Augment.co,Augment.co, a B2B sales enablement company, leveraging cutting-edge AI technology, faced the challenge of consolidating three separate companies into a unified operation. Each company brought its own unique codebase. The goal was to create a powerful AI sales engine that was designed to enhance B2B sales efforts through AI-powered insights.

The core issue was combining these diverse codebases into a single, efficient system while ensuring smooth AI workflows. As a startup, cost was a significant concern, along with the need to scale efficiently while managing complex technological requirements.

Decision to Use Arm

To solve the growing infrastructure challenge, the company made a critical decision to transition from traditional x86 architecture to Arm processors. The key reasons were clear:

• x86 was slow and expensive for the resource-heavy AI workloads.
• Arm was faster and cheaper, allowing them to scale AI operations without inflating costs.

Business Decision to Use Kubernetes

In parallel, the decision was made to begin migrating their existing workloads, which were not containerized, to a Kubernetes-based architecture. This decision provided several advantages:

• Multi-architecture support:The company needed a solution that could seamlessly run both x86 and Arm workloads within the same Kubernetes cluster. This allowed for gradual migration from x86 to Arm without risking downtime or affecting service delivery.
• Scalability and performance: Kubernetes offered the flexibility and scalability needed to handle fluctuating AI demands, allowing for dynamic resource allocation and smoother management of infrastructure.

Navigating Multi-Architecture Kubernetes for Maximum Efficiency

The migration process required rebuilding all their Docker images for Arm-based environments. While Augment faced initial hiccups with AI libraries that were optimized for x86, they successfully rebuilt their pipeline using Arm architecture, eliminating their reliance on x86-based images.

• Multi-architecture interop:was critical to this success, allowing the company to manage both Arm and x86 workloads within a unified cluster.
• They utilized GitHub Runners with Ampere Altra processors to build Arm images, ensuring efficient performance and integration.
• As the migration progressed, they realized they no longer needed x86 images and could fully commit to Arm-based infrastructure, resulting in better performance and cost savings.

Setting Up the Environment for Success

Given the merger of three companies, the task of combining multiple codebases written by different development groups was complex. Kubernetes helped orchestrate this process, making it easier to manage and deploy workloads across the varying architectures.

The successful migration to Arm architecture allowed the company to run all critical AI workflows on faster, cheaper processors, and Kubernetes provided the repeatability and consistency needed to handle this intricate environment. These workloads currently utilize Claude.ai and are in the process of transitioning to Llama 3.1 13b.

In addition to AI workloads, several other integrations were implemented to provide a secure environment that met the needs of Auguria. These integrations included Nginx as webserver, Postgres database, TimescaleDB for handling time-series data efficiently, Redis for in-memory data storage, RabbitMQ to facilitate communication between application, Prometheus for monitoring and alerting, Grafana for visualizing log data, Grafana Loki for log aggregation, and Faktory to handle asynchronous task management.

Key Metrics and Performance Gains

• Cost Efficiency:Migrating to Arm processors (specifically Graviton-based Arm) dramatically reduced the company’s infrastructure costs compared to running on x86. The cost reduction was estimated at 3X to 5X.
• Performance Improvements: AI workloads running on Arm processors saw a significant performance boost, reducing latency and improving overall system efficiency.
• Resource Optimization: Augment was able to reduce the number of servers required by half. This lower resource architecture gave them the ability to create excess capacity to handle additional spiked traffic without the need for autoscaling.
• Scalability: Kubernetes enabled flexible scaling across both architectures, providing repeatable deployment processes and maintaining uptime during the x86-to-Arm migration.
• Migration Time: the entire migration to the Arm based platform took less than a month to complete, indicating minimal disruption to operations in relation to large value in cost savings.

Customer Quote:
“x86 is slow and expensive. Arm is fast and cheap. As a startup with limited resources, switching to Arm allowed us to stay lean while scaling AI operations.”
– Jordan Ritter, CTO

Value Statements Aligned to Ampere and Kubernetes

• Arm processors offered unmatched cost efficiency and performance in handling heavy AI workloads. The ability to manage both x86 and Arm workloads simultaneously within the same Kubernetes cluster allowed for a seamless, no-downtime transition.
• By using Kubernetes, the company ensured their infrastructure was both scalable and flexible, handling the demands of their multi-architecture environment without sacrificing performance.
• The Kubernetes architecture also enables Augment to have a portable environment that can be migrated to other clouds and take advantage of best value competitors.

Conclusion

Through the strategic decisions to adopt Arm architecture and implement Kubernetes for managing their multi-architecture workloads, the B2B sales enablement company not only streamlined their AI-powered operations but also drastically reduced costs and improved system performance. As a result, they could continue innovating while staying within the constraints of a lean startup budget, setting themselves up for scalable growth in the future.