The Challenge

After Directeam led the overall migration of Cyabra’s infrastructure from Azure to AWS, several new challenges emerged around model inferencing and the data scientists’ work environment. The migration had successfully moved the content moderation services to AWS, but Cyabra faced the following hurdles as they looked to further optimize and scale their machine learning workflows:

• Setting Up Persistent Inference for Text and Image Models: Cyabra needed to establish a reliable, always-on inference system for their text and image models. These models had to be consistently available to serve both the existing content moderation services and support new services that Cyabra was planning to launch. The challenge was to select a solution that could meet the demands of both current and future workloads while ensuring high availability and reliability.

• Providing an Environment for ML Experimentation: With the migration to AWS, Cyabra’s data science team required a flexible and optimized environment for model experimentation. The goal was to create an efficient setup that would allow data scientists to quickly test and refine models without the overhead of infrastructure management. The challenge was to build a setup that supported rapid experimentation while ensuring cost-effectiveness and simplicity as the team continued to innovate and iterate on machine learning models.
• Optimizing Experimentation Time and Costs: Cyabra’s data scientists needed an environment that would optimize both the time and cost of experimentation. The challenge was to design the most efficient environment that would allow for quick, high-quality experiments, while minimizing costs associated with the resources required for model training and testing.

In summary, Cyabra needed to create an optimized environment that could efficiently handle persistent inference, enable rapid experimentation, and reduce both time and costs in their machine learning workflows.

The Solution

To address the challenges outlined above, Directeam worked closely with Cyabra to design a tailored solution that optimized both model inference and experimentation workflows. The solution focused on leveraging AWS SageMaker’s managed infrastructure, optimizing GPU selection, and streamlining the integration with Cyabra’s microservices.

• Persistent Inference Setup Using AWS SageMaker: Directeam advised Cyabra on setting up a reliable, always-on inference service for both the text and image models using AWS SageMaker. These models were hosted on dedicated SageMaker endpoints, providing consistent, low-latency inference to serve both the existing content moderation microservices and a new service Cyabra was planning to introduce. By using SageMaker’s managed endpoints, Cyabra ensured the models were available at all times, meeting the demands of both current and future workloads without needing manual intervention for scaling.
• Optimized Environment for ML Experimentation: Directeam helped Cyabra design a flexible and efficient environment for their data scientists to conduct model experimentation. Using AWS SageMaker’s Jupyter notebooks, the data science team could easily prototype, test, and refine models without the complexity of managing infrastructure. This environment enabled rapid iteration while maintaining control over resource utilization. By using SageMaker, Cyabra was able to focus on the experimentation process itself, reducing the overhead and allowing the team to move faster.
• Optimizing Experimentation Time and Costs with the Right GPU Selection: Directeam recommended migrating from G4dn GPU instances to the more powerful G5 instances, equipped with NVIDIA A10G GPUs. G5 instances provided up to 3x higher performance than G4dn, significantly reducing model training time. This allowed Cyabra to complete training cycles faster, enabling more iterations within the same time frame. Furthermore, G5 instances offer up to 40% better price-performance compared to G4dn instances, making them a more cost-effective choice for Cyabra’s machine learning workflows. As a result, Cyabra was able to save on both time and costs associated with experimentation, enabling a more efficient and cost-effective machine learning workflow.
  For further reference, AWS’s official page highlights the cost-effectiveness of G5 instances for training compared to G4dn instances: AWS G5 Instances.
• Integration with Cyabra’s Microservices: The SageMaker endpoints were integrated with Cyabra’s microservices running on Amazon EKS, enabling efficient interaction between the models and the microservices. This streamlined the content moderation pipeline, improving both performance and ease of management.

The Results

In today’s rapidly evolving digital landscape, the rise of generative AI has made the creation of fake content almost effortless, posing significant challenges to organizations striving to maintain trust and authenticity online. The success of this project highlights how leveraging modern cloud-based AI solutions can empower businesses like Cyabra to effectively combat these threats while optimizing resources and scaling for the future. 

By implementing the solution for model inference and experimentation on AWS, Cyabra achieved significant improvements across several key areas:

• Reliable and Always-On Model Inference: The persistent inference setup using AWS SageMaker ensured that both the text and image models were available at all times to serve Cyabra’s content moderation microservices and a new service. This setup provided consistent, low-latency inference, meeting the demands of both current and future workloads without the need for manual scaling, ensuring high availability and reliability.
• Optimized and Efficient Experimentation Environment: With the implementation of SageMaker’s Jupyter notebooks, Cyabra’s data scientists were able to rapidly iterate on model experimentation. The environment enabled quick prototyping and testing without the complexity of managing infrastructure. This resulted in faster experimentation cycles, allowing Cyabra to develop and refine their machine learning models more efficiently, reducing both time and overhead.
• Improved Experimentation Speed and Cost Efficiency: The migration to G5 instances led to a dramatic reduction in model training time, with up to 3x faster performance compared to G4dn instances. This enabled Cyabra to complete training cycles more quickly, facilitating more model iterations within the same timeframe. Additionally, G5 instances offered up to 40% better price-performance, helping Cyabra reduce operational costs while maintaining high-quality experimentation.