H2O.ai – Org-Wide Cost Visibility & Anomaly Detection with the AWS FinOps Deep Agent

Meet H2O.ai
H2O.ai is a leading AI/ML platform company running data-intensive machine-learning, CI/CD and inference workloads on AWS. Its footprint is a large, complex AWS organization of more than 100 accounts spanning production, non-production and shared-services environments.
The AWS FinOps Deep Agent
Directeam’s AWS FinOps Deep Agent is an autonomous agent embedded in every customer’s AWS accounts. It continuously manages and monitors cost inefficiencies, cost and usage anomalies, and security-guardrail posture across the customer’s entire AWS organization, and alerts the customer the moment a new anomaly appears. It advises on building AWS infrastructure in the most cost-effective, well-architected way, with recommendations grounded specifically in AWS services. The agent also exposes an MCP (Model Context Protocol) interface adopted by a growing number of customers that integrates directly into their SDLC and developer tooling, significantly improving time-to-resolution (TTR) for FinOps, DevOps, security and billing issues by delivering precise, accurate recommendations together with the root cause and reasoning behind any incident. The agent is enabled through a read-only IAM role deployed via a CloudFormation Stack (standalone) or StackSet (organization-wide); all processing runs on AWS Bedrock no external/OpenAI models in a per-customer isolated tenant within AWS U.S. regions, collecting only metadata/usage metrics and retaining only investigation summaries, with access revocable at any time by deleting the stack.
Inefficiency detection is part of the agent’s continuous anomaly alerting. Alongside spike anomalies, the AWS FinOps Deep Agent proactively detects inefficiencies that quietly drive wasted cost and for each one it explains the reasoning and root cause, then provides a step-by-step recommendation for how to resolve it. The inefficiency portal presents the potential monthly and annual savings for every finding, and each inefficiency line carries an estimate of how long it will take to resolve (for example, “quick wins” that take under ~4 hours of work), so the team can filter the backlog by savings and by effort and act on the highest-value items first.
Part of Directeam’s full managed-services stack
The agent is delivered within Directeam’s end-to-end managed-services platform, deployed and operated by Directeam for every customer at onboarding: a managed cost-visibility platform that Directeam operates and tunes on the customer’s behalf for cost visibility; landing-zone build and management (Control Tower / LZA); billing, CUR and cost-allocation operated on behalf of the customer; IAM-role and policy management (least-privilege roles, permission boundaries, SCPs); AWS support (PLS) enrollment with a dedicated support channel; and Sombrero Savings — Directeam’s automated Savings Plans / Reserved Instance commitment management that cuts EC2 compute cost without the customer carrying commitment risk. Directeam’s security baseline for this stack is validated through the AWS SPP Foundational Technical Review (FTR).
The Challenge
- A 100+ account estate made org-wide cost visibility and attribution genuinely hard — spend drivers were spread across many accounts and regions.
- Data-intensive ML and CI workloads generated large, variable data-transfer, query and storage costs that were hard to trace to a root cause.
- Networking and S3 data paths (VPC endpoints, Transit Gateway, Network Firewall) silently accrued cost when misconfigured.
- Idle and over-provisioned compute, storage and database resources accumulated across accounts.
- Compute commitment (Savings Plans / Reserved Instances) decisions needed ongoing, data-driven guidance.
The Solution
The engagement between Directeam and HyperGuest moved a workshop concept into a production-grade, partner-facing agent that materially changes how the platform delivers operational support. Key outcomes include:
- Replaced Ticket-Heavy Support Flows with an Autonomous Agent: Routine partner inquiries are now handled by a grounded chat agent operating over HyperGuest’s connectivity knowledge base, freeing the support team for higher-complexity escalations.
- Predictable Per-Conversation Cost Economics: Serverless infrastructure and model selection produced a per-conversation cost profile that compares favorably with the prior support-ticket cost, with no manual capacity planning required as traffic scales.
- Tenant Boundary Preserved End-to-End: Per-tenant identity propagates from the chat surface through the Bedrock Agent invocation into Action Group calls and Knowledge Base retrieval, closing cross-tenant data exposure paths.
- Continuous Quality Loop Improving Grounding Fidelity: Production telemetry, guardrail interventions, and reviewer-queue feedback feed an iterative tuning process — producing measurable improvements in grounding fidelity and tool-use correctness over time.
- Removed Self-Managed Inference Infrastructure: Bedrock Agents, OpenSearch Serverless, Lambda, and Fargate together remove every self-managed inference and indexing component from HyperGuest’s footprint.
- Directeam’s Ongoing Support: Directeam continues to serve as HyperGuest’s technical partner across the AWS estate, including the dedicated CloudFront PPA calculator built for HyperGuest’s specific traffic shape.
Key Results & KPIs
Representative operational metrics observed for H2O.ai with the AWS FinOps Deep Agent:
| AWS accounts managed | 100+ accounts across one AWS Organization, monitored continuously by Directeam |
| Time to detect a new anomaly | ~24 hours surfaced in the next-morning alert with resource, account, region & root cause |
| TTR for a cost investigation | From hours/days of manual work to a single agent session (precise root cause + steps) |
| Cost outcomes | Reductions expressed as % of the affected service line — e.g. EC2 run-rate cut ~40–70% combined (right-sizing, Spot, Graviton & commitment coverage), Athena query spend ~20–40%; absolute figures shared privately |
| Coverage | A missing-VPC-endpoint / NAT data-transfer pattern found across ~40–60% of the 100+ accounts; the per-GB S3 charge eliminated (~100%) where the free Gateway endpoint is added |
| Interactions | Directeam-operated: daily automated anomaly alerts + ad-hoc natural-language cost queries |
Quantitative Outcomes
Representative optimisations Directeam delivered for H2O.ai, expressed as percentage reductions on the affected service line (absolute amounts withheld; available privately):
| What Directeam controlled | Service | Result (% on affected line) | Root cause |
|---|---|---|---|
| Optimised query spend | Amazon Athena | ~20–40% reduction | Full-object scans over high-volume telemetry data; partition pruning, result caching and S3 Select to cut data scanned per query |
| Reduced compute run-rate | Amazon EC2 | ~40–70% reduction (combined) | Right-sizing GPU/CPU instances at 2–3% utilisation; Spot for fault-tolerant CI/CD (~60–90% vs On-Demand); Graviton migration (~20–40%); Savings-Plans coverage on GPU families with zero prior commitment |
| Eliminated billable S3 data transfer | Amazon S3 / VPC | ~100% of the per-GB charge removed per affected account | Traffic routed through NAT / an Interface endpoint instead of the free S3 Gateway endpoint pattern found across ~40–60% of the 100+ accounts |
| Right-sized provisioned IOPS | Amazon EBS | ~80–90% on provisioned-IOPS cost; ~15–25% on overall EBS | Volumes set to 30,000 IOPS at <1% utilisation; unattached volumes and orphaned snapshots (DeleteOnTermination=false) |
| Removed idle / over-provisioned databases | RDS / Redshift | RDS dev/test Multi-AZ→Single-AZ ~50% (~10–20% overall RDS); Redshift ~5–15% | Unnecessary Multi-AZ on dev/test RDS and idle replicas; under-utilised Redshift clusters left running |
Worked Example – An Inefficiency Finding
A representative inefficiency the agent surfaced and walked H2O.ai through, end to end:
Add a free S3 Gateway VPC Endpoint (eliminate billable interface-endpoint S3 traffic)
Detected: Directeam’s agent flagged a recurring inefficiency after a step-change in Amazon VPC (data-transfer) cost – in one case traced to a third-party security scanner (Upwind DSPM) routing S3 GetObject calls through a NAT gateway and generating hundreds of GB/day of avoidable egress then found the same missing-endpoint pattern across ~40–60% of the 100+ accounts.
Reasoning & root cause: S3 traffic was routed through a NAT gateway (or an Interface endpoint with Private DNS) instead of the free S3 Gateway endpoint, so every GB was billed at the per-GB egress rate ($0.045–$0.090/GB) instead of being free. No free S3 Gateway endpoint existed in the route tables for the affected subnets.
Step-by-step recommendation to resolve:
- Confirm the S3 traffic volume and that it is flowing through the Interface endpoint (the agent verifies via VPC/endpoint metrics).
- Create a free S3 Gateway VPC endpoint and associate it with the relevant subnet route tables.
- Disable Private DNS on (or remove) the billable S3 Interface endpoint where it is not required for PrivateLink-only access.
- Validate that S3 traffic now routes through the free Gateway endpoint and the per-GB charge drops to zero.
- Roll the same fix across the remaining affected accounts.
| Potential monthly savings | ~100% of the per-GB S3 data-transfer charge removed on each affected account |
| Potential annual savings | Pattern spanned ~40–60% of the 100+ account fleet – full-year impact compounds across all of them |
| Estimated effort to resolve | Low – under ~1 day per account (‘quick win’) |
| Outcome | Pattern identified fleet-wide and remediated; per-GB S3 charge driven to ~zero where the Gateway endpoint is added |
The Results
- Continuous, Directeam-operated cost visibility across 100+ accounts replaced manual, invoice-time investigation.
- A recurring data-transfer inefficiency spanning ~40–60% of accounts was identified and eliminated (per-GB S3 charge driven to ~zero per account).
- EC2 run-rate was reduced ~40–70% combined via right-sizing, Spot (~60–90% vs On-Demand on CI/CD), Graviton migration and Savings-Plans coverage; Athena query spend was cut ~20–40%; EBS provisioned-IOPS ~80–90%.
- Compute-commitment posture is reviewed regularly with Directeam and tuned to actual usage; engineering time spent chasing cost anomalies was materially reduced.
Directeam continues to operate the AWS FinOps Deep Agent across H2O.ai’s AWS estate as part of its managed-services engagement, with ongoing anomaly detection, inefficiency reviews and architecture guidance.