How Sprawl Happens Without Anyone Deciding It Should
No one made a decision to run four AI vendors. It happened the way most enterprise technology sprawl happens: one reasonable choice at a time, made by different people with different information, in different parts of the organization, at different points in time.
Engineering evaluated models in early 2023 and standardized on Anthropic for document processing. The data science team had existing OpenAI credits from a research project and continued using GPT-4 for their analytical workflows. Product built a feature using Gemini because a vendor gave them early API access. A machine learning engineer set up a Perplexity API key to experiment with retrieval-augmented search and then moved to a different project. The key is still there. The monthly charge is $340. No one is sure what it is doing anymore.
This is AI vendor sprawl — and it is the normal state of almost every organization that has been building with AI for more than 12 months. The question is not whether you have it. The question is how much it is costing you, what risks it is creating, and whether you can see it clearly enough to manage it.
The Costs That Are Easy to Miss
The direct cost of AI vendor sprawl — the total of all your vendor invoices — is only the most visible part. The hidden costs are often larger.
Redundant Capability Investment
When different teams independently select AI vendors for similar use cases, they often build similar infrastructure independently. Team A builds a prompt management system for their OpenAI integration. Team B builds a prompt management system for their Anthropic integration. Neither team knows the other exists. The organization has now funded the same work twice, and neither implementation benefits from the other's learnings.
Negotiating Power Fragmented Across Vendors
AI vendors offer volume discounts, committed spend arrangements, and enterprise pricing tiers. These are negotiated based on total spend. When that total spend is fragmented across four vendors — each of which sees only a fraction of your actual AI investment — you are negotiating from a weakened position with all of them simultaneously.
An organization spending $200,000 per year on AI could be at a meaningful volume threshold with one vendor. Spread across four vendors at $50,000 each, it is below the threshold with all of them. Consolidation is not always the right answer, but it should be an informed choice rather than an accident.
Security and Compliance Surface Area
Every AI vendor integration is a data flow. Prompts contain data. In regulated industries or organizations with sensitive data handling requirements, every new vendor endpoint is a new surface to audit, review, and include in data processing agreements. Sprawl means security reviews lag behind actual data flows — often by months.
The Shadow API Key Problem
API keys provisioned by individual engineers for experimentation often persist long after the experiment ends. They are not tied to service accounts with defined owners. They do not appear in any vendor inventory. They are not subject to key rotation policies. And they are usually charged to a shared credit card or expense account rather than a tracked cost center.
Shadow API keys are not a theoretical risk. They are standard practice. A survey of organizations actively managing AI spend typically finds that 20-40% of active API keys were provisioned by engineers who are no longer directly responsible for the associated usage.
How to Map What You Actually Have
Before you can manage AI vendor sprawl, you need to see it. This requires looking in places that are not typically included in infrastructure audits.
Financial System Scan
Start with your credit card and expense management systems. AI API costs are often paid through corporate cards, engineer expense reimbursements, or departmental purchasing cards that do not route through central IT procurement. A systematic search for vendor names — OpenAI, Anthropic, Google AI, Mistral, Cohere, Perplexity, Replicate, Together AI, Groq, and the dozen others with meaningful market presence — across all payment methods will surface spending that does not appear in your cloud bills or IT budgets.
Network Traffic Analysis
If your organization routes egress traffic through a proxy or monitors DNS resolution, AI vendor API endpoints are identifiable by domain. api.openai.com, api.anthropic.com, generativelanguage.googleapis.com, and similar destinations are distinct and recognizable. A 30-day analysis of outbound DNS or proxy logs will show you which vendors your systems are actually talking to, regardless of whether those vendors appear in any vendor inventory.
Code Repository Search
Search your code repositories for vendor SDK imports and API endpoint strings. This surfaces integrations that may not be in production yet but are in active development, as well as historical integrations in code that is still deployed but no longer actively maintained.
- Python imports:
import openai,from anthropic import,import google.generativeai - Node imports:
require('openai'),from '@anthropic-ai/sdk' - API endpoint strings:
api.openai.com,api.anthropic.com - Environment variable names:
OPENAI_API_KEY,ANTHROPIC_API_KEY,GEMINI_API_KEY
Secrets Manager Audit
If your organization uses a secrets manager — AWS Secrets Manager, HashiCorp Vault, GitHub Secrets, or similar — audit the stored secrets for AI vendor credentials. This surfaces keys that are actively used in automated workflows, even if the associated engineering team is not actively maintaining the integration.
Building the Inventory
The output of a vendor discovery process should be a structured inventory that gives you the information you need to make decisions. A useful AI vendor inventory captures:
| Field | Why It Matters |
|---|---|
| Vendor name | Basic identification |
| Monthly spend (last 90 days) | Materiality assessment |
| Owning team | Accountability and governance |
| Use case | Duplication detection |
| Data classification | Security and compliance risk |
| Integration status | Active, experimental, or orphaned |
| Contract / DPA status | Compliance coverage |
| Key rotation last performed | Security hygiene |
What to Do With the Map
A complete vendor inventory is not a mandate to consolidate. It is the information required to make a rational decision about whether consolidation is the right strategy and, if so, where to start.
Rationalization vs. Standardization
Rationalization means reducing redundancy — eliminating duplicated capabilities and orphaned integrations. This is almost always worth doing. Integrations that are not actively used should be decommissioned, keys revoked, and subscriptions canceled. This reduces cost, security surface, and operational overhead with no downside.
Standardization — mandating a single vendor for all use cases — is a more aggressive intervention that requires careful analysis. Different models have genuinely different strengths. Claude is not GPT-4o. Gemini has capabilities the others do not. Forcing standardization in the name of cost management often results in worse outcomes for use cases that were well-matched to the vendor being eliminated.
The better goal is intentionality: every vendor in your stack should be there because someone made an informed choice, not because someone forgot to cancel a trial.
Establishing Governance Without Friction
The reason AI vendor sprawl happens is that the friction to add a new vendor is very low. An engineer with a corporate card can spin up a new AI vendor integration in 20 minutes. Vendor governance that adds significant friction to this process will be circumvented — engineers will use personal accounts and expense reimbursement.
Effective governance is lightweight enough that the approved path is the easiest path. This usually means:
- A vendor approval process that takes days, not weeks
- Pre-approved vendors that teams can access without a review process
- A centralized key management system that is easier to use than managing keys independently
- Visibility into vendor usage that makes shadow AI apparent without requiring manual disclosure
The Multi-Vendor Reality
The goal of managing AI vendor sprawl is not to arrive at a single vendor. The goal is to know what you have, why you have it, and whether it is working as intended. Most mature AI organizations run multiple vendors intentionally — using different models for different use cases based on cost-performance optimization, and maintaining alternatives for resilience and competitive leverage in vendor negotiations.
The difference between intentional multi-vendor strategy and unmanaged sprawl is visibility. An organization that knows it runs three vendors, can tell you which use cases each serves, and has governance processes to add or remove vendors deliberately is in a fundamentally different position than one that discovers it has seven vendors during a quarterly financial review.
The Role of Unified Observability
Once you know what vendors you have, the operational challenge is maintaining visibility across all of them without requiring your team to log into multiple dashboards. Cost data, usage trends, and anomalies across Anthropic, OpenAI, Google, and others need to appear in a single view to be actionable.
Building this unified layer in-house is possible but requires maintaining integrations with each vendor's reporting API, normalizing different data models, and keeping the integrations current as vendors change their APIs. Platforms like Oberhahn are designed specifically for this layer — aggregating spend and usage data across vendors into a unified view without requiring teams to maintain the underlying integrations.
For organizations managing three or more active AI vendors with meaningful spend, the time cost of manual cross-vendor reconciliation typically exceeds the cost of a platform built to do it automatically.
Start With the Audit, Not the Solution
The natural instinct when confronted with sprawl is to reach for a governance framework — a policy, a process, a platform. The more useful first step is the audit. Until you know what you have, any solution you design is based on assumptions that are probably wrong.
Spend two weeks building a complete picture of your AI vendor landscape. Map the spend, the use cases, the owners, and the integration status. That map will tell you which problem you actually have — whether it is orphaned spend, security risk, redundant capability, or something else entirely. The solution follows from the problem, not from the framework.