The Hidden Stack: The Real Truth About AI Infrastructure Spending (And Why It’s Not on the Charts)
- Tony Grayson
- 3 days ago
- 5 min read
By Tony Grayson, Tech Executive (ex-SVP Oracle, AWS, Meta) & Former Nuclear Submarine Commander
The industry is funding an Apollo Program every 10 months. But if you look past the credit card receipts and follow the megawatts, you'll see the real story of AI infrastructure spending.
The $400 Billion Apollo Project
The scale of investment happening in AI right now is staggering.
Tech companies are projected to reach roughly $400 billion in AI infrastructure spending globally this year (2025). To put that in perspective: adjusted for inflation, that is more money spent in a single year than the entire Apollo Moon Program cost over a decade (~$257 billion in today's terms).
And the trajectory is accelerating. Combined AI capital expenditures in the U.S. alone are on track to exceed $500 billion in 2026.
In essence, the industry is now funding an "Apollo Program" sized infrastructure investment every 10 months.
But here is the question no one is asking: If we are spending $400 billion a year, what are we actually building?
The Map vs. The Territory
In military planning, we employ a mental model known as "The Map is Not the Territory." A map is a reduction of reality...useful for orientation, but dangerous if confused for the truth.
I was reminded of this when I saw a widely circulated chart this week titled "Where Startups Spend on AI." It lists the usual suspects—OpenAI, Anthropic, and a handful of SaaS tools, while Open Source models like Llama appear to have a market share of roughly 11%.
This chart is the Map. It accurately tracks who gets the monthly credit card payments (SaaS revenue). But it completely misses the Territory.
Marketing charts track software licenses (The Map), but the physical reality of the data center (The Territory) proves that the industry is shifting massively toward Open Source.
According to a November 2025 analysis by Tomasz Tunguz, 70% of production AI teams now use open-source models in some capacity.
48% describe their strategy as "mostly open source."
22% are "only open source."
Only 11% stay purely proprietary—ironically matching the exact slice of the market that the spending charts claim is dominant.
The "Hidden Stack" of AI Infrastructure Spending
If 70% of the market is using Open Source, why does it appear as $0 on the "AI Spend" chart?
Because Open Source isn't a Software cost, it is an Infrastructure cost.
When a startup switches from GPT-4 to Llama 3, they stop writing a check to OpenAI (Software License) and start writing a check to AWS, CoreWeave, or a Colocation provider (Compute & Power). The money didn't vanish; it just moved from the "Software" budget to the "Utility" budget.
This is the Hidden Stack. It doesn't appear in SaaS metrics, but it is the primary destination for the $400 billion in AI infrastructure spending.
The Real Driver of AI Infrastructure Spending
To understand why this "Hidden Stack" is growing so fast, we have to distinguish between two parallel shifts that are compounding each other.
Shift 1: The Move to Open Source (The Margin Shift).
This explains where the money is going. When you run proprietary models, the model provider (OpenAI) captures the margin. When you run Open Source on bare metal, the Infrastructure Provider captures the margin.
Shift 2: The Move to Inference (The Volume Multiplier).
This explains why the bills are becoming so large. For the past two years, we have focused on Training (brain development). We are now entering the era of Inference (using the brain).
Training is episodic. You train a model once, like building a factory. You then tune and update as needed.
Inference is continuous. You run the model 24/7/365, like running the assembly line.
As companies move from "experimentation" to "production," the economic pressure to optimize these continuous costs becomes overwhelming. This forces them to move from expensive API calls (SaaS) to efficient, self-hosted open source (Infrastructure).
The Forecast: Gartner projects that spending on inference will overtake training workloads in AI-optimized IaaS in 2026, rising to over 65% of all AI compute spending by 2029.
The Scale: To give you an idea of the volume, analysis suggests OpenAI spent roughly $8.67 billion on inference alone in just the first nine months of 2025.
The Hybrid Reality
To be clear, this isn't a binary switch. Most sophisticated teams are running a hybrid architecture. They use GPT-4 or Claude 3.5 for complex reasoning (low volume), and fine-tuned Llama models for high-volume, repetitive tasks.
But in the last 12 months, the economics have forced a hard shift toward the latter.
The Catalyst: We have reached near "Model Parity," where open models such as Llama 3.1 are sufficiently accurate for many production tasks.
The Result: A Series B fintech company we track recently reduced its inference costs by 28% in one quarter by moving its high-volume document processing from a closed API to self-hosted Llama on bare metal.
When you are a seed-stage startup, you pay for convenience (e.g., APIs). When you scale, you pay for control (Infrastructure).
Conclusion: Follow the Megawatts
It is easy to get distracted by the Map. SaaS funds must demonstrate SaaS revenue to justify their valuations. However, as operators, we have to consider the Territory.
If you want to see where the market is actually going, don't look at SaaS receipts. Look at reserved GPU-hours. Look at the committed colo kW. Look at transformer lead times.
The winners of this shift aren't just the software providers. It is the entire physical stack: the power-secured sites, the switchgear supply chains, the liquid-cooling technologies, and the efficient inference silicon.
The market is voting with its code, not just its credit cards. And the chain of custody is clear: Tokens → GPU-seconds → kWh → MW at the meter.
If you want adoption, don't follow the invoices. Follow the megawatts.
FAQ
Q: Why doesn't Open Source AI show up on startup spending charts?
A: Open Source AI models (like Llama 3.1) do not require a software licensing fee, so they do not appear on "SaaS Spend" reports like those from a16z. Instead, the cost of running these models is recorded as "Cloud Infrastructure" or "Colocation" expenses, because startups pay for compute and power rather than for the software itself.
Q: How is AI infrastructure spending changing in 2025?
A: AI infrastructure spending is shifting from "Training" (building models) to "Inference" (running models). As of 2025, inference accounts for a growing majority of compute, with Gartner predicting it will reach 65% of all AI compute spending by 2029.
Q: What is the "Hidden Stack" in AI infrastructure?
A: The "Hidden Stack" refers to the physical infrastructure—power, cooling, colocation, and bare metal servers—required to run open-source AI. It is termed "hidden" because it does not appear on software vendor lists, yet it represents a substantial share of the projected $400 billion in global AI infrastructure spending in 2025.
____________________________________
Tony Grayson is a recognized Top 10 Data Center Influencer, a successful entrepreneur, and the President & General Manager of Northstar Enterprise + Defense.
A former U.S. Navy Submarine Commander and recipient of the prestigious VADM Stockdale Award, Tony is a leading authority on the convergence of nuclear energy, AI infrastructure, and national defense. His career is defined by building at scale: he led global infrastructure strategy as a Senior Vice President for AWS, Meta, and Oracle before founding and selling a top-10 modular data center company.
Today, he leads strategy and execution for critical defense programs and AI infrastructure, building AI factories and cloud regions that survive contact with reality.