For the last decade, the direction in enterprise infrastructure has seemed pretty set - move to the public cloud, shut down the server room, and don’t look back. That trend is getting more complicated. As AI workloads multiply and cloud costs climb with them, some organizations are bringing part of their infrastructure back in house, while edge computing keeps expanding because the workloads can't tolerate a round trip to a third party’s data center.
The Cloud-Only Model Is Being Reconsidered
The assumption that centralizing everything in the public cloud would automatically be cheaper and more efficient no longer holds up the way it once did. CIO's recent coverage of this shift makes the point. As AI drives demand for specialized, next-generation hardware, cloud provision often isn't the most cost-effective option it used to be. Returning workloads back from the public cloud is already underway at many organizations. Few IT leaders today see an all-in-cloud future for their organization. A blend of cost pressure, privacy requirements, legislation, and the need for operational flexibility is pushing companies toward owning more of their own infrastructure again.
Cost is only part of it. Data sovereignty and regulatory obligations are pushing in the same direction. It's often easier to manage that risk internally than to depend entirely on a third party's controls. Add in sustainability commitments that are straining under the power demands of AI, and the appeal of infrastructure you fully control becomes easier to understand.
None of this means the cloud is going away. It remains the right tool for scalable, well-organized workloads and centralized storage. The real shift is that cloud is becoming one option among several.
The AI Factor Nobody Can Ignore
AI is reshaping this conversation on both sides of the equation. Specialized AI hardware doesn't always make economic sense to rent indefinitely from a cloud provider. On the other hand, AI workloads are also a big part of why the underlying infrastructure question is getting harder, not easier. Brookings' recent research on the future of data centers lays out just how significant the resource demands have become. Data centers already account for a notable share of U.S. electricity consumption. That share is expected to climb sharply as AI adoption accelerates, with some projections putting AI's share of global electricity use as high as a fifth of total demand by the end of the decade. Water use is a growing constraint too, with some facilities drawing hundreds of thousands of gallons a day in regions where water is already scarce.
Every organization weighing on-prem, colocation, or cloud for AI workloads is also weighing exposure to power availability, water availability, and the increasingly complex supply chains for the copper, steel, and semiconductors those facilities depend on.
Resilience and Sovereignty Are Driving the Conversation as Much as Cost
Organizations want to know that a disruption at one provider, one region, or one supplier doesn't take down their entire operation. They want assurance that sensitive data is subject to the laws and controls they choose, not the ones that happen to apply wherever a hyperscaler's region sits. And they want the flexibility to shift workloads as costs, regulations, and business needs change — something that's much harder to do once you're deeply embedded in a single provider's ecosystem.
This is also why data portability deserves more attention than it usually gets. Being able to move data and models between platforms without excessive friction isn't just a technical asset. It's what keeps optionality alive as the infrastructure landscape keeps shifting under our feet.
Where This Leaves IT Leadership
The organizations getting this right are the ones treating infrastructure as a mixed portfolio: cloud for elastic, well-structured workloads; colocation or owned data center capacity for AI, compliance-sensitive, or cost-predictable workloads; and edge for anything where latency is non-negotiable.
Getting that mix right requires a comprehensive look at total cost of ownership — not just monthly cloud spend, but power, staffing, and the opportunity cost of skills gaps that are already showing up across the industry as data center talent gets harder to find.
Read More