Why 93% of Enterprises Are Moving AI Off the Cloud

For most of the last decade, the default advice for any compute-intensive workload was the same: put it in the cloud. Scale on demand. Pay as you go. Don't worry about hardware.

That advice is breaking down for AI workloads. Not because the cloud doesn't work — it works fine for many things — but because enterprise AI has specific characteristics that make cloud-only deployment increasingly impractical.

The numbers tell the story: 93% of enterprises are either actively repatriating AI workloads or evaluating doing so. 79% have already moved at least some AI workloads off the cloud. This isn't a fringe movement. It's a structural shift in how large organizations think about AI infrastructure.

This article covers the three forces driving the shift, what it means for data preparation and model deployment, and how the industry is responding.

The Three Forces Behind AI Repatriation

Force 1: Data Sovereignty and Regulatory Pressure

The regulatory landscape for AI has changed faster than most organizations anticipated. The EU AI Act, DORA (Digital Operational Resilience Act), and sector-specific regulations in healthcare, finance, and defense have created a web of requirements around where data can be processed and by whom.

91% of enterprises now prefer on-premise infrastructure for processing sensitive data with AI systems. That preference isn't ideological — it's practical. When your compliance team needs to demonstrate that patient records, financial transactions, or classified documents never left your controlled environment, the simplest proof is that the infrastructure processing them was never connected to an external network.

The numbers on how this affects actual AI projects are striking:

• 58% of enterprises report that data residency concerns have delayed or blocked AI initiatives entirely
• 74% flag shadow AI — employees using unauthorized cloud AI tools — as a critical security concern
• 91% prefer on-premise for AI workloads involving sensitive data

Shadow AI deserves special attention. When employees can't use company-approved AI tools because the approved tools require sending sensitive data to a cloud API, they find workarounds. They paste customer data into ChatGPT. They upload contracts to Claude. They use personal API keys. The security team doesn't know about it, the compliance team can't audit it, and the risk exposure compounds invisibly.

Organizations that deploy on-premise AI tools — where employees can use AI without data leaving the building — report measurably lower shadow AI usage. The compliance benefit is a side effect of making the approved tool easier to use than the unauthorized one.

Force 2: Cost Unpredictability

Cloud AI pricing looks straightforward until you're running production workloads at scale.

40% of enterprises report that actual cloud AI spending exceeds their initial budget projections. Not by a little — many report costs 2-3x their estimates once you account for data egress, storage growth, token consumption spikes, and the ancillary services (logging, monitoring, vector databases) that a production AI deployment requires.

The problem isn't that cloud is expensive per se. It's that cloud AI costs are hard to predict and harder to cap. A batch processing job that runs inference on 10 million documents will cost what it costs, and you won't know the exact number until the bill arrives. An on-premise GPU cluster has a fixed capital cost and a predictable operational cost (power, cooling, staff). For sustained workloads, the math tips toward on-premise surprisingly quickly — often within 7-12 months.

This is especially true for data preparation, which is the most compute-intensive phase of most AI projects. Cleaning, transforming, and structuring enterprise data for training or fine-tuning involves running that data through multiple processing steps, each of which consumes compute. At cloud token prices, preparing a large corpus can cost more than training the model on it.

Force 3: Latency and Performance Requirements

75% of enterprises report that on-premise deployment is necessary to meet acceptable latency requirements for their AI applications.

This makes intuitive sense for certain workloads. A manufacturing quality inspection system that needs to classify defects in real-time on a production line can't tolerate the 200-500ms round-trip to a cloud endpoint, plus the variability of shared infrastructure. A clinical decision support system embedded in an EMR workflow adds friction if every AI-assisted suggestion requires a network call to a data center 500 miles away.

But latency requirements go beyond just speed. They include:

• Deterministic performance: On-premise inference gives you consistent latency because you're not sharing resources with other tenants
• Offline capability: Many enterprise environments — factories, hospitals, field operations, secure facilities — don't have reliable or any internet connectivity
• Throughput control: When you own the hardware, you can prioritize workloads without competing for capacity

From "Cloud-First" to "Workload-Specific Placement"

The shift isn't anti-cloud. It's post-cloud-first. Enterprises are moving from a default assumption ("everything goes to the cloud") to a deliberate evaluation ("this specific workload belongs in this specific environment").

The emerging pattern looks like this:

This is "workload-specific placement," and it's the dominant strategy among enterprises with mature AI programs. 86% of enterprises expect their AI budgets to increase in 2026, with 40% projecting increases of 25% or more. That money is increasingly being split between cloud and on-premise infrastructure rather than directed solely to cloud providers.

Industry Response: The Infrastructure Is Catching Up

A year ago, running AI on-premise required significant custom engineering. The tooling gap between cloud AI platforms and on-premise alternatives was wide. That gap is closing fast.

Microsoft Foundry Local provides a local runtime for running AI models on enterprise hardware without cloud connectivity. It's Microsoft's acknowledgment that "everything in Azure" isn't what their enterprise customers want for every workload.

Red Hat and Telenor built a sovereign AI factory — a reference architecture for running AI entirely within a national boundary, using Red Hat's OpenShift platform. It's designed for telecom and government customers where data sovereignty isn't optional.

NVIDIA's AI Factory architectures provide reference designs for on-premise GPU clusters optimized for inference, training, and data preparation. They've moved from selling GPUs to selling complete deployment patterns.

These aren't experimental projects. They're production-grade infrastructure offerings from companies that bet on cloud for a decade and are now building on-premise products because that's where customer demand is going.

What This Means for Data Preparation

Here's the part that many organizations miss when planning cloud-to-on-premise migrations: you need on-premise data preparation before you can run on-premise models.

A model running on local hardware is only useful if it has data to work with. For inference, that means the input data needs to be cleaned, structured, and formatted before it reaches the model. For fine-tuning, that means your training data — often drawn from sensitive enterprise documents — needs to go through extraction, cleaning, annotation, and formatting pipelines.

Data preparation is where the most sensitive data touches happen. It's where you're processing raw customer records, medical files, legal documents, and financial transactions. If your model runs on-premise but your data preparation pipeline runs in the cloud, you've shipped all your sensitive data to a cloud provider anyway. The on-premise model gives you nothing from a sovereignty perspective.

This is why data preparation tools that run entirely on-premise — no cloud dependency, no data leaving the network — are a prerequisite for meaningful cloud repatriation. You can't just move the model. You have to move the entire pipeline.

What This Means for Fine-Tuning and Training

Training large foundation models from scratch still requires cloud-scale compute for most organizations. Few enterprises have the thousands of GPUs and the engineering team needed to train a model from zero.

But fine-tuning is a different story. Fine-tuning an existing open-weight model on proprietary data can be done on a single server with 1-4 GPUs. The compute requirements are orders of magnitude lower than pre-training, and the data involved is almost always proprietary and sensitive — exactly the kind of data that sovereignty requirements say should stay on-premise.

The practical pattern for most enterprises in 2026:

1. Select a base model from the open-weight ecosystem (Llama, Mistral, Qwen, etc.)
2. Prepare training data on-premise using local data preparation tools
3. Fine-tune on-premise using local GPU infrastructure
4. Deploy on-premise for inference
5. Use cloud only for initial experimentation and non-sensitive workloads

This pattern keeps sensitive data entirely within the organization's controlled environment while still leveraging the open-source model ecosystem.

The Air-Gapped Frontier

The most extreme version of this trend is air-gapped AI — systems that operate with zero internet connectivity. This was once a niche requirement limited to defense and intelligence agencies. It's expanding.

Healthcare systems processing patient data under HIPAA. Financial institutions handling trading algorithms. Critical infrastructure operators. Government agencies at every level. These organizations are building AI capabilities that run on physically isolated networks, and they need every component of the AI pipeline — data preparation, training, fine-tuning, inference, evaluation — to work without any external network calls.

Air-gapped AI is the logical endpoint of the repatriation trend. Not every organization will get there, but the tools and architectures being built for air-gapped deployments benefit everyone on the spectrum. If your pipeline works in an air-gapped environment, it definitely works in a standard on-premise environment.

What Comes Next

The 93% number will keep climbing. Regulatory pressure is increasing, not decreasing. AI budgets are growing, and organizations that have been running cloud AI for 2-3 years now have enough data to calculate their actual TCO — and many don't like what they see.

The organizations that move fastest will be those that:

• Audit their current cloud AI spending honestly, including all hidden costs
• Classify workloads by sensitivity, latency requirements, and cost characteristics
• Build on-premise data preparation capabilities first, because data prep is where sovereignty requirements bite hardest
• Start with inference migration, which has the best cost-to-complexity ratio
• Keep cloud for what cloud does well: burst compute, experimentation, and elastic workloads

The question isn't whether your organization will move some AI workloads off the cloud. It's which workloads, in what order, and how well-prepared you'll be when you do.

The enterprises that treat this as a deliberate infrastructure strategy — rather than a reaction to a compliance audit or a budget surprise — will be the ones that get the benefits without the disruption.