The infrastructure challenge is no longer simply about having enough computational power to process all that information. It is also about getting the right data to the right computing resources quickly enough to make AI’s capabilities as effective as they can be.
Overwhelmed by Data
AI workloads are fundamentally data hungry. Training requires massive datasets to be stored and moved into computing environments. Inference, which consumes greater amounts of computing power, puts that data to work, whether it comes from a sensor on a manufacturing floor, a logistics operation, a medical tool or a financial system. Those and myriad other systems serving the government, the military, industry, academia and people in general have generated astronomical amounts of data.
The world now creates 402 million terabytes of data per day, according to statistics originating with Statista and IDC. And that number very likely went up while you were reading this sentence. On an annual basis, 181 zettabytes of data were created in 2025, a 23% increase from 2024. This year, the total is projected to be 221 zettabytes, another 22% growth. (In terms of bytes, a zettabyte is represented by a 1 followed by 21 zeros.)
AI is poised to accelerate that growth exponentially, as the impact of agentic AI becomes more evident. AI is being adopted at a rate that dwarfs that of any other technology in history, including PCs, cell phones, the internet or anything else. That even includes electricity and radio, as Microsoft’s AI Diffusion Report points out.
And yet, it’s just getting started, with its penetration of global markets at only 18% of the working population. The majority of that use currently involves chatbots, which makes people wonder how much more data and computational activity awaits as AI systems move into higher realms of activity. But agentic systems can perform far more steps than a simple chatbot interaction, calling tools, retrieving information and exchanging data across multiple systems. That additional activity increases pressure not only on compute resources but also on the networks, storage systems and data architectures feeding those workloads.
Hybrid Infrastructure Is the Practical Answer
With all that data in the works, how do you consistently move the right data to the right place at the right time? This is where AI infrastructure is critical.
For years, the answer was relatively straightforward: Put computing resources in large, centralized data centers, and connect users and systems to them with increasingly high-capacity networks.
That model still has a place, but AI is changing the equation. As it transitions from answering individual prompts to analyzing requests, reasoning, planning, using tools and executing longer workflows, the amount of data moving through these systems grows dramatically. At that scale, relying on a distant centralized environment becomes impractical, with latency, to name one problem, hindering the speed and effectiveness of AI systems.
Instead of asking only where data should reside, organizations also need to consider where workloads should run and where the data supporting those workloads needs to be available. That means processing data closer to where it’s needed, such as with autonomous vehicles, retail points of sale, medical monitors or smart machines in a factory. This makes the cloud and the edge part of the AI infrastructure.
It makes sense: If an AI system needs to make a decision in milliseconds, sending the data hundreds or thousands of miles for processing may not deliver the best, most timely result. But that doesn’t mean everything should move to the edge. The future is more likely to be a combination of centralized, regional, private and public cloud environments and edge infrastructure, with workloads placed where they make the most operational sense.
We are already seeing organizations move in this direction. Research by DataBank found that 64% of AI workloads currently run in public (49%) or private (15%) cloud environments. At the same time, organizations expect their infrastructure strategies to become considerably more distributed over the next five years, with many planning additional private, on-premises and colocation capacity.
Some workloads belong in the public cloud because they need flexibility and scale. Others require dedicated infrastructure because of performance, security, compliance or data sovereignty requirements. Still others need to run close to the source of the data. The answer isn’t cloud versus on-prem versus edge. The answer is putting each workload in the right place and making all those environments work together.
Systems Working as One Depend on Tight Integration
This requires architecting the entire data path: compute, memory, storage, networking, cloud, edge and security as one system. And that can sometimes mean making data accessible where it already exists, rather than looking to move it elsewhere or break down silos, both of which can cause latency. Moving data can also potentially increase network requirements and create additional security and compliance concerns.
The industry is moving toward a decentralized approach. In DataBank’s survey, 76% of respondents said they expect their infrastructures to expand geographically over the next five years, with 44% saying they need to be closer to data sources and 32% aiming to be closer to users. This can improve performance by reducing latency, particularly for real-time applications like self-driving cars, while improving compliance with data sovereignty requirements and keeping data in the countries where it was created.
At SD3IT, we help organizations build resilient, flexible infrastructure by integrating compute, storage, networking, cloud, edge infrastructure, data management and security. We work with a partner ecosystem to address different parts of an efficient AI infrastructure.
SD3IT partner HPE, for example, is building AI infrastructure around accelerated computing, high-performance networking and hybrid cloud capabilities. Another partner, edgeTI, approaches the problem from the data side, with mesh architectures that can make information accessible without having to centralize everything. Both can help organizations achieve the geographic expansions they need.
Our job is to help integrate those capabilities into an architecture that works for the mission while also being resilient and adaptable. The AI environment that organizations build today will not look the same five years from now. Models will evolve, workloads will change and data volumes will continue to grow. More inference will happen at the edge. The infrastructure must be able to evolve with those changes.
An Infrastructure Built for the Future
There is no single answer to the AI infrastructure challenge. Computing power will always matter, but compute without readily accessible data is unrealized potential.
Organizations need to think about AI infrastructure as a complete, evolving architecture rather than as a collection of individual hardware components. Compute, memory, storage, networking, cloud, edge, data management and security all have to work together so that information can be accessed, processed and protected wherever the mission requires it.
As AI workloads grow more distributed and autonomous, the organizations that succeed will not simply be those with the most powerful processors. They will be the ones that can consistently get the right data to the right workload at the moment it is needed.
Because even the most powerful AI in the world can’t make the right decision if the data gets there too late.
