Beyond content delivery: The rise of AI-native internet infrastructure

For decades, the internet has been built around delivering content as quickly and reliably as possible. Advances in cloud computing and content delivery networks made it possible to serve web pages, videos, and applications to billions of users with minimal delay. But the rise of artificial intelligence is changing the nature of internet traffic. Instead of simply requesting information, AI agents increasingly generate, interpret, and act on data in real time, creating demand for infrastructure that can execute computation rather than merely deliver content.

This shift is driving the development of AI-native infrastructure, where networking, compute, and storage are designed to support AI workloads across distributed environments. The most important change is not that models move closer to users; most of a model’s response time comes from generating its answer, which relocating hardware does not shorten. It is that the network becomes an intelligent traffic layer: reusing work through semantic caching, steering requests to the most suitable model or region, and applying security and policy before a request reaches a model. As the agentic web takes shape, the edge continues to serve its traditional delivery role while taking on these new responsibilities.

As this shift accelerated, a deeper realization emerged. The next generation of applications would no longer simply retrieve information. They would generate responses, coordinate with other systems, invoke tools, and make decisions on behalf of users. That shift changes what the internet is expected to do. Connectivity alone is no longer enough. The network itself has to become an intelligent layer that can understand, govern, and optimize AI traffic as it moves between users, models, and services.

Why the internet struggles with AI workloads

The internet was built to transport data efficiently, not to manage AI interactions. Traditional web traffic is predictable and can often be cached or routed using established rules. AI traffic is fundamentally different. Each request carries conversational context, may interact with multiple models or external tools, and varies in computational cost, making it more dynamic, stateful, and expensive to handle.

The challenge is not simply reducing inference latency by moving compute closer to users. Most response time still comes from the model generating its output. Instead, the greater opportunity lies in making the network itself smarter. Semantic caching can reduce unnecessary inference by reusing responses to similar prompts, while intelligent routing directs requests to the fastest or most cost-effective model. Security moves here too. A traditional WAF was built to block malicious requests by pattern; the AI-era equivalent has to inspect meaning, catching prompt-injection and jailbreak attempts, redacting sensitive data, and enforcing content-safety policy on both the prompt and the model’s response. Modern WAFs are evolving in this direction, adding AI-specific protections at the edge so that policy is applied before a request reaches a model.

As AI adoption grows, these capabilities are becoming as important as bandwidth itself. The internet is evolving beyond a transport network into an intelligent traffic layer that routes, secures, and optimizes AI workloads, providing the foundation for the next generation of AI-native applications.

Conversation with Karthik Uthaman

To understand what that next generation of internet infrastructure looks like, we spoke with Karthik Uthaman, a Microsoft engineer and former Amazon infrastructure architect. He shared his perspective on edge computing, AI infrastructure, distributed systems, and the architectural shifts required to support AI-native applications.

Q: How would you describe an AI-native network architecture?

A: The web as we know it is evolving into an Agentic Web where bots and agents have become first-class citizens, increasingly operating on our behalf. An AI-native network architecture is designed for that shift. Instead of treating the network as a passive pipe that moves bytes between a browser and a server, it treats the network as an active, intelligent layer that understands and governs AI traffic as it flows between users, models, tools, and other agents. 

What makes it “AI-native” is not that computation moves to any particular place. Most of a model’s response time comes from generating the answer, and relocating hardware does not change that. What changes is that the network itself becomes aware of the traffic it carries, able to route, reuse, authenticate, and apply policy on behalf of the agents using it. The specific mechanisms, from semantic caching to an AI Perimeter, all follow from one idea: intelligence belongs in the traffic layer, not only at the endpoints.

Q: How does this differ from traditional CDN or cloud models?

A: The industry has a name for the emerging response. Analysts and vendors increasingly describe an “AI Perimeter”: a layer at the edge of the network that sits between agents and the models, tools, and data they reach for, making decisions about identity, cost, safety, and caching before a request ever reaches an origin.

It is the same architectural instinct that produced the CDN, applied to a new class of traffic. The question of who builds and controls that perimeter has become a central point of competition among cloud providers and CDN operators.

Q: What limitations in today’s internet infrastructure most directly motivated your work?

A: The difficulty is not simply that AI traffic is heavier. It is that AI traffic behaves differently along nearly every dimension that CDNs were built around. A traditional web request is stateless, deterministic, and cheap: the same URL returns the same bytes, and the response can be cached by hashing that URL. 

A request to a language model has none of these properties. It carries a conversational state that may span many turns. It is non-deterministic, so the same prompt can produce different completions. It is expensive, with costs measured in tokens rather than fractions of a cent. And it cannot be cached by URL, because the meaningful content is a natural-language prompt rather than an address.

Q: What were the most difficult challenges in designing distributed AI inference systems?

A: Latency is the first. Moving inference to the edge shortens the distance to the user, but edge locations have less capacity than centralized regions, so the system must continuously decide where each request should run. That decision depends on model placement, GPU availability, and how much of a prior computation can be reused.

Reuse itself introduces risk. Karthik et al.’s Semantic Caching Patent (U.S. Pat App. No. 2026/0017456) trades exact correctness for speed, and the central question is how similar two prompts must be before one answer can serve both. Set the threshold too loosely and the system returns a confidently wrong answer; set it too tightly and the cache rarely helps. The right threshold depends on the workload, and getting it wrong carries consequences that a traditional cache miss never did.

Distributed state compounds the difficulty. Multi-turn conversations and long-running agent tasks require memory that persists and stays consistent across many locations, which is demanding in any distributed system and more so when the data changes as an agent works. Security runs through all of it. Anomaly detection, traffic inspection, and cache-integrity protection reflect a long-standing concern with distributed-system safety, and that concern only grows in an agentic setting, where indirect prompt injection, malicious instructions hidden in a web page, document, or tool response that an agent reads as trusted input, remains among the least-solved problems in the field. Finally, all of this must scale globally and remain reliable, which is the discipline that CDN engineering has always demanded.

Q: How do AI workloads change network requirements compared to traditional media delivery?
A: The clearest consequence is that the edge takes on a new role without giving up its old one. The same infrastructure that streams a film to a living room now also runs small language models for intelligent routing AI workloads and applies policy for an agent. That expansion, content delivery and intelligence side by side, has effects that reach well beyond engineering.

One is economic. For years, the value of a web query flowed to the publisher that answered it and the search engine that routed the visitor. As AI systems answer questions directly, that flow is changing. Research from Pew found that when an AI-generated summary appears above search results, users click through to a source roughly half as often as when one does not. The value once captured by publishers is migrating toward model providers, agent platforms, and the edge layer that mediates access, which is precisely why so many infrastructure companies are investing in it.

Another consequence is the arrival of autonomous transactions. When an agent completes a purchase on a user’s behalf, payment and trust have to be handled without a person in the loop. 

Uthaman notes that transaction processing at edge servers anticipated part of this shift, describing how an edge node could carry out a data transaction with parties outside the delivery network, infrastructure for commerce conducted at the edge rather than at a distant origin. The broader industry is now standardizing that capability through a wave of agent-payment protocols, and the edge is again the natural place to enforce it.

Q: Your work also explores extending compute beyond the edge and into space. How do satellites fit into the future of distributed infrastructure?

A: A point of presence in space. The most distinctive theme extends the edge beyond the ground. Our patent, U.S. Patent No. 11,201,939, which was granted in 2021, describes a content-and-compute delivery platform that uses satellites as nodes of a delivery network: a satellite receives a request, serves it if it can, and forwards it otherwise. 

Filed in 2019, before satellite-based delivery became a mainstream part of internet infrastructure, the work anticipates a model in which compute and caching are distributed not only across cities but across orbit, reaching users and devices that terrestrial infrastructure serves poorly.

Q: How will the balance evolve between cloud, edge, and satellite systems?

A: The near-term direction is a hybrid architecture in which cloud, edge, and satellite each handle the work they are best suited to: large-scale training and heavy inference in centralized regions, latency-sensitive inference at the edge, and coverage for remote or mobile users from orbit. Rather than replacing the edge’s traditional role in content delivery, AI is expanding its capabilities through intelligent routing, semantic caching, security, and policy enforcement while continuing to support conventional CDN traffic.

The protocol foundations for coordinating this are consolidating quickly. In a little over a year, the industry moved from having no standard way for agents to use tools or talk to one another to a recognizable stack, including the Model Context Protocol, the Agent2Agent protocol, and Microsoft’s NLWeb, with several of these efforts now placed under shared governance at the Linux Foundation.

AI-native infrastructure: Patenting activity

The rapid growth in AI-native infrastructure patent filings reflects the internet’s transition from content delivery to distributed AI computation. Global patent applications increased from just 56 filings in 2015 to more than 2,000 in 2024 as cloud providers, networking companies, and AI developers invest in architectures that can support real-time inference, intelligent routing, and autonomous agents. Rather than optimizing how data is delivered, innovation is increasingly focused on where computation takes place and how AI workloads are orchestrated across cloud, edge, and emerging satellite networks.

As AI-native architectures mature, researchers are exploring new ways to embed intelligence directly into network infrastructure. Reflecting this trend, Uthaman’s work focuses on moving computation closer to users through distributed inference, programmable edge networks, and semantic caching for retrieval-augmented AI applications. Instead of treating the network as a transport layer, these architectures enable it to participate in decision-making, state management, and inference execution.

We asked Karthik which innovations he believes will define the next generation of AI infrastructure. He points to AI-native network architectures that treat the edge as an execution layer for AI models rather than simply a delivery platform. He also highlights the emergence of the AI Perimeter, a programmable layer that manages identity, security, caching, and routing before requests reach centralized infrastructure. Another area of innovation is semantic caching, which enables retrieval-augmented language models to reuse responses based on the meaning of natural-language prompts instead of traditional URL-based caching, reducing inference costs while preserving enterprise security and access controls.

The internet as a distributed compute platform

Looking ahead, it’s unlikely that one model will define the future of infrastructure. What’s emerging instead is a hybrid system, multiple layers working together, each optimized for a different role.

Cloud will continue to handle centralized, high-density compute. Edge systems will take on low-latency processing closer to users and devices. And satellite networks will extend coverage while enabling more globally distributed coordination.

Together, these layers form a unified system where workloads aren’t fixed, they’re dynamically allocated based on latency, resource availability, and location. The network becomes adaptive, shifting computation across layers in real time. The shift here is as much conceptual as it is technical. The internet is no longer just moving data, it’s starting to function like a globally distributed compute fabric.

Looking beyond connectivity

There’s a broader pattern behind all of this. Breakthroughs don’t usually start with fully formed solutions, they start with a shift in perspective, often a question that challenges existing assumptions.

In this case, it began with two engineers inside one of the world’s most advanced infrastructure environments asking a simple question: is connectivity alone enough? From there, the thinking expanded, into new architectures, and eventually into patents that hint at where the industry is heading.

For anyone navigating the fast-moving space of AI and distributed systems, the takeaway is straightforward. The next phase of infrastructure won’t just be about how efficiently data moves, but about where and how intelligence is executed. That shift isn’t just technical, it’s strategic. Because increasingly, the future of the internet will be defined not by connectivity alone, but by where intelligence live.

---

OUR FEATURED GUEST