Computing-Aware Traffic Steering Q. Li
Internet-Draft H. Wang
Intended status: Informational Pengcheng Laboratory
Expires: 27 February 2027 Y. Jiang
Tsinghua Shenzhen International Graduate School & Pengcheng Laboratory
M. Xu
Tsinghua University
G. Muntean
Dublin City University
26 August 2026
A Framework of Intelligence Delivery Network (IDN) for Deep Learning
Inference
draft-li-cats-idn-01
Abstract
The rapid growth of AI-powered applications is placing increasing
pressure on existing Internet infrastructures. To support more
scalable, latency-aware, and privacy-enhanced AI inference services,
this document introduces the Intelligence Delivery Network (IDN), a
network architecture in which intelligence capabilities are treated
as network services that can be described, placed, routed to, reused,
and secured across distributed heterogeneous computing nodes. This
document describes the motivation, deployment assumptions, system
model, architectural components, terminology, and security
considerations for IDN. It does not specify protocol details or
concrete implementation procedures, which are left to future
documents.
About This Document
This note is to be removed before publishing as an RFC.
Status information for this document may be found at
https://datatracker.ietf.org/doc/draft-li-cats-idn/.
Discussion of this document takes place on the Computing-Aware
Traffic Steering Working Group mailing list (mailto:cats@ietf.org),
which is archived at https://mailarchive.ietf.org/arch/browse/cats/.
Subscribe at https://www.ietf.org/mailman/listinfo/cats/.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Li, et al. Expires 27 February 2027 [Page 1]
Internet-Draft IDN August 2026
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on 27 February 2027.
Copyright Notice
Copyright (c) 2026 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents (https://trustee.ietf.org/
license-info) in effect on the date of publication of this document.
Please review these documents carefully, as they describe your rights
and restrictions with respect to this document. Code Components
extracted from this document must include Revised BSD License text as
described in Section 4.e of the Trust Legal Provisions and are
provided without warranty as described in the Revised BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Background and Challenges . . . . . . . . . . . . . . . . . . 4
2.1. AI Inference Workload Characteristics . . . . . . . . . . 4
2.2. Limitations of Cloud-Centric Serving . . . . . . . . . . 5
2.3. Intelligence Delivery Versus Content Delivery . . . . . . 6
2.4. Architectural Challenges . . . . . . . . . . . . . . . . 6
3. Intelligence Delivery Network Framework . . . . . . . . . . . 7
3.1. Design Principles . . . . . . . . . . . . . . . . . . . . 7
3.2. System Model and Deployment Assumptions . . . . . . . . . 8
3.3. High-Level Architecture . . . . . . . . . . . . . . . . . 8
3.4. Intelligence Capability Abstraction . . . . . . . . . . . 11
3.5. Compute Resource Integration . . . . . . . . . . . . . . 12
3.6. Demand-Driven Capability Deployment . . . . . . . . . . . 12
3.7. Capability-Aware Service Routing . . . . . . . . . . . . 13
3.8. State-Aware Caching and Reuse . . . . . . . . . . . . . . 14
3.9. Model Evolution and Lifecycle . . . . . . . . . . . . . . 15
4. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 16
5. Security, Privacy, and Trust Considerations . . . . . . . . . 19
5.1. Data Privacy and Locality . . . . . . . . . . . . . . . . 19
5.2. Model Integrity and Authenticity . . . . . . . . . . . . 20
Li, et al. Expires 27 February 2027 [Page 2]
Internet-Draft IDN August 2026
5.3. Provider-Scoped Trust and Delegation . . . . . . . . . . 20
5.4. Trust Schema and Object Authentication . . . . . . . . . 21
5.5. Node Trust and Execution Evidence . . . . . . . . . . . . 21
5.6. Reuse, Isolation, and Information Leakage . . . . . . . . 21
5.7. Availability and Abuse Considerations . . . . . . . . . . 22
6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 22
7. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 22
8. Informative References . . . . . . . . . . . . . . . . . . . 22
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 24
1. Introduction
The rapid development of Large Language Models (LLMs) has made AI-
powered applications increasingly common. Examples include
interactive assistants, code copilots, robotics, multimodal
perception systems, and enterprise knowledge services. For these
applications, the core service objective is to complete an inference
task with acceptable latency, quality, cost, privacy, and operational
stability. This motivates an Internet architecture that can support
the delivery of intelligence as a network service.
The dominant deployment model for AI inference remains cloud-centric.
In this model, user inputs are sent across the wide-area network to
remote or regional data centers, where inference is performed by
centralized serving stacks. Such stacks continue to improve
scheduling, batching, memory management, model-variant selection, and
accelerator utilization. However, centralized inference faces
fundamental scalability pressure because each request can consume
substantial accelerator time, memory, and serving capacity. As AI
usage expands to continuous, multimodal, and personalized services,
serving all requests from a small number of remote sites can increase
infrastructure cost, create resource contention, increase queueing
delay, and reduce availability during demand bursts.
AI requests are often geographically, organizationally, or
application-wise clustered. Compute resources are also distributed
across cloud regions, regional points of presence (PoPs), edge sites,
enterprise sites, and local devices. However, current serving
architectures do not expose a common mechanism to describe, select,
and coordinate these resources for inference. As a result, useful
compute capacity remains underutilized, repeated requests for similar
capabilities or reusable state are not handled efficiently, and
sensitive data needs to cross administrative, geographic, or policy
boundaries even when local execution is feasible.
Inspired by Content Delivery Networks (CDNs), this document
introduces the Intelligence Delivery Network (IDN), which delivers AI
capabilities as network services. IDN describes, places, routes to,
Li, et al. Expires 27 February 2027 [Page 3]
Internet-Draft IDN August 2026
reuses, and secures intelligence across heterogeneous resources
according to demand locality, resource availability, service
requirements, and policy constraints. Analogous to how CDNs cache
content closer to users to improve delivery performance [DilleyCDN]
[Akamai], IDN places intelligence capabilities and reusable inference
state closer to request sources in order to improve service quality
and scalability. Unlike CDNs, however, IDN operates on models,
service instances, runtime state, and policy-constrained inference
tasks rather than on static content objects.
IDN is related to Computing-Aware Traffic Steering (CATS) in that an
inference service can have multiple geographically distributed
service instances, and traffic steering needs to account for both
network conditions and computing conditions [CATSFramework]
[CATSUseCases]. IDN adds AI-specific abstractions to this setting,
including intelligence capability descriptors, model and runtime
artifacts, state descriptors, cacheability and sharing scope,
capability lineage, policy labels, and execution receipts. This
document frames IDN as an application of CATS principles to
distributed AI inference, not as a replacement for CATS mechanisms.
The remainder of this document is organized as follows. Section 2
discusses background and challenges. Section 3 presents the IDN
architectural framework. Section 4 defines terminology. Section 5
discusses security, privacy, and trust considerations.
2. Background and Challenges
This section describes the characteristics of AI inference workloads
and identifies the architectural challenges that motivate IDN.
2.1. AI Inference Workload Characteristics
AI inference workloads differ from traditional web services and from
model training workloads in several ways.
First, many inference requests are interactive and latency-sensitive.
A user-facing assistant, a real-time translation system, or a
perception pipeline may be sensitive to time-to-first-token, time-
per-output-token, tail latency, and service degradation under
overload. This differs from batch-oriented training and from many
static content-delivery workloads.
Second, inference workloads are geographically and organizationally
distributed. Demand may be concentrated in particular regions,
access networks, enterprises, campuses, or application communities.
These demand patterns can change quickly as applications, events, and
user behavior change.
Li, et al. Expires 27 February 2027 [Page 4]
Internet-Draft IDN August 2026
Third, inference frequently operates on user-generated, enterprise,
or sensor data. Such data may be sensitive, regulated, or subject to
locality constraints. Moving all input data to remote data centers
can increase wide-area traffic and may create privacy, governance, or
compliance concerns.
Fourth, inference workloads are heterogeneous. Different tasks
require different tradeoffs among quality, latency, model size, cost,
and privacy. A single AI service may include simple tasks that can
be served by a compact model variant and complex tasks that require a
larger or more capable model. Similar heterogeneity has been
observed in distributed inference and mobile vision systems [Elf].
2.2. Limitations of Cloud-Centric Serving
Centralized cloud infrastructure plays an important role in AI
inference because it provides abundant computing resources and can
host the largest models. However, relying solely on a cloud-centric
deployment model introduces several limitations.
Routing all requests to a limited set of data centers can increase
end-to-end latency and wide-area traffic. Centralized serving can
also create resource contention and queueing delay during demand
bursts. Because AI inference consumes accelerator time and memory
per request, it cannot be scaled in exactly the same way as delivery
of static bytes from cache.
Cloud-centric serving can also make poor use of distributed compute
resources. Regional sites, edge servers, customer-premise
environments, and local devices may be able to execute selected
capability variants or reusable stages of a workflow, but they need
to be admitted, described, monitored, and selected safely. Without a
common architectural view, those resources are difficult to use as
part of a coherent inference service.
Finally, centralized serving can be misaligned with policy
constraints. A request involving private or regulated data may need
local execution, stronger isolation, tenant-specific controls, or
processing within a jurisdiction. IDN treats such constraints as
part of service placement and routing.
Li, et al. Expires 27 February 2027 [Page 5]
Internet-Draft IDN August 2026
2.3. Intelligence Delivery Versus Content Delivery
IDN is inspired by CDNs, but intelligence is harder to deliver than
content. A content object is commonly a stable byte sequence that
can be cached and validated with relatively well-understood HTTP
caching mechanisms [RFC9111]. CDN request routing selects an
appropriate serving location using mechanisms such as DNS
redirection, anycast, HTTP redirection, or combinations of these
methods [RFC3568].
An AI service may depend on a base model, adapters, tokenization
logic, safety filters, retrieval indexes, runtime packages, tool
integrations, and reusable inference state such as prompt prefixes,
activations, or key-value (KV) caches. These elements have
compatibility constraints, hardware requirements, loading costs,
sharing restrictions, and policy implications. The selected service
instance is therefore not necessarily the geographically closest
node. It is the node, service instance, or execution plan that best
satisfies capability, quality, state, load, network, and policy
constraints.
This distinction motivates a capability-centric and state-aware
architecture. IDN needs a common way to describe what intelligence
is requested, what capabilities exist, what resources can execute
them, what reusable state exists, how state may be shared, and which
trust and policy constraints apply.
2.4. Architectural Challenges
IDN is motivated by the following architectural challenges:
* Capability abstraction: AI capability needs to be represented at a
granularity that is more flexible than whole-model replication but
still stable and meaningful as a service object.
* Compute resource integration: Heterogeneous compute nodes need to
advertise resource profiles, dynamic state, locality, supported
runtimes, and trust evidence in a scalable manner.
* Demand-driven deployment: Capability variants need to be deployed,
activated, scaled, migrated, or withdrawn according to demand
locality, resource cost, and policy constraints.
* Capability-aware service routing: Requests need to be steered to
suitable service instances or multi-stage execution plans based on
capability, quality, state affinity, network conditions, compute
load, and policy.
Li, et al. Expires 27 February 2027 [Page 6]
Internet-Draft IDN August 2026
* State-aware caching and reuse: Model artifacts, prompt prefixes,
tensor state, and results may be reusable, but their correctness,
compatibility, mobility, and privacy properties differ.
* Trust management: A distributed inference path needs mechanisms
for model provenance, object authentication, node identity,
delegation, policy verification, and auditability.
3. Intelligence Delivery Network Framework
This section presents the IDN framework. The framework describes how
intelligence capabilities, compute resources, service instances,
reusable state, and trust metadata are organized and used to deliver
distributed inference services.
3.1. Design Principles
The IDN framework is guided by the following design principles:
* Capability-centric delivery: Intelligence is represented as
bounded inference capabilities that can be described, placed,
discovered, selected, and executed independently of a single
physical model checkpoint.
* Provider-controlled hierarchical deployment: The deployment
assumes a provider-managed serving hierarchy spanning cloud,
regional, edge, and local resources. The provider retains control
of IDN-specific functions such as placement, request routing,
caching, trust management, policy enforcement, and updates.
* CATS-compatible steering: IDN service routing is aligned with CATS
concepts. An IDN service may have multiple service instances, and
steering decisions need to account for both network and compute
conditions.
* State-aware operation: Reusable model artifacts and inference
state are treated as named, managed objects whose location,
compatibility, sharing scope, and migration cost affect routing
and caching decisions.
* Policy and trust as first-class constraints: Privacy, locality,
isolation, provenance, attestation, and delegation constraints
influence deployment and routing decisions.
* Incremental evolution: IDN is intended to evolve through
prototypes, provider-domain deployments, and later standardization
of common descriptors and interfaces.
Li, et al. Expires 27 February 2027 [Page 7]
Internet-Draft IDN August 2026
3.2. System Model and Deployment Assumptions
IDN assumes a large-scale AI service environment operated by a single
AI provider, i.e., a single provider-controlled administrative
domain. This assumption is analogous to existing CDN deployments,
i.e., infrastructure may be physically hosted in enterprise, campus,
or customer-premise environments, but the service provider controls
application-level functions such as service software, request
steering, cache policy, updates, monitoring, and safety policy.
The resource hierarchy contains four classes of resources:
* Cloud nodes: centralized data centers or cloud regions that host
the largest foundation models, maintain the most complete
capability set, and provide abundant compute, memory, and storage.
* Regional nodes: regional PoPs, metro-scale data centers, or
interconnection sites that aggregate demand across a large
geographic area or access network. They can host frequently used
capabilities with lower latency than centralized cloud data
centers.
* Edge nodes: access-network PoPs, ISP-side cache clusters, MEC
sites, enterprise edge servers, or campus edge servers. They
operate under tighter resource constraints but provide better
proximity to users and may better satisfy local policy
constraints.
* Local nodes: end devices, customer-premise environments, or
lightweight on-premise deployments where limited capabilities may
execute when latency, privacy, or data locality requirements
dominate.
These resources differ in accelerator type, memory capacity, startup
overhead, model residency, connectivity, reliability, distance to
demand, and trust properties. IDN therefore does not attempt to
replicate every capability everywhere. It seeks to place each
capability at the level of the hierarchy where it can best balance
quality, latency, resource cost, and policy compliance.
The model above is compatible with edge-computing discussions in the
IETF and IRTF [RFC9556], but IDN focuses on AI inference services and
the intelligence-specific metadata required for capability placement,
request routing, caching, and trust management.
3.3. High-Level Architecture
Li, et al. Expires 27 February 2027 [Page 8]
Internet-Draft IDN August 2026
+----------------------------------------+
| IDN Provider Control Plane |
| |
| +-----------+ +----------+ +---------+ |
| | Capability| | Resource | | Trust | |
| | Registry | | Broker | | Manager | |
| +-----------+ +----------+ +---------+ |
| +-----------+ +----------+ +---------+ |
| | Deployment| | Service | | Cache | |
| | Controller| | Router | | Manager | |
| +-----------+ +----------+ +---------+ |
+-------------------+--------------------+
|
Capability, Resource, State, and Policy Views
|
+---------------------------+
| Cloud Data Centers |
| (General-Purpose Models) |
| |
| - Large foundation |
| inference models |
| - Capability distillation |
+-------------+-------------+
|
Capability Distribution
|
+-------------------+--------------------+
| |
+------------+-------------+ +------------+-------------+
| Regional IDN Nodes | | Regional IDN Nodes |
|(Specialized Capabilities)| |(Specialized Capabilities)|
| | | |
| - Domain-specific models | | - Task-oriented models |
| - Cached popular skills | | - Cached popular skills |
+------------+-------------+ +------------+-------------+
| |
Inference| | Inference
Requests | | Requests
| |
+------------+-------------+ +-------------+-------------+
| Edge Nodes | | Edge Nodes |
|(Lightweight Capabilities)| | (Lightweight Capabilities)|
| | | |
| - Latency-sensitive | | - Privacy-sensitive |
| inference tasks | | inference tasks |
+------------+-------------+ +-------------+-------------+
| |
+-------------------+--------------------+
Li, et al. Expires 27 February 2027 [Page 9]
Internet-Draft IDN August 2026
|
+------+------+
| Local Nodes |
| Limited |
| capabilities|
+------+------+
|
Clients and
Upstream Applications
At a high level, clients or upstream applications issue intelligence
requests. The provider control plane uses capability descriptors,
resource profiles, state descriptors, and policy information to
select an appropriate service instance or execution plan. A plan may
use a single node or may involve multiple stages, such as execution
of a lightweight capability near the user, fallback to a stronger
regional or cloud variant, or reuse of cached state.
The six main architectural components are:
* Capability registry: stores and advertises intelligence capability
descriptors, capability versions, lineage, and policy labels.
* Resource broker: integrates compute nodes, aggregates resource
profiles, and exposes service-oriented summaries to placement and
routing functions.
* Deployment controller: decides where capability variants and
supporting artifacts should be placed, activated, scaled,
migrated, or withdrawn.
* Service router: maps each request to a service instance or
execution plan using capability, network, compute, state, and
policy information.
* Cache manager: manages model artifacts, prompt prefixes, tensor
state, result objects, and their sharing and invalidation
policies.
* Trust manager: manages provider-scoped identities, delegations,
object authentication, node trust evidence, policy verification,
and execution receipts.
These components are logically distinct. A deployment may combine
them or distribute them across controllers, brokers, site agents, and
service meshes.
Li, et al. Expires 27 February 2027 [Page 10]
Internet-Draft IDN August 2026
3.4. Intelligence Capability Abstraction
A fundamental challenge for IDN is that the intelligence embodied in
a foundation model is not naturally exposed at the right granularity
for network delivery. A large model may support many tasks, but
treating the entire model checkpoint as the deployment unit makes
distribution too coarse-grained. A node would either need to
replicate the full checkpoint or forgo the capability entirely.
IDN defines an intelligence capability as a bounded and deployable
unit of service competence derived from one or more foundation
models. A capability does not need to preserve the full generality
of the originating model. It may represent the subset of
intelligence needed to support a class of requests under explicit
quality, latency, security, and resource constraints. Capabilities
may be produced through provider-specific mechanisms such as
distillation, specialization, compression, adapter-based refinement,
quantization, or other optimization techniques. IDN does not mandate
a capability extraction method.
IDN views capabilities at three levels:
* Capability class: a broad function or service category, such as
translation, code assistance, visual recognition, or vertical-
domain question answering.
* Capability variant: a version of a capability class with specific
tradeoffs among accuracy, latency, model size, context length,
privacy, cost, and trust requirements.
* Deployable realization: a concrete implementation of a capability
variant, such as a distilled model, quantized model, adapter set,
runtime package, or other optimized artifact for a specific
hardware or runtime environment.
This separation allows the IDN control plane to reason about
capability classes and variants while allowing execution platforms to
use the concrete realization that best fits their resources. The
granularity should be service-effective: small enough to enable
differentiated deployment across heterogeneous nodes, but large
enough to remain stable, reusable, and meaningful as a service
object.
Li, et al. Expires 27 February 2027 [Page 11]
Internet-Draft IDN August 2026
3.5. Compute Resource Integration
Distributed compute nodes need to be integrated into IDN as
manageable suppliers of execution capability. A compute node joins
IDN by registering with a provider controller, broker, or site agent
that is responsible for local resource management. Registration
creates or updates a resource profile describing static and dynamic
properties of the node or site.
Resource integration is hierarchical. Fast-changing information such
as queueing condition, available accelerator memory, model residency,
node health, local connectivity, and recent performance can be
collected by local agents. Coarser summaries can then be propagated
to higher-level controllers. This approach avoids exposing raw node-
level telemetry globally and is compatible with CATS-style
abstractions in which traffic steering uses both network and
computing state [CATSFramework].
A resource profile is service-oriented. It exposes what the node or
site can provide to the IDN scheduler, not every hardware detail.
For example, a profile may indicate supported accelerator families,
memory class, supported inference runtimes, resident models, current
load range, locality, jurisdiction, operational policy constraints,
and trust evidence.
Resource integration is also a trust operation. Before a node hosts
sensitive capabilities or serves protected requests, it needs to
expose verifiable identity and relevant security properties. The
exact attestation or verification mechanism is deployment-specific.
3.6. Demand-Driven Capability Deployment
After capabilities have been abstracted and resources have been
integrated, IDN determines where each capability should be placed.
The objective is not to replicate every capability at every node, but
to position each capability where it can best balance demand
locality, quality, latency, resource cost, network cost, and policy
risk.
Deployment can be proactive or reactive. A provider may pre-position
a popular capability variant in a region based on expected demand.
It may also activate, migrate, scale out, scale back, or withdraw a
capability as demand and resource conditions change. When a
recurring task pattern becomes frequent in a region, the provider may
derive or select a compact specialization and deploy it to suitable
regional or edge nodes. When demand subsides, the provider may
remove it to recover constrained resources.
Li, et al. Expires 27 February 2027 [Page 12]
Internet-Draft IDN August 2026
A deployment decision considers at least the following inputs:
* demand locality and predicted reuse;
* capability quality tier and resource requirement;
* model artifact size, loading cost, initialization cost, and memory
footprint;
* node capacity, current load, reliability, and locality;
* network transfer, backhaul, and egress cost;
* privacy, jurisdiction, isolation, and trust constraints; and
* rollback, version compatibility, and operational safety
requirements.
Deployment is a closed-loop process. Demand surges, resource
failures, model updates, policy changes, and security events can
invalidate a previously effective placement. IDN therefore needs
mechanisms for controlled rollout, rollback, version coexistence, and
deprecation.
3.7. Capability-Aware Service Routing
Once capabilities are deployed, IDN needs to determine how each
incoming request should be served. Service routing in IDN selects an
execution location, a capability variant, and an execution context.
The selected plan may be a single service instance or a multi-stage
path across service instances.
An IDN service instance is a reachable execution endpoint that
provides an IDN service or part of an IDN service. It is hosted on a
compute node and implements one or more deployable realizations.
Multiple service instances may provide the same capability variant
with different locality, load, trust, state, and cost properties.
A request may specify the requested capability explicitly, or the
provider may infer it from application context. It may also include
quality targets, response-time requirements, privacy or locality
constraints, state-affinity information, tenant information, and cost
or resource preferences. The service router constructs the feasible
set of service instances or execution plans that satisfy the request
constraints and selects among them according to service policy.
Relevant routing inputs include:
Li, et al. Expires 27 February 2027 [Page 13]
Internet-Draft IDN August 2026
* network path latency, loss, congestion, and locality;
* queueing delay, available accelerator memory, model residency, and
expected execution time;
* capability quality tier, context limit, and compatibility;
* state availability, state transfer cost, and state-affinity value;
* loading or warm-up cost for non-resident capabilities;
* policy constraints, isolation requirements, and trust evidence;
and
* load-balancing, admission-control, and cost policies.
The closest node is not always the best node. A nearby edge node may
lack the requested capability, may be overloaded, may not hold
reusable state, or may not satisfy a policy constraint. Conversely,
a more distant regional node may hold a warm service instance or
reusable state that provides lower end-to-end delay. IDN routing is
therefore a service-quality control loop rather than only a path-
selection mechanism.
State-awareness is especially important for LLM serving. Interactive
sessions, long-context inference, and split execution can accumulate
prompt prefixes, KV caches, retrieved context, embeddings, and other
intermediate state. Moving, reconstructing, or discarding such state
can dominate the latency gain from choosing a shorter network path.
Existing LLM-serving work has shown that memory management and KV-
cache handling materially affect latency and throughput
[PagedAttention].
3.8. State-Aware Caching and Reuse
Caching is essential for improving the efficiency and service quality
of distributed inference, but IDN caching is broader than
conventional content caching. IDN treats caching as state
management: the system decides what state should be cached, where it
should be placed, how it may be shared, when it should be migrated,
and when it should be invalidated.
IDN distinguishes four classes of cacheable objects:
* Artifact caches: stable deployment objects such as model weights,
adapters, quantized variants, tokenizers, safety components, and
runtime packages.
Li, et al. Expires 27 February 2027 [Page 14]
Internet-Draft IDN August 2026
* Prefix caches: reusable prompt modules, system instructions,
retrieved-context templates, or other common input prefixes.
* Tensor-state caches: intermediate execution state such as KV
caches or other activations that can accelerate later inference.
* Result caches: reusable outputs for identical or semantically
equivalent inputs under compatible model, decoding, and policy
settings. Result caches can also include deterministic
subcomputations such as embeddings, retrieval outputs, or tool
results.
Object type is separate from sharing scope. A cached object may be
public, tenant-shared, session-private, user-private, or hardware-
bound. The sharing scope affects cache admission, routing, access
control, encryption, and eviction.
To enable cached objects to be discovered and managed across dynamic
deployments, IDN uses stable names or descriptors rather than only
the address of the node that currently stores the object. This
follows the ICN principle of decoupling object identity from object
location [NDN]. However, IDN does not make location irrelevant. The
control plane needs to resolve each named object to one or more valid
copies and decide whether to route a request to a node that already
holds the object, migrate the object, recompute it, or fall back to a
remote execution plan.
Cache usefulness depends on predicted reuse, latency reduction, state
transfer cost, storage cost, privacy cost, compatibility, and future
routing decisions. Techniques for prompt reuse, KV-cache
compression, and multi-tenant KV-cache reuse provide examples of the
types of state that may be useful in IDN, but the IDN architecture
does not mandate a specific cache implementation [PromptCache]
[CacheGen] [KVShare].
3.9. Model Evolution and Lifecycle
Inference capabilities within an IDN are expected to evolve over
time. Models may be updated, replaced, refined, or specialized as
new data becomes available, as usage patterns change, or as
application requirements evolve. The framework supports incremental
updates and the coexistence of multiple capability versions, enabling
gradual transitions rather than requiring global or disruptive
replacements.
Model evolution may be driven by multiple sources. Updates can be
produced centrally, for example through cloud-side retraining,
refinement, or distillation of models, and subsequently distributed
Li, et al. Expires 27 February 2027 [Page 15]
Internet-Draft IDN August 2026
to appropriate locations within the IDN. In addition, where
permitted by policy and regulatory constraints, inference
capabilities deployed at edge or near-user locations may be locally
adapted using user-provided or locally observed data. Such local
adaptation may follow federated or privacy-preserving learning
approaches, in which locally derived updates contribute to global
model improvement without requiring raw data to leave the local
environment.
Lifecycle management of inference capabilities includes deployment,
update, versioning, deprecation, and removal. Different versions of
a capability may coexist at the same or different locations, allowing
the framework to balance stability, performance, and innovation.
While this document does not specify how lifecycle management
processes are implemented, it assumes that mechanisms for controlled
rollout, compatibility management, and rollback are necessary to
ensure operational stability and consistency within an IDN.
4. Terminology
This section defines terminology used throughout this document.
Phrases in upper-case refer to other defined terms.
AI PROVIDER
An organization or administrative entity that controls an IDN
deployment. In the deployment model assumed by this document, the AI
PROVIDER controls capability placement, request routing, caching
policy, trust management, policy enforcement, software updates, and
monitoring, although some nodes may be physically hosted in other
environments.
CAPABILITY CLASS
A broad service function provided by an INTELLIGENCE CAPABILITY, such
as translation, code assistance, visual recognition, or domain-
specific question answering.
CAPABILITY DESCRIPTOR
Metadata that describes an INTELLIGENCE CAPABILITY for placement,
discovery, routing, compatibility checking, and policy enforcement.
A CAPABILITY DESCRIPTOR can include task, quality tier, response-time
characteristic, security label, resource requirement, supported
runtime, lineage, version, and cost class.
CAPABILITY VARIANT
Li, et al. Expires 27 February 2027 [Page 16]
Internet-Draft IDN August 2026
A version of a CAPABILITY CLASS that reflects a particular tradeoff
among quality, latency, model size, context length, cost, privacy,
and trust requirements.
COMPUTE NODE
A cloud, regional, edge, or local resource that can host one or more
SERVICE INSTANCES or IDN OBJECTS. A COMPUTE NODE may correspond to a
server, cluster, site, device, or managed execution environment.
DEPLOYABLE REALIZATION
A concrete implementation of a CAPABILITY VARIANT, such as a
distilled model, quantized model, adapter set, runtime package, or
other optimized artifact that can run on a particular hardware or
runtime environment.
EXECUTION RECEIPT
A record describing how an INTELLIGENCE REQUEST was served. An
EXECUTION RECEIPT can include the selected plan, SERVICE INSTANCE,
capability version, node attestation, object signatures, delegation
evidence, policy verdict, cache usage, routing decision, and
accounting information.
IDN
Intelligence Delivery Network. IDN is an architectural framework in
which INTELLIGENCE CAPABILITIES and reusable inference state are
described, placed, routed to, reused, and secured across provider-
managed cloud, regional, edge, and local resources.
IDN NODE
A COMPUTE NODE that participates in an IDN deployment. This term is
retained for readability; this document otherwise uses COMPUTE NODE
when referring to the resource abstraction.
IDN OBJECT
A named or described object managed by an IDN. Examples include
CAPABILITY DESCRIPTORS, model artifacts, adapters, runtime packages,
prompt-prefix objects, tensor-state objects, result objects, and
EXECUTION RECEIPTS. An IDN OBJECT may have a location-independent
name and authenticated metadata.
INFERENCE TASK
Li, et al. Expires 27 February 2027 [Page 17]
Internet-Draft IDN August 2026
A task or function for which inference is requested, such as
translation, summarization, object recognition, code completion,
retrieval-augmented question answering, or domain-specific
classification.
INTELLIGENCE
The service competence encoded in one or more models, adapters,
runtime components, indexes, and related artifacts. In this
document, INTELLIGENCE is treated as a distributable and reusable
service capability.
INTELLIGENCE CAPABILITY
A bounded and deployable unit of service competence derived from one
or more foundation models. An INTELLIGENCE CAPABILITY supports a
class of INFERENCE TASKS under specified quality, latency, resource,
security, and policy constraints.
INTELLIGENCE REQUEST
A request to perform an INFERENCE TASK using an explicitly specified
or implicitly inferred INTELLIGENCE CAPABILITY. An INTELLIGENCE
REQUEST can include quality targets, response-time requirements,
policy constraints, state-affinity information, tenant information,
and cost or resource preferences.
LOCATION-INDEPENDENT NAME
A name or descriptor that identifies an IDN OBJECT independently of
the node that currently stores or serves it. The control plane can
resolve the name to valid copies or decide to migrate, recompute, or
ignore the object according to policy.
MODEL
A provider-controlled artifact or set of artifacts used to implement
an INTELLIGENCE CAPABILITY. A MODEL can include a base model, fine-
tuned model, distilled model, quantized model, adapter, tokenizer,
safety component, or related runtime dependency.
RESOURCE PROFILE
Metadata describing what a COMPUTE NODE or site can execute and under
which current conditions. A RESOURCE PROFILE can include accelerator
type, memory, storage, supported runtime, locality, load, model
residency, utilization, trust evidence, policy constraints, and cost
signals.
Li, et al. Expires 27 February 2027 [Page 18]
Internet-Draft IDN August 2026
SERVICE INSTANCE
A reachable execution endpoint that provides an IDN service or part
of an IDN service. A SERVICE INSTANCE is hosted on a COMPUTE NODE
and implements one or more DEPLOYABLE REALIZATIONS. Multiple SERVICE
INSTANCES may provide the same capability with different locality,
load, state, trust, or cost properties.
SERVICE ROUTING
The process of selecting a SERVICE INSTANCE or execution plan for an
INTELLIGENCE REQUEST based on capability, quality, network
conditions, compute state, state affinity, policy, trust, and cost
constraints. SERVICE ROUTING is the IDN-specific use of CATS-
compatible steering.
STATE DESCRIPTOR
Metadata describing a reusable IDN OBJECT such as a model artifact,
prompt prefix, tensor state, or result object. A STATE DESCRIPTOR
can include a LOCATION-INDEPENDENT NAME, object type, compatibility
hash, size, sharing scope, access policy, privacy label, signature
metadata, encryption metadata, decoding configuration, reuse
statistics, and migration cost.
STATE-AWARE CACHING
Caching and reuse of model artifacts, prompt prefixes, tensor state,
result objects, or related inference state while accounting for
compatibility, sharing scope, privacy, location, transfer cost, and
policy.
5. Security, Privacy, and Trust Considerations
IDN distributes inference capabilities, model artifacts, reusable
state, and execution across multiple locations. This distribution
improves scalability and locality, but it also creates security,
privacy, and trust requirements that need to be visible to the
architecture.
5.1. Data Privacy and Locality
Inference requests often include user, enterprise, sensor, or
application data. Such data may be sensitive or regulated. IDN can
reduce unnecessary data movement by enabling selected inference tasks
to execute closer to the data source, but it also increases the
number of places where data may be processed.
Li, et al. Expires 27 February 2027 [Page 19]
Internet-Draft IDN August 2026
Privacy and locality requirements therefore need to be represented as
scheduling and routing constraints. A request may require execution
within a particular site, jurisdiction, tenant boundary, or isolation
domain. A request may also prohibit sharing or caching of derived
state. These constraints influence capability placement, service
routing, cache admission, and object reuse.
5.2. Model Integrity and Authenticity
IDN relies on model artifacts, adapters, runtime packages, cached
state, and result objects that may be distributed and reused. A
tampered or unauthorized object could produce incorrect, unsafe, or
policy-violating output. IDN deployments therefore need mechanisms
to verify that a capability, model artifact, state object, or
execution receipt corresponds to an authorized version and originates
from an authorized producer.
Architectural hooks include signed capability descriptors, object
signatures, compatibility hashes, authenticated metadata, provenance
records, policy labels, controlled rollout, invalidation, and
rollback. These hooks are especially important for reusable state
because caches and routers should not have to trust a storage
location merely because it holds an object.
5.3. Provider-Scoped Trust and Delegation
The deployment assumption in this document is a provider-controlled
environment. Under this assumption, the trust infrastructure can be
scoped to the provider domain rather than relying on a global
certification hierarchy. SDSI provides one relevant design
principle: principals can be represented by public keys, local names
and groups can express authorization, and signed certificates can
delegate restricted authority [SDSI].
An IDN provider can use a provider root key as a trust anchor and
delegate authority to regional controllers, site agents, service
runtimes, or object-producing components. Verification of a
capability descriptor, state object, or execution receipt can then
trace signed metadata through a delegation chain back to the provider
root. Each delegation can restrict what the delegated principal is
allowed to sign, update, host, or authorize.
The exact key-management, revocation, rotation, and audit mechanisms
are deployment-specific and are not specified by this document.
Li, et al. Expires 27 February 2027 [Page 20]
Internet-Draft IDN August 2026
5.4. Trust Schema and Object Authentication
Trust Schema in NDN provides a useful design principle for IDN object
authentication: name patterns can specify which keys are authorized
to sign which classes of objects [TrustSchema]. IDN can use an
analogous schema to define which provider component, capability
version, runtime, or site agent may create, sign, update, or consume
each class of IDN object.
Stable object names provide a common handle for integrity, access
control, encryption metadata, provenance, and audit independent of
the node that currently stores the object. Objects containing
private context or tenant-specific state should be encrypted
according to the applicable policy. Signatures and authenticated
metadata can allow caches, routers, and auditors to verify integrity
and policy eligibility without trusting the storage location itself.
5.5. Node Trust and Execution Evidence
A compute node that advertises resources should expose verifiable
identity and policy-relevant properties before serving protected
requests or hosting sensitive capabilities. Depending on deployment
requirements, such evidence may include node identity, software
version, runtime configuration, isolation properties, trusted
execution evidence, or other attestation results.
Execution receipts can record capability versions, node attestations,
object signatures, delegation evidence, schema-based policy verdicts,
cache usage, and routing or deployment decisions. These receipts
support debugging, accounting, compliance checks, and incident
response. They do not by themselves prove semantic correctness of
inference results.
5.6. Reuse, Isolation, and Information Leakage
State-aware caching and reuse can improve scalability and latency,
but reuse across users, tenants, sessions, or hardware environments
can introduce information leakage. IDN deployments need to enforce
sharing scope, access policy, encryption policy, eviction policy, and
compatibility constraints before reusing cached objects.
Public or tenant-shared objects may be reusable across multiple
requests. Session-private or user-private objects require stronger
isolation. Hardware-bound objects may be unusable after migration.
Result objects may only be safe to reuse when model version, decoding
configuration, prompt normalization, policy labels, and input
equivalence rules are compatible.
Li, et al. Expires 27 February 2027 [Page 21]
Internet-Draft IDN August 2026
5.7. Availability and Abuse Considerations
Popular capabilities and cached state can become high-value targets
for denial-of-service or abuse. A request-routing policy that
overemphasizes locality or state affinity may create hotspots. IDN
deployments therefore need resilience mechanisms such as load-aware
routing, admission control, capability replication, fallback to
compatible variants, early rejection under overload, failure
detection, and rollback of unsafe components.
Security events can also require rapid invalidation of model
artifacts, cached state, or capability descriptors. The architecture
should support object invalidation and service rerouting without
requiring global redeployment.
6. IANA Considerations
This document makes no requests of IANA.
7. Acknowledgments
The authors would like to thank colleagues and reviewers in the
community who provided feedback on earlier version of this draft.
8. Informative References
[Akamai] Nygren, E., Sitaraman, R. K., and J. Sun, "The Akamai
Network: A Platform for High-Performance Internet
Applications", ACM SIGOPS Operating Systems Review, 2010.
[CacheGen] Liu, Y., Li, H., Cheng, Y., Ray, S., Huang, Y., Zhang, Q.,
Du, K., Yao, J., Lu, S., and G. Ananthanarayanan,
"CacheGen: KV Cache Compression and Streaming for Fast
Large Language Model Serving", Proceedings of the ACM
SIGCOMM 2024 Conference, August 2024.
[CATSFramework]
Li, C., Du, Z., Boucadair, M., Contreras, L. M., and J.
Drake, "A Framework for Computing-Aware Traffic Steering
(CATS)", IETF Internet-Draft, work in progress, 2024,
.
Li, et al. Expires 27 February 2027 [Page 22]
Internet-Draft IDN August 2026
[CATSUseCases]
Yao, K., Trossen, D., Boucadair, M., Contreras, L. M.,
Shi, H., Li, Y., Zhang, S., and Q. An, "Computing-Aware
Traffic Steering (CATS) Problem Statement, Use Cases, and
Requirements", IETF Internet-Draft, work in progress,
2024, .
[DilleyCDN]
Dilley, J., Maggs, B., Parikh, J., Prokop, H., Sitaraman,
R., and B. Weihl, "Globally Distributed Content Delivery",
IEEE Internet Computing, 2002.
[Elf] Zhang, W., He, Z., Liu, L., Jia, Z., Liu, Y., Gruteser,
M., Raychaudhuri, D., and Y. Zhang, "Elf: accelerate high-
resolution mobile deep vision with content-aware parallel
offloading", Proceedings of the 27th Annual International
Conference on Mobile Computing and Networking,
DOI 10.1145/3447993.3448628, September 2021,
.
[KVShare] Yang, H., Zhang, R., Huang, M., Wang, W., Tang, Y., Li,
Y., Liu, Y., and D. Zhang, "KVShare: An LLM Service System
with Efficient and Effective Multi-Tenant KV Cache Reuse",
arXiv preprint arXiv:2503.16525, May 2025,
.
[NDN] Zhang, L., Afanasyev, A., Burke, J., Jacobson, V., Claffy,
K. C., Crowley, P., Papadopoulos, C., Wang, L., and B.
Zhang, "Named Data Networking", ACM SIGCOMM Computer
Communication Review, 2014.
[PagedAttention]
Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C.
H., Gonzalez, J., Zhang, H., and I. Stoica, "Efficient
Memory Management for Large Language Model Serving with
PagedAttention", Proceedings of the 29th Symposium on
Operating Systems Principles, October 2023.
[PromptCache]
Gim, I., Chen, G., Lee, S., Sarda, N., Khandelwal, A., and
L. Zhong, "Prompt Cache: Modular Attention Reuse for Low-
Latency Inference", Proceedings of Machine Learning and
Systems, May 2024.
Li, et al. Expires 27 February 2027 [Page 23]
Internet-Draft IDN August 2026
[RFC3568] Barbir, A., Cain, B., Nair, R., and O. Spatscheck, "Known
Content Network (CN) Request-Routing Mechanisms",
RFC 3568, DOI 10.17487/RFC3568, July 2003,
.
[RFC9111] Fielding, R., Ed., Nottingham, M., Ed., and J. Reschke,
Ed., "HTTP Caching", STD 98, RFC 9111,
DOI 10.17487/RFC9111, June 2022,
.
[RFC9556] Hong, J., Hong, Y., de Foy, X., Kovatsch, M., Schooler,
E., and D. Kutscher, "Internet of Things (IoT) Edge
Challenges and Functions", RFC 9556, DOI 10.17487/RFC9556,
April 2024, .
[SDSI] Rivest, R. and B. Lampson, "SDSI - A Simple Distributed
Security Infrastructure", CRYPTO 1996, 1996.
[TrustSchema]
Yu, Y., Afanasyev, A., Clark, D., Claffy, K. C., Jacobson,
V., and L. Zhang, "Schematizing Trust in Named Data
Networking", Proceedings of the 2nd ACM Conference on
Information-Centric Networking, September 2015.
Authors' Addresses
Qing Li
Pengcheng Laboratory
Email: liq@pcl.ac.cn
Hanling Wang
Pengcheng Laboratory
Email: wanghl03@pcl.ac.cn
Yong Jiang
Tsinghua Shenzhen International Graduate School & Pengcheng Laboratory
Email: jiangy@sz.tsinghua.edu.cn
Mingwei Xu
Tsinghua University
Email: xumw@tsinghua.edu.cn
Gabriel-Miro Muntean
Dublin City University
Li, et al. Expires 27 February 2027 [Page 24]
Internet-Draft IDN August 2026
Email: gabriel.muntean@dcu.ie
Li, et al. Expires 27 February 2027 [Page 25]