]> CMCC

No. 58 Kunlunshan Road Suzhou 215000 China lluozy@yeah.net

CMCC

No. 58 Kunlunshan Road Suzhou 215000 China yanhaishuang_yewu@cmss.chinamobile.com

Applications Network Working Group Internet-Draft This document describes an experimental protocol named the Available Session Recovery Protocol (ASRP). The protocol is designed to optimize high-availability network cluster architectures, providing a superior high-availability solution for clusters offering stateful network services such as load balancing and Network Address Translation (NAT ). ASRP defines the procedures for session backup and recovery, as well as the message formats used during these interactions, enabling efficient and streamlined session state management. In contrast to traditional high-availability techniques that back up session state within the cluster itself, the core innovation of ASRP lies in its distributed backup of state information to the client or server side. This approach offers multiple advantages: theoretically unlimited elastic scaling capacity; support for rapid recovery from multi-point failures; reduction of resource redundancy through the elimination of centralized backup nodes; and significant simplification of cluster implementation complexity. The ASRP protocol provides a standardized method for constructing elastic service clusters, facilitating broader participation from software and hardware developers in building elastic cloud network service clusters.

Introduction Traditional high-availability network clusters based on a master-backup architecture rely on session state synchronization between the master and backup nodes. While functionally complete, this architecture faces challenges in the cloud era, such as insufficient flexibility for elastic scaling, resource redundancy, and high implementation complexity. To address these challenges, the industry has moved to the Elastic Stateful Cluster. An Elastic Stateful Cluster is a high-availability network service cluster composed of multiple active nodes whose number can be elastically scaled, providing stateful network services such as load balancing (SLB) and Network Address Translation (NAT). In such a cluster the hard problem is session state: a design has to decide where the state lives, and how a node that receives a packet of a session it does not hold recovers that session.

Comparison with Existing Approaches High-availability mechanisms evolved together with the deployment model they serve. Master/backup HA (e.g., VRRP ) solved the failover problem of the master/backup era: it provides fast takeover of a shared address between two nodes, with session continuity typically left to private synchronization between the pair. The model fits a two-node deployment. In a multi-node cluster where every node forwards traffic simultaneously, there is no natural mapping onto a master/backup pair, and the failover unit no longer corresponds to the service unit. The industry answer is the Elastic Stateful Cluster itself: all nodes are active, and session state is shared through intra-cluster synchronization (e.g., LVS connection synchronization daemons, DPVS), typically as full-mesh or master/replica replication. Conventional Elastic Stateful Clusters realize this with a Fast/Slow Path design that separates session management from packet forwarding:

| Fast Path Node | <--------> | Server | | | | | +----------------+ | | | | +----------+ | | ... | | | | | ... | | | +----------------------+ | +--------------------------+ ]]>

The slow path nodes are responsible for session creation and synchronization, while the fast path nodes are responsible for rapid packet forwarding. The drawback is the weak elastic scaling capability of the slow path nodes: new sessions must be propagated to other nodes so that asymmetric traffic can be handled anywhere, the synchronization traffic grows with the session creation rate and the node count, and a newly added node starts without state. A typical implementation reference is the AWS Hyperplane NFV platform. Stateless scheduling (e.g., Maglev-style consistent hashing, ECMP ) avoids session state on the nodes entirely by hashing each flow to a node. Node failure is handled by rehashing flows, which works only when the service can tolerate a flow landing on a node that holds no state for it; stateful services such as NAT cannot. These mechanisms are what make elastic clusters work in practice, but they are private. The contrast with the previous generation is telling: the master/backup architecture is served by a standard (VRRP), while the Elastic Stateful Cluster - the architecture that large deployments actually use today - has no standard protocol for its session-state layer. Routing, failure detection, and address takeover are individually standardized; session synchronization and recovery are not, and every production cluster ships its own. ASRP fills this gap by removing the session state from the cluster altogether: the backup of a session is stored at one of the session's own endpoints, nodes share no protocol state with each other, and recovery state is pulled on demand (QS/RS) only when a node actually needs a session it does not have. This addresses the hardest problem of the elastic cluster and, as a side effect, simplifies the cluster itself: no inter-node protocol, no synchronization traffic, and no state warm-up for a newly added node. The tradeoff is that the client or server side must support the protocol. Within a single administrative domain - a data center for load balancing, an enterprise edge for SNAT - this is a deployment decision rather than a protocol limitation. summarizes the comparison. Architecture Elasticity Implementation Standard protocol Master/backup HA No (backup idles) Private pairwise sync VRRP Elastic Stateful Cluster (conventional) Fast path: easy; slow path: hard Complex (intra-cluster synchronization) None (private implementations) Elastic Stateful Cluster (ASRP) Yes (all nodes uniform) Simple (state kept outside the cluster) This document

ASRP Elastic Stateful Cluster

| ASRP Node | <--------> | Server | | | | +----------------+ | | | +----------+ | ... | +----------+ | ... | +----------------------+ ]]>

The Available Session Recovery Protocol (ASRP) proposes an innovative high-availability solution, aiming to provide a standardized method for constructing elastic service clusters. This facilitates broader participation from software and hardware developers in building elastic cloud network service clusters. Its core idea is to innovatively distribute session state information to the client or server. The lifecycle of the backup state is synchronized with the real session, eliminating the need for independent keepalive and timeout mechanisms. This design ensures the timeliness and availability of the backup information. ASRP defines corresponding session backup and recovery mechanisms. The protocol allows protocol messages to be transmitted together with the original service data packets, thereby reducing control overhead for state synchronization. In an elastic stateful cluster built on ASRP, network nodes possess atomic and mutually independent properties. There is no need for communication between nodes, nor is session synchronization required within the cluster. This fundamental design provides theoretically unlimited scaling capability and supports rapid recovery from multi-point failures.

Terminology The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here. This document uses the following terms: Network Node: A device that provides stateful network services (e.g., load balancing, NAT) and creates and maintains sessions for the traffic it forwards. Client / Server: The two endpoints of a session. Depending on the operational mode, the server (PSV mode) or the client (ACT mode) also holds the backup of the session. Backup Holder: The client or server that stores the backed-up state of a session. Session: The state a network node maintains for a flow, keyed by the flow's 5-tuple. Explicit First Packet: A packet whose characteristics explicitly indicate the start of a session, e.g., a TCP SYN or a DNS query. Inline Message: An ASRP message carried within a packet of the flow itself, preceded by the ASRP Signature. Standalone Message: An ASRP message carried in an independent UDP packet whose destination port is ASRP-PORT. ASRP-PORT: The UDP destination port used for standalone ASRP messages.

Protocol Overview

Two Operational Modes For the ASRP protocol to function correctly, two prerequisites must be met. First, all network nodes within the cluster MUST run service software supporting the ASRP protocol. Second, the server or client responsible for backing up sessions MUST deploy a kernel module or an eBPF module that supports ASRP. Depending on whether this module is deployed on the server or the client, the protocol operates in one of two corresponding modes: Passive (PSV) Mode and Active (ACT) Mode.

PSV Mode In PSV mode, the network node is typically deployed in the same network domain as the server (e.g., inside a data center). Its typical service is load balancing.

ACT Mode In ACT mode, the network node is typically deployed in the same network domain as the client (e.g., an enterprise intranet). Its typical service is Source Network Address Translation (SNAT). The operational mode (PSV or ACT) is determined by the deployment configuration of the network node and is not carried in ASRP messages.

Two Routing Behaviors

Symmetric Routing

| node X | <----------> | Server | | | | +------------+ | | | +----------+ | ... | +----------+ +------------------+ ]]>

Symmetric routing refers to the path mode where bidirectional traffic of the same session between a client and a server is always routed to the same node within the cluster.

Asymmetric Routing

| node X | -----------> | | | | | +------------+ | | | | Client | | ... | | Server | | | | +------------+ | | | | | <----------- | node Y | <----------- | | +----------+ | +------------+ | +----------+ | ... | +------------------+ ]]>

Asymmetric routing refers to the scenario where bidirectional traffic of the same session may be routed (e.g., by mechanisms such as ECMP , ) to different nodes within a cluster. In cloud networking environments, asymmetric routing is a common phenomenon, which imposes higher demands on the implementation of elastic stateful clusters.

Protocol Message ASRP achieves distributed backup and recovery of session state information by exchanging specific protocol messages among the client, server, and network nodes (such as load balancers or NAT devices). In a load-balancing scenario, session state is distributed and backed up to individual servers; in an SNAT scenario, session state is distributed and backed up to individual clients. ASRP defines the following protocol messages: New Session message (NS), New Session Acknowledge message (NA), New Session Conflict message (NC), Query Session message (QS), Recover Session message (RS), Recovery no-session message (RX), Hello Session message (HS) and Push Session message (PS).

NS Message Generated by the network node, it is used to send session state information to a designated client (in ACT mode) or server (in PSV mode) for backup when creating a new session.

NA Message Generated by the server as a response to an NS message in PSV mode, acknowledging the receipt of the NS message.

NC Message Generated by the client or server when the session state carried in a received NS message conflicts with an existing local session, indicating the session conflict to the network node.

QS Message Generated by the network node, it is used to query the client or server for backup session state information when a received packet cannot match any local session and a session cannot be directly created.

RS Message Generated by the client or server holding the backup as a response to a QS message, it contains the state information required to recover the session. The network node parses the RS message and reconstructs or marks the local session, thereby achieving failure recovery.

RX Message Generated by the client or server holding the backup as a response to a QS message, indicating that the session queried by the QS message was not found. Except for the MsgType field, the RX message is identical to the corresponding QS message.

HS Message Generated by the client, it is used in ACT mode to announce to the network node its capability to support the ASRP protocol and to trigger the network node to return an NS message to complete session backup.

PS Message Generated by the server, it is used in PSV mode to push session state information to the network node. In the case of asymmetric routing, the network node utilizes the PS message to create/update sessions for fast packet forwarding.

Transmission Modes and Signature ASRP messages can be transmitted in two modes: Inline Message and Standalone Message.

Inline Message The Inline Message is transmitted with the same 5-tuples as the flow, either inserted into the original packet or carried in a separate packet that copies the original packet header. When the total packet length does not exceed the MTU, NS, HS, and PS messages SHOULD be inserted into the original packet; otherwise, they are carried in a separate packet. HS messages are sent after forwarded packets, and PS messages are sent before forwarded packets. Traveling with the flow's 5-tuple is a structural requirement for HS and PS: both are sent from an endpoint towards the cluster, and under asymmetric routing the sender cannot predict which network node handles the corresponding direction of the flow. Since per-flow scheduling (e.g., ECMP hashing) maps one 5-tuple to one node, a message carried with the flow's 5-tuple - whether inserted into a packet of the flow or in a separate packet copying its header - is delivered to the same node as the flow itself, which is the node that holds (for HS) or needs (for PS) the session. NS, by contrast, is sent from a network node to a known endpoint (the selected server in PSV mode, or the client in ACT mode) and can also be transmitted as a Standalone Message; carrying it inline is a performance optimization that piggybacks on a packet about to be forwarded. Inserting a message into the first packet of a session makes that packet carry data (e.g., a TCP SYN with a non-empty payload). Some middleboxes drop such packets; this behavior has been observed in the deployment of TCP Fast Open . Where such interference is known or detected on a path, the implementation SHOULD use the separate-packet form instead of inserting into the original packet, since the separate packet carries the message without altering the characteristics of the session's own packets.

Standalone Message The Standalone Message is transmitted in an independent packet with its own 5-tuples. It is encapsulated using IP/UDP , with IP addresses configured to ensure mutual reachability and a fixed destination port ASRP-PORT (e.g., 51200). QS, RS, RX, NA and NC messages typically use this transmission mode.

ASRP Signature In Inline Message mode, an ASRP Signature is used to indicate that a packet contains an ASRP message. Inspired by the Proxy Protocol, an 8-byte ASRP Signature is used: 0x41, 0x53, 0x52, 0x50, 0x0D, 0x0A, 0x00, 0xF1. The signature consists of the ASCII string "ASRP" followed by 0x0D, 0x0A, 0x00, 0xF1. The "ASRP" prefix keeps the marker readable and distinct from other in-stream protocols; the trailing bytes reduce the chance of a false match against application data that happens to begin with the ASCII string. The ASRP Signature is a fixed marker for message identification; it is not a cryptographic signature. It is inserted in front of the ASRP message. In Standalone Message mode, the UDP destination port is ASRP-PORT, and the ASRP Signature is not required.

Session Creation/Recovery Scenarios This section elaborates on, through a series of typical scenarios, how the ASRP protocol achieves session backup and recovery via message interaction in the event of network node failures under different operational modes. Each scenario details the involved protocol message flows and the processing steps of each entity.

PSV-Scenario-1

| | -----2:NS----> | | | | | | <----3:NC----- | | | client | | ASRP Nodes | | server | | | | | | | | | <--5:PKT--- | | <----4:PS----- | | +----------+ +--------------+ +----------+ ]]>

This scenario describes that, in PSV mode, a network node receives an explicit first packet (i.e., a packet whose characteristics explicitly indicate the start of a session) and directly creates a session flow. Common examples of explicit first packets include TCP SYN and DNS query, among others. The processing flow is as follows: Session Creation: Upon receiving a packet from the client (e.g., TCP SYN), the network node first creates a new session and then sends an NS message to the selected server. Server Response: Upon receiving the NS message, the server stores the session state information and associates it with its local session. If a conflict occurs, it sends an NC message. In the case of asymmetric routing, when sending its first response packet, the server sends a PS message to the network node. Session Recovery: Upon receiving the PS message, the network node restores the session and forwards packets accordingly. The session state information backed up by the server is released upon local session termination, without requiring any additional teardown message.

PSV-Scenario-2

| server | | | | | | | | | | | <---3:RS/RX--- | | +----------+ +--------------+ +----------+ ]]>

This scenario describes the session recovery flow triggered by a server packet. The processing flow is as follows: Session Query: Upon receiving a packet from the server, the network node searches its local session table. If no matching session is found, the node SHOULD first buffer the packet for forwarding, then sends a QS message back to the server. Server Response: After receiving the QS message, the server looks up the backed-up session state information and sends an RS/RX message. Session Recovery: Upon receiving an RS message, the network node creates a new local session and forwards packets accordingly. Upon receiving an RX message, the node SHOULD discard the pending packets.

PSV-Scenario-3

| ... | | | ---1:PKT--> | | <---3:RS/RX--- | +--------+ | | | | | ... | | server | | | | | | ... | +--------+ | | client | | ASRP Nodes | ---4:PKT/NS--> | ... | | | | | | +--------+ | | | | | | | server | | | | <--7:PKT--- | | <--5:NA/NC---- | +--------+ | | | | | <----6:PS----- | ... | +----------+ +------------+ +------------+ ]]>

This scenario describes the session creation/recovery flow triggered by a client packet. The processing flow is as follows: Query Local Session: Upon receiving a packet from the client, if no matching session is found, the network node first obtains a list of candidate servers (possibly multiple) for querying. Query Backup Session: The network node sends QS messages to each candidate server. Each server replies with an RS or RX message. Process Query Results: If a session is found, the network node restores the session and forwards the packet. Otherwise, for TCP packets: drop the packet. For UDP packets: create a new local session and send an NS message to the selected server. Server Creates New Session: Upon receiving an NS message, the server stores the session state information and associates it with its local session. If a conflict occurs, it sends an NC message; otherwise, if the F-NA flag is set, it immediately replies with an NA packet. Session Recovery: In an asymmetric routing environment, when sending its first response packet, the server sends a PS message to restore the session at the network node. In this scenario, obtaining the list of candidate servers is a key challenge. Two solutions are proposed: The network node derives an ordered list of candidate servers from the packet's 5-tuple using a deterministic server-selection algorithm, such as consistent hashing or history-aware consistent hashing, and sends QS messages to the candidates in order. Enhance the client so that client packets carry backend server information, allowing the network node to directly extract the target server from the client packet. One such enhancement is specified in , which carries the selected backend identity in a TCP option. For the first solution, the specific algorithm is a local implementation choice; what the protocol relies on are three properties of it: Determinism: The mapping from a session's 5-tuple to its candidate list MUST be identical on every node of the cluster, so that any node receiving a packet of a session it does not hold computes the same candidates to query. History awareness: When the backend pool changes (e.g., a server is added), the algorithm SHOULD keep the servers that previously served the affected hash region in the candidate list, so that sessions created before the change can still locate their backup holder. A removed server simply fails to answer; the recovery attempt moves on to the next candidate or is abandoned. Bounded querying: The number of candidates queried for one recovery MAY be limited to a small constant, bounding the QS traffic and the recovery latency. When the limit is reached without an RS response, the node stops querying and falls back to its local handling of packets without a session ().

ACT-Scenario-1

| | | | | | <----2:NS----- | | ---3:PKT--> | | | client | | ASRP Nodes | | server | | | <----5:QS----- | | <--4:PKT--- | | | | ---6:RS/RX---> | | | | +----------+ +--------------+ +----------+ ]]>

This scenario describes session creation at a network node and server-initiated session restoration. The processing flow is as follows: Session Backup: When sending a packet without receiving an NS message, the client sends an HS message to the network node to request session backup (the client MAY rate-limit the HS messages it generates). Upon receiving the HS message, the node replies with an NS message. Session Lookup: For packets from a server with no matching session, the network node identifies the target client and sends it a QS message. Session Recovery: Upon receiving QS, the client sends an RS/RX message to the network node to restore the session. In step 2, identifying the target client is challenging. Two solutions are proposed: Use static mapping (e.g., map destination port to client). For SNAT, client IP addresses can be statically mapped to distinct port ranges. Enhance the server to embed client information in its packets, allowing the network node to extract the client address directly.

ACT-Scenario-2

| | | | | | | | | | | client | <----2:QS----- | ASRP Nodes | ---4:PKT--> | server | | | | | | | | | ---3:RS/RX---> | | | | +----------+ +--------------+ +----------+ ]]>

This scenario describes the client-packet-triggered session recovery. The processing flow is as follows: Session Query: Upon receiving a packet from a client with no local session and without an HS message, the network node sends a QS message to the client. Session Recovery: The client responds with an RS/RX message; the RS message enables session restoration.

State Consistency ASRP does not need a distributed consistency protocol, because the session state has a single writer at any time and the backup is a read-only copy. The backup of a session is created by the NS message at session creation and is never modified afterwards; it is released when the local session terminates. It therefore reflects the session state at creation time. For the state that ASRP targets - the flow-defining state of load balancing and NAT sessions, keyed by the session's 5-tuple - this snapshot remains valid for the lifetime of the session, since the key does not change. Implementations that maintain additional mutable state (e.g., timeouts or counters) recover a usable approximation of it. Recovery is a local action of one network node: the node rebuilds the session from the backup delivered in an RS message and thereafter owns the session. If, under asymmetric routing, two nodes receive packets of the same session and each recovers it independently, both hold the session locally and each forwards the packets that reach it - which is exactly how an active-active cluster treats any session it holds. No node writes to another node's session state, so there is no split-brain condition to resolve. If recovery fails (an RX response, or no response before the recovery attempt is abandoned), the network node falls back to its local handling of packets without a session, e.g., discarding them or creating a new session.

Protocol Details

Message Format An ASRP message consists of ten fields in total. The message header has a fixed length and is composed of the first six fields. The message body comprises the remaining four fields: the fixed-length Protocol and Cookie fields, and the variable-length Session-Tuple and Session-Data fields. The fields are defined as follows:

Rules for validating these fields on receipt are specified in "Message Validation". ST4(length 12) Format:

ST6 sequence) ST64(length 48) Format: Mixed ST pair(ST6->ST4 sequence) ]]>

For messages carrying two Session-Tuples (NS, NA, NC, RS, and PS), the first Session-Tuple (ST1) MUST be the client-side tuple, and the second Session-Tuple (ST2) MUST be the server-side tuple. For messages carrying a single Session-Tuple (QS and RX), the Session-Tuple MUST be the 5-tuple of the triggering packet.

NS Message Format The NS message contains a Node-Local-IP field and two Session-Tuples. The Node-Local-IP field is located at the end of the NS message and carries the local IP address of the network node that sends the NS message. Its length is 16 octets when the F-LIP6 flag is set, and 4 octets otherwise. The client or server uses this address as the destination IP to send the NA message back to the network node. NS Message Format:

The two Session-Tuples represent the network node's connections with the client and the server, respectively.

NA Message Format The NA message acknowledges the receipt of an NS message that carries the F-NA flag. The NA message has the same structure as the NS message, except that the Node-Local-IP field is not present.

NC Message Format The structure of the message is the same as that of the NA message. In the NC message, the client-side Session-Tuple remains unchanged, while the server-side Session-Tuple may be modified to reflect the conflicting session. The F-CC flag indicates whether the conflict is on the client side or the server side.

QS Message Format QS Message Format:

RS Message Format RS Message Format: The structure of the message is the same as that of the NA message.

RX Message Format RX Message Format: The structure of the message is the same as that of the QS message.

HS Message Format HS Message Format:

PS Message Format PS Message Format: The structure of the message is the same as that of the NA message.

ASRP packet Format A packet that carries a single ASRP message is referred to as an ASRP packet. Based on the two transmission modes of ASRP messages, the format of ASRP packets can also be classified into two types.

Inline-ASRP packet In Inline Message mode, the ASRP packet shares the 5-tuples with the flow. The ASRP message is either inserted into the original packet:

or carried in a separate packet that copies the original packet header, without the original packet data:

Standalone-ASRP packet In Standalone Message mode, the ASRP packet uses its own 5-tuples and is encapsulated using IP/UDP. The packet format is as follows:

Transport-Layer Transparency An inline ASRP message exists only on the wire between the sending side's ASRP module (which inserts it) and the receiving side's ASRP module (which strips it). The message occupies leading octets of the segment payload, but it is never part of either endpoint's TCP sequence space: the sender's transport stack generated the segment without the message, and the receiver's transport stack processes the segment without it. In this respect an inline message travels like application data - the signature and the message simply occupy the head of the payload - yet it is handled entirely below the transport layer, and neither the application nor either transport stack observes it. Consequently, implementations MUST NOT modify TCP sequence or acknowledgment numbers when inserting or stripping an inline message. The only fields requiring adjustment are the IP total length and the transport checksums, which MUST be recomputed after insertion or removal. After stripping, the byte stream each endpoint's transport stack observes is identical to that of a session that never carried ASRP messages.

Message Processing Standalone-ASRP packets are identified by the UDP destination port, while Inline-ASRP packets are identified by the ASRP Signature. Once an ASRP packet is identified, the ASRP messages within it are then parsed and processed.

Message Validation Before processing any ASRP message, a receiver MUST perform the following checks and silently discard the message if any of them fails: The Version field is a version the receiver implements. The Length field matches the number of ASRP message octets actually present in the packet. The STType field matches the MsgType: QS and RX carry a single Session-Tuple (ST4 or ST6); NS, NA, NC, RS, and PS carry a Session-Tuple pair (ST44, ST66, ST46, or ST64); HS carries no Session-Tuple (NULL). The Reserved field and any unassigned flag bits are ignored. A message that passes these checks is processed according to its MsgType as described in the following subsections.

Cookie Handling Every ASRP message carries the 8-octet Cookie field defined in . The cookie associates a response with the request that triggered it. The sender of a request message (QS, NS carrying F-NA, or HS) SHOULD fill the Cookie field with a random value that is unique to this request. A responder (the sender of RS/RX, NA/NC, or the NS replying to HS) MUST copy the Cookie field of the request into the Cookie field of its response, unchanged. A node that receives a response SHOULD verify that the Cookie field matches the outstanding request it is applying the response to, and discard the response otherwise. Messages with no request/response pairing (e.g., PS) set the Cookie field to zero. An implementation that does not use cookie validation MAY set the Cookie field of its requests to zero and skip the verification above; see the Security Considerations for how this affects message validation. Stronger message authentication, e.g., based on a message authentication code, can be introduced as a future extension carried in the variable-length Session-Data field, without any change to the message format.

NS Message Processing When a client or server receives an NS message, it MUST first check for session conflicts on the client side and then on the server side. A client-side conflict occurs when the client-side tuple matches an existing backed-up session on either side. If a conflict is found, it MUST clear the F-NA flag and send an NC message, with the F-CC flag set to indicate a client-side conflict. If no conflict is found, it MUST back up the session state information and hand the embedded packet (if present) over to the system. If the F-NA flag is set, it MUST send an NA message. The F-NA flag SHOULD be set when the session is created from a non-explicit first packet; explicit first packets (e.g., TCP SYN or DNS query) do not require it. If an NS message is lost, in PSV mode a TCP SYN retransmission or subsequent packets regenerate NS messages until an NA message is received; in ACT mode, subsequent packets generate HS messages, prompting the network node to retransmit the NS message. NS messages may be generated in both PSV and ACT modes. The handling procedures are described in , , and .

NA Message Processing The destination IP of the NA packet MUST be set to the Node-Local-IP field carried in the received NS message, and the destination port MUST be set to ASRP-PORT. Upon receiving an NA message, the network node considers that the corresponding NS message has been processed by the responder, and MUST NOT send any further NS messages for this session.

NC Message Processing The destination IP of the NC packet MUST be set to the Node-Local-IP field carried in the received NS message, and the destination port MUST be set to ASRP-PORT. Upon receiving an NC message, the network node considers that the session conflicts with an existing local session on the responder. If the F-CC flag is set, the conflict is on the client side; otherwise, it is on the server side. The network node SHOULD handle the conflict according to its local policy.

QS Message Processing The source IP of the QS packet MUST be set to the network node's local IP (obtainable from configuration), and the destination IP MUST be set to the client's or server's IP (obtainable from the forwarded packet as described in and , or derived via algorithmic mapping to the client or server as described in and ). The source port MUST be randomly generated, and the destination port MUST be set to ASRP-PORT. The backup holder MUST index the backed-up session by both the client-side and the server-side 5-tuples. When a client or server receives a QS packet, it MUST look up the backed-up session whose client-side or server-side 5-tuple matches the Session-Tuple in the QS message. If a match is found, it MUST return an RS message; otherwise, it MUST return an RX message. If a QS message is lost, subsequent packets will trigger the generation of new QS packets, continuing the attempt to recover the session. QS messages may be generated in both PSV and ACT modes. The handling procedures are described in , , and .

RS Message Processing The RS packet MUST reuse the protocol header of the QS packet, with the source and destination IP addresses swapped. When a network node receives an RS packet, it MUST recover the session and forward packets according to the session. If an RS message is lost, subsequent QS messages will continue the attempt to recover the session, thereby triggering retransmission of the RS message. RS messages may be generated in both PSV and ACT modes. The handling procedures are described in , , , , and .

RX Message Processing Upon receiving an RX message, the network node MUST follow the processing procedures described in the respective scenarios under PSV/ACT mode.

HS Message Processing The network node identifies the session by the 5-tuple of the packet carrying the HS message. If a local session is found, it MUST return an NS message for that session to the client, addressed to the source IP of that packet; otherwise, no NS message is returned. HS messages are only generated in ACT mode. The handling procedure is described in .

PS Message Processing The source IP, destination IP, and source port of the PS packet MUST be copied from the packet sent by the server. When a network node receives a PS message, it MUST recover the session; it extracts the forwarded packet (if present) and forwards it according to the session. PS messages are only generated in PSV mode. The handling procedure is described in .

Security Considerations ASRP messages may be subject to forgery, replay, or flooding by attackers who are able to inject traffic into the network that carries them. ASRP deployments can counter this with two layers of message validation; both layers draw on information local to the validating node, so no protocol-level trust relationship between nodes is required. Cookie-based validation: Every ASRP message carries an 8-octet Cookie field. A node that sends a request (a QS message, an NS message carrying F-NA, or an HS message) fills the cookie with a per-request random value and verifies that the corresponding response (RS/RX, NA/NC, or NS, respectively) echoes it. This verifies that the response belongs to an outstanding request of the receiving node; a forged response that guesses the cookie, or a replayed response from an earlier request, can be recognized and discarded. Cookie generation and verification rules are determined by the implementation and deployment; see "Cookie Handling". Session-legitimacy validation: Nodes SHOULD validate the session referenced by a message against application-level policies, such as verifying that the session points to a known and healthy server in a load-balancing deployment, or that the address translation complies with configured rules in a NAT deployment. Nodes that perform this validation MUST discard messages that fail it and MUST NOT create a new session for them. Implementations MAY apply rate limiting to ASRP messages as appropriate, e.g., to contain QS floods . ASRP responses are comparable in size to the requests that trigger them, so the protocol itself provides no amplification.

Privacy Considerations ASRP messages carry the 5-tuple of the session they describe. This is the same information that is visible in the session's own packets; observing ASRP traffic reveals the existence of a session between two endpoints and their addresses, but nothing beyond what the carried traffic already exposes. The Node-Local-IP field of an NS message discloses the address of the network node that created the session to the backup holder. This is information the backup holder requires in order to return NA/NC messages, and the network node address is in any case observable in the forwarded traffic of the session. ASRP introduces no new identifiers that persist across sessions and no tracking mechanism. Operators who wish to limit the visibility of session metadata can apply the usual operational measures to ASRP traffic, e.g., restricting routing of the ASRP-PORT destination to the serving domain.

IANA Considerations This document defines an application-layer protocol (ASRP). All values of the MsgType, STType, and Flags fields used by this version of the protocol are defined in this specification. New values MUST be defined via an update to this document or a successor specification; a future standards-track version of ASRP SHOULD request the creation of IANA registries for these fields. For the implementation of this protocol, a UDP destination port requires allocation:

UDP Destination Port QS/RS/RX/NA/NC messages are encapsulated within UDP datagrams for transmission. A fixed UDP destination port number is required so that the receiving end can identify and process such encapsulated packets. Service Name: asrp Port Number: 51200 (proposed value for current experimentation) Transport Protocol: udp Description: Used for receiving UDP-encapsulated ASRP protocol messages. For experimental implementations and interoperability testing prior to IANA assignment, UDP port 51200 MAY be used as a temporary default. This port falls within the dynamic/private port range (49152-65535) reserved for local or temporary use and documentation examples . IANA is requested to assign a permanent port number in the "User Ports" range (1024-49151) for the "asrp" service in the "Service Name and Transport Protocol Port Number Registry", with a reference to this document.

In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. RFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings. This RFC is the revised basic definition of The Domain Name System. It obsoletes RFC-882. This memo describes the domain style names and their used for host address look up and electronic mail forwarding. It discusses the clients and servers in the domain name system and the protocol used between them. This RFC is the revised specification of the protocol and format used in the implementation of the Domain Name System. It obsoletes RFC-883. This memo documents the details of the domain name client - server communication. The effect of multipath routing on a forwarder is that the forwarder potentially has several next-hops for any given destination and must use some method to choose which next-hop should be used for a given data packet. This memo summarizes current practices, problems, and solutions. This memo provides information for the Internet community. Equal-cost multi-path (ECMP) is a routing technique for routing packets along multiple paths of equal cost. The forwarding engine identifies paths by next-hop. When forwarding a packet the router must decide which next-hop (path) to use. This document gives an analysis of one method for making that decision. The analysis includes the performance of the algorithm and the disruption caused by changes to the set of next-hops. This memo provides information for the Internet community. This document defines basic terminology for describing different types of Network Address Translation (NAT) behavior when handling Unicast UDP and also defines a set of requirements that would allow many applications, such as multimedia communications or online gaming, to work consistently. Developing NATs that meet this set of requirements will greatly increase the likelihood that these applications will function properly. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. This document describes TCP SYN flooding attacks, which have been well-known to the community for several years. Various countermeasures against these attacks, and the trade-offs of each, are described. This document archives explanations of the attack and common defense techniques for the benefit of TCP implementers and administrators of TCP servers or networks, but does not make any standards-level recommendations. This memo provides information for the Internet community. This memo defines the Virtual Router Redundancy Protocol (VRRP) for IPv4 and IPv6. It is version three (3) of the protocol, and it is based on VRRP (version 2) for IPv4 that is defined in RFC 3768 and in "Virtual Router Redundancy Protocol for IPv6". VRRP specifies an election protocol that dynamically assigns responsibility for a virtual router to one of the VRRP routers on a LAN. The VRRP router controlling the IPv4 or IPv6 address(es) associated with a virtual router is called the Master, and it forwards packets sent to these IPv4 or IPv6 addresses. VRRP Master routers are configured with virtual IPv4 or IPv6 addresses, and VRRP Backup routers infer the address family of the virtual addresses being carried based on the transport protocol. Within a VRRP router, the virtual routers in each of the IPv4 and IPv6 address families are a domain unto themselves and do not overlap. The election process provides dynamic failover in the forwarding responsibility should the Master become unavailable. For IPv4, the advantage gained from using VRRP is a higher-availability default path without requiring configuration of dynamic routing or router discovery protocols on every end-host. For IPv6, the advantage gained from using VRRP for IPv6 is a quicker switchover to Backup routers than can be obtained with standard IPv6 Neighbor Discovery mechanisms. [STANDARDS-TRACK] This document defines the procedures that the Internet Assigned Numbers Authority (IANA) uses when handling assignment and other requests related to the Service Name and Transport Protocol Port Number registry. It also discusses the rationale and principles behind these procedures and how they facilitate the long-term sustainability of the registry. This document updates IANA's procedures by obsoleting the previous UDP and TCP port assignment procedures defined in Sections 8 and 9.1 of the IANA Allocation Guidelines, and it updates the IANA service name and port assignment procedures for UDP-Lite, the Datagram Congestion Control Protocol (DCCP), and the Stream Control Transmission Protocol (SCTP). It also updates the DNS SRV specification to clarify what a service name is and how it is registered. This memo documents an Internet Best Current Practice. This document describes an experimental TCP mechanism called TCP Fast Open (TFO). TFO allows data to be carried in the SYN and SYN-ACK packets and consumed by the receiving end during the initial connection handshake, and saves up to one full round-trip time (RTT) compared to the standard TCP, which requires a three-way handshake (3WHS) to complete before data can be exchanged. However, TFO deviates from the standard TCP semantics, since the data in the SYN could be replayed to an application in some rare circumstances. Applications should not use TFO unless they can tolerate this issue, as detailed in the Applicability section. This document specifies the Transmission Control Protocol (TCP). TCP is an important transport-layer protocol in the Internet protocol stack, and it has continuously evolved over decades of use and growth of the Internet. Over this time, a number of changes have been made to TCP as it was specified in RFC 793, though these have only been documented in a piecemeal fashion. This document collects and brings those changes together with the protocol specification from RFC 793. This document obsoletes RFC 793, as well as RFCs 879, 2873, 6093, 6429, 6528, and 6691 that updated parts of RFC 793. It updates RFCs 1011 and 1122, and it should be considered as a replacement for the portions of those documents dealing with TCP requirements. It also updates RFC 5961 by adding a small clarification in reset handling while in the SYN-RECEIVED state. The TCP header control bits from RFC 793 have also been updated based on RFC 3168. CMCC This document defines the Session Recovery Option (SRO) for TCP. SRO improves the reliability of Source Network Address Translation (SNAT) and load balancing (LB) services and simplifies the implementation of elastically scaling SNAT/LB clusters. SRO enables a client and a server to exchange their identifiers during connection establishment. When a session needs to be recovered, an endpoint conveys the identifier of the peer to the network node, which uses it to locate the endpoint holding the session backup and recover the session. SRO is optional: endpoints that do not support it behave as if the option did not exist.

Implementation Status Note to the RFC Editor: Please remove this section before publication. ASRP has been implemented twice, covering both roles the protocol defines, and the implementations have been interoperability tested on a three-node testbed consisting of a client, a load-balancing cluster node, and a backend server: A Linux kernel module extending IPVS, implementing the network-node role in PSV mode: session creation with inline NS insertion, QS/RS exchange for session recovery, NA/NC handling, cookie verification of responses, and the ASRP-PORT UDP listener. An eBPF implementation based on traffic-control programs, implementing the backup-holder role on the server: NS backup from session first packets, session reconstruction from PS messages, and response generation with cookie echoing. The testbed exercised direct session creation (PSV-Scenario-1), recovery after cluster session-table loss simulating node failure (PSV-Scenario-2), and the fallback when no backed-up session exists (RX). Recovery was observed to be driven by the retransmission behavior of the recovered session, as described in the scenarios.

Acknowledgments The authors would like to thank all individuals who have provided valuable feedback and contributions during the development of this document.