]> CMCC

No. 58 Kunlunshan Road Suzhou 215000 China lluozy@yeah.net

CMCC

No. 58 Kunlunshan Road Suzhou 215000 China yanhaishuang_yewu@cmss.chinamobile.com

Internet Network Working Group Internet-Draft This document defines the Session Recovery (SR) option for TCP. The option lets the endpoints of a connection exchange identifiers: during the handshake each endpoint carries its own identifier, and designated segments afterwards carry the identifier of the peer. This places the knowledge of which endpoint a connection belongs to inside the TCP header, where network functions on the path - load balancers, NAT gateways, firewalls - can read it without per-flow state and without payload inspection. The primary use is session recovery in SNAT and load-balancing clusters; further uses include reduced-state forwarding, connection-tracking recovery, and per-backend telemetry. The option is carried in the SYN, the SYN-ACK, and retransmitted segments, or in every segment when so configured; the default adds no overhead to normal data segments. Endpoints that do not support it behave as if the option did not exist.

Introduction SNAT and LB are stateful services: session state resides on the serving node, and a single node cannot recover its sessions after a failure. To provide reliable service, operators deploy high-availability (HA) pairs or clusters. Existing elastically scaling SNAT/LB cluster designs are complex to implement, and the most complex part is session recovery and session migration. Moreover, these services are unfriendly to long-lived connections: an idle connection is typically cleaned up by the serving node after a timeout , breaking the connection. Almost every TCP option standardized to date serves the two endpoints of a connection: MSS, Window Scale, SACK , and timestamps tune the transport between the two stacks. The network between the endpoints has evolved differently. Load balancers, NAT gateways, and firewalls are deployed at scale on virtually every path, and they maintain per-connection state that the endpoints are unaware of; yet no TCP option serves this middle tier, and its session management is implemented privately by every product. The option defined in this document fills this gap: endpoints voluntarily publish, inside the TCP header, the identifier of the endpoint a connection belongs to, so that on-path network functions can consume this information without per-flow state and without payload inspection. The information is provided by the endpoints and is read-only on path; the end-to-end semantics of TCP are preserved. A novel approach to reliable service is to back up the session state to the client or server side, so that even a single serving node can recover a session after a failure. This approach is embodied in the Available Session Recovery Protocol (ASRP) , to which readers may refer for the details of session backup and recovery. A key challenge of this approach is locating the endpoint that holds the session backup. Other mechanisms exist, such as multipath TCP on the endpoint side, but the Session Recovery (SR) option defined in this document addresses this challenge in the most direct way: it locates the backup endpoint in a single step. The SR option is a TCP option . During the TCP handshake, the client and the server exchange their identifiers; when a session needs to be recovered, an endpoint carries the identifier of the peer, and the network node uses it to locate the endpoint holding the session backup. The SR option is lightweight: by default it is carried only in the SYN, the SYN-ACK, and segments sent on timeout retransmission, adding no overhead to normal data segments; a per-segment mode, selected by the deployment, carries it in every segment where the identifier is needed on the forwarding path (see the Other Uses section). Despite this small cost, the option is important for the reliability of SNAT and LB services: it simplifies session recovery and session migration in elastically scaling SNAT/LB clusters, and makes them friendly to long-lived connections. With the SR option, a session that has been cleaned up after an idle timeout is recovered as soon as a segment carrying the identifier arrives. The SR option defines how identifiers are negotiated and carried, not what they are used for. Session recovery is the primary use; other uses, including stateless forwarding, connection-tracking recovery, and per-backend telemetry, are described in .

Terminology The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here. "SR option" refers to the Session Recovery option defined in this document. "Client" refers to TCP's active open side, and "server" refers to TCP's passive open side. "Network node" refers to an on-path device (e.g., a load balancer, a NAT gateway, or a firewall) that forwards packets; in the session-recovery use, it also participates in session backup and recovery. "Client identifier" refers to a value, carried in the SR option in a SYN segment, that identifies a client within a cluster. Its length is configurable and defaults to 4 octets. "Server identifier" refers to a value, carried in the SR option in a SYN-ACK segment, that identifies a server within a cluster. Its length is configurable and defaults to 4 octets. "Session backup" refers to session state information stored on an endpoint for the purpose of session recovery.

The SR Option The SR option is negotiated during the TCP handshake, similar to other TCP options, e.g., the Window Scale option , SACK , the Authentication Option (TCP-AO) , and TCP Fast Open . During the handshake, each endpoint carries its own identifier in the SR option: a client includes the SR option in the SYN segment, and a server that supports the SR option includes it in the SYN-ACK segment. The SR option is only in effect if both segments carry the option. After the handshake, an endpoint carries the identifier of the peer in the SR option, which it stored during the handshake: a client carries the server identifier, and a server carries the client identifier.

Option Formats The SR option carries an identifier. During the handshake, it carries the identifier of its sender: the client identifier in a SYN segment, and the server identifier in a SYN-ACK segment; on session recovery, it carries the identifier of the peer. The identifier length is configurable, from 2 to 8 octets, with a default of 4 octets. The option is 2 + N octets long: 1 octet for the Kind, 1 octet for the Length (Length = N + 2), and N octets for the identifier.

Option Semantics A client MAY include the SR option in a SYN segment; the client decides whether to use the SR option. A server that supports the SR option and receives a SYN segment with the SR option SHOULD include the SR option in the SYN-ACK segment. A server that does not support the SR option MUST NOT include the SR option in the SYN-ACK segment. A server MUST NOT include the SR option in a SYN-ACK segment if the corresponding SYN segment did not include the SR option; the option is in effect only when both endpoints have offered it. A client that receives a SYN-ACK segment without the SR option MUST NOT include the SR option in subsequent segments of that connection. The length of an identifier is configurable, from 2 to 8 octets, with a default of 4 octets; each endpoint determines the length of its own identifier. The Length field MUST equal the identifier length plus 2. A client that receives the SR option in a SYN-ACK segment SHOULD store the server identifier for the lifetime of the connection. A server that receives the SR option in a SYN segment SHOULD store the client identifier for the lifetime of the connection.

Identifier Management An identifier identifies an endpoint within a cluster. The option carries identifiers; it does not assign them. The following rules apply: Uniqueness: An identifier MUST be unique among the endpoints of the cluster it serves. Assignment: Identifiers are assigned by the deployment, e.g., by configuration or by the cluster's management system. How they are derived (randomly, by a keyed function of the address, or otherwise) is a deployment choice; see the Security Considerations. When the SR option is used with a mechanism that resolves identifiers to endpoint addresses, e.g., ASRP , the deployment MUST ensure that the network nodes can perform this resolution. Stability: An identifier MUST remain stable for the lifetime of the connections that carry it, and SHOULD remain stable across reboots. Session recovery relies on the identifier of the backup holder: an identifier that changes after a restart leaves the backups associated with the earlier identifier unreachable.

SR Option Operation

Connection Establishment During the TCP handshake, the client and the server exchange the SR option according to the rules in Section 3.2. After the handshake, each endpoint has stored the identifier of the peer and conveys it for session recovery as described in Section 4.2.

Session Recovery An endpoint conveys the identifier on timeout retransmission. In a load-balancing cluster, where the session state is backed up to the server, a client conveys the server identifier on timeout retransmission. The network node uses the identifier to locate the server holding the backup and recover the session.

| ----- SYN + Client-ID ------> | | <---- SYN-ACK + Server-ID --- | <--- SYN-ACK + Server-ID ---- | | | | | <---------- data -----------> | <---------- data -----------> | | | | | - - - (session lost) - - - x|x - - - (session lost) - - - | | | | | | | | retransmission | | | ------------ + -------------> | | | Server-ID | | | | | | | ------ recovery query ------> | | | <----- session state -------- | | | (session recovered) | | | | | <---------- data -----------> | <---------- data -----------> | ]]>

shows the recovery flow in a load-balancing cluster, where the client conveys the server identifier. The flow in a SNAT cluster is symmetric: the server conveys the client identifier, and the network node locates the client holding the backup. The detailed recovery procedures are described in the Available Session Recovery Protocol (ASRP) . In a SNAT cluster, where the session state is backed up to the client, a server conveys the client identifier on timeout retransmission. The network node uses the identifier to locate the client holding the backup and recover the session. How the network node recovers a session is out of the scope of this document. It is suggested that the node use the Available Session Recovery Protocol (ASRP) to recover the session; however, the SR option does not depend on ASRP. The endpoint includes the identifier of the peer stored during the handshake in the retransmitted segment.

Carrying Modes The identifier of the peer is conveyed in one of two modes. Which mode applies is a deployment choice: Retransmission mode: The identifier is carried only in segments sent on timeout retransmission. This is the protocol default, and it embodies the key design idea of the option: the overhead is synchronized with the need - the identifier appears exactly when a retransmission signals that the session may be in trouble, and normal data segments carry no overhead at all. Per-segment mode: The identifier is carried in every segment of the connection. This is the enabling mode for stateless forwarding (): a load balancer that forwards segments without any session table needs the identifier on every segment. It adds 2 + N octets to every segment. Deployments that consume the identifier on the forwarding path, e.g., stateless load balancing, SHOULD use the per-segment mode; for session recovery alone, the retransmission mode suffices.

Other Uses The SR option defines how identifiers are negotiated and carried, not what consumes them. Besides session recovery, uses include: Stateless forwarding: In deployments where the load balancer forwards only the client-to-server direction (e.g., DSR and one-way DNAT), the load balancer can read the backend identifier from every segment and forward accordingly, with no session table at all; this greatly simplifies its implementation. It requires the per-segment mode (), so that the identifier is present on every segment. In deployments that keep a session table for other processing, the table is reduced to a cache that accelerates the common case: on a miss, the segment is still forwarded correctly. Connection-tracking recovery: A firewall or NAT gateway that loses its connection table - after a restart, a table overflow, or a failover - normally drops established connections. With the SR option, a segment carrying the identifier lets the middlebox rebuild its entry, or locate the backup as in the recovery flow of Section 4.2, without any endpoint involvement. Per-backend telemetry: On-path measurement can attribute traffic to individual backends from the identifiers alone, without inspecting the application payload. Opaque affinity: A network element need not resolve an identifier to an endpoint at all: it can use the identifier as an opaque anchor, e.g., as a hash key that assigns flows to nodes, so that traffic carrying the same identifier follows a stable path without any per-flow state or mapping to maintain.

Compatibility Clients that do not support the SR option do not include it in the SYN segment, and servers that do not support it do not include it in the SYN-ACK segment. Such connections behave exactly as connections without the SR option, and session recovery falls back to the mechanisms implemented by the network node. Similarly, if the SR option is removed by an intermediate device, the endpoints operate without identifiers. The SR option is designed for the common deployment where a single tier of SNAT or LB nodes sits between the client and the server; in multi-tier deployments, only the tier closest to the client or the server can use the SR option.

Security Considerations The SR option only exchanges endpoint identifiers. It carries no session state and no authentication information; the SR option itself performs no validation and promises none. Connections that require integrity protection can use TCP-AO alongside the SR option; the two are independent and can coexist on the same connection. The security of mechanisms built on the identifiers relies on the validation performed by the framework that consumes them. When the SR option is used with ASRP , the message validation described there applies: a network node verifies that a recovery response matches its outstanding request (cookie-based validation) and that the session referenced by the message is legitimate (session-legitimacy validation). A forged identifier cannot recover a non-existent session, because recovery succeeds only if the session backup actually exists on the identified endpoint. A segment carrying an identifier triggers at most one recovery query, which is no more than the number of queries triggered by other recovery mechanisms. How a network node maps an identifier to an endpoint address is an implementation matter. A node SHOULD ignore an identifier that it cannot map to a known endpoint. An identifier SHOULD be randomly assigned or derived with a secret key, rather than being derived directly from the endpoint IP address, so that internal addressing information is not exposed to third parties.

IANA Considerations IANA is requested to allocate one TCP option Kind number from the TCP option kind registry for the Session Recovery option: Kind Length Meaning Reference TBD variable SR This document

In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. RFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings. This document specifies the Transmission Control Protocol (TCP). TCP is an important transport-layer protocol in the Internet protocol stack, and it has continuously evolved over decades of use and growth of the Internet. Over this time, a number of changes have been made to TCP as it was specified in RFC 793, though these have only been documented in a piecemeal fashion. This document collects and brings those changes together with the protocol specification from RFC 793. This document obsoletes RFC 793, as well as RFCs 879, 2873, 6093, 6429, 6528, and 6691 that updated parts of RFC 793. It updates RFCs 1011 and 1122, and it should be considered as a replacement for the portions of those documents dealing with TCP requirements. It also updates RFC 5961 by adding a small clarification in reset handling while in the SYN-RECEIVED state. The TCP header control bits from RFC 793 have also been updated based on RFC 3168. This document attempts to describe the operation of NAT devices and the associated considerations in general, and to define the terminology used to identify various flavors of NAT. This memo provides information for the Internet community. This document defines a set of requirements for NATs that handle TCP that would allow many applications, such as peer-to-peer applications and online games to work consistently. Developing NATs that meet this set of requirements will greatly increase the likelihood that these applications will function properly. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. This document specifies the TCP Authentication Option (TCP-AO), which obsoletes the TCP MD5 Signature option of RFC 2385 (TCP MD5). TCP-AO specifies the use of stronger Message Authentication Codes (MACs), protects against replays even for long-lived TCP connections, and provides more details on the association of security with TCP connections than TCP MD5. TCP-AO is compatible with either a static Master Key Tuple (MKT) configuration or an external, out-of-band MKT management mechanism; in either case, TCP-AO also protects connections when using the same MKT across repeated instances of a connection, using traffic keys derived from the MKT, and coordinates MKT changes between endpoints. The result is intended to support current infrastructure uses of TCP MD5, such as to protect long-lived connections (as used, e.g., in BGP and LDP), and to support a larger set of MACs with minimal other system and operational changes. TCP-AO uses a different option identifier than TCP MD5, even though TCP-AO and TCP MD5 are never permitted to be used simultaneously. TCP-AO supports IPv6, and is fully compatible with the proposed requirements for the replacement of TCP MD5. [STANDARDS-TRACK] This document specifies a set of TCP extensions to improve performance over paths with a large bandwidth * delay product and to provide reliable operation over very high-speed paths. It defines the TCP Window Scale (WS) option and the TCP Timestamps (TS) option and their semantics. The Window Scale option is used to support larger receive windows, while the Timestamps option can be used for at least two distinct mechanisms, Protection Against Wrapped Sequences (PAWS) and Round-Trip Time Measurement (RTTM), that are also described herein. This document obsoletes RFC 1323 and describes changes from it. This document describes an experimental TCP mechanism called TCP Fast Open (TFO). TFO allows data to be carried in the SYN and SYN-ACK packets and consumed by the receiving end during the initial connection handshake, and saves up to one full round-trip time (RTT) compared to the standard TCP, which requires a three-way handshake (3WHS) to complete before data can be exchanged. However, TFO deviates from the standard TCP semantics, since the data in the SYN could be replayed to an application in some rare circumstances. Applications should not use TFO unless they can tolerate this issue, as detailed in the Applicability section. TCP/IP communication is currently restricted to a single path per connection, yet multiple paths often exist between peers. The simultaneous use of these multiple paths for a TCP/IP session would improve resource usage within the network and thus improve user experience through higher throughput and improved resilience to network failure. Multipath TCP provides the ability to simultaneously use multiple paths between peers. This document presents a set of extensions to traditional TCP to support multipath operation. The protocol offers the same type of service to applications as TCP (i.e., a reliable bytestream), and it provides the components necessary to establish and use multiple TCP flows across potentially disjoint paths. This document specifies v1 of Multipath TCP, obsoleting v0 as specified in RFC 6824, through clarifications and modifications primarily driven by deployment experience. CMCC CMCC This document describes an experimental protocol named the Available Session Recovery Protocol (ASRP). The protocol is designed to optimize high-availability network cluster architectures, providing a superior high-availability solution for clusters offering stateful network services such as load balancing and Network Address Translation (NAT [RFC4787]). ASRP defines the procedures for session backup and recovery, as well as the message formats used during these interactions, enabling efficient and streamlined session state management. In contrast to traditional high-availability techniques that back up session state within the cluster itself, the core innovation of ASRP lies in its distributed backup of state information to the client or server side. This approach offers multiple advantages: theoretically unlimited elastic scaling capacity; support for rapid recovery from multi-point failures; reduction of resource redundancy through the elimination of centralized backup nodes; and significant simplification of cluster implementation complexity. The ASRP protocol provides a standardized method for constructing elastic service clusters, facilitating broader participation from software and hardware developers in building elastic cloud network service clusters.

Acknowledgments The authors would like to thank all individuals who have provided valuable feedback and contributions during the development of this document, especially Mike Heard and Michael Tuexen for their detailed review comments.

Changes since draft-cmcc-tcp-sro-01 Renamed the option abbreviation from "SRO" to "SR". Repositioned the option as serving on-path network functions and generalized its description beyond session recovery; rewrote the abstract and Introduction, added the "Other Uses" section, and made the two carrying modes equal deployment choices. Added normative rules: the "Identifier Management" section and negotiation symmetry. Rewrote the Security Considerations.