Encapsulation for In Situ Operations, Administration, and Maintenance Data Using MPLS Network Actions

Encapsulation for In Situ Operations, Administration, and Maintenance Data Using MPLS Network Actions Cisco Systems, Inc.

Canada rgandhi@cisco.com

Ciena Corporation

gregimirsky@gmail.com

Futurewei Technologies

United States of America haoyu.song@futurewei.com

Comcast

United States of America Bin_Wen@cable.comcast.com

Comcast

United States of America Voitek_Kozak@comcast.com

Routing MPLS Working Group Internet-Draft IOAM MPLS Network Action In situ Operations, Administration, and Maintenance (IOAM), defined in RFC 9197, collects operational and telemetry information in the packet using IOAM-Data-Fields while the packet traverses a path between two points in the network. Several IOAM Option-Types are available, for example, Pre-allocated Trace, Proof of Transit (POT), Edge-to-Edge (E2E), and Incremental Trace, that can be used to collect information for calculating various performance metrics. RFC 9326 defines the IOAM Direct Export (IOAM-DEX) Option-Type, which is used as a trigger for IOAM data to be directly exported or locally aggregated without being pushed into in-flight data packets. MPLS Network Actions (MNA) mechanisms indicate actions to be performed on any combination of Label Switched Paths, MPLS packets, and the node itself, and to transport data needed for these actions. This document employs the MNA mechanisms to collect and transport the operational state and telemetry information using IOAM-Data-Fields as well as IOAM-DEX.

Introduction In situ Operations, Administration, and Maintenance (IOAM) collects operational and telemetry information in the packet using IOAM-Data-Fields while the packet traverses a path between two points in the network. Several IOAM Option-Types are available, for example, Pre-allocated Trace, Proof of Transit (POT), Edge-to-Edge (E2E), and Incremental Trace, that can be used to collect information for calculating various performance metrics. Such mechanisms transport the collected information from an IOAM encapsulating node to an IOAM decapsulating node (both typically located at the edge of the IOAM domain within the data path). IOAM Direct Export (IOAM-DEX) is an IOAM Option-Type used as a trigger for IOAM data to be directly exported or locally aggregated without being pushed into in-flight data packets. The exporting method and format are outside the scope of . MPLS Network Actions (MNA) mechanisms indicate actions to be performed on any combination of Label Switched Paths, MPLS packets, and the node itself, and allow for the transport of data needed for these actions. defines mechanisms for carrying a Network Action Sub-Stack (NAS) as part of the MPLS label stack, i.e., the In-Stack MNA solution. defines mechanisms for carrying MNA and Ancillary Data (AD) below the MPLS label stack, i.e., as the Post-Stack MNA solution. describes various use cases that can be realized using the MNA solution, including the IOAM and IOAM-DEX. This document employs the MNA mechanisms to collect and transport the operational state and telemetry information using IOAM Option-Types, including Pre-allocated Trace, POT, and E2E, as well as IOAM-DEX. The mechanism for exporting the collected information for the IOAM-DEX Option-Type is outside the scope of this document. Also, transporting the Incremental Trace IOAM Option-Type is outside the scope of this document.

Conventions Used in This Document

Abbreviations The terminology defined in , , and is used in this document. Abbreviations

Abbreviation	Meaning	Reference
AD	Ancillary Data
Blob	Binary Large Object	This document
BoS	Bottom of Stack
HbH	Hop-by-Hop
I2E	Ingress to Egress
E2E	Edge to Edge
IHS	I2E, HbH, or Select
IOAM	In situ Operations, Administration, and Maintenance
IOAM-DEX	IOAM Direct Export
IOAM-DEX-MNA-ISD	IOAM Direct Export as MPLS Network Action ISD	This document
ISD	In-Stack Data
LSE	Label Stack Entry
MNA	MPLS Network Action
MPLS	Multiprotocol Label Switching
NAI	Network Action Indicator
NAL	Network Action Length
NAS	Network Action Sub-Stack
NASL	Network Action Sub-Stack Length
OAM	Operations, Administration, and Maintenance
P bit	Post-Stack Data Presence bit
POT	Proof of Transit
RLD	Readable Label Depth	and
PFN	Post-Stack First Nibble
PSD	Post-Stack Data
PSMH	Post-Stack MPLS Header
S bit	Bottom of Stack bit
TC	Traffic Class
TTL	Time to Live
U bit	Unknown Network Action Handling bit

Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

Applicability of IOAM and IOAM-DEX in an MPLS Network Pre-allocated Trace, POT, and E2E IOAM Option-Types use user packets to collect and transport the operational state and telemetry information. This document defines the Post-Stack MNA solution for encapsulation of the Pre-allocated Trace, POT, and E2E IOAM Option-Types (see ). However, for some use cases, e.g., mobile backhaul, in which network resources are closely controlled, collecting and transporting telemetry information within a user packet may increase the complexity of network operations. In such environments, IOAM nodes can use IOAM-DEX, an IOAM Option-Type used as a trigger for IOAM data to be directly exported or locally aggregated without being pushed into in-flight data packets, as defined in . IOAM-DEX collects the on-path telemetry information defined as IOAM data in . In this document, encapsulations for IOAM-DEX are realized as In-Stack Data (see ) and as Post-Stack Data (see ). The procedure defined in this document for carrying IOAM Option-Types using the MNA solution can be applied to user traffic packets and active measurement test packets. uses the mechanisms defined in this document to transport IOAM Option-Types using the MNA solution, with Simple Two-Way Active Measurement Protocol (STAMP) test packets for Hop-by-Hop (HbH) and E2E measurements. also describes active measurement methods where the mechanisms defined in this document can carry IOAM Option-Types for on-path telemetry in MPLS networks.

Realization of IOAM and IOAM-DEX as Post-Stack MNA The procedure defined utilizes the In-Stack MNA mechanisms and Post-Stack MNA mechanisms to transport IOAM Option-Types defined in and IOAM-DEX Option-Type defined in ) as Post-Stack MNA in MPLS networks.

NAS for IOAM and IOAM-DEX as Post-Stack MNA The presence of the associated Post-Stack MPLS Header (PSMH) is indicated by setting the P bit to 1 in the NAS as defined in . An example of a NAS with a Format B LSE and the associated PSMH carrying IOAM and IOAM-DEX Option-Types is shown in .

Example of NAS with Format B LSE for Carrying IOAM and IOAM-DEX in PSMH The fields in a NAS are defined as follows:

The MNA Label (value 4) is defined in .
S (1-bit): The BoS .
P (1-bit): The P bit is set as specified in .
The NAL and NASL fields are set as specified in .
Opcode (7-bit): The opcode is set as specified in .
Data: The data is set as specified in .
The U bit is set as specified in for In-Stack Network Actions and updated in .
The IHS scope field is set as defined in and updated in .

If both edge and intermediate nodes need to process the IOAM Option-Types, then the IHS scope MUST be set to "HbH, value 0x1". If only edge nodes need to process the IOAM Option-Types, then the IHS scope MUST be set to "I2E, value 0x0". The I2E scope allows skipping IOAM processing on the intermediate nodes, i.e., it avoids the need to parse all IOAM Option-Types to detect the one that requires HbH processing.

PSMH for IOAM and IOAM-DEX as Post-Stack MNA An example encoding of PSMH carrying IOAM and IOAM-DEX in Post-Stack MNA is shown in .

Example of Post-Stack MPLS Header Carrying IOAM and IOAM-DEX The Post-Stack MPLS Base Header is added and contains the following fields as defined in .

PFN (4-bit): The Post-Stack First Nibble (PFN) (value 0x0) as defined in .
PSMH-Len (8-bit): The PSMH total length in units of 4 octets as defined in . The value excludes the first 4-octet unit for Post-Stack MPLS Base Header. When carrying N IOAM and IOAM-DEX network actions, all with the same PS-NAL value, PSMH-Len equals (1 + PS-NAL) * N.
Type (16-bit): Type for MNA Post-Stack Header (value 1) as defined in .

The PSMH is added after the Post-Stack MPLS Base Header and contains the Post-Stack network action opcode for IOAM and IOAM-DEX, the length in 4-octet units, and the IOAM Option-Type with IOAM-Data-Fields in the Post-Stack ancillary data as shown in and contains the following fields:

MNA-PS-OP (7-bit): Set to Opcode TBA1 (network action opcode for IOAM and IOAM-DEX in PSD) for the IOAM Option-Type defined in , and the IOAM-DEX Option-Type defined in . The U bit for this network action is set as specified in . The IHS scope field for this network action is set to either I2E or HBH .
R (3 x 1-bit): Three reserved bits appear in the PSMH row: two bits following MNA-PS-OP and one bit preceding IOAM-Opt-Type. All reserved bits MUST be set to 0 on transmission and MUST be ignored on receipt.
IOAM-Opt-Type (7-bit): The field to carry the IOAM Option-Type, as defined in the "IOAM Option-Type Registry" in and .
PS-NAL (7-bit): Unsigned integer representing the length of the IOAM-Data-Fields in 4-octet units for the MNA-PS-OP. The value excludes the first 4-octet unit.
IOAM Option-Type and Data Space: IOAM-Data-Fields as specified by the IOAM-Opt-Type field, beginning from the Namespace-ID field. IOAM-Data-Fields are defined according to the IOAM Option-Type (e.g., see and ).
Block-Number (8-bit): The Block-Number is used for the alternate marking method to aggregate the IOAM-Data-Fields collected in the data plane and to correlate IOAM-Data-Fields exported from different nodes along the packet path, e.g., to compute measurement metrics for each block of a data flow.
- When the Flow ID field is present in the IOAM-Data-Fields in the PSMH, the Block-Number is scoped per data flow within the IOAM Namespace identified by the Namespace-ID field.
- When the Flow ID field is absent, the Block-Number is scoped per IOAM Namespace (identified by the Namespace-ID field) and applies to all flows sharing that Namespace.
- The Block-Number MUST be initialized to 0 at the start of operation and wraps from 255 back to 0 on overflow. It is incremented sequentially at every measurement interval configured at the encapsulating node. When the alternate marking method is not used, the Block-Number SHOULD be set to 0.

The following processing rules apply to the network action opcode TBA1.

An MPLS packet MAY carry multiple network action opcode TBA1 in the same PSMH for different Option-Types.
An MPLS packet MAY carry network action opcode TBA1 in more than one PSMH for different processing scopes.
An MPLS packet MUST NOT carry network action opcode TBA1 in a NAS. Otherwise, the packet is treated as malformed and MUST be dropped.
The IHS scope in a NAS MUST NOT be set to Select when the network action opcode TBA1 is carried in the associated PSMH. Otherwise, the packet is treated as malformed and MUST be dropped.

Example of Multiple IOAM and IOAM-DEX Option-Types as Post-Stack MNA An example of multiple Post-Stack Network Actions with the same IHS scope carrying different IOAM and IOAM-DEX Option-Types is shown in .

Example of Multiple IOAM and IOAM-DEX Option-Types as Post-Stack MNA In this example, the PSMH carries the post-stack network action MNA-PS-OP1 with length PS-NAL1 and the post-stack network action MNA-PS-OP2 with length PS-NAL2, both for different Option-Types.

Procedure for IOAM and IOAM-DEX as Post-Stack MNA

Ingress-to-Egress Scope IOAM and IOAM-DEX as Post-Stack MNA The network actions with I2E scope carry IOAM Option-Types that require processing on the encapsulating and egress nodes only. The IOAM Option-Type carried can be the IOAM E2E Option-Type (value 3) defined in or the IOAM-DEX Option-Type (value 4) defined in . The network actions with I2E scope MUST NOT carry any IOAM Option-Type that requires IOAM processing on the intermediate nodes, as it will not be processed by them when the IHS scope is set to "I2E, value 0x0". The IOAM and IOAM-DEX network action procedure with I2E scope is summarized as follows:

The encapsulating node inserts a NAS with the IHS scope set to "I2E, value 0x0", as well as one or more network actions for IOAM and IOAM-DEX in the MPLS packet.
The encapsulating node MUST ensure that the packet with all NASs and PSMHs added does not exceed the path MTU, as specified in and .
The egress node processes the IOAM-Data-Fields using the procedures defined in and . An example of IOAM processing is to export the IOAM-Data-Fields for streaming telemetry.
The egress node MUST remove the NAS and the associated PSMH including network actions for IOAM and IOAM-DEX and IOAM-Data-Fields from the received packet. The decapsulated packet is forwarded downstream similar to the regular data packets or terminated locally.

Hop-by-Hop Scope IOAM and IOAM-DEX as Post-Stack MNA The network actions with HbH scope carry IOAM Option-Types that require processing at the intermediate and/or encapsulating and egress nodes. The IOAM Option-Type carried can be Pre-allocated Trace (value 0), POT (value 2), or E2E (value 3) defined in , or the IOAM-DEX Option-Type (value 4) defined in . Note that the network actions defined in this document do not support the IOAM Incremental Trace Option-Type (value 1), which requires HbH processing. The IOAM and IOAM-DEX network action procedure with HbH scope is summarized as follows:

The encapsulating node inserts a NAS with the IHS scope set to "HbH, value 0x1", as well as one or more network actions for IOAM and IOAM-DEX in the MPLS packet.
The intermediate node that is enabled for the IOAM function processes the IOAM-Data-Fields as defined in and when the node recognizes the HbH scope in the NAS.
The intermediate node forwards the data packet downstream.
The processing on the egress node is the same as in the I2E case.

Both HbH and I2E scope network actions for IOAM and IOAM-DEX MAY be carried in the Post-Stack MNA in an MPLS packet. In this case, the PSMH carrying HbH-scoped network actions MUST be added after the BoS and before the PSMH carrying I2E-scoped network actions. This minimizes the Readable Label Depth (RLD) required on intermediate nodes for processing IOAM and IOAM-DEX.

Realization of IOAM-DEX as In-Stack MNA The procedure defined adopts the IOAM-DEX Option-Type format defined in using MNA In-Stack Data (ISD) to support direct export in MPLS networks.

NAS for IOAM-DEX as In-Stack MNA To transport direct export of the operational state and telemetry information, the IOAM-DEX-MNA-ISD Binary Large Object (blob) is placed in a NAS according to the procedure defined in . An example NAS with Format B LSE and Format D LSE to carry IOAM-DEX-MNA-ISD is shown in .

Example of NAS with Format B LSE and Format D LSE to Carry IOAM-DEX as In-Stack MNA Here, the fields in a NAS are defined as follows:

The MNA Label (value 4) is defined in .
S (1-bit): The BoS .
P (1-bit): The P bit is set as specified in .
The U bit and NAL field are set as specified in .
NASL (4-bit): The number of LSEs that compose the IOAM-DEX-MNA-ISD blob.
Opcode (7-bit): Set to network action opcode TBA2 for IOAM-DEX in ISD. The opcode TBA2 can be carried in Format B or Format C LSE .
Data: MUST be set to 0 in both cases, 13-bit data in Format B LSE and 20-bit data in Format C LSE .
IOAM-DEX-MNA-ISD: IOAM-DEX in MNA ISD encoding in Format D LSE .

By setting the IHS field , the IOAM-DEX-MNA-ISD can be configured to operate in HbH, I2E, or Select scopes to collect the operational state and telemetry information. The following processing rules apply to the network action opcode TBA2.

An MPLS packet MUST NOT carry multiple network action opcode TBA2 in the same NAS. Otherwise, the packet is treated as malformed and MUST be dropped.
An MPLS packet MAY carry network action opcode TBA2 in more than one NAS for different processing scopes.
An MPLS packet MUST NOT carry network action opcode TBA2 in a PSMH. Otherwise, the packet is treated as malformed and MUST be dropped.

The following processing rules apply when carrying both the network action opcode TBA1 for the IOAM-DEX Option-Type in a PSMH and the network action opcode TBA2 in a NAS in an MPLS packet:

The network action opcode TBA2 in a NAS and the network action opcode TBA1 for the IOAM-DEX Option-Type in the associated PSMH MUST NOT be carried, due to their same IHS scope, in an MPLS packet; otherwise, the packet MUST be treated as malformed and MUST be dropped.
The network action opcode TBA1 for IOAM-DEX Option-Type in a PSMH and network action opcode TBA2 in a NAS MAY be carried for different IHS scopes in an MPLS packet.
Each network action for the IOAM-DEX Option-Type operates independently: opcode TBA2 triggers direct export at nodes processing the In-Stack MNA, while opcode TBA1 with IOAM Option-Type triggers direct export at nodes processing the Post-Stack MNA.

IOAM-DEX Option-Type Encoding as MNA ISD The IOAM-DEX-MNA-ISD blob in a NAS uses the Format D LSE , that maps to the IOAM-DEX Option-Type format . In addition to the requirement to preserve the S bit, the most significant bit in Format D LSE is always set to 1, avoiding a possible mix-up of the LSE with one of the Base Special Purpose Labels . The format of the IOAM-DEX-MNA-ISD blob in a NAS is shown in .

IOAM-DEX Option-Type Format for MNA ISD Here, the fields are defined as follows:

Namespace-ID (16-bit): Identifier of the IOAM Namespace, as defined in .
Reserved (6-bit): The Reserved field MUST be set to 0 on transmit and ignored on receipt.
Flags (8-bit): The Flags field comprises eight one-bit subfields. The subfields in the Flags field are allocated by IANA in the IOAM DEX Flags registry, as defined in .
IOAM-Trace-Type-MNA (22-bit): The IOAM-Trace-Type-MNA corresponds to the IOAM-Trace-Type field defined in .
- The interpretation of bit positions, from bit 0 through bit 21, is as specified in .
- Note that bits as specified in are right-shifted one bit, so for example, bit 0 in (leftmost bit for hop_Lim and node_id in short format) is mapped to bit 1 (second leftmost bit) in the MNA encoding.
- The registry that contains the assigned code points is found in IANA's IOAM Trace-Type registry .
O (1-bit): This is a one-bit flag identical to the interpretation of bit 22 of the IOAM-Trace-Type field as defined in , marked as "variable-length Opaque State Snapshot" per ; assigned code points are found in IANA's IOAM Trace-Type registry .
R (1-bit): This is a one-bit flag identical to the interpretation of bit 23 of the IOAM-Trace-Type field as defined in , marked as "reserved" per ; assigned code points are found in IANA's IOAM Trace-Type registry .
- Similarly to , the R bit is reserved to allow for future extensions of the IOAM-Trace-Type-MNA bit field, i.e., it indicates the presence of the extended IOAM-Trace-Type-MNA field in the next LSE in Format D. The format of the extended IOAM-Trace-Type-MNA field in the next LSE in Format D is outside the scope of this document.
The concatenation of the 22-bit IOAM-Trace-Type-MNA field with O and R flags, explained above, is identical to the 24-bit IOAM-Trace-Type field in , and in the interpretation of those bits as defined in and assigned code points found in IANA's IOAM Trace-Type registry .
Ext-Flags (6-bit): The Ext-Flags field comprises six one-bit subfields. The allocation of the subfields in this field is according to the Extension-Flags field defined in .
- Four flags have been assigned by IANA: two as defined in indicating the presence of the optional Flow ID and Sequence Number fields, and two as defined in indicating the presence of the optional Multicast Branch ID field.
- describes the detailed format of the Ext-Flags field.
In the IOAM-DEX Option-Type defined in , the IOAM-Trace-Type and Reserved fields together form the second 32-bit word, aligning the optional fields to 4-octet boundaries.
- In the case of the IOAM-DEX in MNA, for optional fields such as Flow ID MNA and Sequence Number MNA, such alignment is achieved without adding a Reserved field for padding.
Flow ID MNA: An optional 4-octet field containing a 30-bit Flow ID (after removing leading 1 and S bits).
- The semantics of the Flow ID MNA field are the same as those of the Flow ID field defined in , with the following two differences.
  1. The Flow ID MNA field carries a 30-bit value, compared to the 32-bit Flow ID field in ; therefore, the Flow ID space differs accordingly, and the 30-bit value will wrap around faster than the 32-bit value.
  2. The most significant bit MUST be set to 1. Bit 23 MUST be set according to the definition of the S bit in .
Sequence Number MNA: An optional 4-octet field containing a 30-bit sequence number (after removing leading 1 and S bits).
- The semantics of the Sequence Number MNA field are the same as those of the Sequence Number field defined in , with the following two differences.
  1. The Sequence Number MNA field carries a 30-bit value, compared to the 32-bit Sequence Number field in ; therefore, the sequence number space differs accordingly, and the 30-bit value will wrap around faster than the 32-bit value.
  2. The most significant bit MUST be set to 1. Bit 23 MUST be set according to the definition of the S bit in .
- In MPLS networks, when it is known that the label stack information is used for load-balancing flows, the 19-bit-long part of the Sequence Number MNA, starting from the bit 1 position of the LSE, MUST remain immutable for a particular packet flow identified by the value of the Flow ID MNA field.
- In MPLS networks, when it is known that other load-balancing techniques are used, all bits of the Sequence Number MNA field can be varied. Mechanisms to learn about the load-balancing techniques used by the nodes in MPLS networks are outside the scope of this document.

Ext-Flags Definition The length of the Ext-Flags field in the IOAM-DEX Option-Type in MNA is shorter by two one-bit fields compared to the length of the Extension-Flags field defined in . In the 6-bit Ext-Flags field of the IOAM-DEX-MNA-ISD header, four flags have been mapped to those IANA assigned flags in the IOAM-DEX Option-Type, and two flags (corresponding to unassigned bits in the Extension-Flags field) remain unassigned.

Ext-Flags Field Format Here, the Ext-Flags field is defined as follows:

F (1-bit): One-bit flag. When the F flag is set to 1, it indicates the presence of the Flow ID field in the IOAM-DEX-MNA-ISD header.
Q (1-bit): One-bit flag. When the Q flag is set to 1, it indicates the presence of the Sequence Number field in the IOAM-DEX-MNA-ISD header. The designation as Q flag is used to avoid confusion with the existing S bit for Bottom-of-Stack (BoS) in the MPLS header. This flag corresponds to the Sequence Number flag (labeled "S" in ) in the IOAM DEX Extension-Flags registry.
N (1-bit): One-bit flag. When the N flag is set to 1, it indicates the presence of the Multicast Branching Node ID in the Multicast Branch ID field, as defined in .
I (1-bit): One-bit flag. When the I flag is set to 1, it indicates the presence of the Multicast Branching Interface Index in the Multicast Branch ID field, as defined in . The N and I flags MUST be both set or both cleared; otherwise the header is considered malformed and the packet MUST be dropped.
A (1-bit): Unassigned one-bit flags. They MUST be set to zero on transmission and MUST be ignored upon receipt.

Considerations for IOAM and IOAM-DEX in MPLS Networks

Node Capability The following behaviors for node capability apply:

An MNA-capable egress node that supports the network action for IOAM and IOAM-DEX may not support the specific IOAM Option-Type or IOAM-Data-Fields carried in the packet. In that case, the egress node MUST handle the network action according to the U bit as defined in and updated in .
The MNA-capable intermediate node that does not support the network action for IOAM and IOAM-DEX, or the IOAM Option-Type or IOAM-Data-Fields carried in the packet, MUST handle the network action according to the U bit as defined in and updated in .
The intermediate node that does not support the MNA skips processing the IOAM and IOAM-DEX functions.

The encapsulating node needs to know if the intermediate and egress nodes can support the IOAM and IOAM-DEX network actions. Information about the IOAM and IOAM-DEX capabilities of the nodes may be configured, collected through management protocols, or distributed by control protocols (such as advertising by routing protocols). The encapsulating node learns about the IOAM and IOAM-DEX capabilities of nodes using mechanisms that are out of scope for this document. The encapsulating node needs to know about the path maximum transmission unit (MTU) for the nodes on the path . Information about the path MTU of the nodes may be configured, collected through management protocols, or distributed by control protocols (such as MTU signaling for Label Distribution Protocol and Path MTU Discovery for IPv6 ). The mechanisms to learn about the path MTU of nodes in the path are out of scope for this document. The encapsulating node needs to know about the RLD of the nodes in the path as described in and updated in , so that IOAM and IOAM-DEX Option-Types can be read by nodes in the path. Information about the RLD of the nodes may be configured, collected through management protocols, or distributed by control protocols (such as advertising by routing protocols). The mechanisms to learn about the RLD of nodes in the path are out of scope for this document.

Readable Label Depth The encapsulating node needs to ensure that the IOAM Option-Types and their IOAM-Data-Fields and IOAM-DEX data in a NAS and the associated PSMH are added within the RLD of the downstream MNA-capable nodes so that they can process the IOAM-Data-Fields and IOAM-DEX data.

Nested MPLS Encapsulation The following processing rules apply to nested MPLS encapsulation when adding a new MPLS encapsulation:

When a packet is received with a network action opcode TBA2 for IOAM-DEX in ISD or opcode TBA1 for IOAM and IOAM-DEX in PSD, and the nested MPLS encapsulating node needs to add the network action opcode TBA2 for a different scope for IOAM-DEX in ISD, the node MUST add a new NAS with network action opcode TBA2 as part of the new MPLS encapsulation.
When a packet is received with a network action opcode TBA2 for IOAM-DEX in ISD or opcode TBA1 for IOAM and IOAM-DEX in PSD, the nested MPLS encapsulating node MUST NOT add the network action opcode TBA2 for the same scope for IOAM-DEX in ISD as part of the new MPLS encapsulation.
When a packet is received with an MPLS header, the nested MPLS encapsulating node MUST NOT add the new network action opcode TBA1 for IOAM and IOAM-DEX in PSD as part of the new MPLS encapsulation. This is because inserting a PSMH in an MPLS-encapsulated packet received from the network is not supported by .

Operational Considerations Operational considerations discussed in and apply to this document. Management considerations discussed in , management and deployment considerations discussed in , and performance considerations discussed in are also applicable here. Further operational considerations include policies controlling the processing of the collected operational state and telemetry information, and their transport in MPLS networks. Additional considerations are described in for deploying IOAM and IOAM-DEX in MPLS networks. An implementation MAY collect the following counters:

Packets received with IOAM network actions
Packets received with IOAM-DEX in ISD network actions
Packets dropped with IOAM network actions due to an unknown IOAM function
Packets dropped with IOAM-DEX in ISD network actions due to an unknown IOAM-DEX function
Packets dropped with IOAM network actions due to a malformed network action
Packets dropped with IOAM-DEX in ISD network actions due to a malformed network action

Nodes MAY generate rate-limited notifications or alarms for significant operational events, such as sustained high rates of IOAM-related packet drops to alert operators to potential issues. Comprehensive logging of IOAM and IOAM-DEX network action processing details can aid in network diagnostics and post-mortem analysis.

Security Considerations The security considerations discussed in , , and for IOAM and IOAM-DEX apply to this document. The security considerations discussed in for alternate marking and for IOAM deployment apply to this document. The security considerations discussed in , , and for MNA apply to this document. The usage of network actions defined in this document for IOAM and IOAM-DEX is intended for deployment in a single network administrative domain. As such, it assumes that the operator enabling the IOAM and IOAM-DEX operations has previously verified the integrity of the path that packets take. However, operators need to properly secure the IOAM and IOAM-DEX in the domain to avoid malicious configuration and use, which could include injecting malicious IOAM and IOAM-DEX packets into the domain.

IANA Considerations

Network Action Opcodes IANA is requested to assign code points from its Network Action Opcodes registry (created in and updated in ) as specified in . Network Action Opcodes

Opcode	Description	In-Stack Only, Post-Stack Only, In-Stack and Post-Stack	Reference
TBA1	Network Action for IOAM and IOAM-DEX in PSD	Post-Stack Only	This document
TBA2	Network Action for IOAM-DEX in ISD	In-Stack Only	This document

References Normative References Informative References IOAM Trace-Type IANA

Appendix A: In-Stack and Post-Stack Network Action Processing Order Example As described in , the network actions in a NAS are processed in order starting from the top of the label stack. The PSMH Start Offset Network Action opcode TBA3 defined in can be added to interleave Post-Stack Network Actions with In-Stack Network Actions to define processing order. The following example shows how to process the Post-Stack Network Action before some of the In-Stack Network Actions in a NAS.

Example of In-Stack and Post-Stack Network Action Processing Order In this example, shown in , the network actions in the NAS are processed in the following order:

Opcode L (allocated outside of this document) is processed first.
Opcode 1 (for Flag-Based NAIs defined in ) is processed next.
Opcode TBA3 (Network Action for PSMH Start Offset opcode defined in ) is processed next.
Opcode TBA1 (Network Action for IOAM and IOAM-DEX in PSD opcode) in the associated PSMH is processed next. The Data value set to 0 indicates that the PSMH starts immediately after the BoS.
Opcode M (allocated outside of this document) is processed finally.

Note: Opcode TBA3 will be assigned by IANA as part of . This example will be updated by the RFC editor with the IANA assigned value for TBA3 at the time of publication. This note is to be removed once TBA3 value is updated in this example.

Acknowledgments The authors would like to thank Adrian Farrel and Xueyan Song for reviewing this document and providing review comments. The authors would also like to thank Patrick Khordoc, Sagar Soni, Shwetha Bhandari, Vengada Prasad Govindan, Tarek Saad, Stewart Bryant, Xiao Min, Jaganbabu Rajamanickam, Jie Dong, and Cheng Li for reviewing the early version of this document. The authors would also like to thank Mach Chen, Andrew Malis, Matthew Bocci, and Nick Delregno for the MPLS-RT expert review of the early version of this document. The authors also thank Matthew Bocci for the early Rtgdir review, Sheng Jiang for early Opsdir review, and Giuseppe Fioccola for the PerfMetrDir review, which helped improve this document.

Contributors The following people have substantially contributed to this document: