]> intoPIX S.A.

Rue de Rodeuhaie 1 Louvain-la-Neuve B-1348 Belgium +32 10 23 84 70 t.bruylants@intopix.com https://www.intopix.com/

Fraunhofer IIS

Am Wolfsmantel 33 Erlangen D-91058 Germany +49 9131 776 5126 thomas.richter@iis.fraunhofer.de https://www.iis.fraunhofer.de/

intoPIX S.A.

Rue de Rodeuhaie 1 Louvain-la-Neuve B-1348 Belgium +32 10 23 84 70 c.damman@intopix.com https://www.intopix.com/

Université Catholique de Louvain

Ruelle de la Lanterne Magique, 14 Louvain-la-Neuve B-1348 Belgium +32 10 47 27 87 antonin.descampe@uclouvain.be https://uclouvain.be/

General avtcore Internet-Draft JPEG XS video transport real-time protocol This document specifies a Real-Time Transport Protocol (RTP) payload format for transport of a video signal encoded with JPEG XS (ISO/IEC 21122). JPEG XS is a low-latency and low-complexity video coding system. Employing this format allows achieving encoding-decoding latencies confined to a fraction of a video frame. This document revises RFC 9134 to incorporate support for new features introduced in the third edition of JPEG XS. Most notably, it contains the necessary provisions to support the TDC coding mode. This document obsoletes RFC 9134; however, the revised payload format is designed to ensure that existing conforming implementations of RFC 9134 remain valid under the updated specification. Additionally, this document consolidates the errata of RFC 9134 and includes improvements and clarifications for implementers and users.

Introduction This document specifies a payload format for packetization of video signals encoded with JPEG XS into the Real-time Transport Protocol (RTP) . The JPEG XS coding system offers compression and recompression of video signals with very moderate computational resources while remaining robust under multiple compression and decompression cycles as well as mixing of content sources, e.g., embedding of subtitles, overlays, or logos. Typical target compression ratios ensuring visually lossless quality are in the range of 2:1 to 18:1 depending on the nature of the source material. The latency that is introduced by the encoding-decoding process can be confined to a fraction of a video frame, typically expressed in a number of lines. Initially, the first and second editions of JPEG XS only supported intra coding for video content. However, the third edition of the standard introduced the so-called Temporal Differential Coding (TDC) mode, which provides a temporal decorrelation step in the wavelet domain. For progressive video content, a single frame buffer is used for the decorrelation of successive video frames. For interlaced and PsF content, two frame buffers are used, one for each video field. This document revises to incorporate features added in the third edition of JPEG XS. Most notably, it contains the provisions required to support the TDC coding mode. This document obsoletes ; however, the revised payload format is designed to ensure that existing conforming implementations of remain valid under the updated specification. Additionally, this document consolidates the errata of and provides improvements and clarifications for implementers and users. provides further details on the changes between and this revision.

Conventions, Definitions, and Abbreviations The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

Application Data Unit (ADU):

The unit of source data provided as payload to the transport layer. In this RTP payload definition, it corresponds to a single JPEG XS frame.

Color Specification (CS) box:

An ISO Color Specification box defined in that includes color-related metadata required to correctly display JPEG XS video frames, such as color primaries, transfer characteristics, and matrix coefficients.

End of Codestream (EOC) marker:

A marker that consists of the two bytes 0xff11 indicating the end of a JPEG XS codestream, as defined in .

Frame Buffer Bandwidth (FBB):

The bandwidth defined in needed to read from and write to the internal frame buffer when employing the TDC coding mode. This bandwidth is modeled and capped based on an FBB level parameter.

JPEG XS codestream:

A sequence of bytes representing a compressed video frame (progressive), segment (PsF), or field (interlaced), formatted according to .

JPEG XS codestream header:

A sequence of bytes, starting with an SOC marker, at the beginning of each JPEG XS codestream encoded in multiple markers and marker segments, that does not carry entropy coded data, but only metadata such as the video frame (progressive) or field (interlaced and PsF) dimension and component precision.

JPEG XS frame:

In the case of progressive video, a single JPEG XS picture segment. In the case of interlaced and PsF video, the concatenation of two JPEG XS picture segments.

JPEG XS header segment:

The concatenation of a Video Support box , a Color Specification box , and a JPEG XS codestream header.

JPEG XS picture segment:

The concatenation of a Video Support box , a Color Specification box , and a JPEG XS codestream.

JPEG XS stream:

A sequence of JPEG XS frames.

Marker:

A two-byte functional sequence that is part of a JPEG XS codestream starting with a 0xff byte and a subsequent byte defining its function.

Marker segment:

A marker along with a 16-bit marker size and payload data following the size.

Packetization unit:

A portion of an ADU whose boundaries coincide with boundaries of RTP packet payloads (excluding payload header), i.e., the first (or respectively, last) byte of a packetization unit is the first (or respectively, last) byte of an RTP packet payload (excluding its payload header).

Progressive segmented Frame (PsF):

A method of representing progressive video as two segments formatted similarly to interlaced fields, with the odd lines in one field and the even lines in the other field. Technically, the segments are equivalent to interlaced fields, but unlike native interlaced video, both fields originate from the same progressive frame and represent the same point in time.

SLH (Slice header) marker:

A marker that represents a slice header, as defined in .

SLI (TDC enabling slice header) marker:

A marker that represents a TDC enabling slice header, as defined in .

Slice:

The smallest independently decodable unit of a JPEG XS codestream, bearing in mind that it decodes to wavelet coefficients, which still require inverse wavelet filtering before visualization.

Start of Codestream (SOC) marker:

A marker that consists of the two bytes 0xff10 indicating the start of a JPEG XS codestream, as defined in . The SOC marker is considered an integral part of the JPEG XS codestream header.

Temporal Differential Coding (TDC):

An inter-frame coding mode defined in and used by certain JPEG XS profiles specified in .

Video Support (VS) box:

An ISO Video Support box, as defined in , that includes metadata required to play back a JPEG XS stream; such metadata could include its maximum bit rate, its subsampling structure, its buffer model, and its frame rate.

Media Format Description This section explains the terminology and concepts used in this document specific to JPEG XS as specified in , , and .

Data Structures JPEG XS is a low-latency and lightweight coding system for compression of digital continuous-tone grayscale and color signals, like images and videos. This coding system provides an efficient representation of visual content through the mathematical tool of wavelet analysis. The wavelet filter process separates each component into multiple bands, where each band consists of multiple coefficients describing the visual signal of a given component within a frequency domain specific to the wavelet filter type, i.e., the particular filter corresponding to the band. Wavelet coefficients are grouped into precincts, where each precinct includes all coefficients over all bands that contribute to a spatial region of the picture. One or multiple precincts are furthermore combined into slices consisting of an integer number of precincts. Precincts do not cross slice boundaries, and wavelet coefficients in precincts that are part of different slices can be decoded independently of each other. However, note that the wavelet transformation runs across slice boundaries. A slice always extends over the full width of the picture segment but may only cover parts of its height.

Codestream A JPEG XS codestream is formed by (in the given order): a JPEG XS codestream header, which starts with a Start of Codestream (SOC) marker, one or more slices, each starting with either an SLH or SLI marker, an EOC marker to signal the end of the codestream. The JPEG XS codestream format, including the definition of all markers, is further provided in . It represents sample values of a single picture, without any interpretation relative to a color space. As defined in , slices are represented in the codestream as contiguous sequences of bytes, always beginning with a slice header followed by one or more precincts, and optionally including slice-based extension markers. The slice header SHALL be either an SLH or an SLI marker. The last byte of a slice in the codestream SHALL immediately precede either an SLH or SLI marker (indicating the start of the next slice) or an EOC marker (in the case of the final slice). A JPEG XS codestream not using the TDC coding mode can be decoded independently as a stand-alone picture (a video frame or field). However, a codestream that employs the TDC coding mode has a potential dependency on the contents stored in a frame buffer as described in . This frame buffer holds a quantized version of all wavelet coefficients that were reconstructed from decoding the previous codestream. For progressive video streams, a single frame buffer is maintained. For interlaced and PsF video streams, two separate frame buffers are maintained, one for each video field (i.e., video fields are independent of each other).

Video Support box and Color Specification box While the information defined in the codestream is sufficient to reconstruct the sample values of one picture, the interpretation of the samples remains undefined by the codestream itself. The metadata that allows such interpretation is for the purpose of this payload format signaled in the Video Support box and the Color Specification box, which are taken from . These boxes contain the necessary information that is otherwise missing in the raw codestream and allows for correct interpretation and rendering of the JPEG XS stream. The layout and syntax of these boxes, together with their content, are defined in . The Video Support box provides information on the maximum bit rate, the frame rate, the interlaced mode (progressive or interlaced), the color subsampling format, the informative timecode of the current JPEG XS frame or field, the profile, the level/sublevel/fbblevel used, and optionally the buffer model and the mastering display metadata. Note that the profile and level/sublevel/fbblevel, specified respectively by the Ppih and Plev fields , specify limits on the capabilities needed to decode the codestream and handle the output. Profiles represent a limit on the required algorithmic features and parameter ranges used in the codestream. The combination of level and sublevel defines a lower bound on the required throughput for a decoder in the visual (or decoded) domain and the codestream (or coded) domain, respectively. The frame buffer bandwidth (FBB) level defines a lower bound on the required bandwidth to read from and write to the frame buffer(s) when using the TDC coding mode. The actual defined profiles and levels/sublevels/fbblevels, along with the associated values for the Ppih and Plev fields, are defined in . The Color Specification box indicates the color primaries, transfer characteristics, matrix coefficients, and video full range flag needed to specify the color space of the video stream.

JPEG XS frame and picture segment The concatenation of a Video Support box, a Color Specification box, and a JPEG XS codestream forms a JPEG XS picture segment. In the case of a progressive video stream, each JPEG XS frame consists of a single JPEG XS picture segment. In the case of an interlaced video stream, each JPEG XS frame consists of two concatenated JPEG XS picture segments. Each JPEG XS picture segment corresponds exclusively to one of the two fields of the interlaced frame. Packetization of a PsF video stream SHALL follow the same packetization as an interlaced video stream. Each PsF frame consists of two concatenated JPEG XS picture segments. One picture segment corresponds exclusively to the field containing the odd-numbered lines of the progressive frame, and the other picture segment corresponds exclusively to the field containing the even-numbered lines of the progressive frame. Unless explicitly stated otherwise, all provisions, requirements, and procedures specified for interlaced video streams in this document SHALL also apply to PsF video streams. Note that further mandates that all of the Video Support boxes and all of the Color Specification boxes in both picture segments of each JPEG XS frame SHALL have the same respective layouts. Note that the interlaced mode, as signaled by the frat field in the Video Support box, indicates either progressive, interlaced top-field-first, or interlaced bottom-field-first mode. Thus, in the case of interlaced and PsF video, its value SHALL be identical in both picture segments. Note that the frat field in the Video Support box always signals the frame rate, even in the case of interlaced video. This should not be confused with the field rate.

RTP Payload Format This section specifies the payload format for JPEG XS streams over the Real-time Transport Protocol (RTP) . In order to be transported over RTP, each JPEG XS stream is transported in a distinct RTP stream, identified by a distinct synchronization source (SSRC) . A JPEG XS stream is divided into Application Data Units (ADUs), each ADU corresponding to a single JPEG XS frame.

RTP Packetization An ADU is made of several packetization units. If a packetization unit is bigger than the maximum size of an RTP packet payload, the unit is split into multiple RTP packet payloads, as illustrated in . As seen there, each packet SHALL contain (part of) one, and only one, packetization unit. A packetization unit may extend over multiple packets. The payload of every packet SHALL have the same size (e.g., based on the largest packet size supported by the network path) with the possible exception of the last packet of a packetization unit. The boundaries of a packetization unit SHALL coincide with the boundaries of the payload of a packet (excluding the payload header), i.e., the first (or, respectively, last) byte of the packetization unit SHALL be the first (or, respectively, last) byte of the payload (excluding its header). Note that for interlaced frames the requirements of the RTP packetization imply that each packet will only contain data corresponding to exactly one field.

There are two different packetization modes defined for this RTP payload format.

Codestream packetization mode:

In this mode, the packetization unit SHALL be the entire JPEG XS picture segment (i.e., codestream preceded by boxes). This means that a progressive frame will have a single packetization unit, while an interlaced frame will have two. The progressive case is illustrated in .

Slice packetization mode:

In this mode, the packetization unit SHALL be the slice, i.e., there SHALL be data from no more than one slice per RTP packet. The first packetization unit of each JPEG XS picture segment SHALL contain the JPEG XS header segment (i.e., the concatenation of the VS box, the CS box, and the JPEG XS codestream header). This first unit is then followed by successive units, each containing one and only one slice. The packetization unit containing the last slice of a JPEG XS picture segment SHALL also contain the EOC marker immediately following this last slice. This is illustrated in . In the case of an interlaced frame, the JPEG XS header segment of the second field SHALL be in its own packetization unit.

In a constant bitrate (CBR) scenario of JPEG XS, the codestream packetization mode guarantees that a JPEG XS RTP stream will produce both a constant number of bytes per video frame and a constant number of RTP packets per video frame. However, to provide similar guarantees with JPEG XS in a variable bitrate (VBR) mode or when using the slice packetization mode (for either CBR or VBR), additional mechanisms are needed. This can involve a constraint at the rate allocation stage in the JPEG XS encoder to impose a CBR at the slice level, the usage of padding data, or the insertion of empty RTP packets (i.e., an RTP packet whose payload data is empty). But, management of the number of packets produced per video frame is application dependent and not a strict requirement of this RTP payload specification.

RTP Header Usage The format of the RTP header is specified in and reprinted in for convenience. This RTP payload format uses the fields of the header in a manner consistent with that specification. The RTP payload (and the settings for some RTP header bits) for packetization units are specified in .

The version (V), padding (P), extension (X), CSRC count (CC), sequence number, synchronization source (SSRC), and contributing source (CSRC) fields follow their respective definitions in . The remaining RTP header information to be set according to this RTP payload format is set as follows:

Marker (M) [1 bit]:

If progressive video is being transmitted, the marker bit denotes the end of a video frame. If interlaced video is being transmitted, it denotes the end of a field. The marker bit SHALL be set to 1 for the last packet of a JPEG XS picture segment. It SHALL be set to 0 for all other packets.

Payload Type (PT) [7 bits]:

The payload type is a dynamically allocated payload type field that designates the payload as JPEG XS video.

Timestamp [32 bits]:

The RTP timestamp is set to the sampling timestamp of the content (see also and ). A 90 kHz clock rate SHALL be used. If the sampling instant does not correspond to an integer value of the clock, the value SHALL be set to the smallest integer greater than the sampling instant, with no ambiguity. For progressive video, the timestamp denotes the sampling instant of the frame to which the RTP packet belongs. Packets SHALL NOT include data from multiple frames, and all packets belonging to the same frame SHALL have the same timestamp. For interlaced video, the timestamp denotes the sampling instant of the field to which the RTP packet belongs. Packets SHALL NOT include data from multiple fields, and all packets belonging to the same field SHALL have the same timestamp. Use of field timestamps, rather than a frame timestamp and field indicator bit, is needed to support reverse 3-2 pulldown. Several successive RTP packets will consequently have equal timestamps if they belong to the same video frame for progressive content, or the same video field for interlaced content. That is, the time stamp does not change until after the marker bit (M) is set to 1, marking the last packet of the video frame or field. The timestamp is only increased when a new video frame or field begins.

Payload Header Usage The first four bytes of the payload of an RTP packet in this RTP payload format are referred to as the "payload header". illustrates the structure of this payload header.

The payload header consists of the following fields:

Transmission mode (T) [1 bit]:

The T bit indicates whether the transmitter guarantees sequential packet transmission or whether packets may be transmitted out of order. This information allows a receiver to dimension its input buffer(s) accordingly. The T bit SHALL be set to 0 if nothing can be assumed about the transmission order and the transmitter may send out its packets in any order. If T=1, the transmitter SHALL send out the packets as a monotonically increasing sequence according to the F, SEP, and P fields. The T bit value SHALL be identical for all packets of the RTP stream. Note that even with T=1, packets may still arrive out of order relative to the sequence in which they were sent.

pacKetization mode (K) [1 bit]:

The K bit indicates which packetization mode is used. K SHALL be set to 0 to indicate codestream packetization mode is used, whereas K SHALL be set to 1 to indicate that slice packetization mode is used. In the case that the Transmission mode (T) is set to 0 (arbitrary sending order), then K SHALL be set to 1 (slice packetization mode). The K bit value SHALL be identical for all packets of the RTP stream.

Last (L) [1 bit]:

The L bit SHALL be set to 1 to indicate that a packet is the last packet of a packetization unit. The L bit SHALL be set to 0 for all other packets. As the end of a JPEG XS picture segment also ends the packet containing its last packetization unit, the L bit SHALL be set to 1 whenever the M bit is set to 1. In the codestream packetization mode, the L bit and M bit get an equivalent meaning, so they SHALL have identical values in each packet.

Interlaced information (I) [2 bits]:

These two I bits are used to indicate how the JPEG XS frame is scanned (progressive or interlaced). In case of an interlaced frame, they also indicate which JPEG XS picture segment the payload is part of (first or second). 00: The I bits SHALL be set to 00 to indicate that the payload is progressively scanned. 01: This value is reserved for future use. 10: The I bits SHALL be set to 10 to indicate that the payload is part of the first JPEG XS picture segment of an interlaced video frame. The height specified in the included JPEG XS codestream header is half of the height of the entire displayed video frame. 11: The I bits SHALL be set to 11 to indicate that the payload is part of the second JPEG XS picture segment of an interlaced video frame. The height specified in the included JPEG XS codestream header is half of the height of the entire displayed video frame.

F counter [5 bits]:

The Frame (F) counter identifies the video frame number modulo 32 to which a packet belongs. Frame numbers SHALL increment by 1 for each video frame transmitted. The frame number, in addition to the timestamp, may help the decoder manage its input buffer and bring packets back into their natural order. For interlaced frames, both fields SHALL have the same F counter value.

Slice and Extended Packet (SEP) counter [11 bits]:

The SEP counter is used differently depending on the packetization mode. In the case of codestream packetization mode (K=0), this counter resets whenever the Packet counter resets (see ) and increments by 1 whenever the Packet counter overruns. In the case of slice packetization mode (K=1), this counter identifies the slice modulo 2047 to which the packet contributes. If the data belongs to the JPEG XS header segment, this field SHALL have its maximal value, namely 2047=0x07ff. Otherwise, it is the slice index modulo 2047. Slice indices are counted from 0 (corresponding to the top of the picture segment).

P counter [11 bits]:

The Packet (P) counter identifies the packet number modulo 2048 within the current packetization unit. It is set to 0 at the start of the packetization unit and incremented by 1 for every subsequent packet (if any) belonging to the same unit. Practically, if codestream packetization mode is enabled, this field counts the packets within a JPEG XS picture segment and is extended by the SEP counter when it overruns. If slice packetization mode is enabled, this field counts the packets within a slice or within the JPEG XS header segment.

Payload Data The payload data of a JPEG XS RTP stream consists of a concatenation of multiple JPEG XS frames, each consisting of one (for progressive video) or two (for interlaced video) JPEG XS picture segments. Within the RTP stream, all of the Video Support boxes and all of the Color Specification boxes SHALL retain their respective layouts for each JPEG XS picture segment across all JPEG XS frames, for the entirety of the JPEG XS RTP stream. Thus, each Video Support box in the RTP stream SHALL define the same sub boxes, in the same order. The effective values in the boxes are allowed to change under the condition that their relative byte offsets SHALL NOT change. Moreover, any changed value in the boxes SHALL NOT violate any restrictions imposed by the application layer. Each JPEG XS frame is represented by one or more packetization unit(s), as explained in . depicts this layout for a progressive video frame in the codestream packetization mode, depicts this layout for an interlaced video frame in the codestream packetization mode, depicts this layout for a progressive video frame in the slice packetization mode, and depicts this layout for an interlaced video frame in the slice packetization mode. The Frame (F) counter value is not indicated because the value is constant for all packetization units of a given video frame.

Traffic Shaping and Delivery Timing In order to facilitate proper synchronization between senders and receivers, it is RECOMMENDED to implement traffic shaping and delivery timing in accordance with the Network Compatibility Model Compliance Definitions specified in . In such a case, the session description SHALL signal the conformance with the media type parameter TP as described in . The actual applied traffic shaping and timing delivery mechanism is outside the scope of this document and does not influence the payload packetization.

Congestion Control Considerations Congestion control for RTP SHALL be used in accordance with and with any applicable RTP profile, e.g., RTP/AVP or RTP/AVPF . While JPEG XS is mainly designed to be used in controlled network environments, it can also be employed in best-effort network environments, like the Internet. However, if best-effort service is being used, users of this payload format SHALL monitor packet loss to ensure that the packet loss rate is within acceptable parameters. Packet loss is considered acceptable if a TCP flow across the same network path, and experiencing the same network conditions, would achieve an average throughput, measured on a reasonable timescale, that is not less than the RTP flow is achieving. This condition can be satisfied by implementing congestion control mechanisms, for example by means of RTP Control Protocol (RTCP) Feedback for Congestion Control , to allow adapting the transmission rate (or the number of layers subscribed for a layered multicast session) or by arranging for a receiver to leave the session if the loss rate is unacceptably high. Alternatively, this payload format may also be used in networks that provide quality-of-service guarantees. If enhanced service is being used, receivers SHOULD monitor packet loss to ensure that the service that was requested is actually being delivered. If enhanced service is not used, then users of this payload SHOULD assume a best-effort service and behave accordingly. In addition, is an update to that defines criteria for when one is required to stop sending RTP Packet Streams and which can be used for relevant applications. Finally, provides additional information on the best practices for applying congestion control to UDP streams.

Payload Format Parameters This section specifies the required and optional parameters of the payload format and/or the RTP stream. With the exception of the rate parameter, all parameters are declarative, meaning that the information signaled by the parameters is also present in the payload data, namely in the payload header (see ) or in the JPEG XS header segment. When provided, their respective values SHALL be consistent with the payload.

Media Type Registration This registration is done using the template defined in and following . The receiver SHALL ignore any unrecognized parameter.

Type name:

video

Subtype name:

jxsv

Required parameters:

rate:

The RTP timestamp clock rate. Applications using this payload format SHALL use a value of 90000.

packetmode:

This parameter specifies the configured packetization mode as defined by the pacKetization mode (K) bit in the payload header of . This value SHALL be equal to the K-bit value configured in the RTP stream (i.e., 0 for codestream or 1 for slice).

Optional parameters:

transmode:

This parameter specifies the configured transmission mode as defined by the Transmission mode (T) bit in the payload header of . If specified, this value SHALL be equal to the T-bit value configured in the RTP stream (i.e., 0 for out-of-order-allowed or 1 for sequential-only). If not specified, a value 1 (sequential-only) SHALL be assumed and the T bit SHALL be set to 1.

profile:

The JPEG XS profile in use, as specified in . Any white space Unicode character in the profile name SHALL be omitted. Examples of valid profile names include 'Main444.12', 'High444.12', 'CHigh444.12', and 'TDC444.12'. When present, this parameter SHALL identify the same JPEG XS profile as the one signaled in the JPEG XS picture segment.

level:

The JPEG XS level in use, as specified in . Any white space Unicode character in the level name SHALL be omitted. Examples of valid levels are '2k-1' or '4k-2'. When present, this parameter SHALL identify the same JPEG XS level as the one signaled in the JPEG XS picture segment.

sublevel:

The JPEG XS sublevel in use, as specified in . Any white space Unicode character in the sublevel name SHALL be omitted. Examples of valid sublevels are 'Sublev3bpp' or 'Sublev6bpp'. When present, this parameter SHALL identify the same JPEG XS sublevel as the one signaled in the JPEG XS picture segment.

fbblevel:

The JPEG XS frame buffer bandwidth (FBB) level in use, as specified in , when using the TDC coding mode. Any white space Unicode character in the fbblevel name SHALL be omitted. Examples of valid frame buffer bandwidth levels are 'Fbblev3bpp' or 'Fbblev12bpp'. This parameter SHALL only be present when the TDC coding mode is used. When present, this parameter SHALL identify the same JPEG XS fbblevel as the one signaled in the JPEG XS picture segment.

depth:

Determines the number of bits per sample. This is an integer with typical values including 8, 10, 12, and 16. When present, this parameter SHALL match the information signaled in the JPEG XS picture segment.

width:

Determines the number of pixels per line. This is an integer between 1 and 32767, inclusive. When present, this parameter SHALL match the information signaled in the JPEG XS picture segment.

height:

Determines the number of lines per video frame. This is an integer between 1 and 32767, inclusive. For interlaced and PsF video streams, this parameter signals the number of lines of the full video frame, which is twice the height signaled in each JPEG XS codestream header. When present, this parameter SHALL be consistent with the information signaled in the JPEG XS picture segment.

exactframerate:

Signals the video frame rate in frames per second. Integer frame rates SHALL be signaled as a single decimal number (e.g., "25") whilst non-integer frame rates SHALL be signaled as a ratio of two integer decimal numbers separated by a "forward-slash" character (e.g., "30000/1001"), utilizing the numerically smallest numerator value possible. When present, this parameter SHALL identify the same frame rate as the one signaled in the JPEG XS picture segment.

interlace:

If this parameter name is present, it indicates that the video stream is interlaced or Progressive segmented Frame (PsF) (see ). If the segmented parameter name is also present, the video stream is Progressive segmented Frame (PsF); otherwise, it is interlaced. If the interlace parameter name is not present, the video stream SHALL be assumed to be progressive.

segmented:

If this parameter name is present, and the interlace parameter name is also present, then the video stream is Progressive segmented Frame (PsF). Signaling of this parameter name without the interlace parameter name is forbidden.

sampling:

Specifies the color difference signal subsampling structure. Valid values and their specification are the following: For signals utilizing the non-constant luminance Y'C'B C'R signal format of , , , or :

YCbCr-4:4:4

(4:4:4 sampling)

YCbCr-4:2:2

(4:2:2 sampling)

YCbCr-4:2:0

(4:2:0 sampling)

For signals utilizing the constant luminance Y'C C'BC C'RC signal format of :

CLYCbCr-4:4:4

(4:4:4 sampling)

CLYCbCr-4:2:2

(4:2:2 sampling)

CLYCbCr-4:2:0

(4:2:0 sampling)

For signals utilizing the constant intensity I CT CP signal format of :

ICtCp-4:4:4

(4:4:4 sampling)

ICtCp-4:2:2

(4:2:2 sampling)

ICtCp-4:2:0

(4:2:0 sampling)

Signals utilizing the 4:4:4 R' G' B' or RGB signal format (such as that of , , , , , or ) SHALL use the following value for the Media Type Parameter "sampling":

RGB

(RGB or R' G' B' samples)

For signals utilizing the 4:4:4 X' Y' Z' signal format (such as defined in ):

XYZ

(X' Y' Z' samples)

For single-component key signals (such as defined in ):

KEY

(Samples of the key signal)

Signals utilizing a color subsampling other than what is defined here SHALL use the following value for the Media Type Parameter "sampling":

UNSPECIFIED

(Sampling signaled by the payload)

colorimetry:

Specifies the system colorimetry used by the video samples. Valid values and their specification are the following:

BT601-5:

.

BT709-2:

.

SMPTE240M:

.

BT601:

.

BT709:

.

BT2020:

.

BT2100:

, Table 2 titled "System colorimetry".

ST2065-1:

Academy Color Encoding Specification (ACES) .

ST2065-3:

Academy Density Exchange Encoding (ADX) .

XYZ:

, section titled "1931 Observer".

UNSPECIFIED:

Colorimetry SHALL either be signaled in the payload by the Color Specification box of , or be manually coordinated between sender and receiver.

Signals utilizing the colorimetry SHOULD also signal the representational range using the optional parameter RANGE defined below. Signals utilizing the UNSPECIFIED colorimetry might require manual coordination between the sender and the receiver.

TCS:

Transfer Characteristic System. This parameter specifies the transfer characteristic system of the video samples. Valid values and their specification are the following:

SDR:

Standard Dynamic Range video streams that utilize the Optical Electrical Transfer Function (OETF) of or . Such streams SHALL be assumed to target the Electro-Optical Transfer Function (EOTF) specified in .

PQ:

High dynamic range video streams that utilize the Perceptual Quantization system of .

HLG:

High dynamic range video streams that utilize the Hybrid Log-Gamma system of .

UNSPECIFIED:

Video streams whose transfer characteristics SHALL either be signaled by the payload as specified in , or be manually coordinated between sender and receiver.

RANGE:

This parameter SHOULD be used to signal the encoding range of the sample values within the stream. When paired with colorimetry, this parameter has two allowed values, NARROW and FULL, corresponding to the ranges specified in Table 9 of . In any other context, this parameter has three allowed values: NARROW, FULLPROTECT, and FULL, which correspond to the ranges specified in . In the absence of this parameter, and for all but the UNSPECIFIED colorimetry, NARROW SHALL be the assumed value. When paired with the UNSPECIFIED colorimetry, FULL SHALL be the assumed value.

Encoding considerations:

This media type is framed in RTP and contains binary data; see Section 4.8 of .

Security considerations:

See the Security Considerations section of &SELF;.

Interoperability considerations:

None

Published specification:

See the References section of &SELF;.

Applications that use this media type:

Any application that transmits video over RTP (like SMPTE ST 2110).

Fragment identifier considerations:

N/A

Additional information:

None

Person & email address to contact for further information:

T. Bruylants rtp@intopix.com and T. Richter jpeg-xs-techsupport@iis.fraunhofer.de.

Intended usage:

COMMON

Restrictions on usage:

This media type depends on RTP framing; hence, it is only defined for transfer via RTP .

Author:

See the Authors' Addresses section of &SELF;.

Change controller:

IETF Audio/Video Transport Working Group delegated from the IESG.

SDP Parameters A mapping of the parameters into the Session Description Protocol (SDP) is provided for applications that use SDP.

Mapping of Payload Type Parameters to SDP The media type video/jxsv string is mapped to fields in the Session Description Protocol (SDP) as follows: The media type ("video") goes in SDP "m=" as the media name. The media subtype ("jxsv") goes in SDP "a=rtpmap" as the encoding name, followed by a slash ("/") and the required parameter "rate" corresponding to the RTP timestamp clock rate (which for the payload format defined in this document SHALL be 90000). The required parameter "packetmode" and any of the additional optional parameters, as described in , go in the SDP media format description, being the "a=fmtp" attribute (Format Parameters), by copying them directly from the media type string as a semicolon-separated list of parameter=value pairs. All parameters of the media format SHALL correspond to the parameters of the payload. In case of discrepancies between payload parameter values and SDP fields, the values from the payload data SHALL prevail. The receiver SHALL ignore any parameter that is neither defined in nor defined by another specification applicable to the session, such as the TP parameter of (see ). An example SDP mapping for JPEG XS video is as follows:

In this example, a JPEG XS RTP stream is to be sent to UDP destination port 30000, with an RTP dynamic payload type of 112 and a media clock rate of 90000 Hz. Note that the "a=fmtp:" line has been wrapped to fit this page and will be a single long line in the SDP file. This example includes the TP parameter (as specified in ).

Usage with SDP Offer/Answer Model When JPEG XS is offered over RTP using SDP in an offer/answer model for negotiation for unicast usage, the following limitations and rules apply: The "a=fmtp" attribute SHALL be present specifying the required parameter "packetmode" and MAY specify any of the optional parameters, as described in . All parameters in the "a=fmtp" attribute indicate sending capabilities (i.e., properties of the payload). An answerer of the SDP is required to support all parameters and values of the parameters provided by the offerer; otherwise, the answerer SHALL reject the session. It falls on the offerer to use values that are expected to be supported by the answerer. If the answerer accepts the session, it SHALL reply with the exact same parameter values in the "a=fmtp" attribute as they were initially offered. The same RTP payload type number used in the offer SHOULD be used in the answer, as specified in .

IANA Considerations Because this document obsoletes , IANA is asked to change all registration information that references to instead reference &SELF;. IANA is asked to update the media type registration "video/jxsv" as specified in (see https://www.iana.org/assignments/media-types).

Security Considerations RTP packets using the payload format defined in this document are subject to the security considerations of and any applicable RTP profile, such as RTP/AVP , RTP/AVPF , RTP/SAVP , or RTP/SAVPF . Thus, confidentiality of the media streams is achieved through encryption. As discussed in "Securing the RTP Framework: Why RTP Does Not Mandate a Single Media Security Solution" , RTP payload formats neither specify nor mandate the mechanisms used to provide confidentiality, integrity, and source authentication. The responsibility for selecting and deploying appropriate security mechanisms rests with the applications using RTP. Guidance on available mechanisms can be found in "Options for Securing RTP Sessions" , although parts of that document are now dated. Applications SHOULD use current, well-maintained security mechanisms. Examples include: (D)TLS-based protection: provides current recommendations for the use of TLS 1.3 and DTLS. IPsec-based protection: See for Encapsulating Security Payload (ESP) and for Internet Key Exchange Protocol Version 2 (IKEv2). This document does not mandate a specific security mechanism. Applications are responsible for selecting security solutions that provide confidentiality, integrity, and source authentication in accordance with current best practices. Implementations of this RTP payload format SHALL ensure that the decoder is robust against malicious or malformed payloads, so that such payloads do not cause the decoder to overrun its allocated memory or otherwise misbehave. An overrun in allocated memory could lead to arbitrary code execution by an attacker. The same applies to the encoder, even though problems in encoders are typically rarer. This payload format and the JPEG XS encoding do not exhibit any substantial non-uniformity, either in output or in complexity to perform the decoding operation; thus, they are unlikely to pose a denial-of-service threat due to the receipt of pathological datagrams. This payload format and the JPEG XS encoding do not contain code that is executable. It is important to note that high-definition (HD) or ultra-high-definition (UHD) video that is encoded with JPEG XS can have significant bandwidth requirements (typically more than 1 Gbps for UHD video, especially if using high framerate). This is sufficient to cause potential for denial of service if transmitted onto most currently available Internet paths.

Operational Considerations This document revises and obsoletes and has been designed to minimize operational impact on existing deployments. The RTP packetization, header formats, and overall payload structure remain unchanged compared to . As a result, existing implementations conforming to remain compatible with this specification and continue to interoperate without modification when operating in modes defined by . The primary changes introduced by this document are limited to support for new features defined in the third edition of the JPEG XS standard (, , and ), most notably the Temporal Differential Coding (TDC) mode. These extensions are designed to be backward-compatible and do not affect the decoding of non-TDC streams. In particular: For TDC profiles, relies on a specific slice header marker called SLI, in addition to the original SLH marker. The SLI marker indicates that the slice encodes TDC-enabled content. This distinction is not directly relevant to this specification, and for the purposes of this RFC, both the SLH and SLI markers serve the same function: to define the boundaries of packetization units when using the Slice Packetization mode, as described in . Yet, this document was updated to reflect the possibility of using either SLH or SLI markers. In addition to the level and sublevel, the TDC coding mode introduces an fbblevel in that needs to be supported as an optional payload parameter. A new parameter for signaling the fbblevel is defined in . This document now provides more clarifications and improved descriptions for correctly handling interlaced video. provides a more detailed definition of a Slice to clarify that this RTP payload format supports the optional slice-based extension marker functionality defined in . The erratum of concerning the RTP timestamp for interlaced video signals has been incorporated into this specification. As a consequence, deployment of this updated specification does not require an explicit or coordinated upgrade, and can be introduced incrementally. Implementations that support the new features can negotiate their use via existing SDP mechanisms, while legacy implementations remain fully operational within their supported feature set. Overall, the design of this specification ensures a straightforward upgrade path, preserves interoperability with existing systems, and does not introduce additional operational complexity for configuration, debugging, or network management.

RFC Editor Considerations Note to RFC Editor: This section may be removed after carrying out all the instructions of this section. &SELF; is to be replaced by the RFC number this specification receives when published.

This document defines a mechanism by which two entities can make use of the Session Description Protocol (SDP) to arrive at a common view of a multimedia session between them. In the model, one participant offers the other a description of the desired session from their perspective, and the other participant answers with the desired session from their perspective. This offer/answer model is most useful in unicast sessions where information from both participants is needed for the complete view of the session. The offer/answer model is used by protocols like the Session Initiation Protocol (SIP). [STANDARDS-TRACK] This memorandum describes RTP, the real-time transport protocol. RTP provides end-to-end network transport functions suitable for applications transmitting real-time data, such as audio, video or simulation data, over multicast or unicast network services. RTP does not address resource reservation and does not guarantee quality-of- service for real-time services. The data transport is augmented by a control protocol (RTCP) to allow monitoring of the data delivery in a manner scalable to large multicast networks, and to provide minimal control and identification functionality. RTP and RTCP are designed to be independent of the underlying transport and network layers. The protocol supports the use of RTP-level translators and mixers. Most of the text in this memorandum is identical to RFC 1889 which it obsoletes. There are no changes in the packet formats on the wire, only changes to the rules and algorithms governing how the protocol is used. The biggest change is an enhancement to the scalable timer algorithm for calculating when to send RTCP packets in order to minimize transmission in excess of the intended rate when many participants join a session simultaneously. [STANDARDS-TRACK] This document describes a profile called "RTP/AVP" for the use of the real-time transport protocol (RTP), version 2, and the associated control protocol, RTCP, within audio and video multiparticipant conferences with minimal control. It provides interpretations of generic fields within the RTP specification suitable for audio and video conferences. In particular, this document defines a set of default mappings from payload type numbers to encodings. This document also describes how audio and video data may be carried within RTP. It defines a set of standard encodings and their names when used within RTP. The descriptions provide pointers to reference implementations and the detailed standards. This document is meant as an aid for implementors of audio, video and other real-time multimedia applications. This memorandum obsoletes RFC 1890. It is mostly backwards-compatible except for functions removed because two interoperable implementations were not found. The additions to RFC 1890 codify existing practice in the use of payload formats under this profile and include new payload formats defined since RFC 1890 was published. [STANDARDS-TRACK] This document specifies the procedure to register RTP payload formats as audio, video, or other media subtype names. This is useful in a text-based format description or control protocol to identify the type of an RTP transmission. [STANDARDS-TRACK] This document defines procedures for the specification and registration of media types for use in HTTP, MIME, and other Internet protocols. This memo documents an Internet Best Current Practice. This memo defines the Session Description Protocol (SDP). SDP is intended for describing multimedia sessions for the purposes of session announcement, session invitation, and other forms of multimedia session initiation. This document obsoletes RFC 4566. ISO/IEC ISO/IEC ISO/IEC SMPTE In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. RFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings. This document describes the Secure Real-time Transport Protocol (SRTP), a profile of the Real-time Transport Protocol (RTP), which can provide confidentiality, message authentication, and replay protection to the RTP traffic and to the control traffic for RTP, the Real-time Transport Control Protocol (RTCP). [STANDARDS-TRACK] This memo specifies a packetization scheme for encapsulating uncompressed video into a payload format for the Real-time Transport Protocol, RTP. It supports a range of standard- and high-definition video formats, including common television formats such as ITU BT.601, and standards from the Society of Motion Picture and Television Engineers (SMPTE), such as SMPTE 274M and SMPTE 296M. The format is designed to be applicable and extensible to new video formats as they are developed. [STANDARDS-TRACK] This document describes an updated version of the Encapsulating Security Payload (ESP) protocol, which is designed to provide a mix of security services in IPv4 and IPv6. ESP is used to provide confidentiality, data origin authentication, connectionless integrity, an anti-replay service (a form of partial sequence integrity), and limited traffic flow confidentiality. This document obsoletes RFC 2406 (November 1998). [STANDARDS-TRACK] Real-time media streams that use RTP are, to some degree, resilient against packet losses. Receivers may use the base mechanisms of the Real-time Transport Control Protocol (RTCP) to report packet reception statistics and thus allow a sender to adapt its transmission behavior in the mid-term. This is the sole means for feedback and feedback-based error repair (besides a few codec-specific mechanisms). This document defines an extension to the Audio-visual Profile (AVP) that enables receivers to provide, statistically, more immediate feedback to the senders and thus allows for short-term adaptation and efficient feedback-based repair mechanisms to be implemented. This early feedback profile (AVPF) maintains the AVP bandwidth constraints for RTCP and preserves scalability to large groups. [STANDARDS-TRACK] An RTP profile (SAVP) for secure real-time communications and another profile (AVPF) to provide timely feedback from the receivers to a sender are defined in RFC 3711 and RFC 4585, respectively. This memo specifies the combination of both profiles to enable secure RTP communications with feedback. [STANDARDS-TRACK] The Real-time Transport Protocol (RTP) is used in a large number of different application domains and environments. This heterogeneity implies that different security mechanisms are needed to provide services such as confidentiality, integrity, and source authentication of RTP and RTP Control Protocol (RTCP) packets suitable for the various environments. The range of solutions makes it difficult for RTP-based application developers to pick the most suitable mechanism. This document provides an overview of a number of security solutions for RTP and gives guidance for developers on how to choose the appropriate security mechanism. This memo discusses the problem of securing real-time multimedia sessions. It also explains why the Real-time Transport Protocol (RTP) and the associated RTP Control Protocol (RTCP) do not mandate a single media security mechanism. This is relevant for designers and reviewers of future RTP extensions to ensure that appropriate security mechanisms are mandated and that any such mechanisms are specified in a manner that conforms with the RTP architecture. This document describes version 2 of the Internet Key Exchange (IKE) protocol. IKE is a component of IPsec used for performing mutual authentication and establishing and maintaining Security Associations (SAs). This document obsoletes RFC 5996, and includes all of the errata for it. It advances IKEv2 to be an Internet Standard. The Real-time Transport Protocol (RTP) is widely used in telephony, video conferencing, and telepresence applications. Such applications are often run on best-effort UDP/IP networks. If congestion control is not implemented in these applications, then network congestion can lead to uncontrolled packet loss and a resulting deterioration of the user's multimedia experience. The congestion control algorithm acts as a safety measure by stopping RTP flows from using excessive resources and protecting the network from overload. At the time of this writing, however, while there are several proprietary solutions, there is no standard algorithm for congestion control of interactive RTP flows. This document does not propose a congestion control algorithm. It instead defines a minimal set of RTP circuit breakers: conditions under which an RTP sender needs to stop transmitting media data to protect the network from excessive congestion. It is expected that, in the absence of long-lived excessive congestion, RTP applications running on best-effort IP networks will be able to operate without triggering these circuit breakers. To avoid triggering the RTP circuit breaker, any Standards Track congestion control algorithms defined for RTP will need to operate within the envelope set by these RTP circuit breaker algorithms. The User Datagram Protocol (UDP) provides a minimal message-passing transport that has no inherent congestion control mechanisms. This document provides guidelines on the use of UDP for the designers of applications, tunnels, and other protocols that use UDP. Congestion control guidelines are a primary focus, but the document also provides guidance on other topics, including message sizes, reliability, checksums, middlebox traversal, the use of Explicit Congestion Notification (ECN), Differentiated Services Code Points (DSCPs), and ports. Because congestion control is critical to the stable operation of the Internet, applications and other protocols that choose to use UDP as an Internet transport must employ mechanisms to prevent congestion collapse and to establish some degree of fairness with concurrent traffic. They may also need to implement additional mechanisms, depending on how they use UDP. Some guidance is also applicable to the design of other protocols (e.g., protocols layered directly on IP or via IP-based tunnels), especially when these protocols do not themselves provide congestion control. This document obsoletes RFC 5405 and adds guidelines for multicast UDP usage. An effective RTP congestion control algorithm requires more fine-grained feedback on packet loss, timing, and Explicit Congestion Notification (ECN) marks than is provided by the standard RTP Control Protocol (RTCP) Sender Report (SR) and Receiver Report (RR) packets. This document describes an RTCP feedback message intended to enable congestion control for interactive real-time traffic using RTP. The feedback message is designed for use with a sender-based congestion control algorithm, in which the receiver of an RTP flow sends back to the sender RTCP feedback packets containing the information the sender needs to perform congestion control. This document specifies a Real-Time Transport Protocol (RTP) payload format to be used for transporting video encoded with JPEG XS (ISO/IEC 21122). JPEG XS is a low-latency, lightweight image coding system. Compared to an uncompressed video use case, it allows higher resolutions and video frame rates while offering visually lossless quality, reduced power consumption, and encoding-decoding latency confined to a fraction of a video frame. Transport Layer Security (TLS) and Datagram Transport Layer Security (DTLS) are used to protect data exchanged over a wide range of application protocols and can also form the basis for secure transport protocols. Over the years, the industry has witnessed several serious attacks on TLS and DTLS, including attacks on the most commonly used cipher suites and their modes of operation. This document provides the latest recommendations for ensuring the security of deployed services that use TLS and DTLS. These recommendations are applicable to the majority of use cases. RFC 7525, an earlier version of the TLS recommendations, was published when the industry was transitioning to TLS 1.2. Years later, this transition is largely complete, and TLS 1.3 is widely available. This document updates the guidance given the new environment and obsoletes RFC 7525. In addition, this document updates RFCs 5288 and 6066 in view of recent attacks. ITU-R ITU-R ITU-R ITU-R ITU-R ITU-R ITU-R ISO/CIE SMPTE SMPTE SMPTE SMPTE SMPTE SMPTE

Acknowledgements This document is a revision of . As such the authors would like to thank the following people for their valuable contributions that made and this document possible: Siegfried Foessel, Arnaud Germain, Jean-Baptiste Lorent, Sébastien Lugan, Gaël Rouvroy, and Alexandre Willème.