<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE rfc [
  <!ENTITY nbsp "&#160;">
  <!ENTITY zwsp "&#8203;">
  <!ENTITY nbhy "&#8209;">
  <!ENTITY wj "&#8288;">
]>
<rfc xmlns:xi="http://www.w3.org/2001/XInclude"
     docName="draft-msri-bpm-bmp-convergence-benchmarking-00"
     category="info"
     ipr="trust200902"
     submissionType="IETF"
     consensus="true"
     version="3"
     xml:lang="en"
     tocInclude="true"
     tocDepth="4"
     symRefs="true"
     sortRefs="true">

  <front>
    <title abbrev="BMP Convergence Benchmarking">Benchmarking Methodology for BGP Monitoring Protocol (BMP) Convergence and Scaling</title>
    <seriesInfo name="Internet-Draft" value="draft-msri-bpm-bmp-convergence-benchmarking-00"/>

    <author fullname="Mukul Srivastava" initials="M." surname="Srivastava">
      <organization>Hewlett Packard Enterprise</organization>
      <address>
        <postal>
          <street>10 Technology Park Dr</street>
          <city>Westford</city>
          <region>MA</region>
          <code>01886</code>
          <country>United States of America</country>
        </postal>
        <email>mukul.srivastava@hpe.com</email>
      </address>
    </author>

    <date year="2026" month="November" day="16"/>

    <area>Operations and Management</area>
    <workgroup>Network Working Group</workgroup>

    <keyword>BMP</keyword>
    <keyword>BGP</keyword>
    <keyword>benchmarking</keyword>
    <keyword>convergence</keyword>

    <abstract>
      <t>The BGP Monitoring Protocol (BMP) is widely used by operators to
      stream BGP routing state from routers to monitoring and automation
      systems.  Although BGP convergence has well-established benchmarking
      methodologies, there is no standardized way to measure how quickly and
      how completely a router exports BGP state over BMP, or how the BMP
      export affects the router and the collector.  This document defines
      terminology, a reference test setup, metrics, and test procedures to
      benchmark the BMP convergence of a router (the Device Under Test)
      in a vendor-independent and repeatable way.  It covers all five BMP
      monitoring views (Adj-RIB-In pre-policy and post-policy, Loc-RIB,
      and Adj-RIB-Out pre-policy and post-policy), initial table dump speed,
      convergence during routing events and churn, impact on BGP itself, scaling with the number of
      monitored views, the Adj-RIB-Out scale, and BMP stations, and behavior under TCP backpressure.</t>
    </abstract>
  </front>

  <middle>

    <!-- ===================================================== -->
    <section anchor="sec-intro" numbered="true">
      <name>Introduction</name>

      <t>The BGP Monitoring Protocol (BMP) <xref target="RFC7854"/> allows a
      monitored router to stream its BGP state, over a TCP connection, to
      one or more BMP stations.  The base protocol has been extended to
      cover the Adj-RIB-Out <xref target="RFC8671"/> and the Loc-RIB
      <xref target="RFC9069"/>, and additional statistics have been defined
      <xref target="RFC9972"/>.  Operators use the resulting feed for
      troubleshooting, anomaly detection, auditing, and increasingly as
      real-time input to automated traffic engineering and path computation
      systems.</t>

      <t>Benchmarking methodologies exist for BGP control-plane convergence
      <xref target="RFC4098"/> and BGP data-plane convergence
      <xref target="RFC7747"/>.  These documents measure how fast a router
      processes routing changes and updates its forwarding state.  They say
      nothing about how fast the same router tells a BMP station about those
      changes.  BMP is an observer of BGP, but it is also
      a consumer of the router's control-plane resources and, through TCP
      backpressure, a potential source of interference with BGP itself.</t>

      <section anchor="sec-motivation" numbered="true">
        <name>Motivation</name>

        <t>No published standard describes how to benchmark the performance
        or scale of a BMP implementation.  The IETF has standardized the
        BMP data formats and state reporting, but the performance of BMP
        implementations across vendors is not characterized in a uniform
        way.  In practice, operators and vendors each build their own
        private test plans, which makes results incomparable.</t>

        <t>Telemetry protocols have historically been treated as secondary
        management-plane functions rather than core routing functions.  That
        view no longer holds.  Automated systems now make routing and
        traffic engineering decisions based on BMP data, and the usefulness of those decisions depends on how
        fresh and how complete the BMP feed is.  Operators therefore need
        answers to questions such as:</t>

        <ul spacing="normal">
          <li>After a router reboots or a BMP session is (re)established, how
          long until a collector has a complete view of the router's tables?</li>
          <li>During a routing event or a routing churn period, how far
          behind the router's real BGP state is the BMP feed?</li>
          <li>Does enabling BMP, or enabling more BMP monitoring views or more
          BMP stations, slow down BGP convergence or exhaust control-plane resources?</li>
          <li>What happens to the router when a BMP collector cannot keep up
          and the TCP connection applies backpressure?</li>
          <li>How fast can a collector itself absorb the feed when it does real work
          with it, such as storing it in a database?</li>
        </ul>

        <t>A common methodology allows these questions to be answered
        consistently, and allows different implementations to be compared
        under the same conditions.</t>
      </section>

      <section anchor="sec-scope" numbered="true">
        <name>Scope</name>

        <t>This document defines a methodology for benchmarking, in a
        laboratory environment, the following aspects of a BMP-capable
        router acting as the Device Under Test (DUT):</t>

        <ul spacing="normal">
          <li>Initial dump speed for each BMP monitoring view.</li>
          <li>BMP convergence after BGP events, and during churn.</li>
          <li>Impact of BMP on BGP convergence and on control-plane resources.</li>
          <li>Scaling with the number of enabled monitoring views, the
          Adj-RIB-Out scale (number of peers advertised to), and the
          number of BMP stations.</li>
          <li>Behavior with a slow or stalled BMP station (TCP backpressure).</li>
        </ul>

        <t>The following are out of scope: the correctness of the BMP
        encoding, the BMP Route Mirroring function, the benchmarking of
        BMP collector products as such (except as needed to isolate the DUT
        from collector effects), the accuracy of router-internal clocks, and
        data-plane convergence.  The methodology is intended for
        controlled laboratory use only.</t>
      </section>

      <section anchor="sec-reqlang" numbered="true">
        <name>Requirements Language</name>
        <t>The key words "<bcp14>MUST</bcp14>", "<bcp14>MUST NOT</bcp14>",
        "<bcp14>REQUIRED</bcp14>", "<bcp14>SHALL</bcp14>",
        "<bcp14>SHALL NOT</bcp14>", "<bcp14>SHOULD</bcp14>",
        "<bcp14>SHOULD NOT</bcp14>", "<bcp14>RECOMMENDED</bcp14>",
        "<bcp14>NOT RECOMMENDED</bcp14>", "<bcp14>MAY</bcp14>", and
        "<bcp14>OPTIONAL</bcp14>" in this document are to be interpreted as
        described in BCP 14 <xref target="RFC2119"/> <xref target="RFC8174"/>
        when, and only when, they appear in all capitals, as shown here.</t>
      </section>
    </section>

    <!-- ===================================================== -->
    <section anchor="sec-terminology" numbered="true">
      <name>Terminology</name>

      <section anchor="sec-existing-terms" numbered="true">
        <name>Terms Defined Elsewhere</name>

        <t>This document does not redefine existing terms.  The following
        terms are used as defined in the referenced documents.</t>

        <ul spacing="normal">
          <li>Monitored Router, BMP Station, Initiation Message, Peer Up
          Notification, Peer Down Notification, Route Monitoring Message,
          Statistics Report, Termination Message, Per-Peer Header, and
          Adj-RIB-In (pre-policy and post-policy): <xref target="RFC7854"/>.
          The terms "producer" and "collector" are used in some documents as
          synonyms for "Monitored Router" and "BMP Station"
          <xref target="RFC9972"/>; this document uses "BMP Station".</li>
          <li>Adj-RIB-Out (pre-policy and post-policy): <xref target="RFC8671"/>.</li>
          <li>Loc-RIB and the Loc-RIB Instance Peer Type: <xref target="RFC4271"/>
          and <xref target="RFC9069"/>.</li>
          <li>End-of-RIB marker: <xref target="RFC4724"/>.</li>
          <li>Route packing, route mixture, minimal policy, and BGP
          control-plane convergence: <xref target="RFC4098"/>.</li>
          <li>Device Under Test (DUT) and benchmarking terminology in general:
          <xref target="RFC1242"/>, <xref target="RFC2544"/>, and
          <xref target="RFC7747"/>.</li>
        </ul>
      </section>

      <section anchor="sec-new-terms" numbered="true">
        <name>Terms Defined in This Document</name>

        <dl newline="true" spacing="normal">
          <dt>Monitoring View (View):</dt>
          <dd>One of the five BMP-exportable RIB views.  The abbreviations in
          <xref target="tbl-views"/> are used throughout this document.</dd>

          <dt>Peering Router (PR):</dt>
          <dd>A router, or a tester emulating a router, that establishes BGP
          sessions with the DUT and generates or receives BGP routes.</dd>

          <dt>Test Controller:</dt>
          <dd>The entity that orchestrates the test, provides the reference
          clock, triggers events, and records measurements.</dd>

          <dt>Sink Station:</dt>
          <dd>A BMP Station that accepts the TCP connection, acknowledges all
          data promptly and discards the content, without doing any
          processing that could create backpressure (see
          <xref target="sec-station-types"/>).</dd>

          <dt>Processing Station:</dt>
          <dd>A BMP Station that performs real collector work with the feed,
          for example parsing it fully and storing it in a database or
          forwarding it to a message bus.</dd>

          <dt>Trigger:</dt>
          <dd>The stimulus that starts a measurement, for example the start of
          a BMP session, an UPDATE burst, a BGP session reset, or a link
          failure.</dd>

          <dt>Reference State:</dt>
          <dd>For a given View and a given moment, the exact set of routes (and
          attributes, as relevant) that the DUT is expected to export through
          BMP.  Because the Test Controller generates all BGP stimuli, it
          knows the Reference State without needing to query the DUT.</dd>

          <dt>View State:</dt>
          <dd>The state of a View as reconstructed by a BMP Station by
          applying the received Route Monitoring and Peer Down messages.</dd>

          <dt>BMP Convergence:</dt>
          <dd>The condition in which the View State of a given View, for all
          monitored peers and AFI/SAFIs, is equal to the Reference State
          following a Trigger and remains equal until the next Trigger.</dd>

          <dt>BMP Convergence Time (BCT):</dt>
          <dd>The time from a Trigger until BMP Convergence is reached for a given View.</dd>

          <dt>BMP Initial Dump:</dt>
          <dd>The set of Route Monitoring messages that a DUT generates to
          report the existing content of a View after a BMP session is
          established (or a View is enabled on an established session).</dd>

          <dt>BMP Initial Dump Time (BIDT):</dt>
          <dd>The time from the start of the BMP Initial Dump until BMP
          Convergence for that View (see <xref target="sec-metrics"/>).</dd>

          <dt>BMP Event Latency (BEL):</dt>
          <dd>For a single event, the time from the moment the event is
          injected by the Test Controller until the BMP message that reports
          that event is received by the BMP Station.  An event is either a
          route event (announcement, withdrawal, attribute change) or a BGP
          session event (session down, session up).</dd>

          <dt>Quiescence Interval (QI):</dt>
          <dd>A configured period without any BMP message from the DUT, used
          by some completion-detection methods.</dd>

          <dt>BGP Propagation Time (BPT):</dt>
          <dd>The time for a routing event injected at one Peering Router to be
          re-advertised by the DUT and observed at another Peering Router.  This is the
          control-plane convergence metric used as the reference for measuring
          the impact of BMP on BGP (see <xref target="RFC4098"/>).</dd>

          <dt>Loss-Free Churn Rate (LFCR):</dt>
          <dd>The highest sustained BGP event rate at which BMP Convergence
          is reached, with no BMP message loss, no BMP session reset, and a
          bounded BEL.</dd>

          <dt>Receive-Only Peer:</dt>
          <dd>A Peering Router that has a BGP session with the DUT and receives
          the DUT's advertisements, but advertises no routes to the DUT.  It adds to
          the Adj-RIB-Out scale without changing the Adj-RIB-In or Loc-RIB scale.</dd>

          <dt>BMP Fan-Out:</dt>
          <dd>For a single routing event and a given View, the number of BMP
          messages that the DUT generates to report the event (see
          <xref target="sec-fanout"/>).</dd>
        </dl>

        <table anchor="tbl-views" align="center">
          <name>Monitoring Views and Abbreviations</name>
          <thead>
            <tr><th>Abbreviation</th><th>Monitoring View</th><th>Defined in</th></tr>
          </thead>
          <tbody>
            <tr><td>IN-PRE</td><td>Adj-RIB-In, pre-policy</td><td><xref target="RFC7854"/></td></tr>
            <tr><td>IN-POST</td><td>Adj-RIB-In, post-policy</td><td><xref target="RFC7854"/></td></tr>
            <tr><td>LOC</td><td>Loc-RIB</td><td><xref target="RFC9069"/></td></tr>
            <tr><td>OUT-PRE</td><td>Adj-RIB-Out, pre-policy</td><td><xref target="RFC8671"/></td></tr>
            <tr><td>OUT-POST</td><td>Adj-RIB-Out, post-policy</td><td><xref target="RFC8671"/></td></tr>
          </tbody>
        </table>
      </section>
    </section>

    <!-- ===================================================== -->
    <section anchor="sec-setup" numbered="true">
      <name>Test Setup</name>

      <section anchor="sec-topo-basic" numbered="true">
        <name>Reference Topology</name>

        <t>The reference topology has three types of nodes: the DUT, one or more
        Peering Routers, and a BMP Station.  A Test Controller, which
        may be implemented in the same tester as the Peering Routers and the
        BMP Station, provides time reference and orchestration.</t>

        <figure anchor="fig-basic">
          <name>Reference Topology</name>
          <artwork type="ascii-art" align="center"><![CDATA[
                      +-----------------+
                      | Test Controller |
                      | (clock, trigger,|
                      |  measurement)   |
                      +--+----+------+--+
                         |    |      |
          +--------------+    |      +--------------+
          |                   |                     |
  +-------+-------+   +-------+-------+   +---------+-----+
  |  Peering      |   |      DUT      |   |  BMP Station  |
  |  Router(s)    |<->| (Monitored    |-->|  (Sink or     |
  |  PR-1..PR-n   |   |    Router)    |   |   Processing) |
  +---------------+   +---------------+   +---------------+
               BGP sessions          BMP over TCP
]]></artwork>
        </figure>

        <t>The Peering Routers establish the BGP sessions to the DUT, inject
        routes and events, and receive the DUT's advertisements.  The BGP
        sessions and the BMP session MAY use different interfaces.  The BMP
        TCP session is initiated by the DUT toward the BMP Station, as defined
        in <xref target="RFC7854"/>.</t>

        <t>A Peering Router may be a real router or an emulation function of
        a tester.  Using a tester is RECOMMENDED because it allows precise
        control of route generation and timestamping.</t>
      </section>

      <section anchor="sec-topo-multi" numbered="true">
        <name>Multiple BMP Stations</name>

        <t>For the tests that scale the number of BMP stations
        (<xref target="sec-tc-stations"/>), the topology in
        <xref target="fig-multi"/> is used.  All stations are configured on the
        DUT with the same monitoring views, unless the test case specifies
        otherwise.</t>

        <figure anchor="fig-multi">
          <name>Topology with Multiple BMP Stations</name>
          <artwork type="ascii-art" align="center"><![CDATA[
                                          +-----------------+
                                     +--->| BMP Station 1   |
  +---------------+   +---------+    |    +-----------------+
  |  Peering      |   |         |----+
  |  Router(s)    |<->|   DUT   |-------->  ...
  |  PR-1..PR-n   |   |         |----+
  +---------------+   +---------+    |    +-----------------+
       BGP sessions                  +--->| BMP Station M   |
                                          +-----------------+
                                  BMP over TCP (one per station)
]]></artwork>
        </figure>
      </section>

      <section anchor="sec-station-types" numbered="true">
        <name>BMP Station Types and Measurement Points</name>

        <t>A principal goal of the methodology is to separate the
        performance of the DUT from the performance of the collector.  The
        BMP Station therefore has to be able to operate in two modes.</t>

        <dl newline="true" spacing="normal">
          <dt>Sink Station:</dt>
          <dd>
            <t>The Sink Station is used to measure the true BMP export performance of the
            DUT.  It MUST NOT cause TCP backpressure under any load
            in the test.  Three levels of processing are defined, in
            increasing order of cost:</t>
            <ul spacing="normal">
              <li>Level 0 (raw): the station reads and discards the TCP
              byte stream.  Timestamps are taken from a packet capture on the
              Test Controller's interface (see <xref target="sec-timestamps"/>)
              and the messages are decoded offline after the test.  This
              level is RECOMMENDED for all benchmarks whose purpose is to
              measure the DUT.</li>
              <li>Level 1 (framing): the station parses only the BMP Common
              Header and the Per-Peer Header, and records message type, peer,
              and arrival time in memory.</li>
              <li>Level 2 (decode): the station fully decodes each message in
              memory, to count and verify routes online.  No disk or network
              I/O is permitted in the receive path.</li>
            </ul>
          </dd>

          <dt>Processing Station:</dt>
          <dd>The Processing Station is used to measure the effect of a
          realistic collector on the system.  It does the real work, for
          example storing the feed in a database, and is therefore allowed to
          create backpressure.  The nature of the processing
          (decoding, storage engine, hardware, and so on) MUST be fully
          described in the report.</dd>
        </dl>

        <t>Unless a test case states otherwise, it MUST be performed with a
        Sink Station first, and MAY be repeated with a Processing Station (see
        <xref target="sec-tc-processing"/>).  A Sink Station MUST have its TCP receive
        buffers sized so that it never advertises a closed window
        during a test, except in the backpressure test
        (<xref target="sec-tc-backpressure"/>), where the window is
        manipulated on purpose.</t>
      </section>

      <section anchor="sec-timestamps" numbered="true">
        <name>Time Reference and Timestamping</name>

        <t>All timestamps used to compute the metrics in this document MUST
        be taken by the Test Controller, from a single clock, or from
        multiple clocks synchronized to a common reference.  The
        synchronization uncertainty MUST be at least an order of magnitude
        smaller than the smallest time reported and MUST be documented.
        Timestamps carried in BMP messages, such as the timestamp in the
        Per-Peer Header, are produced by the DUT's clock and MAY be recorded
        for diagnostic purposes, but MUST NOT be used to compute the benchmark
        metrics defined here.</t>

        <t>The two principal observation points are:</t>

        <dl newline="true" spacing="normal">
          <dt>Injection timestamp (T_inj):</dt>
          <dd>The time at which the first byte of the BGP UPDATE (or
          other BGP message or physical event) that constitutes the Trigger
          leaves the Peering Router.  For a bulk event this is the time of
          the first UPDATE; the time of the last UPDATE is also recorded
          (T_inj_last).  For BMP-session Triggers, it is the time at which
          the TCP SYN from the DUT arrives at the BMP Station (T_syn) and, when
          available, the time at which the BMP configuration was committed
          on the DUT (T_cfg).</dd>

          <dt>Reception timestamp (T_rx):</dt>
          <dd>The time at which the last TCP segment carrying a given BMP
          message arrives at the BMP Station's interface.  A hardware or
          packet-capture timestamp is RECOMMENDED; application-layer
          timestamps are acceptable only if their error is documented.</dd>
        </dl>
      </section>
    </section>

    <!-- ===================================================== -->
    <section anchor="sec-considerations" numbered="true">
      <name>Test Considerations</name>

      <t>This section lists the parameters that have to be fixed, or at
      least documented, for every test.  The default values define the
      "Basic Configuration".  Each test case in
      <xref target="sec-tests"/> uses the Basic Configuration, and then
      adds its own configuration and Trigger.</t>

      <section anchor="sec-peers-routes" numbered="true">
        <name>Number of Peers, Address Families, and Routes</name>

        <t>The following scale profiles are defined.  The Basic Profile is
        REQUIRED for all tests.  The numbers are initial recommendations
        and are expected to evolve with the operational state of the art.</t>

        <table anchor="tbl-scale" align="center">
          <name>Scale Profiles</name>
          <thead>
            <tr>
              <th>Parameter</th>
              <th>Basic Profile (REQUIRED)</th>
              <th>Full-Table Profile (OPTIONAL)</th>
            </tr>
          </thead>
          <tbody>
            <tr><td>BGP peers</td><td>10 eBGP</td><td>4 eBGP</td></tr>
            <tr><td>Address families</td><td>IPv4 unicast, IPv6 unicast</td><td>IPv4 unicast, IPv6 unicast</td></tr>
            <tr><td>IPv4 routes per peer</td><td>100,000 (unique per peer)</td><td>approx. 1,000,000 (same prefixes on all peers)</td></tr>
            <tr><td>IPv6 routes per peer</td><td>20,000 (unique per peer)</td><td>approx. 200,000 (same prefixes on all peers)</td></tr>
            <tr><td>Resulting Adj-RIB-In paths</td><td>1,000,000 IPv4 and 200,000 IPv6</td><td>approx. 4,000,000 IPv4 and 800,000 IPv6</td></tr>
            <tr><td>Resulting Loc-RIB routes</td><td>1,000,000 IPv4 and 200,000 IPv6</td><td>approx. 1,000,000 IPv4 and 200,000 IPv6</td></tr>
            <tr><td>Routes advertised by the DUT to each peer (post-policy Adj-RIB-Out per peer)</td><td>100,000 IPv4 and 20,000 IPv6 (the routes learned from one other peer)</td><td>approx. 250,000 IPv4 and 50,000 IPv6 (one quarter of the Loc-RIB, selected by prefix)</td></tr>
            <tr><td>Resulting post-policy Adj-RIB-Out paths, all peers</td><td>1,000,000 IPv4 and 200,000 IPv6</td><td>approx. 1,000,000 IPv4 and 200,000 IPv6</td></tr>
          </tbody>
        </table>

        <t>The report MUST give the route mixture (prefix length
        distribution), route packing (prefixes per UPDATE) and the
        AS_PATH length distribution, as defined in <xref target="RFC4098"/>.
        The time sequence in which multiple peers advertise their routes
        MUST be documented.  When more than one eBGP peer advertises the same prefix, the
        policy MUST make the best path deterministic, so that the Reference
        State is known in advance.</t>

        <t>The scale of the Adj-RIB-Out views (OUT-PRE and OUT-POST) is
        the number of routes the DUT advertises to its BGP peers, and it is
        independent of the Adj-RIB-In scale.  It is the product of the number of
        peers that receive routes and the number of routes advertised to each of
        them.  In <xref target="tbl-scale"/>, the outbound policy advertises to
        each peer a fixed subset of the Loc-RIB, called the advertised set of
        that peer, so that the total of the post-policy Adj-RIB-Out is of the
        same order as the total of the Adj-RIB-In.  In the Basic Profile, the
        advertised set of peer i is the set of routes learned from peer
        (i+1), the last peer wrapping around to the first.  In the Full-Table
        Profile, the Loc-RIB is split by prefix into four equal parts, and each
        peer is advertised one of them.  The advertised sets MUST be
        documented.  The advertised sets MUST NOT contain a route learned from
        the peer it is advertised to.</t>

        <t>OUT-POST contains the advertised sets.  OUT-PRE is the result before
        the outbound policy is applied and, as defined in
        <xref target="RFC8671"/>, it normally matches the Loc-RIB, so it can be
        larger than OUT-POST, up to the size of the Loc-RIB for each peer.  The
        size of OUT-PRE as reported by the DUT MUST be documented, and the
        Reference State of OUT-PRE is the content that the DUT is expected to
        report for it.  With an outbound policy that additionally rejects a
        fraction of the advertised set, OUT-POST is smaller by the rejection
        ratio.</t>

        <t>Because the Adj-RIB-Out size grows with the number of receiving
        peers, and not with the Loc-RIB size, the tests that involve OUT-PRE
        and OUT-POST SHOULD be repeated with a varying number of peers that
        only receive routes (they advertise nothing to the DUT), to measure the
        effect of the Adj-RIB-Out scale alone, as specified in
        <xref target="sec-tc-ribout"/>.  The number of such
        receive-only peers MUST be documented.</t>
      </section>

      <section anchor="sec-policy" numbered="true">
        <name>Policy</name>

        <t>The DUT MUST be tested first with the minimal policy defined in
        <xref target="RFC4098"/>, modified as follows so that the pre-policy and
        post-policy views carry different information:</t>

        <ul spacing="normal">
          <li>The inbound policy accepts all routes and modifies at least one
          attribute (for example, sets LOCAL_PREF or adds a community).</li>
          <li>The outbound policy accepts only the advertised set of the peer
          (see <xref target="sec-peers-routes"/>) and modifies at least one
          attribute (for example, AS_PATH prepending).</li>
        </ul>

        <t>Additional runs MAY use policies that reject a fraction of the
        routes.  The rejection ratio MUST then be documented, and the
        Reference State for IN-POST and OUT-POST MUST be adjusted accordingly.
        The policy settings MUST be fully documented and MUST be identical across all
        iterations of a test.</t>
      </section>

      <section anchor="sec-params" numbered="true">
        <name>Configured Parameters</name>

        <t>The parameters in <xref target="RFC7747"/> that influence BGP
        behavior (keepalive timer, hold time, update delay timer,
        ConnectRetry timer, MRAI, MAOI, route flap damping, TCP segment size,
        TCP authentication, MTU) apply here too.  The values recommended in
        Section 4.4 of <xref target="RFC7747"/> are the Basic Configuration
        defaults.  In addition, the following BMP-related parameters MUST be
        documented, and MUST remain constant across iterations:</t>

        <ul spacing="normal">
          <li>The set of Views enabled on each BMP session, and the set of
          monitored peers.</li>
          <li>Any BMP statistics reporting interval and the list of Statistics
          Report types enabled.</li>
          <li>BMP TCP parameters: local/remote ports, MSS, send buffer size,
          receive window of the station, any BMP-specific buffering or queue limit
          configured on the DUT, and the BMP session reconnect timer.</li>
          <li>Any BMP rate limiting, batching, or throttling feature configured on
          the DUT.  These features SHOULD be disabled in the Basic
          Configuration; if they cannot be disabled, their settings MUST be
          reported.</li>
          <li>Whether BMP is carried over an in-band or out-of-band
          (management) interface.</li>
          <li>Whether any transport security or TCP authentication is
          used with the BMP session.  If used, it MUST be applied to all
          iterations of a test and to its baseline.  The effect of any
          such mechanism can be assessed by repeating a test with and without it.</li>
        </ul>
      </section>

      <section anchor="sec-logging" numbered="true">
        <name>Tracing, Logging, and Other Processes</name>

        <t>Debug tracing and verbose logging on the DUT, and on the BMP
        Station (other than what is needed to record the measurement), MUST be
        disabled during all tests, as they can have a significant effect on
        timing.  Any logging that cannot be disabled MUST be documented.</t>

        <t>The DUT MUST be loaded only with the configuration necessary for the test.
        Unless a test case says otherwise, all other routing protocols MUST
        have converged and be stable before the Trigger, and no
        configuration change, software upgrade, or management query that is
        not part of the measurement is allowed during the test.  Management
        access used to collect CPU and memory samples (see
        <xref target="sec-resource"/>) MUST be documented, since it
        too consumes control-plane resources.</t>
      </section>

      <section anchor="sec-ha" numbered="true">
        <name>High Availability</name>
        <t>As in <xref target="RFC7747"/>, redundancy features of the
        routing engine (for example, non-stop routing) SHOULD be disabled.
        If they cannot be, the results may not be comparable with other
        devices and that MUST be stated in the report.</t>
      </section>

      <section anchor="sec-trials" numbered="true">
        <name>Repeatability and Statistics</name>

        <t>Each test MUST be repeated at least 10 times, with identical
        configuration, starting from the same initial state.  For every metric
        the report MUST give the minimum, maximum, mean, median, and standard
        deviation.  For per-event metrics such as BEL, the 95th and 99th
        percentiles over all events and all iterations MUST also be given.  Outlier handling
        MUST be documented, and no result may be silently dropped.</t>
      </section>
    </section>

    <!-- ===================================================== -->
    <section anchor="sec-metrics" numbered="true">
      <name>Metrics and Measurement Methods</name>

      <section anchor="sec-bct" numbered="true">
        <name>BMP Convergence Time</name>

        <t>BMP Convergence Time for a View is:</t>
        <artwork type="ascii-art"><![CDATA[
   BCT(view) = T_converged(view) - T_inj
]]></artwork>
        <t>where T_converged is the time at which the View State first equals
        the Reference State (and stays equal), observed using one of the
        methods in <xref target="sec-completion"/>.  When the Trigger is a bulk
        event, T_inj is the time of the first message of the stimulus; the
        report SHOULD also give BCT measured from T_inj_last.  The BCT is
        reported separately for each View, each peer, and each AFI/SAFI, as
        well as for the whole DUT (the maximum over all of them).</t>
      </section>

      <section anchor="sec-bidt" numbered="true">
        <name>BMP Initial Dump Time and Dump Rate</name>

        <artwork type="ascii-art"><![CDATA[
   BIDT(view)      = T_complete(view) - T_init
   Dump Rate(view) = N_routes(view) / BIDT(view)    [routes per second]
]]></artwork>
        <t>where T_init is the time the Initiation message is received by the
        BMP Station, T_complete is the time at which the dump is
        determined to be complete by one of the methods in
        <xref target="sec-completion"/>, and N_routes is the number of route
        entries in the Reference State for the View.  When T_syn and T_cfg
        are available, the report SHOULD also give the start-up delays
        (T_init - T_syn) and (T_syn - T_cfg), because they may account for a
        significant portion of the time to visibility.</t>
      </section>

      <section anchor="sec-bel" numbered="true">
        <name>BMP Event Latency</name>

        <artwork type="ascii-art"><![CDATA[
   BEL(e) = T_rx(m(e)) - T_inj(e)
]]></artwork>
        <t>where m(e) is the first BMP message that reports event e itself:</t>
        <ul spacing="normal">
          <li>Route event: m(e) is the first Route Monitoring message reporting
          that route change in the View under test.  BEL is reported per View.</li>
          <li>Session event: m(e) is the Peer Down or Peer Up Notification for
          that peer.  It reports session state independent of the Views
          <xref target="RFC8671"/>, so BEL is reported once per event and peer,
          not per View.</li>
        </ul>
        <t>The BEL of a session event measures only the delay of the Peer Down
        or Peer Up Notification.  It does not include the delay of any route
        changes that the session event causes.  Those are separate route
        events, measured per View as above, and MUST NOT be reported as the
        BEL of the session event.</t>

        <t>A Peer Down Notification implicitly withdraws all routes associated
        with the peer, and an implementation MAY omit explicit withdrawals
        <xref target="RFC7854"/>.  The View State is computed by treating a
        Peer Down Notification as the withdrawal of all routes of that peer.
        The report MUST state whether explicit withdrawals were observed.</t>

        <t>Each event that is injected MUST be identified.  Before an
        announcement is sent, the Test Controller SHOULD stamp it with a
        sequence value, which is a unique, monotonically increasing number
        carried in an attribute that is preserved by the configured policies
        (for example, a BGP Large Community <xref target="RFC8092"/>).  The
        sequence value allows each Route Monitoring message to be traced
        back to the event that caused it, and allows loss, duplication, and
        re-ordering to be identified even during churn.  Withdrawals are
        correlated by prefix and by their time relative to the event.  If
        the DUT removes or rewrites the stamp in some Views, the
        correlation method MUST be documented.</t>
      </section>

      <section anchor="sec-fanout" numbered="true">
        <name>BMP Fan-Out and Per-Peer Spread</name>

        <t>One routing event can lead to several BMP messages, in particular
        in the Adj-RIB-Out Views, where each receiving peer has its own
        Adj-RIB-Out and is reported with its own Per-Peer Header
        <xref target="RFC8671"/>.  For an event e and a View:</t>

        <artwork type="ascii-art"><![CDATA[
   Fan-Out(e)    = number of BMP messages generated for e
   Spread(e)     = T_rx(last message for e) - T_rx(first message for e)
]]></artwork>
        <t>Fan-Out is counted per View and per peer.  Spread shows how unevenly
        the DUT serves the peers, and it is the difference between the BEL of the
        last peer and of the first peer reported.  Both MUST be reported for
        the tests in <xref target="sec-tc-ribout"/>, and SHOULD be reported
        for all event tests that involve the Adj-RIB-Out Views.</t>
      </section>

      <section anchor="sec-bpt" numbered="true">
        <name>BGP Propagation Time and BMP Overhead</name>

        <t>To determine the effect of BMP on BGP convergence, the BPT of
        the DUT is measured with BMP disabled (BPT_off) and with BMP
        enabled (BPT_on), for the same event.</t>

        <artwork type="ascii-art"><![CDATA[
   BMP Overhead (absolute) = BPT_on - BPT_off
   BMP Overhead (relative) = (BPT_on - BPT_off) / BPT_off
]]></artwork>
        <t>BPT is measured at the Peering Routers as defined in
        <xref target="RFC4098"/>: from the injection of the event at the
        source Peering Router to the receipt of the corresponding UPDATE at the destination
        Peering Router.</t>
      </section>

      <section anchor="sec-completion" numbered="true">
        <name>Detecting Completion of a Dump or of Convergence</name>

        <t>A dump or a convergence period is complete when the BMP Station
        has received everything that it is supposed to receive.  This is
        not trivial to detect externally.  The following methods are defined.
        The method used MUST be reported.  More than one method SHOULD be
        applied to the same run and the results compared.</t>

        <section anchor="sec-m-complete" numbered="true">
          <name>Method A: Reference-State Completeness (Primary)</name>
          <t>The Test Controller knows the Reference State.  T_complete is the time T_rx of the BMP
          message by which the View State first equals the Reference State.
          This requires the decoding of every BMP message, in the receive path
          (Sink Station Level 2) or offline from a packet capture (Level 0).
          This method does not depend on any signal sent by the DUT, and
          is therefore the REQUIRED reference method for steady-state tests.
          It also detects routes that are missing, duplicated, or stale.
          The method applies to churn, too, as long as the Reference State is
          defined at each point in time from the sequence-stamped events.</t>
        </section>

        <section anchor="sec-m-eor" numbered="true">
          <name>Method B: End-of-RIB Markers</name>
          <t>T_complete is the time of the last End-of-RIB marker
          <xref target="RFC4724"/>, carried in a BMP Route Monitoring message, for all monitored peers and
          AFI/SAFIs in the View.  BIDT is then measured from the Initiation
          message to this marker.</t>
          <t><strong>Caveat:</strong> The End-of-RIB marker is generated
          by BGP and it may be generated as soon as the BGP
          process has finished enumerating a table, and not when the BMP
          messages for that table have actually been serialized and written to the
          TCP socket.  An implementation can send an End-of-RIB marker before
          the dump is complete or, in some cases, not send it at all.  The EoR time
          therefore has to be regarded as an indication, or at most as a lower bound,
          and MUST be checked against Method A before it is reported as a
          dump time.  A result obtained only through Method B MUST be labeled
          as such.</t>
        </section>

        <section anchor="sec-m-quiesce" numbered="true">
          <name>Method C: Quiescence of the BMP Stream</name>
          <t>T_complete is the time of the last Route Monitoring message before
          a period of at least one Quiescence Interval (QI) in which the
          DUT sends no Route Monitoring message.  The report MUST give the QI, and
          the computed times MUST NOT include it.  This method is suitable
          only for steady-state tests with no background churn, because any
          churn would reset the quiescence period.  A DUT that pauses
          during a dump (for example because it is yielding the CPU)
          for longer than QI would give a false completion time;
          therefore QI SHOULD be chosen larger than the longest observed
          intra-dump pause.</t>
        </section>

        <section anchor="sec-m-cpu" numbered="true">
          <name>Method D: Control-Plane CPU</name>
          <t>In steady state, with BMP being the only activity on the DUT, the
          dump appears as a rise of the CPU utilization (of the routing
          engine or of the process or processes responsible for BMP) above
          its baseline.  T_complete is the time at which the utilization
          comes back within a tolerance of the pre-test baseline and remains
          there for a hold time.  The baseline, tolerance, hold time, sampling
          method, and sampling interval MUST be documented.  This method does
          not depend on BMP messages, but its resolution is limited by the
          sampling interval, and it is not suitable when other
          protocols, or BGP churn, are contributing to the CPU load.  It SHOULD
          be used as a cross-check only.</t>
        </section>

        <section anchor="sec-m-canary" numbered="true">
          <name>Method E: Canary Routes (for Churn Conditions)</name>
          <t>When a dump or convergence period happens during churn,
          the methods B, C, and D are not reliable.  In this case, in addition to Method A,
          the Test Controller injects canary routes: uniquely-identified routes, on a dedicated set of prefixes
          and at a low rate (for example one per second per View), from the
          start of the test.  The BEL of the canary routes shows the delay in the
          BMP export pipeline at that point in time.  When the BMP export
          queue is FIFO, the BEL of a canary equals the time it takes to drain
          the backlog that was in front of it.  Canary routes MUST be
          included in the Reference State, and the churn level in which they
          were measured MUST be reported.  Whether the DUT delivers
          BMP messages in strict order is implementation-specific, and so canary
          measurements describe the latency of the pipeline, not necessarily
          the dump completion time.</t>
        </section>

        <section anchor="sec-m-stats" numbered="true">
          <name>Precondition Verification with Statistics Reports</name>
          <t>Before each run, the Test Controller MUST verify that the DUT
          holds the expected state, i.e. that the Reference State matches the DUT.  The counts in the
          DUT's BGP tables, as read through the management interface, and
          optionally, the Statistics Report counters
          <xref target="RFC7854"/> <xref target="RFC9972"/>, can be used for that purpose.
          These counters report what the DUT holds, and not what the DUT has
          exported over BMP, so they MUST NOT be used as a measure of
          completeness of the BMP feed.</t>
        </section>
      </section>

      <section anchor="sec-resource" numbered="true">
        <name>Resource and Stability Metrics</name>

        <t>For each run, the following DUT-side quantities MUST be collected
        at a documented sampling interval, with minimal impact on the DUT:</t>

        <ul spacing="normal">
          <li>CPU utilization of the control plane, overall and, if available,
          of the BGP and the BMP processes, and peak value during the test.</li>
          <li>Memory utilization of the control plane, overall and, if
          available, of the BGP and the BMP processes; peak value and value
          after the test.</li>
          <li>Time for the CPU utilization to return to the baseline after the Trigger.</li>
          <li>The number of BGP session flaps (any session reset, any
          Notification, any hold timer expiry) and the number of BMP
          session resets during the test.  Both MUST be zero for a test to be
          considered successful, unless the test is intended to cause them.</li>
          <li>The BMP queue depth, drop counters, or any similar BMP internal
          counters that the DUT exposes.</li>
        </ul>
      </section>

      <section anchor="sec-scaling" numbered="true">
        <name>Scaling Metrics</name>

        <t>When a test is repeated while varying a scaling parameter
        (the number of Views, the number of receiving peers, or the number of
        BMP Stations), the following are reported, where X(k) is a benchmark metric (BCT, BIDT,
        peak CPU, ...) at k units of the parameter:</t>

        <artwork type="ascii-art"><![CDATA[
   Scaling Ratio       SR(k) = X(k) / X(1)
   Incremental Cost    IC(k) = X(k) - X(k-1)
]]></artwork>
        <t>SR(k) = 1 means that the parameter has no effect, SR(k) = k means that
        cost increases linearly with k (each additional unit is as expensive
        as the first), SR(k) less than k indicates that a part of the work is
        shared (for example, a single route lookup used for several Views or
        stations), and SR(k) greater than k indicates a super-linear
        degradation (for example, contention for a common queue or memory).  The
        report MUST present the whole series X(1)..X(K) and
        MAY include a fitted curve.</t>
      </section>
    </section>

    <!-- ===================================================== -->
    <section anchor="sec-tests" numbered="true">
      <name>Test Cases</name>

      <t>All test cases start from the Basic Configuration described in
      <xref target="sec-setup"/> and <xref target="sec-considerations"/>, and
      use the Sink Station (Level 0 or Level 2) unless stated.  Unless
      otherwise noted, every test case is performed once for each of the
      five Views in <xref target="tbl-views"/>, with that View alone
      enabled, and in addition for the combinations specified in
      <xref target="sec-tc-views"/>.</t>

      <section anchor="sec-tc-common" numbered="true">
        <name>Common Procedure</name>
        <ol spacing="normal" type="1">
          <li>Configure the DUT, the Peering Routers, and the BMP Station as
          in the Basic Configuration.  Disable BMP (or, for event tests,
          configure BMP as required by the test case).</li>
          <li>Establish all BGP sessions and have the Peering Routers
          advertise their routes.  Verify that the DUT holds the Reference
          State (<xref target="sec-m-stats"/>).</li>
          <li>Wait until the DUT is in a stable state: no BGP messages are
          exchanged other than keepalives, and the CPU and memory utilization have been constant
          within the tolerance for a hold time.  Record the baseline.</li>
          <li>Start packet capture and resource sampling.</li>
          <li>Apply the Trigger of the test case and record T_inj.</li>
          <li>Wait for BMP Convergence by the methods that apply (<xref target="sec-completion"/>),
          then for the resource utilization to be back to the baseline.</li>
          <li>Verify that the View State equals the Reference State and
          record any deviation (missing, duplicate, stale, or out-of-order
          routes).</li>
          <li>Record the metrics in <xref target="sec-metrics"/>, restore the
          initial state, and repeat the test (<xref target="sec-trials"/>).</li>
        </ol>
      </section>

      <section anchor="sec-tc-dump" numbered="true">
        <name>Initial Dump in Steady State</name>

        <section anchor="sec-tc-dump-view" numbered="true">
          <name>Single-View Initial Dump</name>
          <t><strong>Objective:</strong> Measure the BIDT and the Dump Rate
          of the DUT for each View, when BMP is the only activity.</t>
          <t><strong>Procedure:</strong> BMP is disabled at the start.  The
          Trigger is the commit of the BMP configuration that
          enables a single View toward one Sink Station.  Perform the Common
          Procedure for each View.  Measure BIDT by Methods A (REQUIRED),
          and B, C, and D (RECOMMENDED), and report all of them.  Also report the
          time from T_cfg to T_syn to T_init.</t>
          <t><strong>Reporting:</strong> BIDT and Dump Rate per View, per
          AFI/SAFI, per method; resource peak values; any difference between
          the methods and its probable cause (for example, an EoR sent
          before the dump is complete).</t>
        </section>

        <section anchor="sec-tc-dump-reconnect" numbered="true">
          <name>Initial Dump after BMP Session Re-establishment</name>
          <t><strong>Objective:</strong> Measure the time to visibility after a
          BMP session failure on an otherwise unchanged DUT.</t>
          <t><strong>Procedure:</strong> The BMP session is established and
          converged.  The Trigger is the abrupt closure of the TCP connection by the
          BMP Station (TCP reset), followed by the Station accepting a new connection.
          Measure T_syn and the resulting BIDT.  The DUT's BMP reconnect behavior
          (timer, backoff) MUST be documented, and the delay between the reset and T_syn
          reported separately from BIDT.  Repeat with a graceful close, if the implementation supports it.</t>
        </section>

        <section anchor="sec-tc-dump-reboot" numbered="true">
          <name>Time to Visibility after DUT Restart</name>
          <t><strong>Objective:</strong> Measure the time to complete visibility
          after a restart of the DUT.  This is the main figure for
          traffic-engineering controllers.</t>
          <t><strong>Procedure:</strong> BMP and BGP are both configured in the
          startup configuration.  The Trigger is the restart of the routing
          software (or a reload) of the DUT.  Record the time at which each BGP
          session is established, the time of the Initiation message, and
          the BIDT.  Report the time from the restart Trigger to
          BMP Convergence, and the part of it that is attributable to
          BGP convergence (for example, the time for the last BGP session to
          reach established state, and to receive its EoR).  This test is
          OPTIONAL.</t>
        </section>
      </section>

      <section anchor="sec-tc-dump-churn" numbered="true">
        <name>Initial Dump during Churn</name>
        <t><strong>Objective:</strong> Measure the BIDT when the DUT is
        concurrently processing BGP changes.</t>
        <t><strong>Procedure:</strong> A set of Peering Routers generates
        background churn (a continuous, repeating series of announcements
        and withdrawals of routes that are not part of the dump
        verification set, or that are sequence-stamped) at a rate R.  The
        tests are run for R of 0 (as in <xref target="sec-tc-dump-view"/>) and at increasing rates, up to, at
        least, the LFCR found in <xref target="sec-tc-churn"/>.  While the
        churn is in progress, the Trigger of <xref target="sec-tc-dump-view"/>
        is applied.  Methods A and E are REQUIRED; B and D are not
        reliable under churn, but MAY be reported, with that statement.</t>
        <t><strong>Reporting:</strong> BIDT as a function of R, the canary
        BEL distribution over time, and the time-to-drain of the backlog after the
        churn stops.</t>
      </section>

      <section anchor="sec-tc-events" numbered="true">
        <name>BMP Convergence after BGP Events</name>

        <t>For the tests in this section BMP is already established and in
        sync, with the View or Views under test enabled, before the
        Trigger.  For each test, T_inj, BCT (from T_inj and from T_inj_last), and BEL
        distribution are reported for each enabled View.</t>

        <section anchor="sec-tc-ev-adv" numbered="true">
          <name>Route Advertisement</name>
          <t>A Peering Router advertises N new routes (N = 1, 1,000,
          10,000, 100,000), in a single burst, to the DUT.  IN-PRE, IN-POST,
          and LOC views show the new routes; OUT-PRE is expected to show them for
          all peers other than the sender, and OUT-POST only for the peers whose
          advertised set includes them.</t>
        </section>

        <section anchor="sec-tc-ev-wd" numbered="true">
          <name>Route Withdrawal</name>
          <t>A Peering Router withdraws N routes in a burst (N as above).  The
          BMP convergence for the Adj-RIB-In view is the time at which the
          withdrawals are reported; for LOC, OUT-PRE and OUT-POST, the time at which
          the resulting changes are reported (withdrawals, or replacement by
          alternative paths where available).</t>
        </section>

        <section anchor="sec-tc-ev-attr" numbered="true">
          <name>Path Attribute Change and Best-Path Change</name>
          <t>A Peering Router re-advertises N routes with changed attributes.
          Two cases are tested: (a) the change does not affect the best path
          (visible in IN-PRE and IN-POST only); and (b) the change makes the route
          the new best path (visible in all Views).  Case (b) requires a
          second path to the same prefixes, from another Peering Router,
          and the Full-Table profile has this by construction.</t>
        </section>

        <section anchor="sec-tc-ev-peerdown" numbered="true">
          <name>BGP Peer Down</name>
          <t>The Trigger is the loss of one BGP session (hard reset from
          the Peering Router, or session shutdown by the Test Controller).
          Measure and report separately:</t>
          <ol spacing="normal" type="a">
            <li>Session event: the BEL of the Peer Down Notification, with its
            reason code.  This is a single-message result.</li>
            <li>Route events caused by the session event: for LOC, and for
            OUT-PRE and OUT-POST of the other peers, the BEL, BCT and Fan-Out
            of the resulting withdrawals or replacements.</li>
          </ol>
          <t>For IN-PRE, IN-POST and the failed peer's own Adj-RIB-Out Views,
          the Peer Down Notification itself removes the routes, so BCT equals
          the BEL in (a).  If explicit withdrawals follow, BCT is the arrival
          of the last one.  Results (a) and (b) measure different things and
          MUST NOT be compared or added.</t>
        </section>

        <section anchor="sec-tc-ev-peerup" numbered="true">
          <name>BGP Peer Up with Full Table</name>
          <t>The Trigger is the establishment of an additional BGP session,
          on which the Peering Router advertises its full set of routes.
          Measure the BEL of the Peer Up Notification (session event).  Report
          the BIDT and BCT of the Views for that peer separately, as
          route-level results.  This test measures the
          dump of one peer in the middle of normal operation, as opposed to
          the dump of the entire router in <xref target="sec-tc-dump"/>.</t>
        </section>

        <section anchor="sec-tc-ev-link" numbered="true">
          <name>Physical Link Failure</name>
          <t>The Trigger is the failure of the physical link between the
          DUT and a Peering Router (carrier loss at the tester).  The
          DUT's interface failure detection timer MUST be documented (see
          <xref target="RFC7747"/>).  The measured BCT includes the failure
          detection time; the report SHOULD give the point where the DUT
          detected the event, if it can be observed.</t>
        </section>

        <section anchor="sec-tc-ev-policy" numbered="true">
          <name>Policy Change and Route Refresh</name>
          <t>The Trigger is a change of the inbound or outbound policy on
          the DUT that changes the post-policy content of N routes, or the
          sending of a Route Refresh request by a Peering Router.  The test
          applies primarily to IN-POST, LOC, and OUT-POST; the IN-PRE and
          OUT-PRE views are expected not to change for a policy-only change,
          and it SHOULD be verified and reported that no messages are
          sent for them.</t>
        </section>

        <section anchor="sec-tc-ev-igp" numbered="true">
          <name>Next-Hop or IGP Metric Change</name>
          <t>This test is OPTIONAL.  The Trigger is a change to the IGP metric or
          reachability of the BGP next hop, which changes the best path for
          N routes without any BGP message from a Peering Router.  This is
          visible only in the LOC and the Adj-RIB-Out Views.  A Peering Router or the Test
          Controller emulates the IGP change.</t>
        </section>
      </section>

      <section anchor="sec-tc-churn" numbered="true">
        <name>Sustained Churn and Loss-Free Churn Rate</name>
        <t><strong>Objective:</strong> Determine the highest event rate that
        the BMP export keeps up with, and how it degrades beyond.</t>
        <t><strong>Procedure:</strong> With BMP established and in sync, the
        Peering Routers generate a sustained stream of sequence-stamped
        announcements, withdrawals, and attribute changes at rate R
        (events per second, summed over all peers), for a duration D (at
        least 5 minutes).  R is increased in steps (a binary search is
        RECOMMENDED) between 0 and the rate at which BGP itself stops converging.
        For each R, record: BEL distribution; the number of events not reported
        (lost), reported more than once, or reported out of order; BMP and
        BGP session stability; resource utilization; and the time after the
        stop of the churn at which BMP Convergence is reached (backlog drain
        time).</t>
        <t><strong>Reporting:</strong> The LFCR for each View (and for all
        Views together), plus the curves of BEL percentiles, CPU, and
        backlog drain time as functions of R.  A rate is loss-free only
        if no event is lost, no session resets, and the BEL at the 99th
        percentile is stable over time (no unbounded growth).</t>
        <t>The churn profile (the share of announcements, withdrawals, and
        attribute changes; whether it is spread uniformly or in bursts; and the
        number of peers it comes from) MUST be documented.</t>
      </section>

      <section anchor="sec-tc-bgp-impact" numbered="true">
        <name>Impact of BMP on BGP Convergence</name>
        <t><strong>Objective:</strong> Determine whether, and by how much,
        enabling BMP slows down the DUT's BGP convergence.</t>
        <t><strong>Procedure:</strong> Select one or more of the events from
        <xref target="sec-tc-events"/>, and the sustained churn of
        <xref target="sec-tc-churn"/> at a fixed rate.  For each, measure
        the BPT with BMP disabled (BPT_off), then with each View
        enabled individually and with all Views enabled, to a Sink Station
        (BPT_on).  Keep everything else equal.  Repeat with the
        Processing Station.  The test is also performed with all Views enabled
        at the largest Adj-RIB-Out scale of <xref target="sec-tc-ribout"/>.</t>
        <t><strong>Reporting:</strong> BPT_off, BPT_on, and BMP Overhead
        (<xref target="sec-bpt"/>), CPU and memory impact, and any
        BGP session instability.</t>
      </section>

      <section anchor="sec-tc-views" numbered="true">
        <name>Scaling with the Number of Monitoring Views</name>
        <t><strong>Objective:</strong> Characterize the cost of each
        additional View, and determine whether it is linear, shared, or
        super-linear.</t>
        <t><strong>Procedure:</strong> Repeat the initial dump tests
        (<xref target="sec-tc-dump-view"/>), the dump-under-churn test
        (<xref target="sec-tc-dump-churn"/>), the event tests
        (<xref target="sec-tc-events"/>), and the BMP/BGP impact tests
        (<xref target="sec-tc-bgp-impact"/>), enabling Views cumulatively
        on the same BMP session, in the following order:
        IN-PRE; IN-PRE + IN-POST; + LOC; + OUT-PRE; + OUT-POST.  The test
        MUST be repeated with at least one other order, for example the
        reverse, to expose any order dependence.  The Views that are common
        in operation (for example, IN-PRE and LOC) SHOULD also be tested as
        sets.  The Adj-RIB-Out scale is that of the Basic Profile in this test;
        its variation is covered in <xref target="sec-tc-ribout"/>.</t>
        <t><strong>Reporting:</strong> X(k), SR(k), and IC(k) as defined in
        <xref target="sec-scaling"/> for BIDT, BCT, the Dump Rate, peak
        CPU, and peak memory, for each k = 1..5; and, in addition, the time at
        which each individual View finishes, as the DUT may serialize or
        interleave the Views.  The report MUST state whether Views are sent
        sequentially or concurrently, if it can be determined.</t>
      </section>

      <section anchor="sec-tc-ribout" numbered="true">
        <name>Scaling of the Adj-RIB-Out Views</name>
        <t><strong>Objective:</strong> Characterize the cost of the OUT-PRE and
        OUT-POST Views as a function of the Adj-RIB-Out scale, that is, the
        number of peers the DUT advertises to and the number of routes it
        advertises to each of them, with the Adj-RIB-In and Loc-RIB scale
        held constant.</t>
        <t><strong>Procedure:</strong> The Basic Profile peers advertise their
        routes as in <xref target="tbl-scale"/>.  The Adj-RIB-Out scale is
        then varied along two independent dimensions, each with the Views
        OUT-PRE alone, OUT-POST alone, and both together:</t>
        <ol spacing="normal" type="a">
          <li>Number of receiving peers.  Receive-only Peering Routers, which
          advertise nothing to the DUT, are added to the Basic Profile peers
          to reach a total of P = 10, 20, 50, and 100 peers, or the largest
          number the DUT supports.  Each receive-only peer is advertised a set of the
          same size as the per-peer advertised set of the Basic Profile, with
          the outbound policy of <xref target="sec-policy"/>.</li>
          <li>Outbound policy rejection ratio.  With the Basic Profile peers,
          the outbound policy additionally rejects 0%, 10%, 50%, and 90% of the
          routes in the advertised set of each peer.  The OUT-PRE
          content stays the same, while the OUT-POST content shrinks.  This shows
          whether the cost of the DUT follows the pre-policy or the post-policy
          size.</li>
        </ol>
        <t>For each point, the following tests are repeated:
        the initial dump of <xref target="sec-tc-dump-view"/>; the route
        advertisement, withdrawal, and best-path change events of
        <xref target="sec-tc-events"/>; the peer-up of an additional
        receive-only peer on an already converged DUT (the dump of one
        Adj-RIB-Out in normal operation); and the sustained churn of
        <xref target="sec-tc-churn"/> at a fixed rate.</t>
        <t>The Reference State of OUT-PRE and OUT-POST can be checked in two
        ways: against what the Test Controller computes from the policy, and,
        for OUT-POST, against what the receive-only Peering Routers actually
        receive, since the post-policy Adj-RIB-Out is defined to convey what
        is sent to the peer <xref target="RFC8671"/>.  Any discrepancy MUST be
        reported.</t>
        <t>A single change in the Loc-RIB can lead to one BMP message for each
        receiving peer that is advertised the route, so the number of BMP messages
        generated for one
        injected event grows with P.  Per-peer data MUST therefore be recorded: the
        BEL and BCT for the first and the last peer reported, and the number of
        BMP messages generated per injected event (the BMP Fan-Out and the
        Spread of <xref target="sec-fanout"/>).  This holds even if the DUT
        generates a single BGP UPDATE for a group of peers.</t>
        <t><strong>Reporting:</strong> BIDT, Dump Rate (in total
        Adj-RIB-Out route entries per second), BCT, BEL percentiles, LFCR, and
        peak CPU and memory for each point, as X(P) with SR(P) and IC(P) for
        dimension (a) (<xref target="sec-scaling"/>), and as a function of the
        rejection ratio for dimension (b); the fan-out per event; per-peer
        first and last values; and the BPT overhead of
        <xref target="sec-tc-bgp-impact"/> at the largest P tested.</t>
      </section>

      <section anchor="sec-tc-stations" numbered="true">
        <name>Scaling with the Number of BMP Stations</name>
        <t><strong>Objective:</strong> Determine how BMP convergence and
        DUT resource usage change as the number of BMP stations grows.</t>
        <t><strong>Procedure:</strong> Using the topology of
        <xref target="fig-multi"/>, and the same View set on each
        station, repeat the initial dump
        (<xref target="sec-tc-dump-view"/>), the event
        (<xref target="sec-tc-events"/>), sustained churn
        (<xref target="sec-tc-churn"/>), and BMP/BGP impact
        (<xref target="sec-tc-bgp-impact"/>) tests with M = 1, 2, 4, 8, and,
        if supported, 16 Sink Stations, started simultaneously, first with the
        Views of the Basic Configuration and then with all five Views enabled at
        the largest Adj-RIB-Out scale of <xref target="sec-tc-ribout"/>, since
        that is the largest BMP load that each additional Station has to receive.  The
        test SHOULD be repeated with Stations started in staggered fashion
        (a new Station connects while the others are in sync) to measure
        the cost of an additional dump during normal operation.  All
        stations MUST be measured, and the report MUST include the BCT or
        BIDT of the first and of the last station to converge, as the
        DUT might serve them unevenly.</t>
        <t><strong>Reporting:</strong> X(M), SR(M), and IC(M) for BIDT, BCT,
        LFCR, and resource peak values, per Station, plus the maximum number
        of Stations at which the DUT meets the loss-free criteria.</t>
      </section>

      <section anchor="sec-tc-processing" numbered="true">
        <name>Effect of the Collector (Sink versus Processing Station)</name>
        <t><strong>Objective:</strong> Separate the performance of the DUT
        from the performance of a collector, and quantify the latter.</t>
        <t><strong>Procedure:</strong> Repeat selected tests from this
        section with a Processing Station in place of the Sink Station, in
        the same topology and with the same configuration.  The
        Sink Station results are the baseline for the DUT.  The Processing
        Station is allowed to perform its work in the usual way (for example,
        decoding, normalization, and storage in a database).  The Processing
        Station SHOULD record, for each message, the time it was received
        (T_rx) and the time its content was available to the consumers
        (T_commit).</t>
        <t><strong>Reporting:</strong> The difference in the BIDT, BCT, and
        LFCR between the Processing and the Sink Station; the collector-side
        lag distribution (T_commit - T_rx); the TCP receive window
        behavior (the time it was reduced or closed), which is the cause
        of any backpressure on the DUT; and the full description of the
        collector, its hardware, and its software.</t>
      </section>

      <section anchor="sec-tc-backpressure" numbered="true">
        <name>BMP TCP Backpressure and Buffer Behavior</name>
        <t><strong>Objective:</strong> Characterize how the DUT behaves when
        the BMP Station reads the stream slower than the DUT writes it.</t>
        <t><strong>Procedure:</strong> With BMP established and in sync,
        the Peering Routers generate churn at a fixed rate R (which SHOULD
        be a rate below the LFCR, and a rate at or above the LFCR), with all
        five Views enabled, at the Basic Profile and at the largest
        Adj-RIB-Out scale of <xref target="sec-tc-ribout"/>, because the amount
        of data buffered during a stall depends on both.  The Test
        Controller then reduces the rate at which the Station reads the
        TCP stream, without closing the connection, as follows:</t>
        <ol spacing="normal" type="a">
          <li>Complete stall (receive window to zero) for a duration D, with D =
          10 s, 60 s, 300 s, and 1,800 s, and then resume at full rate.</li>
          <li>Constant slow reading at a fraction of the rate at which the DUT
          generates BMP data (for example 90%, 50%, and 10%) for a duration D.</li>
          <li>Repeat (a), with a full initial dump pending.</li>
        </ol>
        <t>For each case, record:</t>
        <ul spacing="normal">
          <li>Whether the DUT keeps the BMP session, resets it, or sends a Termination
          Message, and the time elapsed before it does that.</li>
          <li>Whether messages are dropped, aggregated, or delayed, and how it can
          be observed by the Station (for example, by a gap in the sequence-stamped
          events or by the Reference-State differences after recovery).</li>
          <li>The growth of the DUT's memory over time, and the peak value; whether
          the memory is released after recovery (possible leak).</li>
          <li>The effect on BGP: sessions flaps, BPT (<xref target="sec-bpt"/>),
          and CPU.  BGP processing MUST NOT be adversely impacted by a stalled
          BMP Station, and any impact is a significant finding.</li>
          <li>The time to BMP Convergence after the Station resumes, and
          whether the DUT recovers by sending the backlog, or by
          resetting the session and doing a new Initial Dump.</li>
          <li>Any limit, configured or hard-coded, on BMP buffering or
          queue size that is observed.</li>
        </ul>
      </section>
    </section>

    <!-- ===================================================== -->
    <section anchor="sec-reporting" numbered="true">
      <name>Reporting Format</name>

      <t>The report for each test MUST include the following, so that others
      can reproduce the test and compare it with other results.</t>

      <section anchor="sec-rep-dut" numbered="true">
        <name>DUT and Test Bed Description</name>
        <table anchor="tbl-rep-setup" align="center">
          <name>Test Bed Parameters</name>
          <thead><tr><th>Parameter</th><th>Value</th></tr></thead>
          <tbody>
            <tr><td>DUT hardware platform, routing engine / CPU model, memory</td><td></td></tr>
            <tr><td>DUT software release, BMP feature version / draft or RFC level</td><td></td></tr>
            <tr><td>Redundancy features (disabled/enabled)</td><td></td></tr>
            <tr><td>Peering Router / tester model and software</td><td></td></tr>
            <tr><td>BMP Station type (Sink L0 / L1 / L2, Processing), hardware, software</td><td></td></tr>
            <tr><td>Timestamping point and clock synchronization uncertainty</td><td></td></tr>
            <tr><td>Interfaces: type, speed, in-band / out-of-band for BMP</td><td></td></tr>
            <tr><td>Logging / tracing state on DUT and Station</td><td></td></tr>
          </tbody>
        </table>
      </section>

      <section anchor="sec-rep-config" numbered="true">
        <name>Test Configuration</name>
        <table anchor="tbl-rep-config" align="center">
          <name>Test Configuration Parameters</name>
          <thead><tr><th>Parameter</th><th>Value</th></tr></thead>
          <tbody>
            <tr><td>Scale profile; number of BGP peers (iBGP / eBGP)</td><td></td></tr>
            <tr><td>Address families; routes per peer, route mixture, route packing, AS_PATH distribution</td><td></td></tr>
            <tr><td>Inbound / outbound policy; rejection ratio</td><td></td></tr>
            <tr><td>Adj-RIB-Out scale: number of receiving peers (of which receive-only), routes advertised per peer, advertised-set assignment</td><td></td></tr>
            <tr><td>BGP timers (keepalive, hold, MRAI, ...), TCP parameters, authentication</td><td></td></tr>
            <tr><td>BMP Views enabled; monitored peers; statistics interval</td><td></td></tr>
            <tr><td>Number of BMP stations; BMP rate-limit or batching features</td><td></td></tr>
            <tr><td>Background churn rate and profile (if any)</td><td></td></tr>
            <tr><td>Number of iterations</td><td></td></tr>
          </tbody>
        </table>
      </section>

      <section anchor="sec-rep-results" numbered="true">
        <name>Results</name>
        <table anchor="tbl-rep-results" align="center">
          <name>Result Reporting Template (one row per View and metric)</name>
          <thead>
            <tr>
              <th>Test case</th><th>View</th><th>Metric</th><th>Method</th>
              <th>Min</th><th>Max</th><th>Mean</th><th>Median</th><th>Std. dev.</th>
              <th>P95</th><th>P99</th>
            </tr>
          </thead>
          <tbody>
            <tr><td>6.x</td><td>IN-PRE</td><td>BIDT (s)</td><td>A</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr>
            <tr><td>6.x</td><td>IN-PRE</td><td>Dump Rate (routes/s)</td><td>A</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr>
            <tr><td>6.x</td><td>LOC</td><td>BCT (s)</td><td>A</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr>
            <tr><td>6.x</td><td>OUT-POST</td><td>BEL (ms)</td><td>A</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr>
            <tr><td>6.x</td><td>OUT-PRE</td><td>BIDT (s) at P receiving peers</td><td>A</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr>
            <tr><td>6.x</td><td>OUT-POST</td><td>Fan-Out (messages per event)</td><td>A</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr>
            <tr><td>6.x</td><td>OUT-POST</td><td>Spread (ms)</td><td>A</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr>
            <tr><td>6.x</td><td>n/a (session event)</td><td>BEL, session event (ms): peer-down and peer-up, with Peer Down reason code</td><td>A</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr>
          </tbody>
        </table>

        <t>In addition, the following MUST be reported: the method used to
        detect completion (<xref target="sec-completion"/>) for each
        figure, with any discrepancy between the methods; the CPU and memory
        baseline, peak, and time-to-baseline; the number of lost, duplicated, and
        out-of-order BMP-reported routes; the BMP Fan-Out and Spread for the
        Adj-RIB-Out Views; the BGP and BMP session stability
        statistics; and, for scaling tests, the series X(k), SR(k), and IC(k).</t>
      </section>
    </section>

    <!-- ===================================================== -->
    <section anchor="sec-security" numbered="true">
      <name>Security Considerations</name>

      <t>Benchmarking activities as described in this document are limited
      to technology characterization of a DUT using controlled stimuli in a
      laboratory environment, with dedicated address space and the
      constraints specified here.  The test setup MUST be isolated from
      production networks and from the Internet.  Routes, AS numbers, and
      address space used in the tests MUST NOT be advertised outside of the
      test bed.  See also <xref target="RFC6815"/> on the use of
      benchmarking methodologies on production networks.</t>

      <t>The security considerations of BMP <xref target="RFC7854"/>,
      <xref target="RFC8671"/> and <xref target="RFC9069"/> apply.  The tests
      in this document, especially those with a stalled BMP Station and
      those with a high rate of churn, can exhaust the resources of the
      DUT, and could be used as a denial-of-service technique
      against a production device.  They MUST NOT be applied to production
      devices.  The BMP data used or obtained in the tests can reveal
      the complete routing state of the DUT, and it SHOULD be handled
      as sensitive when it comes from a non-lab configuration.</t>

      <t>This document does not define any protocol or protocol extension
      and has no direct impact on the security of the Internet.</t>
    </section>

    <section anchor="sec-iana" numbered="true">
      <name>IANA Considerations</name>
      <t>This document has no IANA actions.</t>
    </section>

  </middle>

  <back>
    <references>
      <name>References</name>

      <references anchor="sec-normative-refs">
        <name>Normative References</name>
        <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.2119.xml"/>
        <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.4098.xml"/>
        <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.4271.xml"/>
        <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.4724.xml"/>
        <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.7854.xml"/>
        <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.8174.xml"/>
        <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.8671.xml"/>
        <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.9069.xml"/>
      </references>

      <references anchor="sec-informative-refs">
        <name>Informative References</name>
        <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.1242.xml"/>
        <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.2544.xml"/>
        <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.6815.xml"/>
        <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.7747.xml"/>
        <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.8092.xml"/>
        <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.9972.xml"/>
      </references>
    </references>
  </back>
</rfc>
