rosotacom-dev 2.5.dev57__tar.gz → 2.5.dev60__tar.gz

{rosotacom_dev-2.5.dev57 → rosotacom_dev-2.5.dev60}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: rosotacom-dev
-Version: 2.5.dev57
+Version: 2.5.dev60
 Summary: Host CLI for ROS OTA communication sessions.
 Author: develNor
 License-Expression: BSD-3-Clause

{rosotacom_dev-2.5.dev57 → rosotacom_dev-2.5.dev60}/docs/findings/README.md

@@ -32,6 +32,8 @@ living verification hook.
 - [A bridge that learns types from the graph cannot use a transport that does not carry the graph](graph-derived-types-block-a-non-graph-transport.md) - over `zenoh_ros2dds` the payload topic never arrived in either run: the bridge routes a topic once a local reader exists and the reader waited for the bridge, leaving only `Pending=1` in a log.
 - [A Fast DDS datagram cap stops every message larger than it](fastdds-datagram-cap-drops-large-samples.md) - copying CycloneDDS's 1200 B fragment size into the Fast DDS OTA profile makes every sample above the cap never arrive, at 1200 B and at 8192 B, sync and async — while the 84 B heartbeat keeps flowing at 10 Hz.
 - [A lost keyframe silences exactly the next gop_size−2 deltas](keyframe-loss-kills-the-next-deltas.md) - libavcodec emits nothing for them, the last delta decodes with artifacts, the next keyframe restores clean decode; a lost delta never blocks decode — so the display hole is gop_size × frame period, deterministically.
+- [Any transport element in a Fast DDS profile doubles every sample on the wire](a-transport-element-doubles-every-fastdds-sample.md) - 366 datagrams for 180 samples against 183 without it; `<builtinTransports>DEFAULT` is a no-op selection and doubles too, so the trigger is the element's presence — removing it takes the shipped OTA profile from 2.10x the payload to 1.04x at identical delivery.
+- [Lifespan is enforced by one DDS reader and not the other](lifespan-is-enforced-by-one-reader-and-not-the-other.md) - under the same 20% oversubscription both stacks queue alike (472 and 433 delivered at ~1.1-1.2 s); with `lifespan: 0.7` CycloneDDS is unchanged and delivers 9.1% inside the bound, while Fast DDS drops to 4 delivered and never exceeds 570 ms — the policy decides shown-vs-suppressed, not how long the wait is.
 - [Oversubscription queues — it does not lose until the queue itself dies](oversubscription-queues-not-losses.md) - 16% over an emulated cap climbed one-way delay 340 → 3250 ms across twelve bins with zero loss in every one; the losses arrived only when the queue died, all at once.
 - [Best-effort reordering becomes reader-side loss, independent of history depth](reorder-becomes-reader-loss.md) - under pure delay jitter a Cyclone best-effort reader discards every overtaken sample (57% at 100 Hz under 50+-45 ms), delivery stays strictly monotonic, and depth 50 loses the same as depth 1.
 - [Jitter causes loss while bandwidth shortage builds latency](jitter-loss-bandwidth-latency.md) - 18 KB at 20 Hz separates jitter-induced loss from bandwidth-induced queueing.

rosotacom_dev-2.5.dev60/docs/findings/a-transport-element-doubles-every-fastdds-sample.md

@@ -0,0 +1,117 @@
+# Any Transport Element In A Fast DDS Profile Doubles Every Sample On The Wire
+
+## Claim
+
+A Fast DDS participant profile that carries **any** transport element transmits
+every sample twice. It is not a property of the transport that gets selected:
+`<builtinTransports>DEFAULT</builtinTransports>` asks for exactly the built-in
+set — a semantic no-op — and doubles the wire just as a full custom
+`<userTransports>` descriptor does. Removing the element is the entire fix.
+
+Measured against a 960 kbit/s payload, on the profile this repository ships:
+
+| participant profile carries | interface tx |
+|---|---|
+| `<userTransports>` + `<useBuiltinTransports>false` (as shipped) | 1741 kbit/s |
+| `<builtinTransports non_blocking="true">UDPv4` | 1741 kbit/s |
+| `<builtinTransports>UDPv4` | 1741 kbit/s |
+| `<builtinTransports>DEFAULT` | 1741 kbit/s |
+| **no transport element** | **1007 kbit/s** |
+
+Counted per datagram: **366** to the reader's data locator for **180** published
+samples, against **183** without the element — each carrying a full 11 951 B of
+UDP payload, to the same address and the same port. The reader discards the
+second copy by sequence number, so it is delivered exactly once and costs half
+the link.
+
+The consequence for any published wire-volume comparison is direct: a Fast DDS
+row measured with such a profile is a measurement of this defect, not of the
+middleware.
+
+## Setup
+
+- Host pair / topology: one host, two containers on a private docker bridge
+  (172.31.77.0/24), publisher and subscriber in separate containers; and the
+  same profile on the host network of a multi-homed machine (38 interfaces:
+  docker bridges, a VPN tun, WLAN) for the discovery checks.
+- Session: a two-process rclpy probe, not the session pipeline — one
+  `std_msgs/String` topic at 12000 B and 10 Hz, best_effort / KEEP_LAST depth 1,
+  publisher started after the subscriber. Bytes are read from
+  `/sys/class/net/eth0/statistics/tx_bytes` around the run; datagrams are counted
+  from a `tshark` capture with `ip.defragment:FALSE`, so the packet count and the
+  interface counter agree.
+- rosotacom SHA: `fastdds_tuned.xml.template` as of this commit, rendered by
+  `get_ota_xml.py`. ROS 2 Kilted, Fast DDS 3.2.4, `rmw_fastrtps_cpp` 9.3.4,
+  image `ros-communication:latest`.
+- Profile: no shaping for the wire-volume rows — the point is what the stack
+  offers, not what a link does with it.
+- Seed policy: deterministic; one run per cell, and the cells that share a
+  configuration agree to the byte (4 996 526 B across three different descriptor
+  shapes), which is what makes single runs readable here.
+
+## Evidence
+
+Evidence grade: interface counters plus a packet capture of the same runs, on
+both a synthetic profile and the rendered shipped one.
+
+The trigger is the element, not its contents. Four descriptor shapes, all 366
+datagrams for 180 samples: a bare `<transport_descriptor>` with nothing but
+`transport_id` and `type`; the same plus `maxInitialPeersRange`; the same with
+`<useBuiltinTransports>` set to `true` instead of `false`; and the same
+allowlisted to a single interface. The single-interface row also refutes the
+obvious alternative explanation — that the participant sends one copy per bound
+interface.
+
+On the rendered `fastdds_tuned.xml`, correcting for the ~40 messages published
+before discovery completes (a Fast DDS writer with no matched reader puts nothing
+on the wire, and this profile takes 3.9–4.2 s to match):
+
+| | interface tx | transmitted payload | ratio |
+|---|---|---|---|
+| as shipped | 6516 kB | 3109 kB | **2.10×** |
+| with the element removed | 3270 kB | 3145 kB | **1.04×** |
+
+Both delivered 245 of 245, at a sub-millisecond median age. Nothing is traded for
+the halving. For scale, CycloneDDS on the same bench and payload costs 963 kbit/s
+— **1.00×** — so removing the element brings Fast DDS from twice Cyclone's wire
+need to within 4% of it.
+
+Two properties the removed descriptor carried were checked rather than assumed,
+on the multi-homed host the profile exists for:
+
+- **The same-host path stays open.** 195 of 195 delivered, discovery in 4.13 s,
+  across 38 interfaces. The pinned `defaultUnicastLocatorList` /
+  `metatrafficUnicastLocatorList` already say what the interface allowlist said.
+- **Discovery survives the lost `maxInitialPeersRange`.** With 12 extra
+  participants started first, so the probe processes take high participant
+  indices: 195 of 195, discovery in 4.24 s, unchanged.
+
+What is genuinely given up: `<non_blocking_send>true</non_blocking_send>` goes
+with the descriptor, so a full send buffer blocks the writer instead of dropping.
+That is the smaller cost — a blocking send stalls only while the link is behind,
+where the duplicate costs half the link at all times.
+
+Verification: automated:
+`pytest tests/unit/test_ota_link_configs.py::test_no_fastdds_profile_names_a_transport`
+guards the deletion in every packaged Fast DDS profile. Manual, for the effect
+itself: two containers on a private bridge, one 12 kB/10 Hz stream, read
+`tx_bytes` around a 30 s run with and without a `<builtinTransports>DEFAULT</builtinTransports>`
+line in the participant profile — the ratio must be 2:1.
+
+## Status
+
+confirmed, 2026-08-19.
+
+## Publication notes
+
+This is a Fast DDS 3.2.4 / `rmw_fastrtps_cpp` 9.3.4 behaviour and is worth
+reporting upstream; the `DEFAULT` row is the reproducer to lead with, because it
+removes every question about what the profile was asking for.
+
+It retires the "Fast DDS's lower loss is bought with bandwidth" reading of this
+project's wire-volume table: roughly half of the excess over CycloneDDS was this,
+and the remainder is still unattributed. Pair with
+[lifespan-is-enforced-by-one-reader-and-not-the-other.md](lifespan-is-enforced-by-one-reader-and-not-the-other.md)
+— together they account for both columns on which Fast DDS looked like a
+different kind of stack: it costs about what Cyclone costs, and it delivers less
+because it refuses to deliver stale.

rosotacom_dev-2.5.dev60/docs/findings/lifespan-is-enforced-by-one-reader-and-not-the-other.md

@@ -0,0 +1,108 @@
+# Lifespan Is Enforced By One DDS Reader And Not The Other
+
+## Claim
+
+Under identical oversubscription the two DDS stacks queue identically — 472 and
+433 delivered of ~490 offered, at medians of 1146 ms and 1220 ms. The stacks
+diverge only when `lifespan` is switched on, and then completely: CycloneDDS is
+unchanged (472 delivered, 1143 → 1146 ms, **9.1%** of samples inside the 0.7 s
+bound they were published with), while Fast DDS drops from 433 delivered to
+**4**, at a 344 ms median, and **never delivers a sample older than 570 ms**.
+
+`rmw_cyclonedds` does not enforce lifespan at the reader; Fast DDS does. Neither
+enforcement shortens the delay — the backlog is below DDS and unrecallable
+either way. What lifespan decides is whether a late sample is **shown or
+suppressed**, and for an operator picture those are different failures: 7.9 Hz of
+video more than a second old, against 0.07 Hz of video that is current.
+
+The corollary matters for reading any oversubscription result: **Fast DDS's
+apparent collapse under a cap is not a transport failure.** With the policy off
+it carries the same load as CycloneDDS. The missing messages are its reader
+discarding what arrived too late.
+
+## Setup
+
+- Host pair / topology: one host, two containers on a private docker bridge
+  (172.31.77.0/24), publisher and subscriber in separate containers. Single host
+  on purpose: both sides read the same `CLOCK_REALTIME`, so a sample's age is
+  measured without a clock-offset term.
+- Session: a two-process rclpy probe, not the session pipeline — one
+  `std_msgs/String` topic at 12000 B and 10 Hz (960 kbit/s payload), QoS
+  best_effort / KEEP_LAST depth 1, publisher started after the subscriber so
+  discovery is not inside the measurement. The payload carries its own sequence
+  number and publish stamp.
+- rosotacom SHA: not involved; ROS 2 Kilted, Fast DDS 3.2.4 /
+  `rmw_fastrtps_cpp` 9.3.4, CycloneDDS via `rmw_cyclonedds_cpp`, image
+  `ros-communication:latest`.
+- Profile: `tc qdisc add dev eth0 root tbf rate 800kbit burst 64kb latency
+  2000ms` on the publisher container's egress — 20% below the offered payload,
+  with a 2 s buffer, i.e. a deliberate standing queue. 60 s per run.
+- Both middlewares configured with explicit unicast discovery (the bridge
+  carries unicast but not the DDS discovery multicast), so neither runs on
+  defaults.
+- Seed policy: no stochastic netem elements (rate only); one run per cell,
+  reported as the four-cell A/B rather than as a distribution.
+
+## Evidence
+
+Evidence grade: single-host container runs, per-sample age computed from a
+publish stamp inside the payload against one clock, sequence numbers giving loss
+independently of the age.
+
+| middleware | lifespan | delivered | age p50 | age p95 | age max | inside 0.7 s |
+|---|---|---|---|---|---|---|
+| CycloneDDS | 0.7 s | 472 | 1146 ms | 1449 ms | 1533 ms | **9.1%** |
+| CycloneDDS | 86400 s (off) | 472 | 1143 ms | 1449 ms | 1533 ms | — |
+| Fast DDS | 0.7 s | **4** | **344 ms** | 570 ms | **570 ms** | **100%** |
+| Fast DDS | 86400 s (off) | **433** | 1220 ms | 1459 ms | 1557 ms | — |
+
+Three readings:
+
+1. **The two stacks queue the same.** With the policy off, 472 against 433 and
+   1143 ms against 1220 ms. Interface accounting agrees: 811 and 812 kbit/s, both
+   pinned to the 800 kbit/s cap. Whatever separates them under `lifespan: 0.7` is
+   not capacity and not the transport.
+2. **CycloneDDS's A/B is a null result, and that is the finding.** 472/472,
+   1146/1143 ms — turning the policy off changes nothing measurable, because the
+   policy was doing nothing. Only 9.1% of what it delivers is inside the bound it
+   was published with.
+3. **Fast DDS's A/B is the opposite.** 4 against 433, and the max age under the
+   policy is 570 ms — below 700 ms, not approximately below it. The samples that
+   vanish are the ones the shaper delayed past their lifespan.
+
+The same pair of runs also shows the delay is a queueing discipline rather than a
+property of the path: replacing `tbf` with `htb` + `fq_codel` at the same
+800 kbit/s takes CycloneDDS from 472 delivered at 1146 ms to 8 delivered at
+**124 ms**, with 100% inside the lifespan. An AQM converts staleness into loss;
+it does not create capacity.
+
+Verification: manual: two containers on a private bridge, `tbf` as above on the
+publisher's egress, one 12 kB/10 Hz best_effort depth-1 stream, run once with
+`lifespan: 0.7` and once with it effectively off, for each RMW. The Cyclone pair
+must be indistinguishable and the Fast DDS pair must differ by two orders of
+magnitude in delivered count, with the `lifespan: 0.7` arm's maximum age below
+the lifespan.
+
+## Status
+
+confirmed, 2026-08-19.
+
+## Publication notes
+
+This corrects, from the reader side, the conclusion drawn in the operator
+harness's middleware comparison that lifespan is "indistinguishable for both DDS
+stacks" — that A/B's Fast DDS arm ran against a zero-byte profile. It is the
+enforcement half of
+[oversubscription-queues-not-losses.md](oversubscription-queues-not-losses.md):
+that finding shows the excess becomes delay rather than loss, this one shows what
+each reader then does with the delayed sample.
+
+For papers, the useful framing is that "loss" and "delivered" are not
+middleware-comparable numbers unless the lifespan enforcement is stated with
+them. A table that puts Fast DDS's 4 next to CycloneDDS's 472 without that column
+reads as a transport difference and is not one.
+
+Practical consequence: neither behaviour is usable at 20% oversubscription, so
+this is a choice about how to fail, not a fix. The fix is below DDS (offer fewer
+bytes than the link carries) or above it (receiver-side age gating that can skip
+rather than render).

{rosotacom_dev-2.5.dev57 → rosotacom_dev-2.5.dev60}/src/rosotacom/cli.py

@@ -2912,6 +2912,66 @@ def _ota_load_check(plan: OtaSmokePlan, *, dry_run: bool) -> None:
             print(f"+ {peer.name}: {summary}")
 
 
+_REACH_ATTEMPTS = 3
+_REACH_BACKOFF_SECONDS = 2.0
+_REACH_TIMEOUT_SECONDS = 20.0
+
+
+def _ota_reach_check(peer: OtaSmokePeer, *, dry_run: bool) -> None:
+    """Open the control channel to a peer, retrying a transport that flaked.
+
+    This is the only preflight step that retries, and the reason is the probe
+    itself: the remote command is ``true``. It cannot fail. Every non-zero exit
+    and every timeout here is therefore the transport — an ssh handshake that
+    timed out because the peer was loaded, a link that had not come up yet —
+    and never a condition on the machine that a second look would paper over.
+
+    The checks after this one deliberately do not retry. A container that is
+    already running is still running a moment later, and a peer somebody else is
+    saturating is still saturated; retrying those would convert a real refusal
+    into a delayed one. Retrying is right where the answer is stateless and the
+    failure is transient, and that is here alone.
+
+    Each attempt is printed. A run that needed three tries to reach a peer was
+    taken under conditions worth knowing about, so the artefact says so rather
+    than reporting a clean first-try connection.
+    """
+    if dry_run:
+        _ota_run(peer, "true", label=f"{peer.name}: reachable", dry_run=True, batch=True)
+        return
+
+    last_detail = ""
+    for attempt in range(1, _REACH_ATTEMPTS + 1):
+        suffix = "" if attempt == 1 else f" (attempt {attempt} of {_REACH_ATTEMPTS})"
+        try:
+            result = _ota_run(
+                peer,
+                "true",
+                label=f"{peer.name}: reachable{suffix}",
+                batch=True,
+                check=False,
+                timeout=_REACH_TIMEOUT_SECONDS,
+            )
+        except subprocess.TimeoutExpired:
+            last_detail = f"no answer within {_REACH_TIMEOUT_SECONDS:.0f}s"
+        else:
+            if result.returncode == 0:
+                if attempt > 1:
+                    print(
+                        f"! {peer.name}: reachable on attempt {attempt} of {_REACH_ATTEMPTS} — the transport was flaky"
+                    )
+                return
+            last_detail = ((result.stdout or "") + (result.stderr or "")).strip() or f"exit code {result.returncode}"
+        if attempt < _REACH_ATTEMPTS:
+            print(f"! {peer.name}: not reachable ({last_detail}); retrying")
+            time.sleep(_REACH_BACKOFF_SECONDS * attempt)
+
+    raise RuntimeError(
+        f"{peer.name}: not reachable via {_ota_peer_target(peer)} after "
+        f"{_REACH_ATTEMPTS} attempts.\nLast failure: {last_detail}"
+    )
+
+
 def _ota_preflight(
     plan: OtaSmokePlan,
     *,
@@ -2928,7 +2988,7 @@ def _ota_preflight(
     sudo_passwords = sudo_passwords or {}
     for peer in plan.peers.values():
         if _ota_peer_is_remote(peer):
-            _ota_run(peer, "true", label=f"{peer.name}: reachable", dry_run=dry_run, batch=True)
+            _ota_reach_check(peer, dry_run=dry_run)
         required = ["python3", "docker", "tar"]
         if require_tmux:
             required.append("tmux")

{rosotacom_dev-2.5.dev57 → rosotacom_dev-2.5.dev60}/src/rosotacom/resources/ws/ota_configs/fastdds_tuned.xml.template

@@ -26,44 +26,6 @@
 -->
 <dds xmlns="http://www.eprosima.com/XMLSchemas/fastRTPS_Profiles">
   <profiles>
-    <transport_descriptors>
-      <transport_descriptor>
-        <transport_id>rosotacom_ota_udp</transport_id>
-        <type>UDPv4</type>
-
-        <sendBufferSize>1048576</sendBufferSize>
-        <receiveBufferSize>4194304</receiveBufferSize>
-
-        <!-- Drop rather than block when the socket buffer is full: on a shaped
-             uplink the blocking default turns a link-layer backlog into
-             application-level latency. -->
-        <non_blocking_send>true</non_blocking_send>
-
-        <!-- An initial peer given without a port makes Fast DDS probe
-             participant indices 0..range on that address. The default is 4, and
-             an OTA domain carrying a domain bridge plus the IN and OUT splits
-             can hand out a higher index, so whichever process starts late is
-             never found. CycloneDDS probes 10; this matches it. -->
-        <maxInitialPeersRange>10</maxInitialPeersRange>
-
-        <!-- Two addresses, both load-bearing. #host_ip is the OTA address, and
-             pinning it is what keeps a multi-homed VPN host from announcing a
-             locator the peer cannot route to. 127.0.0.1 is the same-host half:
-             with a split OTA domain the stock `domain_bridge` process sits on
-             that domain too and speaks whatever RMW the local side uses, so
-             this participant has to be able to discover a peer on its own
-             machine. An allowlist of the OTA address alone takes the loopback
-             path away and the bridge and the OTA processes never see each
-             other — a failure that reads as a dead link and is not one. -->
-        <interfaces>
-          <allowlist>
-            <interface name="127.0.0.1" netmask_filter="OFF"/>
-            <interface name="#host_ip" netmask_filter="OFF"/>
-          </allowlist>
-        </interfaces>
-      </transport_descriptor>
-    </transport_descriptors>
-
     <participant profile_name="rosotacom_fastdds_tuned" is_default_profile="true">
       <rtps>
         <defaultUnicastLocatorList>
@@ -128,10 +90,46 @@
           </initialPeersList>
         </builtin>
 
-        <userTransports>
-          <transport_id>rosotacom_ota_udp</transport_id>
-        </userTransports>
-        <useBuiltinTransports>false</useBuiltinTransports>
+        <!-- No transport element here, deliberately, and this is load-bearing.
+
+             Measured on a two-container bench (2026-08-19, ROS 2 Kilted, Fast
+             DDS 3.2.4, rmw_fastrtps_cpp 9.3.4): a participant profile that
+             carries *any* transport element transmits every sample twice.
+
+               <userTransports> + <useBuiltinTransports>false   1741 kbit/s
+               <builtinTransports non_blocking="true">UDPv4     1741 kbit/s
+               <builtinTransports>UDPv4                         1741 kbit/s
+               <builtinTransports>DEFAULT                       1741 kbit/s
+               no transport element at all                      1007 kbit/s
+
+             against a 960 kbit/s payload. The DEFAULT row is what settles it:
+             that selection is a no-op — it asks for exactly the built-in set —
+             and it still doubles. Counted per datagram, 366 to the reader's data
+             locator for 180 published samples against 183 without the element,
+             each carrying a full 11 951 B of UDP payload to the same address and
+             port. The reader discards the second copy by sequence number, so it
+             buys nothing and costs half the link.
+
+             What removing it gives up, named rather than hidden:
+               * shared memory is enabled again. It is unused between machines,
+                 and between processes on one host it is faster than the loopback
+                 path this profile also names. fastdds_local.xml records that SHM
+                 was measured and is not the cause of the local-domain failure.
+               * <non_blocking_send>true</non_blocking_send> is gone with the
+                 descriptor, so a full send buffer blocks instead of dropping.
+                 That is the smaller cost: a blocking send stalls the writer for
+                 as long as the link is behind, where the duplicate costs half the
+                 link all the time.
+               * maxInitialPeersRange falls back to 4 from 10, and the interface
+                 allowlist is gone. The locator lists above already pin what this
+                 participant announces, which is what the allowlist was for. -->
+
+        <!-- What the removed descriptor's sendBufferSize/receiveBufferSize were.
+             These are participant-level and asymmetric on purpose: the receive
+             side absorbs a burst that arrives after a stall, the send side
+             should not hold one. -->
+        <sendSocketBufferSize>1048576</sendSocketBufferSize>
+        <listenSocketBufferSize>4194304</listenSocketBufferSize>
       </rtps>
     </participant>
 

rosotacom_dev-2.5.dev60/src/rosotacom/resources/ws/ota_configs/fastdds_unicast.xml.template

@@ -0,0 +1,93 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<dds xmlns="http://www.eprosima.com/XMLSchemas/fastRTPS_Profiles">
+  <profiles>
+    <participant profile_name="rosotacom_fastdds_unicast" is_default_profile="true">
+      <rtps>
+        <defaultUnicastLocatorList>
+          <locator>
+            <udpv4>
+              <address>#host_ip</address>
+              <port>0</port>
+            </udpv4>
+          </locator>
+        </defaultUnicastLocatorList>
+
+        <builtin>
+          <metatrafficUnicastLocatorList>
+            <locator>
+              <udpv4>
+                <address>#host_ip</address>
+                <port>0</port>
+              </udpv4>
+            </locator>
+          </metatrafficUnicastLocatorList>
+
+          <initialPeersList>
+            <locator>
+              <udpv4>
+                <address>#host_ip</address>
+                <port>0</port>
+              </udpv4>
+            </locator>
+            <!--peer-block--><locator>
+              <udpv4>
+                <address>#peer</address>
+                <port>0</port>
+              </udpv4>
+            </locator><!--/peer-block-->
+          </initialPeersList>
+        </builtin>
+
+        <!-- No transport element here, deliberately, and this is load-bearing.
+
+             Measured on a two-container bench (2026-08-19, ROS 2 Kilted, Fast
+             DDS 3.2.4, rmw_fastrtps_cpp 9.3.4): a participant profile that
+             carries *any* transport element transmits every sample twice.
+
+               <userTransports> + <useBuiltinTransports>false   1741 kbit/s
+               <builtinTransports non_blocking="true">UDPv4     1741 kbit/s
+               <builtinTransports>UDPv4                         1741 kbit/s
+               <builtinTransports>DEFAULT                       1741 kbit/s
+               no transport element at all                      1007 kbit/s
+
+             against a 960 kbit/s payload. The DEFAULT row is what settles it:
+             that selection is a no-op — it asks for exactly the built-in set —
+             and it still doubles. Counted per datagram, 366 to the reader's data
+             locator for 180 published samples against 183 without the element,
+             each carrying a full 11 951 B of UDP payload to the same address and
+             port. The reader discards the second copy by sequence number, so it
+             buys nothing and costs half the link.
+
+             What removing it gives up, named rather than hidden:
+               * shared memory is enabled again. It is unused between machines,
+                 and between processes on one host it is faster than the loopback
+                 path this profile also names. fastdds_local.xml records that SHM
+                 was measured and is not the cause of the local-domain failure.
+               * <non_blocking_send>true</non_blocking_send> is gone with the
+                 descriptor, so a full send buffer blocks instead of dropping.
+                 That is the smaller cost: a blocking send stalls the writer for
+                 as long as the link is behind, where the duplicate costs half the
+                 link all the time.
+               * maxInitialPeersRange falls back to 4 from 10, and the interface
+                 allowlist is gone. The locator lists above already pin what this
+                 participant announces, which is what the allowlist was for. -->
+      </rtps>
+    </participant>
+
+    <data_writer profile_name="rosotacom_fastdds_writer" is_default_profile="true">
+      <qos>
+        <data_sharing>
+          <kind>OFF</kind>
+        </data_sharing>
+      </qos>
+    </data_writer>
+
+    <data_reader profile_name="rosotacom_fastdds_reader" is_default_profile="true">
+      <qos>
+        <data_sharing>
+          <kind>OFF</kind>
+        </data_sharing>
+      </qos>
+    </data_reader>
+  </profiles>
+</dds>

{rosotacom_dev-2.5.dev57 → rosotacom_dev-2.5.dev60}/src/rosotacom/resources/ws/ros2src/com_py/com_py/playout_pacer_core.py

@@ -32,7 +32,10 @@ class PacerConfig:
     adaptive: bool = True
     percentile: float = 0.95  # of (arrival - stamp), window-local
     margin_ms: float = 40.0  # headroom above the tracked percentile
-    window: int = 400  # samples (~40 s at 10 Hz)
+    #: Samples (~40 s at 10 Hz). Bounds both the tracked percentile and the
+    #: delay floor, so how fast the schedule re-anchors after the path changes
+    #: is this number: a longer window is steadier and slower to recover.
+    window: int = 400
 
     def __post_init__(self) -> None:
         if not 0.0 < self.percentile < 1.0:
@@ -72,8 +75,18 @@ class PlayoutSchedule:
         """
         d = arrival_s - stamp_s
         self._d.append(d)
-        if self._d_floor is None or d < self._d_floor:
-            self._d_floor = d
+        # The minimum over the *window*, not over all time. An all-time floor
+        # can only fall, so any upward shift in `arrival - stamp` -- a clock
+        # step, an NTP correction, a route change, a looping replay -- leaves
+        # every later message past `stamp + floor + budget`, and the pacer
+        # degrades to a pass-through it can never come back from. Found on the
+        # fleet e2e, where a bag loop moved the delay by the bag's length and
+        # the centre's camera ran unpaced for the rest of the run with every
+        # panel healthy. Windowed, the same shift re-anchors within one window.
+        #
+        # O(window) per message: 400 floats at 10 Hz. A monotonic deque would
+        # make it O(1) and is not worth the machinery at these rates.
+        self._d_floor = min(self._d)
         release = stamp_s + self._d_floor + self.budget_ms / 1000.0
         if release < arrival_s:  # already late: pass through immediately
             release = arrival_s

{rosotacom_dev-2.5.dev57 → rosotacom_dev-2.5.dev60}/src/rosotacom_dev.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: rosotacom-dev
-Version: 2.5.dev57
+Version: 2.5.dev60
 Summary: Host CLI for ROS OTA communication sessions.
 Author: develNor
 License-Expression: BSD-3-Clause

{rosotacom_dev-2.5.dev57 → rosotacom_dev-2.5.dev60}/src/rosotacom_dev.egg-info/SOURCES.txt

@@ -61,6 +61,7 @@ docs/release.md
 docs/testing.md
 docs/work-items.md
 docs/findings/README.md
+docs/findings/a-transport-element-doubles-every-fastdds-sample.md
 docs/findings/cyclone-spdp-discovery-bursts.md
 docs/findings/delay-alone-no-loss.md
 docs/findings/fastdds-datagram-cap-drops-large-samples.md
@@ -68,6 +69,7 @@ docs/findings/graph-derived-types-block-a-non-graph-transport.md
 docs/findings/head-of-line-irregularity.md
 docs/findings/jitter-loss-bandwidth-latency.md
 docs/findings/keyframe-loss-kills-the-next-deltas.md
+docs/findings/lifespan-is-enforced-by-one-reader-and-not-the-other.md
 docs/findings/lighter-message-loss.md
 docs/findings/oversubscription-queues-not-losses.md
 docs/findings/reorder-becomes-reader-loss.md

{rosotacom_dev-2.5.dev57 → rosotacom_dev-2.5.dev60}/src/rosotacom_dev.egg-info/scm_file_list.json

@@ -46,6 +46,7 @@
     "docs/ci.md",
     "docs/ffmpeg-keyframes.md",
     "docs/findings/README.md",
+    "docs/findings/a-transport-element-doubles-every-fastdds-sample.md",
     "docs/findings/cyclone-spdp-discovery-bursts.md",
     "docs/findings/delay-alone-no-loss.md",
     "docs/findings/fastdds-datagram-cap-drops-large-samples.md",
@@ -53,6 +54,7 @@
     "docs/findings/head-of-line-irregularity.md",
     "docs/findings/jitter-loss-bandwidth-latency.md",
     "docs/findings/keyframe-loss-kills-the-next-deltas.md",
+    "docs/findings/lifespan-is-enforced-by-one-reader-and-not-the-other.md",
     "docs/findings/lighter-message-loss.md",
     "docs/findings/oversubscription-queues-not-losses.md",
     "docs/findings/reorder-becomes-reader-loss.md",

{rosotacom_dev-2.5.dev57 → rosotacom_dev-2.5.dev60}/src/rosotacom_dev.egg-info/scm_version.json

@@ -1,7 +1,7 @@
 {
   "tag": "2.4",
-  "distance": 57,
-  "node": "gd5c85e0fb2b514ef6c7827c2706cb96b8467fdea",
+  "distance": 60,
+  "node": "g3cd0295dbb4fd3e1533c5da3267cfa2ba8b2b58c",
   "dirty": false,
   "branch": "HEAD",
   "node_date": "2026-08-19"