topicforge 0.4.0__tar.gz → 0.5.1__tar.gz

{topicforge-0.4.0 → topicforge-0.5.1}/.gitignore

@@ -217,6 +217,12 @@ CLAUDE*.md
 # remain reproducible. Tracked, not part of sdist.
 !/docs/projet-file/references/
 !/docs/projet-file/references/**
+# Archive of superseded strategic artifacts (old audit reports, dated
+# launch-post drafts). Kept in git so the historical context survives
+# but tucked away so the active `projet-file/` folder stays focused on
+# what is currently load-bearing.
+!/docs/projet-file/archive/
+!/docs/projet-file/archive/**
 /docs/assets/screencast-raw/
 
 

{topicforge-0.4.0 → topicforge-0.5.1}/CHANGELOG.md

@@ -5,7 +5,331 @@ All notable changes to TopicForge are documented in this file.
 The format is based on [Keep a Changelog 1.1.0](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning 2.0.0](https://semver.org/spec/v2.0.0.html).
 
-## [0.4.0]
+## [Unreleased]
+
+## [0.5.1] - 2026-08-22
+
+### Fixed (hotfix — MCP SDK 2.0 incompatibility)
+
+- **Hard-pinned `mcp < 2` — TopicForge was uninstallable from 2026-07-28
+  to 2026-08-22.** The MCP Python SDK released `2.0.0` on 2026-07-28
+  alongside the `2026-07-28` protocol revision. That release removes the
+  `mcp.server.fastmcp` module entirely (`mcp/server/` now ships `auth`,
+  `lowlevel`, and `mcpserver`), so `server/app.py`'s
+  `from mcp.server.fastmcp import FastMCP` raises `ImportError` against
+  it. The dependency was declared as an unbounded `mcp>=1.0.0`, so every
+  fresh `pip install topicforge` resolved to `2.0.0` and produced a
+  server that could not start. Local development and CI both masked the
+  break — the dev environment had `mcp 1.27.1` already installed, and
+  the last CI run predates the SDK release. Pinning `<2` restores
+  installability on the 1.x line; migrating to the 2.x API
+  (`FastMCP` -> `MCPServer`, transport options moved from the
+  constructor to `.run()`, stateless protocol) is tracked separately and
+  is deliberately **not** bundled into this hotfix.
+
+### Note on scope
+
+- This release also publishes the external-audit work (Lots 0-5,
+  2026-07-08) that had accumulated under `[Unreleased]`. The Cyclone
+  `take_iter` -> `read_iter` change documented below is still **validated
+  by static analysis only** (`ruff` + `py_compile`); it has not been
+  exercised against a real multi-vendor DDS bus. It ships here because
+  leaving the package uninstallable was the larger harm — a broken
+  install affects every user, while this change can only affect users
+  running the Cyclone backend against a live bus. Real-bus validation
+  remains open.
+
+### Changed
+
+- **DDS pure logic extracted for testability (Lot 0, external audit
+  2026-07-08).** The QoS-profile normalizers (`_cyclone_qos_to_profile` /
+  `_fast_qos_to_profile`) and the discovery-sample field extractors
+  (`_extract_guid` / `_extract_vendor_id` / `_extract_hostname` /
+  `_extract_topic_name` / `_is_removal`) were moved out of
+  `adapters/dds_cyclone/adapter.py` and `adapters/dds_fast/adapter.py` —
+  which import their vendor binding at module top level and were therefore
+  never exercised by the test suite — into the binding-free
+  `adapters/common/qos_normalize.py` and `adapters/common/dds_introspection.py`.
+  The adapters import them back under their original private names, so every
+  call site is unchanged (verified by `ruff check` static analysis, since the
+  adapters are not importable without their SDKs). `fast_qos_to_profile`
+  takes the binding's int→str enum maps as parameters so it stays
+  import-free. This is the highest-value item from the audit: the QoS
+  normalization feeds `detect_qos_mismatches` (the flagship DDS diagnostic)
+  and was previously untestable and untested.
+
+### Added
+
+- `tests/test_dds_qos_normalization.py` and `tests/test_dds_introspection.py`
+  drive the extracted logic with synthetic duck-typed objects (no
+  `cyclonedds` / `fastdds` needed), including regression guards for the
+  "renamed policy key silently yields no QoS profile → no mismatch ever
+  reported" failure mode. The extracted modules are now ~91–92% covered.
+- `pytest-cov` and `rosbags` added to the `[dev]` extra, plus `[tool.coverage]`
+  config with a `fail_under = 85` floor (the two binding-only adapter shells
+  are `omit`ted as structurally unreachable without their SDKs). Adding
+  `rosbags` un-skips the real `.db3` bag-analysis I/O test.
+
+### Fixed
+
+- `tests/test_bag_service.py` bag-generation helper updated for the current
+  `rosbags` API (`Writer(..., version=Writer.VERSION_LATEST)`; typestore keyed
+  by `std_msgs/msg/String`, not `std_msgs__msg__String`). The test was
+  previously auto-skipped and had gone silently stale — un-skipping it in CI
+  surfaced the drift.
+- **`topic_metrics` frequency was wrong (Lot 2, audit C5).** It divided the
+  sample count by `(now − oldest_sample)` — folding in idle time since the
+  last peek — and counted N intervals instead of N−1. Now measured as
+  `(N−1) / (newest − oldest)` over the samples' own arrival span; samples
+  surfaced by a single opportunistic peek share one timestamp (span 0) and
+  correctly yield `frequency_hz_observed = null` instead of a fabricated rate.
+- **`topic_metrics` sequence-gap count exploded on multi-writer topics,
+  publisher restarts, and counter wrap (Lot 2, audit C6).** Gaps are now
+  counted per writer (new best-effort `MetricsSample.writer_guid`), so
+  independent writers' counter offsets are not read as phantom gaps, and a
+  single jump wider than 10 000 is treated as a reset/wrap discontinuity
+  rather than that many losses.
+- **QoS Deadline false negative (Lot 2, audit C3/P1-3).**
+  `detect_qos_mismatches` now flags a reader that requests a finite Deadline
+  against a writer that offers none — an absent deadline is the infinite
+  (loosest) period and cannot satisfy a finite request. The previous rule
+  required both sides non-null and silently missed this incompatibility.
+- **`LifecycleBuffer` participant map was unbounded (Lot 3, audit P1-4 / M1 /
+  P1).** Only the event ring was capped; the participant dict grew one entry
+  per GUID ever seen (a churny bus mints a fresh RTPS GUID on each node
+  restart), and `list_participants` returned every tombstone forever. Now
+  capped at `MAX_PARTICIPANTS = 4096`, evicting `"left"` tombstones first then
+  the oldest-inserted entry — the docstring's "Bounded" claim is now true.
+- **`MetricsBuffer` topic map was unbounded (Lot 3, audit P2-5).** Per-topic
+  rings were capped but the number of topic keys was not; now capped at
+  `MAX_TOPICS = 4096`, oldest-inserted topic evicted on overflow.
+- **OpenDDS stub `is_available()` always returns False (Lot 4, audit S1).** A
+  stub that advertised availability (when a `pyopendds` module happened to be
+  importable) could be auto-selected by the factory, after which every tool
+  call raised. Now consistent with the Dust stub.
+- **`iter_field_names` mis-decoded a string `__slots__` (Lot 4, audit C2).**
+  `__slots__ = "value"` was exploded into `['v','a','l','u','e']`; a bare
+  string slot is now treated as a single field name.
+- **`decode_field_value` recursion is depth-capped at 32 (Lot 4, audit M6).**
+  A pathologically deep decoded object graph collapses to `repr()` instead of
+  risking `RecursionError`.
+- **`_encode_raw_bytes` slices before hex-encoding (Lot 4, audit M5).** A large
+  raw payload no longer allocates its full 2×-size hex string only to truncate
+  it to the 4096-char preview.
+
+### Added (Lot 4 — test hardening)
+
+- End-to-end test that a failing tool call surfaces as an MCP error
+  (`ToolError`) rather than being masked as a success — pins the thin-handler
+  contract (CLAUDE.md §8, audit test-gap #4).
+- Test pinning that every canonical vendor tag is a valid
+  `ParticipantInfo`/`ParticipantEvent.vendor` Literal (guards against
+  vendor-map ↔ schema drift, audit P2-3). Scenario allowlist `_KNOWN_TOOLS`
+  now includes the 11th tool `peek_bag_samples`.
+
+### Removed (Lot 4)
+
+- Dead `annotate_full` / `annotate_partial` imports and the `_ = (...)`
+  unused-suppressor from `services/bag_service.py`.
+
+### Documentation (Lot 1 — reconcile the strategic source of truth)
+
+- **`docs/product-plan.md` realigned on the shipped 11-tool surface (audit
+  M6/P1-6).** §1 and §4 said "five typed tools today" / DDS "roadmapped"
+  while six DDS/observability tools had shipped across v0.2.0–v0.4.0. §11's
+  risk register carried a self-imposed governance gate — "any 9th tool needs
+  an explicit re-scope discussion documented in this register before code
+  lands" — that was crossed during v0.4.0 without the discussion being
+  recorded. Added a retroactive re-scope decision closing that gap: the three
+  ceiling-breaking tools are accepted, the new ceiling is 11 tools, a 12th
+  needs a documented re-scope.
+- **User-topic `raw` decode honesty (audit C1).** README and the
+  `peek_dds_samples` tool description no longer imply the `raw` fallback
+  preserves the payload in `_raw_bytes_hex` — on the current user-topic raw
+  path that field is empty (a `raw` status means "present but not decoded").
+  Capturing the on-wire CDR bytes is stated as roadmapped rather than done.
+
+### Changed (Lot 5 — DDS adapter deduplication)
+
+- **QoS-mismatch endpoint pairing deduplicated (audit D1/M7/P2).** The ~40
+  identical lines in each adapter's `detect_qos_mismatches` (group endpoints by
+  topic, pair reader × writer, build `MismatchReport`) moved to the binding-free
+  `common/qos_endpoints.detect_mismatches_across_endpoints`, unit-tested without
+  a binding. Each writer's QoS profile is now parsed once per topic instead of
+  once per reader (fixes the O(readers × writers) re-parse). Both adapters
+  delegate to it.
+- **Shared `validate_domain_id` (`common/dds_helpers`).** The identical 0..232
+  bound check in all four DDS adapter constructors (Cyclone, Fast, OpenDDS,
+  Dust) is now defined once.
+- **Cyclone discovery/sample reads switched from `take_iter` to `read_iter`
+  (audit A1/P1-5).** Destructive `take` drained the builtin discovery cache,
+  risking spurious lost / re-discovered participant flapping across polls;
+  `read` is non-destructive — the correct choice for read-only observability.
+  ⚠️ **Requires real-bus validation on `scripts/integration/` before release**:
+  the read-vs-take semantics cannot be exercised without `cyclonedds` installed
+  (the adapter is not importable in the unit environment; these edits are
+  validated only by `ruff` static analysis + `py_compile`).
+
+Baseline: 399 → 485 passed, 24 → 23 skipped, ruff clean, coverage 89.12%.
+
+## [0.5.0] - 2026-05-21
+
+### Sprint v0.5.0 — Polish + validation (pre-marketing-publication)
+
+> Branch `feat/v0.5.0-polish-and-validation`. Three sub-milestones
+> (5.1 real validation, 5.2 audit closure + error polish, 5.3 docs +
+> examples + repo polish). No new MCP tool, no schema change. Last
+> sprint before marketing publication ; the next version bump is
+> the maintainer's manual `release` commit + tag.
+
+#### Changed (sub-milestone 5.1 — Real validation)
+
+- **CI matrix expanded** (`.github/workflows/ci.yml`) from
+  `ubuntu-latest × {3.11, 3.12}` to `{ubuntu-latest, windows-latest} ×
+  {3.11, 3.12, 3.13}` (6 cells, `fail-fast: false`). Windows-latest
+  coverage closes the largest unverified surface — TopicForge's primary
+  dev environment is Windows per `CLAUDE.md §7` and was previously only
+  hand-validated. Python 3.13 added now that wheels are stable across
+  the dependency footprint.
+- **`pyproject.toml` classifiers** widened with `Programming Language ::
+  Python :: 3.13`.
+
+#### Changed (sub-milestone 5.2 — AdapterError polish)
+
+- **`DDS_ONLY_ERROR_MSG`** (`adapters/common/dds_helpers.py`) rewritten
+  with the v0.4.0 `CompositeAdapter` remediation path explicit, and
+  with every affected ROS2 tool name listed inline. Substring
+  `"DDS observability only"`, `"TOPICFORGE_DDS_BACKEND"`,
+  `"TOPICFORGE_MODE"` preserved — existing `pytest.raises(match=...)`
+  contracts honored. New token assertions added :
+  `"CompositeAdapter"`, the 4 affected tool names.
+- **CycloneDDS adapter errors** (`adapters/dds_cyclone/adapter.py`)
+  enriched with the underlying exception type, the active domain id,
+  the topic name (where relevant), and common-cause diagnostic hints
+  (`CYCLONEDDS_URI` misconfiguration, multicast firewall, domain
+  mismatch). Three sites : participant discovery, endpoint discovery,
+  sample peek.
+- **Fast DDS participant-init error** (`adapters/dds_fast/adapter.py`)
+  enriched with an ABI-mismatch diagnostic — Fast DDS 2.6.x Python
+  binding wheels frequently desynchronize with system-installed Fast
+  DDS native libraries, and the v0.4.0 wording masked the cause behind
+  a bare `"returned None"`. New message points at the pyproject pin
+  (`fastdds>=2.6.1,<3`) and the `FastDDS_DEFAULT_PROFILES_FILE` env
+  var as the two likely culprits.
+- **`BagService` IO errors** (`services/bag_service.py`) wrap the
+  rosbags-side exception class name into the AdapterError message so
+  the LLM caller can pivot on `PermissionError` / `IsADirectoryError`
+  / etc. without inspecting `__cause__`.
+- **`_ROSBAGS_REQUIRED_MSG`** rewording — clarifies that `analyze_bag`
+  has a v0.3.0 text-parse fallback while `peek_bag_samples` does not.
+
+#### Closed (sub-milestone 5.2 — audit triage)
+
+- **`docs/projet-file/audit-followup-triage-v0.2.0.md` refreshed**
+  against the current tree (was last touched pre-v0.3.0). Strict
+  B-class : 3 items now CLOSED (B6 in v0.2.0, B9 in v0.3.0, B10 in
+  v0.5.0). 6 items remain DEFER (B1-B5 hosted-context security
+  hardening ; B7, B8 wire-contract decisions for v0.6+).
+- **`TODO(roadmap, audit-2026-05-14)` at `services/inspector.py:76`
+  retired as WONT-FIX-by-design** — the `list_topics` Inspector gate
+  is intentionally empty (no MCP-level args to validate). The comment
+  block is now a permanent design note rather than a roadmap pointer.
+
+#### Added (sub-milestone 5.2 — regression tests)
+
+- **5 new tests** in `tests/test_dds_helpers.py` and
+  `tests/test_bag_service.py` (regex-token assertions, not exact
+  wording) — pin the polished message contract without locking the
+  exact prose. Baseline grows from 394 to 399 passed ; 24 skipped
+  unchanged ; ruff clean.
+
+#### Changed (sub-milestone 5.3 — Documentation cascade)
+
+- **`README.md`** — CI badge added ; tagline refreshed from "v0.3.0"
+  to "v0.4.0" framing emphasising observability + bag analysis ; the
+  3-row DDS tool mini-table grew into a 6-row table listing every
+  DDS / observability tool with its sub-milestone of origin ;
+  `peek_dds_samples` scope rewritten around `_decode_status` (full /
+  partial / raw) ; telemetry contract field description switched from
+  "five MVP tools" to "eleven MCP tools" ; `TOPICFORGE_DDS_BACKEND`
+  Literal values listed in the config reference ; Roadmap section
+  pruned of items that shipped in v0.4.0 (Composite adapter, XTypes
+  Cyclone push).
+- **`docs/DDS_QUICKSTART.md`** — header bumped to v0.4.0+ ; §4
+  "Single-adapter limitation (v0.3.0)" replaced by "Composite adapter
+  (v0.4.0 Phase 1+)" with the new routing table ; §5 documents the
+  v0.4.0 Phase 1.5 best-effort XTypes story (no more
+  `AdapterError` on user topics) ; §6 "What's next" pruned of shipped
+  items ; §7 Troubleshooting updated.
+- **`docs/TESTING.md`** — "five MCP tools" → "eleven MCP tools" in the
+  three documented occurrences ("Pick your path" table row, the lead
+  paragraph, Path 1 header). New v0.4.0 tool-surface callout above
+  the path picker.
+
+#### Added (sub-milestone 5.3 — new docs and examples)
+
+- **`docs/MIGRATION_v0.3_to_v0.4.md`** (new, sibling to the existing
+  `MIGRATION_v0.2_to_v0.3.md`). 8 sections : new tools, env vars and
+  extras, soft-breaking schema widening (`ParticipantInfo` +4 fields,
+  `BagAnalysis` +4 fields, `HealthReport.ros_backend`, `dds_backend`
+  widening, `AdapterName` widening, new `TopicMetrics` /
+  `ParticipantEvent` schemas), `CompositeAdapter`, `peek_dds_samples`
+  user-topic story, protocol expansions, plus a quick checklist.
+- **`docs/TROUBLESHOOTING.md`** (new). One section per polished
+  AdapterError message, plus the cross-cutting "ROS2 CLI not found on
+  PATH" / `auto` fallback to mock case. Each section quotes the
+  message and lists the diagnostics in order.
+- **`examples/`** (new). 4 mock-mode-runnable walkthroughs covering
+  the headline value props :
+  - `01-discover-ros2-stack.md` — `health_check` + `list_topics` +
+    `get_topic_info` + `sample_messages`
+  - `02-debug-qos-mismatch.md` — `list_participants` +
+    `detect_qos_mismatches` + `peek_dds_samples` (canonical
+    Reliability mismatch story)
+  - `03-analyze-recording.md` — `analyze_bag` + `peek_bag_samples`
+    post-mortem inspection
+  - `04-monitor-topic-frequency.md` — `topic_metrics` +
+    `participant_events` with the opportunistic-fill caveat
+  Each example pairs an MCP-client prompt with the expected tool
+  calls and a short LLM-facing synthesis.
+
+#### Added (sub-milestone 5.3 — repo polish)
+
+- **`CONTRIBUTING.md`** (new). What contributions land easily vs hard,
+  development setup, the `make check` contract, mock-first
+  development convention, layer separation, pure-parser convention,
+  commit conventions.
+- **`SECURITY.md`** (new). Local-trust threat model, the
+  read-only-by-architecture stance, vulnerability disclosure email,
+  response SLAs, supported version policy.
+- **`.github/ISSUE_TEMPLATE/bug_report.yml`** (new) — structured form
+  with version, mode, OS, Python, repro, env vars, troubleshooting
+  check.
+- **`.github/ISSUE_TEMPLATE/feature_request.yml`** (new) —
+  problem-first framing, tier disambiguation (OSS / Pro), explicit
+  read-only-by-architecture acknowledgment.
+- **`.github/ISSUE_TEMPLATE/config.yml`** (new) — disables blank
+  issues, links to `SECURITY.md`, `docs/TROUBLESHOOTING.md`,
+  `docs/DDS_QUICKSTART.md`.
+- **`.github/PULL_REQUEST_TEMPLATE.md`** (new) — summary, test plan,
+  backward-compatibility checklist (covers the 11-tool cap, schema
+  additive-only invariant, telemetry 6-field contract, env var docs,
+  public API removal flag).
+
+#### Notes
+
+- **Backward compat preserved.** Zero schema changes, zero new tools,
+  zero env-var renames. The `DDS_ONLY_ERROR_MSG` substring tokens
+  pinned by existing tests (`"DDS observability only"`,
+  `"TOPICFORGE_DDS_BACKEND"`, `"TOPICFORGE_MODE"`) are preserved
+  intact ; v0.4.0 producers and clients keep working byte-for-byte.
+- **CHANGELOG entry deferred to release time.** This branch leaves
+  `pyproject.toml` at `0.4.0`, `__version__` at `"0.4.0"`, and the
+  `## [Unreleased]` heading populated with the polish notes above.
+  The version bump and `v0.5.0` tag are the maintainer's manual
+  steps after final review.
+
+## [0.4.0] - 2026-05-15
 
 ### Sprint v0.4.0 — Phase 3 (bag analysis multi-format)
 
@@ -554,7 +878,10 @@ Initial MVP release of TopicForge — ROS Topic Inspector & Bag Analyzer MCP ser
 - The write path (publishing, commanding robots) is intentionally out of scope for the MVP.
 - `analyze_bag` in live mode parses `ros2 bag info` text output; deeper anomaly detection remains mock-only for now.
 
-[Unreleased]: https://github.com/yaniswav/TopicForge/compare/v0.3.0...HEAD
+[Unreleased]: https://github.com/yaniswav/TopicForge/compare/v0.5.1...HEAD
+[0.5.1]: https://github.com/yaniswav/TopicForge/compare/v0.5.0...v0.5.1
+[0.5.0]: https://github.com/yaniswav/TopicForge/compare/v0.4.0...v0.5.0
+[0.4.0]: https://github.com/yaniswav/TopicForge/compare/v0.3.0...v0.4.0
 [0.3.0]: https://github.com/yaniswav/TopicForge/compare/v0.2.0...v0.3.0
 [0.2.0]: https://github.com/yaniswav/TopicForge/compare/v0.1.2...v0.2.0
 [0.1.2]: https://github.com/yaniswav/TopicForge/compare/v0.1.1...v0.1.2

{topicforge-0.4.0 → topicforge-0.5.1}/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.4
+Metadata-Version: 2.5
 Name: topicforge
-Version: 0.4.0
+Version: 0.5.1
 Summary: ROS Topic Inspector & Bag Analyzer MCP server for AI agents
 Project-URL: Homepage, https://github.com/yaniswav/TopicForge
 Project-URL: Repository, https://github.com/yaniswav/TopicForge
@@ -37,8 +37,9 @@ Classifier: Operating System :: OS Independent
 Classifier: Programming Language :: Python :: 3
 Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
 Requires-Python: >=3.11
-Requires-Dist: mcp>=1.0.0
+Requires-Dist: mcp<2,>=1.0.0
 Requires-Dist: pydantic>=2.6
 Provides-Extra: all
 Requires-Dist: cyclonedds>=0.10; extra == 'all'
@@ -62,18 +63,21 @@ Requires-Dist: fastdds<3,>=2.6.1; extra == 'dds-fast'
 Provides-Extra: dds-opendds
 Requires-Dist: pyopendds>=0.1; extra == 'dds-opendds'
 Provides-Extra: dev
+Requires-Dist: pytest-cov>=5.0; extra == 'dev'
 Requires-Dist: pytest>=8.0; extra == 'dev'
+Requires-Dist: rosbags>=0.9; extra == 'dev'
 Requires-Dist: ruff>=0.4; extra == 'dev'
 Description-Content-Type: text/markdown
 
 # TopicForge
 
 [![PyPI version](https://img.shields.io/pypi/v/topicforge.svg)](https://pypi.org/project/topicforge/)
+[![CI](https://github.com/yaniswav/TopicForge/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/yaniswav/TopicForge/actions/workflows/ci.yml)
 [![Python versions](https://img.shields.io/pypi/pyversions/topicforge.svg)](https://pypi.org/project/topicforge/)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://github.com/yaniswav/TopicForge/blob/main/LICENSE)
 [![Read-only by architecture](https://img.shields.io/badge/safety-read--only_by_architecture-2563eb)](https://github.com/yaniswav/TopicForge#security-model)
 
-> **The safety-first read-only MCP for ROS2 robotics — now with multi-vendor OMG DDS-RTPS observability (v0.3.0).** TopicForge lets AI agents inspect your ROS2 graph, ROS bag files, and (since v0.2.0) the raw DDS layer beneath ROS, without ever publishing back to the bus. v0.3.0 ships two OSS Python adapters — Eclipse CycloneDDS and eProsima Fast DDS — each joining the bus as a read-only DDS-RTPS participant that observes **every conformant vendor on the wire** (RTI Connext, OpenDDS, CoreDX, Dust DDS in Rust, etc.) via the OMG protocol guarantee. See [`docs/dds-interop-matrix.md`](docs/dds-interop-matrix.md) for the canonical multi-vendor positioning and the OMG May 2025 interop reference.
+> **The safety-first read-only MCP for ROS2 robotics — observability + multi-vendor OMG DDS-RTPS (v0.4.0).** TopicForge lets AI agents inspect your ROS2 graph, recorded bag files, and the raw DDS layer beneath ROS — without ever publishing back to the bus. **Eleven typed read-only tools** (5 ROS2 graph + 3 DDS + 3 observability/bag) share a single Pydantic envelope so an LLM caller reads one schema across the whole stack. v0.4.0 adds participant lifecycle tracking (`participant_events`), temporal metrics (`topic_metrics`), and post-mortem bag sample peek (`peek_bag_samples`) on top of v0.3.0's two OSS Python DDS participants — Eclipse CycloneDDS and eProsima Fast DDS — each observing **every conformant vendor on the wire** (RTI Connext, OpenDDS, CoreDX, Dust DDS in Rust, etc.) via the OMG protocol guarantee. See [`docs/dds-interop-matrix.md`](docs/dds-interop-matrix.md) for the canonical multi-vendor positioning and the OMG May 2025 interop reference.
 
 TopicForge is a production-minded MCP (Model Context Protocol) server that lets AI agents — such as Claude — inspect ROS2 topics, analyze ROS bag files, and (since v0.2.0) observe the raw DDS layer through a clean, structured tool interface. It is read-only by **architecture**, not by configuration: there is no write path to misconfigure, no permission system to audit, no liability conversation to have. The MCP client can see the robot stack; it cannot touch it.
 
@@ -239,21 +243,24 @@ CoreDX, InterCOM) ship under the optional `topicforge-pro` package with
 BYO vendor license. See [`docs/pro.md`](docs/pro.md) for the early-access
 slot and pricing terms ; nothing is collected today.
 
-Three new MCP tools (in addition to the five ROS2 tools above) :
+Six DDS / observability tools (in addition to the five ROS2 tools above) :
 
-| Tool                    | Purpose                                                                          |
-| ----------------------- | -------------------------------------------------------------------------------- |
-| `list_participants`     | DDS participants discovered on a domain, with vendor and hostname                |
-| `detect_qos_mismatches` | Reader/writer QoS incompatibilities preventing communication on a topic          |
-| `peek_dds_samples`      | Recent samples on a raw DDS topic (distinct from `sample_messages` on ROS2 graph)|
+| Tool                    | Since   | Purpose                                                                                                        |
+| ----------------------- | ------- | -------------------------------------------------------------------------------------------------------------- |
+| `list_participants`     | v0.2.0  | DDS participants discovered on a domain, with vendor, hostname, and (v0.4.0) lifecycle fields                  |
+| `detect_qos_mismatches` | v0.2.0  | Reader/writer QoS incompatibilities preventing communication on a topic                                        |
+| `peek_dds_samples`      | v0.2.0  | Recent samples on a raw DDS topic — v0.4.0 adds best-effort XTypes user-topic decode (distinct from `sample_messages` on ROS2 graph) |
+| `participant_events`    | v0.4.0  | Lifecycle stream — `discovered` / `lost` participant events over a configurable window                         |
+| `topic_metrics`         | v0.4.0  | Temporal metrics — observed frequency, sequence gaps, latency p50/p95/p99 over a sliding window                |
+| `peek_bag_samples`      | v0.4.0  | Post-mortem inspection — decoded samples from a recorded `.mcap` / `.db3` / `.bag` file                        |
 
-**Composite adapter (v0.4.0 Phase 1+).** When `TOPICFORGE_MODE=live` is paired with a DDS backend (`cyclone`, `fast`, …), TopicForge instantiates **both** a ROS2 CLI adapter and the chosen DDS adapter and routes per-tool category — the 5 ROS2 tools hit the CLI, the 3 DDS tools (+ `participant_events` from Phase 1) hit the DDS backend. ROS2-only or DDS-only setups still work — the missing half is skipped and the present half serves what it can. The mock backend continues to expose all 9 tools against deterministic fixtures for local development.
+**Composite adapter (v0.4.0 Phase 1+).** When `TOPICFORGE_MODE=live` is paired with a DDS backend (`cyclone`, `fast`, …), TopicForge instantiates **both** a ROS2 CLI adapter and the chosen DDS adapter and routes per-tool category — the 5 ROS2 tools hit the CLI, the DDS / observability tools hit the DDS backend. ROS2-only or DDS-only setups still work — the missing half is skipped and the present half serves what it can. The mock backend continues to expose all 11 tools against deterministic fixtures for local development.
 
-**v0.3.0 limitation — `peek_dds_samples` scope.** Full-fidelity on the 4 builtin DCPS topics (`DCPSParticipant`, `DCPSSubscription`, `DCPSPublication`) ; arbitrary user topics raise an `AdapterError` pointing at the v0.3.x XTypes/IDL roadmap. The other two DDS tools (`list_participants`, `detect_qos_mismatches`) work end-to-end on any user-topic deployment.
+**`peek_dds_samples` payload shape (v0.4.0 Phase 1.5).** Full-fidelity on the 4 builtin DCPS topics (`DCPSParticipant`, `DCPSSubscription`, `DCPSPublication`). Arbitrary user topics return best-effort decoded samples with a `_decode_status` annotation : `"full"` (every IDL field decoded — currently a v0.4.0+ Cyclone XTypes path), `"partial"` (some fields decoded, others opaque), or `"raw"` (binding could not resolve the dynamic XTypes). The diagnostic key `_decode_note` carries a short explanation when the status is non-`full`. The wire shape is identical across Cyclone and Fast backends. **Caveat (v0.5.x):** on the current user-topic *raw* path `_raw_bytes_hex` is **empty** — a `"raw"` status means "topic present on the bus but not decoded", not "here are the serialized bytes to re-decode". Capturing the on-wire CDR bytes into the fallback is roadmapped ; until then use the Cyclone full/partial XTypes path for actual user-topic payloads. The 4 builtin DCPS topics are unaffected (always structured).
 
 **`RTI Connext`** is v0.4.0+ Pro tier (BYO license — see `docs/pro.md`).
 
-**Full 5-minute walkthrough** — backend selection, the canonical QoS-mismatch debugging scenario, troubleshooting — lives in [`docs/DDS_QUICKSTART.md`](docs/DDS_QUICKSTART.md). Migration from v0.2.0 in [`docs/MIGRATION_v0.2_to_v0.3.md`](docs/MIGRATION_v0.2_to_v0.3.md).
+**Full 5-minute walkthrough** — backend selection, the canonical QoS-mismatch debugging scenario, troubleshooting — lives in [`docs/DDS_QUICKSTART.md`](docs/DDS_QUICKSTART.md). Migration history : [v0.2 → v0.3](docs/MIGRATION_v0.2_to_v0.3.md), [v0.3 → v0.4](docs/MIGRATION_v0.3_to_v0.4.md).
 
 ### Configure with Claude Desktop
 
@@ -308,7 +315,7 @@ make check    # both, plus tests (CI bundle)
 | `TOPICFORGE_LOG_LEVEL`      | `INFO`  | `DEBUG`, `INFO`, `WARNING`, `ERROR`                                           |
 | `TOPICFORGE_ROS2_BIN`       | `ros2`  | Name (or path) of the ROS2 CLI binary                                         |
 | `TOPICFORGE_TELEMETRY`      | `off`   | Opt-in anonymous usage telemetry. See [Telemetry](#telemetry).                |
-| `TOPICFORGE_DDS_BACKEND`    | `mock`  | DDS module backend: `mock`, `cyclone`, `fast`, `rti`, or `auto`. `auto` resolves to Fast > Cyclone > Mock. See [Multi-vendor DDS support](#multi-vendor-dds-support-v030). |
+| `TOPICFORGE_DDS_BACKEND`    | `mock`  | DDS module backend: `mock`, `cyclone`, `fast`, `rti`, `opensplice`, `coredx`, `intercom`, `opendds`, `dust`, or `auto`. The v0.4.0 Phase 1.5 auto-detect chain resolves to: `rti > opensplice > coredx > intercom` (Pro tier, if installed) `> opendds > fast > cyclone > dust > mock`. See [Multi-vendor DDS support](#multi-vendor-dds-support-v030). |
 | `TOPICFORGE_DDS_DOMAIN_ID`  | `0`     | DDS domain id observed (0..232) when a DDS backend is active.                 |
 
 See [`.env.example`](.env.example).
@@ -336,7 +343,7 @@ When telemetry is on, each MCP tool call emits a single event with **only** thes
 
 | Field             | Example          | Notes                                                                  |
 | ----------------- | ---------------- | ---------------------------------------------------------------------- |
-| `tool_name`       | `"list_topics"`  | One of the five MVP tools — never argument values.                     |
+| `tool_name`       | `"list_topics"`  | One of the eleven MCP tools — never argument values.                   |
 | `latency_ms`      | `12.34`          | Wall-clock duration of the handler, rounded to 2 decimals.             |
 | `mode`            | `"mock"`         | Effective runtime mode: `mock` or `live`.                              |
 | `version`         | `"0.1.2"`        | TopicForge server version.                                             |
@@ -382,16 +389,14 @@ See [`docs/product-plan.md`](docs/product-plan.md) for the full product trajecto
 
 Near-term additions on the bench:
 
-- `rclpy`-backed live adapter for faster & richer sampling
-- XTypes/IDL discovery to extend `peek_dds_samples` to arbitrary user topics (today: 4 builtin DCPS topics only) — v0.3.x patch
-- Extended QoS coverage (Liveliness, Ownership, Partition, TimeBasedFilter, LatencyBudget) — v0.3.x patch
-- Composite adapter delegating per-tool category, so ROS2 + DDS surfaces work simultaneously — v0.3.x patch
-- `RtiConnextAdapter` in the Pro tier (BYO RTI Connext license, gated by `TOPICFORGE_LICENSE_KEY`) — v0.4.0+
-- URDF inspector / validator MCP tools
-- Bag anomaly detection (clock jumps, gaps, dropped frames, TF tree health)
+- `rclpy`-backed live adapter for faster & richer sampling (per-message rmw receive timestamps, windowed sampling) — gated on external user demand
+- Extended QoS coverage (Liveliness, Ownership, Partition, TimeBasedFilter, LatencyBudget) — v0.5.x patch
+- Real `RtiConnextAdapter` in the Pro tier (BYO RTI Connext license, gated by `TOPICFORGE_LICENSE_KEY`) — the v0.4.0 Phase 1.5 framework is in place ; production binding pending Pro tier launch
+- URDF inspector / validator MCP tools (Pro tier)
+- Bag anomaly detection (clock jumps, gaps, dropped frames, TF tree health) — Pro tier
 - Dataset export helpers (rosbag → COCO / HF Datasets)
 - Synthetic data pipeline controller (Blender, Gazebo, Isaac Sim)
-- Hosted MCP endpoint with auth
+- Hosted MCP endpoint with auth (Phase 3 — depends on Pro tier traction)
 
 ## Project layout
 

{topicforge-0.4.0 → topicforge-0.5.1}/README.md

@@ -1,11 +1,12 @@
 # TopicForge
 
 [![PyPI version](https://img.shields.io/pypi/v/topicforge.svg)](https://pypi.org/project/topicforge/)
+[![CI](https://github.com/yaniswav/TopicForge/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/yaniswav/TopicForge/actions/workflows/ci.yml)
 [![Python versions](https://img.shields.io/pypi/pyversions/topicforge.svg)](https://pypi.org/project/topicforge/)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://github.com/yaniswav/TopicForge/blob/main/LICENSE)
 [![Read-only by architecture](https://img.shields.io/badge/safety-read--only_by_architecture-2563eb)](https://github.com/yaniswav/TopicForge#security-model)
 
-> **The safety-first read-only MCP for ROS2 robotics — now with multi-vendor OMG DDS-RTPS observability (v0.3.0).** TopicForge lets AI agents inspect your ROS2 graph, ROS bag files, and (since v0.2.0) the raw DDS layer beneath ROS, without ever publishing back to the bus. v0.3.0 ships two OSS Python adapters — Eclipse CycloneDDS and eProsima Fast DDS — each joining the bus as a read-only DDS-RTPS participant that observes **every conformant vendor on the wire** (RTI Connext, OpenDDS, CoreDX, Dust DDS in Rust, etc.) via the OMG protocol guarantee. See [`docs/dds-interop-matrix.md`](docs/dds-interop-matrix.md) for the canonical multi-vendor positioning and the OMG May 2025 interop reference.
+> **The safety-first read-only MCP for ROS2 robotics — observability + multi-vendor OMG DDS-RTPS (v0.4.0).** TopicForge lets AI agents inspect your ROS2 graph, recorded bag files, and the raw DDS layer beneath ROS — without ever publishing back to the bus. **Eleven typed read-only tools** (5 ROS2 graph + 3 DDS + 3 observability/bag) share a single Pydantic envelope so an LLM caller reads one schema across the whole stack. v0.4.0 adds participant lifecycle tracking (`participant_events`), temporal metrics (`topic_metrics`), and post-mortem bag sample peek (`peek_bag_samples`) on top of v0.3.0's two OSS Python DDS participants — Eclipse CycloneDDS and eProsima Fast DDS — each observing **every conformant vendor on the wire** (RTI Connext, OpenDDS, CoreDX, Dust DDS in Rust, etc.) via the OMG protocol guarantee. See [`docs/dds-interop-matrix.md`](docs/dds-interop-matrix.md) for the canonical multi-vendor positioning and the OMG May 2025 interop reference.
 
 TopicForge is a production-minded MCP (Model Context Protocol) server that lets AI agents — such as Claude — inspect ROS2 topics, analyze ROS bag files, and (since v0.2.0) observe the raw DDS layer through a clean, structured tool interface. It is read-only by **architecture**, not by configuration: there is no write path to misconfigure, no permission system to audit, no liability conversation to have. The MCP client can see the robot stack; it cannot touch it.
 
@@ -171,21 +172,24 @@ CoreDX, InterCOM) ship under the optional `topicforge-pro` package with
 BYO vendor license. See [`docs/pro.md`](docs/pro.md) for the early-access
 slot and pricing terms ; nothing is collected today.
 
-Three new MCP tools (in addition to the five ROS2 tools above) :
+Six DDS / observability tools (in addition to the five ROS2 tools above) :
 
-| Tool                    | Purpose                                                                          |
-| ----------------------- | -------------------------------------------------------------------------------- |
-| `list_participants`     | DDS participants discovered on a domain, with vendor and hostname                |
-| `detect_qos_mismatches` | Reader/writer QoS incompatibilities preventing communication on a topic          |
-| `peek_dds_samples`      | Recent samples on a raw DDS topic (distinct from `sample_messages` on ROS2 graph)|
+| Tool                    | Since   | Purpose                                                                                                        |
+| ----------------------- | ------- | -------------------------------------------------------------------------------------------------------------- |
+| `list_participants`     | v0.2.0  | DDS participants discovered on a domain, with vendor, hostname, and (v0.4.0) lifecycle fields                  |
+| `detect_qos_mismatches` | v0.2.0  | Reader/writer QoS incompatibilities preventing communication on a topic                                        |
+| `peek_dds_samples`      | v0.2.0  | Recent samples on a raw DDS topic — v0.4.0 adds best-effort XTypes user-topic decode (distinct from `sample_messages` on ROS2 graph) |
+| `participant_events`    | v0.4.0  | Lifecycle stream — `discovered` / `lost` participant events over a configurable window                         |
+| `topic_metrics`         | v0.4.0  | Temporal metrics — observed frequency, sequence gaps, latency p50/p95/p99 over a sliding window                |
+| `peek_bag_samples`      | v0.4.0  | Post-mortem inspection — decoded samples from a recorded `.mcap` / `.db3` / `.bag` file                        |
 
-**Composite adapter (v0.4.0 Phase 1+).** When `TOPICFORGE_MODE=live` is paired with a DDS backend (`cyclone`, `fast`, …), TopicForge instantiates **both** a ROS2 CLI adapter and the chosen DDS adapter and routes per-tool category — the 5 ROS2 tools hit the CLI, the 3 DDS tools (+ `participant_events` from Phase 1) hit the DDS backend. ROS2-only or DDS-only setups still work — the missing half is skipped and the present half serves what it can. The mock backend continues to expose all 9 tools against deterministic fixtures for local development.
+**Composite adapter (v0.4.0 Phase 1+).** When `TOPICFORGE_MODE=live` is paired with a DDS backend (`cyclone`, `fast`, …), TopicForge instantiates **both** a ROS2 CLI adapter and the chosen DDS adapter and routes per-tool category — the 5 ROS2 tools hit the CLI, the DDS / observability tools hit the DDS backend. ROS2-only or DDS-only setups still work — the missing half is skipped and the present half serves what it can. The mock backend continues to expose all 11 tools against deterministic fixtures for local development.
 
-**v0.3.0 limitation — `peek_dds_samples` scope.** Full-fidelity on the 4 builtin DCPS topics (`DCPSParticipant`, `DCPSSubscription`, `DCPSPublication`) ; arbitrary user topics raise an `AdapterError` pointing at the v0.3.x XTypes/IDL roadmap. The other two DDS tools (`list_participants`, `detect_qos_mismatches`) work end-to-end on any user-topic deployment.
+**`peek_dds_samples` payload shape (v0.4.0 Phase 1.5).** Full-fidelity on the 4 builtin DCPS topics (`DCPSParticipant`, `DCPSSubscription`, `DCPSPublication`). Arbitrary user topics return best-effort decoded samples with a `_decode_status` annotation : `"full"` (every IDL field decoded — currently a v0.4.0+ Cyclone XTypes path), `"partial"` (some fields decoded, others opaque), or `"raw"` (binding could not resolve the dynamic XTypes). The diagnostic key `_decode_note` carries a short explanation when the status is non-`full`. The wire shape is identical across Cyclone and Fast backends. **Caveat (v0.5.x):** on the current user-topic *raw* path `_raw_bytes_hex` is **empty** — a `"raw"` status means "topic present on the bus but not decoded", not "here are the serialized bytes to re-decode". Capturing the on-wire CDR bytes into the fallback is roadmapped ; until then use the Cyclone full/partial XTypes path for actual user-topic payloads. The 4 builtin DCPS topics are unaffected (always structured).
 
 **`RTI Connext`** is v0.4.0+ Pro tier (BYO license — see `docs/pro.md`).
 
-**Full 5-minute walkthrough** — backend selection, the canonical QoS-mismatch debugging scenario, troubleshooting — lives in [`docs/DDS_QUICKSTART.md`](docs/DDS_QUICKSTART.md). Migration from v0.2.0 in [`docs/MIGRATION_v0.2_to_v0.3.md`](docs/MIGRATION_v0.2_to_v0.3.md).
+**Full 5-minute walkthrough** — backend selection, the canonical QoS-mismatch debugging scenario, troubleshooting — lives in [`docs/DDS_QUICKSTART.md`](docs/DDS_QUICKSTART.md). Migration history : [v0.2 → v0.3](docs/MIGRATION_v0.2_to_v0.3.md), [v0.3 → v0.4](docs/MIGRATION_v0.3_to_v0.4.md).
 
 ### Configure with Claude Desktop
 
@@ -240,7 +244,7 @@ make check    # both, plus tests (CI bundle)
 | `TOPICFORGE_LOG_LEVEL`      | `INFO`  | `DEBUG`, `INFO`, `WARNING`, `ERROR`                                           |
 | `TOPICFORGE_ROS2_BIN`       | `ros2`  | Name (or path) of the ROS2 CLI binary                                         |
 | `TOPICFORGE_TELEMETRY`      | `off`   | Opt-in anonymous usage telemetry. See [Telemetry](#telemetry).                |
-| `TOPICFORGE_DDS_BACKEND`    | `mock`  | DDS module backend: `mock`, `cyclone`, `fast`, `rti`, or `auto`. `auto` resolves to Fast > Cyclone > Mock. See [Multi-vendor DDS support](#multi-vendor-dds-support-v030). |
+| `TOPICFORGE_DDS_BACKEND`    | `mock`  | DDS module backend: `mock`, `cyclone`, `fast`, `rti`, `opensplice`, `coredx`, `intercom`, `opendds`, `dust`, or `auto`. The v0.4.0 Phase 1.5 auto-detect chain resolves to: `rti > opensplice > coredx > intercom` (Pro tier, if installed) `> opendds > fast > cyclone > dust > mock`. See [Multi-vendor DDS support](#multi-vendor-dds-support-v030). |
 | `TOPICFORGE_DDS_DOMAIN_ID`  | `0`     | DDS domain id observed (0..232) when a DDS backend is active.                 |
 
 See [`.env.example`](.env.example).
@@ -268,7 +272,7 @@ When telemetry is on, each MCP tool call emits a single event with **only** thes
 
 | Field             | Example          | Notes                                                                  |
 | ----------------- | ---------------- | ---------------------------------------------------------------------- |
-| `tool_name`       | `"list_topics"`  | One of the five MVP tools — never argument values.                     |
+| `tool_name`       | `"list_topics"`  | One of the eleven MCP tools — never argument values.                   |
 | `latency_ms`      | `12.34`          | Wall-clock duration of the handler, rounded to 2 decimals.             |
 | `mode`            | `"mock"`         | Effective runtime mode: `mock` or `live`.                              |
 | `version`         | `"0.1.2"`        | TopicForge server version.                                             |
@@ -314,16 +318,14 @@ See [`docs/product-plan.md`](docs/product-plan.md) for the full product trajecto
 
 Near-term additions on the bench:
 
-- `rclpy`-backed live adapter for faster & richer sampling
-- XTypes/IDL discovery to extend `peek_dds_samples` to arbitrary user topics (today: 4 builtin DCPS topics only) — v0.3.x patch
-- Extended QoS coverage (Liveliness, Ownership, Partition, TimeBasedFilter, LatencyBudget) — v0.3.x patch
-- Composite adapter delegating per-tool category, so ROS2 + DDS surfaces work simultaneously — v0.3.x patch
-- `RtiConnextAdapter` in the Pro tier (BYO RTI Connext license, gated by `TOPICFORGE_LICENSE_KEY`) — v0.4.0+
-- URDF inspector / validator MCP tools
-- Bag anomaly detection (clock jumps, gaps, dropped frames, TF tree health)
+- `rclpy`-backed live adapter for faster & richer sampling (per-message rmw receive timestamps, windowed sampling) — gated on external user demand
+- Extended QoS coverage (Liveliness, Ownership, Partition, TimeBasedFilter, LatencyBudget) — v0.5.x patch
+- Real `RtiConnextAdapter` in the Pro tier (BYO RTI Connext license, gated by `TOPICFORGE_LICENSE_KEY`) — the v0.4.0 Phase 1.5 framework is in place ; production binding pending Pro tier launch
+- URDF inspector / validator MCP tools (Pro tier)
+- Bag anomaly detection (clock jumps, gaps, dropped frames, TF tree health) — Pro tier
 - Dataset export helpers (rosbag → COCO / HF Datasets)
 - Synthetic data pipeline controller (Blender, Gazebo, Isaac Sim)
-- Hosted MCP endpoint with auth
+- Hosted MCP endpoint with auth (Phase 3 — depends on Pro tier traction)
 
 ## Project layout