chiptime 0.4.0__tar.gz

chiptime-0.4.0/.gitignore

@@ -0,0 +1,35 @@
+# macOS
+.DS_Store
+
+# Python
+__pycache__/
+*.py[cod]
+.venv/
+venv/
+dist/
+build/
+*.egg-info/
+.pytest_cache/
+.mypy_cache/
+.ruff_cache/
+.coverage
+htmlcov/
+
+# Node (for the future JS implementation)
+node_modules/
+*.tsbuildinfo
+
+# Editors
+.idea/
+.vscode/
+
+# Claude workspace — local-only, never committed
+.claude/
+CLAUDE.md
+
+# Real-device corpus tier: NEVER committed (ADR-0007 — PII)
+corpus/private/
+.site-build/
+
+# Internal-only research (competitive analysis stays private)
+docs/internal/

chiptime-0.4.0/.python-version

@@ -0,0 +1 @@
+3.13

chiptime-0.4.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Maxim Grigoryev
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

chiptime-0.4.0/PKG-INFO

@@ -0,0 +1,60 @@
+Metadata-Version: 2.5
+Name: chiptime
+Version: 0.4.0
+Summary: Recovery-grade FIT file processing: parse anything, lose nothing silently, explain everything.
+Project-URL: Homepage, https://maxgrgrv.github.io/chiptime/
+Project-URL: Documentation, https://maxgrgrv.github.io/chiptime/
+Project-URL: Repository, https://github.com/MaxGrgrv/chiptime
+Project-URL: Changelog, https://maxgrgrv.github.io/chiptime/changelog/
+Project-URL: Issues, https://github.com/MaxGrgrv/chiptime/issues
+Author: Maxim Grigoryev
+License-Expression: MIT
+License-File: LICENSE
+Keywords: activity,cycling,fit,fitness,garmin,parser,recovery,repair,running
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Information Analysis
+Classifier: Typing :: Typed
+Requires-Python: >=3.11
+Provides-Extra: pandas
+Requires-Dist: pandas>=2.0; extra == 'pandas'
+Description-Content-Type: text/markdown
+
+# chiptime
+
+**Recovery-grade FIT file processing.** Parse anything, lose nothing silently, explain everything. Zero runtime dependencies.
+
+Every other open-source FIT parser stops at the first bad byte. chiptime resynchronizes mid-file, salvages truncated files, rebuilds missing session summaries, and can write the result back out as a valid, uploadable `.fit` — with a machine-readable provenance trail for every repair.
+
+```python
+import chiptime
+
+result = chiptime.parse("broken.fit")  # lenient: recover + annotate
+result.recovery  # what was salvaged
+result.activity.sessions[0].records.stream("power")  # 0 is real, None is absent
+result.to_canonical_json()  # RFC 8785 — byte-identical every run
+
+fixed = chiptime.repair("broken.fit")  # → a valid .fit
+```
+
+```bash
+chiptime parse ride.fit --json      # agent exit codes: 0/2/3/4/64
+chiptime repair broken.fit -o fixed.fit
+chiptime validate fixed.fit --platform garmin-connect
+chiptime inspect weird.fit          # wire-level forensics
+```
+
+- Three modes: `strict` / `lenient` / `forensic` (forensic never drops data)
+- Sentinels → null before any statistics; zero ≠ null, always
+- Deterministic canonical JSON; errors carry stable codes + suggested next steps
+- Conformance corpus: 104-item edge-case taxonomy, golden-file tested
+- Optional extras: `chiptime[pandas]` for DataFrames; `chiptime.metrics` for mean-max curves, zone time, SWOLF
+
+Docs, taxonomy, corpus and research: https://github.com/MaxGrgrv/chiptime
+
+chiptime is an independent project, not affiliated with or endorsed by Garmin. FIT and Garmin are trademarks of Garmin Ltd. This package ships no Garmin SDK files.

chiptime-0.4.0/README.md

@@ -0,0 +1,33 @@
+# chiptime
+
+**Recovery-grade FIT file processing.** Parse anything, lose nothing silently, explain everything. Zero runtime dependencies.
+
+Every other open-source FIT parser stops at the first bad byte. chiptime resynchronizes mid-file, salvages truncated files, rebuilds missing session summaries, and can write the result back out as a valid, uploadable `.fit` — with a machine-readable provenance trail for every repair.
+
+```python
+import chiptime
+
+result = chiptime.parse("broken.fit")  # lenient: recover + annotate
+result.recovery  # what was salvaged
+result.activity.sessions[0].records.stream("power")  # 0 is real, None is absent
+result.to_canonical_json()  # RFC 8785 — byte-identical every run
+
+fixed = chiptime.repair("broken.fit")  # → a valid .fit
+```
+
+```bash
+chiptime parse ride.fit --json      # agent exit codes: 0/2/3/4/64
+chiptime repair broken.fit -o fixed.fit
+chiptime validate fixed.fit --platform garmin-connect
+chiptime inspect weird.fit          # wire-level forensics
+```
+
+- Three modes: `strict` / `lenient` / `forensic` (forensic never drops data)
+- Sentinels → null before any statistics; zero ≠ null, always
+- Deterministic canonical JSON; errors carry stable codes + suggested next steps
+- Conformance corpus: 104-item edge-case taxonomy, golden-file tested
+- Optional extras: `chiptime[pandas]` for DataFrames; `chiptime.metrics` for mean-max curves, zone time, SWOLF
+
+Docs, taxonomy, corpus and research: https://github.com/MaxGrgrv/chiptime
+
+chiptime is an independent project, not affiliated with or endorsed by Garmin. FIT and Garmin are trademarks of Garmin Ltd. This package ships no Garmin SDK files.

chiptime-0.4.0/pyproject.toml

@@ -0,0 +1,73 @@
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[project]
+name = "chiptime"
+version = "0.4.0"
+description = "Recovery-grade FIT file processing: parse anything, lose nothing silently, explain everything."
+readme = "README.md"
+requires-python = ">=3.11"
+license = "MIT"
+authors = [{ name = "Maxim Grigoryev" }]
+keywords = ["fit", "garmin", "fitness", "activity", "parser", "recovery", "repair", "cycling", "running"]
+classifiers = [
+    "Development Status :: 3 - Alpha",
+    "Intended Audience :: Developers",
+    "Operating System :: OS Independent",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
+    "Topic :: Scientific/Engineering :: Information Analysis",
+    "Typing :: Typed",
+]
+# Contract: the core has ZERO runtime dependencies. Any addition here is a
+# design decision requiring an ADR (see docs/dependencies/DEPENDENCY_MAP.md).
+dependencies = []
+
+[project.urls]
+Homepage = "https://maxgrgrv.github.io/chiptime/"
+Documentation = "https://maxgrgrv.github.io/chiptime/"
+Repository = "https://github.com/MaxGrgrv/chiptime"
+Changelog = "https://maxgrgrv.github.io/chiptime/changelog/"
+Issues = "https://github.com/MaxGrgrv/chiptime/issues"
+
+[project.optional-dependencies]
+pandas = ["pandas>=2.0"]
+
+[project.scripts]
+chiptime = "chiptime.cli:main"
+
+[dependency-groups]
+docs = [
+    "mkdocs-material>=9.5",
+    "mkdocstrings[python]>=0.26",
+    "mkdocs-llmstxt>=0.2",
+    "mkdocs-git-revision-date-localized-plugin>=1.2",
+]
+dev = [
+    "pytest>=8.3",
+    "hypothesis>=6.112",
+    "mypy>=1.11",
+    "ruff>=0.6.8",
+]
+# Only for corpus/tools/scoreboard.py (M2) and profile cross-checks.
+# Never installed by default; never imported by chiptime itself.
+baselines = [
+    "fitparse>=1.2",
+    "fitdecode>=0.10",
+]
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/chiptime"]
+
+
+[tool.mypy]
+python_version = "3.11"
+strict = true
+files = ["src"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+addopts = "-q"

chiptime-0.4.0/src/chiptime/__init__.py

@@ -0,0 +1,43 @@
+"""chiptime — recovery-grade FIT file processing.
+
+Parse anything, lose nothing silently, explain everything.
+
+    import chiptime
+    result = chiptime.parse("ride.fit")          # lenient by default
+    result.ok, result.file_type, result.recovery
+    result.to_canonical_json()                    # deterministic (RFC 8785)
+"""
+
+from chiptime._api import iter_frames, iter_messages, parse
+from chiptime.errors import (
+    CrcMismatchError,
+    EmptyFileError,
+    FitError,
+    HeaderError,
+    NotFitError,
+    ProtocolError,
+    TruncatedError,
+)
+from chiptime.repair import NotRepairableError, RepairResult, repair
+from chiptime.result import Mode, ParseResult
+
+__version__ = "0.4.0"
+
+__all__ = [
+    "CrcMismatchError",
+    "EmptyFileError",
+    "FitError",
+    "HeaderError",
+    "Mode",
+    "NotFitError",
+    "NotRepairableError",
+    "ParseResult",
+    "ProtocolError",
+    "RepairResult",
+    "TruncatedError",
+    "__version__",
+    "iter_frames",
+    "iter_messages",
+    "parse",
+    "repair",
+]

chiptime-0.4.0/src/chiptime/__main__.py

@@ -0,0 +1,3 @@
+from chiptime.cli import main
+
+raise SystemExit(main())

chiptime-0.4.0/src/chiptime/_api.py

@@ -0,0 +1,394 @@
+"""parse() — the one-call entry point; iter_frames/iter_messages streaming layers.
+
+Mode policy per ADR-0003: strict raises the first defect; lenient recovers and
+records; forensic is lenient that never drops (divergence begins in F5/F10).
+"""
+
+from __future__ import annotations
+
+import dataclasses
+import hashlib
+from collections.abc import Iterator
+from os import PathLike
+from pathlib import Path
+from typing import Any, BinaryIO
+
+from chiptime.decode import Decoder
+from chiptime.errors import (
+    Defect,
+    Diagnostic,
+    FitError,
+    ProvenanceEntry,
+    defect_to_error,
+)
+from chiptime.frames import (
+    CrcFrame,
+    DataFrame,
+    EndOfStream,
+    FileHeader,
+    FrameEvent,
+    SkippedBytes,
+    read_stream,
+)
+from chiptime.intake import unwrap
+from chiptime.message import FieldValue, Message
+from chiptime.result import FitPart, Mode, ParseResult, RecoveryReport, SourceInfo
+from chiptime.semantics import build_activity
+
+Source = str | PathLike[str] | bytes | bytearray | BinaryIO
+
+# Structural defects that do NOT stop the stream — they surface as warnings
+# in lenient/forensic ("seen and continued").
+_CONTINUE_CODES = {
+    "FIT_HEADER_INVALID",
+    "FIT_HEADER_CRC_MISMATCH",
+    "FIT_CRC_MISMATCH",
+    "FIT_CRC_MISSING",
+    "FIT_DATA_SIZE_MISMATCH",
+}
+
+_SUGGESTIONS = {
+    "NOT_FIT_FORMAT": "route this file to a parser for the named format",
+    "FIT_TRUNCATED": 'rerun with mode="lenient" to salvage the decodable prefix',
+    "FIT_CRC_MISMATCH": 'rerun with mode="lenient" to decode despite the bad CRC',
+    "FIT_HEADER_CRC_MISMATCH": 'rerun with mode="lenient" to decode despite the bad header CRC',
+    "FIT_UNDEFINED_LOCAL_TYPE": 'rerun with mode="lenient" to salvage the decodable prefix',
+    "FIT_DEFINITION_INVALID": 'rerun with mode="lenient" to salvage the decodable prefix',
+    "FIT_DATA_SIZE_MISMATCH": 'rerun with mode="lenient" to parse the actual content',
+}
+
+_PII_MESSAGES = {"user_profile"}
+_PII_FIELDS = {"serial_number"}
+
+
+def _read_source(src: Source) -> tuple[bytes, str | None]:
+    if isinstance(src, (bytes, bytearray)):
+        return bytes(src), None
+    if isinstance(src, (str, PathLike)):
+        p = Path(src)
+        return p.read_bytes(), str(p)
+    data = src.read()
+    return bytes(data), getattr(src, "name", None)
+
+
+def iter_frames(src: Source, *, mode: Mode = "lenient") -> Iterator[FrameEvent]:
+    """Lossless wire-level frame events (forensics layer)."""
+    data, _ = _read_source(src)
+    offset = 0
+    while offset < len(data):
+        consumed = offset
+        for ev in read_stream(data, offset=offset):
+            if isinstance(ev, Defect) and mode == "strict":
+                raise defect_to_error(ev, suggestion=_SUGGESTIONS.get(ev.code))
+            if isinstance(ev, EndOfStream):
+                consumed = ev.consumed
+            yield ev
+        if consumed <= offset or not _looks_like_header(data, consumed):
+            break
+        offset = consumed
+
+
+def iter_messages(src: Source, *, mode: Mode = "lenient") -> Iterator[Message]:
+    """Profile-applied message stream without building the semantic model."""
+    decoder = Decoder()
+    for ev in iter_frames(src, mode=mode):
+        if isinstance(ev, DataFrame):
+            yield decoder.decode(ev)
+
+
+def _looks_like_header(data: bytes, offset: int) -> bool:
+    if len(data) - offset < 12:
+        return False
+    return data[offset + 8 : offset + 12] == b".FIT" or data[offset] in (12, 14)
+
+
+def parse(
+    src: Source,
+    *,
+    mode: Mode = "lenient",
+    strip_pii: bool = False,
+    include_unknown: bool = True,
+    include_raw: bool = False,
+) -> ParseResult:
+    """Parse a FIT source. lenient (default) recovers and annotates; strict
+    raises the first FitError; forensic maximizes salvage and never drops."""
+    raw, path = _read_source(src)
+    source_hash = hashlib.sha256(raw).hexdigest()
+    intake_result = unwrap(raw)
+    data = intake_result.data
+    source = SourceInfo(
+        path=path,
+        size_bytes=len(raw),
+        sha256=source_hash,
+        unwrapped=intake_result.unwrapped,
+    )
+
+    parts: list[FitPart] = []
+    provenance: list[ProvenanceEntry] = list(intake_result.provenance)
+    warnings: list[Diagnostic] = []
+    errors: list[FitError] = []
+
+    for d in intake_result.defects:
+        if mode == "strict":
+            raise defect_to_error(d, suggestion=_SUGGESTIONS.get(d.code))
+        errors.append(defect_to_error(d, suggestion=_SUGGESTIONS.get(d.code)))
+    if any(d.severity == "fatal" for d in intake_result.defects):
+        return ParseResult(
+            ok=False,
+            mode=mode,
+            source=source,
+            parts=[],
+            provenance=provenance,
+            warnings=warnings,
+            errors=errors,
+            recovery=None,
+            include_raw=include_raw,
+        )
+    total_recovered = 0
+    total_skipped = 0
+    resync_count = 0
+    recovery_engaged = False
+    est_total: int | None = None
+
+    offset = 0
+    part_index = 0
+    while True:  # runs at least once so empty input still yields its defect
+        decoder = Decoder()
+        messages: list[Message] = []
+        stream_defects: list[Defect] = []
+        skips: list[SkippedBytes] = []
+        header: FileHeader | None = None
+        consumed = len(data)
+        body_bytes_decoded = 0
+
+        for ev in read_stream(data, offset=offset):
+            if isinstance(ev, Defect):
+                if mode == "strict":
+                    raise defect_to_error(ev, suggestion=_SUGGESTIONS.get(ev.code))
+                stream_defects.append(ev)
+            elif isinstance(ev, SkippedBytes):
+                skips.append(ev)
+            elif isinstance(ev, DataFrame):
+                messages.append(decoder.decode(ev))
+                body_bytes_decoded = (
+                    ev.offset
+                    + 1
+                    + len(ev.payload)
+                    - (header.offset + header.size if header else offset)
+                )
+            elif isinstance(ev, FileHeader):
+                header = ev
+            elif isinstance(ev, CrcFrame):
+                pass  # mismatch already surfaced as a Defect
+            elif isinstance(ev, EndOfStream):
+                consumed = ev.consumed
+
+        decode_out = decoder.finish()
+        messages = decode_out.messages  # finish() may rebuild (late dev-field back-fill)
+        provenance.extend(decode_out.provenance)
+        warnings.extend(decode_out.diagnostics)
+        for d in decode_out.defects:  # data-severity defects from decoding
+            if mode == "strict":
+                raise defect_to_error(d, suggestion=_SUGGESTIONS.get(d.code))
+            warnings.append(Diagnostic(d.code, d.detail, f"byte {d.offset}"))
+
+        scope = f"part[{part_index}]"
+        skip_offsets = {s.offset for s in skips}
+        for skip in skips:
+            recovery_engaged = True
+            total_skipped += skip.length
+            if skip.reason == "preamble-garbage":
+                provenance.append(
+                    ProvenanceEntry(
+                        "PREAMBLE_GARBAGE_SKIPPED",
+                        "repaired",
+                        scope,
+                        f"skipped {skip.length} garbage byte(s) before the FIT header",
+                        byte_offset=skip.offset,
+                        data={"length": skip.length},
+                    )
+                )
+            else:
+                resync_count += 1
+                provenance.append(
+                    ProvenanceEntry(
+                        "RESYNC_SKIPPED_BYTES",
+                        "repaired",
+                        scope,
+                        f"skipped {skip.length} undecodable byte(s) after {skip.reason}"
+                        f" at offset {skip.offset}; decoding resumed",
+                        byte_offset=skip.offset,
+                        data={"length": skip.length, "defect_code": skip.reason},
+                    )
+                )
+        for defect in stream_defects:
+            if defect.offset in skip_offsets and defect.severity == "structural":
+                continue  # resynchronized: the SkippedBytes provenance tells the story
+            if defect.severity == "fatal":
+                errors.append(defect_to_error(defect, suggestion=_SUGGESTIONS.get(defect.code)))
+            elif defect.code in _CONTINUE_CODES:
+                warnings.append(Diagnostic(defect.code, defect.detail, f"byte {defect.offset}"))
+            else:
+                # Structural defect that stopped the stream: prefix salvage (F5 → resync).
+                recovery_engaged = True
+                code = (
+                    "TRUNCATED_TAIL_SALVAGED"
+                    if defect.code == "FIT_TRUNCATED"
+                    else "STREAM_STOPPED_AT_DEFECT"
+                )
+                provenance.append(
+                    ProvenanceEntry(
+                        code,
+                        "repaired",
+                        scope,
+                        f"{defect.detail}; salvaged {len(messages)} complete message(s)",
+                        byte_offset=defect.offset,
+                        data={"defect_code": defect.code},
+                    )
+                )
+                if (
+                    defect.code == "FIT_TRUNCATED"
+                    and header is not None
+                    and header.data_size
+                    and body_bytes_decoded > 0
+                ):
+                    est_total = round(len(messages) * header.data_size / body_bytes_decoded)
+
+        if messages or header is not None:
+            part = _build_part(messages)
+            if strip_pii:
+                _strip_pii(part, provenance, scope)
+            if not include_unknown:
+                _drop_unknown(part, provenance, scope)
+            if part.file_type == "activity":
+                part.activity = build_activity(
+                    part.messages,
+                    warnings,
+                    provenance,
+                    scope,
+                    skipped_ranges=[(sk.offset, sk.offset + sk.length) for sk in skips],
+                    forensic=(mode == "forensic"),
+                )
+            parts.append(part)
+            total_recovered += len(messages)
+
+        part_index += 1
+        if consumed <= offset:
+            break
+        offset = consumed
+        if offset >= len(data):
+            break
+        if not _looks_like_header(data, offset):
+            junk = Defect(
+                "FIT_TRAILING_JUNK",
+                f"{len(data) - offset} byte(s) after the final CRC are not a chained FIT file",
+                offset,
+                "structural",
+            )
+            if mode == "strict":
+                raise defect_to_error(junk)
+            if not any(
+                d.severity == "structural" and d.code not in _CONTINUE_CODES for d in stream_defects
+            ):
+                warnings.append(Diagnostic(junk.code, junk.detail, f"byte {offset}"))
+            break
+
+    ok = any(p.messages for p in parts) and not any(
+        e.code in ("FIT_EMPTY", "FIT_TOO_SMALL", "NOT_FIT_FORMAT") for e in errors
+    )
+    if not ok and not errors:
+        # Contract #5: ok=false must always be explained. The valid-but-empty
+        # shell (taxonomy #16, seen in the wild as 16-byte tool output).
+        errors.append(
+            FitError(
+                "FIT_NO_CONTENT",
+                "structurally valid FIT container with no messages — the data is"
+                " genuinely absent, not recoverable",
+                suggestion="nothing to salvage; check the device/app that wrote it",
+            )
+        )
+    recovery = (
+        RecoveryReport(
+            recovered_records=total_recovered,
+            estimated_total_records=est_total,
+            bytes_read=len(data),
+            bytes_skipped=total_skipped,
+            resync_count=resync_count,
+        )
+        if recovery_engaged
+        else None
+    )
+    return ParseResult(
+        ok=ok,
+        mode=mode,
+        source=source,
+        parts=parts,
+        provenance=provenance,
+        warnings=warnings,
+        errors=errors,
+        recovery=recovery,
+        include_raw=include_raw,
+    )
+
+
+def _build_part(messages: list[Message]) -> FitPart:
+    file_id: dict[str, Any] | None = None
+    file_type = "unknown"
+    for m in messages:
+        if m.global_num == 0:
+            file_id = {k: fv.value for k, fv in m.fields.items()}
+            t = m.get("type")
+            if isinstance(t, str):
+                file_type = t
+            elif t is not None:
+                file_type = f"unknown_{t}"
+            break
+    return FitPart(file_type=file_type, file_id=file_id, messages=messages)
+
+
+def _strip_pii(part: FitPart, provenance: list[ProvenanceEntry], scope: str) -> None:
+    removed_msgs = 0
+    nulled_fields = 0
+    kept: list[Message] = []
+    for m in part.messages:
+        if m.name in _PII_MESSAGES:
+            removed_msgs += 1
+            continue
+        if any(f in m.fields for f in _PII_FIELDS):
+            fields = dict(m.fields)
+            for f in _PII_FIELDS:
+                if f in fields:
+                    fields[f] = FieldValue(None, None, fields[f].units)
+                    nulled_fields += 1
+            m = dataclasses.replace(m, fields=fields)
+        kept.append(m)
+    part.messages = kept
+    if part.file_id and "serial_number" in part.file_id:
+        part.file_id["serial_number"] = None
+    if removed_msgs or nulled_fields:
+        provenance.append(
+            ProvenanceEntry(
+                "PII_STRIPPED",
+                "dropped",
+                scope,
+                f"removed {removed_msgs} PII message(s), nulled {nulled_fields}"
+                f" serial-number field(s) (strip_pii=True)",
+                data={"messages_removed": removed_msgs, "fields_nulled": nulled_fields},
+            )
+        )
+
+
+def _drop_unknown(part: FitPart, provenance: list[ProvenanceEntry], scope: str) -> None:
+    known = [m for m in part.messages if not m.name.startswith("unknown_")]
+    dropped = len(part.messages) - len(known)
+    part.messages = known
+    if dropped:
+        provenance.append(
+            ProvenanceEntry(
+                "UNKNOWN_MESSAGES_OMITTED",
+                "ignored",
+                scope,
+                f"{dropped} unknown message(s) omitted from output (include_unknown=False)",
+                data={"count": dropped},
+            )
+        )