ledgerworks 0.2.0__tar.gz

ledgerworks-0.2.0/.gitignore

@@ -0,0 +1,13 @@
+__pycache__/
+*.py[cod]
+.venv/
+dist/
+build/
+*.egg-info/
+.pytest_cache/
+.ruff_cache/
+.env
+# example working directories (the committed inputs and expected outputs stay)
+examples/*/work/
+examples/*/ledger/
+.DS_Store

ledgerworks-0.2.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Gabin Taibi
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

ledgerworks-0.2.0/PKG-INFO

@@ -0,0 +1,184 @@
+Metadata-Version: 2.5
+Name: ledgerworks
+Version: 0.2.0
+Summary: Run, record, verify and trace pipeline artifacts: immutable outputs with provenance sidecars, lineage and a rebuildable index
+Project-URL: Homepage, https://github.com/GabinTB/ledgerworks
+Project-URL: Repository, https://github.com/GabinTB/ledgerworks
+Project-URL: Issues, https://github.com/GabinTB/ledgerworks/issues
+Author: Gabin Taibi
+License: MIT
+License-File: LICENSE
+Keywords: artifacts,data-management,lineage,provenance,reproducibility,research
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering
+Classifier: Topic :: Software Development :: Version Control
+Requires-Python: >=3.11
+Provides-Extra: all
+Requires-Dist: numpy>=1.26; extra == 'all'
+Requires-Dist: pandas>=2.0; extra == 'all'
+Requires-Dist: polars>=0.20; extra == 'all'
+Requires-Dist: pyarrow>=14.0; extra == 'all'
+Requires-Dist: python-dotenv>=1.0; extra == 'all'
+Provides-Extra: cli
+Requires-Dist: python-dotenv>=1.0; extra == 'cli'
+Provides-Extra: pandas
+Requires-Dist: pandas>=2.0; extra == 'pandas'
+Provides-Extra: relink
+Requires-Dist: numpy>=1.26; extra == 'relink'
+Requires-Dist: polars>=0.20; extra == 'relink'
+Requires-Dist: pyarrow>=14.0; extra == 'relink'
+Description-Content-Type: text/markdown
+
+# ledgerworks
+
+Run, record, verify and trace pipeline artifacts.
+
+Every output of a pipeline becomes an immutable artifact directory with a
+`meta.json` sidecar that says what it is, what it was built from, which code
+and model produced it, and what every file hashed to when it was written.
+A SQLite index makes lookups fast and can always be rebuilt from the sidecars,
+so the files are the source of truth and the index is a cache.
+
+| | |
+|---|---|
+| PyPI | `pip install ledgerworks` |
+| Python | >= 3.11 |
+| Core dependencies | none (stdlib only) |
+| License | MIT |
+
+## What it does
+
+| Layer | What you get |
+|---|---|
+| **Record** | A deterministic artifact id (`{kind}__{model}__{version}__{params}__{minute}`), a `meta.json` and `README.md` sidecar per artifact, one `RunRecord` per execution (start, end, git commit, files produced, Python, platform, `uv.lock` digest and direct dependency versions), blake2b hash and size of every output file |
+| **Query** | `latest(kind)`, `get(id)`, `list(...)`, `parents` / `children` / `ancestors` / `descendants`, DuckDB views by kind |
+| **Verify** | Re-hash files, flag crashed (partial) runs and stray temp files, find lineage edges that point to nothing, run per-kind content checks you register |
+| **Run** | `ledgerworks.jobs`: long pipelines as ordered work units with resume, update, pause, a heartbeat lock and a status file |
+| **Migrate** | `ledgerworks.relink`: re-register an artifact's files under new inputs or new code without recomputing, after equivalence checks (needs the `relink` extra) |
+
+Guarantees that hold everywhere:
+
+- a run is **partial** until it exits cleanly, and `latest()` never returns a partial artifact;
+- nothing is mutated or deleted: new data appends an execution, a wrong artifact is deprecated;
+- the index holds nothing the sidecars lack, so `reindex` recovers it from disk at any time.
+
+## Installation
+
+```bash
+pip install ledgerworks              # core, stdlib only
+pip install "ledgerworks[cli]"       # + .env support for the command lines
+pip install "ledgerworks[relink]"    # + numpy / polars / pyarrow for relink checks
+pip install "ledgerworks[all]"
+```
+
+For development:
+
+```bash
+git clone https://github.com/GabinTB/ledgerworks
+cd ledgerworks
+uv sync --group dev
+uv run pytest
+```
+
+## Quick start
+
+```python
+from ledgerworks import Ledger, ModelCard
+
+ledger = Ledger("/data/ledger")
+
+# 1. register a raw input
+with ledger.run(kind="documents", pipeline="my-project", pipeline_version="v0.1.0",
+                hyperparams={"source": "fomc", "start": "2021-01-01"},
+                layer="raw") as run:
+    write_parquet(run.out_dir / "documents.parquet", docs)
+
+# 2. derive something from it, with lineage and a model card
+source = ledger.latest("documents")
+card = ModelCard(model_id="my-embedder", version="1.0", repo="https://hf.co/...",
+                 weights_public=True, dim=768)
+
+with ledger.run(kind="document_embeddings", pipeline="my-project", pipeline_version="v0.1.0",
+                hyperparams={"pooling": "mean"}, sources=[source], model_card=card,
+                verifier="document_embeddings") as run:
+    embed(source.path, run.out_dir)
+
+# 3. query
+art = ledger.latest("document_embeddings")
+print(art.artifact_id, art.meta.pipeline_commit, art.file_hashes)
+print(ledger.ancestors(art.artifact_id))
+```
+
+On clean exit the run's record is marked complete, the outputs are hashed and
+the sidecars written. On an exception the record stays partial, so the crash is
+visible instead of silent.
+
+The artifact directory:
+
+```
+{root}/derived/document_embeddings/document_embeddings__my-embedder-1.0__v0.1.0__poolingmean__20261014T1530/
+    meta.json        # the record: identity, hyperparams, sources, model card, runs[], file hashes
+    README.md        # the same, rendered for humans
+    2021-01.parquet
+    ...
+```
+
+## Command lines
+
+```
+ledgerworks init | ls | show ID | verify | lineage ID | deprecate ID --reason ... | reindex | move | relink
+ledgerworks-jobs start KIND | resume ID | update ID | pause ID | status | list | check ID
+```
+
+The root comes from `--root`, else `$LEDGER_ROOT` (also read from `.env` when
+the `cli` extra is installed).
+
+## Plugging in your own kinds
+
+A package contributes jobs, content verifiers and equivalence checks through
+entry points in its `pyproject.toml`:
+
+```toml
+[project.entry-points."ledgerworks.jobs"]
+document_embeddings = "my_project.embed:EmbedJob"
+
+[project.entry-points."ledgerworks.verifiers"]
+document_embeddings = "my_project.embed:verify_artifact"
+```
+
+A `Job` declares what it computes (`kind`, `pipeline`, `pipeline_version`,
+`params()`, `sources()`, `units()`, `run_unit()`); `JobRunner` owns how it
+runs. See `ledgerworks/jobs.py` for the contract and `examples/codecheck` for
+a complete job.
+
+## Examples
+
+- [`examples/codecheck`](examples/codecheck): a self-contained reproducibility
+  package (FOMC statements, an open embedding model, a macro-narrative
+  taxonomy) submitted to the University of Twente CodeCheck 2026. Run it with
+  one command and compare the outputs to the committed reference.
+  Reproduce it: `cd examples/codecheck && uv sync --extra model && uv run codecheck-fomc all`
+  (see [`examples/codecheck/README.md`](examples/codecheck/README.md)).
+
+## Development
+
+```bash
+uv run pytest             # tests
+uv run ruff check .       # lint
+uv run ruff format .      # format
+uv build                  # wheel + sdist
+```
+
+Releases: push a tag `vX.Y.Z` matching the version in `pyproject.toml`. The
+release workflow runs CI, publishes to TestPyPI, installs from there as a
+smoke test, then publishes to PyPI and creates a GitHub release.
+
+## License
+
+MIT

ledgerworks-0.2.0/README.md

@@ -0,0 +1,147 @@
+# ledgerworks
+
+Run, record, verify and trace pipeline artifacts.
+
+Every output of a pipeline becomes an immutable artifact directory with a
+`meta.json` sidecar that says what it is, what it was built from, which code
+and model produced it, and what every file hashed to when it was written.
+A SQLite index makes lookups fast and can always be rebuilt from the sidecars,
+so the files are the source of truth and the index is a cache.
+
+| | |
+|---|---|
+| PyPI | `pip install ledgerworks` |
+| Python | >= 3.11 |
+| Core dependencies | none (stdlib only) |
+| License | MIT |
+
+## What it does
+
+| Layer | What you get |
+|---|---|
+| **Record** | A deterministic artifact id (`{kind}__{model}__{version}__{params}__{minute}`), a `meta.json` and `README.md` sidecar per artifact, one `RunRecord` per execution (start, end, git commit, files produced, Python, platform, `uv.lock` digest and direct dependency versions), blake2b hash and size of every output file |
+| **Query** | `latest(kind)`, `get(id)`, `list(...)`, `parents` / `children` / `ancestors` / `descendants`, DuckDB views by kind |
+| **Verify** | Re-hash files, flag crashed (partial) runs and stray temp files, find lineage edges that point to nothing, run per-kind content checks you register |
+| **Run** | `ledgerworks.jobs`: long pipelines as ordered work units with resume, update, pause, a heartbeat lock and a status file |
+| **Migrate** | `ledgerworks.relink`: re-register an artifact's files under new inputs or new code without recomputing, after equivalence checks (needs the `relink` extra) |
+
+Guarantees that hold everywhere:
+
+- a run is **partial** until it exits cleanly, and `latest()` never returns a partial artifact;
+- nothing is mutated or deleted: new data appends an execution, a wrong artifact is deprecated;
+- the index holds nothing the sidecars lack, so `reindex` recovers it from disk at any time.
+
+## Installation
+
+```bash
+pip install ledgerworks              # core, stdlib only
+pip install "ledgerworks[cli]"       # + .env support for the command lines
+pip install "ledgerworks[relink]"    # + numpy / polars / pyarrow for relink checks
+pip install "ledgerworks[all]"
+```
+
+For development:
+
+```bash
+git clone https://github.com/GabinTB/ledgerworks
+cd ledgerworks
+uv sync --group dev
+uv run pytest
+```
+
+## Quick start
+
+```python
+from ledgerworks import Ledger, ModelCard
+
+ledger = Ledger("/data/ledger")
+
+# 1. register a raw input
+with ledger.run(kind="documents", pipeline="my-project", pipeline_version="v0.1.0",
+                hyperparams={"source": "fomc", "start": "2021-01-01"},
+                layer="raw") as run:
+    write_parquet(run.out_dir / "documents.parquet", docs)
+
+# 2. derive something from it, with lineage and a model card
+source = ledger.latest("documents")
+card = ModelCard(model_id="my-embedder", version="1.0", repo="https://hf.co/...",
+                 weights_public=True, dim=768)
+
+with ledger.run(kind="document_embeddings", pipeline="my-project", pipeline_version="v0.1.0",
+                hyperparams={"pooling": "mean"}, sources=[source], model_card=card,
+                verifier="document_embeddings") as run:
+    embed(source.path, run.out_dir)
+
+# 3. query
+art = ledger.latest("document_embeddings")
+print(art.artifact_id, art.meta.pipeline_commit, art.file_hashes)
+print(ledger.ancestors(art.artifact_id))
+```
+
+On clean exit the run's record is marked complete, the outputs are hashed and
+the sidecars written. On an exception the record stays partial, so the crash is
+visible instead of silent.
+
+The artifact directory:
+
+```
+{root}/derived/document_embeddings/document_embeddings__my-embedder-1.0__v0.1.0__poolingmean__20261014T1530/
+    meta.json        # the record: identity, hyperparams, sources, model card, runs[], file hashes
+    README.md        # the same, rendered for humans
+    2021-01.parquet
+    ...
+```
+
+## Command lines
+
+```
+ledgerworks init | ls | show ID | verify | lineage ID | deprecate ID --reason ... | reindex | move | relink
+ledgerworks-jobs start KIND | resume ID | update ID | pause ID | status | list | check ID
+```
+
+The root comes from `--root`, else `$LEDGER_ROOT` (also read from `.env` when
+the `cli` extra is installed).
+
+## Plugging in your own kinds
+
+A package contributes jobs, content verifiers and equivalence checks through
+entry points in its `pyproject.toml`:
+
+```toml
+[project.entry-points."ledgerworks.jobs"]
+document_embeddings = "my_project.embed:EmbedJob"
+
+[project.entry-points."ledgerworks.verifiers"]
+document_embeddings = "my_project.embed:verify_artifact"
+```
+
+A `Job` declares what it computes (`kind`, `pipeline`, `pipeline_version`,
+`params()`, `sources()`, `units()`, `run_unit()`); `JobRunner` owns how it
+runs. See `ledgerworks/jobs.py` for the contract and `examples/codecheck` for
+a complete job.
+
+## Examples
+
+- [`examples/codecheck`](examples/codecheck): a self-contained reproducibility
+  package (FOMC statements, an open embedding model, a macro-narrative
+  taxonomy) submitted to the University of Twente CodeCheck 2026. Run it with
+  one command and compare the outputs to the committed reference.
+  Reproduce it: `cd examples/codecheck && uv sync --extra model && uv run codecheck-fomc all`
+  (see [`examples/codecheck/README.md`](examples/codecheck/README.md)).
+
+## Development
+
+```bash
+uv run pytest             # tests
+uv run ruff check .       # lint
+uv run ruff format .      # format
+uv build                  # wheel + sdist
+```
+
+Releases: push a tag `vX.Y.Z` matching the version in `pyproject.toml`. The
+release workflow runs CI, publishes to TestPyPI, installs from there as a
+smoke test, then publishes to PyPI and creates a GitHub release.
+
+## License
+
+MIT

ledgerworks-0.2.0/examples/codecheck/README.md

@@ -0,0 +1,220 @@
+# CodeCheck 2026: FOMC statements scored against a monetary-policy narrative taxonomy
+
+A self-contained reproducibility package, submitted to the University of Twente
+CodeCheck / Reproducibility Hackathon (20 October 2026). It is one slice of an
+ongoing PhD project on narrative dynamics in financial markets (Gabin Taibi,
+University of Twente, BMS-IEBIS), rebuilt on open data and an open model so
+that every step can be re-run and checked by a reviewer.
+
+**What is reproduced:** the two computational steps at the core of the research
+pipeline, (1) embedding a corpus of documents with a finance-adapted language
+model and (2) scoring every document against a taxonomy of narratives, applied
+to FOMC statements. The outputs are the embedding tables and the score tables
+under `expected/`. There is no paper table to match: this is the "ongoing
+project" route of the CodeCheck call, and the research taxonomy itself is not
+published here (the file in `taxonomy/` is a public stand-in with the same
+structure).
+
+**What the reviewer checks:** that `codecheck-fomc all` rebuilds, from the files
+in this directory, outputs equal to `expected/` within the stated tolerances,
+and that every artifact's provenance record (`meta.json`) is intact. The
+package is also the end-to-end example of
+[`ledgerworks`](../../README.md), the artifact-provenance library the project
+is built on.
+
+## Quick start
+
+Requirements: Python >= 3.11, [uv](https://docs.astral.sh/uv/), about 2.7 GB of
+disk (measured on macOS, Apple silicon: 1.2 GB for `.venv`, 1.5 GB for the model
+in the Hugging Face cache), a network connection the first time (to
+download the model from the Hugging Face Hub). CPU is enough. On Linux, `uv.lock`
+takes torch from the PyTorch CPU index (`2.14.1+cpu`), not PyPI, whose Linux
+wheel pulls several GB of CUDA libraries a CPU run never loads; macOS and
+Windows use PyPI's torch, which is CPU-only there.
+
+```bash
+git clone https://github.com/GabinTB/ledgerworks
+cd ledgerworks/examples/codecheck
+uv sync --extra model          # creates .venv from uv.lock (pinned)
+uv run codecheck-fomc all      # ingest, embed, score, verify, compare
+```
+
+The corpus is 46 statements from 2021-01-27 to 2026-09-16, split into 190
+paragraphs. Measured runtime on an Apple M4 Pro (CPU), model already cached:
+about 15 seconds wall clock, almost all of it in the embedding step (a
+395M-parameter model over 190 paragraphs). The first run also downloads the
+model weights (1.6 GB). The last lines of the output are the comparison with
+`expected/` and a verdict (here, on the machine that produced `expected/`):
+
+```
+fomc_narrative_scores: ...
+  ok    fomc_narrative_scores/2021.parquet: 80 scores, max|diff| 0.00e+00
+  ...
+  ok    fomc_narrative_scores/2026.parquet: 60 scores, max|diff| 0.00e+00
+  ok    fomc_narrative_scores/narratives.parquet: 10 vectors, max|diff| 0.00e+00, min cosine 1.000000
+        7/7 files hash-identical to the reference
+
+REPRODUCED
+```
+
+The score tables hold 460 rows in total (46 statements × 10 narratives).
+
+`uv run codecheck-fomc --help` lists the individual steps. Everything lands in
+`./ledger/` (deleted and rebuilt freely; it is in `.gitignore`).
+
+To check the plumbing without the model (seconds, no download):
+
+```bash
+uv sync
+uv run codecheck-fomc --backend hashing all     # NOT REPRODUCED is expected here:
+                                                # expected/ was produced with the real model
+uv run pytest                                   # offline tests on a fixture corpus
+```
+
+## What is in the box
+
+| Path | What | Who produced it |
+|---|---|---|
+| `data/raw/html/*.htm` | The FOMC statement pages, byte for byte as served by federalreserve.gov | author, with `fetch_fomc.py` |
+| `data/raw/MANIFEST.json` | URL, date, SHA-256 and retrieval time of every page | `fetch_fomc.py` |
+| `taxonomy/monetary_policy.yaml` | Ten monetary-policy narratives: label, description, seed phrases | author |
+| `codecheck_fomc/` | The pipeline: three `ledgerworks` jobs, the embedding backends, the comparison | author |
+| `expected/` | The reference run: the three artifact directories with their `meta.json` | author, with `codecheck-fomc freeze` |
+| `uv.lock` | Every Python dependency pinned, model stack included | `uv` |
+| `tests/` | Offline tests of the whole pipeline on a fixture corpus (hashing backend) | author |
+
+### Data
+
+FOMC statements from January 2021 to the retrieval date, one HTML page per
+meeting statement (other FOMC press releases, such as the August 2025 update of
+the Statement on Longer-Run Goals and Monetary Policy Strategy, are excluded), downloaded from the Board's
+[FOMC calendars page](https://www.federalreserve.gov/monetarypolicy/fomccalendars.htm).
+Content on the Board's website is in the public domain and may be copied and
+distributed without permission
+([disclaimer](https://www.federalreserve.gov/disclaimer.htm)); the Board of
+Governors of the Federal Reserve System is the source. The committed HTML is the
+immutable input: parsing it into a table is the first job of the pipeline, so
+it is recorded and re-run like every other step. `fetch_fomc.py` is kept for
+provenance; the reviewer does not need to run it.
+
+### Model
+
+[`BatuhanECB/FinModernBERT-embed-large-v1`](https://huggingface.co/BatuhanECB/FinModernBERT-embed-large-v1),
+a ModernBERT-large encoder domain-adapted on financial text and contrastively
+fine-tuned for embeddings (Apache-2.0, 395M parameters, 1024 dimensions). It is
+pinned to Hub commit `4fd33d2335e75c79df57b345aee06d49c5f85bc8` in
+`codecheck_fomc/backends.py` and loaded through `sentence-transformers`; the
+exact library versions are in `uv.lock` and in every artifact's model card
+(`meta.json` → `model_card.serving`). It runs on CPU in float32 with
+`torch.use_deterministic_algorithms(True)`.
+
+### Taxonomy
+
+`taxonomy/monetary_policy.yaml` holds ten narratives. Where a label coincides
+with a topic of Bybee, Kelly, Manela and Xiu (2024), *Business News and
+Business Cycles*, Journal of Finance 79(5),
+[doi:10.1111/jofi.13377](https://doi.org/10.1111/jofi.13377), the file says so
+(`bkmx: true`); the descriptions and seed phrases are this package's own and are
+the part a researcher swaps for their own taxonomy.
+
+## The pipeline
+
+Three jobs, each producing one `ledgerworks` artifact (an immutable directory
+with a `meta.json` record), each partitioned one file per year:
+
+| Step | Kind | Input | Output |
+|---|---|---|---|
+| 1 ingest | `fomc_statements` (raw layer) | `data/raw/` | one row per statement: `doc_id`, `date`, `url`, `html_sha256`, `paragraphs`, `text` |
+| 2 embed | `fomc_embeddings` | step 1 | one row per paragraph: `doc_id`, `para_idx`, `text_sha256`, `embedding` (unit norm) |
+| 3 score | `fomc_narrative_scores` | step 2 + taxonomy | one row per statement × narrative: `cos_doc`, `cos_para_mean`, `cos_para_max`; plus `narratives.parquet` (the narrative vectors) |
+
+Paragraphs are the embedding unit because FOMC statements are written as short
+self-contained paragraphs and the model was trained at 512 tokens. The parser
+(`codecheck_fomc/corpus.py`, `PARSER_VERSION = "3"`, recorded in the
+`fomc_statements` hyperparameters) keeps only what the Committee says. From the
+`<p>` elements of the press release it drops:
+
+- page furniture ("For release at", "Share", "Implementation Note", "Last Update", ...);
+- the release-date line printed above the title ("January 27, 2021");
+- sidebar panels (`<div class="panel ...">`, "Related Information" links);
+- vote paragraphs, in both formats: "Voting for/against the monetary policy
+  action ..." (with any dissents inside it), the approval line that opens
+  statements from June 2026 ("The Federal Open Market Committee approved the
+  following statement for release by a 12 – 0 vote:"), and standalone dissent
+  paragraphs ("<Name>, who preferred ...", "<Name> dissented").
+ A narrative
+vector is the mean of the embeddings of its description and seed phrases, from
+the same model. `cos_doc` compares the narrative with the mean of a statement's
+paragraph vectors; `cos_para_mean` and `cos_para_max` aggregate paragraph-level
+cosines. These are raw scores; calibration is downstream work and out of scope.
+
+Every artifact records: its hyperparameters (which define its id), its input
+artifact (lineage), the model card (id, revision, dimension, pooling, library
+versions, device), the git commit of the code, start and end times, the
+execution environment (Python, platform, `uv.lock` digest, versions of the
+direct dependencies), and the blake2b digest of every output file. `ledgerworks verify --root ledger` re-hashes
+everything and checks the lineage; `ledgerworks lineage ID` prints the graph.
+
+## How the comparison works
+
+`codecheck-fomc compare` joins each table of the fresh run with the reference
+on its keys and checks:
+
+| Table | Check |
+|---|---|
+| statements | rows identical (text and HTML digest) |
+| embeddings | same paragraphs (text digest); vectors within `--atol 1e-4` element-wise and cosine >= `--min-cos 0.9999` |
+| scores | within `--atol 1e-4` |
+
+Neural-network outputs are reproducible to floating-point tolerance across
+machines, not bit for bit (different BLAS kernels, instruction sets and thread
+counts reorder floating-point sums), hence the tolerances. The ingest step and
+the hashing test backend are exact, and `compare` reports how many files are
+hash-identical to the reference. The comparison exits non-zero on any failure;
+with no reference in `expected/` yet (first run) it says so and exits 0.
+
+## Cross-machine check
+
+`expected/` was produced on an Apple M4 Pro (macOS, arm64, CPython 3.12.13, PyPI
+torch 2.14.1; Python is pinned by `.python-version`). The manual workflow
+[`reproduce.yml`](../../.github/workflows/reproduce.yml) rebuilds everything on
+GitHub-hosted runners with `uv sync --extra model` and `codecheck-fomc all`, and
+uploads every `meta.json` and the comparison output. Run of 14 October 2026 at
+commit `7619ab4`:
+
+| | Linux (ubuntu-latest) | Windows (windows-latest) |
+|---|---|---|
+| Machine | x86_64, CPython 3.12.3, torch 2.14.1+cpu | AMD64, CPython 3.12.10, torch 2.14.1 |
+| Verdict | REPRODUCED | REPRODUCED |
+| statements | 46 identical, 6/6 files hash-identical | 46 identical, 6/6 files hash-identical |
+| embeddings, max\|diff\| | 6.97e-06 | 5.45e-06 |
+| embeddings, min cosine | 1.000000 | 1.000000 |
+| scores, max\|diff\| | 5.01e-06 | 3.76e-06 |
+| narrative vectors, max\|diff\| / min cosine | 3.73e-07 / 1.000000 | 3.80e-07 / 1.000000 |
+| Wall time, `all` (model cached) | 135 s | 175 s |
+
+The largest difference is about 14 times below the tolerance (`atol 1e-4`), and
+no file of the embedding or score tables is bit-identical across machines, as
+expected from different CPUs and BLAS kernels. The artifact ids (up to the
+creation minute) and the recorded `uv.lock` digest are the same on all three
+machines: `.gitattributes` keeps text files in LF on every checkout and the raw
+HTML byte for byte.
+
+## Reproducing the reference run (author's procedure)
+
+```bash
+python fetch_fomc.py                       # network: download the statements
+git add data/raw && git commit -m "data: FOMC statements to <date>"
+uv sync --extra model
+uv run codecheck-fomc all                  # from a clean tree, so the commit is recorded
+uv run codecheck-fomc freeze               # copy the run into expected/
+git add expected && git commit -m "expected: reference run"
+```
+
+A dirty working tree is refused by the jobs unless `--allow-dirty`, and
+`verify` flags artifacts whose recorded commit ends in `-dirty`.
+
+## Contact
+
+Gabin Taibi, University of Twente (BMS-IEBIS), gabin.taibi@gmail.com.
+Issues and questions: https://github.com/GabinTB/ledgerworks/issues.

ledgerworks-0.2.0/examples/codecheck/data/raw/README.md

@@ -0,0 +1,12 @@
+# data/raw
+
+Produced once by the package author with `python fetch_fomc.py` (see the
+example README), then committed:
+
+- `html/monetaryYYYYMMDDa.htm`: each FOMC statement page, byte for byte as
+  served by federalreserve.gov (public domain; source: Board of Governors of the
+  Federal Reserve System).
+- `MANIFEST.json`: URL, date, SHA-256, size and retrieval time of every page.
+
+The pipeline's first job (`codecheck-fomc ingest`) parses these files; it
+refuses a file whose digest no longer matches the manifest.

ledgerworks-0.2.0/examples/codecheck/expected/README.md

@@ -0,0 +1,10 @@
+# expected
+
+The reference run, produced by the package author with
+`codecheck-fomc all` followed by `codecheck-fomc freeze`: one directory per
+artifact kind (`fomc_statements`, `fomc_embeddings`, `fomc_narrative_scores`),
+each a verbatim copy of the artifact directory including its `meta.json`
+(hyperparameters, inputs, model card, git commit, file digests), plus
+`RUN.json` with the three artifact ids.
+
+`codecheck-fomc compare` checks a fresh run against these files.