awkward-monetizer 0.1.1__tar.gz

awkward_monetizer-0.1.1/.gitignore

@@ -0,0 +1,50 @@
+# --- Data (not committed; see data/README.md) ---
+/data/*.root
+*.root
+*.csv
+*.parquet
+*.arrow
+*.dbfarm/
+hep-dbfarm/
+
+# --- Local awkward source checkout (built via pixi path deps) ---
+/awkward/
+
+# --- pixi ---
+/.pixi/
+# Remove the next line to commit the lockfile for reproducibility.
+pixi.lock
+
+# --- Python / packaging ---
+__pycache__/
+*.py[cod]
+*$py.class
+build/
+dist/
+*.egg-info/
+.eggs/
+.venv/
+venv/
+env/
+src/*/_version.py
+
+# --- Test / tooling caches ---
+.pytest_cache/
+.ruff_cache/
+.mypy_cache/
+htmlcov/
+.coverage
+
+# --- Notebooks / outputs ---
+.ipynb_checkpoints/
+__marimo__/
+*.png
+
+# --- Cowork desktop app ---
+/Claude outputs/
+
+# --- Editors / OS ---
+.vscode/
+.idea/
+.DS_Store
+*.swp

awkward_monetizer-0.1.1/LICENSE

@@ -0,0 +1,28 @@
+BSD 3-Clause License
+
+Copyright (c) 2026, Ianna Osborne
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+1. Redistributions of source code must retain the above copyright notice, this
+   list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright notice,
+   this list of conditions and the following disclaimer in the documentation
+   and/or other materials provided with the distribution.
+
+3. Neither the name of the copyright holder nor the names of its
+   contributors may be used to endorse or promote products derived from
+   this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

awkward_monetizer-0.1.1/PKG-INFO

@@ -0,0 +1,256 @@
+Metadata-Version: 2.5
+Name: awkward-monetizer
+Version: 0.1.1
+Summary: Hybrid HEP analysis engine: Awkward Array + MonetDB + Apache Arrow
+Project-URL: Homepage, https://github.com/ianna/awkward-monetizer
+Project-URL: Issues, https://github.com/ianna/awkward-monetizer/issues
+Author: Ianna Osborne
+License-Expression: BSD-3-Clause
+License-File: LICENSE
+Keywords: awkward,columnar,hep,monetdb,scikit-hep,uproot
+Requires-Python: >=3.10
+Requires-Dist: awkward>=2
+Requires-Dist: numpy>=1.24
+Requires-Dist: pandas>=2
+Requires-Dist: pyarrow>=12
+Requires-Dist: pymonetdb>=1.7
+Requires-Dist: uproot>=5
+Requires-Dist: vector>=1
+Provides-Extra: dev
+Requires-Dist: pre-commit; extra == 'dev'
+Requires-Dist: pytest>=7; extra == 'dev'
+Requires-Dist: ruff; extra == 'dev'
+Provides-Extra: embedded
+Requires-Dist: monetdbe; extra == 'embedded'
+Provides-Extra: notebook
+Requires-Dist: hist; extra == 'notebook'
+Requires-Dist: marimo; extra == 'notebook'
+Requires-Dist: matplotlib; extra == 'notebook'
+Requires-Dist: mplhep; extra == 'notebook'
+Description-Content-Type: text/markdown
+
+# awkward-monetizer
+
+A hybrid High-Energy Physics (HEP) analysis engine that combines:
+
+- **[MonetDB](https://www.monetdb.org/)** — fast, columnar SQL over event-level data
+- **[Awkward Array](https://awkward-array.org/)** — high-performance nested (NF2) physics analysis
+- **[Apache Arrow](https://arrow.apache.org/)** — the bridge between the two
+
+Interactive exploration is done in [Marimo](https://marimo.io/) notebooks; the aim
+is a Python-native alternative to [ROOT/RDataFrame](https://root.cern/doc/master/classROOT_1_1RDataFrame.html).
+
+The idea: flatten nested ROOT events into relational tables, push selections down
+to SQL, then reconstruct the nested structure in Awkward for the physics. It's
+motivated by *Evaluating Query Languages and Systems for High-Energy Physics
+Data* (Graur et al., PVLDB vol. 15, 2022; [arXiv:2104.12615](https://arxiv.org/abs/2104.12615)),
+which found SQL systems struggle with nested data while ROOT stays dominant
+thanks to its NF2-friendly execution model.
+
+## Pipeline
+
+```
+ROOT ──uproot──▶ Awkward ──flatten──▶ pandas ──▶ MonetDB
+                                                     │
+   Awkward (NF2) ◀──unflatten── pandas ◀──SQL slice──┘
+        │
+        └─▶ physics: invariant mass, ΔR, combinatorics ─▶ plots
+```
+
+## Layout
+
+```
+src/awkward_monetizer/
+  datasets.py     dataset registry (dimuon, nanoaod) + collection types
+  ingest.py       ROOT → Awkward → flat tables → MonetDB
+  reconstruct.py  flat tables → Awkward NF2 (ak.unflatten)
+  db.py           MonetDB connections (server / embedded) + packaged schema
+  physics.py      4-vectors (scikit-hep vector), invariant mass, charge pairs
+  adl.py          ADL Q1–Q8 benchmark queries
+  roundtrip.py    full ingest → DB → slice → reconstruct → validate cycle
+  sampledata.py   synthetic NanoAOD generator
+  cli.py          `awkward-monetizer` command-line interface
+  schemas/        packaged SQL (dimuon.sql, nanoaod.sql, jets.sql)
+examples/         analysis.py (Marimo notebook), make_sample_nanoaod.py
+tests/            pytest suite (reconstruction, round-trip, ADL)
+data/             inputs (git-ignored) — see data/README.md
+```
+
+## Install
+
+Development environment via **pixi** (builds Awkward from a local source
+checkout; see *Awkward from source* below):
+
+```bash
+git clone --recursive https://github.com/scikit-hep/awkward.git
+pixi install
+```
+
+Or plain pip (uses released Awkward):
+
+```bash
+pip install -e ".[notebook,dev]"
+```
+
+The MonetDB **server** is separate (not on PyPI/conda): `brew install monetdb`
+on macOS. The embedded engine (`pip install ".[embedded]"`, i.e. `monetdbe`) needs
+Python ≤ 3.10.
+
+## Quickstart
+
+```bash
+awkward-monetizer ingest data/cms.root --dry-run    # ROOT → flat tables
+awkward-monetizer reconstruct data/cms.root         # → Awkward NF2 + validate
+awkward-monetizer make-nano                          # synthetic NanoAOD
+awkward-monetizer adl                                # ADL Q1–Q8 on it
+```
+
+With pixi these are tasks: `pixi run ingest-dryrun`, `pixi run reconstruct-demo`,
+`pixi run make-nano`, `pixi run adl`, `pixi run notebook`, `pixi run test`.
+
+## Datasets
+
+`event_id` is the join key across tables; `*_index` columns preserve each
+object's position so nesting round-trips exactly. Two layouts (`--dataset`):
+
+- **dimuon** (default; `data/cms.root`) — flat CMS dimuon ntuple: one row per
+  event, two muons as parallel columns (`pt1/pt2`, `Q1/Q2`, …), precomputed mass
+  `M`. Un-pivoted into a `muons` table. Schema: `schemas/dimuon.sql`.
+- **nanoaod** — jagged per-event lists (`Jet_pt`, `Muon_pt`, …) exploded into
+  `jets`/`muons`, plus MET. Schema: `schemas/nanoaod.sql`. Matches real CMS
+  NanoAODv9 branch names/types; `event_id` is synthesized from a row index
+  (because `event` alone isn't unique in NanoAOD — the key is run+lumi+event,
+  kept as `run`/`lumi`/`event_number` columns).
+
+## Live MonetDB round-trip
+
+```bash
+brew install monetdb
+monetdbd create ~/hep-dbfarm && monetdbd start ~/hep-dbfarm
+monetdb create hep && monetdb release hep
+awkward-monetizer roundtrip --root-file data/cms.root      # server backend
+```
+
+`roundtrip` ingests, creates the schema, loads (`COPY INTO`), pushes a
+`WHERE mass BETWEEN 60 AND 120` slice to SQL, reconstructs, and checks the
+reconstructed dimuon mass against the stored `M`. `--backend embedded` runs the
+same cycle in-process via `monetdbe` (loads with `INSERT`).
+
+Round-trip mass validation supports only `dimuon`; unsupported datasets are
+rejected before connecting to the database. If the SQL filter selects no events,
+validation reports that it did not run and returns `passed=False`.
+
+## ADL benchmark
+
+`adl.py` implements the eight IRIS-HEP ADL benchmark queries (Q1 MET, Q2/Q3 jet
+pT, Q4 ≥2 jets, Q5 OS dimuon, Q6 trijet, Q7 lepton-cleaned HT, Q8 MT of MET +
+lead lepton) on reconstructed NanoAOD events using `vector`. No real NanoAOD file
+ships here — `make-nano` writes a synthetic one; point `--from-root` at a real
+CMS Open Data NanoAOD file to run on data. Scope: only muons are present, so
+Q7/Q8 use muons as leptons; Q6 omits the b-tag term (no b-tag branch yet).
+
+## Awkward from source
+
+`pixi.toml` builds Awkward from a local `./awkward` clone (path dependencies:
+`awkward-cpp` compiled, `awkward` editable) rather than a wheel — a `{ git=... }`
+spec resolves to the released wheel. Verify with `pixi run check-awkward`: the
+`file:` path should resolve into `./awkward/src/awkward/`. On macOS the source
+build pins `MACOSX_DEPLOYMENT_TARGET` so the compiled wheel matches conda-forge's
+Python; if a build is rejected as `macosx_15_0` incompatible, run
+`MACOSX_DEPLOYMENT_TARGET=11.0 pixi install`.
+
+## Benchmarks
+
+`benchmark.py` compares backends on identical data for the dimuon Z→μμ selection
+(opposite-charge pair, 60–120 GeV):
+
+- **awkward** — pure Awkward (uproot read → reconstruct → analyze in memory)
+- **hybrid** — MonetDB + Awkward: data pre-loaded, the event cut pushed to SQL,
+  survivors reconstructed and finished in Awkward
+- **monetdb** — the *whole* selection in SQL via a `dimuon_mass` user-defined
+  function (`schemas/udf_dimuon.sql`): self-join the two muons per event, compute
+  the mass and filter in-database, return only the surviving masses — no Awkward
+  reconstruction. Runs on a server or embedded `monetdbe`.
+- **rdataframe** — ROOT `RDataFrame` (gated: runs only if `import ROOT` works)
+
+`--scale K` tiles the input into a K× larger physical ROOT file every backend
+reads, so timings are meaningful; the query phase is timed (median of `--repeats`)
+while one-time setup (reconstruction / DB ingest) is reported separately, since a
+database amortizes load cost across many queries.
+
+```bash
+awkward-monetizer bench --root-file data/cms.root --scale 50 \\
+    --backends awkward,hybrid,rdataframe --repeats 5
+```
+
+The backends cross-check on selected-event count and mean mass (the harness prints
+`agreement across backends: OK`). Pushing the physics into the database with the UDF (`monetdb` backend) returns
+only the final selection, so it can beat the fetch-and-reconstruct `hybrid` path
+by a wide margin on this query. The embedded hybrid backend loads via `INSERT`
+(slow); use `--hybrid-backend server` (a real MonetDB server with `COPY INTO`) to
+benchmark a realistic load path.
+
+### When SQL wins vs when Awkward wins
+
+The harness ships two analyses (`--analysis`) that show the crossover:
+
+- `zmumu` (flat: two muons/event, a scalar mass) **favors the database** — the
+  in-DB `monetdb` backend runs the whole selection in SQL and beats
+  fetch-and-reconstruct.
+- `trijet` (ADL Q6, jagged: every 3-jet combination, keep the mass closest to
+  172.5 GeV) **favors Awkward** — `ak.combinations(jets, 3)` is one vectorized
+  line, whereas the SQL equivalent (`schemas/udf_trijet.sql`) needs two UDFs, a
+  3-way self-join over jet triples, and a window function — and runs *slower* on
+  the same data.
+
+```bash
+awkward-monetizer bench --analysis trijet --root-file data/nano_synth.root \\
+    --backends awkward,monetdb
+```
+
+The relational store is the right tool for columnar filtering, materialization,
+and scalar arithmetic; Awkward is the right tool for variable-length
+combinatorics and nested (NF2) physics. The point of the hybrid design is to use
+each where it wins, not to replace one with the other.
+
+## Ingesting real CMS Open Data NanoAOD
+
+The `nanoaod` dataset reads real CMS NanoAODv9 files directly (uproot pulls only
+the branches we map, so a multi-GB file isn't fully read). Get a file — e.g. the
+DoubleMuon sample from the [CMS NanoAOD guide](https://opendata.cern.ch/docs/cms-getting-started-nanoaod) —
+via XRootD and stream it into a running MonetDB `hep` server in chunks:
+
+```bash
+xrdcp root://eospublic.cern.ch//eos/opendata/cms/Run2016H/DoubleMuon/NANOAOD/UL2016_MiniAODv2_NanoAODv9-v1/2510000/127C2975-1B1C-A046-AABF-62B77E757A86.root .
+awkward-monetizer ingest 127C2975-*.root --dataset nanoaod --database hep \\
+    --create-schema --step-size "100 MB" --entry-stop 200000
+```
+
+`--step-size` streams with `uproot.iterate` (each chunk is offset so `event_id`
+stays globally unique), keeping memory flat regardless of file size;
+`--create-schema` (re)creates the tables first. Then the `nanoaod` ADL queries
+and benchmarks run on real physics.
+
+`--step-size 50000` sets an entry count instead of a memory size. Add `--dry-run`
+to build and summarize chunks without connecting to MonetDB, or `--truncate`
+to clear the dataset's tables once before loading the chunks.
+
+## Testing
+
+```bash
+pytest        # or: pixi run test
+```
+
+The ADL suite runs on generated synthetic data; the dimuon and round-trip tests
+use `data/cms.root` and skip cleanly if it isn't present.
+
+## Status
+
+Ingestion, reconstruction, the live MonetDB round-trip, and ADL Q1–Q8 are
+implemented and validated (round-trip verified on a real MonetDB server; the
+reconstructed dimuon mass matches the stored `M` to ~1e-8). See `docs/roadmap.md`
+for what's next (real NanoAOD data, ROOT/RDataFrame benchmarks, Arrow Flight).
+
+## License
+
+BSD-3-Clause. Contributions welcome — see `CONTRIBUTING.md`.

awkward_monetizer-0.1.1/README.md

@@ -0,0 +1,225 @@
+# awkward-monetizer
+
+A hybrid High-Energy Physics (HEP) analysis engine that combines:
+
+- **[MonetDB](https://www.monetdb.org/)** — fast, columnar SQL over event-level data
+- **[Awkward Array](https://awkward-array.org/)** — high-performance nested (NF2) physics analysis
+- **[Apache Arrow](https://arrow.apache.org/)** — the bridge between the two
+
+Interactive exploration is done in [Marimo](https://marimo.io/) notebooks; the aim
+is a Python-native alternative to [ROOT/RDataFrame](https://root.cern/doc/master/classROOT_1_1RDataFrame.html).
+
+The idea: flatten nested ROOT events into relational tables, push selections down
+to SQL, then reconstruct the nested structure in Awkward for the physics. It's
+motivated by *Evaluating Query Languages and Systems for High-Energy Physics
+Data* (Graur et al., PVLDB vol. 15, 2022; [arXiv:2104.12615](https://arxiv.org/abs/2104.12615)),
+which found SQL systems struggle with nested data while ROOT stays dominant
+thanks to its NF2-friendly execution model.
+
+## Pipeline
+
+```
+ROOT ──uproot──▶ Awkward ──flatten──▶ pandas ──▶ MonetDB
+                                                     │
+   Awkward (NF2) ◀──unflatten── pandas ◀──SQL slice──┘
+        │
+        └─▶ physics: invariant mass, ΔR, combinatorics ─▶ plots
+```
+
+## Layout
+
+```
+src/awkward_monetizer/
+  datasets.py     dataset registry (dimuon, nanoaod) + collection types
+  ingest.py       ROOT → Awkward → flat tables → MonetDB
+  reconstruct.py  flat tables → Awkward NF2 (ak.unflatten)
+  db.py           MonetDB connections (server / embedded) + packaged schema
+  physics.py      4-vectors (scikit-hep vector), invariant mass, charge pairs
+  adl.py          ADL Q1–Q8 benchmark queries
+  roundtrip.py    full ingest → DB → slice → reconstruct → validate cycle
+  sampledata.py   synthetic NanoAOD generator
+  cli.py          `awkward-monetizer` command-line interface
+  schemas/        packaged SQL (dimuon.sql, nanoaod.sql, jets.sql)
+examples/         analysis.py (Marimo notebook), make_sample_nanoaod.py
+tests/            pytest suite (reconstruction, round-trip, ADL)
+data/             inputs (git-ignored) — see data/README.md
+```
+
+## Install
+
+Development environment via **pixi** (builds Awkward from a local source
+checkout; see *Awkward from source* below):
+
+```bash
+git clone --recursive https://github.com/scikit-hep/awkward.git
+pixi install
+```
+
+Or plain pip (uses released Awkward):
+
+```bash
+pip install -e ".[notebook,dev]"
+```
+
+The MonetDB **server** is separate (not on PyPI/conda): `brew install monetdb`
+on macOS. The embedded engine (`pip install ".[embedded]"`, i.e. `monetdbe`) needs
+Python ≤ 3.10.
+
+## Quickstart
+
+```bash
+awkward-monetizer ingest data/cms.root --dry-run    # ROOT → flat tables
+awkward-monetizer reconstruct data/cms.root         # → Awkward NF2 + validate
+awkward-monetizer make-nano                          # synthetic NanoAOD
+awkward-monetizer adl                                # ADL Q1–Q8 on it
+```
+
+With pixi these are tasks: `pixi run ingest-dryrun`, `pixi run reconstruct-demo`,
+`pixi run make-nano`, `pixi run adl`, `pixi run notebook`, `pixi run test`.
+
+## Datasets
+
+`event_id` is the join key across tables; `*_index` columns preserve each
+object's position so nesting round-trips exactly. Two layouts (`--dataset`):
+
+- **dimuon** (default; `data/cms.root`) — flat CMS dimuon ntuple: one row per
+  event, two muons as parallel columns (`pt1/pt2`, `Q1/Q2`, …), precomputed mass
+  `M`. Un-pivoted into a `muons` table. Schema: `schemas/dimuon.sql`.
+- **nanoaod** — jagged per-event lists (`Jet_pt`, `Muon_pt`, …) exploded into
+  `jets`/`muons`, plus MET. Schema: `schemas/nanoaod.sql`. Matches real CMS
+  NanoAODv9 branch names/types; `event_id` is synthesized from a row index
+  (because `event` alone isn't unique in NanoAOD — the key is run+lumi+event,
+  kept as `run`/`lumi`/`event_number` columns).
+
+## Live MonetDB round-trip
+
+```bash
+brew install monetdb
+monetdbd create ~/hep-dbfarm && monetdbd start ~/hep-dbfarm
+monetdb create hep && monetdb release hep
+awkward-monetizer roundtrip --root-file data/cms.root      # server backend
+```
+
+`roundtrip` ingests, creates the schema, loads (`COPY INTO`), pushes a
+`WHERE mass BETWEEN 60 AND 120` slice to SQL, reconstructs, and checks the
+reconstructed dimuon mass against the stored `M`. `--backend embedded` runs the
+same cycle in-process via `monetdbe` (loads with `INSERT`).
+
+Round-trip mass validation supports only `dimuon`; unsupported datasets are
+rejected before connecting to the database. If the SQL filter selects no events,
+validation reports that it did not run and returns `passed=False`.
+
+## ADL benchmark
+
+`adl.py` implements the eight IRIS-HEP ADL benchmark queries (Q1 MET, Q2/Q3 jet
+pT, Q4 ≥2 jets, Q5 OS dimuon, Q6 trijet, Q7 lepton-cleaned HT, Q8 MT of MET +
+lead lepton) on reconstructed NanoAOD events using `vector`. No real NanoAOD file
+ships here — `make-nano` writes a synthetic one; point `--from-root` at a real
+CMS Open Data NanoAOD file to run on data. Scope: only muons are present, so
+Q7/Q8 use muons as leptons; Q6 omits the b-tag term (no b-tag branch yet).
+
+## Awkward from source
+
+`pixi.toml` builds Awkward from a local `./awkward` clone (path dependencies:
+`awkward-cpp` compiled, `awkward` editable) rather than a wheel — a `{ git=... }`
+spec resolves to the released wheel. Verify with `pixi run check-awkward`: the
+`file:` path should resolve into `./awkward/src/awkward/`. On macOS the source
+build pins `MACOSX_DEPLOYMENT_TARGET` so the compiled wheel matches conda-forge's
+Python; if a build is rejected as `macosx_15_0` incompatible, run
+`MACOSX_DEPLOYMENT_TARGET=11.0 pixi install`.
+
+## Benchmarks
+
+`benchmark.py` compares backends on identical data for the dimuon Z→μμ selection
+(opposite-charge pair, 60–120 GeV):
+
+- **awkward** — pure Awkward (uproot read → reconstruct → analyze in memory)
+- **hybrid** — MonetDB + Awkward: data pre-loaded, the event cut pushed to SQL,
+  survivors reconstructed and finished in Awkward
+- **monetdb** — the *whole* selection in SQL via a `dimuon_mass` user-defined
+  function (`schemas/udf_dimuon.sql`): self-join the two muons per event, compute
+  the mass and filter in-database, return only the surviving masses — no Awkward
+  reconstruction. Runs on a server or embedded `monetdbe`.
+- **rdataframe** — ROOT `RDataFrame` (gated: runs only if `import ROOT` works)
+
+`--scale K` tiles the input into a K× larger physical ROOT file every backend
+reads, so timings are meaningful; the query phase is timed (median of `--repeats`)
+while one-time setup (reconstruction / DB ingest) is reported separately, since a
+database amortizes load cost across many queries.
+
+```bash
+awkward-monetizer bench --root-file data/cms.root --scale 50 \\
+    --backends awkward,hybrid,rdataframe --repeats 5
+```
+
+The backends cross-check on selected-event count and mean mass (the harness prints
+`agreement across backends: OK`). Pushing the physics into the database with the UDF (`monetdb` backend) returns
+only the final selection, so it can beat the fetch-and-reconstruct `hybrid` path
+by a wide margin on this query. The embedded hybrid backend loads via `INSERT`
+(slow); use `--hybrid-backend server` (a real MonetDB server with `COPY INTO`) to
+benchmark a realistic load path.
+
+### When SQL wins vs when Awkward wins
+
+The harness ships two analyses (`--analysis`) that show the crossover:
+
+- `zmumu` (flat: two muons/event, a scalar mass) **favors the database** — the
+  in-DB `monetdb` backend runs the whole selection in SQL and beats
+  fetch-and-reconstruct.
+- `trijet` (ADL Q6, jagged: every 3-jet combination, keep the mass closest to
+  172.5 GeV) **favors Awkward** — `ak.combinations(jets, 3)` is one vectorized
+  line, whereas the SQL equivalent (`schemas/udf_trijet.sql`) needs two UDFs, a
+  3-way self-join over jet triples, and a window function — and runs *slower* on
+  the same data.
+
+```bash
+awkward-monetizer bench --analysis trijet --root-file data/nano_synth.root \\
+    --backends awkward,monetdb
+```
+
+The relational store is the right tool for columnar filtering, materialization,
+and scalar arithmetic; Awkward is the right tool for variable-length
+combinatorics and nested (NF2) physics. The point of the hybrid design is to use
+each where it wins, not to replace one with the other.
+
+## Ingesting real CMS Open Data NanoAOD
+
+The `nanoaod` dataset reads real CMS NanoAODv9 files directly (uproot pulls only
+the branches we map, so a multi-GB file isn't fully read). Get a file — e.g. the
+DoubleMuon sample from the [CMS NanoAOD guide](https://opendata.cern.ch/docs/cms-getting-started-nanoaod) —
+via XRootD and stream it into a running MonetDB `hep` server in chunks:
+
+```bash
+xrdcp root://eospublic.cern.ch//eos/opendata/cms/Run2016H/DoubleMuon/NANOAOD/UL2016_MiniAODv2_NanoAODv9-v1/2510000/127C2975-1B1C-A046-AABF-62B77E757A86.root .
+awkward-monetizer ingest 127C2975-*.root --dataset nanoaod --database hep \\
+    --create-schema --step-size "100 MB" --entry-stop 200000
+```
+
+`--step-size` streams with `uproot.iterate` (each chunk is offset so `event_id`
+stays globally unique), keeping memory flat regardless of file size;
+`--create-schema` (re)creates the tables first. Then the `nanoaod` ADL queries
+and benchmarks run on real physics.
+
+`--step-size 50000` sets an entry count instead of a memory size. Add `--dry-run`
+to build and summarize chunks without connecting to MonetDB, or `--truncate`
+to clear the dataset's tables once before loading the chunks.
+
+## Testing
+
+```bash
+pytest        # or: pixi run test
+```
+
+The ADL suite runs on generated synthetic data; the dimuon and round-trip tests
+use `data/cms.root` and skip cleanly if it isn't present.
+
+## Status
+
+Ingestion, reconstruction, the live MonetDB round-trip, and ADL Q1–Q8 are
+implemented and validated (round-trip verified on a real MonetDB server; the
+reconstructed dimuon mass matches the stored `M` to ~1e-8). See `docs/roadmap.md`
+for what's next (real NanoAOD data, ROOT/RDataFrame benchmarks, Arrow Flight).
+
+## License
+
+BSD-3-Clause. Contributions welcome — see `CONTRIBUTING.md`.

awkward_monetizer-0.1.1/data/README.md

@@ -0,0 +1,20 @@
+# data/
+
+Data files are **not** committed (see `.gitignore`). Put inputs here.
+
+## `cms.root` — dimuon sample
+The flat CMS dimuon teaching ntuple (tree `events`, 2,304 events; branches
+`Run, Event, Type, M` and two muons as `pt1/pt2`, `Q1/Q2`, …). It's the default
+input for the `dimuon` dataset. This is a well-known CMS open-data outreach file
+(the "Dimuon" / DoubleMu sample); drop your copy here as `data/cms.root`.
+
+## `nano_synth.root` — synthetic NanoAOD
+Generated on demand for the NanoAOD path and the ADL benchmark:
+
+```
+awkward-monetizer make-nano            # writes data/nano_synth.root
+# or:  python examples/make_sample_nanoaod.py
+```
+
+To run the ADL queries on **real** data, point `--from-root` at a CMS Open Data
+NanoAOD file instead.