readcon-db 0.2.0__tar.gz → 0.2.1__tar.gz

readcon_db-0.2.1/.github/workflows/lint.yml

@@ -0,0 +1,47 @@
+name: Lint
+
+on:
+  push:
+    branches: [main, "fix/**", "feat/**", "ci/**"]
+  pull_request:
+    branches: [main]
+  workflow_dispatch:
+jobs:
+  lint:
+    runs-on: ubuntu-latest
+
+    steps:
+    - uses: actions/checkout@v6
+      with:
+        fetch-depth: 0
+
+    - name: Conventional commits check
+      uses: cocogitto/cocogitto-action@v3
+      with:
+        check-latest-tag-only: true
+
+    # large-file-auditor is omitted: this clone already has historical
+    # target/debug blobs above 1M. Rewriting shared history is a separate job.
+
+  version:
+    runs-on: ubuntu-latest
+
+    steps:
+    - uses: actions/checkout@v6
+
+    - uses: actions/setup-python@v6
+      with:
+        python-version: '3.12'
+
+    - run: python -m pip install cogapp==3.6.0 meson==1.12.1
+
+    - name: Every version copy matches Cargo.toml
+      run: python scripts/check_version_lockstep.py
+
+    - name: Drift the sync misses still fails the check
+      run: python scripts/test_check_version_lockstep.py
+
+    - name: The sync leaves the tree unchanged
+      run: |
+        python scripts/sync-version.py
+        git diff --exit-code

{readcon_db-0.2.0 → readcon_db-0.2.1}/CHANGELOG.md

@@ -1,5 +1,12 @@
 # Changelog
 
+## v0.2.1 - 2026-11-11
+#### Bug Fixes
+- stream corpus append and parse one stored frame - (623ff00) - *HaoZeke*
+#### Documentation
+- document streaming append and publish the API reference - (16baa43) - *HaoZeke*
+- rebuild the tutorial on recorded output and fix stale install, C, and link defects (#26) - (9cf3abf) - *HaoZeke*
+
 ## v0.2.0 - 2026-10-15
 #### Features
 - (**frames**) store CON text as a tagged zstd frame - (838df21) - *HaoZeke*

{readcon_db-0.2.0 → readcon_db-0.2.1}/CITATION.cff

@@ -9,7 +9,9 @@ authors:
     orcid: "https://orcid.org/0000-0002-2393-8056"
 license: MIT
 repository-code: "https://github.com/lode-org/readcon-db"
-version: 0.2.0
+# [[[cog cog.outl(f'version: {VERSION}') ]]]
+version: 0.2.1
+# [[[end]]]
 date-released: "2026-08-15"
 keywords:
   - CON

{readcon_db-0.2.0 → readcon_db-0.2.1}/Cargo.lock

@@ -868,7 +868,7 @@ dependencies = [
 
 [[package]]
 name = "readcon-db"
-version = "0.2.0"
+version = "0.2.1"
 dependencies = [
  "heed",
  "libc",

{readcon_db-0.2.0 → readcon_db-0.2.1}/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "readcon-db"
-version = "0.2.0"
+version = "0.2.1"
 edition = "2021"
 description = "Mmap-backed CON frame corpus (Heed/LMDB), xxHash exact match, multi-language FFI"
 license = "MIT"

{readcon_db-0.2.0 → readcon_db-0.2.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: readcon-db
-Version: 0.2.0
+Version: 0.2.1
 Summary: Mmap CON corpus store (LMDB/Heed) with xxHash exact match
 Home-Page: https://lode-org.github.io/readcon-db/
 License: MIT
@@ -86,6 +86,8 @@ let h = db.frame_hash(keys[0])?;
 
 Foreign trajectories: **`readcon.read_chemfiles("traj.xyz")` → frames → ingest into readcon-db** (chemfiles-enabled build), not `ase.io.read`.
 
+Python `append_trajectory_frames` commits 64 frames at a time. `get_frame` returns a `StoredFrame` (`atoms[i]` builds that atom only). Gzip and zstd files are read with `readcon.read_con`, then appended. Python `append_trajectory` and Rust `append_trajectory_path` read UTF-8 and do not decompress. `readcon-db` does not register an `ase.ioformats` entry point. Guide: [`docs/orgmode/howto/streaming.org`](docs/orgmode/howto/streaming.org). Upgrade from 0.2.0: [`docs/orgmode/howto/migration.org`](docs/orgmode/howto/migration.org).
+
 ## Design
 
 - **No SQL engine** — explicit indexes + in-process intersection, with **ASE.db-competitive screening fields** (mass, volume, PBC, reserved metadata, charge/magmom; see [design matrix](docs/orgmode/explanation/design.org)).
@@ -119,6 +121,11 @@ Foreign trajectories: **`readcon.read_chemfiles("traj.xyz")` → frames → inge
 
 Full ABI table, logo, Sphinx docs, and site: see `docs/`, `website/`, `assets/logo/`, `CHANGELOG.md`. Fortran module notes: `fortran/README.md`, `fortran/ReadConDb/`.
 
+## Questions and bug reports
+
+Ask questions and report bugs on the [GitHub issue tracker](https://github.com/lode-org/readcon-db/issues).
+Report a security problem by email instead, as [`SECURITY.md`](SECURITY.md) describes.
+
 ## License
 
 MIT

{readcon_db-0.2.0 → readcon_db-0.2.1}/README.md

@@ -73,6 +73,8 @@ let h = db.frame_hash(keys[0])?;
 
 Foreign trajectories: **`readcon.read_chemfiles("traj.xyz")` → frames → ingest into readcon-db** (chemfiles-enabled build), not `ase.io.read`.
 
+Python `append_trajectory_frames` commits 64 frames at a time. `get_frame` returns a `StoredFrame` (`atoms[i]` builds that atom only). Gzip and zstd files are read with `readcon.read_con`, then appended. Python `append_trajectory` and Rust `append_trajectory_path` read UTF-8 and do not decompress. `readcon-db` does not register an `ase.ioformats` entry point. Guide: [`docs/orgmode/howto/streaming.org`](docs/orgmode/howto/streaming.org). Upgrade from 0.2.0: [`docs/orgmode/howto/migration.org`](docs/orgmode/howto/migration.org).
+
 ## Design
 
 - **No SQL engine** — explicit indexes + in-process intersection, with **ASE.db-competitive screening fields** (mass, volume, PBC, reserved metadata, charge/magmom; see [design matrix](docs/orgmode/explanation/design.org)).
@@ -106,6 +108,11 @@ Foreign trajectories: **`readcon.read_chemfiles("traj.xyz")` → frames → inge
 
 Full ABI table, logo, Sphinx docs, and site: see `docs/`, `website/`, `assets/logo/`, `CHANGELOG.md`. Fortran module notes: `fortran/README.md`, `fortran/ReadConDb/`.
 
+## Questions and bug reports
+
+Ask questions and report bugs on the [GitHub issue tracker](https://github.com/lode-org/readcon-db/issues).
+Report a security problem by email instead, as [`SECURITY.md`](SECURITY.md) describes.
+
 ## License
 
 MIT

{readcon_db-0.2.0 → readcon_db-0.2.1}/docs/orgmode/contributing/index.org

@@ -39,8 +39,8 @@ pixi lock
 | Prek             | =ci_prek.yml=                       | push, PR                                 | =prek run -a=                                                    |
 | Documentation    | =ci_docs.yml=                       | push, PR                                 | Locked Sphinx HTML (=pixi.lock=) + assembled site + lychee       |
 | Pages            | =pages.yml=                         | push to main                             | Deploy site + locked Sphinx docs                                 |
-| Lint             | =lint.yml=                          | PR                                       | Conventional commits + large-file audit                          |
-| Version lockstep | =scripts/check_version_lockstep.sh= | =cargo test=                             | Cargo, Python, meson, pixi, Fortran, CITATION, Sphinx, core pin  |
+| Lint             | =lint.yml=                          | PR                                       | Conventional commits; version copies match Cargo.toml            |
+| Version lockstep | =scripts/check_version_lockstep.py= | =lint.yml=, =set-version.sh=             | Each copy parsed and compared with Cargo.toml; readcon-core pin  |
 | C/C++ dist       | =ci_cxx.yml=                        | push, PR                                 | CMake/Meson/pkg-config without cbindgen; clib fixture gate       |
 | crates.io        | =crates_publish.yml=                | =v*= tag                                 | =cargo publish --locked=                                         |
 | Python wheels    | =python-wheels.yml=                 | =v*= tag, PR                             | maturin matrix → PyPI                                            |

{readcon_db-0.2.0 → readcon_db-0.2.1}/docs/orgmode/cooked-soa.org

@@ -1,13 +1,17 @@
-#+title: Cooked SoA tier (RCSO / frames_soa)
-#+options: toc:2
+#+OPTIONS: toc:nil num:nil
 
-* Contract
+* Cooked structure-of-arrays (SoA) tier (=RCSO= / =frames_soa=)
+:PROPERTIES:
+:CUSTOM_ID: cooked-soa
+:END:
+
+** Contract
 
 RCSO is *non-authoritative*. CON text in =frames= is the sole authority
 for hash, dedup, join/split, =reindex=, and CON export. Cooked numerics
 in =frames_soa= are derived POD only; they never replace the text tier.
 
-* Summary
+** Summary
 
 =readcon-db= stores two tiers per frame key:
 
@@ -18,11 +22,11 @@ in =frames_soa= are derived POD only; they never replace the text tier.
 
 *Default ingest does not cook.* Call =cook_frame= / =recook_all= / CLI =cook-frame= / =recook-all= (or =append_trajectory_path_cook(..., true)= in Rust) to populate RCSO. =delete_cooked_soa= drops only the cooked tier; CON text and indexes stay.
 
-* Why CON text is still required
+** Why the =.con= text is still required
 
 RCSO is *not* fully equivalent to CON: it omits element symbols, masses, cell/angles, constraints, JSON metadata, and exact on-disk bytes. Those are required for xxHash3 dedup, symbol/formula indexes, =reindex=, join/split, and CON export. Do *not* omit =frames= when cooked exists.
 
-* Numeric hot path
+** Numeric hot path
 
 =get_positions= / =get_forces= / =get_velocities= (Rust, Python, C
 =rkrdb_get_positions= / =rkrdb_get_forces= / =rkrdb_get_velocities=, CLI
@@ -31,7 +35,7 @@ Missing or corrupt cooked falls back to parsing CON text. Authority APIs
 (=frame_hash=, =find_by_hash=, =get_frame_text=, =reindex=, shard join)
 never read =frames_soa=.
 
-* Language surfaces
+** Language surfaces
 
 | Surface | Cook / delete / has-valid | Positions / forces / velocities |
 |---------+---------------------------+--------------------------------|
@@ -41,7 +45,7 @@ never read =frames_soa=.
 | Fortran | =db_cook_frame=, =db_delete_cooked=, =db_has_valid_cooked=, =db_recook_all= | =db_get_positions=, =db_get_forces=, =db_get_velocities= |
 | CLI | =cook-frame=, =delete-cooked=, =recook-all= | =positions= |
 
-* See also
+** See also
 
-- Internal design notes: =docs/orgmode/explanation/design.org= (engineer-facing; mirrors this contract)
-- Architecture Sphinx page (synced summary from this org)
+- Internal design notes: @@rst::doc:`/explanation/design`@@ (engineer-facing; mirrors this contract)
+- Architecture summary: @@rst::doc:`/explanation/architecture`@@

{readcon_db-0.2.0 → readcon_db-0.2.1}/docs/orgmode/explanation/architecture.org

@@ -49,7 +49,10 @@ Environment (Heed / LMDB)
 
 *Drain/join:* node-local =shard-ingest=, =drain_to= compact-snapshots =data.mdb= (refuse overwrite), then =join-drained=. =compact-join= is the single-root join (=open_existing=). Ops: @@rst::doc:`/howto/campaign`@@.
 
-*Cooked SoA tier:* optional RCSO in =frames_soa= accelerates =get_positions= / =get_forces= / =get_velocities= without CON parse when valid. RCSO is non-authoritative: CON text in =frames= remains sole authority for hash/dedup/join/reindex. RCSO is not fully equivalent (no symbols/metadata/exact bytes). See =docs/orgmode/cooked-soa.org=.
+*Cooked SoA tier:* optional =RCSO= in =frames_soa= accelerates =get_positions= / =get_forces= / =get_velocities= without a =.con= parse when valid.
+=RCSO= is non-authoritative: the =.con= text in =frames= remains sole authority for hash/dedup/join/reindex.
+=RCSO= is not fully equivalent (no symbols/metadata/exact bytes).
+See @@rst::doc:`/cooked-soa`@@.
 
 =FrameKey= is 12 bytes: =traj_id= (BE u64) + =frame_idx= (BE u32) so lexicographic order matches numeric order.
 

{readcon_db-0.2.0 → readcon_db-0.2.1}/docs/orgmode/howto/campaign.org

@@ -114,7 +114,7 @@ ConCorpus("/lustre/orion/proj/campaign_single", readonly=True).export_h5md(1, "t
 :PROPERTIES:
 :CUSTOM_ID: units
 :END:
-Stamp units on ingest (=--units= JSON, or C =rkrdb_append_trajectory_units= / Fortran =db_append_units=). Aliases canonicalize (=A= -> =angstrom=). =set_units= converts stored numbers; append/extend stamp incoming values. CON line-2 =units= is the authority for length, energy, and the clock. A v3.1 frame with =time= or =timestep= and no =units.time= is an error. =--velocity-time-unit= supplies that missing clock unit and overrides the D10 velocity rule for a file that breaks it. A present =units.time= still scales the clock. v1, v2, and v3.0 clocks with no =units.time= are read as =fs=. H5MD velocities follow D10: v1, v2, and v3.0 use the natural unit =(amu*angstrom^2/eV)^0.5= whatever the time label says; v3.1 uses =units.time= and fails when velocities have no =units.time=.
+Stamp units on ingest (=--units= JSON, or C =rkrdb_append_trajectory_units= / Fortran =db_append_units=). Aliases canonicalize (=A= -> =angstrom=). =set_units= converts stored numbers and refuses a v3.1 clock or velocities frame with no =units.time= instead of assuming =fs=; append/extend stamp incoming values. CON line-2 =units= is the authority for length, energy, and the clock. A v3.1 frame with =time= or =timestep= and no =units.time= is an error. =--velocity-time-unit= supplies that missing clock unit and overrides the D10 velocity rule for a file that breaks it. A present =units.time= still scales the clock. v1, v2, and v3.0 clocks with no =units.time= are read as =fs=. H5MD velocities follow D10: v1, v2, and v3.0 use the natural unit =(amu*angstrom^2/eV)^0.5= whatever the time label says; v3.1 uses =units.time= and fails when velocities have no =units.time=.
 
 ** C / Fortran writers
 :PROPERTIES:

{readcon_db-0.2.0 → readcon_db-0.2.1}/docs/orgmode/howto/howto.org

@@ -44,17 +44,17 @@ Exact match: =Select::new().exact_hash(h.to_bytes())=.
 :CUSTOM_ID: python
 :END:
 #+begin_src python
-import readcon_db
-
-db = readcon_db.ConCorpus.open("/tmp/corpus")
-db.append_trajectory_path(1, "run.con")
-sel = readcon_db.Select().require_symbol("Cu")
-keys = db.select(sel)
-h = db.frame_hash(keys[0])
-text = db.get_frame_text(keys[0])
+from readcon_db import ConCorpus
+
+db = ConCorpus("/tmp/corpus")
+db.append_trajectory(1, "run.con")
+keys = db.select(symbol="Cu")
+traj_id, frame_idx = keys[0]
+h = db.frame_hash(traj_id, frame_idx)
+text = db.get_frame_text(traj_id, frame_idx)
 #+end_src
 
-=maturin develop --features python= from a checkout. Module name is =readcon_db=.
+=maturin develop --features python= from a checkout. Module name is =readcon_db=. Streaming append, =StoredFrame=, and compressed files: @@rst::doc:`/howto/streaming`@@. The 0.2.1 changes: @@rst::doc:`/howto/migration`@@.
 
 ** C / C++
 :PROPERTIES:
@@ -63,12 +63,21 @@ text = db.get_frame_text(keys[0])
 #+begin_src C
 #include "readcon-db.h"
 
-RkrdbCorpus *db = rkrdb_open("/tmp/corpus");
-rkrdb_append_trajectory_path(db, 1, "run.con");
-/* Select via rkrdb_select_* / rkrdb_select_meta; see api_c */
-rkrdb_close(db);
+size_t id;
+uint32_t n_frames;
+rkrdb_open("/tmp/corpus", &id);
+rkrdb_append_trajectory(id, 1, "run.con", &n_frames);
+/* traj_id -1 matches every trajectory; limit 0 means no limit */
+rkrdb_select_basic(id, -1, "Cu", 0, UINT32_MAX, 0);
+for (int i = 0; i < rkrdb_result_count(id); i++) {
+    uint64_t traj;
+    uint32_t frame;
+    rkrdb_result_key(id, i, &traj, &frame);
+}
+rkrdb_close(id);
 #+end_src
 
+Each call returns =RKRDB_OK= on success, and =rkrdb_last_error= copies the message of the last failure into a caller buffer.
 C++ RAII: =readcon_db::Corpus=. MPI pack/bcast takes the *caller* communicator (=readcon-db-mpi.h=). The library does not call =MPI_Init=. Details: @@rst::doc:`/reference/c`@@.
 
 ** Fortran

{readcon_db-0.2.0 → readcon_db-0.2.1}/docs/orgmode/howto/install.org

@@ -14,7 +14,7 @@ cargo add readcon-db
 cargo install readcon-db --locked
 #+end_src
 
-Requires [[https://crates.io/crates/readcon-core][readcon-core]] *^0.14* (pulled automatically).
+Pulls in [[https://crates.io/crates/readcon-core][readcon-core]] at the version requirement ~{{{readcon-core-req}}}~ from =Cargo.toml=.
 
 ** Python (PyPI)
 :PROPERTIES:
@@ -38,7 +38,8 @@ cargo test --locked
 cargo build --release   # libreadcon_db + CLI readcon-db
 #+end_src
 
-=readcon-core= comes from crates.io (=^0.14=). To develop against a local core tree, add an *untracked* =.cargo/config.toml=:
+=readcon-core= comes from crates.io (~{{{readcon-core-req}}}~).
+Add an *untracked* =.cargo/config.toml= with the patch below to build against a local core tree.
 
 #+begin_src toml
 [patch.crates-io]
@@ -64,7 +65,7 @@ CMake:
 include(FetchContent)
 FetchContent_Declare(
   readcon-db
-  URL https://github.com/lode-org/readcon-db/releases/download/v0.1.8/readcon-db-cxx-0.1.8.tar.gz
+  URL https://github.com/lode-org/readcon-db/releases/download/v{{{readcon-db-version}}}/readcon-db-cxx-{{{readcon-db-version}}}.tar.gz
   URL_HASH SHA256=<sha256 from the .sha256 sidecar on the GitHub Release>
 )
 FetchContent_MakeAvailable(readcon-db)
@@ -91,7 +92,7 @@ A process that uses both C APIs must link the *shared* objects (=libreadcon_db.s
 Unpack =readcon-db-clib-$VERSION-$target.tar.gz= from the GitHub Release, then =export READCON_DB_LIB= / =PKG_CONFIG_PATH=. Attach assets to an already-published tag with Actions → *C ABI library tarball* → ~tag=vX.Y.Z~.
 
 #+begin_src sh
-VER=0.1.8
+VER={{{readcon-db-version}}}
 TARGET=x86_64-unknown-linux-gnu
 curl -fsSL -O "https://github.com/lode-org/readcon-db/releases/download/v${VER}/readcon-db-clib-${VER}-${TARGET}.tar.gz"
 tar -xzf "readcon-db-clib-${VER}-${TARGET}.tar.gz"
@@ -137,7 +138,8 @@ Campaign shard/drain/join is the =readcon-db= CLI (in =bin/= of the clib tarball
 pixi r -e docs docbld
 #+end_src
 
-=pixi.lock= pins Sphinx, Furo, and MyST to the same set Pages publishes. Refresh with =pixi lock= after editing =pixi.toml= =[feature.docs]=.
+=pixi.lock= pins Sphinx, Shibuya, and Markedly Structured Text (MyST) to the same set Pages publishes.
+Refresh with =pixi lock= after editing =pixi.toml= =[feature.docs]=.
 
 ** Static marketing page
 :PROPERTIES:

readcon_db-0.2.1/docs/orgmode/howto/migration.org

@@ -0,0 +1,38 @@
+#+OPTIONS: toc:nil num:nil
+
+* Migrating to 0.2.1
+:PROPERTIES:
+:CUSTOM_ID: migrating-to-0-2-1
+:END:
+
+0.2.1 keeps the same corpus files as 0.2.0. Two Python calls behave differently. Rust =get_frame= still returns a =readcon_core::ConFrame=. Rust =append_trajectory_frames= still commits the slice you pass in one transaction.
+
+** =get_frame= returns a =StoredFrame=
+:PROPERTIES:
+:CUSTOM_ID: get-frame-returns-a-storedframe
+:END:
+In 0.2.0, Python =get_frame= parsed the stored text with =readcon.read_con_string= and returned a =readcon.ConFrame=. Reading =atoms= on that object built a Python atom for every site.
+
+In 0.2.1 it returns a =StoredFrame=:
+
+| Still works | =frame.xyz=, =frame.atoms[i].symbol=, =frame.atoms[i].x= / =y= / =z=, =frame.cell=, =frame.angles=, =len(frame)= |
+| Changed | =type(frame)= is =StoredFrame=, not =ConFrame=. There is no =to_ase=, no metadata dict, and no velocity or force attributes on =StoredAtom=. |
+
+=atoms[i]= builds that one atom. Use =get_frame_text= when you need the text, and =readcon.read_con_string= when you need a =ConFrame=:
+
+#+begin_src python
+text = db.get_frame_text(traj_id, frame_idx)
+frame = readcon.read_con_string(text)[0]
+#+end_src
+
+Positions, forces, and velocities are also still on =get_positions=, =get_forces=, and =get_velocities=.
+
+** Append commits 64 frames at a time
+:PROPERTIES:
+:CUSTOM_ID: append-commits-64-frames-at-a-time
+:END:
+In 0.2.0, Python =append_trajectory_frames= turned the whole trajectory into one CON string, then prepared and committed every frame in one transaction. The string and the prepared rows were resident together, and walking a =read_con= result built a Python frame for every index.
+
+In 0.2.1 the same call commits groups of 64 frames. The first group creates the trajectory. Each later group extends it. The return value is still the number of frames in the trajectory after the last commit. A =read_con= list is serialized from the parser's stored frames, so those Python frames are not built.
+
+Python =append_trajectory= and Rust =append_trajectory_path= are unchanged: they read the path as UTF-8 and do not decompress gzip or zstd. Compressed files go through =readcon.read_con=, then =append_trajectory_frames=. The guide is @@rst::doc:`/howto/streaming`@@.

readcon_db-0.2.1/docs/orgmode/howto/streaming.org

@@ -0,0 +1,149 @@
+#+OPTIONS: toc:nil num:nil
+
+* Streaming append and random reads
+:PROPERTIES:
+:CUSTOM_ID: streaming-append-and-random-reads
+:END:
+
+#+BEGIN_EXPORT rst
+.. note::
+
+   Upgrading from 0.2.0: :doc:`/howto/migration`.
+#+END_EXPORT
+
+=readcon-db= stores CON text. =readcon= (the =readcon-core= Python package) parses and writes that text. ASE is a different store. This page is the path from a trajectory file to one stored frame.
+
+** Create a corpus
+:PROPERTIES:
+:CUSTOM_ID: create-a-corpus
+:END:
+A corpus is a directory. Creating =ConCorpus= creates it. The directory holds =data.mdb= and =lock.mdb=. One process writes. Many processes may open the same directory with =readonly=True=.
+
+#+begin_src python
+from readcon_db import ConCorpus
+
+db = ConCorpus("/tmp/corpus")
+ro = ConCorpus("/tmp/corpus", readonly=True)
+#+end_src
+
+Rust is =ConCorpus::open= and =ConCorpus::open_readonly=. The command line is =readcon-db ingest=.
+
+** Stream an append
+:PROPERTIES:
+:CUSTOM_ID: stream-an-append
+:END:
+=append_trajectory_frames= commits 64 frames at a time. The first group is a new trajectory. Each later group extends it. A list returned by =read_con= is written from the frames the parser already stored, so the append does not build a Python atom for every site.
+
+Rust =ConCorpus::append_trajectory_frames= still commits the slice you pass in one transaction. The grouping is the Python binding.
+
+** Random access
+:PROPERTIES:
+:CUSTOM_ID: random-access
+:END:
+=get_frame(traj_id, frame_idx)= returns a =StoredFrame=.
+
+| Attribute | Contents |
+|-----------+----------|
+| =xyz= | one =(x, y, z)= row per atom |
+| =atoms= | a =StoredAtoms= list |
+| =atoms[i]= | one =StoredAtom= with =symbol=, =x=, =y=, =z= |
+| =cell= | three box lengths |
+| =angles= | three box angles, in degrees |
+| =len(frame)= | the atom count |
+
+Indexing =atoms= builds that atom and no others. =get_frame_text= returns the CON text. =readcon.read_con_string= on that text returns a =readcon.ConFrame= when you need the full object.
+
+** Compressed inputs
+:PROPERTIES:
+:CUSTOM_ID: compressed-inputs
+:END:
+Python =append_trajectory= and Rust =append_trajectory_path= read the file as UTF-8. They do not decompress gzip or zstd. Read the compressed file with =readcon.read_con=, which selects gzip or zstd from the extension, then append the frames:
+
+#+begin_src python
+frames = readcon.read_con("traj.con.gz")  # or traj.con.zst
+db.append_trajectory_frames(1, frames, source="traj.con.gz")
+#+end_src
+
+A =.con.zst= file requires a =readcon= build with the =zstd= feature. Plain =.con= files can go straight to =append_trajectory= (Python) or =append_trajectory_path= (Rust).
+
+** readcon-core
+:PROPERTIES:
+:CUSTOM_ID: readcon-core
+:END:
+=readcon-core= is the CON parser and writer. The corpus stores the text that parser produces and decodes a hit with the same library. Foreign formats (XYZ, PDB, GRO) enter through =readcon.read_chemfiles= on a chemfiles build, then =append_trajectory_frames=. They do not enter through ASE.
+
+** ASE
+:PROPERTIES:
+:CUSTOM_ID: ase
+:END:
+=readcon-db= does not register an =ase.ioformats= entry point, and neither does =readcon=. ASE reads a CON file with its built-in =eon= format:
+
+#+begin_src python
+import ase.io
+ase.io.read("traj.con", format="eon")
+#+end_src
+
+That call does not open a corpus and does not use =readcon=. =ase.db= and =ase.io.Trajectory= are the stores a benchmark compares against. They are not how this corpus stores frames.
+
+A calculator that needs an =ase.Atoms= object can be built from a =StoredFrame=. =readcon.ConFrame.to_ase= applies to a frame parsed by =readcon=, including one recovered with =read_con_string(db.get_frame_text(...))=.
+
+** Example
+:PROPERTIES:
+:CUSTOM_ID: example
+:END:
+This block is executed by =python/tests/test_streaming_guide.py=.
+
+#+begin_src python
+# runnable
+import gzip
+import importlib.metadata as metadata
+from pathlib import Path
+
+import readcon
+from readcon_db import ConCorpus
+
+root = Path("guide")
+root.mkdir()
+sites = [readcon.Atom("Cu", float(i), 0.0, 1.0, atom_id=i) for i in range(4)]
+frame = readcon.ConFrame((12.0, 12.0, 12.0), (90.0, 90.0, 90.0), sites)
+plain = root / "traj.con"
+readcon.write_con(str(plain), [frame] * 70)
+gz = root / "traj.con.gz"
+gz.write_bytes(gzip.compress(plain.read_bytes()))
+
+db = ConCorpus(str(root / "db"))
+rejected = False
+try:
+    db.append_trajectory(1, str(gz))
+except RuntimeError:
+    rejected = True
+assert rejected
+
+loaded = readcon.read_con(str(gz))
+n = db.append_trajectory_frames(1, loaded, source=gz.name)
+assert n == 70
+got = db.get_frame(1, 64)
+assert type(got).__name__ == "StoredFrame"
+assert type(got.atoms).__name__ == "StoredAtoms"
+assert got.atoms[0].symbol == "Cu"
+assert got.atoms[-1].x == 3.0
+assert tuple(got.cell) == (12.0, 12.0, 12.0)
+text = db.get_frame_text(1, 0)
+full = readcon.read_con_string(text)[0]
+assert full.atoms[0].symbol == got.atoms[0].symbol
+
+from ase import Atoms
+
+atoms = Atoms(
+    symbols=[atom.symbol for atom in got.atoms],
+    positions=got.xyz,
+    cell=list(got.cell),
+    pbc=True,
+)
+assert len(atoms) == len(got)
+
+eps = metadata.entry_points()
+group = eps.select(group="ase.ioformats") if hasattr(eps, "select") else eps.get("ase.ioformats", [])
+registered = [ep for ep in group if "readcon_db" in ep.value]
+assert registered == []
+#+end_src

{readcon_db-0.2.0 → readcon_db-0.2.1}/docs/orgmode/index.org

@@ -20,9 +20,11 @@
 
 * readcon-db
 
-Rare-event codes already checkpoint on CON. This crate keeps thousands of
-those frames in one mmap tree: ingest CON text, filter without loading every
-atom, decode a hit with [[https://lode-org.github.io/readcon-core/][readcon-core]].
+Rare-event codes already checkpoint frames in the plain-text structure format (CON).
+This crate keeps thousands of those frames in one memory-mapped tree: ingest CON text, filter without loading every atom, decode a hit with [[https://lode-org.github.io/readcon-core/][readcon-core]].
+
+Ask questions and report bugs on the [[https://github.com/lode-org/readcon-db/issues][GitHub issue tracker]].
+Report a security problem by email instead, as =SECURITY.md= in the repository describes.
 
 #+BEGIN_EXPORT rst
 :doc:`/tutorials/getting-started` · :doc:`/tutorials/tutorial` · :doc:`/howto/howto` · :doc:`/explanation/architecture` ·
@@ -44,11 +46,11 @@ atom, decode a hit with [[https://lode-org.github.io/readcon-core/][readcon-core
 .. grid:: 1 1 2 2
    :gutter: 2
 
-   .. grid-item-card:: Tutorial: first corpus
+   .. grid-item-card:: Tutorial: Your first corpus
       :link: tutorials/tutorial
       :link-type: doc
 
-      Open a directory, ingest a fixture, select copper, print the hash.
+      Ingest three files, filter them by element and energy, and find a repeated trajectory.
 
    .. grid-item-card:: How-to by language
       :link: howto/howto
@@ -83,6 +85,8 @@ Site map
    :caption: How-to guides
 
    howto/howto
+   howto/streaming
+   howto/migration
    howto/topology
    howto/install
    howto/campaign
@@ -97,6 +101,7 @@ Site map
    explanation/overview
    explanation/architecture
    explanation/design
+   cooked-soa
 
 .. toctree::
    :maxdepth: 1
@@ -105,6 +110,7 @@ Site map
    reference/rust
    reference/c
    reference/python
+   reference/python-api
    reference/fortran
    reference/citation
 

{readcon_db-0.2.0 → readcon_db-0.2.1}/docs/orgmode/reference/c.org

@@ -92,7 +92,7 @@ rkrdb_close(id);
 
 Status: =RKRDB_OK= (0), =RKRDB_ERR= (-1), =RKRDB_NOT_FOUND= (-2), =RKRDB_NULL= (-3).
 
-H5MD velocities follow D10 and are written as =Angstrom ps-1=. v1, v2, and v3.0 frames use =(amu*angstrom^2/eV)^0.5= whatever the time label says. v3.1 frames use =units.time= and fail when a velocities section has no =units.time=. =rkrdb_set_velocity_time_unit= (C++ =Corpus::set_velocity_time_unit=) overrides that rule for a file that breaks it. Pass =fs= when the file really stored Angstrom/fs. The simulation clock uses =units.time= when that key is present. A v3.1 frame with =time= or =timestep= and no =units.time= is an error, and the same override supplies that missing clock unit. A present =units.time= still scales the clock. v1, v2, and v3.0 clocks with no =units.time= are read as =fs=.
+=rkrdb_h5md_*= returns =RKRDB_NULL= for an invalid handle. A collect failure returns =RKRDB_ERR= and leaves the message in =rkrdb_last_error=. =rkrdb_set_units= returns that missing-=units.time= error for a v3.1 clock or velocities frame instead of assuming =fs=. v1, v2, and v3.0 clocks with no =units.time= still convert as =fs=. H5MD velocities follow D10 and are written as =Angstrom ps-1=. v1, v2, and v3.0 frames use =(amu*angstrom^2/eV)^0.5= whatever the time label says. v3.1 frames use =units.time= and fail when a velocities section has no =units.time=. =rkrdb_set_velocity_time_unit= (C++ =Corpus::set_velocity_time_unit=) overrides that rule for a file that breaks it. Pass =fs= when the file really stored Angstrom/fs. The simulation clock uses =units.time= when that key is present. A v3.1 frame with =time= or =timestep= and no =units.time= is an error, and the same override supplies that missing clock unit. A present =units.time= still scales the clock. v1, v2, and v3.0 clocks with no =units.time= are read as =fs=.
 
 Link =libreadcon_db= from =cargo build --release= (=cdylib= / =staticlib=), or unpack the prebuilt =readcon-db-clib-$VERSION-$target.tar.gz= Release asset and =pkg-config --libs readcon-db=. See @@rst::doc:`/howto/install`@@.
 
@@ -167,4 +167,6 @@ C++ =readcon_db::Corpus::pack_frame= (~buf == NULL~ size query) / =pack_frames=
 :PROPERTIES:
 :CUSTOM_ID: cooked-soa-rcso
 :END:
-See =docs/orgmode/cooked-soa.org=. Tier is opt-in; CON text remains authority. Bindings expose cook / delete / has-valid / positions / forces / velocities.
+See @@rst::doc:`/cooked-soa`@@.
+Tier is opt-in; the =.con= text remains authority.
+Bindings expose cook / delete / has-valid / positions / forces / velocities.

{readcon_db-0.2.0 → readcon_db-0.2.1}/docs/orgmode/reference/fortran.org

@@ -52,4 +52,6 @@ H5MD interchange and caller units: =db_append_units=, =db_set_units=, =db_h5md_t
 :PROPERTIES:
 :CUSTOM_ID: cooked-soa-rcso
 :END:
-See =docs/orgmode/cooked-soa.org=. Tier is opt-in; CON text remains authority. Fortran exposes cook / delete / has-valid / =db_get_positions= / =db_get_forces= / =db_get_velocities=.
+See @@rst::doc:`/cooked-soa`@@.
+Tier is opt-in; the =.con= text remains authority.
+Fortran exposes cook / delete / has-valid / =db_get_positions= / =db_get_forces= / =db_get_velocities=.

readcon_db-0.2.1/docs/orgmode/reference/python-api.org

@@ -0,0 +1,14 @@
+#+OPTIONS: toc:nil num:nil
+
+* Generated Python API
+:PROPERTIES:
+:CUSTOM_ID: generated-python-api
+:END:
+
+This page is generated from the docstrings on =readcon_db=. The narrative guide is @@rst::doc:`/howto/streaming`@@.
+
+#+BEGIN_EXPORT rst
+.. automodule:: readcon_db
+   :members:
+   :undoc-members:
+#+END_EXPORT