readcon-db 0.2.1__cp315-cp315-win_amd64.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
readcon_db/__init__.py
ADDED
|
Binary file
|
|
@@ -0,0 +1,139 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: readcon-db
|
|
3
|
+
Version: 0.2.1
|
|
4
|
+
Summary: Mmap CON corpus store (LMDB/Heed) with xxHash exact match
|
|
5
|
+
Home-Page: https://lode-org.github.io/readcon-db/
|
|
6
|
+
License: MIT
|
|
7
|
+
Requires-Python: >=3.9
|
|
8
|
+
Description-Content-Type: text/markdown
|
|
9
|
+
Project-URL: Documentation, https://lode-org.github.io/readcon-db/
|
|
10
|
+
Project-URL: Homepage, https://github.com/lode-org/readcon-db
|
|
11
|
+
Project-URL: Issues, https://github.com/lode-org/readcon-db/issues
|
|
12
|
+
Project-URL: Repository, https://github.com/lode-org/readcon-db
|
|
13
|
+
|
|
14
|
+
# readcon-db
|
|
15
|
+
|
|
16
|
+
**Mmap-backed CON/convel corpus store** (LMDB via [Heed](https://github.com/meilisearch/heed)), **non-SQL selection**, **xxHash3-128 exact match**, and **Rust / C / C++ / Python / Fortran** bindings.
|
|
17
|
+
|
|
18
|
+
Part of the **readcon ecosystem** with [`readcon-core`](https://github.com/lode-org/readcon-core) (Python package **`readcon`**):
|
|
19
|
+
|
|
20
|
+
| Crate / package | Role | Docs |
|
|
21
|
+
|-----------------|------|------|
|
|
22
|
+
| **[readcon-core](https://github.com/lode-org/readcon-core)** / **`readcon`** | CON interchange (parse/write/spec v2–v3). **XYZ/PDB/GRO → `ConFrame` via chemfiles** (`read_chemfiles*`), not ASE. Optional `to_ase` only for calculators. | Core README, `docs/orgmode/` |
|
|
23
|
+
| **readcon-db** / **`readcon_db`** (this repo) | mmap CON corpus: indexes (natoms, symbols, **energy range**, **forces/velocities/energy flags**), multi-reader, dedup. Blobs are **CON text** decoded with readcon-core. | [docs](https://lode-org.github.io/readcon-db/docs/), [`docs/orgmode/explanation/design.org`](docs/orgmode/explanation/design.org) |
|
|
24
|
+
|
|
25
|
+
ASE is not on the CON or XYZ read path. Optional `to_ase` is calculator hand-off only.
|
|
26
|
+
|
|
27
|
+
## Install
|
|
28
|
+
|
|
29
|
+
```bash
|
|
30
|
+
cargo add readcon-db
|
|
31
|
+
cargo install readcon-db --locked # CLI
|
|
32
|
+
pip install readcon-db # module readcon_db (PyPI)
|
|
33
|
+
# C/C++: FetchContent / meson dependency('readcon-db') / pkg-config
|
|
34
|
+
# headers in include/ are shipped; cbindgen is not required
|
|
35
|
+
# Prebuilt C ABI (no cargo): readcon-db-clib-$VER-$target.tar.gz on the GitHub Release
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
Docs: <https://lode-org.github.io/readcon-db/> · crate: <https://crates.io/crates/readcon-db>
|
|
39
|
+
|
|
40
|
+
## Quick start (from source)
|
|
41
|
+
|
|
42
|
+
```bash
|
|
43
|
+
git clone https://github.com/lode-org/readcon-db
|
|
44
|
+
cd readcon-db
|
|
45
|
+
cargo test --locked
|
|
46
|
+
cargo build --release # libreadcon_db + CLI readcon-db
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
Optional LODE sibling checkout (edit core + db together): clone both under the same parent, then create **untracked** `.cargo/config.toml` in `readcon-db`:
|
|
50
|
+
|
|
51
|
+
```toml
|
|
52
|
+
[patch.crates-io]
|
|
53
|
+
readcon-core = { path = "../readcon-core" }
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
Python extension from a checkout (`python/` + maturin):
|
|
57
|
+
|
|
58
|
+
```bash
|
|
59
|
+
pip install maturin
|
|
60
|
+
maturin develop --release --features python --manifest-path python/pyproject.toml
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
```rust
|
|
64
|
+
use readcon_db::{ConCorpus, Select};
|
|
65
|
+
let db = ConCorpus::open("/tmp/corpus")?;
|
|
66
|
+
db.append_trajectory_path(1, "run.con")?;
|
|
67
|
+
// XYZ in: use readcon-core chemfiles → ConFrame → append (see workflows)
|
|
68
|
+
let keys = db.select(
|
|
69
|
+
&Select::new()
|
|
70
|
+
.require_symbol("Cu")
|
|
71
|
+
.require_forces()
|
|
72
|
+
.exact_composition("Cu:2|H:2")
|
|
73
|
+
.fmax_range(0.0, 1.0)
|
|
74
|
+
.energy_range(-50.0, 0.0),
|
|
75
|
+
)?;
|
|
76
|
+
let h = db.frame_hash(keys[0])?;
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
```bash
|
|
80
|
+
./target/release/readcon-db ingest-dir /tmp/corpus /path/to/con_files
|
|
81
|
+
./target/release/readcon-db select /tmp/corpus --formula 'Cu:2|H:2' --require-forces \
|
|
82
|
+
--fmax-max 1.0 --energy-min -50 --energy-max 0
|
|
83
|
+
./target/release/readcon-db reindex /tmp/corpus
|
|
84
|
+
./target/release/readcon-db dedup-export /tmp/corpus --symbol Cu -o subset.xyz # only if a tool demands XYZ on disk
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
Foreign trajectories: **`readcon.read_chemfiles("traj.xyz")` → frames → ingest into readcon-db** (chemfiles-enabled build), not `ase.io.read`.
|
|
88
|
+
|
|
89
|
+
Python `append_trajectory_frames` commits 64 frames at a time. `get_frame` returns a `StoredFrame` (`atoms[i]` builds that atom only). Gzip and zstd files are read with `readcon.read_con`, then appended. Python `append_trajectory` and Rust `append_trajectory_path` read UTF-8 and do not decompress. `readcon-db` does not register an `ase.ioformats` entry point. Guide: [`docs/orgmode/howto/streaming.org`](docs/orgmode/howto/streaming.org). Upgrade from 0.2.0: [`docs/orgmode/howto/migration.org`](docs/orgmode/howto/migration.org).
|
|
90
|
+
|
|
91
|
+
## Design
|
|
92
|
+
|
|
93
|
+
- **No SQL engine** — explicit indexes + in-process intersection, with **ASE.db-competitive screening fields** (mass, volume, PBC, reserved metadata, charge/magmom; see [design matrix](docs/orgmode/explanation/design.org)).
|
|
94
|
+
- **Decode via readcon-core** — CON semantics never fork.
|
|
95
|
+
- **Metadata indexes** — finite `energy` bins; flags for forces, velocities, energy presence.
|
|
96
|
+
- **xxHash3-128** on the CON text. A new `frames` value is `0xFD 0x01` plus one zstd level-1 frame of that UTF-8, written outside the LMDB transaction. An aligned run of 64 frames shares one `0xFD 0x03` zstd frame when that group is shorter than the per-frame values, and the other keys store a `0xFD 0x02` redirect. Untagged UTF-8 still reads as a 0.1.8 span. `readcon-db migrate` rewrites legacy values once and leaves an already packed value unchanged. `--no-compress` on `ingest`, `ingest-dir`, and `shard-ingest` stores the UTF-8. There is no dictionary. Both MPI ingest routes send the stored bytes. The hash is the decompressed text.
|
|
97
|
+
- **Many readers, one writer** (LMDB). Same-frame MPI: rank 0 of the
|
|
98
|
+
**caller communicator** packs RCSO and `MPI_Bcast` on that handle
|
|
99
|
+
(`include/readcon-db-mpi.h`, Python `bcast_packed_frame` /
|
|
100
|
+
`bcast_packed_frames`). The library never `MPI_Init`s and never names
|
|
101
|
+
the process-wide world communicator; LAMMPS / mpi4py pass the comm they
|
|
102
|
+
already own.
|
|
103
|
+
- **H5MD interchange** — `export_h5md` / `collect_h5md` writes one
|
|
104
|
+
`[T][N][3]` trajectory (CON stays authority). Engine dest is Å / ps /
|
|
105
|
+
kJ mol^{-1} Angstrom^{-1}; velocity dest is `Angstrom ps-1`.
|
|
106
|
+
Velocities follow D10: v1, v2, and v3.0 frames use the natural unit
|
|
107
|
+
`(amu*angstrom^2/eV)^0.5` whatever the time label says; v3.1 frames use
|
|
108
|
+
`units.time` and fail when a velocities section has no `units.time`.
|
|
109
|
+
`--velocity-time-unit`, Rust `H5mdCollectOpts`, the Python
|
|
110
|
+
`velocity_time_unit` argument, and C `rkrdb_set_velocity_time_unit`
|
|
111
|
+
override that rule for a file that breaks it.
|
|
112
|
+
The simulation clock uses `units.time` when that key is present.
|
|
113
|
+
A v3.1 frame with `time` or `timestep` and no `units.time` is an error.
|
|
114
|
+
The same override supplies that missing clock unit.
|
|
115
|
+
A present `units.time` still scales the clock.
|
|
116
|
+
v1, v2, and v3.0 clocks with no `units.time` are read as `fs`.
|
|
117
|
+
- **Node-local drain/join** — `shard-ingest` then `drain` to a unique
|
|
118
|
+
dest (`data.mdb` only, refuse overwrite), then `join-drained`.
|
|
119
|
+
`compact-join` joins one sharded root (`open_existing`). Campaign
|
|
120
|
+
ops: [`docs/orgmode/howto/campaign.org`](docs/orgmode/howto/campaign.org).
|
|
121
|
+
|
|
122
|
+
Full ABI table, logo, Sphinx docs, and site: see `docs/`, `website/`, `assets/logo/`, `CHANGELOG.md`. Fortran module notes: `fortran/README.md`, `fortran/ReadConDb/`.
|
|
123
|
+
|
|
124
|
+
## Questions and bug reports
|
|
125
|
+
|
|
126
|
+
Ask questions and report bugs on the [GitHub issue tracker](https://github.com/lode-org/readcon-db/issues).
|
|
127
|
+
Report a security problem by email instead, as [`SECURITY.md`](SECURITY.md) describes.
|
|
128
|
+
|
|
129
|
+
## License
|
|
130
|
+
|
|
131
|
+
MIT
|
|
132
|
+
|
|
133
|
+
## Cooked SoA tier
|
|
134
|
+
|
|
135
|
+
Optional RCSO numerics in `frames_soa` (opt-in cook). Ingest and `migrate` do not cook. RCSO is
|
|
136
|
+
non-authoritative: CON text in `frames` is the sole authority for hash,
|
|
137
|
+
dedup, join/split, and reindex. User doc:
|
|
138
|
+
[`docs/orgmode/cooked-soa.org`](docs/orgmode/cooked-soa.org).
|
|
139
|
+
|
|
@@ -0,0 +1,6 @@
|
|
|
1
|
+
readcon_db/__init__.py,sha256=fPwHtXUOuSf9vQPEC63UzPQqX8obc7xjJhzDJdj9H3w,123
|
|
2
|
+
readcon_db/readcon_db.cp315-win_amd64.pyd,sha256=iYfE4efqZzYW0Le9ESJQ0CzqqoAe76dUwu6uDO7ENd8,2827776
|
|
3
|
+
readcon_db-0.2.1.dist-info/METADATA,sha256=5myiD-Gox_aGeEkzCQIz2W4QQ9bA6nvbz2qVKXnYP6Y,7616
|
|
4
|
+
readcon_db-0.2.1.dist-info/WHEEL,sha256=Q5QdGvjBfn8x6ihg-B4VQuuGKk7DT67L9DVrwEH_qj8,97
|
|
5
|
+
readcon_db-0.2.1.dist-info/sboms/readcon-db.cyclonedx.json,sha256=ye_DeaxLbidHU0VbZJblVIg6GiO8_UxV77eQoXBhk3o,118867
|
|
6
|
+
readcon_db-0.2.1.dist-info/RECORD,,
|