testerkit 0.4.0__py3-none-any.whl

testerkit/_docs/concepts/configuration/stations.md

@@ -0,0 +1,214 @@
+# Stations
+
+A **Station** is where you test — a physical bench with instruments. Station configs define what instruments are available and how to connect to them.
+
+## Station Configuration
+
+Station configs are YAML files in `stations/`:
+
+```yaml
+# stations/bench_1.yaml
+id: bench_1
+name: "Production Bench 1"
+location: "Lab A"
+
+instruments:
+  dmm:
+    type: dmm
+    driver: pymeasure.instruments.keysight.Keysight34461A
+    resource: "TCPIP::192.168.1.100::INSTR"
+
+  psu:
+    type: psu
+    driver: pymeasure.instruments.keysight.KeysightE36312A
+    resource: "GPIB0::5::INSTR"
+```
+
+## Instrument Configuration
+
+Each instrument has:
+
+| Field | Description |
+|-------|-------------|
+| `type` | Instrument type (dmm, scope, psu, eload, etc.) |
+| `driver` | Import path to the driver class (PyMeasure / PyVISA / vendor) |
+| `resource` | VISA address or connection string |
+| `mock_config` | Canned return values used in `--mock-instruments` mode |
+| `mock` | Force mock mode for this instrument |
+
+Instruments can be shared across multiple UUT sites in a multi-UUT fixture. TesterKit connects a shared instrument once and lets each UUT's test use it one at a time, so two tests never drive the same instrument at the same moment. Sharing is detected automatically — no extra flags. See [Configuring Stations](../../how-to/configuration/configuring-stations.md#shared-instruments-multi-uut) for details.
+
+### Common Instrument Types
+
+| Type | Description | Typical Capabilities |
+|------|-------------|---------------------|
+| `dmm` | Digital Multimeter | voltage (DC/AC), current, resistance |
+| `scope` | Oscilloscope | voltage (AC), frequency, time |
+| `psu` | DC Power Supply | voltage output, current output |
+| `eload` | Electronic Load | current sink |
+| `funcgen` | Function Generator | waveform output |
+
+## Mock Mode
+
+For development without hardware, use `--mock-instruments`:
+
+```bash
+pytest tests/ --station=stations/bench_1.yaml --mock-instruments --uut-serial=SIM001
+```
+
+Configure mock values in the station:
+
+```yaml
+instruments:
+  dmm:
+    type: dmm
+    driver: pymeasure.instruments.keysight.Keysight34461A
+    resource: "TCPIP::192.168.1.100::INSTR"
+    mock_config:
+      measure_dc_voltage: 3.31
+      measure_current: 0.1
+```
+
+### When to Use Mock Mode
+
+| Approach | Use Case |
+|----------|----------|
+| `--mock-instruments` flag | Development, CI, station without hardware |
+| Real hardware | Production, calibration |
+
+## Station Types and Instances
+
+A station can be split into a reusable template and the actual bench that fills it in:
+
+### Station Types (Templates)
+
+List the instrument roles and drivers a station needs:
+
+```yaml
+# stations/types/voltage_tester.yaml
+id: voltage_tester
+description: "Station for voltage testing"
+instruments:
+  dmm: {type: dmm, driver: pymeasure.instruments.keysight.Keysight34461A}
+  psu: {type: psu, driver: pymeasure.instruments.keysight.KeysightE36312A}
+capabilities:                                # catalog capability ids
+  - keysight.34461a.dc_voltage
+  - keysight.e36312a.dc_source
+```
+
+One file per station type under `stations/types/`. Every role listed under `instruments:` is required (there is no `required:` field). `capabilities:` is a list of catalog capability id strings — not inline `Capability` dicts.
+
+### Station Instances (Real Benches)
+
+Concrete stations that implement a type:
+
+```yaml
+# stations/bench_1.yaml
+id: bench_1
+name: "Bench 1"
+station_type: voltage_tester
+location: "Lab A, Bench 1"
+
+instruments:
+  dmm:
+    type: dmm
+    driver: pymeasure.instruments.keysight.Keysight34461A
+    resource: "TCPIP::192.168.1.100::INSTR"
+  psu:
+    type: psu
+    driver: pymeasure.instruments.keysight.KeysightE36312A
+    resource: "GPIB0::5::INSTR"
+```
+
+### Instrument aliases per test
+
+Station configs define the physical instrument inventory. You can override which station instrument a fixture name points to for a specific test or class — without editing the station config — using a per-test override file. See [Writing Tests](../../how-to/execution/writing-tests.md) for details.
+
+## Using Stations in Tests
+
+### Via pytest
+
+```bash
+pytest tests/ --station=bench_1 --uut-serial=SN001
+```
+
+### Via fixtures
+
+```python
+def test_voltage(dmm, measure):
+    """Each instrument role in the station config is available as a fixture (here: dmm)."""
+    measure("voltage", dmm.measure_voltage())
+```
+
+### Via CLI
+
+```bash
+testerkit serve                    # Start UI
+testerkit runs                     # List recent runs
+testerkit show <run_id>            # Show run details
+```
+
+## Supported Test Phases
+
+Stations can optionally declare which test phases they support, via `supported_phases`. This lets you filter stations when selecting one for a run or profile (see [Profiles](../../how-to/execution/profiles.md)):
+
+```yaml
+id: bench_1
+name: "Production Bench 1"
+
+supported_phases:
+  - validation
+  - production
+  - debug
+
+instruments:
+  # ...
+```
+
+## Multiple Stations
+
+Large test environments may have multiple stations:
+
+```
+stations/
+├── _base.yaml           # Station type definitions
+├── bench_1.yaml         # Production bench 1
+├── bench_2.yaml         # Production bench 2
+├── debug_bench.yaml     # Debug station
+└── ci_station.yaml      # CI (simulated) station
+```
+
+### CI Station Example
+
+```yaml
+# stations/ci_station.yaml
+id: ci_station
+name: "CI Environment"
+description: "For CI/CD with --mock-instruments"
+
+instruments:
+  dmm:
+    type: dmm
+    mock: true
+    catalog_ref: generic_dmm
+    mock_config:
+      measure_dc_voltage: 3.3
+      measure_current: 0.1
+  psu:
+    type: psu
+    mock: true
+    catalog_ref: generic_psu
+    mock_config:
+      measure_voltage: 5.0
+```
+
+Run in CI:
+```bash
+pytest tests/ --station=stations/ci_station.yaml --mock-instruments --uut-serial=CI-TEST
+```
+
+## Next Steps
+
+- [Capabilities](capabilities.md) — Understanding what stations can do
+- [Fixtures](fixtures.md) — Mapping UUT pins to instruments
+- [Configuration Reference](../../reference/configuration.md) — YAML schema details

testerkit/_docs/concepts/data/data-stores.md

@@ -0,0 +1,199 @@
+# The TesterKit data stores
+
+TesterKit uses four complementary data stores, each optimized for a different access pattern. Together they provide a complete picture of test activity.
+
+## Overview
+
+| Store | Data | Format |
+|-------|------|--------|
+| **EventStore** | Typed events (sessions, measurements, diagnostics) — the source of truth | Arrow files (DuckDB-queryable) |
+| **ChannelStore** | Time-series instrument data (waveforms, readings) | Arrow segments |
+| **FileStore** | Captured artifacts — images, video, vendor capture files (`file://`) | files + index |
+| **RunStore** | Flat test results, one row per measurement | Parquet files |
+
+## EventStore — Source of Truth
+
+The event log captures every significant action as a typed event. It is the canonical record of what happened.
+
+- **Write path:** events are buffered and streamed into the DuckDB-backed event log (see [flight-streaming](flight-streaming.md) for the cross-process detail).
+- **Read path:** SQL queries, or direct reads of the Arrow files.
+- **Storage:** `<data_dir>/events/{date}/{session_id}-{pid}.arrow`
+
+Each event carries only its own fields — `SessionStarted` carries the session/station/operator metadata once (and never a `run_id`); `RunStarted` carries the run/UUT context; `MeasurementRecorded` carries just the measurement. The flat run rows fill in the shared context when they're written.
+
+## ChannelStore — Time-Series Data
+
+Instrument reads that produce arrays, waveforms, or high-frequency scalar streams go to the ChannelStore. The event carries a compact reference URI (`channel://scope.ch1.waveform?session=…`) that points at the data, instead of the bulk samples.
+
+- **Write path:** samples are buffered and written to rotating Arrow segments (each closed and readable after every flush).
+- **Read path:** query the channel back, with optional downsampling for plotting.
+- **Storage:** `<data_dir>/channels/{date}/{channel_id}_{session_short}.arrow`
+
+Each channel gets its own IPC file with a schema inferred from the first write. Segment rotation ensures files are always readable (closed after each flush).
+
+## RunStore — Analysis-Ready Results
+
+The run store produces analysis-ready parquet files with one row per measurement, every row carrying its full context. This is what you query with DuckDB, Polars, or Spark for yield analysis and SPC.
+
+- **Write path:** the runs daemon collects events as a run executes and writes the per-run parquet when the run ends.
+- **Read path:** any Parquet reader (DuckDB, Polars, pandas).
+- **Storage:** `<data_dir>/runs/{date}/{timestamp}_{run_id8}_{serial}.parquet`
+
+The parquet files are built from the event stream — if the schema changes, they can be rebuilt from events.
+
+### Live streaming + crash safety
+
+During execution, the materializer holds row state in-process and flushes to a single per-run parquet at run end. The operator UI subscribes to the in-process event stream for live updates — there is no separate JSONL journal on disk. If the process is killed before the parquet is finalized, the close-time fallback writes whatever rows reached the materializer with `run_outcome = aborted`.
+
+## FileStore — Captured Artifacts
+
+Bulk artifacts — scope screenshots, camera frames, vendor capture files (`.tdms`, NPZ), any blob — go to the FileStore. Like the ChannelStore, the event carries a compact `file://` reference instead of the bytes.
+
+- **Write path:** `observe(name, value)` with an image / `bytes` / `Path`, or `files.write` / `files.stream` directly.
+- **Read path:** fetch the artifact by its `file://` reference.
+- **Storage:** `<data_dir>/files/{date}/{session_id}/{filename}`
+
+See [the three verbs](three-verbs.md) for how a test routes a value to the FileStore.
+
+## How They Relate
+
+```
+Events (source of truth)
+  ├── RunStore — per-run parquet, written when the run ends
+  ├── ChannelStore — time-series; events carry a channel:// reference
+  └── FileStore — artifacts; events carry a file:// reference
+```
+
+1. All activity flows through the EventStore — the source of truth.
+2. The runs daemon collects events and writes a per-run parquet when the run ends.
+3. Array/waveform values write to the ChannelStore and artifacts to the FileStore; the event keeps a `channel://` or `file://` reference.
+4. Queries join across stores using `session_id`.
+
+## Storage Layout
+
+```
+<data_dir>/
+├── events/                    # EventStore
+│   └── 2026-03-10/
+│       └── {session_id}-{pid}.arrow
+├── channels/                  # ChannelStore
+│   ├── 2026-03-10/
+│   │   ├── dmm.voltage_{session_short}.arrow
+│   │   └── scope.ch1.waveform_{session_short}.arrow
+│   └── _index.duckdb          # channel descriptors + query index (single file, disposable)
+├── files/                     # FileStore
+│   ├── 2026-03-10/
+│   │   └── {session_id}/
+│   │       └── setup_photo.png
+│   └── _index.duckdb          # file catalog + query index (single file, disposable)
+└── runs/                      # RunStore (parquet)
+    ├── 2026-03-10/
+    │   └── 20260310T143022Z_a1b2c3d4_SN001.parquet
+    └── _index.<fingerprint>.duckdb   # content-addressed query index (see below)
+```
+
+Sessions are not a stored entity — they're derived from events at query time.
+
+## Where the data dir lives
+
+`<data_dir>` defaults to a shared per-user directory so every project on the machine sees the same results pool — `testerkit runs`, `testerkit serve`, and DuckDB queries see everything.
+
+Resolution order (first match wins):
+
+1. Explicit `--data-dir` argument or `data_dir=` parameter
+2. `data_dir` field in the project's `testerkit.yaml`
+3. `TESTERKIT_HOME` environment variable
+4. `~/.local/share/testerkit/data/` (platform default via `platformdirs`)
+
+To isolate a project's results from the shared pool, add to `testerkit.yaml`:
+
+```yaml
+name: my-project
+data_dir: data       # writes to ./data/ instead of the global pool
+```
+
+## Schema evolution — HARD contract
+
+Parquet files are the permanent record. Each testerkit version may add columns; older files simply lack them. The parquet artifact is a **HARD contract**: changes must be additive because written files cannot be retroactively rewritten when a new version ships.
+
+Until the 1.0 cut, the following invariants hold:
+
+- **New columns only.** Every release may add columns. Existing column names, types, and semantics are stable across 0.x releases.
+- **No removals or type changes** in 0.x. If a column would otherwise be removed or repurposed, it stays in the schema and reads as NULL for newly-written rows; the old meaning is documented as deprecated.
+- **PK stability.** `(run_id, step_path, vector_index)` is the per-step identity in the materialized table; `(run_id, step_path, vector_index, measurement_name, vector_retry)` discriminates measurement rows. These tuples do not change shape in 0.x.
+- **`record_type` discriminator stable.** The at-rest `'run'` / `'step'` / `'vector'` values are part of the wire format and do not change (the daemon also projects a `'measurement'` type at query time).
+- **Read with `union_by_name=true`.** Consumer queries that follow the recommended `read_parquet(..., union_by_name=true)` pattern survive every additive evolution automatically.
+
+```sql
+-- DuckDB handles mixed schemas automatically
+SELECT station_id, project_name, run_outcome
+FROM read_parquet('~/.local/share/testerkit/data/runs/**/*.parquet',
+                  union_by_name=true)
+```
+
+Schema rewrites and column removals are deferred to the 1.0 cut, when a migration story for old files lands.
+
+## The DuckDB query index
+
+Every store keeps a DuckDB index alongside its data files, to speed up queries like `testerkit runs` and the web UI. The index is always a **disposable cache** — delete it and it rebuilds from the parquet or Arrow files underneath. Deleting an index file never loses data.
+
+Two index shapes are in use, depending on the store:
+
+**RunStore — content-addressed.** The runs index file name embeds a short fingerprint of the query it serves — the shape of the projection plus the TesterKit version that built it — as `_index.<fingerprint>.duckdb`. A query or schema change never rewrites an existing file in place: it produces a different fingerprint, and the next build writes a *new* file alongside any older ones. Multiple fingerprints can sit in the same `<data_dir>/runs/` directory at once — an older TesterKit version (or another project pinned to an older release) reading the same results keeps opening its own file, untouched by a newer index being built next to it.
+
+**ChannelStore and FileStore — a single `_index.duckdb`.** These two stores have not adopted the content-addressed model yet; each keeps one index file that's reused (or, on a schema change, deleted and rebuilt) in place. Same disposable-cache guarantee, no coexisting versions.
+
+### Managing runs-index files
+
+`testerkit data index` manages the RunStore's index files (called *epochs* — one per fingerprint):
+
+```cli
+$ testerkit data index list
+  FINGERPRINT   SCHEMA  BUILT BY  SEEN BY       RUNS   SIZE    LAST SEEN
+* a1b2c3d4e5f6  3       0.3.0     0.3.0         1,204  18.2 MB just now
+  f0e1d2c3b4a5  2       0.2.4     0.2.4, 0.3.0  1,204  17.9 MB 6 days ago
+
+$ testerkit data index build            # warm the current epoch
+$ testerkit data index rm f0e1d2c3b4a5  # drop one epoch by its fingerprint prefix
+$ testerkit data index prune            # reap epochs no recent version has touched
+```
+
+- `list` — every epoch present, with its fingerprint, schema version, the TesterKit version that built it, every version that has since read it, row count, size, and last access.
+- `build` — warms the epoch matching the currently installed TesterKit version by ingesting from parquet. `--rebuild` discards it first for a clean rebuild.
+- `rm <fingerprint-prefix>` — deletes one epoch. Refuses to remove the epoch actively serving queries unless `--force`.
+- `prune` — removes epochs no recent TesterKit version has touched, always keeping the current epoch and (by default) the three most-recently-seen others.
+
+None of these touch the parquet files. Every command is safe to run at any time — the worst case is that the next query pays the cost of a rebuild.
+
+To force a full rebuild of the channels or files index, delete `_index.duckdb` under that store's directory — it rebuilds from the Arrow segments or file catalog on next access.
+
+## Mixed versions on one machine
+
+When multiple projects use different TesterKit versions but share the global results directory:
+
+| Layer | What happens | User impact |
+|---|---|---|
+| Parquet files | Each version writes its own schema. Newer files may have more columns. | NULL values for columns that didn't exist when the file was written. |
+| Runs query index | Content-addressed: each TesterKit version reads and writes its own epoch file, keyed by its own fingerprint. Old and new epochs coexist in `<data_dir>/runs/` without contention. | `testerkit data index list` shows every epoch present; nothing to migrate by hand. Run `testerkit data index prune` to reclaim disk from epochs no version uses anymore. |
+| Channels / files query index | A single shared index file per store. | Rare cross-version friction can still surface here; delete `_index.duckdb` in that store's directory to force a rebuild. |
+| Web UI / CLI | Shows whatever the current process's index has. | Some fields may be empty for older runs. |
+
+The rule: newer is always a superset. An older TesterKit version reading newer results ignores unknown columns; a newer version reading older results sees NULL for missing columns. No version corrupts or downgrades another's data.
+
+## See also
+
+**Same topic, other quadrants:**
+
+- [Reference → Parquet schema](../../reference/data/parquet-schema.md) — column-level reference for the materialized run rows
+- [Reference → Query API](../../reference/data/query-api.md) — `RunsQuery` / `StepsQuery` / `MeasurementsQuery` — the read path over the run store
+- [Reference → `testerkit data index`](../../reference/cli.md#cli-data-index) — full flag reference for `list` / `build` / `rm` / `prune`
+- [How-to → Querying events](../../how-to/data/querying-events.md), [Querying channels](../../how-to/data/querying-channels.md) — task recipes against each store
+- [How-to → Export results](../../how-to/data/export-results.md) — pulling rows out of the parquet store
+- [Integration → Lakehouse import](../../integration/data/lakehouse-import.md) — pulling TesterKit parquet into your warehouse
+
+**Sibling concepts:**
+
+- [Event log](event-log.md) — the source-of-truth event stream
+- [Event sourcing](event-sourcing.md) — why the platform is event-sourced
+- [Sessions](sessions.md) — the observation window that keys events
+- [Flight streaming](flight-streaming.md) — the cross-process query model

testerkit/_docs/concepts/data/event-log.md

@@ -0,0 +1,185 @@
+# The Event Log
+
+The event log is TesterKit's unified record of everything that happens during testing — sessions, instrument connections, measurements, diagnostics, and more.
+
+## One stream, in order
+
+Every significant action emits a typed event into a single ordered stream. That stream lets you reconstruct what happened during a test, line up instrument reads with measurements, and watch a test live as it runs.
+
+## Event Hierarchy
+
+Every event carries the same common fields:
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `id` | UUID | Unique event identifier |
+| `occurred_at` | datetime | When the event happened |
+| `received_at` | datetime | When the log processed it for storage |
+| `session_id` | UUID | Which session this event belongs to |
+| `run_id` | UUID | Which test run (if applicable) |
+
+Each event also carries an `event_type` string (e.g. `test.measurement`) naming which kind of event it is.
+
+## Event Categories
+
+TesterKit defines events across 12 categories.
+
+### Session (2 events)
+| Event | Type String | Description |
+|-------|-------------|-------------|
+| `SessionStarted` | `session.started` | Session-wide metadata: station, operator, fixture |
+| `SessionEnded` | `session.ended` | Session outcome |
+
+### Run (3 events)
+| Event | Type String | Description |
+|-------|-------------|-------------|
+| `RunStarted` | `run.started` | Full run context: UUT, part, operator, config snapshots |
+| `RunEnded` | `run.ended` | Run outcome |
+| `RunMaterialized` | `run.materialized` | Emitted after the run's Parquet file is durably written; signals that the run is ready for downstream consumers |
+
+### Site (2 events — multi-UUT)
+| Event | Type String | Description |
+|-------|-------------|-------------|
+| `SiteStarted` | `site.started` | A multi-UUT site subprocess begins |
+| `SiteCompleted` | `site.completed` | A multi-UUT site subprocess finishes |
+
+### Sync (2 events — multi-UUT)
+| Event | Type String | Description |
+|-------|-------------|-------------|
+| `SyncArrived` | `sync.arrived` | A worker reached a synchronization barrier |
+| `SyncRelease` | `sync.release` | All workers arrived; barrier released |
+
+### Route (2 events — signal switching)
+| Event | Type String | Description |
+|-------|-------------|-------------|
+| `RouteClosed` | `route.closed` | Switch route closed (signal connected) |
+| `RouteOpened` | `route.opened` | Switch route opened (signal disconnected) |
+
+### Fixture (5 events)
+| Event | Type String | Description |
+|-------|-------------|-------------|
+| `InstrumentConnected` | `fixture.instrument_connected` | Instrument identified and connected |
+| `IdentityVerified` | `fixture.identity_verified` | Expected vs actual instrument identity |
+| `CalibrationWarning` | `fixture.calibration_warning` | Calibration due date approaching |
+| `UutScanned` | `fixture.uut_scanned` | UUT serial barcode scanned |
+| `InstrumentDisconnected` | `fixture.instrument_disconnected` | Instrument released during teardown |
+
+### Test (7 events)
+| Event | Type String | Description |
+|-------|-------------|-------------|
+| `StepsDiscovered` | `test.steps_discovered` | Full list of collected test items |
+| `StepStarted` | `test.step_started` | A test step begins execution |
+| `MeasurementRecorded` | `test.measurement` | A single measurement with limits and outcome |
+| `StepEnded` | `test.step_ended` | A test step finishes |
+| `Observation` | `test.observation` | An environmental or contextual reading recorded during a step |
+| `VectorStarted` | `test.vector_started` | A parametric sweep vector begins |
+| `VectorEnded` | `test.vector_ended` | A parametric sweep vector finishes |
+
+### Instrument (2 events)
+| Event | Type String | Description |
+|-------|-------------|-------------|
+| `InstrumentSet` | `instrument.set` | Driver set via proxy |
+| `InstrumentConfigure` | `instrument.configure` | Driver configure via proxy |
+
+### Diagnostic (2 events)
+| Event | Type String | Description |
+|-------|-------------|-------------|
+| `DiagnosticWarning` | `diagnostic.warning` | Non-fatal warning |
+| `DiagnosticError` | `diagnostic.error` | Error condition |
+
+### Channel (3 events)
+| Event | Type String | Description |
+|-------|-------------|-------------|
+| `ChannelStarted` | `channel.started` | A channel received its first sample in this session |
+| `ChannelEnded` | `channel.ended` | A channel was sealed for this session |
+| `ChannelCheckpoint` | `channel.checkpoint` | Liveness + progress marker from an active channel producer |
+
+### File (3 events)
+| Event | Type String | Description |
+|-------|-------------|-------------|
+| `FileStarted` | `file.started` | A data stream begins |
+| `FileEnded` | `file.ended` | A data stream ends |
+| `FileCheckpoint` | `file.checkpoint` | Liveness + progress marker from an active file sink |
+
+### Dialog (2 events)
+| Event | Type String | Description |
+|-------|-------------|-------------|
+| `DialogOpened` | `dialog.opened` | Operator dialog shown, execution paused |
+| `DialogResponded` | `dialog.responded` | Operator responded to dialog |
+
+## Event Timeline
+
+A typical test session emits events in this order:
+
+```
+SessionStarted          # Session-wide metadata (station, operator)
+├── RunStarted          # Run context (UUT, part, config snapshots)
+├── InstrumentConnected # One per instrument role
+├── IdentityVerified    # Optional identity check
+├── StepsDiscovered     # Full list of collected test items
+├── StepStarted         # First test step
+│   ├── InstrumentSet
+│   └── MeasurementRecorded
+├── StepEnded
+├── StepStarted         # Next test step...
+│   └── ...
+├── StepEnded
+├── RunEnded            # All steps complete
+├── RunMaterialized     # Parquet file durably written
+├── InstrumentDisconnected
+└── SessionEnded        # Cleanup complete
+```
+
+## How the log is written
+
+When an event is written:
+
+1. **`received_at` is stamped** and the event is buffered for batched writes to an Arrow file (a fast columnar on-disk format)
+2. **On `RunEnded`** the run's Parquet file is written. The same path runs when you replay stored events with `testerkit export`.
+3. **Flush** happens every 50 events (configurable), writing a batch to the Arrow file
+
+The set of writers (Parquet, the live UI feed) is built in; adding a new output format is a change to TesterKit itself, not a drop-in plugin.
+
+## Storage
+
+Events are stored as Arrow files, date-partitioned:
+
+```
+<data_dir>/events/
+├── 2026-03-10/
+│   ├── {session_id}-{pid}.arrow
+│   └── {session_id}-{pid}_0001.arrow   # rotation for large sessions
+└── 2026-03-11/
+    └── ...
+```
+
+Each Arrow file contains index columns (`id`, `event_type`, `occurred_at`, `received_at`, `session_id`, `run_id`) plus a `json` column with the full serialized event for lossless replay.
+
+## Dual-Write Pattern
+
+Events are also loaded into an in-memory database as they're written, so you can query them with SQL right away:
+
+1. **Arrow file** — crash-safe append-only storage
+2. **In-memory DuckDB** — immediate SQL queryability
+
+Each batch is loaded into an in-memory SQL database as it's written, so a query sees an event the instant after it's written.
+
+## What stays stable across releases
+
+Once an event type and its fields exist, they don't change or disappear within 0.x. Only new event types and new optional fields are added. A query or report you write today keeps working as the platform evolves.
+
+## See also
+
+**Same topic, other quadrants:**
+
+- [Reference → Event types](../../reference/data/event-types.md) — generated field reference for every event class
+- [Reference → Parquet schema](../../reference/data/parquet-schema.md) — the materialized projection of these events
+- [How-to → Querying events](../../how-to/data/querying-events.md) — MCP / HTTP / Python recipes for reading the event log
+- [Operator UI → Events](../../reference/operator-ui/events.md) — the browser view of the event log
+- [Tutorial → Step 10: Live monitoring](../../tutorial/10-live-monitoring.md) — first hands-on with events as they happen
+
+**Sibling concepts:**
+
+- [Event sourcing](event-sourcing.md) — why the platform is event-sourced rather than mutation-based
+- [Data stores](data-stores.md) — how EventStore fits with ChannelStore, FileStore, and RunStore
+- [Sessions](sessions.md) — the observation window the event log keys events by