engineering-behavior-observatory 0.2.1

package/docs/harnesses/openhands-agent-server.md

@@ -0,0 +1,153 @@
+# OpenHands Agent Server adapter
+
+The OpenHands adapter executes one EBO attempt through a pinned Agent Server
+REST/WebSocket boundary. It preserves the source records before projecting any
+uniform events. It does not import the OpenHands Python SDK or embed the agent
+loop.
+
+## Pinned boundary
+
+The runtime pin is Agent Server `1.46.0`. The matching release OpenAPI digest,
+server commit, image, event discriminator list, WebSocket route, and
+authentication mode are recorded in the
+[contract manifest](../../contracts/openhands-agent-server-v1.46.0.json).
+
+The 1.46.0 upgrade preserves the existing conversation endpoints and consumed
+event schemas. The upstream schema changes concern ACP agents and skill requests,
+which this adapter does not use. Historical 1.44.1 fixtures and the contract
+manifest remain unchanged; retained evaluation keeps their original runtime and
+adapter identity.
+
+For a native macOS ARM64 installation, download the release's
+`agent-server-1.46.0-macos-arm64` and verify it against `SHA256SUMS` (also pinned
+in the contract manifest). Mark the binary executable and launch it with:
+
+```sh
+agent-server-1.46.0-macos-arm64 --host 127.0.0.1 --port 18080
+curl --fail http://127.0.0.1:18080/server_info
+```
+
+The identity response must report `1.46.0`. Server readiness does not establish
+a working model route; the opt-in conversation smoke below verifies that separately.
+
+`@openhands/typescript-client` `1.39.0` was checked against that contract. Its
+generated types target Agent Server `1.44.0`, and its WebSocket wrapper neither
+exposes reconnect lifecycle evidence nor uses the current first-message
+authentication path. EBO therefore uses Node's native `fetch` and `WebSocket`
+APIs for this boundary and does not add the client as a runtime dependency.
+
+## Capture flow
+
+`captureOpenHandsAgentServerRun` coordinates one caller-supplied run definition:
+
+The run configuration must expose the actual model at `agent.llm.model` or
+`agent_settings.llm.model`; EBO rejects a request whose executable model differs
+from the run manifest identity.
+
+1. Verify `/server_info` reports exactly `1.46.0`.
+2. Create one conversation with the supplied agent configuration and the
+   workspace path visible to the server. Authenticated REST requests use the
+   pinned server's `X-Session-API-Key` header.
+3. Open `/sockets/events/{conversation_id}` with `resend_mode=all`, authenticate
+   in the first frame when required, and submit the message through REST.
+4. Poll the conversation until `finished`, `error`, or `stuck`.
+5. Read every final event page in timestamp order, reconcile by native event ID,
+   then clean up the conversation.
+6. Package native JSONL, exposed hook events, the workspace outcome, and the EBO
+   verifier result before finalizing the run bundle. If workspace packaging
+   fails, preserve and report the disposable workspace path for recovery.
+
+Container callers can set `serverWorkspacePath` when the Agent Server sees a
+different mount path from the local EBO coordinator.
+
+## Reconciliation and completeness
+
+Unexpected socket closure reconnects with `resend_mode=since` from the last
+native event timestamp. Inclusive replay can deliver duplicates. EBO retains
+every receipt as native evidence, selects the final REST copy as the canonical
+normalization source, and emits one uniform event per native event ID.
+
+The result reports streamed-only and final-only IDs. A failed final REST read
+keeps the streamed records and marks reconciliation partial. Unknown `kind`
+values remain in native JSONL and are listed as unmapped; they never crash the
+capture.
+
+REST responses and WebSocket frames are bounded before parsing, event capture
+has a fixed upper limit, and repeated pagination cursors fail into partial
+evidence. A coordinator abort closes the socket, stops polling, attempts final
+REST recovery and cleanup with a short independent bound, and retains the
+records already received.
+
+Matching REST and WebSocket IDs proves agreement between those two public API
+views only. The boundary does not expose enough evidence to prove delivery of
+the complete in-process `EventLog`, so every otherwise usable capture records
+`EVENT_LOG_COMPLETENESS_UNPROVEN` as a qualification gap.
+
+## Uniform projection
+
+The adapter maps only fields present in the pinned event contract:
+
+| Native record | Uniform family | Preserved distinctions |
+|---|---|---|
+| `MessageEvent`, `SystemPromptEvent` | `message` | actor comes from native `source`, not LLM role |
+| `ActionEvent` | `tool` / before | tool and call identities |
+| `ObservationEvent` | `tool` / after | `action_id` causal relation |
+| `AgentErrorEvent` | `tool` / after | `agent-tool` error scope |
+| `ConversationErrorEvent` | `runtime` | conversation error scope |
+| `ServerErrorEvent` | `runtime` | server error scope |
+| condensation records | `context` | exposed forgotten-event IDs stay native attributes |
+| `HookExecutionEvent` | `runtime` | hook type, result, and exposed action/message association |
+| state, pause, and interrupt records | `runtime` | source status facts |
+| final conversation record | `outcome` | terminal status and workspace identity |
+
+Workspace and verifier evidence are packaged through the existing EBO outcome
+contracts. Agent Server product telemetry is not treated as complete native
+OpenTelemetry evidence.
+
+The native run bundle is finalized and capture-qualified before uniform-event
+projection. A projection error is returned separately and does not replace the
+recorded terminal state or make the retained native/workspace evidence
+inaccessible. Unqualified capture is not normalized.
+
+## Validation
+
+The normal test suite uses pinned streamed/final fixtures for reconnect,
+deduplication, errors, condensation, hooks, unknown variants, and partial REST
+failure. The live smoke is opt-in because it requires a running pinned server
+and a permitted model route:
+
+```sh
+export EBO_LIVE_OPENHANDS_WORKSPACE_ROOT=/path-visible-to-host-and-server
+# Set SESSION_API_KEY and EBO_OPENHANDS_SESSION_API_KEY to the same local
+# smoke-only value in the invoking environment.
+mkdir -p "$EBO_LIVE_OPENHANDS_WORKSPACE_ROOT"
+docker run --rm --name ebo-openhands-smoke -p 127.0.0.1:8010:8000 \
+  -v "$EBO_LIVE_OPENHANDS_WORKSPACE_ROOT:$EBO_LIVE_OPENHANDS_WORKSPACE_ROOT" \
+  -e SESSION_API_KEY \
+  ghcr.io/openhands/agent-server:1.46.0-python --host 0.0.0.0
+
+npm run build
+EBO_LIVE_OPENHANDS_SMOKE=1 \
+EBO_OPENHANDS_SERVER_URL=http://127.0.0.1:8010 \
+node --test --test-name-pattern='approved live Agent Server smoke' \
+  dist/test/openhands.test.js
+```
+
+The smoke asks the agent to modify one file, verifies the retained workspace,
+and requires native stream/final records, hook evidence, a verifier result, a
+run bundle, normalized events, and the explicit EventLog completeness gap.
+`LLM_MODEL` and `LLM_API_KEY` must name an approved route before the test runs.
+
+For an OpenAI-compatible endpoint, use its LiteLLM `openai/` model prefix and
+set `LLM_BASE_URL`. For example, the Z.ai coding-plan smoke used
+`LLM_MODEL=openai/glm-5.3-flash` and
+`LLM_BASE_URL=https://api.z.ai/api/coding/paas/v4`, with `LLM_API_KEY` supplied
+from the operator's `ZAI_API_KEY` environment variable.
+
+Set `EBO_LIVE_OPENHANDS_KEEP_ARTIFACTS=1` to retain the smoke directory and print
+its location, including on failure. These native artifacts remain restricted.
+On the tested macOS installation, the default tmux pool failed during tool
+initialization. `EBO_LIVE_OPENHANDS_TERMINAL_TYPE=subprocess` selects OpenHands'
+supported subprocess terminal for this smoke. With that setting, the 1.46.0
+server completed the Z.ai run, verified the workspace, and produced normalized
+events with the declared EventLog completeness gap.

package/docs/harnesses/pi-sdk.md

@@ -0,0 +1,66 @@
+# Pi TypeScript SDK harness
+
+EBO integrates `@earendil-works/pi-coding-agent` `0.85.1` directly through its public TypeScript SDK. `ebo pi run` executes one caller-selected frozen queue entry; it does not add RPC, ACP, a Pi fork, or another scheduler.
+
+## Operator command
+
+```sh
+ebo pi run \
+  <bundle-root> <queue.json> <run-id> <output-root> \
+  [--workspace-root <path>]
+```
+
+The queue uses the existing model, harness, native-limits, native-tool-policy, and capture-profile references. Each referenced record declares `schemaVersion: "ebo.pi-config/v1"`. The checked-in synthetic shapes are under `test/fixtures/pi/configs/`; their provider URL is intentionally non-routable.
+
+The model record owns the queue model-condition ID, effective provider/model, API dialect, base URL, credential environment-variable name, thinking level, optional provider-specific thinking-level map, advertised input types, context/output limits, and the pricing inputs Pi requires. Pi's opt-in `xhigh` and `max` levels require an explicit non-null mapping in this digest-bound record so the SDK cannot silently clamp them. Credentials remain process environment only; the runtime uses an empty attempt-local credential store so the user's Pi home cannot override the selected route. The retained composition also records Pi's effective non-secret prompt-cache retention (`long` or `short`) so ambient `PI_CACHE_RETENTION` differences remain explicit. The harness record pins Pi and may explicitly reference digest-bound extensions, skills, prompt templates, context files, and a system prompt. Admitted extension modules must be self-contained; import, dynamic-import, re-export, and CommonJS dependency syntax is rejected so the entrypoint digest covers the complete executable extension. Candidate workspace resources and the user's Pi home are not auto-discovered. The capture profile can set `passiveObserver: false` for a controlled observer-disabled condition; it defaults to enabled.
+
+The tool-policy record selects Pi's public built-in tools and a child-process environment allowlist. The provider credential variable is rejected if it appears in that allowlist. This is configuration containment, not an OS sandbox: Pi still runs in the local trusted environment and its file tools can access paths allowed by the host account.
+
+## Native evidence
+
+One attempt owns one native Pi session. EBO retains three separately addressable restricted artifacts:
+
+| Artifact | Authority | Contents |
+|---|---|---|
+| `pi-session.jsonl` | authoritative history | Pi session header and append-only entries with original IDs, parent links, messages, usage, compaction, model and thinking changes |
+| `pi-events.jsonl` | transient lifecycle | subscription events such as agent/turn/message/tool/retry/compaction lifecycle plus adapter cleanup/error records |
+| `pi-observer.jsonl` | passive metadata | ordered public extension hooks for context, provider request/response stages, tool calls/results, compaction and session lifecycle |
+
+Callback values are snapshotted at receipt and writes are serialized and drained before finalization. Provider payload content is off by default; when explicitly enabled it stays restricted. Header values are never recorded. The observer returns no context, prompt, provider-payload, tool-call, or tool-result modifications.
+
+Pi history owns final messages and usage. Streamed message updates retain only their incremental event fields; cumulative `message` and `partial` snapshots are omitted to avoid quadratic evidence growth, while final content remains authoritative in native history. Streamed message deltas and passive tool hooks remain reachable but are explicitly unmapped where projecting them would double-count history or tool operations. Adapter receipt time is labeled in attributes and is never promoted to a native timestamp. Errors from selected extension hooks are retained as observer diagnostics and prevent the capture from qualifying.
+
+On timeout or cancellation EBO calls the public `session.abort()` API, waits for idle, copies the original persisted native session tree, emits shutdown, disposes the session, drains recorders, and then packages the workspace. The SDK branch-export helper is only a fallback when native persistence is unavailable. Creation, provider, export, recorder, and cleanup failures retain a partial attempt; recorder loss cannot qualify as success. The single configured retry bound applies to Pi's agent-level loop while provider-library retries remain disabled, preventing multiplicative requests.
+
+## Capability and gap matrix
+
+| Surface | Status | Evidence/gap |
+|---|---|---|
+| messages and native session tree | available | persisted Pi JSONL plus subscription start/end records |
+| model requests | available | passive `before_provider_request`; payload content opt-in restricted |
+| tool operations | available | subscription start/update/end keyed by native tool-call ID |
+| context and compaction | available | context/compaction hooks plus native history |
+| retries | available | public session retry lifecycle events |
+| provider usage | available | native assistant-message usage; treated as per-message increments |
+| permissions | unsupported | no distinct selected public-SDK permission-decision record |
+| delegation/child histories | unsupported | one owned Pi session per attempt; no branching/resume UI |
+| native OTLP receipt | unsupported | no verified native Pi `0.85.1` OTLP receipt surface |
+| isolation | limited | local SDK execution is not an OS sandbox |
+
+## Validation
+
+The deterministic fixture exercises the real queue, workspace, bundle, qualification, export/readback, observation, comparison, judge-package, calibration, aggregation, and Atlas paths with an SDK-shaped session seam:
+
+```sh
+npm run build
+node --test dist/test/pi.test.js
+```
+
+The approved live smoke performs one synthetic file edit through Z.ai's OpenAI-compatible Chat Completions route. It is opt-in and never prints the key:
+
+```sh
+EBO_LIVE_PI_SDK_SMOKE=1 node --test \
+  --test-name-pattern='approved live Pi SDK smoke' dist/test/pi.test.js
+```
+
+Set `ZAI_API_KEY` in the environment. The smoke selects `glm-5.3-flash` and `https://api.z.ai/api/coding/paas/v4` only in its generated test input; neither is a product default.

package/docs/reference/README.md

@@ -0,0 +1,19 @@
+# Reference
+
+Use guides to learn a workflow; use these pages when implementing or checking
+an exact contract.
+
+| Reference | Answers |
+| :--- | :--- |
+| [CLI](cli.md) | Commands grouped by preparation, capture, evidence, evaluation, and reports |
+| [Task packets and experiments](contracts.md) | Archive rules, admission, freeze, configuration digests, run matrices |
+| [Run bundles](run-bundle-contract.md) | Evidence descriptors, terminal states, qualification, workspace and export contracts |
+| [Run lifecycle](run-lifecycle.md) | Attempt ownership, interruption, retries, process boundary |
+| [Agent SDK configuration](agent-sdk-operational-runner.md) | Five source-specific queue records and runner behavior |
+| [Uniform events](../evaluation/uniform-events.md) | Families, native references, coverage, explicit unknown/unsupported values |
+| [Extension contracts](../development/extension-contracts.md) | How new adapters and evaluators fit existing interfaces |
+
+Machine-readable contracts are shipped under `schemas/`, `contracts/`, and
+`ontology/`. Paths in versioned records are interpreted by their contract,
+not automatically relative to your current shell directory. CLI filesystem
+arguments are operator paths; packet locators remain bundle-relative.

package/docs/reference/agent-sdk-operational-runner.md

@@ -0,0 +1,118 @@
+# Agent SDK queue configuration
+
+This reference describes the implemented `ebo agent-sdk run` input contract.
+For authentication and native evidence, start with
+[the harness guide](../harnesses/claude-agent-sdk.md). For packet preparation,
+see [the operator workflow](../guides/operator-guide.md).
+
+## Command
+
+```sh
+ebo agent-sdk run <bundle-root> <queue.json> <run-id> <output-root> [--workspace-root <path>]
+```
+
+The runner validates a persisted queue entry, rechecks its frozen packet and
+configuration digests, materializes a disposable workspace, and calls the
+capture API once. It retains a starting baseline until outcome capture finishes,
+then reopens the manifest before returning a summary.
+
+The summary contains `runId`, `attemptId`, `bundlePath`,
+`assessmentMode`, `terminal`, `classification`, and
+`captureQualification`, plus native session/trace IDs when available.
+A captured failure or stop is a valid observation, not necessarily a nonzero
+CLI exit. Preflight or final validation failures return nonzero.
+
+## Five configuration records
+
+Each JSON block below is a **separate file**, referenced and SHA-256 pinned by
+the experiment/queue. Bounds and model values are examples, not research
+defaults. Authentication stays in the process environment.
+
+```json
+{ "schemaVersion": "ebo.agent-sdk-config/v1", "kind": "model", "model": "sonnet" }
+```
+
+```json
+{ "schemaVersion": "ebo.agent-sdk-config/v1", "kind": "harness", "adapter": "claude-agent-sdk" }
+```
+
+```json
+{
+  "schemaVersion": "ebo.agent-sdk-config/v1",
+  "kind": "native-limits",
+  "maxTurns": 8,
+  "maxBudgetUsd": 1
+}
+```
+
+```json
+{
+  "schemaVersion": "ebo.agent-sdk-config/v1",
+  "kind": "native-tool-policy",
+  "tools": ["Read", "Edit", "Bash"],
+  "allowedTools": ["Read", "Edit", "Bash"],
+  "disallowedTools": [],
+  "permissionMode": "dontAsk"
+}
+```
+
+```json
+{
+  "schemaVersion": "ebo.agent-sdk-config/v1",
+  "kind": "capture-profile",
+  "telemetry": {
+    "endpoint": "http://127.0.0.1:4318",
+    "protocol": "http/json",
+    "exportIntervalMs": 1000,
+    "logUserPrompts": false,
+    "logToolDetails": false,
+    "logToolContent": false,
+    "logRawApiBodies": false
+  },
+  "workspaceOutcome": {
+    "excludeDirectoryNames": ["node_modules", "coverage"],
+    "respectGitignore": true,
+    "omitEmptyDirectories": true
+  }
+}
+```
+
+Rules:
+
+- The queue entry's model, harness, native-limits, native-tool-policy, and queue-level capture-profile references must resolve to the corresponding `kind`.
+- Duplicate JSON keys, unknown fields, unsafe numbers, wrong kinds, digest mismatches, malformed UTF-8, and malformed JSON are rejected before SDK launch.
+- The model record's `model` must equal the queue entry's model ID, matching the existing capture invariant.
+- The harness adapter must be `claude-agent-sdk`.
+- `maxTurns` and `maxBudgetUsd` are optional positive limits. The queue's coordinator `maxWallClockMs` remains the outer wall-clock limit.
+- Tool lists are arrays of unique nonempty strings. Preserve Agent SDK names; do not invent a common tool ontology.
+- `bypassPermissions` is accepted only when `allowDangerouslySkipPermissions` is explicitly true, matching the existing executor safeguard.
+- Telemetry content flags default to false. Credentials and environment overrides are never read from these JSON files; authentication stays in the process environment.
+- A configured telemetry endpoint without a receipt checker remains explicit `not-checked` or missing receipt evidence. Do not claim collector receipt from a successful HTTP configuration alone.
+
+## Retention and recovery
+
+The output bundle is written under `<output-root>/<run-id>/<attempt-id>`.
+Each invocation has a fresh attempt identity. The command does not iterate,
+resume, or silently retry; ordinary CLI reruns do not claim linked retry lineage.
+
+Only the admitted `agentInput` prompt and materialized fixture enter execution.
+Observational packets have no verifier. Verified packets run their admitted
+verifier on the retained workspace snapshot and keep its result separate from
+capture qualification.
+
+The capture layer retains complete or partial native evidence after execution
+starts. If workspace packaging fails, the summary may expose
+`retainedWorkspacePath` for local recovery. Do not delete that directory until
+the outcome has been recovered. Cleanup failure after successful packaging
+also leaves the path visible; it does not erase the retained bundle.
+
+## Library use
+
+`runAgentSdkQueueEntry` accepts the same paths and selected run ID, plus
+an optional abort signal and in-process test/receipt callbacks.
+`captureClaudeAgentSdkRun` accepts an already-resolved attempt and explicit
+workspace composition. Neither is a general task generator or study scheduler.
+
+See [lifecycle](run-lifecycle.md) and
+[run-bundle contracts](run-bundle-contract.md) for ownership, interruption,
+qualification, and artifact invariants.

package/docs/reference/cli.md

@@ -0,0 +1,114 @@
+# CLI reference
+
+Install `ebo` with [the quickstart](../guides/quickstart.md). `ebo --help`
+prints the current top-level syntax. For a source checkout without `npm link`,
+use `npm run ebo -- <arguments>`.
+
+For one-offs without a global install, use
+`npm exec --package=engineering-behavior-observatory -- ebo --help`.
+Avoid `npx ebo`, which targets a different package.
+
+`<angle-bracket>` values are required placeholders; `[square-bracket]` items
+are optional. Do not paste these templates unchanged. `packet.json` and freeze
+locators are relative to the declared bundle root; other filesystem arguments
+are operator paths. Record destinations must be outside immutable evidence.
+
+## Validate artifacts
+
+```sh
+ebo validate <artifact.json>...
+ebo task-packet <validate|admit|freeze|status> <bundle-root> <packet.json> [freeze-record.json]
+```
+
+`validate` checks registered artifact schemas. The source-specific readers,
+capture qualification, and export readback perform additional evidence checks.
+
+## Prepare inputs
+
+Validate/admit/freeze a caller-authored packet, then compile a deterministic queue. These commands do not generate tasks or run models.
+
+```sh
+ebo matrix compile <experiment.json> <bundle-root> <queue.json> [--freeze-locator <task-id>=<path>]
+ebo queue inspect <queue.json>
+ebo queue validate <queue.json> [experiment.json] [--bundle-root <bundle-root>]
+```
+
+## Capture one attempt
+
+Execute one selected frozen entry. Use the matching harness configuration and authentication. OpenHands and DeepSeek expose library APIs, not CLI run commands.
+
+```sh
+ebo agent-sdk run <bundle-root> <queue.json> <run-id> <output-root> [--workspace-root <path>]
+ebo codex run <bundle-root> <queue.json> <run-id> <output-root> [--workspace-root <path>]
+ebo cursor run <bundle-root> <queue.json> <run-id> <output-root> [--workspace-root <path>]
+ebo pi run <bundle-root> <queue.json> <run-id> <output-root> [--workspace-root <path>]
+```
+
+## Inspect and share evidence
+
+Index retained manifests, query exact metadata, or create and transport a policy-checked derivative. Export and packing do not publish.
+
+```sh
+ebo export create <run-bundle-root> <policy.json> <export-root>
+ebo corpus build <corpus-root> <index.jsonl>
+ebo corpus query <index.jsonl> [--kind|--run|--attempt|--task|--model|--harness|--assessment-mode|--terminal|--failure-class|--verifier-status|--capture|--export-status|--sharing-class <value>]
+ebo corpus validate <corpus-root> <index.jsonl>
+ebo corpus pack <export-root> <policy.json> <archive.tar.gz>
+ebo corpus unpack <archive.tar.gz> <destination-root>
+```
+
+## Derive facts and evaluate
+
+Observations qualify and normalize retained evidence before extracting structural facts. Judging requires an explicit request and may call a live provider; assertions remain proposals until reviewed.
+
+```sh
+ebo observations create <run-bundle-root> <output.json>
+ebo observations corpus <corpus-root> <index.jsonl> <output-root> [corpus query flags]
+ebo assertions validate <run-bundle-root> <assertion.json> [review.json]
+ebo judge run <run-bundle-root> <observations.json> <request.json> <output-root>
+```
+
+## Record human review
+
+Select evidence and build static review packets. Import human-authored decisions; these commands do not manufacture reviewer identities or judgments.
+
+```sh
+ebo calibration sample <sources.json> <criteria.json> <selection.json>
+ebo calibration packet <selection.json> <output-root>
+ebo calibration inspect <packet.json> <assertion-id> [event-id]
+ebo calibration binding <selection.json> <assertion-id> [history.json]
+ebo calibration <import|adjudicate> <selection.json> <history.json> <decision.json>
+ebo calibration summarize <selection.json> <history.json> <summary.json>
+```
+
+## Compare and present
+
+Check comparison eligibility, aggregate declared populations, and build or serve a local Atlas. These commands consume existing judgments rather than invoking a judge.
+
+```sh
+ebo comparison check <request.json>
+ebo aggregate build <request.json> <output.json>
+ebo atlas build <request.json> <output-root> [--share] [--filter <name=value>]
+ebo atlas serve <request.json> [--port <port>]
+```
+
+## Operational behavior
+
+- A queue-run command executes one entry and creates a new attempt. It does not
+  schedule the whole queue or silently retry.
+- A successful exit can mean a valid **captured failure or stop**. Read the run
+  summary, manifest, and capture report before interpreting the result.
+- `queue inspect` prints summary information. Read `queue.json` for run IDs.
+- Corpus filters use exact matches. Indexes are rebuildable; native manifests
+  and records remain authoritative.
+- There is no standalone `normalize` command: observations invoke the
+  qualified retained-source path.
+- `comparison check` prints a report to stdout; redirect it to a file when an
+  aggregation request needs it. Unsupported comparisons return nonzero.
+- New export, unpack, judgment, and report destinations must not already exist.
+- Atlas defaults to a loopback-only service. Ordinary reports remain restricted;
+  `--share` requires policy-validated source exports and excludes unsupported
+  semantic/review fields.
+
+For preparation and recovery, use the [operator guide](../guides/operator-guide.md).
+For field definitions, use the [reference index](README.md).