openmerit 0.1.6-preview.4 → 0.1.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,18 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.1.6 — 2026-10-11
4
+
5
+ - Promote the code from `0.1.6-preview.5` to the default npm release. Install with `npm install -g openmerit`, then run `openmerit dash` from an application directory.
6
+ - Update the packaged README and terminal guides to present current capabilities and limits. The terminal is a starting point for OpenMerit; this release introduces no new routing or evaluation behavior.
7
+ - Retain the existing coordinator, protocol, Pi adapter, package exports, and Node.js requirement. Earlier npm versions remain available.
8
+
9
+ ## 0.1.6-preview.5 — 2026-10-10
10
+
11
+ - Open the terminal before initial collection, then index repository declarations in a background worker with visible discovery, indexing, and linking progress. Replace repository-wide scan caps with continuing work batches.
12
+ - Reuse unchanged source/configuration metadata across scans, re-resolve changed dotenv bindings, and remove deleted declarations. Keep activity collection independent and disclose skipped files or interrupted indexing.
13
+ - Reuse unchanged reference catalogs in the browser and report complete issue counts alongside the bounded detail list. Request-history and runtime inventory limits are unchanged.
14
+ - Preserve ordinary function names containing `task_` instead of mistaking their suffix for a credential prefix.
15
+
3
16
  ## 0.1.6-preview.4 — 2026-10-10
4
17
 
5
18
  - Index supported application code, nested configuration, custom endpoints, and conservative model/call-site references automatically during `openmerit dash`, with file-and-line evidence.
package/README.md CHANGED
@@ -1,137 +1,76 @@
1
1
  # OpenMerit
2
2
 
3
- OpenMerit is a harness-neutral orchestration layer that helps a coding harness continuously find and maintain the Pareto frontier of models for a real product task.
3
+ OpenMerit is being built to help applications make better inference decisions using evidence about their models, workloads, quality, cost, and latency.
4
4
 
5
- OpenMerit does not perform the intelligent work. It nudges a connected coding harness, such as Pi, with typed outcome requests. The harness understands the project, establishes evaluations and observability, gathers evidence, discovers and tests candidates, calculates the Pareto frontier, and performs an authorized model change. OpenMerit validates the returned contracts, preserves evidence and preferences, and advances the approved lifecycle.
5
+ Version **0.1.6** starts with visibility: the Inference Terminal gives you a local view of the inference your application declares and records. It is a starting point for OpenMerit, not the full product vision. This release brings the existing npm preview into the default installation.
6
6
 
7
- The npm package `openmerit` contains the Pi extension, the reusable coordinator,
8
- and the protocol in one install. The Pi adapter is the first integration; other
9
- harnesses can use the same core and protocol exports.
7
+ ## Install and run
10
8
 
11
- ## Why OpenMerit
9
+ Requires **Node.js 22.19 or newer**. In your application's directory:
12
10
 
13
- The cheapest model, fastest model, and highest-quality model are often different. Public benchmarks are useful before product evidence exists, but they cannot establish which model is best for a particular workload. OpenMerit connects the initial choice to production evidence and repeated reassessment.
14
-
15
- It distinguishes:
16
-
17
- - one-run observations from rate, percentile, consistency, and reliability claims;
18
- - missing evidence from poor performance;
19
- - feasible candidates from dominated, unresolved, or ineligible candidates;
20
- - an automatic nudge from an intelligent action performed by the harness;
21
- - a recommendation from an authorized, verified model swap.
22
-
23
- ## Current capabilities
24
-
25
- - Pi 0.87 extension with a single `/openmerit` command surface.
26
- - Harness-inferred, user-confirmed task profiles with explicit metrics, sampling designs, constraints, and evaluation budgets.
27
- - The complete initial quality, reliability, cost, latency, efficiency, and human-intervention metric catalogue.
28
- - Setup-time preliminary model research in the source checkout, followed by baseline evidence collection, refreshed candidate discovery, and controlled challenger trials. The published `0.1.5` package still begins candidate discovery after the baseline.
29
- - A normative, uncertainty-aware Pareto definition owned by OpenMerit.
30
- - Harness-calculated frontier results independently checked by OpenMerit's conformance verifier.
31
- - Versioned result schemas for all ten intents, reusable across harness adapters and exposed dynamically by Pi.
32
- - Configurable swap approval, post-swap verification requests, and policy-based rollback requests. Automatic swaps can begin immediately if explicitly enabled with a zero verified-swap threshold.
33
- - Project-local snapshots, redacted append-only audit events, evidence manifests, and optional best-effort event exporters under `.openmerit/`.
34
- - Model-catalog change detection that can nudge the harness to reassess a mature baseline.
35
- - User-confirmed task-count, elapsed-time, regression, catalogue-change, and post-swap check policies.
36
- - Idempotent harness signals, durable pending work, native wakeup planning, and explicit scheduler capability gaps.
37
- - In the unreleased source checkout, an opt-in macOS Pi LaunchAgent for time-based checks and a durable pause; published `0.1.5` still requires an external scheduler and has a partial, session-local pause.
38
-
39
- ## Responsibility boundary
40
-
41
- | OpenMerit | Coding harness |
42
- | --- | --- |
43
- | Defines typed outcomes and Pareto conformance rules | Understands the product and user task |
44
- | Tracks lifecycle, policy, budget, and evidence references | Builds evals and observability |
45
- | Determines which approved nudge is due | Collects and calculates metrics |
46
- | Verifies returned structure and frontier conformance | Discovers candidates and calculates the frontier |
47
- | Selects intents according to confirmation and rollback policy | Applies and verifies an authorized model change |
48
-
49
- Jev System One, when configured, belongs to the coding harness. Pi may use it aggressively for bounded ranking, triage, and fast judgment. OpenMerit contains no Jev client and accepts no Jev credential.
50
-
51
- ## Install with Pi
11
+ ```sh
12
+ npm install -g openmerit
13
+ openmerit dash
14
+ ```
52
15
 
53
- Requirements:
16
+ The terminal opens in your browser while collection starts. It runs locally, bound to `127.0.0.1`, and prints its URL. Keep the command running while using it; press Ctrl+C to stop. Pi and model-provider credentials are not required for the terminal.
54
17
 
55
- - Node.js 22.19 or newer
56
- - Pi 0.87 with a supported model provider configured
18
+ You can also point it at a project explicitly:
57
19
 
58
20
  ```sh
59
- pi install npm:openmerit
60
- pi list
21
+ openmerit dash /path/to/your/project
61
22
  ```
62
23
 
63
- When Pi enters an unconfigured project with an interactive UI, the adapter first looks for an actual application LLM call pattern. If the project starts empty, the unreleased source checkout scans again after each settled build turn and automatically begins setup when Pi creates the application call. `/openmerit setup` remains a recovery command rather than a required demo step. Pi confirms the application target and exact incumbent application model, infers task-relevant metrics and their repeated-input or representative-case sampling designs, explains the proposed collection plan and estimated cost, and asks the user to confirm or edit it. It then establishes task-specific eval and observability artifacts, verifies them, and reports evidence to OpenMerit. Pi's own model is never the application target.
24
+ To update from an earlier release or `openmerit@preview`, run `npm install -g openmerit`, stop the old terminal process, and launch it again. To pin this release, use `npm install -g openmerit@0.1.6`. Remove it with `npm uninstall -g openmerit`.
64
25
 
65
- Run `/openmerit doctor` inside Pi to check the installation. OpenMerit keeps
66
- project evidence under `.openmerit/` in the product repository. Version 0.1.5
67
- replaces the 0.1.4 background watcher and standalone CLI with this
68
- harness-driven lifecycle; it does not migrate the earlier home-directory state.
26
+ ## What you can see today
69
27
 
70
- ## Commands
28
+ - **Models and services:** application declarations from supported code and configuration, recorded provider connections, and supported project-linked runtime inventory.
29
+ - **Inference activity:** requests, tokens, latency, costs, and agent/task identifiers where supported application logs or explicit exports provide them.
30
+ - **Code and configuration evidence:** file-and-line references, indexing progress, unresolved references, and source coverage.
31
+ - **Compute and deployments:** explicitly linked hosts, model residency, and supplied hardware or capacity measurements, with their freshness and scope shown.
71
32
 
72
- - `/openmerit` or `/openmerit status` — show evidence readiness and lifecycle stage.
73
- - `/openmerit setup` — explicitly rerun or recover automatic setup.
74
- - `/openmerit assess` — check stored application metric windows against the baseline thresholds now (in the unreleased source checkout); start candidate discovery only when ready. Published `0.1.5` still asks Pi to assess.
75
- - `/openmerit logs` — show the canonical audit, exporter-error, evidence, and state paths.
76
- - `/openmerit frontier` — ask the harness to calculate a frontier now; OpenMerit verifies it.
77
- - `/openmerit approve` — approve a verified proposal during supervised graduation.
78
- - `/openmerit pause` and `/openmerit resume` — control some proactive nudges in the current Pi session; task-triggered checks can still run in 0.1.5. See [pause limitations](docs/commands.md#pause-and-resume).
79
- - `/openmerit doctor` — show sanitized adapter diagnostics.
33
+ The terminal keeps declarations, runtime inventory, and recorded usage separate. Missing evidence stays unknown. Code alone cannot reconstruct request history, usage, costs, or running agents. Discovery supports specific source formats and syntax; it is not complete coverage of every application or provider.
80
34
 
81
- ## Inference Terminal (test preview)
35
+ ## Local and read-only
82
36
 
83
- A local, dark-only overview of application inference inventory, connections, activity, and compute. Coding-harness usage is outside its scope. It reads existing metadata passively and makes no model calls.
37
+ The terminal reads supported project files, logs, exports, and authorized loopback runtime inventory endpoints. It does not execute your application code, call models, route inference, run evaluations, or change your model configuration. It does not retain prompts, responses, or credential values in its snapshots. Closing it leaves your workloads running.
84
38
 
85
- With Node.js 22.19+ installed, run this from your application's directory:
39
+ Repository indexing runs in the background and reuses unchanged metadata in memory. Request history and runtime readers have documented limits. The local API can be reached by other processes or users on the same machine, so treat its model names, endpoint identifiers, and file paths as project data.
40
+
41
+ ## Useful commands
86
42
 
87
43
  ```sh
88
- npm install -g openmerit@preview
89
- openmerit dash
44
+ openmerit dash --no-open # Print the local URL without opening a browser
45
+ openmerit dash --json # Print one collected snapshot, then exit
46
+ openmerit dash --diagnostics # Print repository-index coverage counts only
47
+ openmerit dash --demo # Explore fictional data without local collection
48
+ openmerit dash --help
90
49
  ```
91
50
 
92
- The preview uses the existing `openmerit` npm package, including its coordinator, protocol, Pi adapter, and compiled terminal assets. It opens its localhost URL in your default browser. The stable npm package `openmerit@0.1.5` does not include this command. See the [Inference Terminal guide](docs/inference-terminal.md) for supported sources, updating or removing the preview, source builds, and read-only agent access.
93
-
94
- ## Documentation
95
-
96
- Read the searchable [OpenMerit documentation](https://openmerit.site/docs/). The Markdown sources below also ship with the package.
51
+ See the [Inference Terminal guide](docs/inference-terminal.md) for supported sources, setup, options, data handling, and limitations. You can also [explore the hosted fictional sample](https://dash.openmerit.site/) without installing anything; it does not inspect your environment.
97
52
 
98
- - [Getting started](docs/getting-started.md)
99
- - [Your first trial](docs/first-trial.md)
100
- - [Command reference](docs/commands.md)
101
- - [Budgets and permissions](docs/budgets.md)
102
- - [Troubleshooting](docs/troubleshooting.md)
103
- - [Architecture and responsibility boundary](docs/architecture.md)
104
- - [Metrics and evidence](docs/metrics-and-evidence.md)
105
- - [Improvement lifecycle](docs/lifecycle.md)
106
- - [Harness-neutral automation](docs/automation.md)
107
- - [Pi extension guide](docs/pi-extension.md)
108
- - [Building another harness adapter](docs/adapter-guide.md)
109
- - [Pareto conformance specification](docs/pareto-spec.md)
110
- - [Project state and operations](docs/operations.md)
111
- - [Security](docs/security.md)
112
- - [Testing](docs/testing.md)
113
- - [Current limitations and roadmap](docs/roadmap.md)
53
+ ## Optional Pi integration
114
54
 
115
- ### Maintaining documentation
55
+ The package also retains the Pi extension, reusable coordinator, and protocol. These are separate from the terminal. Pi can gather application evidence and carry out user-authorized model comparisons and changes; OpenMerit coordinates typed outcomes and validates the returned structure and frontier evidence.
116
56
 
117
- Edit the Markdown in `docs/` alongside product changes. The public documentation is generated from these same files; there is no separate article copy to update in `site/`. Run `npm run check:docs` to rebuild and validate it. The normal Cloudflare deployment rebuilds and publishes the documentation. See [the documentation workflow](docs/site-documentation.md) for adding guides and previewing changes.
57
+ For Pi 0.87 with a supported model provider configured:
118
58
 
119
- ## Repository layout
59
+ ```sh
60
+ pi install npm:openmerit
61
+ ```
120
62
 
121
- - `packages/protocol` — harness-neutral types for intents, results, metrics, policies, and frontier evidence.
122
- - `packages/core` — adapter SDK, durable coordinator, pluggable persistence, policy, and frontier conformance verification.
123
- - `packages/pi` — Pi extension that turns OpenMerit intents into harness work.
124
- - `docs` — user, operator, architecture, and normative documentation.
63
+ Then use `/openmerit doctor` inside Pi. See [Pi setup](docs/getting-started.md), [commands](docs/commands.md), [budgets and permissions](docs/budgets.md), and [current limitations](docs/roadmap.md). The terminal itself does not perform this evaluation or routing work.
125
64
 
126
- The workspace modules are private development boundaries. The published
127
- package exposes `openmerit/core`, `openmerit/protocol`, and `openmerit/pi`.
65
+ The existing `openmerit/core`, `openmerit/protocol`, and `openmerit/pi` exports remain available for integrations.
128
66
 
129
- ## Development from source
67
+ ## Documentation
130
68
 
131
- ```sh
132
- npm ci
133
- npm run verify
134
- ./node_modules/.bin/pi --no-extensions -e ./packages/pi/src/index.ts
135
- ```
69
+ - [Inference Terminal](docs/inference-terminal.md)
70
+ - [Security and data handling](docs/security.md)
71
+ - [Project files and logs](docs/operations.md)
72
+ - [Architecture and integrations](docs/architecture.md)
73
+ - [Testing and validation](docs/testing.md)
74
+ - [Changelog](CHANGELOG.md)
136
75
 
137
- `--no-extensions` prevents an installed npm copy of OpenMerit from registering the same tools as the source extension. Credentials must never be committed. The repository intentionally contains no provider or Jev API key.
76
+ The bundled Inference Terminal guide follows this package version. The separately deployed [documentation website](https://openmerit.site/docs/) may reflect a different release.