@bjoernrochel/sevro 0.1.0-rc.3 → 0.1.0-rc.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CONTRIBUTING.md +54 -7
- package/README.md +6 -6
- package/docs/builtin-graders-v1.md +9 -1
- package/docs/codex-host.md +4 -4
- package/docs/development-cli.md +4 -4
- package/docs/documentation-quality.md +11 -0
- package/docs/getting-started.md +4 -4
- package/docs/identity-v1.md +5 -0
- package/docs/installing.md +21 -11
- package/docs/native-hosts.md +2 -1
- package/docs/releases.md +62 -39
- package/docs/results-v1.md +43 -0
- package/docs/runtime-v1.md +144 -6
- package/docs/typescript-quality.md +56 -21
- package/package.json +2 -1
- package/schemas/runtime-v1.schema.json +7 -0
- package/src/case-validation.ts +82 -0
- package/src/cli.ts +31 -103
- package/src/evaluation-check-plan.ts +510 -0
- package/src/evaluation-context.ts +257 -0
- package/src/evaluation-fixture.ts +3 -5
- package/src/evaluation-grading-host.ts +28 -0
- package/src/evaluation-invocation-plan.ts +548 -0
- package/src/evaluation-plan.ts +15 -1034
- package/src/evaluation-run.ts +19 -1386
- package/src/evaluation-semantic.ts +287 -0
- package/src/evaluation-trial.ts +946 -0
- package/src/evaluation-types.ts +2 -0
- package/src/extension-session.ts +8 -1
- package/src/generated/runtime.cjs +1 -1
- package/src/graders/shell.ts +10 -3
- package/src/hosts/claude-credential.ts +6 -58
- package/src/hosts/claude-events.ts +6 -9
- package/src/hosts/claude-keychain.ts +59 -0
- package/src/hosts/claude-nested-skills.ts +7 -10
- package/src/hosts/claude-process.ts +24 -59
- package/src/hosts/claude-repository-invocation.ts +4 -7
- package/src/hosts/claude-settings.ts +97 -17
- package/src/hosts/claude-state.ts +376 -0
- package/src/hosts/claude-tool-calls.ts +10 -15
- package/src/hosts/claude.ts +73 -291
- package/src/hosts/codex-command-output.ts +7 -11
- package/src/hosts/codex-events.ts +5 -7
- package/src/hosts/codex-native-calls.ts +14 -16
- package/src/hosts/codex-process.ts +26 -66
- package/src/hosts/codex-profile.ts +144 -14
- package/src/hosts/codex-state.ts +386 -0
- package/src/hosts/codex.ts +77 -321
- package/src/hosts/isolation-common.ts +84 -0
- package/src/hosts/isolation-roots.ts +20 -3
- package/src/hosts/isolation.ts +10 -0
- package/src/hosts/linux-sandbox.ts +135 -0
- package/src/hosts/mac-sandbox.ts +23 -85
- package/src/hosts/native-controls.ts +5 -9
- package/src/hosts/process-lifecycle.ts +60 -0
- package/src/hosts/protected-worktrees.ts +19 -3
- package/src/identity.ts +6 -3
- package/src/native-transcript-state.ts +112 -0
- package/src/native-transcript-view.ts +51 -0
- package/src/native-transcripts.ts +267 -0
- package/src/preparation.ts +2 -2
- package/src/provenance.ts +28 -12
- package/src/runtime-config.ts +106 -10
- package/src/runtime-discovery-apple.ts +90 -0
- package/src/runtime-discovery-providers.ts +63 -0
- package/src/runtime-discovery.ts +70 -0
- package/src/runtime-hooks.ts +97 -16
- package/src/runtime-paths.ts +121 -12
- package/src/runtime-query.ts +85 -0
- package/src/runtime-seeds.ts +2 -5
- package/src/runtime-state.ts +13 -2
- package/src/value-guards.ts +6 -0
package/CONTRIBUTING.md
CHANGED
|
@@ -26,8 +26,9 @@ by applicable law.
|
|
|
26
26
|
|
|
27
27
|
## Set up development
|
|
28
28
|
|
|
29
|
-
Use Bun 1.3.13, Git, Node 24, and npm. Full suite isolation
|
|
30
|
-
`
|
|
29
|
+
Use Bun 1.3.13, Git, Node 24, and npm. Full suite isolation uses `sandbox-exec`
|
|
30
|
+
on macOS or `bubblewrap` and `socat` on Ubuntu. Windows acceptance is unverified.
|
|
31
|
+
From a fresh checkout:
|
|
31
32
|
|
|
32
33
|
```sh
|
|
33
34
|
bun install --frozen-lockfile
|
|
@@ -51,8 +52,10 @@ bun run check:typescript
|
|
|
51
52
|
It performs frozen installation, schema freshness, formatting, lint, typing,
|
|
52
53
|
documentation and guide dry-run checks, tutorial examples, the real Bun suite,
|
|
53
54
|
coverage integrity and thresholds, and installed-package validation. Statements
|
|
54
|
-
and branches must each reach at least 95% using raw exact counts over the
|
|
55
|
-
authored production inventory, including unimported files.
|
|
55
|
+
and branches must each reach at least 95% using raw exact counts over the
|
|
56
|
+
platform-reachable authored production inventory, including unimported files.
|
|
57
|
+
The macOS and Ubuntu gates each measure their reachable host modules; the exact
|
|
58
|
+
module and branch-outcome exclusions are recorded in coverage artifacts. The five explicitly
|
|
56
59
|
inventoried generated AJV validators are excluded from coverage because this
|
|
57
60
|
target measures authored runner behavior; their syntax, schema freshness, typed
|
|
58
61
|
boundaries and runtime tests remain checked. Tests and fixtures remain linted and
|
|
@@ -68,9 +71,44 @@ Run `bun run check:docs:external` separately and [triage network failures](docs/
|
|
|
68
71
|
Guide changes also need [native evals](docs/guide-evaluation.md).
|
|
69
72
|
Documentation checks supplement #1 rather than replacing its contract.
|
|
70
73
|
|
|
74
|
+
## Delivery and CI ownership
|
|
75
|
+
|
|
76
|
+
An implementation PR is complete only when required CI checks pass for its exact
|
|
77
|
+
published commit. A request to open or update a PR includes monitoring CI and
|
|
78
|
+
repairing in-scope failures; it does not authorize merging or unrelated changes.
|
|
79
|
+
|
|
80
|
+
- Use GitHub CI in `.github/workflows/verify.yml` for full acceptance validation.
|
|
81
|
+
It provides the configured OS, architecture, runtime and native-tool versions,
|
|
82
|
+
non-root user, and isolation prerequisites. Reproducing that environment locally
|
|
83
|
+
with Docker is optional for diagnosis and is not a publication prerequisite.
|
|
84
|
+
- After the local pre-commit checks, publish the authorized candidate to its PR
|
|
85
|
+
and require `bun run check:typescript` to succeed in GitHub CI before reporting
|
|
86
|
+
delivery as complete. A local run, fast check, targeted suite, or historical
|
|
87
|
+
result does not establish that the current Ubuntu gate passes.
|
|
88
|
+
- Keep the quality bar intact. Repair behavior or add meaningful coverage for
|
|
89
|
+
accepted behavior; do not lower thresholds, exempt authored sources, skip tests,
|
|
90
|
+
bypass checks, or add artificial tests just to move a counter. Remove impossible
|
|
91
|
+
branches by clarifying validated invariants rather than inventing impossible inputs.
|
|
92
|
+
- Bind validation to the candidate's exact source and commit. Changes to source,
|
|
93
|
+
tests, tooling, dependencies, schemas, workflows, or guidance invalidate prior
|
|
94
|
+
candidate evidence; require the applicable CI gates on the updated published head.
|
|
95
|
+
- After authorized publication, observe required checks for the exact PR head
|
|
96
|
+
until they finish. Queued or running checks, a successful older commit, and a
|
|
97
|
+
passing subset are not completion.
|
|
98
|
+
- On failure, inspect the failed job and retained artifacts, reproduce the cause
|
|
99
|
+
locally when useful, repair within the authorized scope, and publish a non-force
|
|
100
|
+
update to the same PR. Continue monitoring without waiting for the user to notice
|
|
101
|
+
the failure or request correction; do not open a duplicate PR or merge it.
|
|
102
|
+
- If an external blocker prevents progress, report its concrete evidence and the
|
|
103
|
+
unfinished checks. Keep the PR's status and description honest; do not report
|
|
104
|
+
delivery as complete while required checks remain failing or unobserved.
|
|
105
|
+
|
|
106
|
+
Release metadata follows `docs/releases.md` and its existing verified-source CI
|
|
107
|
+
reuse policy; these implementation gates do not require release test reruns.
|
|
108
|
+
|
|
71
109
|
## Tests, fixtures, and schemas
|
|
72
110
|
|
|
73
|
-
Preserve real Bun filesystem/process integration tests and
|
|
111
|
+
Preserve real Bun filesystem/process integration tests and native isolation.
|
|
74
112
|
Use deterministic fixtures under `tests/fixtures/`; keep evaluator-only inputs
|
|
75
113
|
outside candidate workspaces. Keep credentials and raw trial evidence out of Git.
|
|
76
114
|
Add meaningful regression coverage for reproducible bugs. Fixtures should
|
|
@@ -102,7 +140,16 @@ native trials or remote CI results.
|
|
|
102
140
|
|
|
103
141
|
## Maintainer releases
|
|
104
142
|
|
|
105
|
-
Follow [releases](docs/releases.md):
|
|
106
|
-
the
|
|
143
|
+
Follow [releases](docs/releases.md): reuse successful source CI, choose a new
|
|
144
|
+
version, and submit the release commit through a PR. Release PRs reuse the
|
|
145
|
+
existing full quality gate and run static metadata checks; releases do not
|
|
146
|
+
rerun local or CI tests. After the PR merges, prepare and inspect the exact
|
|
147
|
+
tarball and verify its license, provenance, and checksums before publication.
|
|
148
|
+
After changing `package.json.version`, run `bun run docs:sync` to refresh the
|
|
149
|
+
[current release](docs/installing.md#current-release). `check:docs`, release
|
|
150
|
+
preparation, and package-install validation reject stale release documentation.
|
|
151
|
+
Package-install, native, and Darrow compatibility tests belong to development
|
|
152
|
+
or an explicitly requested diagnostic, not the release process. Darrow's
|
|
153
|
+
dependency update is a separate PR.
|
|
107
154
|
Publication, tags, registry changes, and alternative licensing are separate
|
|
108
155
|
maintainer actions; preparing a contribution does not authorize them.
|
package/README.md
CHANGED
|
@@ -16,15 +16,15 @@ account is required.
|
|
|
16
16
|
## Status and environments
|
|
17
17
|
|
|
18
18
|
Sevro is under active development. The npm package is `@bjoernrochel/sevro`;
|
|
19
|
-
its command is `sevro`.
|
|
19
|
+
its command is `sevro`. The [current release](docs/installing.md#current-release) includes runtime
|
|
20
20
|
configuration, native goals, selected plugin hooks, and the current licensing terms.
|
|
21
21
|
Earlier `rc.1` and `rc.2` packages retain their original BSL grants.
|
|
22
22
|
See [installation and release differences](docs/installing.md).
|
|
23
23
|
|
|
24
|
-
Use Bun 1.3.13. The
|
|
25
|
-
|
|
24
|
+
Use Bun 1.3.13. The development checkout runs native isolation on macOS with
|
|
25
|
+
`sandbox-exec` and on Ubuntu with `bubblewrap` and `socat`.
|
|
26
26
|
Contributor package/release checks also need Node 24 and npm.
|
|
27
|
-
|
|
27
|
+
Windows support is unverified. The basic example uses an injected
|
|
28
28
|
deterministic adapter and needs no native model credentials.
|
|
29
29
|
|
|
30
30
|
## Install and get a first result
|
|
@@ -33,9 +33,9 @@ deterministic adapter and needs no native model credentials.
|
|
|
33
33
|
mkdir sevro-demo
|
|
34
34
|
cd sevro-demo
|
|
35
35
|
bun init -y
|
|
36
|
-
bun add --exact @bjoernrochel/sevro@0.1.0-rc.3
|
|
37
36
|
```
|
|
38
37
|
|
|
38
|
+
Install the [current release](docs/installing.md#current-release) in that directory.
|
|
39
39
|
Continue with [your first evaluation](docs/getting-started.md): the exact command,
|
|
40
40
|
expected `passed` result, and retained evidence. Read [installation](docs/installing.md)
|
|
41
41
|
for updates, removal, and source checkouts.
|
|
@@ -72,7 +72,7 @@ environment.
|
|
|
72
72
|
|
|
73
73
|
## License and acknowledgements
|
|
74
74
|
|
|
75
|
-
|
|
75
|
+
The [current release](docs/installing.md#current-release) uses the [Sevro Source Available License 1.0](LICENSE).
|
|
76
76
|
Use is free in every context, including commercial work and paid education.
|
|
77
77
|
Selling copies, rebranded versions, or hosted access to Sevro's functionality
|
|
78
78
|
requires a separate license. Modification and free redistribution remain allowed.
|
|
@@ -38,7 +38,7 @@ The shell-check process runner accepts a bounded evaluator-owned `run` string,
|
|
|
38
38
|
optional `expectedExitCode` and `timeoutMs`, plus `expectExact`, `expectRegex`,
|
|
39
39
|
`notRegex`, and regex `flags` for stdout. Exact matching removes one final
|
|
40
40
|
newline; regex matching always uses multiline mode. The runner executes with
|
|
41
|
-
`sh -e`. On macOS it runs through the outer sandbox with network access denied,
|
|
41
|
+
`sh -e`. On macOS and Linux it runs through the outer sandbox with network access denied,
|
|
42
42
|
a credential-free environment, and caller-declared protected roots. Without a
|
|
43
43
|
stdout assertion, process output is discarded. With one, the runner captures
|
|
44
44
|
at most 1 MiB in memory and retains only its digest and byte length. Oversized
|
|
@@ -87,3 +87,11 @@ grading an error. Checks are grouped by source, so response and document checks
|
|
|
87
87
|
remain independent in the same trial. Artifact outcomes retain the resolved
|
|
88
88
|
path and content digest. The semantic host uses an empty workspace and receives
|
|
89
89
|
the document as untrusted data.
|
|
90
|
+
|
|
91
|
+
When [native transcripts are enabled](runtime-v1.md#native-transcript-access),
|
|
92
|
+
shell checks and semantic hosts can read retained candidate native files through
|
|
93
|
+
`SEVRO_CANDIDATE_TRANSCRIPTS`. The view has `index.json` plus a `codex/` or
|
|
94
|
+
`claude/` native tree. It is read-only and separate from the grading host's own
|
|
95
|
+
live native tree. Native transcripts are untrusted candidate evidence; a grader
|
|
96
|
+
must not follow instructions inside them. Missing or partial capture remains
|
|
97
|
+
explicit in the index and does not establish required complete evidence.
|
package/docs/codex-host.md
CHANGED
|
@@ -130,16 +130,16 @@ available through the engine's injected host interface and the development
|
|
|
130
130
|
CLI's explicit `--host codex` route. Its protocol and evidence host ID is
|
|
131
131
|
`sevro.host.codex`; `codex` remains the CLI route selector.
|
|
132
132
|
|
|
133
|
-
`prepareMacSandboxCommand`
|
|
133
|
+
`prepareMacSandboxCommand` and `prepareLinuxSandboxCommand` provide the shell-grading boundary. Callers
|
|
134
134
|
provide absolute protected roots and a private state root; the primitive
|
|
135
135
|
canonicalizes them, refuses overlap with the fixture workspace, and creates a
|
|
136
|
-
temporary
|
|
137
|
-
|
|
136
|
+
temporary native sandbox that denies reads and writes. It fails if
|
|
137
|
+
isolation is unavailable. The caller releases the sandbox resources after the
|
|
138
138
|
process exits. The runner assembles the same protected-root set for Codex
|
|
139
139
|
candidate commands and shell grading.
|
|
140
140
|
|
|
141
141
|
Codex cannot start its native sandbox inside an outer `sandbox-exec` process
|
|
142
|
-
on
|
|
142
|
+
on macOS. Its runner-generated permission profile therefore sets an
|
|
143
143
|
explicit workspace rule, denies all other filesystem roots except minimal
|
|
144
144
|
runtime files and the selected executable's install directories, and
|
|
145
145
|
specifically denies protected roots and the private Codex home. An executable
|
package/docs/development-cli.md
CHANGED
|
@@ -107,7 +107,7 @@ The same worktree protection applies to other declared source and private roots;
|
|
|
107
107
|
a package without Git metadata remains usable. Result and active-run storage
|
|
108
108
|
stay independent through `--results-root` and `--run-state-root`.
|
|
109
109
|
|
|
110
|
-
A bundled Claude Code candidate route is available on macOS:
|
|
110
|
+
A bundled Claude Code candidate route is available on macOS and Linux:
|
|
111
111
|
|
|
112
112
|
```sh
|
|
113
113
|
sevro run --json \
|
|
@@ -133,7 +133,7 @@ values are excluded from retained evidence and configuration digests. Pass
|
|
|
133
133
|
`--claude-credential-file` with an absolute path to select a separate saved login.
|
|
134
134
|
It accepts declared Claude
|
|
135
135
|
plugin directories and explicit skill invocations from extension preparation.
|
|
136
|
-
It runs candidate tools under Claude Code's native
|
|
136
|
+
It runs candidate tools under Claude Code's native sandbox with private,
|
|
137
137
|
source, and result paths denied. The route disables hooks by default and does not expose
|
|
138
138
|
the Write tool. It currently supports passive candidate turns only. Use
|
|
139
139
|
`--protected-root` to exclude additional private paths from the candidate's
|
|
@@ -317,8 +317,8 @@ For shell checks, add
|
|
|
317
317
|
`--shell-isolation` and repeat `--protected-root /absolute/path` for every
|
|
318
318
|
additional source worktree or private root. The engine always protects the
|
|
319
319
|
selected project, results, runner source, user home, configured host homes,
|
|
320
|
-
and active peer fixtures. Shell checks require macOS `sandbox-exec`
|
|
321
|
-
|
|
320
|
+
and active peer fixtures. Shell checks require macOS `sandbox-exec` or Linux
|
|
321
|
+
`bubblewrap`.
|
|
322
322
|
Injected host adapters may return namespaced observations and bounded evidence
|
|
323
323
|
artifacts. A case can list their IDs in `requiredEvidence`; missing or partial
|
|
324
324
|
host evidence produces unavailable grading rather than a passing task. The
|
|
@@ -15,6 +15,17 @@ marked tutorial commands in fresh consumer/contributor directories, including
|
|
|
15
15
|
expected verdicts and saved evidence. Candidate and released-package runs are
|
|
16
16
|
reported separately. These commands create and clean temporary directories.
|
|
17
17
|
|
|
18
|
+
`package.json.version` is the authoritative current version. `bun run docs:sync`
|
|
19
|
+
updates only the marked [Current release](installing.md#current-release) section
|
|
20
|
+
with its exact installation command. Keep its markers and heading in place;
|
|
21
|
+
the command refuses missing, duplicate, or reversed markers and invalid versions.
|
|
22
|
+
The rest of the page stays ordinary Markdown. `check:docs` compares this section
|
|
23
|
+
without writing; release preparation and package-install validation also reject
|
|
24
|
+
stale output. Run synchronization after a version bump and include the resulting
|
|
25
|
+
documentation change. Link installation and update instructions to this section.
|
|
26
|
+
Keep historical versions, licensing transitions, and minimum supported versions
|
|
27
|
+
fixed and explain their purpose beside each reference.
|
|
28
|
+
|
|
18
29
|
## External links
|
|
19
30
|
|
|
20
31
|
Run `bun run check:docs:external` separately. Transient failures are retried
|
package/docs/getting-started.md
CHANGED
|
@@ -7,17 +7,17 @@ This teaches the runner lifecycle, not the quality of an actual model.
|
|
|
7
7
|
|
|
8
8
|
## From an installed package
|
|
9
9
|
|
|
10
|
-
|
|
10
|
+
Prepare a new directory:
|
|
11
11
|
|
|
12
12
|
```sh
|
|
13
13
|
mkdir sevro-demo
|
|
14
14
|
cd sevro-demo
|
|
15
15
|
bun init -y
|
|
16
|
-
bun add --exact @bjoernrochel/sevro@0.1.0-rc.3
|
|
17
16
|
```
|
|
18
17
|
|
|
19
|
-
|
|
20
|
-
|
|
18
|
+
Install the [current release](installing.md#current-release) there, then return
|
|
19
|
+
here to run the example. Read [version differences](installing.md#published-package-and-checkout)
|
|
20
|
+
for the current docs, source-available license, and earlier BSL releases.
|
|
21
21
|
|
|
22
22
|
Make the local installation available on `PATH` for this shell, then call
|
|
23
23
|
`sevro` directly. Bun must also be on `PATH`.
|
package/docs/identity-v1.md
CHANGED
|
@@ -69,3 +69,8 @@ Selected [runtime configuration](runtime-v1.md) is retained under
|
|
|
69
69
|
It includes declared non-secret environment values, canonical read roots,
|
|
70
70
|
private seed digests, and hook authority. Per-role output paths are resolved
|
|
71
71
|
only during execution; runtime placeholders remain symbolic in identity.
|
|
72
|
+
Runtime roots also retain lexical aliases, metadata discovery sources,
|
|
73
|
+
and skipped optional directories. Moving or
|
|
74
|
+
changing the effective host grants changes identity, even when the portable
|
|
75
|
+
configuration file is unchanged. External installation contents remain outside
|
|
76
|
+
this digest unless explicitly captured as private seed inputs.
|
package/docs/installing.md
CHANGED
|
@@ -10,13 +10,26 @@ Linux and Windows are unverified.
|
|
|
10
10
|
mkdir sevro-demo
|
|
11
11
|
cd sevro-demo
|
|
12
12
|
bun init -y
|
|
13
|
-
bun add --exact @bjoernrochel/sevro@0.1.0-rc.3
|
|
14
13
|
```
|
|
15
14
|
|
|
15
|
+
Then install the [current release](#current-release) in that directory.
|
|
16
|
+
|
|
16
17
|
Run the [packaged deterministic example](getting-started.md#from-an-installed-package)
|
|
17
18
|
to verify installation. It needs neither model credentials nor Darrow.
|
|
18
19
|
Local installation records the selected version in the project lockfile.
|
|
19
20
|
|
|
21
|
+
<!-- sevro-current-release:start -->
|
|
22
|
+
|
|
23
|
+
## Current release
|
|
24
|
+
|
|
25
|
+
The current release is `0.1.0-rc.5`. Install this exact version:
|
|
26
|
+
|
|
27
|
+
```sh
|
|
28
|
+
bun add --exact @bjoernrochel/sevro@0.1.0-rc.5
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
<!-- sevro-current-release:end -->
|
|
32
|
+
|
|
20
33
|
## Make the command available
|
|
21
34
|
|
|
22
35
|
The examples call `sevro` from `PATH`. For a local installation, run this from
|
|
@@ -33,10 +46,11 @@ no adjustment is needed. Bun must also be on `PATH`: the executable uses
|
|
|
33
46
|
|
|
34
47
|
## Published package and checkout
|
|
35
48
|
|
|
36
|
-
The
|
|
49
|
+
The [current release](#current-release) includes runtime
|
|
37
50
|
configuration, native goals, selected plugin hooks, current onboarding, and
|
|
38
51
|
[source-available licensing terms](licensing.md). Earlier `rc.1` and `rc.2`
|
|
39
|
-
packages retain BUSL-1.1.
|
|
52
|
+
packages retain BUSL-1.1. The license transition began with `0.1.0-rc.3`.
|
|
53
|
+
Source checkouts can include changes beyond their
|
|
40
54
|
package version; compare the exact release tag when establishing provenance.
|
|
41
55
|
|
|
42
56
|
The candidate npm package contains sources, schemas, examples, documentation,
|
|
@@ -57,14 +71,10 @@ directories; it does not publish. [Contributing](../CONTRIBUTING.md) lists check
|
|
|
57
71
|
|
|
58
72
|
## Update
|
|
59
73
|
|
|
60
|
-
Review a release's license and compatibility, then
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
```
|
|
65
|
-
|
|
66
|
-
This selects `rc.3` explicitly. For later updates, substitute the intended
|
|
67
|
-
published version, then rerun the example.
|
|
74
|
+
Review a release's license and compatibility, then use the exact installation
|
|
75
|
+
command for the [current release](#current-release) from your existing project.
|
|
76
|
+
To select a different published version, substitute that exact version in the
|
|
77
|
+
command, then rerun the [packaged example](getting-started.md#from-an-installed-package).
|
|
68
78
|
Rollback uses the prior exact version and another verification run.
|
|
69
79
|
[Releases](releases.md) explains artifact provenance.
|
|
70
80
|
|
package/docs/native-hosts.md
CHANGED
|
@@ -1,7 +1,8 @@
|
|
|
1
1
|
# Run native model hosts
|
|
2
2
|
|
|
3
3
|
The basic tutorial needs none of this setup. Bundled model routes and isolated
|
|
4
|
-
shell checks require macOS `sandbox-exec
|
|
4
|
+
shell checks require macOS `sandbox-exec` or Linux `bubblewrap` (and `socat` for
|
|
5
|
+
Claude Code), a supported native CLI, and model
|
|
5
6
|
authentication. Complete commands/rules live in the [CLI reference](development-cli.md)
|
|
6
7
|
and [Codex host reference](codex-host.md).
|
|
7
8
|
|
package/docs/releases.md
CHANGED
|
@@ -7,7 +7,7 @@ exact semantic version; development versions containing `-dev` are not release
|
|
|
7
7
|
candidates. Darrow pins the published version in development dependencies and
|
|
8
8
|
its lockfile. Marketplace plugins do not depend on Sevro.
|
|
9
9
|
|
|
10
|
-
|
|
10
|
+
The [current release](installing.md#current-release) uses the current [licensing terms](licensing.md).
|
|
11
11
|
Published `rc.1` and `rc.2` retain their original BSL grants. Always select a new
|
|
12
12
|
version for publication; a local development tarball does not replace an
|
|
13
13
|
already published artifact.
|
|
@@ -28,8 +28,9 @@ The record identifies the package, version, release tag, distribution tag,
|
|
|
28
28
|
tarball SHA-256 and npm SHA-512 integrity, and the preparation runtimes.
|
|
29
29
|
Preparation does not publish a package or create a Git tag.
|
|
30
30
|
|
|
31
|
-
|
|
32
|
-
|
|
31
|
+
For development or explicitly requested diagnostics, the package installation
|
|
32
|
+
check accepts `--tarball <absolute file>` to verify an exact candidate without
|
|
33
|
+
repacking. It is not a release step. The supplied tarball remains intact.
|
|
33
34
|
Its installed name and version must match the source metadata. Installation,
|
|
34
35
|
CLI execution, generic grading, prompt-only execution, release provenance,
|
|
35
36
|
reporting, and supported deterministic native fixtures run without source Git
|
|
@@ -37,56 +38,78 @@ metadata in the installed package.
|
|
|
37
38
|
|
|
38
39
|
## Maintainer workflow
|
|
39
40
|
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
41
|
+
Release already verified code. Do not rerun tests locally or in CI as part of a
|
|
42
|
+
release: reuse the successful source CI run. Code, dependencies, schemas, tests,
|
|
43
|
+
examples, license changes, and other quality inputs belong in ordinary changes
|
|
44
|
+
with their own CI before starting the release.
|
|
45
|
+
|
|
46
|
+
1. Start a release branch from verified `main`. Update `package.json.version`
|
|
47
|
+
to a new release version, keep the selected license, and run `bun run docs:sync`.
|
|
48
|
+
Set `publishConfig.tag` to `latest` for the recommended default or `next`
|
|
49
|
+
for an opt-in preview. Commit and open a pull request; every release commit
|
|
50
|
+
must reach `main` through a PR. Never attempt a direct release push to `main`.
|
|
51
|
+
2. Merge the release PR after its static checks and reused CI evidence pass.
|
|
52
|
+
Release CI compares the candidate against a successful ancestor run on
|
|
53
|
+
`main`, permitting only version/distribution-tag changes and the exact
|
|
54
|
+
release documentation and workflow files inventoried in
|
|
55
|
+
[the CI selector](../.github/scripts/release-ci.py). All other tracked inputs
|
|
56
|
+
must match. The original run must have completed the full quality gate;
|
|
57
|
+
another release's reused status is insufficient. CI retains the source SHA
|
|
58
|
+
and run URL in `release-ci.json`. Missing evidence fails the release check;
|
|
59
|
+
it does not trigger another test run. Ordinary code changes and explicit
|
|
60
|
+
workflow dispatches continue to run the full gate.
|
|
61
|
+
3. From the clean, merged release commit, use macOS, Bun 1.3.13, Node 24,
|
|
62
|
+
and npm to prepare one exact candidate:
|
|
47
63
|
|
|
48
64
|
```sh
|
|
65
|
+
bun install --frozen-lockfile
|
|
49
66
|
bun run release:prepare --tag "v${SEVRO_RELEASE_VERSION:?set the selected new release version}" --output /absolute/new/release-directory
|
|
50
|
-
bun run test:package-install --tarball "/absolute/new/release-directory/bjoernrochel-sevro-${SEVRO_RELEASE_VERSION}.tgz"
|
|
51
67
|
```
|
|
52
68
|
|
|
53
|
-
4.
|
|
69
|
+
4. Inspect the version, repository, license, distribution tag, file inventory,
|
|
70
|
+
and retained checksums. Publish that tarball with lifecycle scripts disabled
|
|
71
|
+
using the commands below. Do not run `check:typescript`, `test:package-install`,
|
|
72
|
+
Darrow's archive/installed gates, or native trials during release. Diagnostic
|
|
73
|
+
tests require a separate explicit request.
|
|
74
|
+
5. Once npm serves the version, compare `dist.integrity` with `release.json`
|
|
75
|
+
and verify the intended distribution tag:
|
|
54
76
|
|
|
55
77
|
```sh
|
|
56
|
-
|
|
78
|
+
npm view "@bjoernrochel/sevro@${SEVRO_RELEASE_VERSION}" version dist.integrity --json
|
|
79
|
+
npm view @bjoernrochel/sevro dist-tags --json
|
|
57
80
|
```
|
|
58
81
|
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
6. After publication, verify registry integrity, install the exact version in
|
|
63
|
-
Darrow, and run its frozen installed compatibility gate. Retain the prior
|
|
64
|
-
version's evidence for updates. Roll back through a new dependency/lockfile
|
|
65
|
-
commit and rerun the installed gate; never relabel failed-version evidence.
|
|
82
|
+
npm may take a few minutes to process the accepted upload. Wait for the
|
|
83
|
+
registry record rather than republishing. Retain the reviewed tarball,
|
|
84
|
+
checksums, and original CI evidence.
|
|
66
85
|
|
|
67
|
-
|
|
86
|
+
6. Create the matching Git tag at the merged release commit and push only the tag:
|
|
68
87
|
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
88
|
+
```sh
|
|
89
|
+
git tag -a "v${SEVRO_RELEASE_VERSION}" -m "Release ${SEVRO_RELEASE_VERSION}"
|
|
90
|
+
git push origin "refs/tags/v${SEVRO_RELEASE_VERSION}"
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
Darrow dependency/lockfile updates and compatibility gates are a separate
|
|
94
|
+
Darrow PR, not prerequisites for npm publication. A prerelease does not establish
|
|
95
|
+
full Darrow migration completion. Preserve prior-version evidence for updates
|
|
96
|
+
and rollback; never relabel failed-version evidence.
|
|
97
|
+
|
|
98
|
+
## Local macOS publication
|
|
73
99
|
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
100
|
+
There is no GitHub release job. The maintainer prepares, inspects, and publishes
|
|
101
|
+
the reviewed tarball from macOS with an authenticated npm account. Check its
|
|
102
|
+
retained checksum immediately before publication:
|
|
77
103
|
|
|
78
104
|
```sh
|
|
79
|
-
|
|
105
|
+
cd /absolute/reviewed
|
|
106
|
+
shasum -a 256 -c SHA256SUMS
|
|
107
|
+
npm publish ./bjoernrochel-sevro-0.1.0-rc.1.tgz --ignore-scripts --access public --tag latest
|
|
80
108
|
```
|
|
81
109
|
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
The package repository URL must match this public repository. See npm's
|
|
86
|
-
[trusted publishing documentation](https://docs.npmjs.com/trusted-publishers/).
|
|
87
|
-
The workflow checks the downloaded artifact's SHA-256 before publishing and
|
|
88
|
-
retains its release record. Configuring an owner or workflow does not itself
|
|
89
|
-
authorize this coding session to publish or push.
|
|
110
|
+
Replace the example version and distribution tag with the reviewed candidate's
|
|
111
|
+
values. The package repository URL must match this public repository. Preparing
|
|
112
|
+
a tarball or opening a pull request does not authorize publication.
|
|
90
113
|
|
|
91
114
|
An existing published candidate can become the default without republishing:
|
|
92
115
|
|
|
@@ -99,7 +122,7 @@ Promotion changes the registry tag; it does not update the published tarball.
|
|
|
99
122
|
|
|
100
123
|
The first published candidate was `0.1.0-rc.1`, using the owner-selected
|
|
101
124
|
`BUSL-1.1` license. Its `LICENSE` preserves Darrow's terms and parameters,
|
|
102
|
-
changing only the Licensed Work name and description to Sevro.
|
|
103
|
-
|
|
125
|
+
changing only the Licensed Work name and description to Sevro. Local checks do
|
|
126
|
+
not claim a remote CI run or publication. Darrow's exact
|
|
104
127
|
registry pin, default caller switch, remaining workflow migrations, focused live validation, and
|
|
105
128
|
generic runner removal remain separate extraction gates.
|
package/docs/results-v1.md
CHANGED
|
@@ -116,3 +116,46 @@ reports `failed` / `not_requested` / `not_assessed` with exit code `2`. A final
|
|
|
116
116
|
message that happens to match a check cannot turn that failure into a pass.
|
|
117
117
|
Claude retains its bounded event stream and, when present, up to 64 KiB of
|
|
118
118
|
stderr as private host artifacts. Failure bodies do not appear in CLI diagnostics.
|
|
119
|
+
|
|
120
|
+
## Native transcript bundles
|
|
121
|
+
|
|
122
|
+
With [runtime transcript opt-in](runtime-v1.md#native-transcript-access), each
|
|
123
|
+
bundled host captures full native files before removing its private state.
|
|
124
|
+
Candidate artifact `sevro.native-transcripts.bundle` has format
|
|
125
|
+
`sevro.native-transcripts.v1`. It contains `host` (`codex` or `claude`), `role`,
|
|
126
|
+
native `rootSessionId` (or `null`), `completeness`, `issues`, and `files`.
|
|
127
|
+
Each file records its native relative `path`, SHA-256 `sha256`, and exact bytes
|
|
128
|
+
as `bytesBase64`. Decode the bytes without rewriting native records. Native
|
|
129
|
+
session paths, root/child records and sidecar metadata preserve associations;
|
|
130
|
+
the bundle does not normalize model, effort, timing, token, or cost statistics.
|
|
131
|
+
Existing unknown metrics remain unknown.
|
|
132
|
+
|
|
133
|
+
`sevro.host.native-transcripts` records the same capture state and file hashes
|
|
134
|
+
without raw bytes. Semantic and advisory bundles retain their role in the
|
|
135
|
+
bundle and use the existing `sevro.semantic[.<source digest>].` and
|
|
136
|
+
`sevro.advisory.` artifact ID prefixes. Run/trial artifact references supply
|
|
137
|
+
stable private file URLs and whole-bundle hashes after native home cleanup.
|
|
138
|
+
Extensions receive these references through the existing `evaluate` protocol;
|
|
139
|
+
benchmarks can use the same references and decode the native files.
|
|
140
|
+
|
|
141
|
+
Capture includes native transcript-tree sidecars and refuses descendant links,
|
|
142
|
+
special files and escaping paths. It is bounded to 4,096 directory entries,
|
|
143
|
+
5 MiB of native file bytes and 7 MiB of encoded file metadata/content. The
|
|
144
|
+
bundle remains inside the existing 8 MiB item and 32 MiB host artifact budgets;
|
|
145
|
+
it groups sessions instead of spending an artifact per child. Limits produce
|
|
146
|
+
`byte_limit`, `entry_limit`, or `bundle_limit` issues. Unreadable, unsafe or
|
|
147
|
+
changing entries, malformed/empty native records, malformed JSON sidecars,
|
|
148
|
+
an unknown/missing original session, or incomplete host
|
|
149
|
+
execution produce explicit partial/unavailable capture. `complete` means the
|
|
150
|
+
identified root's tree was captured within those bounds after completed host
|
|
151
|
+
execution and generic syntax/basic record validation. JSONL must contain
|
|
152
|
+
nonempty JSON object records with a nonempty string `type`; JSON sidecars must
|
|
153
|
+
parse as JSON. UTF-8 decoding is strict. One trailing newline and CRLF are
|
|
154
|
+
accepted; blank interior records, invalid encoding and truncated JSON are
|
|
155
|
+
malformed. Unknown native fields and record types remain valid. Issues
|
|
156
|
+
`empty_transcript`, `malformed_transcript` and `malformed_sidecar` downgrade
|
|
157
|
+
capture without rewriting or discarding retained bytes. This validates basic
|
|
158
|
+
structure, not native semantic correctness.
|
|
159
|
+
An empty/missing native JSONL tree is `unavailable`. Failure, cancellation and
|
|
160
|
+
timeout retain available native bytes with incomplete capture state. Raw
|
|
161
|
+
bundles and views are private evidence and must stay out of Git.
|