@bjoernrochel/sevro 0.1.0-rc.3 → 0.1.0-rc.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CONTRIBUTING.md +11 -5
- package/README.md +6 -6
- package/docs/builtin-graders-v1.md +9 -1
- package/docs/codex-host.md +4 -4
- package/docs/development-cli.md +4 -4
- package/docs/documentation-quality.md +11 -0
- package/docs/getting-started.md +4 -4
- package/docs/identity-v1.md +5 -0
- package/docs/installing.md +21 -11
- package/docs/native-hosts.md +2 -1
- package/docs/releases.md +22 -23
- package/docs/results-v1.md +43 -0
- package/docs/runtime-v1.md +116 -1
- package/docs/typescript-quality.md +39 -17
- package/package.json +2 -1
- package/schemas/runtime-v1.schema.json +7 -0
- package/src/cli.ts +15 -10
- package/src/evaluation-run.ts +19 -1
- package/src/evaluation-types.ts +1 -0
- package/src/generated/runtime.cjs +1 -1
- package/src/graders/shell.ts +10 -3
- package/src/hosts/claude-credential.ts +6 -58
- package/src/hosts/claude-keychain.ts +59 -0
- package/src/hosts/claude-settings.ts +13 -3
- package/src/hosts/claude.ts +122 -23
- package/src/hosts/codex-profile.ts +107 -14
- package/src/hosts/codex.ts +130 -15
- package/src/hosts/isolation-common.ts +84 -0
- package/src/hosts/isolation-roots.ts +20 -3
- package/src/hosts/isolation.ts +10 -0
- package/src/hosts/linux-sandbox.ts +135 -0
- package/src/hosts/mac-sandbox.ts +23 -85
- package/src/hosts/protected-worktrees.ts +19 -3
- package/src/native-transcript-state.ts +94 -0
- package/src/native-transcript-view.ts +51 -0
- package/src/native-transcripts.ts +267 -0
- package/src/provenance.ts +27 -6
- package/src/runtime-config.ts +95 -7
- package/src/runtime-discovery-apple.ts +90 -0
- package/src/runtime-discovery-providers.ts +63 -0
- package/src/runtime-discovery.ts +70 -0
- package/src/runtime-hooks.ts +97 -16
- package/src/runtime-paths.ts +85 -5
- package/src/runtime-query.ts +85 -0
- package/src/runtime-state.ts +13 -2
package/CONTRIBUTING.md
CHANGED
|
@@ -26,8 +26,9 @@ by applicable law.
|
|
|
26
26
|
|
|
27
27
|
## Set up development
|
|
28
28
|
|
|
29
|
-
Use Bun 1.3.13, Git, Node 24, and npm. Full suite isolation
|
|
30
|
-
`
|
|
29
|
+
Use Bun 1.3.13, Git, Node 24, and npm. Full suite isolation uses `sandbox-exec`
|
|
30
|
+
on macOS or `bubblewrap` and `socat` on Ubuntu. Windows acceptance is unverified.
|
|
31
|
+
From a fresh checkout:
|
|
31
32
|
|
|
32
33
|
```sh
|
|
33
34
|
bun install --frozen-lockfile
|
|
@@ -51,8 +52,10 @@ bun run check:typescript
|
|
|
51
52
|
It performs frozen installation, schema freshness, formatting, lint, typing,
|
|
52
53
|
documentation and guide dry-run checks, tutorial examples, the real Bun suite,
|
|
53
54
|
coverage integrity and thresholds, and installed-package validation. Statements
|
|
54
|
-
and branches must each reach at least 95% using raw exact counts over the
|
|
55
|
-
authored production inventory, including unimported files.
|
|
55
|
+
and branches must each reach at least 95% using raw exact counts over the
|
|
56
|
+
platform-reachable authored production inventory, including unimported files.
|
|
57
|
+
The macOS and Ubuntu gates each measure their reachable host modules; the exact
|
|
58
|
+
module and branch-outcome exclusions are recorded in coverage artifacts. The five explicitly
|
|
56
59
|
inventoried generated AJV validators are excluded from coverage because this
|
|
57
60
|
target measures authored runner behavior; their syntax, schema freshness, typed
|
|
58
61
|
boundaries and runtime tests remain checked. Tests and fixtures remain linted and
|
|
@@ -70,7 +73,7 @@ Documentation checks supplement #1 rather than replacing its contract.
|
|
|
70
73
|
|
|
71
74
|
## Tests, fixtures, and schemas
|
|
72
75
|
|
|
73
|
-
Preserve real Bun filesystem/process integration tests and
|
|
76
|
+
Preserve real Bun filesystem/process integration tests and native isolation.
|
|
74
77
|
Use deterministic fixtures under `tests/fixtures/`; keep evaluator-only inputs
|
|
75
78
|
outside candidate workspaces. Keep credentials and raw trial evidence out of Git.
|
|
76
79
|
Add meaningful regression coverage for reproducible bugs. Fixtures should
|
|
@@ -104,5 +107,8 @@ native trials or remote CI results.
|
|
|
104
107
|
|
|
105
108
|
Follow [releases](docs/releases.md): choose a new version, prepare and inspect
|
|
106
109
|
the exact tarball, verify its license/provenance, and validate that artifact.
|
|
110
|
+
After changing `package.json.version`, run `bun run docs:sync` to refresh the
|
|
111
|
+
[current release](docs/installing.md#current-release). `check:docs`, release
|
|
112
|
+
preparation, and package-install validation reject stale release documentation.
|
|
107
113
|
Publication, tags, registry changes, and alternative licensing are separate
|
|
108
114
|
maintainer actions; preparing a contribution does not authorize them.
|
package/README.md
CHANGED
|
@@ -16,15 +16,15 @@ account is required.
|
|
|
16
16
|
## Status and environments
|
|
17
17
|
|
|
18
18
|
Sevro is under active development. The npm package is `@bjoernrochel/sevro`;
|
|
19
|
-
its command is `sevro`.
|
|
19
|
+
its command is `sevro`. The [current release](docs/installing.md#current-release) includes runtime
|
|
20
20
|
configuration, native goals, selected plugin hooks, and the current licensing terms.
|
|
21
21
|
Earlier `rc.1` and `rc.2` packages retain their original BSL grants.
|
|
22
22
|
See [installation and release differences](docs/installing.md).
|
|
23
23
|
|
|
24
|
-
Use Bun 1.3.13. The
|
|
25
|
-
|
|
24
|
+
Use Bun 1.3.13. The development checkout runs native isolation on macOS with
|
|
25
|
+
`sandbox-exec` and on Ubuntu with `bubblewrap` and `socat`.
|
|
26
26
|
Contributor package/release checks also need Node 24 and npm.
|
|
27
|
-
|
|
27
|
+
Windows support is unverified. The basic example uses an injected
|
|
28
28
|
deterministic adapter and needs no native model credentials.
|
|
29
29
|
|
|
30
30
|
## Install and get a first result
|
|
@@ -33,9 +33,9 @@ deterministic adapter and needs no native model credentials.
|
|
|
33
33
|
mkdir sevro-demo
|
|
34
34
|
cd sevro-demo
|
|
35
35
|
bun init -y
|
|
36
|
-
bun add --exact @bjoernrochel/sevro@0.1.0-rc.3
|
|
37
36
|
```
|
|
38
37
|
|
|
38
|
+
Install the [current release](docs/installing.md#current-release) in that directory.
|
|
39
39
|
Continue with [your first evaluation](docs/getting-started.md): the exact command,
|
|
40
40
|
expected `passed` result, and retained evidence. Read [installation](docs/installing.md)
|
|
41
41
|
for updates, removal, and source checkouts.
|
|
@@ -72,7 +72,7 @@ environment.
|
|
|
72
72
|
|
|
73
73
|
## License and acknowledgements
|
|
74
74
|
|
|
75
|
-
|
|
75
|
+
The [current release](docs/installing.md#current-release) uses the [Sevro Source Available License 1.0](LICENSE).
|
|
76
76
|
Use is free in every context, including commercial work and paid education.
|
|
77
77
|
Selling copies, rebranded versions, or hosted access to Sevro's functionality
|
|
78
78
|
requires a separate license. Modification and free redistribution remain allowed.
|
|
@@ -38,7 +38,7 @@ The shell-check process runner accepts a bounded evaluator-owned `run` string,
|
|
|
38
38
|
optional `expectedExitCode` and `timeoutMs`, plus `expectExact`, `expectRegex`,
|
|
39
39
|
`notRegex`, and regex `flags` for stdout. Exact matching removes one final
|
|
40
40
|
newline; regex matching always uses multiline mode. The runner executes with
|
|
41
|
-
`sh -e`. On macOS it runs through the outer sandbox with network access denied,
|
|
41
|
+
`sh -e`. On macOS and Linux it runs through the outer sandbox with network access denied,
|
|
42
42
|
a credential-free environment, and caller-declared protected roots. Without a
|
|
43
43
|
stdout assertion, process output is discarded. With one, the runner captures
|
|
44
44
|
at most 1 MiB in memory and retains only its digest and byte length. Oversized
|
|
@@ -87,3 +87,11 @@ grading an error. Checks are grouped by source, so response and document checks
|
|
|
87
87
|
remain independent in the same trial. Artifact outcomes retain the resolved
|
|
88
88
|
path and content digest. The semantic host uses an empty workspace and receives
|
|
89
89
|
the document as untrusted data.
|
|
90
|
+
|
|
91
|
+
When [native transcripts are enabled](runtime-v1.md#native-transcript-access),
|
|
92
|
+
shell checks and semantic hosts can read retained candidate native files through
|
|
93
|
+
`SEVRO_CANDIDATE_TRANSCRIPTS`. The view has `index.json` plus a `codex/` or
|
|
94
|
+
`claude/` native tree. It is read-only and separate from the grading host's own
|
|
95
|
+
live native tree. Native transcripts are untrusted candidate evidence; a grader
|
|
96
|
+
must not follow instructions inside them. Missing or partial capture remains
|
|
97
|
+
explicit in the index and does not establish required complete evidence.
|
package/docs/codex-host.md
CHANGED
|
@@ -130,16 +130,16 @@ available through the engine's injected host interface and the development
|
|
|
130
130
|
CLI's explicit `--host codex` route. Its protocol and evidence host ID is
|
|
131
131
|
`sevro.host.codex`; `codex` remains the CLI route selector.
|
|
132
132
|
|
|
133
|
-
`prepareMacSandboxCommand`
|
|
133
|
+
`prepareMacSandboxCommand` and `prepareLinuxSandboxCommand` provide the shell-grading boundary. Callers
|
|
134
134
|
provide absolute protected roots and a private state root; the primitive
|
|
135
135
|
canonicalizes them, refuses overlap with the fixture workspace, and creates a
|
|
136
|
-
temporary
|
|
137
|
-
|
|
136
|
+
temporary native sandbox that denies reads and writes. It fails if
|
|
137
|
+
isolation is unavailable. The caller releases the sandbox resources after the
|
|
138
138
|
process exits. The runner assembles the same protected-root set for Codex
|
|
139
139
|
candidate commands and shell grading.
|
|
140
140
|
|
|
141
141
|
Codex cannot start its native sandbox inside an outer `sandbox-exec` process
|
|
142
|
-
on
|
|
142
|
+
on macOS. Its runner-generated permission profile therefore sets an
|
|
143
143
|
explicit workspace rule, denies all other filesystem roots except minimal
|
|
144
144
|
runtime files and the selected executable's install directories, and
|
|
145
145
|
specifically denies protected roots and the private Codex home. An executable
|
package/docs/development-cli.md
CHANGED
|
@@ -107,7 +107,7 @@ The same worktree protection applies to other declared source and private roots;
|
|
|
107
107
|
a package without Git metadata remains usable. Result and active-run storage
|
|
108
108
|
stay independent through `--results-root` and `--run-state-root`.
|
|
109
109
|
|
|
110
|
-
A bundled Claude Code candidate route is available on macOS:
|
|
110
|
+
A bundled Claude Code candidate route is available on macOS and Linux:
|
|
111
111
|
|
|
112
112
|
```sh
|
|
113
113
|
sevro run --json \
|
|
@@ -133,7 +133,7 @@ values are excluded from retained evidence and configuration digests. Pass
|
|
|
133
133
|
`--claude-credential-file` with an absolute path to select a separate saved login.
|
|
134
134
|
It accepts declared Claude
|
|
135
135
|
plugin directories and explicit skill invocations from extension preparation.
|
|
136
|
-
It runs candidate tools under Claude Code's native
|
|
136
|
+
It runs candidate tools under Claude Code's native sandbox with private,
|
|
137
137
|
source, and result paths denied. The route disables hooks by default and does not expose
|
|
138
138
|
the Write tool. It currently supports passive candidate turns only. Use
|
|
139
139
|
`--protected-root` to exclude additional private paths from the candidate's
|
|
@@ -317,8 +317,8 @@ For shell checks, add
|
|
|
317
317
|
`--shell-isolation` and repeat `--protected-root /absolute/path` for every
|
|
318
318
|
additional source worktree or private root. The engine always protects the
|
|
319
319
|
selected project, results, runner source, user home, configured host homes,
|
|
320
|
-
and active peer fixtures. Shell checks require macOS `sandbox-exec`
|
|
321
|
-
|
|
320
|
+
and active peer fixtures. Shell checks require macOS `sandbox-exec` or Linux
|
|
321
|
+
`bubblewrap`.
|
|
322
322
|
Injected host adapters may return namespaced observations and bounded evidence
|
|
323
323
|
artifacts. A case can list their IDs in `requiredEvidence`; missing or partial
|
|
324
324
|
host evidence produces unavailable grading rather than a passing task. The
|
|
@@ -15,6 +15,17 @@ marked tutorial commands in fresh consumer/contributor directories, including
|
|
|
15
15
|
expected verdicts and saved evidence. Candidate and released-package runs are
|
|
16
16
|
reported separately. These commands create and clean temporary directories.
|
|
17
17
|
|
|
18
|
+
`package.json.version` is the authoritative current version. `bun run docs:sync`
|
|
19
|
+
updates only the marked [Current release](installing.md#current-release) section
|
|
20
|
+
with its exact installation command. Keep its markers and heading in place;
|
|
21
|
+
the command refuses missing, duplicate, or reversed markers and invalid versions.
|
|
22
|
+
The rest of the page stays ordinary Markdown. `check:docs` compares this section
|
|
23
|
+
without writing; release preparation and package-install validation also reject
|
|
24
|
+
stale output. Run synchronization after a version bump and include the resulting
|
|
25
|
+
documentation change. Link installation and update instructions to this section.
|
|
26
|
+
Keep historical versions, licensing transitions, and minimum supported versions
|
|
27
|
+
fixed and explain their purpose beside each reference.
|
|
28
|
+
|
|
18
29
|
## External links
|
|
19
30
|
|
|
20
31
|
Run `bun run check:docs:external` separately. Transient failures are retried
|
package/docs/getting-started.md
CHANGED
|
@@ -7,17 +7,17 @@ This teaches the runner lifecycle, not the quality of an actual model.
|
|
|
7
7
|
|
|
8
8
|
## From an installed package
|
|
9
9
|
|
|
10
|
-
|
|
10
|
+
Prepare a new directory:
|
|
11
11
|
|
|
12
12
|
```sh
|
|
13
13
|
mkdir sevro-demo
|
|
14
14
|
cd sevro-demo
|
|
15
15
|
bun init -y
|
|
16
|
-
bun add --exact @bjoernrochel/sevro@0.1.0-rc.3
|
|
17
16
|
```
|
|
18
17
|
|
|
19
|
-
|
|
20
|
-
|
|
18
|
+
Install the [current release](installing.md#current-release) there, then return
|
|
19
|
+
here to run the example. Read [version differences](installing.md#published-package-and-checkout)
|
|
20
|
+
for the current docs, source-available license, and earlier BSL releases.
|
|
21
21
|
|
|
22
22
|
Make the local installation available on `PATH` for this shell, then call
|
|
23
23
|
`sevro` directly. Bun must also be on `PATH`.
|
package/docs/identity-v1.md
CHANGED
|
@@ -69,3 +69,8 @@ Selected [runtime configuration](runtime-v1.md) is retained under
|
|
|
69
69
|
It includes declared non-secret environment values, canonical read roots,
|
|
70
70
|
private seed digests, and hook authority. Per-role output paths are resolved
|
|
71
71
|
only during execution; runtime placeholders remain symbolic in identity.
|
|
72
|
+
Runtime roots also retain lexical aliases, metadata discovery sources,
|
|
73
|
+
and skipped optional directories. Moving or
|
|
74
|
+
changing the effective host grants changes identity, even when the portable
|
|
75
|
+
configuration file is unchanged. External installation contents remain outside
|
|
76
|
+
this digest unless explicitly captured as private seed inputs.
|
package/docs/installing.md
CHANGED
|
@@ -10,13 +10,26 @@ Linux and Windows are unverified.
|
|
|
10
10
|
mkdir sevro-demo
|
|
11
11
|
cd sevro-demo
|
|
12
12
|
bun init -y
|
|
13
|
-
bun add --exact @bjoernrochel/sevro@0.1.0-rc.3
|
|
14
13
|
```
|
|
15
14
|
|
|
15
|
+
Then install the [current release](#current-release) in that directory.
|
|
16
|
+
|
|
16
17
|
Run the [packaged deterministic example](getting-started.md#from-an-installed-package)
|
|
17
18
|
to verify installation. It needs neither model credentials nor Darrow.
|
|
18
19
|
Local installation records the selected version in the project lockfile.
|
|
19
20
|
|
|
21
|
+
<!-- sevro-current-release:start -->
|
|
22
|
+
|
|
23
|
+
## Current release
|
|
24
|
+
|
|
25
|
+
The current release is `0.1.0-rc.4`. Install this exact version:
|
|
26
|
+
|
|
27
|
+
```sh
|
|
28
|
+
bun add --exact @bjoernrochel/sevro@0.1.0-rc.4
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
<!-- sevro-current-release:end -->
|
|
32
|
+
|
|
20
33
|
## Make the command available
|
|
21
34
|
|
|
22
35
|
The examples call `sevro` from `PATH`. For a local installation, run this from
|
|
@@ -33,10 +46,11 @@ no adjustment is needed. Bun must also be on `PATH`: the executable uses
|
|
|
33
46
|
|
|
34
47
|
## Published package and checkout
|
|
35
48
|
|
|
36
|
-
The
|
|
49
|
+
The [current release](#current-release) includes runtime
|
|
37
50
|
configuration, native goals, selected plugin hooks, current onboarding, and
|
|
38
51
|
[source-available licensing terms](licensing.md). Earlier `rc.1` and `rc.2`
|
|
39
|
-
packages retain BUSL-1.1.
|
|
52
|
+
packages retain BUSL-1.1. The license transition began with `0.1.0-rc.3`.
|
|
53
|
+
Source checkouts can include changes beyond their
|
|
40
54
|
package version; compare the exact release tag when establishing provenance.
|
|
41
55
|
|
|
42
56
|
The candidate npm package contains sources, schemas, examples, documentation,
|
|
@@ -57,14 +71,10 @@ directories; it does not publish. [Contributing](../CONTRIBUTING.md) lists check
|
|
|
57
71
|
|
|
58
72
|
## Update
|
|
59
73
|
|
|
60
|
-
Review a release's license and compatibility, then
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
```
|
|
65
|
-
|
|
66
|
-
This selects `rc.3` explicitly. For later updates, substitute the intended
|
|
67
|
-
published version, then rerun the example.
|
|
74
|
+
Review a release's license and compatibility, then use the exact installation
|
|
75
|
+
command for the [current release](#current-release) from your existing project.
|
|
76
|
+
To select a different published version, substitute that exact version in the
|
|
77
|
+
command, then rerun the [packaged example](getting-started.md#from-an-installed-package).
|
|
68
78
|
Rollback uses the prior exact version and another verification run.
|
|
69
79
|
[Releases](releases.md) explains artifact provenance.
|
|
70
80
|
|
package/docs/native-hosts.md
CHANGED
|
@@ -1,7 +1,8 @@
|
|
|
1
1
|
# Run native model hosts
|
|
2
2
|
|
|
3
3
|
The basic tutorial needs none of this setup. Bundled model routes and isolated
|
|
4
|
-
shell checks require macOS `sandbox-exec
|
|
4
|
+
shell checks require macOS `sandbox-exec` or Linux `bubblewrap` (and `socat` for
|
|
5
|
+
Claude Code), a supported native CLI, and model
|
|
5
6
|
authentication. Complete commands/rules live in the [CLI reference](development-cli.md)
|
|
6
7
|
and [Codex host reference](codex-host.md).
|
|
7
8
|
|
package/docs/releases.md
CHANGED
|
@@ -7,7 +7,7 @@ exact semantic version; development versions containing `-dev` are not release
|
|
|
7
7
|
candidates. Darrow pins the published version in development dependencies and
|
|
8
8
|
its lockfile. Marketplace plugins do not depend on Sevro.
|
|
9
9
|
|
|
10
|
-
|
|
10
|
+
The [current release](installing.md#current-release) uses the current [licensing terms](licensing.md).
|
|
11
11
|
Published `rc.1` and `rc.2` retain their original BSL grants. Always select a new
|
|
12
12
|
version for publication; a local development tarball does not replace an
|
|
13
13
|
already published artifact.
|
|
@@ -38,11 +38,18 @@ metadata in the installed package.
|
|
|
38
38
|
## Maintainer workflow
|
|
39
39
|
|
|
40
40
|
1. Select the license, update `package.json` to the intended release version,
|
|
41
|
-
and commit those changes. Preserve the existing public schemas and document
|
|
41
|
+
run `bun run docs:sync`, and commit those changes. Preserve the existing public schemas and document
|
|
42
42
|
behavior changes. Set `publishConfig.tag` to `latest` for the recommended
|
|
43
43
|
default or `next` for an opt-in preview. A prerelease does not establish full
|
|
44
44
|
Darrow migration completion.
|
|
45
|
-
2.
|
|
45
|
+
2. On macOS with Bun 1.3.13, Node 24, npm, and `sandbox-exec`, run frozen
|
|
46
|
+
installation and the full quality gate:
|
|
47
|
+
|
|
48
|
+
```sh
|
|
49
|
+
bun install --frozen-lockfile
|
|
50
|
+
bun run check:typescript
|
|
51
|
+
```
|
|
52
|
+
|
|
46
53
|
3. Prepare and test the exact candidate:
|
|
47
54
|
|
|
48
55
|
```sh
|
|
@@ -64,29 +71,21 @@ metadata in the installed package.
|
|
|
64
71
|
version's evidence for updates. Roll back through a new dependency/lockfile
|
|
65
72
|
commit and rerun the installed gate; never relabel failed-version evidence.
|
|
66
73
|
|
|
67
|
-
##
|
|
68
|
-
|
|
69
|
-
`.github/workflows/release.yml` runs only through explicit dispatch against a
|
|
70
|
-
release tag. Its default prepares and checks the artifact. `publish: true`
|
|
71
|
-
additionally publishes that verified artifact using npm trusted publishing.
|
|
72
|
-
No push or pull-request event publishes a package.
|
|
74
|
+
## Local macOS publication
|
|
73
75
|
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
76
|
+
There is no GitHub release job. The maintainer prepares, verifies, and publishes
|
|
77
|
+
the reviewed tarball from macOS with an authenticated npm account. Check its
|
|
78
|
+
retained checksum immediately before publication:
|
|
77
79
|
|
|
78
80
|
```sh
|
|
79
|
-
|
|
81
|
+
cd /absolute/reviewed
|
|
82
|
+
shasum -a 256 -c SHA256SUMS
|
|
83
|
+
npm publish ./bjoernrochel-sevro-0.1.0-rc.1.tgz --ignore-scripts --access public --tag latest
|
|
80
84
|
```
|
|
81
85
|
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
The package repository URL must match this public repository. See npm's
|
|
86
|
-
[trusted publishing documentation](https://docs.npmjs.com/trusted-publishers/).
|
|
87
|
-
The workflow checks the downloaded artifact's SHA-256 before publishing and
|
|
88
|
-
retains its release record. Configuring an owner or workflow does not itself
|
|
89
|
-
authorize this coding session to publish or push.
|
|
86
|
+
Replace the example version and distribution tag with the reviewed candidate's
|
|
87
|
+
values. The package repository URL must match this public repository. Preparing
|
|
88
|
+
a tarball or opening a pull request does not authorize publication.
|
|
90
89
|
|
|
91
90
|
An existing published candidate can become the default without republishing:
|
|
92
91
|
|
|
@@ -99,7 +98,7 @@ Promotion changes the registry tag; it does not update the published tarball.
|
|
|
99
98
|
|
|
100
99
|
The first published candidate was `0.1.0-rc.1`, using the owner-selected
|
|
101
100
|
`BUSL-1.1` license. Its `LICENSE` preserves Darrow's terms and parameters,
|
|
102
|
-
changing only the Licensed Work name and description to Sevro.
|
|
103
|
-
|
|
101
|
+
changing only the Licensed Work name and description to Sevro. Local checks do
|
|
102
|
+
not claim a remote CI run or publication. Darrow's exact
|
|
104
103
|
registry pin, default caller switch, remaining workflow migrations, focused live validation, and
|
|
105
104
|
generic runner removal remain separate extraction gates.
|
package/docs/results-v1.md
CHANGED
|
@@ -116,3 +116,46 @@ reports `failed` / `not_requested` / `not_assessed` with exit code `2`. A final
|
|
|
116
116
|
message that happens to match a check cannot turn that failure into a pass.
|
|
117
117
|
Claude retains its bounded event stream and, when present, up to 64 KiB of
|
|
118
118
|
stderr as private host artifacts. Failure bodies do not appear in CLI diagnostics.
|
|
119
|
+
|
|
120
|
+
## Native transcript bundles
|
|
121
|
+
|
|
122
|
+
With [runtime transcript opt-in](runtime-v1.md#native-transcript-access), each
|
|
123
|
+
bundled host captures full native files before removing its private state.
|
|
124
|
+
Candidate artifact `sevro.native-transcripts.bundle` has format
|
|
125
|
+
`sevro.native-transcripts.v1`. It contains `host` (`codex` or `claude`), `role`,
|
|
126
|
+
native `rootSessionId` (or `null`), `completeness`, `issues`, and `files`.
|
|
127
|
+
Each file records its native relative `path`, SHA-256 `sha256`, and exact bytes
|
|
128
|
+
as `bytesBase64`. Decode the bytes without rewriting native records. Native
|
|
129
|
+
session paths, root/child records and sidecar metadata preserve associations;
|
|
130
|
+
the bundle does not normalize model, effort, timing, token, or cost statistics.
|
|
131
|
+
Existing unknown metrics remain unknown.
|
|
132
|
+
|
|
133
|
+
`sevro.host.native-transcripts` records the same capture state and file hashes
|
|
134
|
+
without raw bytes. Semantic and advisory bundles retain their role in the
|
|
135
|
+
bundle and use the existing `sevro.semantic[.<source digest>].` and
|
|
136
|
+
`sevro.advisory.` artifact ID prefixes. Run/trial artifact references supply
|
|
137
|
+
stable private file URLs and whole-bundle hashes after native home cleanup.
|
|
138
|
+
Extensions receive these references through the existing `evaluate` protocol;
|
|
139
|
+
benchmarks can use the same references and decode the native files.
|
|
140
|
+
|
|
141
|
+
Capture includes native transcript-tree sidecars and refuses descendant links,
|
|
142
|
+
special files and escaping paths. It is bounded to 4,096 directory entries,
|
|
143
|
+
5 MiB of native file bytes and 7 MiB of encoded file metadata/content. The
|
|
144
|
+
bundle remains inside the existing 8 MiB item and 32 MiB host artifact budgets;
|
|
145
|
+
it groups sessions instead of spending an artifact per child. Limits produce
|
|
146
|
+
`byte_limit`, `entry_limit`, or `bundle_limit` issues. Unreadable, unsafe or
|
|
147
|
+
changing entries, malformed/empty native records, malformed JSON sidecars,
|
|
148
|
+
an unknown/missing original session, or incomplete host
|
|
149
|
+
execution produce explicit partial/unavailable capture. `complete` means the
|
|
150
|
+
identified root's tree was captured within those bounds after completed host
|
|
151
|
+
execution and generic syntax/basic record validation. JSONL must contain
|
|
152
|
+
nonempty JSON object records with a nonempty string `type`; JSON sidecars must
|
|
153
|
+
parse as JSON. UTF-8 decoding is strict. One trailing newline and CRLF are
|
|
154
|
+
accepted; blank interior records, invalid encoding and truncated JSON are
|
|
155
|
+
malformed. Unknown native fields and record types remain valid. Issues
|
|
156
|
+
`empty_transcript`, `malformed_transcript` and `malformed_sidecar` downgrade
|
|
157
|
+
capture without rewriting or discarding retained bytes. This validates basic
|
|
158
|
+
structure, not native semantic correctness.
|
|
159
|
+
An empty/missing native JSONL tree is `unavailable`. Failure, cancellation and
|
|
160
|
+
timeout retain available native bytes with incomplete capture state. Raw
|
|
161
|
+
bundles and views are private evidence and must stay out of Git.
|
package/docs/runtime-v1.md
CHANGED
|
@@ -53,12 +53,27 @@ shell startup overrides, subprocess injection flags, and `UV_OFFLINE` are refuse
|
|
|
53
53
|
An inherited or explicit `PATH` must contain nonempty absolute entries. Missing
|
|
54
54
|
entries are inert. Sevro grants read-only access to existing directories,
|
|
55
55
|
resolved symlinked tools, and bounded installation prefixes above `bin`.
|
|
56
|
+
An inaccessible individual symlink target is inert; an unreadable PATH directory
|
|
57
|
+
still fails configuration. Explicitly declared read roots must remain readable.
|
|
56
58
|
It does not widen a `bin` grant to the whole home, `/usr`, `/usr/local`, or
|
|
57
59
|
`/opt/homebrew`. Direct versioned mise paths can reuse their existing Node or
|
|
58
60
|
Python installations. A mise shim may also need explicitly declared mise data
|
|
59
61
|
and configuration; inheriting `PATH` does not change the shim's own discovery
|
|
60
62
|
rules. Fixture binaries keep precedence over declared host tools.
|
|
61
63
|
|
|
64
|
+
When `PATH` is declared, Sevro also queries available installation metadata
|
|
65
|
+
before execution. Homebrew's `--prefix` and `--cellar` identify its existing
|
|
66
|
+
`Cellar`, `opt`, and `etc` support trees on macOS and Linux. The whole Homebrew
|
|
67
|
+
prefix is not granted. On macOS, selected Apple tool shims (including Git,
|
|
68
|
+
Python, and compiler tools) use `xcode-select --print-path` to locate their
|
|
69
|
+
developer support tree. A declared `DEVELOPER_DIR` participates in that query;
|
|
70
|
+
an unrelated ambient override does not. Installation locations are not assumed.
|
|
71
|
+
Each query has a three-second deadline and an 8 KiB combined output limit.
|
|
72
|
+
Absent providers and the known no-active-Apple-installation result add no roots.
|
|
73
|
+
Other query failures, invalid metadata, or a missing declared developer override
|
|
74
|
+
fail before execution. Diagnostics identify the provider and operation without
|
|
75
|
+
retaining arbitrary command output.
|
|
76
|
+
|
|
62
77
|
Declare supporting directories outside `PATH` with `filesystem.readOnlyRoots`:
|
|
63
78
|
|
|
64
79
|
```json
|
|
@@ -77,6 +92,43 @@ the real home are allowed only through bounded runtime grants. Source and
|
|
|
77
92
|
configuration worktrees, evaluator inputs, credentials, results, run ownership
|
|
78
93
|
state, and peer fixtures remain protected. A conflicting or broad grant fails.
|
|
79
94
|
|
|
95
|
+
Use `filesystem.optionalReadOnlyRoots` for supporting directories that may not
|
|
96
|
+
be installed on every host, with the same path rules as `readOnlyRoots`:
|
|
97
|
+
|
|
98
|
+
```json
|
|
99
|
+
{
|
|
100
|
+
"format": "sevro.runtime.v1",
|
|
101
|
+
"filesystem": {
|
|
102
|
+
"optionalReadOnlyRoots": [
|
|
103
|
+
"../public-python-installations",
|
|
104
|
+
"../public-tool-cache"
|
|
105
|
+
]
|
|
106
|
+
}
|
|
107
|
+
}
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
With `sevro.json` at the repository root, these paths name sibling support
|
|
111
|
+
directories outside the protected repository. Keep them bounded to public tool
|
|
112
|
+
data; directories inside the repository remain protected even when absent.
|
|
113
|
+
|
|
114
|
+
Only absent directories are skipped. Invalid variables or paths, unreadable
|
|
115
|
+
directories, regular files, and protected overlaps remain errors. Required roots
|
|
116
|
+
still fail when absent. Effective roots retain both lexical directory paths and
|
|
117
|
+
canonical targets, and both are checked against protected inputs. Native aliases
|
|
118
|
+
inside granted support trees (for example, `opt/package` pointing into `Cellar`)
|
|
119
|
+
remain usable. Codex may canonicalize a standalone symlink-root permission under
|
|
120
|
+
protected home, denying its lexical lookup; declare the intended bounded parent
|
|
121
|
+
support tree and its target rather than relying on that standalone alias.
|
|
122
|
+
|
|
123
|
+
On Linux, Codex starts candidate commands from an empty filesystem with bounded
|
|
124
|
+
read mounts. Sevro keeps a normal home ancestor denied by that empty-root policy
|
|
125
|
+
rather than adding a native deny mount that would hide its read-only descendants.
|
|
126
|
+
Protected source and credential descendants remain explicitly denied, and no
|
|
127
|
+
home ancestor receives content access. If the operator's home lies inside a
|
|
128
|
+
minimal system read tree such as `/usr` or `/etc`, Sevro retains its explicit deny;
|
|
129
|
+
the native preflight may refuse that overlapping setup rather than expose home
|
|
130
|
+
contents.
|
|
131
|
+
|
|
80
132
|
## Private writable state
|
|
81
133
|
|
|
82
134
|
`runtime.seedDirectories` copies prepared public dependencies into a private
|
|
@@ -113,7 +165,7 @@ the policy does not itself prove that a goal was created or completed.
|
|
|
113
165
|
|
|
114
166
|
`hooks.plugins` authorizes all command hooks from the named prepared plugins.
|
|
115
167
|
Sevro copies immutable mounts, filters unselected hooks, and wraps selected
|
|
116
|
-
handlers in a credential-free macOS sandbox. It covers default hook files,
|
|
168
|
+
handlers in a credential-free macOS or Linux sandbox. It covers default hook files,
|
|
117
169
|
manifest path/inline declarations in both host formats, and Markdown skill or
|
|
118
170
|
agent frontmatter. Unsupported handler types and exec-form arguments fail
|
|
119
171
|
instead of being skipped. The wrapper denies network access and protected
|
|
@@ -129,6 +181,61 @@ cases on their explicit project-settings route with hooks disabled. Unrelated
|
|
|
129
181
|
user settings and hooks are not imported. Sevro does not use Claude's `--bare`
|
|
130
182
|
mode, which would change authentication and behavior.
|
|
131
183
|
|
|
184
|
+
## Native transcript access
|
|
185
|
+
|
|
186
|
+
Set top-level `"nativeTranscripts": true` to enable live, read-only access to
|
|
187
|
+
the current role's full native root and subagent transcript tree. Absent or
|
|
188
|
+
`false` leaves this access and capture disabled. The selection is snapshotted
|
|
189
|
+
with the runtime policy and enters configuration identity; temporary native
|
|
190
|
+
homes and transcript views do not enter that identity. Codex `exec` persists
|
|
191
|
+
sessions when enabled, including runs without continuation.
|
|
192
|
+
|
|
193
|
+
Commands discover their own tree through `SEVRO_NATIVE_TRANSCRIPT_ROOT`.
|
|
194
|
+
Codex commands also receive runner-owned `CODEX_HOME`; native
|
|
195
|
+
`CODEX_THREAD_ID` selects `sessions/**/rollout-*-<thread>.jsonl`. Read the latest
|
|
196
|
+
native `turn_context` for that session's `model` and `effort`. Claude shell
|
|
197
|
+
commands receive a credential-free `CLAUDE_CONFIG_DIR` view whose `projects/`
|
|
198
|
+
tree contains `projects/*/<CLAUDE_CODE_SESSION_ID>.jsonl` and subagent files.
|
|
199
|
+
The latest native assistant `message.model` records the observed model.
|
|
200
|
+
Claude supplies `CLAUDE_CODE_SESSION_ID` and `CLAUDE_EFFORT` itself; Sevro does
|
|
201
|
+
not replace these with the root's requested route. Requested CLI effort is
|
|
202
|
+
configuration, and is not proof of an effective child effort. Missing native
|
|
203
|
+
route evidence stays unknown.
|
|
204
|
+
|
|
205
|
+
Observed with Claude Code 2.1.284: a native Agent's Bash may receive the parent's
|
|
206
|
+
`CLAUDE_CODE_SESSION_ID` and no `CLAUDE_EFFORT`. In that situation the environment
|
|
207
|
+
pair selects the parent transcript. Analyze the child's native JSONL and
|
|
208
|
+
`.meta.json` sidecar to establish its own model/session association; the
|
|
209
|
+
parent's model is not the child's observed model, and missing effort remains
|
|
210
|
+
unknown. Sevro preserves these native values without synthesizing a child
|
|
211
|
+
context or copying the root's requested effort into the child.
|
|
212
|
+
|
|
213
|
+
Native hosts keep their credential-bearing homes. Claude's host writes through
|
|
214
|
+
a runner-created, canonical-bound `projects` link into the isolated source;
|
|
215
|
+
runner-owned Bash and zsh startup files supply the shell compatibility view.
|
|
216
|
+
These sources and startup files sit outside candidate-writable state. Commands
|
|
217
|
+
cannot modify transcripts or read credential/settings siblings, evaluator
|
|
218
|
+
inputs, or peer role/trial state. A fixed private namespace is protected before
|
|
219
|
+
commands start, including peers created later. Pre-existing namespaces must be
|
|
220
|
+
owned private directories; links and unsafe permissions fail. Ordinary
|
|
221
|
+
filesystem declarations cannot authorize access to this namespace.
|
|
222
|
+
Namespace read roots and seed sources, including declaration symlinks into it,
|
|
223
|
+
are refused before private trees are hashed or copied.
|
|
224
|
+
On Linux, Codex keeps this namespace hidden through its empty-root policy and
|
|
225
|
+
mounts only runner-owned session/helper trees and retained views. It avoids an
|
|
226
|
+
ancestor deny mount that would mask these narrower read-only grants. Ordinary
|
|
227
|
+
declarations cannot select this exception; parent content, credentials, and
|
|
228
|
+
present or future peer state receive no grants.
|
|
229
|
+
|
|
230
|
+
After candidate execution, isolated shell checks and semantic/advisory hosts
|
|
231
|
+
receive `SEVRO_CANDIDATE_TRANSCRIPTS`, a separate read-only retained view with
|
|
232
|
+
`index.json` and `codex/` or `claude/` native relative file paths. This does not
|
|
233
|
+
replace a grading host's own native home or live tree. Trusted injected adapters
|
|
234
|
+
receive the same view through request `candidateTranscriptRoot`. The temporary
|
|
235
|
+
view is removed after grading; stable retained artifact references remain in
|
|
236
|
+
run evidence. Extensions and external benchmarks consume the
|
|
237
|
+
[native transcript bundles](results-v1.md#native-transcript-bundles).
|
|
238
|
+
|
|
132
239
|
## Evidence and compatibility
|
|
133
240
|
|
|
134
241
|
`configuration.redacted.runtimePolicy` retains the selected environment,
|
|
@@ -139,6 +246,12 @@ execution uses per-role paths. This binds declared setup and seed contents;
|
|
|
139
246
|
it does not claim that every external installation file or OS dependency is
|
|
140
247
|
immutable.
|
|
141
248
|
|
|
249
|
+
The retained policy also records metadata providers, their selected executables
|
|
250
|
+
and effective support roots, plus skipped optional paths. These values and the
|
|
251
|
+
actual effective roots enter identity; private execution paths and external tool
|
|
252
|
+
file contents do not. The same discovery and skip evidence reaches
|
|
253
|
+
`sevro.host.runtime` observations.
|
|
254
|
+
|
|
142
255
|
Bundled hosts supply `sevro.host.runtime` with the applied policy digest, role,
|
|
143
256
|
environment names, read roots, and seed slots. `sevro.host.hooks` records source
|
|
144
257
|
and effective plugin digests plus observed executions and their exit codes.
|
|
@@ -158,3 +271,5 @@ Programmatic consumers can import `RuntimeConfiguration`, `RuntimePolicy`, and
|
|
|
158
271
|
`loadRuntimeConfiguration` from `src/engine.ts`, then pass the resolved policy
|
|
159
272
|
to `runEvaluation` or a bundled host request. They must provide the same
|
|
160
273
|
protected-source declarations required by their execution route.
|
|
274
|
+
`loadRuntimeConfiguration` accepts those canonical protected roots as its third
|
|
275
|
+
argument, so provider queries themselves cannot run from protected source trees.
|