@bjoernrochel/sevro 0.1.0-rc.2 → 0.1.0-rc.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CONTRIBUTING.md +108 -0
- package/LICENSE +63 -136
- package/README.md +78 -66
- package/docs/README.md +40 -0
- package/docs/acknowledgements.md +14 -0
- package/docs/architecture.md +25 -0
- package/docs/assets/README.md +27 -0
- package/docs/assets/sevro-logo.png +0 -0
- package/docs/assets/sevro-symbol.svg +1 -0
- package/docs/assets/sevro-wordmark.svg +1 -0
- package/docs/builtin-graders-v1.md +12 -9
- package/docs/codex-host.md +15 -4
- package/docs/contracts.md +18 -0
- package/docs/creating-cases.md +132 -0
- package/docs/development-cli.md +23 -16
- package/docs/documentation-quality.md +40 -0
- package/docs/evidence.md +17 -0
- package/docs/extension-protocol-v1.md +24 -13
- package/docs/getting-started.md +81 -0
- package/docs/guide-evaluation.md +85 -0
- package/docs/guide-verification.json +239 -0
- package/docs/identity-v1.md +6 -0
- package/docs/installing.md +78 -0
- package/docs/licenses/BUSL-1.1-0.1.0-rc.2.txt +159 -0
- package/docs/licensing.md +42 -0
- package/docs/native-hosts.md +38 -0
- package/docs/reading-results.md +39 -0
- package/docs/releases.md +25 -8
- package/docs/running-evaluations.md +31 -0
- package/docs/runtime-v1.md +160 -0
- package/docs/specs/repository-guide.md +58 -0
- package/docs/troubleshooting.md +30 -0
- package/docs/typescript-quality.md +160 -0
- package/examples/basic/host.ts +2 -2
- package/package.json +38 -6
- package/schemas/runtime-v1.schema.json +84 -0
- package/src/advisory-fixture.ts +185 -128
- package/src/advisory.ts +8 -3
- package/src/cli-invocation.ts +592 -0
- package/src/cli.ts +472 -652
- package/src/engine.ts +11 -2373
- package/src/evaluation-fixture.ts +315 -0
- package/src/evaluation-plan.ts +1615 -0
- package/src/evaluation-run.ts +1565 -0
- package/src/evaluation-types.ts +147 -0
- package/src/extension-client.ts +251 -109
- package/src/extension-session.ts +295 -147
- package/src/fixture-bin.ts +24 -54
- package/src/fixture-setup.ts +177 -98
- package/src/fixture-tools.ts +88 -0
- package/src/generated/cli-result.cjs +2 -0
- package/src/generated/cli-result.d.cts +3 -0
- package/src/generated/extension.cjs +2 -0
- package/src/generated/extension.d.cts +3 -0
- package/src/generated/report.cjs +2 -0
- package/src/generated/report.d.cts +3 -0
- package/src/generated/run-evidence.cjs +2 -0
- package/src/generated/run-evidence.d.cts +3 -0
- package/src/generated/runtime.cjs +2 -0
- package/src/generated/runtime.d.cts +3 -0
- package/src/generated-fixture.ts +172 -92
- package/src/git-hooks.ts +11 -50
- package/src/graders/artifact.ts +56 -23
- package/src/graders/git-head.ts +98 -49
- package/src/graders/output.ts +222 -140
- package/src/graders/semantic.ts +102 -43
- package/src/graders/shell.ts +360 -185
- package/src/hosts/claude-continuation.ts +63 -18
- package/src/hosts/claude-credential.ts +72 -39
- package/src/hosts/claude-events.ts +82 -44
- package/src/hosts/claude-native-goal.ts +57 -33
- package/src/hosts/claude-nested-skills.ts +107 -50
- package/src/hosts/claude-process.ts +87 -0
- package/src/hosts/claude-repository-invocation.ts +222 -105
- package/src/hosts/claude-settings.ts +80 -18
- package/src/hosts/claude-tool-calls.ts +171 -101
- package/src/hosts/claude.ts +645 -402
- package/src/hosts/codex-app-server.ts +294 -170
- package/src/hosts/codex-command-output.ts +246 -131
- package/src/hosts/codex-configuration.ts +44 -18
- package/src/hosts/codex-events.ts +94 -55
- package/src/hosts/codex-executor-command.ts +155 -59
- package/src/hosts/codex-invocation.ts +70 -0
- package/src/hosts/codex-marketplace.ts +275 -0
- package/src/hosts/codex-native-calls.ts +628 -473
- package/src/hosts/codex-process.ts +101 -0
- package/src/hosts/codex-profile.ts +81 -23
- package/src/hosts/codex-session-files.ts +114 -0
- package/src/hosts/codex-skill-reads.ts +460 -242
- package/src/hosts/codex.ts +903 -799
- package/src/hosts/isolation-roots.ts +33 -16
- package/src/hosts/mac-sandbox.ts +122 -31
- package/src/hosts/native-controls.ts +93 -53
- package/src/hosts/protected-worktrees.ts +34 -18
- package/src/hosts/workspace-fingerprint.ts +55 -35
- package/src/identity.ts +96 -45
- package/src/instrumentation.ts +109 -48
- package/src/preparation.ts +182 -123
- package/src/provenance.ts +111 -67
- package/src/report.ts +110 -105
- package/src/repository-fixture.ts +193 -80
- package/src/resolved-case.ts +33 -0
- package/src/results.ts +119 -68
- package/src/run-owner.ts +153 -62
- package/src/runtime-config.ts +222 -0
- package/src/runtime-hook-processes.ts +97 -0
- package/src/runtime-hooks.ts +349 -0
- package/src/runtime-paths.ts +202 -0
- package/src/runtime-seeds.ts +68 -0
- package/src/runtime-state.ts +119 -0
- package/src/schema-types.ts +286 -0
- package/src/schema.ts +15 -19
- package/src/trial-scheduler.ts +3 -1
- package/src/value-guards.ts +14 -0
package/CONTRIBUTING.md
ADDED
|
@@ -0,0 +1,108 @@
|
|
|
1
|
+
# Contributing to Sevro
|
|
2
|
+
|
|
3
|
+
Contributions are welcome: runner fixes, extension examples, tests, documentation,
|
|
4
|
+
and useful bug reports. Agent guidance lives in `AGENTS.md` in a source checkout.
|
|
5
|
+
The [public contracts](docs/contracts.md) define the behavior compatibility boundary.
|
|
6
|
+
|
|
7
|
+
## Contribution licensing
|
|
8
|
+
|
|
9
|
+
By intentionally submitting a contribution for inclusion, you confirm that you
|
|
10
|
+
created it or have permission to license it, including employer-owned work.
|
|
11
|
+
Identify third-party material and its license separately; do not grant rights
|
|
12
|
+
you lack. You make your contribution available under [Sevro's public license](LICENSE).
|
|
13
|
+
|
|
14
|
+
In addition, you grant Björn Rochel a worldwide, perpetual, non-exclusive,
|
|
15
|
+
irrevocable, royalty-free copyright license to use, reproduce, modify, publicly
|
|
16
|
+
display, publicly perform, distribute, and license or sublicense the contribution
|
|
17
|
+
and its derivatives under other terms, including open-source and commercial
|
|
18
|
+
licenses. You retain copyright. This grants permission, not ownership.
|
|
19
|
+
|
|
20
|
+
Other recipients receive the public terms unless separately licensed. This is
|
|
21
|
+
not an Apache-2.0 grant to every recipient. The additional grant lets the
|
|
22
|
+
maintainer offer alternatives without obtaining new copyright permission from
|
|
23
|
+
each contributor. If you cannot agree, raise that before submitting work for
|
|
24
|
+
inclusion. Contributions are provided without warranties to the extent permitted
|
|
25
|
+
by applicable law.
|
|
26
|
+
|
|
27
|
+
## Set up development
|
|
28
|
+
|
|
29
|
+
Use Bun 1.3.13, Git, Node 24, and npm. Full suite isolation needs macOS and
|
|
30
|
+
`sandbox-exec`; Linux/Windows acceptance is unverified. From a fresh checkout:
|
|
31
|
+
|
|
32
|
+
```sh
|
|
33
|
+
bun install --frozen-lockfile
|
|
34
|
+
bun run test:docs-examples
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
The second command checks the tutorial in fresh consumer/contributor directories.
|
|
38
|
+
Native model trials need their own [credentials and prerequisites](docs/native-hosts.md).
|
|
39
|
+
|
|
40
|
+
## Quality contract and checks
|
|
41
|
+
|
|
42
|
+
[The TypeScript quality contract](docs/typescript-quality.md) defines the checked
|
|
43
|
+
source inventory, type-aware lint, strict typing, focused functions, reproducible
|
|
44
|
+
properties, and independent statement and branch coverage requirements.
|
|
45
|
+
Run the complete acceptance command:
|
|
46
|
+
|
|
47
|
+
```sh
|
|
48
|
+
bun run check:typescript
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
It performs frozen installation, schema freshness, formatting, lint, typing,
|
|
52
|
+
documentation and guide dry-run checks, tutorial examples, the real Bun suite,
|
|
53
|
+
coverage integrity and thresholds, and installed-package validation. Statements
|
|
54
|
+
and branches must each reach at least 95% using raw exact counts over the complete
|
|
55
|
+
authored production inventory, including unimported files. The five explicitly
|
|
56
|
+
inventoried generated AJV validators are excluded from coverage because this
|
|
57
|
+
target measures authored runner behavior; their syntax, schema freshness, typed
|
|
58
|
+
boundaries and runtime tests remain checked. Tests and fixtures remain linted and
|
|
59
|
+
typed; they do not enter the production coverage denominator. New or changed
|
|
60
|
+
source dispositions require review.
|
|
61
|
+
|
|
62
|
+
Before committing, use `bun run check:typescript --fast` after installing the
|
|
63
|
+
locked dependencies. This runs inventory, schema, formatting, lint, typing and
|
|
64
|
+
documentation checks; it does not replace full acceptance. CI retains machine-readable coverage
|
|
65
|
+
reports under `.quality/` and exposes the stable `TypeScript quality` status.
|
|
66
|
+
|
|
67
|
+
Run `bun run check:docs:external` separately and [triage network failures](docs/documentation-quality.md#external-links).
|
|
68
|
+
Guide changes also need [native evals](docs/guide-evaluation.md).
|
|
69
|
+
Documentation checks supplement #1 rather than replacing its contract.
|
|
70
|
+
|
|
71
|
+
## Tests, fixtures, and schemas
|
|
72
|
+
|
|
73
|
+
Preserve real Bun filesystem/process integration tests and macOS isolation.
|
|
74
|
+
Use deterministic fixtures under `tests/fixtures/`; keep evaluator-only inputs
|
|
75
|
+
outside candidate workspaces. Keep credentials and raw trial evidence out of Git.
|
|
76
|
+
Add meaningful regression coverage for reproducible bugs. Fixtures should
|
|
77
|
+
establish observable behavior, not mirror implementation.
|
|
78
|
+
|
|
79
|
+
After changing a schema, regenerate its checked-in validators:
|
|
80
|
+
|
|
81
|
+
```sh
|
|
82
|
+
bun run schemas:generate
|
|
83
|
+
bun run schemas:check
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
Commit schemas and validators together. Public v1 schemas and extension
|
|
87
|
+
capabilities are compatibility boundaries. Document changed meaning, validate
|
|
88
|
+
external inputs, and keep missing evidence from becoming a pass.
|
|
89
|
+
|
|
90
|
+
## Documentation and pull requests
|
|
91
|
+
|
|
92
|
+
Give each page one reader need: tutorial, how-to, reference, or explanation.
|
|
93
|
+
Maintain the [hub](docs/README.md), old destinations, prerequisites, expected
|
|
94
|
+
results, and limitations. Preserve historical observations as evidence rather
|
|
95
|
+
than rewriting them to describe current behavior.
|
|
96
|
+
|
|
97
|
+
Use imperative Conventional Commit subjects without trailing periods. Do not
|
|
98
|
+
add `Co-authored-by` or AI attribution trailers. Do not change author metadata
|
|
99
|
+
without explicit instruction. A PR explains the problem, changed behavior,
|
|
100
|
+
compatibility limits, and checks actually performed. Do not claim unexecuted
|
|
101
|
+
native trials or remote CI results.
|
|
102
|
+
|
|
103
|
+
## Maintainer releases
|
|
104
|
+
|
|
105
|
+
Follow [releases](docs/releases.md): choose a new version, prepare and inspect
|
|
106
|
+
the exact tarball, verify its license/provenance, and validate that artifact.
|
|
107
|
+
Publication, tags, registry changes, and alternative licensing are separate
|
|
108
|
+
maintainer actions; preparing a contribution does not authorize them.
|
package/LICENSE
CHANGED
|
@@ -1,159 +1,86 @@
|
|
|
1
|
-
|
|
1
|
+
Sevro Source Available License 1.0
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
"Business Source License" is a trademark of MariaDB Corporation Ab.
|
|
3
|
+
Copyright (c) 2026 Björn Rochel
|
|
5
4
|
|
|
6
|
-
|
|
5
|
+
1. Definitions
|
|
7
6
|
|
|
8
|
-
|
|
7
|
+
"Software" means Sevro source code, executable forms, documentation, and
|
|
8
|
+
modifications or derivative works of those materials, except material
|
|
9
|
+
explicitly identified as having separate terms. Logo/branding permissions
|
|
10
|
+
are separate and stated in docs/assets/README.md. Third-party material
|
|
11
|
+
retains its identified license.
|
|
9
12
|
|
|
10
|
-
|
|
13
|
+
"You" means the person or legal entity exercising these rights.
|
|
11
14
|
|
|
12
|
-
|
|
13
|
-
The Licensed Work is (c) 2026 Björn Rochel.
|
|
15
|
+
2. Free use and modification
|
|
14
16
|
|
|
15
|
-
|
|
16
|
-
|
|
17
|
+
Subject to this license, you may use, copy, modify, and create derivative
|
|
18
|
+
works of the Software without a license fee, for any purpose. This includes
|
|
19
|
+
commercial and non-commercial work, paid education, research, evaluation,
|
|
20
|
+
consulting, and organizational use.
|
|
17
21
|
|
|
18
|
-
|
|
19
|
-
purpose, commercial or non-commercial, provided
|
|
20
|
-
that such use is not Commercial Educational
|
|
21
|
-
Production Use;
|
|
22
|
+
3. Free redistribution and origin
|
|
22
23
|
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
24
|
+
You may redistribute the Software in source or executable form, including
|
|
25
|
+
modified versions, without charge and under this license. Include this
|
|
26
|
+
license, retain applicable copyright/attribution/origin notices, identify
|
|
27
|
+
Sevro as the source of covered code, and prominently identify modifications.
|
|
28
|
+
Do not claim another person's contribution was authored by you or imply
|
|
29
|
+
that a modified version is an official Sevro release.
|
|
27
30
|
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
publishing blog posts, articles, or other
|
|
32
|
-
educational content, provided that such
|
|
33
|
-
production use is not Commercial Educational
|
|
34
|
-
Production Use;
|
|
31
|
+
These conditions also apply to covered code in combined distributions.
|
|
32
|
+
Independently authored material containing no covered Software may have
|
|
33
|
+
separate terms that neither restrict these rights nor circumvent section 4.
|
|
35
34
|
|
|
36
|
-
|
|
37
|
-
projects, provided that such use is not
|
|
38
|
-
Commercial Educational Production Use;
|
|
35
|
+
4. No sale or paid hosted access under this public license
|
|
39
36
|
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
functionality of the Licensed Work.
|
|
37
|
+
You may not charge directly or indirectly for copies of the Software, a
|
|
38
|
+
license to it, or access to its functionality as a hosted or managed service.
|
|
39
|
+
This includes modified, derivative, renamed, and rebranded versions. Bundling
|
|
40
|
+
or relabeling does not avoid this condition, including when the principal
|
|
41
|
+
value of a paid offering is access to the Software's functionality.
|
|
46
42
|
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
43
|
+
You may charge for independent professional services, training, research,
|
|
44
|
+
analysis, or other work performed using the Software, provided the payment
|
|
45
|
+
is not for copies, licensing, or hosted access to its functionality. Calling
|
|
46
|
+
a charge a hosting, subscription, service, or support fee does not permit
|
|
47
|
+
charging for otherwise restricted access.
|
|
52
48
|
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
49
|
+
Restricted distribution or hosted access requires a separate license from
|
|
50
|
+
the relevant rights holder. Contact Björn Rochel at bjoern@bjro.de for
|
|
51
|
+
alternative terms. This public license grants no general right to relicense
|
|
52
|
+
covered Software under other terms.
|
|
56
53
|
|
|
57
|
-
|
|
58
|
-
or promote, a paid product, service, membership,
|
|
59
|
-
or engagement; or
|
|
54
|
+
5. Contributions and separate grants
|
|
60
55
|
|
|
61
|
-
|
|
62
|
-
|
|
56
|
+
Contributions intentionally submitted for inclusion are governed by the
|
|
57
|
+
terms in CONTRIBUTING.md, including the additional copyright license to
|
|
58
|
+
Björn Rochel. Contributors retain copyright. Additional/alternative grants
|
|
59
|
+
apply only to their stated recipients and terms, not to every recipient.
|
|
63
60
|
|
|
64
|
-
|
|
65
|
-
constitute Commercial Educational Production Use.
|
|
66
|
-
Commercial Educational Production Use is not included
|
|
67
|
-
in this Additional Use Grant.
|
|
61
|
+
6. Independent inputs, outputs, and branding
|
|
68
62
|
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
63
|
+
This license does not by itself apply to independently authored cases,
|
|
64
|
+
datasets, extensions, prompts, or outputs merely because Sevro processes
|
|
65
|
+
them. Material copying or deriving from covered Software remains subject
|
|
66
|
+
to its applicable terms. Other material/input licenses still apply.
|
|
72
67
|
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
themselves offered free of charge under this License,
|
|
76
|
-
are permitted.
|
|
68
|
+
This software license grants no trademark or logo rights. Separate project
|
|
69
|
+
identification permissions appear in docs/assets/README.md.
|
|
77
70
|
|
|
78
|
-
|
|
79
|
-
version of the Licensed Work is first made publicly
|
|
80
|
-
available. The Change Date is determined separately
|
|
81
|
-
for each released version.
|
|
71
|
+
7. Termination
|
|
82
72
|
|
|
83
|
-
|
|
73
|
+
Your permissions terminate if you violate these conditions. Other recipients'
|
|
74
|
+
compliant permissions are not terminated by your violation. The relevant
|
|
75
|
+
rights holder may reinstate permissions.
|
|
84
76
|
|
|
85
|
-
|
|
86
|
-
please contact: bjoern@bjro.de
|
|
77
|
+
8. Disclaimer and liability
|
|
87
78
|
|
|
88
|
-
|
|
79
|
+
To the fullest extent permitted by applicable law, the Software is provided
|
|
80
|
+
"as is", without express or implied warranties, including merchantability,
|
|
81
|
+
fitness for a particular purpose, title, or non-infringement. No author,
|
|
82
|
+
contributor, or rights holder is liable for damages or other liability
|
|
83
|
+
arising from the Software or its use, in contract, tort, or otherwise.
|
|
89
84
|
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
Business Source License 1.1
|
|
93
|
-
|
|
94
|
-
Terms
|
|
95
|
-
|
|
96
|
-
The Licensor hereby grants you the right to copy, modify, create derivative
|
|
97
|
-
works, redistribute, and make non-production use of the Licensed Work. The
|
|
98
|
-
Licensor may make an Additional Use Grant, above, permitting limited
|
|
99
|
-
production use.
|
|
100
|
-
|
|
101
|
-
Effective on the Change Date, or the fourth anniversary of the first publicly
|
|
102
|
-
available distribution of a specific version of the Licensed Work under this
|
|
103
|
-
License, whichever comes first, the Licensor hereby grants you rights under
|
|
104
|
-
the terms of the Change License, and the rights granted in the paragraph
|
|
105
|
-
above terminate.
|
|
106
|
-
|
|
107
|
-
If your use of the Licensed Work does not comply with the requirements
|
|
108
|
-
currently in effect as described in this License, you must purchase a
|
|
109
|
-
commercial license from the Licensor, its affiliated entities, or authorized
|
|
110
|
-
resellers, or you must refrain from using the Licensed Work.
|
|
111
|
-
|
|
112
|
-
All copies of the original and modified Licensed Work, and derivative works
|
|
113
|
-
of the Licensed Work, are subject to this License. This License applies
|
|
114
|
-
separately for each version of the Licensed Work and the Change Date may vary
|
|
115
|
-
for each version of the Licensed Work released by Licensor.
|
|
116
|
-
|
|
117
|
-
You must conspicuously display this License on each original or modified copy
|
|
118
|
-
of the Licensed Work. If you receive the Licensed Work in original or
|
|
119
|
-
modified form from a third party, the terms and conditions set forth in this
|
|
120
|
-
License apply to your use of that work.
|
|
121
|
-
|
|
122
|
-
Any use of the Licensed Work in violation of this License will automatically
|
|
123
|
-
terminate your rights under this License for the current and all other
|
|
124
|
-
versions of the Licensed Work.
|
|
125
|
-
|
|
126
|
-
This License does not grant you any right in any trademark or logo of
|
|
127
|
-
Licensor or its affiliates (provided that you may use a trademark or logo of
|
|
128
|
-
Licensor as expressly required by this License).
|
|
129
|
-
|
|
130
|
-
TO THE EXTENT PERMITTED BY APPLICABLE LAW, THE LICENSED WORK IS PROVIDED ON
|
|
131
|
-
AN "AS IS" BASIS. LICENSOR HEREBY DISCLAIMS ALL WARRANTIES AND CONDITIONS,
|
|
132
|
-
EXPRESS OR IMPLIED, INCLUDING (WITHOUT LIMITATION) WARRANTIES OF
|
|
133
|
-
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NON-INFRINGEMENT, AND
|
|
134
|
-
TITLE.
|
|
135
|
-
|
|
136
|
-
MariaDB hereby grants you permission to use this License's text to license
|
|
137
|
-
your works, and to refer to it using the trademark "Business Source License",
|
|
138
|
-
as long as you comply with the Covenants of Licensor below.
|
|
139
|
-
|
|
140
|
-
Covenants of Licensor
|
|
141
|
-
|
|
142
|
-
In consideration of the right to use this License's text and the "Business
|
|
143
|
-
Source License" name and trademark, Licensor covenants to MariaDB, and to all
|
|
144
|
-
other recipients of the licensed work to be provided by Licensor:
|
|
145
|
-
|
|
146
|
-
1. To specify as the Change License the GPL Version 2.0 or any later version,
|
|
147
|
-
or a license that is compatible with GPL Version 2.0 or a later version,
|
|
148
|
-
where "compatible" means that software provided under the Change License
|
|
149
|
-
can be included in a program with software provided under GPL Version 2.0
|
|
150
|
-
or a later version. Licensor may specify additional Change Licenses
|
|
151
|
-
without limitation.
|
|
152
|
-
|
|
153
|
-
2. To either: (a) specify an additional grant of rights to use that does not
|
|
154
|
-
impose any additional restriction on the right granted in this License, as
|
|
155
|
-
the Additional Use Grant; or (b) insert the text "None".
|
|
156
|
-
|
|
157
|
-
3. To specify a Change Date.
|
|
158
|
-
|
|
159
|
-
4. Not to modify this License in any other way.
|
|
85
|
+
This is a source-available license, not an OSI open-source license. It has
|
|
86
|
+
no automatic change date. Earlier releases retain their original grants.
|
package/README.md
CHANGED
|
@@ -1,72 +1,84 @@
|
|
|
1
1
|
# Sevro
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
response per extension process and negotiates the v1 protocol. It enforces
|
|
32
|
-
message limits, timeouts, response identity, and process-group cancellation.
|
|
33
|
-
|
|
34
|
-
[`extension-session.ts`](src/extension-session.ts) runs the negotiated
|
|
35
|
-
`resolve`, `prepare`, and `evaluate` methods and checks source stability and
|
|
36
|
-
evidence references. The engine accepts additive extension checks for inline
|
|
37
|
-
cases.
|
|
38
|
-
|
|
39
|
-
[`results.ts`](src/results.ts) reduces checks to separate execution, grading,
|
|
40
|
-
and task states, applies case thresholds, and maps the aggregate to a CLI exit
|
|
41
|
-
category.
|
|
42
|
-
[`identity.ts`](src/identity.ts) computes versioned comparison identities from
|
|
43
|
-
canonical JSON. The [identity contract](docs/identity-v1.md) defines the input
|
|
44
|
-
dimensions.
|
|
45
|
-
|
|
46
|
-
The [built-in output graders](docs/builtin-graders-v1.md) evaluate regex, JSON,
|
|
47
|
-
and inline schema assertions on bounded final-message observations.
|
|
48
|
-
|
|
49
|
-
The [first engine slice](docs/engine-slice.md) executes resolved cases through
|
|
50
|
-
an injected host adapter and retains trial evidence before fixture cleanup.
|
|
51
|
-
The [advisory review fixture](docs/advisory-fixture.md) builds a separate Git
|
|
52
|
-
view of the complete candidate change while withholding evaluator paths.
|
|
53
|
-
The [development CLI](docs/development-cli.md) runs that path with a trusted
|
|
54
|
-
adapter module or the bundled Codex and Claude Code routes and emits a
|
|
55
|
-
machine-readable result.
|
|
56
|
-
The [native control observation](docs/native-controls.md) exposes bounded tool
|
|
57
|
-
labels for extension policy on both Codex and Claude, with explicit completeness.
|
|
58
|
-
|
|
59
|
-
The packaged [basic example](examples/basic/) runs without Darrow or a model
|
|
60
|
-
account. From this checkout, run the graded case with:
|
|
3
|
+
<p align="center">
|
|
4
|
+
<img src="docs/assets/sevro-logo.png" alt="Sevro logo: a black S-shaped track with red segments above the sevro wordmark" width="360">
|
|
5
|
+
</p>
|
|
6
|
+
|
|
7
|
+
Sevro runs repeatable evaluations of coding agents. Give it a case, a host
|
|
8
|
+
adapter, and success criteria; it retains execution, grading results, and
|
|
9
|
+
evidence. Projects own their benchmark cases and policy through Sevro's
|
|
10
|
+
[extension boundary](docs/extension-protocol-v1.md).
|
|
11
|
+
|
|
12
|
+
Sevro grew out of [Darrow](https://github.com/BjRo/darrow), but works without a
|
|
13
|
+
Darrow checkout. Try [your first evaluation](docs/getting-started.md): no model
|
|
14
|
+
account is required.
|
|
15
|
+
|
|
16
|
+
## Status and environments
|
|
17
|
+
|
|
18
|
+
Sevro is under active development. The npm package is `@bjoernrochel/sevro`;
|
|
19
|
+
its command is `sevro`. Release candidate `0.1.0-rc.3` includes runtime
|
|
20
|
+
configuration, native goals, selected plugin hooks, and the current licensing terms.
|
|
21
|
+
Earlier `rc.1` and `rc.2` packages retain their original BSL grants.
|
|
22
|
+
See [installation and release differences](docs/installing.md).
|
|
23
|
+
|
|
24
|
+
Use Bun 1.3.13. The full deterministic suite and native isolation target macOS;
|
|
25
|
+
bundled Codex/Claude routes and isolated shell checks need `sandbox-exec`.
|
|
26
|
+
Contributor package/release checks also need Node 24 and npm.
|
|
27
|
+
Linux and Windows support is unverified. The basic example uses an injected
|
|
28
|
+
deterministic adapter and needs no native model credentials.
|
|
29
|
+
|
|
30
|
+
## Install and get a first result
|
|
61
31
|
|
|
62
32
|
```sh
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
--condition passive --trials 1 --threshold 1
|
|
33
|
+
mkdir sevro-demo
|
|
34
|
+
cd sevro-demo
|
|
35
|
+
bun init -y
|
|
36
|
+
bun add --exact @bjoernrochel/sevro@0.1.0-rc.3
|
|
68
37
|
```
|
|
69
38
|
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
39
|
+
Continue with [your first evaluation](docs/getting-started.md): the exact command,
|
|
40
|
+
expected `passed` result, and retained evidence. Read [installation](docs/installing.md)
|
|
41
|
+
for updates, removal, and source checkouts.
|
|
42
|
+
|
|
43
|
+
## Find your next task
|
|
44
|
+
|
|
45
|
+
The [documentation hub](docs/README.md) routes to running evaluations, authoring
|
|
46
|
+
cases and extensions, reading results, troubleshooting, and architecture.
|
|
47
|
+
Public v1 contracts are the [extension protocol](docs/extension-protocol-v1.md),
|
|
48
|
+
[results and evidence](docs/results-v1.md), and [report](docs/report-v1.md).
|
|
49
|
+
Machine-readable contracts live in [schemas/](schemas/).
|
|
50
|
+
|
|
51
|
+
## Ask the repository guide
|
|
52
|
+
|
|
53
|
+
Open this checkout in a fresh Codex or Claude Code session and ask
|
|
54
|
+
“What is Sevro, and how do I run my first evaluation?” Explicit invocation is
|
|
55
|
+
`$sevro-guide` in Codex or `/sevro-guide` in Claude Code.
|
|
56
|
+
The guide reads sources, cites its answers, and handles follow-ups. It explains
|
|
57
|
+
commands; execution requires a separate request. See the
|
|
58
|
+
[guide contract and verification limits](docs/specs/repository-guide.md).
|
|
59
|
+
The npm package does not install the repository guide.
|
|
60
|
+
|
|
61
|
+
## Contribute
|
|
62
|
+
|
|
63
|
+
[CONTRIBUTING.md](CONTRIBUTING.md) covers setup, checks, fixtures, schema
|
|
64
|
+
regeneration, compatibility, and releases. [Documentation quality](docs/documentation-quality.md)
|
|
65
|
+
defines documentation checks and review.
|
|
66
|
+
|
|
67
|
+
Run `bun run check:typescript` for the complete quality gate and
|
|
68
|
+
`bun run check:typescript --fast` before committing. The
|
|
69
|
+
[TypeScript quality contract](docs/typescript-quality.md) defines the source
|
|
70
|
+
inventory, strict typing, property tests, coverage integrity, and supported
|
|
71
|
+
environment.
|
|
72
|
+
|
|
73
|
+
## License and acknowledgements
|
|
74
|
+
|
|
75
|
+
Release candidate `0.1.0-rc.3` uses the [Sevro Source Available License 1.0](LICENSE).
|
|
76
|
+
Use is free in every context, including commercial work and paid education.
|
|
77
|
+
Selling copies, rebranded versions, or hosted access to Sevro's functionality
|
|
78
|
+
requires a separate license. Modification and free redistribution remain allowed.
|
|
79
|
+
It is source-available, not an OSI open-source license. Read
|
|
80
|
+
[licensing](docs/licensing.md) for contribution grants and earlier BSL releases.
|
|
81
|
+
|
|
82
|
+
Björn Rochel created the logo. Sevro's name is inspired by the character in
|
|
83
|
+
Pierce Brown's _Red Rising_. Thank you, Pierce Brown, for the world and characters
|
|
84
|
+
that inspired both Sevro and Darrow. See [acknowledgements](docs/acknowledgements.md).
|
package/docs/README.md
ADDED
|
@@ -0,0 +1,40 @@
|
|
|
1
|
+
# Sevro documentation
|
|
2
|
+
|
|
3
|
+
<img src="assets/sevro-wordmark.svg" alt="Sevro wordmark" width="220">
|
|
4
|
+
|
|
5
|
+
Start with the task you want to complete. These pages work without an agent.
|
|
6
|
+
|
|
7
|
+
| I want to… | Read |
|
|
8
|
+
| --------------------------------------------- | -------------------------------------------------------- |
|
|
9
|
+
| Get a result without a model account | [Your first evaluation](getting-started.md) |
|
|
10
|
+
| Install, update, remove, or use a checkout | [Installation](installing.md) |
|
|
11
|
+
| Run evaluations and select a host | [Running evaluations](running-evaluations.md) |
|
|
12
|
+
| Create cases and reusable policy | [Cases and extensions](creating-cases.md) |
|
|
13
|
+
| Interpret verdicts, evidence, and comparisons | [Reading results](reading-results.md) |
|
|
14
|
+
| Run Codex or Claude with isolation | [Native hosts](native-hosts.md) |
|
|
15
|
+
| Reuse host tools or enable goals and hooks | [Runtime configuration](runtime-v1.md) |
|
|
16
|
+
| Resolve a symptom | [Troubleshooting](troubleshooting.md) |
|
|
17
|
+
| Understand runner responsibilities | [Architecture](architecture.md) |
|
|
18
|
+
| Change Sevro or its documentation | [Contributing](../CONTRIBUTING.md) |
|
|
19
|
+
| Understand licensing and branding | [Licensing](licensing.md) and [assets](assets/README.md) |
|
|
20
|
+
|
|
21
|
+
## Ask the guide
|
|
22
|
+
|
|
23
|
+
Ask ordinary Sevro questions in a fresh repository session, or use `$sevro-guide`
|
|
24
|
+
in Codex and `/sevro-guide` in Claude Code. The
|
|
25
|
+
canonical guide at `.agents/skills/sevro-guide/SKILL.md` reads evidence and
|
|
26
|
+
explains it; it does not run installations, tests, or evaluations.
|
|
27
|
+
It belongs to this repository, not an npm installation.
|
|
28
|
+
|
|
29
|
+
## Sources and authority
|
|
30
|
+
|
|
31
|
+
[Current contracts](contracts.md) define interfaces. The
|
|
32
|
+
[guide specification](specs/repository-guide.md) defines the repository assistant.
|
|
33
|
+
[Historical validation](evidence.md) records observations, not current guarantees.
|
|
34
|
+
There is no accepted Rust migration decision here;
|
|
35
|
+
[issue #2](https://github.com/BjRo/sevro/issues/2) tracks research.
|
|
36
|
+
|
|
37
|
+
Read [documentation quality](documentation-quality.md),
|
|
38
|
+
[guide evaluation](guide-evaluation.md), [releases](releases.md), and
|
|
39
|
+
[acknowledgements](acknowledgements.md) for maintenance.
|
|
40
|
+
Return to the [Sevro introduction](../README.md).
|
|
@@ -0,0 +1,14 @@
|
|
|
1
|
+
# Acknowledgements
|
|
2
|
+
|
|
3
|
+
Sevro is named after the character in [Pierce Brown's _Red Rising_ series](https://www.piercebrown.com/redrisingsaga).
|
|
4
|
+
Thank you, Pierce Brown, for the world and characters that inspired both Sevro
|
|
5
|
+
and Darrow. This acknowledges inspiration, not ownership of the fictional
|
|
6
|
+
character or affiliation with the author.
|
|
7
|
+
|
|
8
|
+
Björn Rochel created the Sevro logo. Its black/red track motif gives the runner
|
|
9
|
+
its own visual identity alongside [Darrow](https://github.com/BjRo/darrow).
|
|
10
|
+
Read [brand assets and permitted uses](assets/README.md).
|
|
11
|
+
|
|
12
|
+
Sevro's standalone runner grew out of Darrow's evaluation work. Darrow remains
|
|
13
|
+
an integration and source of ideas; the deterministic example works independently.
|
|
14
|
+
Return to the [documentation hub](README.md).
|
|
@@ -0,0 +1,25 @@
|
|
|
1
|
+
# How Sevro fits together
|
|
2
|
+
|
|
3
|
+
Sevro owns generic execution/evidence contracts. Projects own cases and policy.
|
|
4
|
+
This lets Darrow and other integrations use the same runner without importing
|
|
5
|
+
their benchmark policy into it.
|
|
6
|
+
|
|
7
|
+
| Responsibility | Sources |
|
|
8
|
+
| ------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ |
|
|
9
|
+
| Parse commands/select explicit routes | [CLI](../src/cli.ts), [CLI reference](development-cli.md) |
|
|
10
|
+
| Prepare, execute, grade, retain, clean up | [Engine](../src/engine.ts), [engine slice](engine-slice.md) |
|
|
11
|
+
| Negotiate trusted extension calls | [Client](../src/extension-client.ts), [session](../src/extension-session.ts), [protocol](extension-protocol-v1.md) |
|
|
12
|
+
| Run native hosts and retain bounded observations | [Codex](codex-host.md), [native controls](native-controls.md), [host sources](../src/hosts/) |
|
|
13
|
+
| Reduce execution/grading/task states | [Result source](../src/results.ts), [contract](results-v1.md) |
|
|
14
|
+
| Bind comparison identities and release provenance | [Identity](identity-v1.md), [releases](releases.md) |
|
|
15
|
+
| Summarize retained results | [Report](report-v1.md) |
|
|
16
|
+
|
|
17
|
+
Completed evidence is retained before cleanup. Missing observations remain
|
|
18
|
+
unavailable; unknown measurements do not become zero. Extensions cannot
|
|
19
|
+
silently replace isolation/persistence or built-in policy. The
|
|
20
|
+
[advisory fixture](advisory-fixture.md) gives review a separate Git view with
|
|
21
|
+
evaluator inputs withheld.
|
|
22
|
+
|
|
23
|
+
These paths describe this checkout. [Historical validation](evidence.md) records
|
|
24
|
+
bounded past observations. [Rust research](https://github.com/BjRo/sevro/issues/2)
|
|
25
|
+
does not establish a migration; public interfaces remain useful independently.
|
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
# Sevro brand assets
|
|
2
|
+
|
|
3
|
+
Björn Rochel created the supplied logo in 2026. Its original artwork and white
|
|
4
|
+
background are preserved in `sevro-logo.png`.
|
|
5
|
+
|
|
6
|
+
- [Full logo](sevro-logo.png): supplied 1254 × 1254 PNG including the wordmark.
|
|
7
|
+
- [Compact symbol](sevro-symbol.svg): the same artwork framed around the S track.
|
|
8
|
+
- [Wordmark](sevro-wordmark.svg): the same artwork framed around the name.
|
|
9
|
+
|
|
10
|
+
SVG variants embed the supplied PNG; they are not traced vector redrawings.
|
|
11
|
+
Keep a white backing on dark pages; no transparent variant is claimed. Use the
|
|
12
|
+
full lockup for introductions, the symbol for compact navigation, and the
|
|
13
|
+
wordmark for wide layouts. Preserve colors, proportions, and a readable size.
|
|
14
|
+
The black/red track motif relates to Darrow while giving Sevro its own identity.
|
|
15
|
+
|
|
16
|
+
## Ownership and identification
|
|
17
|
+
|
|
18
|
+
Logo artwork: © 2026 Björn Rochel. Name/logo rights are reserved separately
|
|
19
|
+
from software licensing. You may display the unmodified logo or supplied
|
|
20
|
+
variants to identify or link to this project. Preserve the credit when
|
|
21
|
+
redistributing assets, do not imply endorsement, and do not present this
|
|
22
|
+
branding as your own product identity. Other uses require permission from
|
|
23
|
+
Björn Rochel.
|
|
24
|
+
|
|
25
|
+
The name is inspired by Pierce Brown's _Red Rising_. This artwork credit does
|
|
26
|
+
not assert ownership of his fictional names or characters. See
|
|
27
|
+
[acknowledgements](../acknowledgements.md) and [licensing](../licensing.md).
|
|
Binary file
|