@hraness/sys1 0.17.0 → 0.18.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +155 -590
- package/dist/cli.js +2944 -1047
- package/dist/gateway.d.ts +1 -1
- package/dist/index.js +1 -1
- package/dist/packs/core/fixtures/calibration.jsonl +2 -2
- package/dist/packs/core/fixtures/heldout.jsonl +2 -2
- package/dist/packs/core/pack.yaml +18 -14
- package/package.json +21 -5
package/README.md
CHANGED
|
@@ -1,677 +1,242 @@
|
|
|
1
1
|
# Sys1
|
|
2
2
|
|
|
3
|
-
Sys1
|
|
4
|
-
|
|
5
|
-
|
|
3
|
+
Sys1 gives coding agents tools to review code, check completion claims, and get
|
|
4
|
+
structured answers from Jev or a local model. Jev is TypeSafe’s hosted decision
|
|
5
|
+
model. Project skills work with Codex, Claude Code, and Devin; applications can
|
|
6
|
+
use the same tools through a CLI, Node/Bun client, or HTTP API.
|
|
6
7
|
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
[Jev](https://docs.typesafe.ai/models), a local model on your machine, or a
|
|
10
|
-
compatible server you run.
|
|
8
|
+
Latest release: v0.18.0. Install the GitHub release with npm and run it with
|
|
9
|
+
Bun 1.3.14 or newer. MIT licensed.
|
|
11
10
|
|
|
12
|
-
|
|
13
|
-
local daemon that serves the Jev-compatible `POST /v1/systemone` API.
|
|
11
|
+
[Get started](#install) · [Agent skills](https://sys1.io/skills) · [Documentation](https://sys1.io/docs) · [Releases](https://github.com/hraness/sys1/releases)
|
|
14
12
|
|
|
15
|
-
|
|
16
|
-
on Bun 1.3.14 or newer.
|
|
13
|
+
## Choose what helps your workflow
|
|
17
14
|
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
## Choose a workflow
|
|
21
|
-
|
|
22
|
-
| Task | Start here | What you get |
|
|
15
|
+
| You want to | Use | What you receive |
|
|
23
16
|
| --- | --- | --- |
|
|
24
|
-
|
|
|
25
|
-
| Check a
|
|
26
|
-
|
|
|
27
|
-
|
|
|
28
|
-
|
|
|
29
|
-
|
|
30
|
-
The review and verification skills work in Git repositories with Codex,
|
|
31
|
-
Claude Code, or Devin; other agents can use the same CLI. Their workflows do
|
|
32
|
-
not depend on a specific framework or hosting provider. The bundled review
|
|
33
|
-
rules cover two narrow JavaScript and TypeScript changes: newly empty catch
|
|
34
|
-
blocks and removed test assertions. Add and evaluate your own rules for other
|
|
35
|
-
conventions or languages.
|
|
17
|
+
| Save a check and continue its review later | [`sys1 workflow`](#save-a-check-and-continue-its-review) | A saved execution record, a private command log, and input checks before resume. |
|
|
18
|
+
| Check a change against a repository rule | [`sys1 review`](#review-changes-with-your-agent) | Candidates tied to the rule and diff, with feedback and reuse of unchanged reviews. |
|
|
19
|
+
| Check a proposed completion message | [`sys1 verify`](#check-an-agents-completion-message) | A comparison with Git state, linked pull requests, and live pages. |
|
|
20
|
+
| Add a decision to application code | [Client and HTTP API](#use-as-a-module) | Yes/no, choice, or score answers with probabilities and validated response shapes. |
|
|
21
|
+
| Keep long test logs out of the agent’s context | [System One Skills](https://sys1.io/skills#compact-checks), a separate package | A short result, the command’s exit status, and the full log saved locally. No model or API key. |
|
|
22
|
+
|
|
36
23
|
|
|
37
24
|
## Install
|
|
38
25
|
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
publishing. `--allow-scripts=node-llama-cpp` lets only the pinned native
|
|
42
|
-
inference package run its install script. The installed `sys1` command runs
|
|
43
|
-
with Bun.
|
|
26
|
+
Install [Bun](https://bun.sh/docs/installation) 1.3.14 or newer and have npm
|
|
27
|
+
available, then run:
|
|
44
28
|
|
|
45
29
|
```sh
|
|
46
30
|
npm install --global --allow-scripts=node-llama-cpp \
|
|
47
|
-
https://github.com/hraness/sys1/releases/download/v0.
|
|
48
|
-
sys1
|
|
49
|
-
```
|
|
50
|
-
|
|
51
|
-
To build the current source instead:
|
|
52
|
-
|
|
53
|
-
```sh
|
|
54
|
-
git clone https://github.com/hraness/sys1.git
|
|
55
|
-
cd sys1
|
|
56
|
-
bun install
|
|
57
|
-
bun run build:dist
|
|
58
|
-
ln -sf "$PWD/dist/cli.js" ~/.local/bin/sys1
|
|
31
|
+
https://github.com/hraness/sys1/releases/download/v0.18.0/hraness-sys1-0.18.0.tgz
|
|
32
|
+
sys1 --version
|
|
59
33
|
```
|
|
60
34
|
|
|
61
|
-
|
|
35
|
+
The version command prints the installed release number. The release supports macOS, Linux,
|
|
36
|
+
and Windows. `--allow-scripts=node-llama-cpp` allows the optional local inference
|
|
37
|
+
runtime’s install script; installation downloads no model weights and enables
|
|
38
|
+
no hosted backend. The release includes a SHA-256 checksum.
|
|
62
39
|
|
|
63
|
-
|
|
64
|
-
candidate, record feedback, and check its completion message. It installs
|
|
65
|
-
instructions in your repository, without activating a model or adding hooks.
|
|
40
|
+
## Save a check and continue its review
|
|
66
41
|
|
|
67
|
-
|
|
68
|
-
your environment, or choose another configured backend. Install the skill and
|
|
69
|
-
preview selected files before sending source to that backend:
|
|
42
|
+
From a Git worktree on macOS or Linux, run your repository’s check command:
|
|
70
43
|
|
|
71
44
|
```sh
|
|
72
|
-
sys1
|
|
73
|
-
sys1
|
|
74
|
-
--max-requests 10 --dry-run --json -- src test
|
|
45
|
+
sys1 workflow check -- bun test
|
|
46
|
+
sys1 workflow list
|
|
75
47
|
```
|
|
76
48
|
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
49
|
+
Sys1 saves the result and command output locally. This check needs no model or
|
|
50
|
+
API key. After configuring a backend, `sys1 workflow review` can run the check
|
|
51
|
+
and then review the selected changes. You can pause after the check and resume
|
|
52
|
+
with the same inputs; uncertain interrupted steps are never silently repeated.
|
|
53
|
+
ALGAL handles execution and saved state inside Sys1.
|
|
82
54
|
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
skipped evidence and incomplete coverage.
|
|
55
|
+
The [workflow guide](docs/workflows.md) covers scoped reviews, resume, private
|
|
56
|
+
logs, and inspecting a saved run. Keep running fresh required checks before
|
|
57
|
+
delivery.
|
|
87
58
|
|
|
88
|
-
##
|
|
59
|
+
## Review changes with your agent
|
|
89
60
|
|
|
90
|
-
|
|
91
|
-
|
|
61
|
+
Run these commands inside the Git repository you are working on. They install
|
|
62
|
+
project instructions and preview a review without calling a model:
|
|
92
63
|
|
|
93
64
|
```sh
|
|
94
|
-
sys1
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
Use `setup claude-code` or `setup devin` for those agents. The installed
|
|
98
|
-
instructions tell the agent to save its proposed final message and compare
|
|
99
|
-
its claims with the repository, linked pull requests, and live pages. Save the
|
|
100
|
-
draft outside the Git worktree so it does not appear as an uncommitted change.
|
|
101
|
-
You can also run the command directly:
|
|
102
|
-
|
|
103
|
-
```sh
|
|
104
|
-
sys1 verify --message /tmp/final-message.txt --model typesafe/jev-1.13.0
|
|
105
|
-
```
|
|
106
|
-
|
|
107
|
-
Use `--message -` for piped text and `--url https://example.com` to include a
|
|
108
|
-
live page the message does not link. Without `--message`, verify reads the
|
|
109
|
-
newest matching local Devin session, including that turn's check-command
|
|
110
|
-
results. File and stdin input work with any agent; check-command results are
|
|
111
|
-
available only through Devin transcript discovery. Contradicted claims exit 7;
|
|
112
|
-
missing evidence is unverifiable. A clean report does not prove task completion.
|
|
113
|
-
See [the verification guide](docs/verify.md) for evidence and limits.
|
|
114
|
-
|
|
115
|
-
## Use as a module
|
|
116
|
-
|
|
117
|
-
For a Node 24 or Bun application that calls a running gateway, install the
|
|
118
|
-
release package without the optional native runtime:
|
|
119
|
-
|
|
120
|
-
```sh
|
|
121
|
-
npm install --omit=optional \
|
|
122
|
-
https://github.com/hraness/sys1/releases/download/v0.17.0/hraness-sys1-0.17.0.tgz
|
|
123
|
-
```
|
|
124
|
-
|
|
125
|
-
```ts
|
|
126
|
-
import { createClient } from "@hraness/sys1/client";
|
|
127
|
-
|
|
128
|
-
const sys1 = createClient(); // http://127.0.0.1:13900
|
|
129
|
-
const { response, metadata } = await sys1.evaluate({
|
|
130
|
-
state: "The build failed after a dependency upgrade.",
|
|
131
|
-
questions: {
|
|
132
|
-
action: {
|
|
133
|
-
type: "choice",
|
|
134
|
-
criteria: { repair: "Fix the build", continue: "Continue work" },
|
|
135
|
-
},
|
|
136
|
-
},
|
|
137
|
-
}, { signal: AbortSignal.timeout(5_000) });
|
|
138
|
-
|
|
139
|
-
console.log(response.answers.action, metadata.backend);
|
|
65
|
+
sys1 review setup codex
|
|
66
|
+
sys1 review checkpoint --worktree --model typesafe/jev-1.13.0 \
|
|
67
|
+
--max-requests 10 --dry-run --json -- src test
|
|
140
68
|
```
|
|
141
69
|
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
Import schemas and request/response types from the same `/client` entry point.
|
|
70
|
+
For Claude Code, use `setup claude-code`; for Devin, use `setup devin`. Replace
|
|
71
|
+
`src test` with the paths you changed. The preview lists selected files, rules,
|
|
72
|
+
skipped evidence, and the number of requests. If no requests are planned, inspect the skipped
|
|
73
|
+
evidence, selected paths, and active rules. Setup adds no automatic hooks and preserves
|
|
74
|
+
existing instructions.
|
|
148
75
|
|
|
149
|
-
|
|
76
|
+
The bundled rules cover newly empty catch blocks and removed test assertions
|
|
77
|
+
in JavaScript and TypeScript. Add [your repository’s rules](docs/review.md#draft-a-repository-rule)
|
|
78
|
+
for conventions that need judgment. For an exact syntax pattern, use a linter.
|
|
79
|
+
Review and completion checks are experimental and advisory: investigate
|
|
80
|
+
findings and keep the repository’s normal tests and review.
|
|
150
81
|
|
|
151
|
-
|
|
152
|
-
import { createRouter, DEFAULT_CONFIG } from "@hraness/sys1";
|
|
153
|
-
|
|
154
|
-
const router = createRouter({
|
|
155
|
-
config: DEFAULT_CONFIG,
|
|
156
|
-
env: process.env,
|
|
157
|
-
home: "/absolute/path/to/sys1-state", // previously installed models
|
|
158
|
-
});
|
|
159
|
-
try {
|
|
160
|
-
const result = await router.evaluate({
|
|
161
|
-
state: "All required checks passed.",
|
|
162
|
-
questions: { ready: { type: "noul", instructions: "Are the checks passing?" } },
|
|
163
|
-
});
|
|
164
|
-
console.log(result.response.answers.ready);
|
|
165
|
-
} finally {
|
|
166
|
-
await router.dispose();
|
|
167
|
-
}
|
|
168
|
-
```
|
|
82
|
+
### Add hosted Jev
|
|
169
83
|
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
the `/client` entry point is portable to Node. Keep one router per application,
|
|
173
|
-
not one per request. The root package also exposes lower-level routing and
|
|
174
|
-
model-management APIs; applications should normally use `createClient` or
|
|
175
|
-
`createRouter`.
|
|
176
|
-
|
|
177
|
-
### Adopting Sys1 in an existing Jev application
|
|
178
|
-
|
|
179
|
-
Keep domain questions, deterministic fallback, action authorization, and quality
|
|
180
|
-
thresholds in the application. Put endpoint configuration, transport, routing,
|
|
181
|
-
response validation, and local engine lifecycle behind Sys1. Existing HTTP
|
|
182
|
-
clients in other languages can use the same daemon without a JavaScript module.
|
|
183
|
-
|
|
184
|
-
Use `model: "auto"` or omit `model` to use the configured routing policy and
|
|
185
|
-
selected local model. A hardcoded `jev-1.13.0` remains a model pin and cannot
|
|
186
|
-
select an unrelated local model.
|
|
187
|
-
Local calls need no hosted API key; hosted activation stays explicit. A remote
|
|
188
|
-
server's loopback address points to that server. A browser running on the user's
|
|
189
|
-
machine can address local services, so Sys1's network listener rejects browser
|
|
190
|
-
origins and Fetch Metadata site headers, requires a loopback request authority,
|
|
191
|
-
and accepts decision POSTs only as `application/json`.
|
|
192
|
-
|
|
193
|
-
Start with an opt-in, non-authoritative pilot. Compare decisions on the
|
|
194
|
-
application's representative fixtures and record backend/adapter identity,
|
|
195
|
-
latency, errors, abstentions, and disagreement with the current decision path.
|
|
196
|
-
Do not reuse Jev probability thresholds for generic GGUF output
|
|
197
|
-
without model-specific evidence. A local-only policy also constrains explicit
|
|
198
|
-
pins; a pin never bypasses the policy. Broad production adoption requires the
|
|
199
|
-
consumer's own quality and operational acceptance, not just wire compatibility.
|
|
200
|
-
|
|
201
|
-
## Quickstart: experimental local decisions
|
|
84
|
+
Get a key from [TypeSafe](https://console.typesafe.ai/) and provide it as
|
|
85
|
+
`TYPESAFE_API_KEY` through your shell or secret manager. Then enable Jev:
|
|
202
86
|
|
|
203
87
|
```sh
|
|
204
|
-
sys1
|
|
205
|
-
sys1
|
|
206
|
-
sys1 status
|
|
88
|
+
sys1 jev enable
|
|
89
|
+
sys1 jev status
|
|
207
90
|
```
|
|
208
91
|
|
|
209
|
-
|
|
210
|
-
|
|
211
|
-
|
|
212
|
-
|
|
92
|
+
The key stays in the environment. Enabling Jev selects `hosted-only` routing,
|
|
93
|
+
so a failed hosted request cannot silently use a local model. Selected source
|
|
94
|
+
and diff context go to Jev; inspect the preview before sending them. Provider
|
|
95
|
+
usage is billed by TypeSafe at its [published rates](https://docs.typesafe.ai/models).
|
|
213
96
|
|
|
214
|
-
|
|
215
|
-
persists that choice as `local.model`, regardless of system memory. Inspect
|
|
216
|
-
without changing anything:
|
|
97
|
+
Run the previewed check by removing `--dry-run`:
|
|
217
98
|
|
|
218
99
|
```sh
|
|
219
|
-
sys1
|
|
100
|
+
sys1 review checkpoint --worktree --model typesafe/jev-1.13.0 \
|
|
101
|
+
--max-requests 10 --json -- src test
|
|
220
102
|
```
|
|
221
103
|
|
|
222
|
-
|
|
223
|
-
|
|
224
|
-
|
|
225
|
-
|
|
226
|
-
|
|
104
|
+
No background gateway is needed for this hosted workflow. Investigate each
|
|
105
|
+
candidate and record it as useful, incorrect, or unverifiable. Complete,
|
|
106
|
+
unchanged batches reuse their review for up to 24 hours. The [review guide](docs/review.md)
|
|
107
|
+
covers feedback, rechecks, and rule drafts; [`sys1 audit`](docs/audit.md) runs a
|
|
108
|
+
check without saved review history.
|
|
227
109
|
|
|
228
|
-
|
|
229
|
-
| --- | --- |
|
|
230
|
-
| macOS ARM64 | Metal (the pinned runtime's only automatic selection) |
|
|
231
|
-
| macOS x64 | CPU |
|
|
232
|
-
| Linux x64 | CUDA, Vulkan, then CPU |
|
|
233
|
-
| Linux ARM64 | CPU |
|
|
234
|
-
| Windows x64 | CUDA, Vulkan, then CPU |
|
|
235
|
-
| Windows ARM64 | CPU |
|
|
110
|
+
## Check an agent’s completion message
|
|
236
111
|
|
|
237
|
-
|
|
238
|
-
release artifact and package smoke are exercised on Ubuntu, macOS, and Windows.
|
|
239
|
-
This matrix does not prove every OS/architecture pair above; run
|
|
240
|
-
`sys1 doctor` on the actual host before use.
|
|
241
|
-
|
|
242
|
-
Send a decision:
|
|
112
|
+
After setting up a backend, install the standalone verification instructions:
|
|
243
113
|
|
|
244
114
|
```sh
|
|
245
|
-
sys1
|
|
246
|
-
{
|
|
247
|
-
"state": "Help! My payouts have been failing for 3 days.",
|
|
248
|
-
"questions": {
|
|
249
|
-
"urgent": {
|
|
250
|
-
"type": "noul",
|
|
251
|
-
"instructions": "Does this need immediate attention?",
|
|
252
|
-
"criteria": {
|
|
253
|
-
"true": "A customer-impacting incident is ongoing",
|
|
254
|
-
"false": "This can wait for normal triage"
|
|
255
|
-
}
|
|
256
|
-
}
|
|
257
|
-
}
|
|
258
|
-
}
|
|
259
|
-
EOF
|
|
115
|
+
sys1 verify setup codex
|
|
260
116
|
```
|
|
261
117
|
|
|
262
|
-
|
|
263
|
-
|
|
264
|
-
|
|
265
|
-
|
|
266
|
-
Hosted Jev is disabled by default, even if `TYPESAFE_API_KEY` is already set in
|
|
267
|
-
the environment. Add it explicitly:
|
|
118
|
+
Use `setup claude-code` or `setup devin` for those agents. Ask the agent to save
|
|
119
|
+
its proposed final message outside the Git worktree, then compare the draft
|
|
120
|
+
with available evidence:
|
|
268
121
|
|
|
269
122
|
```sh
|
|
270
|
-
|
|
271
|
-
|
|
272
|
-
sys1 jev status
|
|
123
|
+
sys1 verify --message /tmp/final-message.txt \
|
|
124
|
+
--model typesafe/jev-1.13.0 --dry-run --json
|
|
273
125
|
```
|
|
274
126
|
|
|
275
|
-
|
|
276
|
-
|
|
277
|
-
|
|
278
|
-
|
|
279
|
-
|
|
280
|
-
|
|
281
|
-
sys1 jev disable
|
|
282
|
-
```
|
|
127
|
+
Choose an equivalent external file path on Windows. The preview reports the
|
|
128
|
+
input type and model route without model calls or page fetches. Read the message
|
|
129
|
+
file and inspect its links before rerunning without `--dry-run`; the message and
|
|
130
|
+
any fetched page excerpts go to the selected backend. Use `--message -` for
|
|
131
|
+
piped text and `--url https://example.com` to include a live page the message
|
|
132
|
+
does not link.
|
|
283
133
|
|
|
284
|
-
|
|
285
|
-
|
|
286
|
-
|
|
287
|
-
|
|
288
|
-
the installed model named by `local.model`. Disabling Jev returns a hosted-only
|
|
289
|
-
configuration to `auto` for the selected local model. Installing additional
|
|
290
|
-
models does not change the selection. If no eligible route is available, Sys1
|
|
291
|
-
reports an error instead of silently choosing another installed model.
|
|
134
|
+
Contradictions exit 7; unavailable evidence is marked unverifiable. A clean
|
|
135
|
+
report does not prove completion. File input works with any coding agent;
|
|
136
|
+
check-command evidence requires local Devin transcript discovery. See the
|
|
137
|
+
[verification guide](docs/verify.md) for input modes, evidence, and exit codes.
|
|
292
138
|
|
|
293
|
-
##
|
|
139
|
+
## Use as a module
|
|
294
140
|
|
|
295
|
-
|
|
296
|
-
`~/.sys1/models` (or `$SYS1_HOME/models`). Downloads stream to a temporary
|
|
297
|
-
file, enforce an 8 GiB ceiling, verify SHA-256, run bounded GGUF structural
|
|
298
|
-
validation, and only then atomically enter the model store. Manifest filenames
|
|
299
|
-
cannot escape the store, symbolic-link weights are not admitted, and the daemon
|
|
300
|
-
never downloads weights implicitly.
|
|
141
|
+
For an application using Node 24 or Bun, install the portable client:
|
|
301
142
|
|
|
302
143
|
```sh
|
|
303
|
-
|
|
304
|
-
sys1
|
|
305
|
-
sys1 model list
|
|
306
|
-
sys1 model verify qwen3-1.7b
|
|
144
|
+
npm install --omit=optional \
|
|
145
|
+
https://github.com/hraness/sys1/releases/download/v0.18.0/hraness-sys1-0.18.0.tgz
|
|
307
146
|
```
|
|
308
147
|
|
|
309
|
-
|
|
310
|
-
|
|
311
|
-
| Model | Kind | Download | Role |
|
|
312
|
-
| --- | --- | ---: | --- |
|
|
313
|
-
| `qwen3-1.7b` | `gguf` | 1.03 GiB | default local experiment |
|
|
314
|
-
| `qwen3-0.6b` | `gguf` | 365 MiB | diagnostic model; explicit selection only |
|
|
315
|
-
| `qwen3.5-4b` | `gguf` | 2.55 GiB | larger candidate; explicit selection only |
|
|
316
|
-
|
|
317
|
-
All entries are pinned to the publisher's Hugging Face LFS SHA-256. Weight
|
|
318
|
-
licenses and terms remain those of their publishers; weights are not included
|
|
319
|
-
in the Sys1 package.
|
|
320
|
-
|
|
321
|
-
`sys1 pull` defaults to Qwen3 1.7B and only installs an artifact; it does not
|
|
322
|
-
change `local.model`. To change the local route, install the model first, then
|
|
323
|
-
select its installed ID with `sys1 config set local.model MODEL`. Other
|
|
324
|
-
installed models require a request-level model pin. This keeps a new download
|
|
325
|
-
from becoming an unintended fallback.
|
|
326
|
-
|
|
327
|
-
Older inventories containing removed CUA-S1 or Needle artifacts fail closed.
|
|
328
|
-
Sys1 preserves those files and the manifest; use a new `SYS1_HOME` for the
|
|
329
|
-
current GGUF store.
|
|
330
|
-
|
|
331
|
-
### Generic GGUF adapter
|
|
332
|
-
|
|
333
|
-
For builtin GGUF models, Sys1 renders a bounded question prompt, evaluates
|
|
334
|
-
the full first-token vocabulary distribution with llama.cpp, and sums
|
|
335
|
-
probability mass over constrained answer labels. Choice and score use unique
|
|
336
|
-
one-character labels to avoid ambiguous multi-token option names. Builtin
|
|
337
|
-
inference supports up to 35 options per question; hosted and external
|
|
338
|
-
backends retain the protocol's 255-option limit. Builtin answers use the
|
|
339
|
-
official Jev wire shapes and disclose `generic-gguf` in the
|
|
340
|
-
`x-sys1-local-adapter` response header:
|
|
341
|
-
|
|
342
|
-
- Noul returns only `type` and probability-of-yes `noul`;
|
|
343
|
-
- Choice returns `choice`, keyed `probabilities`, and `confidence`;
|
|
344
|
-
- Score returns a zero-based probability-weighted fractional `score`, keyed
|
|
345
|
-
`legend`, keyed `probabilities`, and `confidence`.
|
|
346
|
-
|
|
347
|
-
Adapter input bounds fail closed: Sys1 never silently truncates state,
|
|
348
|
-
instructions, or criteria. Generic GGUF accepts up to 6,000 state characters,
|
|
349
|
-
2,000 instruction characters, 96 characters per option name/criterion, and
|
|
350
|
-
16,000 characters for the complete rendered prompt. An input beyond the adapter's
|
|
351
|
-
bounds returns an error without being sent to a different backend.
|
|
352
|
-
|
|
353
|
-
Adapter quality signals stay outside those answer objects:
|
|
354
|
-
`x-sys1-local-min-coverage` is the least total probability mass assigned to
|
|
355
|
-
allowed labels, and `x-sys1-local-min-concentration` is the least
|
|
356
|
-
distribution concentration in the batch. Low coverage means the model did not
|
|
357
|
-
cleanly follow the decision instruction. These are useful local signals, not
|
|
358
|
-
a calibration guarantee. Use hosted Jev or a task-qualified System One-specific
|
|
359
|
-
backend when its behavior has been evaluated for your task.
|
|
360
|
-
|
|
361
|
-
An unlisted public Hugging Face GGUF can be installed explicitly:
|
|
148
|
+
With a backend configured, run `sys1 up` to start the gateway, then call it:
|
|
362
149
|
|
|
363
|
-
```
|
|
364
|
-
|
|
365
|
-
```
|
|
366
|
-
|
|
367
|
-
## The endpoint
|
|
150
|
+
```ts
|
|
151
|
+
import { createClient } from "@hraness/sys1/client";
|
|
368
152
|
|
|
369
|
-
|
|
370
|
-
|
|
371
|
-
|
|
372
|
-
|
|
373
|
-
|
|
374
|
-
|
|
375
|
-
|
|
376
|
-
|
|
377
|
-
from a remote backend, including 4xx or 5xx, is definitive. Only a transport
|
|
378
|
-
failure may re-dispatch, at most once, and never for a pinned `backend/model`.
|
|
379
|
-
|
|
380
|
-
`state`, `instructions`, and criterion descriptions accept text, JSON objects,
|
|
381
|
-
JSON arrays, or `null` where the official Jev contract permits it. The public
|
|
382
|
-
package exports request and response schemas for boundary validation.
|
|
383
|
-
|
|
384
|
-
### Request example
|
|
385
|
-
|
|
386
|
-
```json
|
|
387
|
-
{
|
|
388
|
-
"model": "auto",
|
|
389
|
-
"state": { "tests": "failing", "branch": "main" },
|
|
390
|
-
"questions": {
|
|
391
|
-
"action": {
|
|
392
|
-
"type": "choice",
|
|
393
|
-
"instructions": "What should the agent do next?",
|
|
394
|
-
"criteria": {
|
|
395
|
-
"fix": "Repair the failure before continuing",
|
|
396
|
-
"continue": "The failure is unrelated and safe to defer",
|
|
397
|
-
"escalate": "Human judgment is required"
|
|
398
|
-
}
|
|
153
|
+
const sys1 = createClient(); // http://127.0.0.1:13900
|
|
154
|
+
const { response } = await sys1.evaluate({
|
|
155
|
+
model: "typesafe/jev-1.13.0",
|
|
156
|
+
state: "Customers cannot complete checkout after today’s release.",
|
|
157
|
+
questions: {
|
|
158
|
+
urgent: {
|
|
159
|
+
type: "noul",
|
|
160
|
+
instructions: "Does this describe an active customer-impacting incident?",
|
|
399
161
|
},
|
|
400
|
-
|
|
401
|
-
|
|
402
|
-
"instructions": "Rate merge risk",
|
|
403
|
-
"criteria": ["low", "moderate", "high"]
|
|
404
|
-
}
|
|
405
|
-
}
|
|
406
|
-
}
|
|
407
|
-
```
|
|
408
|
-
|
|
409
|
-
## Routing
|
|
410
|
-
|
|
411
|
-
For unpinned requests (`model: "auto"` or omitted), `routing.policy` controls
|
|
412
|
-
the order of enabled hosted Jev and the installed model named by `local.model`:
|
|
413
|
-
|
|
414
|
-
| Policy | Order |
|
|
415
|
-
| --- | --- |
|
|
416
|
-
| `auto` (default) | hosted Jev, then the selected local model |
|
|
417
|
-
| `prefer-local` | selected local model, then hosted Jev |
|
|
418
|
-
| `prefer-hosted` | hosted Jev, then the selected local model |
|
|
419
|
-
| `local-only` | selected local model only |
|
|
420
|
-
| `hosted-only` | hosted Jev only |
|
|
421
|
-
|
|
422
|
-
Other installed models and all registered HTTP services require an explicit
|
|
423
|
-
request model or backend/model pin. They never receive unpinned fallback
|
|
424
|
-
traffic. A missing selected model does not promote another installed model.
|
|
425
|
-
Registered HTTP services are local only when their URL uses a
|
|
426
|
-
loopback host; off-machine URLs count as hosted. Redirects are never followed.
|
|
427
|
-
`local-only` decisions neither probe nor dispatch to hosted endpoints. Explicit
|
|
428
|
-
model discovery and doctor may probe all configured backends. Backend names must
|
|
429
|
-
be unique; `typesafe` and `local-*` are reserved for managed candidates. Requests can pin either a model id or an exact backend/model:
|
|
430
|
-
|
|
431
|
-
- `"model": "jev-1.13.0"` selects a backend serving that hosted model;
|
|
432
|
-
- `"model": "local-qwen3-1.7b/qwen3-1.7b"` pins the builtin Qwen runner;
|
|
433
|
-
- `"model": "local-qwen3-0.6b/qwen3-0.6b"` explicitly selects the experimental model;
|
|
434
|
-
- `"model": "openjev/openjev-latest"` pins a registered HTTP backend that advertises that alias.
|
|
435
|
-
|
|
436
|
-
Selection is capability-aware. Sys1 compares each request's largest option
|
|
437
|
-
count and total question count against the backend's published limits. A backend the request exceeds is skipped; when no configured backend
|
|
438
|
-
can serve the request at all the gateway answers `422 request_unsupported`
|
|
439
|
-
rather than dispatching a request that would fail downstream. Builtin
|
|
440
|
-
backends publish their adapter limit (`generic-gguf` 35 options); remote backends
|
|
441
|
-
are probed at `GET /v1/limits` (openjev-style `max_answers_per_question` and
|
|
442
|
-
`max_questions`). A backend that publishes nothing has unknown capacity; Sys1 can enforce only
|
|
443
|
-
its configured limits and the common protocol envelope. Missing limits never
|
|
444
|
-
mean zero capability.
|
|
445
|
-
|
|
446
|
-
## External System One backends
|
|
447
|
-
|
|
448
|
-
Sys1 can route to an [OpenJev](https://github.com/razorback16/openjev) server
|
|
449
|
-
through its standard System One decision API. OpenJev's image, chat, and
|
|
450
|
-
advanced sampling extensions are not supported. A server you register answers
|
|
451
|
-
only requests that select it; adding one does not change the default route.
|
|
452
|
-
|
|
453
|
-
Any service implementing `POST /v1/systemone` and `GET /v1/models` can join the
|
|
454
|
-
same router. For an existing OpenJev server, first inspect its advertised model
|
|
455
|
-
IDs. The example uses OpenJev's documented `openjev-latest` alias; replace it if
|
|
456
|
-
your server advertises a different ID:
|
|
457
|
-
|
|
458
|
-
```sh
|
|
459
|
-
curl http://127.0.0.1:8080/v1/models
|
|
460
|
-
sys1 backend add \
|
|
461
|
-
--name openjev \
|
|
462
|
-
--url http://127.0.0.1:8080 \
|
|
463
|
-
--model openjev-latest
|
|
464
|
-
```
|
|
465
|
-
|
|
466
|
-
Before routing agents to an operator backend, qualify its discovery, limits,
|
|
467
|
-
and all three answer shapes:
|
|
162
|
+
},
|
|
163
|
+
}, { signal: AbortSignal.timeout(5_000) });
|
|
468
164
|
|
|
469
|
-
|
|
470
|
-
sys1 backend check --name openjev
|
|
165
|
+
console.log(response.answers.urgent);
|
|
471
166
|
```
|
|
472
167
|
|
|
473
|
-
|
|
474
|
-
|
|
168
|
+
The answer contains the model’s probability of yes. Sys1 validates requests and
|
|
169
|
+
responses, supports cancellation, and returns stable error codes. Your
|
|
170
|
+
application decides which actions are allowed and evaluates the model on its
|
|
171
|
+
own examples. A valid answer can still be wrong.
|
|
475
172
|
|
|
476
|
-
The
|
|
477
|
-
|
|
478
|
-
normalization, and Score arithmetic; and never prints or persists request or
|
|
479
|
-
response bodies. Backends that do not publish limits receive a warning unless
|
|
480
|
-
static caps were configured. All configured HTTP processes remain
|
|
481
|
-
operator-owned: Sys1 probes and forwards to them but does not download their
|
|
482
|
-
weights, mutate credentials, or own their lifecycle.
|
|
173
|
+
The [client and embedded router guide](docs/runtime.md#use-as-a-module) covers
|
|
174
|
+
custom endpoints, the in-process Bun router, and existing Jev applications.
|
|
483
175
|
|
|
484
|
-
##
|
|
176
|
+
## The endpoint
|
|
485
177
|
|
|
486
|
-
|
|
487
|
-
|
|
488
|
-
|
|
489
|
-
with Jev, local models, or a separately trained Kev checkpoint.
|
|
490
|
-
[Kev and tuning guide](docs/kev.md).
|
|
178
|
+
The loopback gateway serves `POST /v1/systemone` for decisions,
|
|
179
|
+
`GET /v1/models` for discovery, and `GET /healthz` for liveness.
|
|
180
|
+
[HTTP request and response reference](docs/runtime.md#the-endpoint).
|
|
491
181
|
|
|
492
|
-
|
|
182
|
+
## Local models
|
|
493
183
|
|
|
494
|
-
|
|
495
|
-
|
|
496
|
-
|
|
497
|
-
sys1
|
|
498
|
-
```
|
|
184
|
+
Local Qwen models are experimental. In the broader recorded studies, Qwen3
|
|
185
|
+
1.7B answered 32/72 cases correctly; Qwen3.5 4B answered 44/72 on a different
|
|
186
|
+
fresh fixture. Evaluate the selected model on your task before relying on it.
|
|
187
|
+
[Model studies](https://sys1.io/docs/evaluations).
|
|
499
188
|
|
|
500
|
-
|
|
501
|
-
|
|
189
|
+
Preview the download with `sys1 setup --dry-run --json`. The [local setup guide](docs/runtime.md#experimental-local-decisions)
|
|
190
|
+
covers supported platforms, downloads, and model selection. Setup explicitly
|
|
191
|
+
downloads and selects Qwen3 1.7B; other installed models are never automatic
|
|
192
|
+
substitutes.
|
|
502
193
|
|
|
503
|
-
|
|
504
|
-
import { createClient, createProfile } from "@hraness/sys1/client";
|
|
194
|
+
## External System One backends
|
|
505
195
|
|
|
506
|
-
|
|
507
|
-
|
|
508
|
-
|
|
509
|
-
|
|
510
|
-
},
|
|
511
|
-
});
|
|
512
|
-
const result = await createClient().evaluate(triage.request("Checkout is unavailable."));
|
|
513
|
-
```
|
|
196
|
+
Connect a server that implements the System One API, including OpenJev or Kev.
|
|
197
|
+
Register and check the server, then select it in each request. Adding a server
|
|
198
|
+
does not change default routing. [Compatible server setup](docs/runtime.md#external-system-one-backends)
|
|
199
|
+
· [Kev and decision profiles](docs/kev.md).
|
|
514
200
|
|
|
515
|
-
|
|
516
|
-
validated and frozen; each request is a fresh ordinary System One request.
|
|
517
|
-
Only model, state, and questions cross the wire. No templating, hidden prompt
|
|
518
|
-
injection, global profile registry, or automatic training is involved. Save
|
|
519
|
-
the definition as JSON for `sys1 eval --profile triage.json`, which accepts
|
|
520
|
-
only `{"state": ...}` on stdin or `--file`.
|
|
521
|
-
[Start from the ticket-triage profile](examples/ticket-triage.profile.json).
|
|
522
|
-
|
|
523
|
-
For a direct Kev endpoint, use `createClient({ baseUrl: "http://127.0.0.1:8009",
|
|
524
|
-
adapter: "kev" })` with an ordinary request containing `model: "kev-latest"`.
|
|
525
|
-
When calling the Sys1 gateway, leave the client adapter unset. Routing metadata
|
|
526
|
-
identifies Kev and its two-decimal precision; probabilities are not renormalized.
|
|
527
|
-
|
|
528
|
-
The backend/model pin identifies a route, not its weights. Kev always advertises
|
|
529
|
-
`kev-latest`; verify the server's actual checkpoint separately. The
|
|
530
|
-
[setup and tuning guide](docs/kev.md) covers pinned checkpoints, prompt revisions,
|
|
531
|
-
fine-tuning, calibration, and held-out evaluation. Model downloads and training
|
|
532
|
-
remain explicit Kev operations. Protocol tests do not establish model quality.
|
|
533
|
-
|
|
534
|
-
## Diagnostics
|
|
535
|
-
|
|
536
|
-
`sys1 doctor` checks the install and prints one line per check (✓, ⚠ or ✗),
|
|
537
|
-
a count, and the command to run next. It verifies the
|
|
538
|
-
Bun floor, state-directory access, config, native llama.cpp runtime/backend,
|
|
539
|
-
the installed-model list, every installed model's GGUF header, leftover or
|
|
540
|
-
unknown files in the model folder, routing candidates, and daemon ownership. It does not hash entire model
|
|
541
|
-
files; use `sys1 model verify MODEL` for exact SHA-256 verification.
|
|
201
|
+
## Reference and troubleshooting
|
|
542
202
|
|
|
543
|
-
|
|
544
|
-
|
|
545
|
-
|
|
546
|
-
|
|
203
|
+
<a id="diagnostics"></a>
|
|
204
|
+
<a id="routing"></a>
|
|
205
|
+
<a id="configuration"></a>
|
|
206
|
+
<a id="commands"></a>
|
|
207
|
+
<a id="kev-and-decision-profiles"></a>
|
|
547
208
|
|
|
548
|
-
|
|
549
|
-
|
|
550
|
-
|
|
551
|
-
|
|
552
|
-
|
|
553
|
-
|
|
554
|
-
sys1 setup [--tier compact|quality] [--dry-run]
|
|
555
|
-
sys1 jev status|enable|disable
|
|
556
|
-
sys1 up|down|serve|status|doctor
|
|
557
|
-
sys1 pull [MODEL]|pull --list
|
|
558
|
-
sys1 model list|verify|remove
|
|
559
|
-
sys1 models
|
|
560
|
-
sys1 eval
|
|
561
|
-
sys1 audit --staged|--worktree|--since <ref> --model <backend/model>
|
|
562
|
-
sys1 review checkpoint|issues|feedback|recheck|setup
|
|
563
|
-
sys1 rules list|check|draft
|
|
564
|
-
sys1 verify setup codex|claude-code|devin
|
|
565
|
-
sys1 verify --model <backend/model> [--message <file|->]
|
|
566
|
-
sys1 usage [--days <n>]
|
|
567
|
-
sys1 backend list|add|check|remove
|
|
568
|
-
sys1 config path|get|set|unset
|
|
569
|
-
sys1 --version|--help
|
|
570
|
-
```
|
|
209
|
+
- [`sys1 doctor` and diagnostics](docs/runtime.md#diagnostics): inspect the runtime, routing, model store, and gateway after setup.
|
|
210
|
+
- [Routing](docs/runtime.md#routing): select a model and control fallback.
|
|
211
|
+
- [Configuration](docs/runtime.md#configuration): settings, credentials, and gateway access.
|
|
212
|
+
- [Commands](docs/runtime.md#commands): CLI reference and JSON output.
|
|
213
|
+
- [Decision profiles](docs/runtime.md#kev-and-decision-profiles): reuse questions across application requests.
|
|
214
|
+
- [Model comparison](https://sys1.io/compare) and [evaluation reports](https://sys1.io/docs/evaluations): inspect model-specific evidence.
|
|
571
215
|
|
|
572
|
-
|
|
573
|
-
|
|
574
|
-
`
|
|
575
|
-
|
|
576
|
-
command has its own help (`sys1 setup --help`). Errors are one sentence and the
|
|
577
|
-
command to run next; with `--json` they are one `{"ok":false,"error":{...}}`
|
|
578
|
-
object on stdout. `sys1 setup --dry-run` shows the model, its size and the
|
|
579
|
-
folder it would be downloaded to.
|
|
580
|
-
|
|
581
|
-
## Configuration
|
|
582
|
-
|
|
583
|
-
`~/.sys1/config.json` is created on the first write. `SYS1_HOME` overrides
|
|
584
|
-
the state directory. Settable keys:
|
|
585
|
-
|
|
586
|
-
- `routing.policy`;
|
|
587
|
-
- `gateway.host` (loopback addresses only), `gateway.port`,
|
|
588
|
-
`gateway.request_timeout_ms`, `gateway.probe_timeout_ms`;
|
|
589
|
-
- `hosted.base_url`, `hosted.model`, `hosted.api_key_env` (activation uses
|
|
590
|
-
`sys1 jev`);
|
|
591
|
-
- `local.enabled`, `local.model`, `local.context_tokens`, `local.eval_timeout_ms`,
|
|
592
|
-
`local.max_loaded_models`.
|
|
593
|
-
|
|
594
|
-
Fresh config uses `routing.policy: auto`, `local.enabled: true`,
|
|
595
|
-
`local.model: qwen3-1.7b`, `hosted.model: jev-1.13.0`, and
|
|
596
|
-
`hosted.enabled: false`. The daemon reads config per request, so routing and
|
|
597
|
-
backend changes do not need a restart. Restart after changing local runtime
|
|
598
|
-
context, timeout, or residency settings. Environment variables are inherited when
|
|
599
|
-
the daemon starts, so restart it after exporting a new Jev credential. Already
|
|
600
|
-
loaded GGUFs stay resident up to `local.max_loaded_models` (default one) and are
|
|
601
|
-
released on eviction or daemon shutdown. Local requests are serialized
|
|
602
|
-
to keep context state isolated and residency bounded. GGUF inference lives in
|
|
603
|
-
an owned worker process; abort, timeout, or disposal terminates and collects
|
|
604
|
-
that worker before the next request can reuse the engine slot.
|
|
605
|
-
|
|
606
|
-
The decision endpoint accepts loopback binds only and has no application
|
|
607
|
-
authentication. Network admission blocks browser-originated decision dispatch and
|
|
608
|
-
non-loopback Host authorities, but it does not authenticate local processes.
|
|
609
|
-
Any process that can connect locally can dispatch decisions using the gateway's
|
|
610
|
-
enabled backends and credentials; use it only on a trusted local machine.
|
|
611
|
-
The in-process `createRouter`/`createFetchHandler` surface leaves admission to
|
|
612
|
-
its owning application. Daemon shutdown uses a per-instance
|
|
613
|
-
secret from its private pid file and an authenticated control endpoint. Sys1
|
|
614
|
-
never signals an arbitrary PID read from that file.
|
|
216
|
+
If `sys1` is missing after installation, check that npm’s global binary
|
|
217
|
+
directory is on your `PATH` and that `bun --version` works. If a request cannot
|
|
218
|
+
find a model, inspect `sys1 jev status` and `sys1 doctor`. Restart an existing
|
|
219
|
+
gateway after changing its credential environment.
|
|
615
220
|
|
|
616
221
|
## Releases
|
|
617
222
|
|
|
618
223
|
An annotated `v<version>` tag at the exact current `main` head requests a
|
|
619
|
-
release
|
|
620
|
-
|
|
621
|
-
|
|
622
|
-
|
|
623
|
-
|
|
624
|
-
|
|
625
|
-
Each release page copies its summary and changes from the version's section of
|
|
626
|
-
[`CHANGELOG.md`](CHANGELOG.md) and adds the install command, the tarball's
|
|
627
|
-
SHA-256, and the source commit. The workflow stops before publishing when that
|
|
628
|
-
section is missing, and it fails when a published page no longer matches the
|
|
629
|
-
changelog and the attached files.
|
|
224
|
+
release and must match `package.json`. The release workflow reruns the complete
|
|
225
|
+
gate, creates one npm-format tarball and `SHA256SUMS`, and installs those exact
|
|
226
|
+
bytes with the native dependency on Ubuntu, macOS, and Windows before
|
|
227
|
+
publishing an immutable GitHub Release. Release notes come from
|
|
228
|
+
[CHANGELOG.md](CHANGELOG.md). The optional npm mirror publishes that same
|
|
229
|
+
tarball after the package’s one-time registry setup.
|
|
630
230
|
|
|
631
231
|
## Development
|
|
632
232
|
|
|
633
233
|
```sh
|
|
234
|
+
git clone https://github.com/hraness/sys1.git
|
|
235
|
+
cd sys1
|
|
634
236
|
bun install
|
|
635
237
|
bun run check
|
|
636
238
|
```
|
|
637
239
|
|
|
638
|
-
|
|
639
|
-
|
|
640
|
-
|
|
641
|
-
the packed candidate in a disposable prefix and checks real native runtime
|
|
642
|
-
readiness before a release tag is needed. This does not rebuild the package
|
|
643
|
-
or download model weights. Model inference and hosted calls remain excluded
|
|
644
|
-
from ordinary CI; the release workflow still verifies its exact uploaded
|
|
645
|
-
artifact on all three platforms.
|
|
646
|
-
|
|
647
|
-
## Comparing models
|
|
648
|
-
|
|
649
|
-
Use [the model comparison](https://sys1.io/compare) for external JevBench
|
|
650
|
-
accuracy on the hosted Jev and operator-registered OpenJev routes, route
|
|
651
|
-
availability, and a workload cost calculator. It uses the pinned
|
|
652
|
-
[JevBench v1.2.6 snapshot](https://github.com/fstandhartinger/jevbench/tree/v1.2.6);
|
|
653
|
-
its methodology and limits are recorded in the [evidence appendix](docs/model-comparison.md).
|
|
654
|
-
Built-in Qwen has no matching JevBench result: SemIf's Qwen3.5 4B uses a
|
|
655
|
-
different adapter and BF16 checkpoint, so its score does not apply to Sys1's
|
|
656
|
-
GGUF path. An external OpenJev GPU score likewise does not establish Mac MLX
|
|
657
|
-
quality or qualify a Sys1 deployment.
|
|
658
|
-
|
|
659
|
-
[Sys1's original evaluation studies](https://sys1.io/docs/evaluations) remain
|
|
660
|
-
available with raw reports, failure analysis, and token-accounting details.
|
|
661
|
-
The [opt-in Sys1 benchmarks](benchmarks/README.md) support adapter qualification
|
|
662
|
-
and reproduction; they are not the cross-model leaderboard. No local candidate
|
|
663
|
-
is generally qualified, and the original fixtures do not justify an automatic
|
|
664
|
-
application migration.
|
|
665
|
-
|
|
666
|
-
## Related
|
|
667
|
-
|
|
668
|
-
[System One Skills](https://github.com/hraness/system-one-skills) is a separate
|
|
669
|
-
skill for Devin, Claude Code, and Codex. It runs a known noisy test or build
|
|
670
|
-
once, returns a short result with the exit status, and keeps the full log on
|
|
671
|
-
disk. It needs no model, API key, or Sys1 installation, and installing either
|
|
672
|
-
project does not configure the other. [Skills guide](https://sys1.io/skills).
|
|
673
|
-
|
|
674
|
-
[The thread through hraness](https://hraness.com/writing/the-thread-through-hraness)
|
|
675
|
-
describes the design Sys1 shares with every Hraness project: a decision has a
|
|
676
|
-
declared shape before a model is asked, and the answer comes back validated
|
|
677
|
-
instead of as prose.
|
|
240
|
+
See [CONTRIBUTING.md](CONTRIBUTING.md) for the development and release workflow.
|
|
241
|
+
Report a bug in [GitHub Issues](https://github.com/hraness/sys1/issues), or follow
|
|
242
|
+
[SECURITY.md](SECURITY.md) for a security report.
|