sneakoscope 0.4.0 → 0.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +318 -5
- package/docs/PERFORMANCE.md +21 -1
- package/package.json +15 -4
- package/src/cli/main.mjs +756 -13
- package/src/core/db-safety.mjs +7 -1
- package/src/core/evaluation.mjs +238 -0
- package/src/core/fsx.mjs +1 -1
- package/src/core/hooks-runtime.mjs +65 -29
- package/src/core/hproof.mjs +6 -0
- package/src/core/init.mjs +205 -16
- package/src/core/research.mjs +143 -0
package/README.md
CHANGED
|
@@ -10,10 +10,187 @@ Sneakoscope Codex is a zero-runtime-dependency Node.js harness for running Codex
|
|
|
10
10
|
npm i -g sneakoscope
|
|
11
11
|
```
|
|
12
12
|
|
|
13
|
-
The npm package name is `sneakoscope`; the command is branded as SKS and exposed as lowercase `sks` for shell portability.
|
|
13
|
+
The npm package name is `sneakoscope`; the command is branded as SKS and exposed as lowercase `sks` for shell portability. The package also exposes a `sneakoscope` command alias, so `sks setup` and `sneakoscope setup` are equivalent.
|
|
14
|
+
Global installation is the default and recommended setup. During `sks setup` or `sks init`, SKS resolves the global binary when possible and writes that absolute path into `.codex/hooks.json`, which avoids PATH issues in GUI or hook execution environments. For a project-only install, use `npm i -D sneakoscope` and initialize hooks with `npx sks setup --install-scope project`; this writes hook commands that call the local `node_modules/sneakoscope` binary.
|
|
14
15
|
|
|
15
16
|
`@openai/codex` is intentionally not bundled. Install Codex separately, or set `SKS_CODEX_BIN` to the Codex executable you want Sneakoscope Codex to supervise.
|
|
16
17
|
|
|
18
|
+
## One-Prompt LLM Install
|
|
19
|
+
|
|
20
|
+
If you are using Codex App, ChatGPT, Claude Code, Cursor, or another coding agent, copy this prompt into the agent from your target project directory:
|
|
21
|
+
|
|
22
|
+
````text
|
|
23
|
+
You are installing Sneakoscope Codex in the current project. Do the setup end to end without asking follow-up questions unless a command needs user approval.
|
|
24
|
+
|
|
25
|
+
Repository:
|
|
26
|
+
https://github.com/mandarange/Sneakoscope-Codex.git
|
|
27
|
+
|
|
28
|
+
Requirements:
|
|
29
|
+
- Node.js must be >=20.11.
|
|
30
|
+
- Codex CLI is installed separately; if missing, report that @openai/codex must be installed or SKS_CODEX_BIN must be set.
|
|
31
|
+
- Prefer safe, local verification. Do not modify application source files unless needed for SKS setup.
|
|
32
|
+
|
|
33
|
+
Run:
|
|
34
|
+
```bash
|
|
35
|
+
npm i -g git+https://github.com/mandarange/Sneakoscope-Codex.git
|
|
36
|
+
sks setup
|
|
37
|
+
sks doctor --fix
|
|
38
|
+
sks selftest --mock
|
|
39
|
+
sks commands
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
If the global command is not on PATH, use:
|
|
43
|
+
```bash
|
|
44
|
+
npx -y -p git+https://github.com/mandarange/Sneakoscope-Codex.git sks setup
|
|
45
|
+
npx -y -p git+https://github.com/mandarange/Sneakoscope-Codex.git sks doctor --fix
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
After setup, explain these outputs to the user:
|
|
49
|
+
- `.sneakoscope/` mission state and policy
|
|
50
|
+
- `.codex/config.toml` Codex App profiles
|
|
51
|
+
- `.codex/hooks.json` SKS hook integration
|
|
52
|
+
- `.codex/skills/` local Codex App skills
|
|
53
|
+
- `.codex/SNEAKOSCOPE.md` Codex App quick reference
|
|
54
|
+
- `AGENTS.md` repository rules
|
|
55
|
+
|
|
56
|
+
Show the user how to discover commands:
|
|
57
|
+
```bash
|
|
58
|
+
sks help
|
|
59
|
+
sks commands
|
|
60
|
+
sks usage ralph
|
|
61
|
+
sks quickstart
|
|
62
|
+
sks codex-app
|
|
63
|
+
sks dollar-commands
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
Tell the user they can use these prompt commands inside Codex App:
|
|
67
|
+
```text
|
|
68
|
+
$DF 글자 색 바꿔줘
|
|
69
|
+
$DF 내용을 영어로 바꿔줘
|
|
70
|
+
$SKS show me available workflows
|
|
71
|
+
$Ralph implement this with mandatory clarification
|
|
72
|
+
$Research investigate this idea
|
|
73
|
+
$DB check this migration safely
|
|
74
|
+
```
|
|
75
|
+
````
|
|
76
|
+
|
|
77
|
+
After SKS is installed, you can print this prompt again from the CLI:
|
|
78
|
+
|
|
79
|
+
```bash
|
|
80
|
+
sks install-prompt
|
|
81
|
+
sks install-prompt --project
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
## Repository
|
|
85
|
+
|
|
86
|
+
```bash
|
|
87
|
+
npm i -g git+https://github.com/mandarange/Sneakoscope-Codex.git
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
Source repository: <https://github.com/mandarange/Sneakoscope-Codex.git>
|
|
91
|
+
|
|
92
|
+
Local development checkout:
|
|
93
|
+
|
|
94
|
+
```bash
|
|
95
|
+
git clone https://github.com/mandarange/Sneakoscope-Codex.git
|
|
96
|
+
cd Sneakoscope-Codex
|
|
97
|
+
npm i
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
## Installed Commands
|
|
101
|
+
|
|
102
|
+
Installing the package exposes two equivalent shell commands:
|
|
103
|
+
|
|
104
|
+
```bash
|
|
105
|
+
sks <command>
|
|
106
|
+
sneakoscope <command>
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
Use `sks --help` or `sneakoscope --help` to inspect the installed CLI. The user-facing subcommands are listed in [Commands](#commands).
|
|
110
|
+
|
|
111
|
+
Useful discovery commands:
|
|
112
|
+
|
|
113
|
+
```bash
|
|
114
|
+
sks commands
|
|
115
|
+
sks usage install
|
|
116
|
+
sks usage ralph
|
|
117
|
+
sks quickstart
|
|
118
|
+
sks install-prompt
|
|
119
|
+
sks codex-app
|
|
120
|
+
sks dollar-commands
|
|
121
|
+
sks df
|
|
122
|
+
sks aliases
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
## Prompt Pipeline and $ Commands
|
|
126
|
+
|
|
127
|
+
SKS installs a Codex App `UserPromptSubmit` hook that adds a lightweight prompt-optimization context to every user request. You do not need to type a command for basic routing: SKS will infer the lightest path before work starts.
|
|
128
|
+
|
|
129
|
+
Use `$` prompt commands inside Codex App or another coding agent when you want to force a route:
|
|
130
|
+
|
|
131
|
+
```text
|
|
132
|
+
$DF fast design/content fix
|
|
133
|
+
$SKS general Sneakoscope workflow/help
|
|
134
|
+
$Ralph clarification-gated Ralph mission
|
|
135
|
+
$Research frontier research mission
|
|
136
|
+
$DB database/Supabase safety check
|
|
137
|
+
$GX deterministic visual context
|
|
138
|
+
$Help command and workflow help
|
|
139
|
+
```
|
|
140
|
+
|
|
141
|
+
`$DF` is intentionally small and fast. Use it for changes like text color, visible copy, labels, spacing, button text, or translation:
|
|
142
|
+
|
|
143
|
+
```text
|
|
144
|
+
$DF 글자 색 파란색으로 바꿔줘
|
|
145
|
+
$DF 내용을 영어로 바꿔줘
|
|
146
|
+
$DF Change the CTA label to "Start"
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
DF should not start Ralph, Research, evaluation, or a broad redesign unless you explicitly ask for that.
|
|
150
|
+
|
|
151
|
+
## Codex App
|
|
152
|
+
|
|
153
|
+
Sneakoscope Codex can also be used from Codex App when the repository is opened in the app. Run setup once in the project:
|
|
154
|
+
|
|
155
|
+
```bash
|
|
156
|
+
sks setup
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
This creates the app-facing control surface:
|
|
160
|
+
|
|
161
|
+
```text
|
|
162
|
+
.codex/config.toml Codex App profiles for SKS Ralph, research, and default work
|
|
163
|
+
.codex/hooks.json Codex App hook entrypoints routed through SKS guards
|
|
164
|
+
.codex/skills/ local project skills for Ralph, DB safety, GX, research, and design work
|
|
165
|
+
.codex/SNEAKOSCOPE.md quick reference for using SKS inside Codex App
|
|
166
|
+
AGENTS.md repository rules loaded by Codex agents
|
|
167
|
+
.sneakoscope/ mission state, gates, logs, policy, GX cartridges, and reports
|
|
168
|
+
```
|
|
169
|
+
|
|
170
|
+
Inside Codex App, you can ask the agent to use the local SKS control surface, for example:
|
|
171
|
+
|
|
172
|
+
```text
|
|
173
|
+
$DF 글자 색 바꿔줘
|
|
174
|
+
$DF 내용을 영어로 바꿔줘
|
|
175
|
+
Use Sneakoscope Ralph mode to prepare this task.
|
|
176
|
+
Run the latest Ralph mission with the sealed decision contract.
|
|
177
|
+
Use SKS DB safety before touching database or Supabase files.
|
|
178
|
+
Use SKS research mode for this investigation.
|
|
179
|
+
```
|
|
180
|
+
|
|
181
|
+
If Codex App cannot find `sks` from hooks, run:
|
|
182
|
+
|
|
183
|
+
```bash
|
|
184
|
+
sks fix-path
|
|
185
|
+
```
|
|
186
|
+
|
|
187
|
+
For a project-only install, use:
|
|
188
|
+
|
|
189
|
+
```bash
|
|
190
|
+
npm i -D sneakoscope
|
|
191
|
+
npx sks setup --install-scope project
|
|
192
|
+
```
|
|
193
|
+
|
|
17
194
|
## Requirements
|
|
18
195
|
|
|
19
196
|
- Node.js `>=20.11`
|
|
@@ -24,11 +201,29 @@ The npm package name is `sneakoscope`; the command is branded as SKS and exposed
|
|
|
24
201
|
## Quick Start
|
|
25
202
|
|
|
26
203
|
```bash
|
|
27
|
-
sks
|
|
28
|
-
sks init
|
|
204
|
+
sks setup
|
|
29
205
|
sks selftest --mock
|
|
30
206
|
```
|
|
31
207
|
|
|
208
|
+
Project-only setup:
|
|
209
|
+
|
|
210
|
+
```bash
|
|
211
|
+
npm i -D sneakoscope
|
|
212
|
+
npx sks setup --install-scope project
|
|
213
|
+
```
|
|
214
|
+
|
|
215
|
+
If a GUI hook, Codex session, or another project cannot find `sks`, refresh the hook command with the resolved binary path:
|
|
216
|
+
|
|
217
|
+
```bash
|
|
218
|
+
sks fix-path
|
|
219
|
+
```
|
|
220
|
+
|
|
221
|
+
If your shell cannot find the global command yet, run through npm without relying on PATH:
|
|
222
|
+
|
|
223
|
+
```bash
|
|
224
|
+
npx -y -p sneakoscope sks setup
|
|
225
|
+
```
|
|
226
|
+
|
|
32
227
|
Create a Ralph mission:
|
|
33
228
|
|
|
34
229
|
```bash
|
|
@@ -51,15 +246,27 @@ For a local smoke test that does not call a model:
|
|
|
51
246
|
sks ralph run latest --mock
|
|
52
247
|
```
|
|
53
248
|
|
|
249
|
+
Run a research mission:
|
|
250
|
+
|
|
251
|
+
```bash
|
|
252
|
+
sks research prepare "LLM 에이전트의 새로운 평가 방법론"
|
|
253
|
+
sks research run latest --max-cycles 3
|
|
254
|
+
```
|
|
255
|
+
|
|
54
256
|
## What Sneakoscope Codex Adds
|
|
55
257
|
|
|
56
258
|
- **Mandatory clarification**: `ralph prepare` generates required decision slots before autonomous execution can start.
|
|
57
259
|
- **Sealed decision contract**: `ralph answer` validates answers and writes `decision-contract.json`.
|
|
58
260
|
- **No-question Ralph loop**: after `ralph run` starts, Ralph must resolve ambiguity with the sealed contract instead of asking the user.
|
|
261
|
+
- **Research mode**: `research` runs a frontier-discovery loop for non-obvious hypotheses, falsification, novelty ledgers, and testable experiments.
|
|
262
|
+
- **Prompt pipeline and `$` routes**: user prompts are lightly optimized by default, and Codex App users can force routes such as `$DF`, `$Ralph`, `$Research`, `$DB`, and `$GX`.
|
|
263
|
+
- **Fast DF mode**: `$DF` handles small design/content edits like color, copy, labels, spacing, and translation without unnecessary Ralph, Research, or evaluation loops.
|
|
59
264
|
- **Database guard**: destructive DB operations, production writes, unsafe Supabase MCP configuration, and direct live SQL mutations are blocked or warned on.
|
|
60
265
|
- **H-Proof done gate**: completion requires supported critical claims, reviewed DB safety state, acceptable visual/wiki drift, and required test evidence.
|
|
266
|
+
- **Performance evaluation**: `sks eval` produces deterministic token, accuracy-proxy, recall, support, and runtime metrics for before/after evidence.
|
|
61
267
|
- **Bounded runtime state**: child process output is tailed, logs are rotated/compacted, and old mission artifacts can be pruned.
|
|
62
268
|
- **Visual cartridges**: `gx` creates deterministic SVG/HTML visual context from `vgraph.json` and `beta.json`; no generated-image service is required.
|
|
269
|
+
- **Design artifact skill**: `sks init` installs a local skill for high-fidelity HTML/UI/prototype work with design-context gathering and rendered verification.
|
|
63
270
|
|
|
64
271
|
## Ralph Workflow
|
|
65
272
|
|
|
@@ -91,9 +298,26 @@ Core invariants:
|
|
|
91
298
|
|
|
92
299
|
## Commands
|
|
93
300
|
|
|
301
|
+
All examples below use `sks`, but the same commands can be run with the `sneakoscope` alias.
|
|
302
|
+
|
|
94
303
|
```bash
|
|
95
|
-
sks
|
|
96
|
-
sks
|
|
304
|
+
sks help [topic]
|
|
305
|
+
sks commands [--json]
|
|
306
|
+
sks usage [install|setup|ralph|research|db|codex-app|df|dollar|eval|gx]
|
|
307
|
+
sks quickstart
|
|
308
|
+
sks install-prompt [--project]
|
|
309
|
+
sks codex-app
|
|
310
|
+
sks dollar-commands [--json]
|
|
311
|
+
sks df
|
|
312
|
+
sks aliases
|
|
313
|
+
|
|
314
|
+
sks --help
|
|
315
|
+
sneakoscope --help
|
|
316
|
+
|
|
317
|
+
sks setup [--install-scope global|project] [--force] [--json]
|
|
318
|
+
sks fix-path [--install-scope global|project] [--json]
|
|
319
|
+
sks doctor [--fix] [--json] [--install-scope global|project]
|
|
320
|
+
sks init [--force] [--install-scope global|project]
|
|
97
321
|
sks selftest [--mock]
|
|
98
322
|
|
|
99
323
|
sks ralph prepare "task"
|
|
@@ -101,6 +325,10 @@ sks ralph answer <mission-id|latest> <answers.json>
|
|
|
101
325
|
sks ralph run <mission-id|latest> [--mock] [--max-cycles N]
|
|
102
326
|
sks ralph status <mission-id|latest>
|
|
103
327
|
|
|
328
|
+
sks research prepare "topic" [--depth frontier]
|
|
329
|
+
sks research run <mission-id|latest> [--mock] [--max-cycles N]
|
|
330
|
+
sks research status <mission-id|latest>
|
|
331
|
+
|
|
104
332
|
sks db policy
|
|
105
333
|
sks db scan [--migrations] [--json]
|
|
106
334
|
sks db mcp-config --project-ref <ref> [--features database,docs]
|
|
@@ -110,7 +338,12 @@ sks db check --sql "SELECT * FROM users LIMIT 10"
|
|
|
110
338
|
sks db check --command "supabase db reset"
|
|
111
339
|
sks db check --file ./migration.sql
|
|
112
340
|
|
|
341
|
+
sks eval run [--json] [--out report.json] [--iterations N]
|
|
342
|
+
sks eval compare --baseline old.json --candidate new.json [--json]
|
|
343
|
+
sks eval thresholds
|
|
344
|
+
|
|
113
345
|
sks hproof check [mission-id|latest]
|
|
346
|
+
sks team "task"
|
|
114
347
|
sks gx init [name]
|
|
115
348
|
sks gx render [name] [--format svg|html|all]
|
|
116
349
|
sks gx validate [name]
|
|
@@ -119,11 +352,38 @@ sks gx snapshot [name]
|
|
|
119
352
|
sks profile show
|
|
120
353
|
sks profile set <model>
|
|
121
354
|
sks gc [--dry-run] [--json]
|
|
355
|
+
sks memory [--dry-run] [--json]
|
|
122
356
|
sks stats [--json]
|
|
123
357
|
```
|
|
124
358
|
|
|
125
359
|
`sks memory` is currently an alias for garbage collection/retention handling.
|
|
126
360
|
|
|
361
|
+
## Research Mode
|
|
362
|
+
|
|
363
|
+
Research mode is for exploratory work where the desired output is a possible new insight, mechanism, prediction, or experiment, not a summary. It uses a frontier-discovery loop:
|
|
364
|
+
|
|
365
|
+
```text
|
|
366
|
+
R0 frame discovery criteria
|
|
367
|
+
R1 map assumptions and baselines
|
|
368
|
+
R2 generate competing hypotheses
|
|
369
|
+
R3 falsify with counterexamples and missing evidence
|
|
370
|
+
R4 synthesize surviving mechanisms
|
|
371
|
+
R5 propose tests, predictions, or probes
|
|
372
|
+
R6 write novelty ledger and research gate
|
|
373
|
+
```
|
|
374
|
+
|
|
375
|
+
Artifacts are written under `.sneakoscope/missions/<MISSION_ID>/`:
|
|
376
|
+
|
|
377
|
+
```text
|
|
378
|
+
research-plan.md
|
|
379
|
+
research-plan.json
|
|
380
|
+
research-report.md
|
|
381
|
+
novelty-ledger.json
|
|
382
|
+
research-gate.json
|
|
383
|
+
```
|
|
384
|
+
|
|
385
|
+
`sks research run` uses the `sks-research` Codex profile with maximum configured reasoning effort. `--mock` exercises the local artifact flow without calling a model.
|
|
386
|
+
|
|
127
387
|
## Database Safety
|
|
128
388
|
|
|
129
389
|
Sneakoscope Codex treats database access as high risk across Supabase MCP, Supabase CLI, Postgres, Prisma, Drizzle, Knex, Sequelize, `psql`, SQL files, and MCP-shaped payloads.
|
|
@@ -172,6 +432,24 @@ sks db check --command "supabase db reset"
|
|
|
172
432
|
|
|
173
433
|
Hooks are strongest for Codex tool execution paths, but Sneakoscope Codex does not rely on hooks alone. Ralph startup also scans DB/MCP configuration, and the supervised prompt embeds the DB policy.
|
|
174
434
|
|
|
435
|
+
## Performance Evaluation
|
|
436
|
+
|
|
437
|
+
`sks eval run` benchmarks the current SKS flow with a deterministic context-selection scenario. It compares an uncompressed all-claims baseline against the TriWiki compressed capsule and reports:
|
|
438
|
+
|
|
439
|
+
```text
|
|
440
|
+
estimated_tokens
|
|
441
|
+
token_savings_pct
|
|
442
|
+
accuracy_proxy
|
|
443
|
+
required_recall
|
|
444
|
+
relevance_precision
|
|
445
|
+
support_ratio
|
|
446
|
+
unsupported_critical_selected
|
|
447
|
+
context_build_ms_per_run
|
|
448
|
+
meaningful_improvement
|
|
449
|
+
```
|
|
450
|
+
|
|
451
|
+
`accuracy_proxy` is an evidence-weighted context quality metric, not a live model task score. Use `sks eval compare --baseline old.json --candidate new.json` to compare saved JSON reports across versions or experiments.
|
|
452
|
+
|
|
175
453
|
## H-Proof Done Gate
|
|
176
454
|
|
|
177
455
|
Ralph completion is evaluated through `.sneakoscope/missions/<MISSION_ID>/done-gate.json`.
|
|
@@ -183,6 +461,8 @@ A mission cannot pass when:
|
|
|
183
461
|
- a database safety violation or destructive DB attempt is recorded
|
|
184
462
|
- DB safety logs exist but have not been reviewed
|
|
185
463
|
- required tests lack evidence
|
|
464
|
+
- required performance evaluation evidence is missing
|
|
465
|
+
- required design verification evidence is missing
|
|
186
466
|
- visual or wiki drift is marked `high`
|
|
187
467
|
|
|
188
468
|
Run the evaluator directly with:
|
|
@@ -199,10 +479,21 @@ sks hproof check latest
|
|
|
199
479
|
.sneakoscope/ mission state, policy, retention, logs, GX cartridges
|
|
200
480
|
.codex/config.toml Codex profiles used by Sneakoscope Codex
|
|
201
481
|
.codex/hooks.json hook entrypoints
|
|
482
|
+
.codex/skills/ Codex App local project skills
|
|
483
|
+
.codex/SNEAKOSCOPE.md Codex App quick reference
|
|
202
484
|
.agents/skills/ Sneakoscope Codex helper skills
|
|
203
485
|
AGENTS.md managed repository rules block
|
|
204
486
|
```
|
|
205
487
|
|
|
488
|
+
Install scope controls `.codex/hooks.json`:
|
|
489
|
+
|
|
490
|
+
```text
|
|
491
|
+
global -> /absolute/path/to/sks hook ... when resolvable, otherwise sks hook ...
|
|
492
|
+
project -> node ./node_modules/sneakoscope/bin/sks.mjs hook ...
|
|
493
|
+
```
|
|
494
|
+
|
|
495
|
+
If no scope is provided, SKS uses `global`.
|
|
496
|
+
|
|
206
497
|
Storage is intentionally bounded:
|
|
207
498
|
|
|
208
499
|
- process stdout/stderr are kept as bounded tails
|
|
@@ -264,9 +555,11 @@ Q0 raw logs only when necessary
|
|
|
264
555
|
bin/sks.mjs CLI executable
|
|
265
556
|
src/cli/main.mjs command router and Ralph loop
|
|
266
557
|
src/core/db-safety.mjs SQL, CLI, and MCP payload classifier
|
|
558
|
+
src/core/evaluation.mjs token, accuracy-proxy, and context-quality evaluator
|
|
267
559
|
src/core/gx-renderer.mjs deterministic SVG/HTML visual context renderer
|
|
268
560
|
src/core/hproof.mjs done-gate evaluator
|
|
269
561
|
src/core/init.mjs project bootstrap and hook/skill installation
|
|
562
|
+
src/core/research.mjs research-mode plan, novelty ledger, and gate helpers
|
|
270
563
|
src/core/retention.mjs storage report and garbage collection policy
|
|
271
564
|
src/core/triwiki-attention.mjs
|
|
272
565
|
docs/PERFORMANCE.md resource and leak policy
|
|
@@ -278,12 +571,32 @@ The published npm package is allowlisted to `bin`, `src`, `docs`, `README.md`, a
|
|
|
278
571
|
## Development
|
|
279
572
|
|
|
280
573
|
```bash
|
|
574
|
+
npm run repo-audit
|
|
281
575
|
npm run packcheck
|
|
282
576
|
npm run selftest
|
|
283
577
|
npm run sizecheck
|
|
284
578
|
npm run doctor
|
|
285
579
|
```
|
|
286
580
|
|
|
581
|
+
`npm run repo-audit` checks tracked files for risky local paths and high-confidence secret material such as private keys, npm/GitHub/OpenAI-style tokens, local MCP configs, DB dumps, and credential files. It is included in `prepack` and `prepublishOnly`.
|
|
582
|
+
|
|
287
583
|
`npm run sizecheck` blocks accidental package bloat before `npm pack` or `npm publish`. Defaults: packed tarball `<=96 KiB`, unpacked package `<=320 KiB`, package files `<=40`, and each tracked file `<=256 KiB`. Override only for an intentional release with `SKS_MAX_PACK_BYTES`, `SKS_MAX_UNPACKED_BYTES`, `SKS_MAX_PACK_FILES`, or `SKS_MAX_TRACKED_FILE_BYTES`.
|
|
288
584
|
|
|
289
585
|
`npm run selftest` uses the mock path and does not call a model. Live Ralph runs require a working Codex CLI installation and authentication.
|
|
586
|
+
|
|
587
|
+
## Publishing
|
|
588
|
+
|
|
589
|
+
The npm package is published as public package `sneakoscope`. You must be logged in as an npm owner for that package before publishing.
|
|
590
|
+
|
|
591
|
+
```bash
|
|
592
|
+
npm whoami
|
|
593
|
+
npm owner ls sneakoscope
|
|
594
|
+
npm run publish:dry
|
|
595
|
+
npm run publish:npm
|
|
596
|
+
```
|
|
597
|
+
|
|
598
|
+
If `npm whoami` returns `E401 Unauthorized`, run `npm login` with an owner account or ask an existing owner to add your npm username:
|
|
599
|
+
|
|
600
|
+
```bash
|
|
601
|
+
npm owner add <your-npm-username> sneakoscope
|
|
602
|
+
```
|
package/docs/PERFORMANCE.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Sneakoscope Codex performance and leak policy
|
|
2
2
|
|
|
3
|
-
Sneakoscope Codex v0.
|
|
3
|
+
Sneakoscope Codex v0.6 is designed to keep runtime, package size, RAM, and storage bounded.
|
|
4
4
|
|
|
5
5
|
## Speed
|
|
6
6
|
|
|
@@ -10,6 +10,26 @@ Sneakoscope Codex v0.4 is designed to keep runtime, package size, RAM, and stora
|
|
|
10
10
|
- GX visual context renders deterministic SVG/HTML from JSON sources, avoiding external image-generation latency, cost, and nondeterminism.
|
|
11
11
|
- `sks gc` runs after Ralph cycles by default.
|
|
12
12
|
|
|
13
|
+
## Evaluation metrics
|
|
14
|
+
|
|
15
|
+
`sks eval run` creates a deterministic JSON report in `.sneakoscope/reports/` unless `--no-save` is used. The built-in scenario compares an uncompressed all-claims baseline with a TriWiki compressed context capsule.
|
|
16
|
+
|
|
17
|
+
Tracked metrics:
|
|
18
|
+
|
|
19
|
+
- `estimated_tokens`: deterministic chars/4 prompt-size estimate for local regression tracking
|
|
20
|
+
- `token_savings_pct`: prompt-size reduction versus baseline
|
|
21
|
+
- `accuracy_proxy`: evidence-weighted context-selection quality score
|
|
22
|
+
- `required_recall`: required claim coverage
|
|
23
|
+
- `relevance_precision`: selected required claims divided by selected claims
|
|
24
|
+
- `support_ratio`: selected claims that are supported or weakly supported
|
|
25
|
+
- `unsupported_critical_selected`: critical/high unsupported claims that survived compression
|
|
26
|
+
- `context_build_ms_per_run`: local context construction runtime
|
|
27
|
+
- `meaningful_improvement`: true only when token savings, accuracy delta, recall, unsupported-critical filtering, and runtime thresholds pass
|
|
28
|
+
|
|
29
|
+
Default meaningful-improvement thresholds are intentionally explicit: at least 25% token savings, at least +0.03 accuracy-proxy delta, at least 0.95 required recall, zero unsupported critical claims selected, and candidate context construction under 25 ms per run. `sks eval compare --baseline old.json --candidate new.json` compares saved reports across implementations.
|
|
30
|
+
|
|
31
|
+
The accuracy metric is not a live model task score. It is a deterministic proxy for whether the context handed to a model is smaller, better supported, and less contaminated by unsupported critical claims.
|
|
32
|
+
|
|
13
33
|
## Package size
|
|
14
34
|
|
|
15
35
|
- The npm package has zero runtime dependencies.
|
package/package.json
CHANGED
|
@@ -1,11 +1,16 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "sneakoscope",
|
|
3
3
|
"displayName": "Sneakoscope Codex",
|
|
4
|
-
"version": "0.
|
|
4
|
+
"version": "0.6.0",
|
|
5
5
|
"description": "Sneakoscope Codex: database-safe, performance-bounded Codex CLI harness with Ralph no-question loop, H-Proof gates, deterministic GX visual context, and TriWiki compression.",
|
|
6
6
|
"type": "module",
|
|
7
7
|
"bin": {
|
|
8
|
-
"sks": "bin/sks.mjs"
|
|
8
|
+
"sks": "bin/sks.mjs",
|
|
9
|
+
"sneakoscope": "bin/sks.mjs"
|
|
10
|
+
},
|
|
11
|
+
"publishConfig": {
|
|
12
|
+
"access": "public",
|
|
13
|
+
"registry": "https://registry.npmjs.org/"
|
|
9
14
|
},
|
|
10
15
|
"files": [
|
|
11
16
|
"bin",
|
|
@@ -18,12 +23,15 @@
|
|
|
18
23
|
"node": ">=20.11"
|
|
19
24
|
},
|
|
20
25
|
"scripts": {
|
|
26
|
+
"repo-audit": "node ./scripts/repo-audit.mjs",
|
|
21
27
|
"selftest": "node ./bin/sks.mjs selftest --mock",
|
|
22
28
|
"doctor": "node ./bin/sks.mjs doctor",
|
|
23
29
|
"packcheck": "find bin src scripts -name '*.mjs' -print0 | xargs -0 -n1 node --check",
|
|
24
30
|
"sizecheck": "node ./scripts/sizecheck.mjs",
|
|
25
|
-
"
|
|
26
|
-
"
|
|
31
|
+
"publish:dry": "npm --cache /tmp/sks-npm-cache pack --dry-run",
|
|
32
|
+
"publish:npm": "npm --cache /tmp/sks-npm-cache publish --access public",
|
|
33
|
+
"prepack": "npm run repo-audit && npm run packcheck && npm run selftest && npm run sizecheck",
|
|
34
|
+
"prepublishOnly": "npm run repo-audit && npm run packcheck && npm run selftest && npm run sizecheck"
|
|
27
35
|
},
|
|
28
36
|
"keywords": [
|
|
29
37
|
"sneakoscope",
|
|
@@ -33,6 +41,9 @@
|
|
|
33
41
|
"ai-agent",
|
|
34
42
|
"harness",
|
|
35
43
|
"ralph",
|
|
44
|
+
"research",
|
|
45
|
+
"hypothesis",
|
|
46
|
+
"discovery",
|
|
36
47
|
"llm-wiki",
|
|
37
48
|
"gx",
|
|
38
49
|
"svg",
|