@n8r/surface 0.0.0-stage → 0.1.0-experimental.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +202 -0
- package/LICENSE-DOCS +395 -0
- package/LICENSING.md +36 -0
- package/README.md +207 -2
- package/dist/activity.d.ts +16 -0
- package/dist/activity.js +44 -0
- package/dist/cli.d.ts +2 -0
- package/dist/cli.js +412 -0
- package/dist/codec.d.ts +6 -0
- package/dist/codec.js +56 -0
- package/dist/codex.d.ts +72 -0
- package/dist/codex.js +526 -0
- package/dist/contracts.d.ts +235 -0
- package/dist/contracts.js +1 -0
- package/dist/cyclonedx-schemas/bom-1.7.schema.json +6700 -0
- package/dist/cyclonedx-schemas/cryptography-defs.schema.json +576 -0
- package/dist/cyclonedx-schemas/jsf-0.82.schema.json +240 -0
- package/dist/cyclonedx-schemas/spdx.schema.json +786 -0
- package/dist/cyclonedx-validation.d.ts +1 -0
- package/dist/cyclonedx-validation.js +45 -0
- package/dist/html.d.ts +5 -0
- package/dist/html.js +74 -0
- package/dist/index.d.ts +7 -0
- package/dist/index.js +5 -0
- package/dist/notes.d.ts +52 -0
- package/dist/notes.js +184 -0
- package/dist/portable.d.ts +10 -0
- package/dist/portable.js +71 -0
- package/dist/recent.d.ts +9 -0
- package/dist/recent.js +71 -0
- package/dist/revisions.d.ts +2 -0
- package/dist/revisions.js +60 -0
- package/dist/settings.d.ts +23 -0
- package/dist/settings.js +100 -0
- package/dist/setup.d.ts +9 -0
- package/dist/setup.js +161 -0
- package/dist/skill-artifacts-cli.d.ts +1 -0
- package/dist/skill-artifacts-cli.js +13 -0
- package/dist/skill-artifacts.d.ts +15 -0
- package/dist/skill-artifacts.js +184 -0
- package/dist/skill-cyclonedx.d.ts +26 -0
- package/dist/skill-cyclonedx.js +214 -0
- package/dist/skill-presentation.d.ts +10 -0
- package/dist/skill-presentation.js +108 -0
- package/dist/skill-surface-cli.d.ts +1 -0
- package/dist/skill-surface-cli.js +37 -0
- package/dist/skill-surface-store.d.ts +7 -0
- package/dist/skill-surface-store.js +126 -0
- package/dist/source-tools.d.ts +102 -0
- package/dist/source-tools.js +201 -0
- package/dist/startup.d.ts +7 -0
- package/dist/startup.js +74 -0
- package/dist/surface-bom.d.ts +73 -0
- package/dist/surface-bom.js +139 -0
- package/dist/surface-draft.d.ts +22 -0
- package/dist/surface-draft.js +112 -0
- package/dist/surface-instructions.d.ts +3 -0
- package/dist/surface-instructions.js +32 -0
- package/dist/surface-overview.d.ts +15 -0
- package/dist/surface-overview.js +86 -0
- package/dist/surface-store.d.ts +34 -0
- package/dist/surface-store.js +213 -0
- package/dist/surface-view.d.ts +3 -0
- package/dist/surface-view.js +50 -0
- package/dist/validate.d.ts +4 -0
- package/dist/validate.js +482 -0
- package/docs/n8r-surface/README.md +133 -0
- package/docs/n8r-surface/cli-bundles.md +72 -0
- package/docs/n8r-surface/codex-research-transcript.md +51 -0
- package/docs/n8r-surface/codex-research-verification.md +176 -0
- package/docs/n8r-surface/command-outcomes.md +711 -0
- package/docs/n8r-surface/contracts.md +220 -0
- package/docs/n8r-surface/critical-path.md +184 -0
- package/docs/n8r-surface/design.md +289 -0
- package/docs/n8r-surface/first-use-access.md +143 -0
- package/docs/n8r-surface/first-use-overview.md +262 -0
- package/docs/n8r-surface/getting-started.md +74 -0
- package/docs/n8r-surface/guided-journey-verification.md +137 -0
- package/docs/n8r-surface/host-contract.md +176 -0
- package/docs/n8r-surface/implementation-decisions.md +96 -0
- package/docs/n8r-surface/instance-first-research.md +209 -0
- package/docs/n8r-surface/npm-distribution.md +107 -0
- package/docs/n8r-surface/npm-release.md +315 -0
- package/docs/n8r-surface/product-direction.md +327 -0
- package/docs/n8r-surface/recreation-handoff.md +117 -0
- package/docs/n8r-surface/researcher-discovery.md +157 -0
- package/docs/n8r-surface/researcher-prepared-check.md +117 -0
- package/docs/n8r-surface/researcher-reconciliation-trial.md +149 -0
- package/docs/n8r-surface/researcher-repository-check.md +95 -0
- package/docs/n8r-surface/researcher-skeleton-check.md +84 -0
- package/docs/n8r-surface/researcher-skill-verification.md +725 -0
- package/docs/n8r-surface/researcher-skill.md +309 -0
- package/docs/n8r-surface/researcher-vault.md +134 -0
- package/docs/n8r-surface/save-reopen-transcript.md +211 -0
- package/docs/n8r-surface/save-reopen-verification.md +152 -0
- package/docs/n8r-surface/security-assurance-strategy.md +267 -0
- package/docs/n8r-surface/skill-bom-verification.md +142 -0
- package/docs/n8r-surface/skill-cyclonedx-verification.md +144 -0
- package/docs/n8r-surface/skill-cyclonedx.md +85 -0
- package/docs/n8r-surface/skill-manual.md +129 -0
- package/docs/n8r-surface/skill-surface-artifacts.md +234 -0
- package/docs/n8r-surface/skill-usefulness-verification.md +135 -0
- package/docs/n8r-surface/skill-workspace-automation.md +359 -0
- package/docs/n8r-surface/startup-human-check.md +172 -0
- package/docs/n8r-surface/startup-verification.md +161 -0
- package/docs/n8r-surface/startup.md +151 -0
- package/docs/n8r-surface/surface-description-contract.md +224 -0
- package/docs/n8r-surface/surface-descriptions.md +173 -0
- package/docs/n8r-surface/surface-relationships.md +231 -0
- package/docs/n8r-surface/surface-revisions-transcript.md +552 -0
- package/docs/n8r-surface/surface-revisions-verification.md +161 -0
- package/docs/n8r-surface/tester-candidate.md +158 -0
- package/docs/n8r-surface/vault-setup-verification.md +210 -0
- package/docs/n8r-surface/verification.md +117 -0
- package/docs/n8r-surface/versioning.md +172 -0
- package/docs/n8r-surface/workspace-human-check.md +101 -0
- package/docs/n8r-surface/workspace-installation.md +68 -0
- package/docs/n8r-surface/workspace-verification.md +350 -0
- package/docs/n8r-surface/workspace.md +83 -0
- package/examples/researcher/README.md +100 -0
- package/examples/researcher/composed.surface.json +430 -0
- package/examples/researcher/composed.surface.yaml +283 -0
- package/examples/researcher/cyclonedx/configured/README.md +7 -0
- package/examples/researcher/cyclonedx/configured/manifest.json +10 -0
- package/examples/researcher/cyclonedx/configured/mapping.json +323 -0
- package/examples/researcher/cyclonedx/configured/surface-records.json +1275 -0
- package/examples/researcher/cyclonedx/configured/surface.cdx.json +79 -0
- package/examples/researcher/descriptions/README.md +29 -0
- package/examples/researcher/descriptions/bound-r1.json +141 -0
- package/examples/researcher/descriptions/bound-r2.json +153 -0
- package/examples/researcher/descriptions/capability.json +167 -0
- package/examples/researcher/descriptions/research-r1.json +153 -0
- package/examples/researcher/descriptions/research-r2.json +157 -0
- package/examples/researcher/descriptions/sources.md +24 -0
- package/examples/researcher/harbor-r1.research.html +193 -0
- package/examples/researcher/harbor-r2.research.html +226 -0
- package/examples/researcher/lumen-manual-edit.pending.html +271 -0
- package/examples/researcher/lumen-r1.research.html +271 -0
- package/examples/researcher/lumen-r2.research.html +280 -0
- package/examples/researcher/relationships/README.md +34 -0
- package/examples/researcher/relationships/bound-r1.json +346 -0
- package/examples/researcher/relationships/bound-r2.json +358 -0
- package/examples/researcher/relationships/capability.json +325 -0
- package/examples/researcher/relationships/config/assistant.env.example +4 -0
- package/examples/researcher/relationships/research-r1.json +300 -0
- package/examples/researcher/relationships/research-r2.json +304 -0
- package/examples/researcher/relationships/sources.md +38 -0
- package/examples/researcher/sources/harbor-api.md +16 -0
- package/examples/researcher/sources/harbor-auth.md +8 -0
- package/examples/researcher/sources/lumen-guide.md +28 -0
- package/examples/researcher/sources/lumen-settings.md +9 -0
- package/examples/researcher/surface-bom/README.md +17 -0
- package/examples/researcher/surface-bom/initial.cdx.json +188 -0
- package/package.json +45 -4
- package/plugins/n8r-surface/.codex-plugin/plugin.json +31 -0
- package/plugins/n8r-surface/LICENSE +202 -0
- package/plugins/n8r-surface/LICENSE-DOCS +395 -0
- package/plugins/n8r-surface/LICENSING.md +39 -0
- package/plugins/n8r-surface/assets/icon.svg +5 -0
- package/plugins/n8r-surface/plugin.json +27 -0
- package/plugins/n8r-surface/skills/research/SKILL.md +163 -0
- package/plugins/n8r-surface/skills/research/agents/openai.yaml +4 -0
- package/plugins/n8r-surface/skills/research/references/artifacts.md +152 -0
- package/plugins/n8r-surface/skills/research/references/cyclonedx.md +207 -0
- package/plugins/n8r-surface/skills/research/references/discovery-map.md +178 -0
- package/plugins/n8r-surface/skills/research/references/examples/bound-r1.json +346 -0
- package/plugins/n8r-surface/skills/research/references/examples/capability.json +325 -0
- package/plugins/n8r-surface/skills/research/references/examples/research-r1.json +300 -0
- package/plugins/n8r-surface/skills/research/references/examples/surface.cdx.json +188 -0
- package/plugins/n8r-surface/skills/research/references/first-use-access.md +97 -0
- package/plugins/n8r-surface/skills/research/references/first-use-overview.md +101 -0
- package/plugins/n8r-surface/skills/research/references/instance-first.md +125 -0
- package/plugins/n8r-surface/skills/research/references/model-meanings.md +55 -0
- package/plugins/n8r-surface/skills/research/references/skeleton-surface.md +105 -0
- package/plugins/n8r-surface/skills/research/references/working-notes.md +121 -0
- package/plugins/n8r-surface/skills/research/references/workspace.md +89 -0
- package/plugins/n8r-surface/skills/research/scripts/THIRD-PARTY-NOTICES.txt +273 -0
- package/plugins/n8r-surface/skills/research/scripts/artifacts.mjs +8348 -0
- package/plugins/n8r-surface/skills/research/scripts/cyclonedx.mjs +28113 -0
- package/plugins/n8r-surface/skills/research/scripts/surface.mjs +27800 -0
- package/plugins/n8r-surface/skills/research/scripts/workspace.mjs +120 -0
|
@@ -0,0 +1,135 @@
|
|
|
1
|
+
# Installed Surface skill: two-product usefulness check
|
|
2
|
+
|
|
3
|
+
Related to [#52](https://github.com/injectionator/n8r-surface/issues/52).
|
|
4
|
+
This checks the standalone Surface skill before CLI integration. The products
|
|
5
|
+
are research subjects, not new integrations or deployment targets.
|
|
6
|
+
|
|
7
|
+
## Run identity and scope
|
|
8
|
+
|
|
9
|
+
- Date: 8 November 2026.
|
|
10
|
+
- Skill: 0.1.11, source commit `8e4cb3ade3c9e5ca59ab0a37838ccaf874b7fc42`
|
|
11
|
+
from PR #54. An isolated Codex marketplace installation matched every tracked
|
|
12
|
+
plugin file to that commit before research.
|
|
13
|
+
- Host: Codex CLI 0.166.0 app-server, ChatGPT sign-in, configured model
|
|
14
|
+
`gpt-6-astra`, Node 26.13.0. Fresh ephemeral sessions had no prior turns;
|
|
15
|
+
previous-thread and receipt access were independently denied.
|
|
16
|
+
- Source access: pinned public file copies, read-only research; no source
|
|
17
|
+
execution, additional fetches or deployment inspection. Each product was
|
|
18
|
+
bounded to model API, storage and identity connections.
|
|
19
|
+
- Configuration and reviewer inputs were fictional test fixtures. They do not
|
|
20
|
+
describe a real deployment or constitute human acceptance.
|
|
21
|
+
|
|
22
|
+
| Product | Pinned source | Files supplied |
|
|
23
|
+
| --- | --- | --- |
|
|
24
|
+
| LibreChat | [f4154b6](https://github.com/danny-avila/LibreChat/tree/f4154b649efa37e1345ccc517a4d8d1ed2178c82) | README, database connection, OpenID strategy |
|
|
25
|
+
| Open WebUI | [b7a005e](https://github.com/open-webui/open-webui/tree/b7a005e5a5c0c0d869f2aebf4b8f051c5fb37245) | README, SQL database layer, OAuth implementation, model API router |
|
|
26
|
+
|
|
27
|
+
## Interaction exercised
|
|
28
|
+
|
|
29
|
+
1. Ask the installed Researcher which model API, storage and identity connections
|
|
30
|
+
a reviewer should examine; save a checked Capability with source hashes and
|
|
31
|
+
bounded evidence. Do not infer selected targets from supported options.
|
|
32
|
+
2. Supply a fictional configuration declaring Model Service A, Database Service A
|
|
33
|
+
and Identity Service A. Permit only six named nonsecret fields to be projected
|
|
34
|
+
from a fixture containing synthetic sensitive sentinels. Supply a second
|
|
35
|
+
same-revision handover note declaring Identity Service B.
|
|
36
|
+
3. Give a scripted Test reviewer preference for Identity Service A in a future
|
|
37
|
+
revision, explicitly leaving the current contradiction unresolved and runtime
|
|
38
|
+
verification deferred. Save a Bound Surface; retain both source claims and
|
|
39
|
+
keep the reviewer decision separate from evidence.
|
|
40
|
+
4. Start a fresh session, reopen the saved Bound artifacts and supply a later
|
|
41
|
+
configuration selecting Model Service B. Save a successor with stable
|
|
42
|
+
identities, exact parents, unchanged prior files, retained decision and
|
|
43
|
+
unresolved identity conflict. Independently read all saved directories.
|
|
44
|
+
|
|
45
|
+
Shell/file tools and Node are permitted. Only the `n8r-surface` product executable
|
|
46
|
+
is prohibited. The initial fresh-session prompt mistakenly said “no CLI use”;
|
|
47
|
+
both sessions declined file access, accurately reported the blocker and created
|
|
48
|
+
no successor. Those receipts are retained. The retry clarified this instruction
|
|
49
|
+
in new sessions without changing skill code or overwriting previous artifacts.
|
|
50
|
+
|
|
51
|
+
For a repeat of the corrected revision step, start a fresh Researcher chat with
|
|
52
|
+
that trial's workspace selected and send:
|
|
53
|
+
|
|
54
|
+
```text
|
|
55
|
+
Reopen bound-001 and read change.md. Shell/file tools and Node.js are allowed,
|
|
56
|
+
including the installed scripts/artifacts.mjs helper. Do not invoke the
|
|
57
|
+
n8r-surface product executable. Save a checked successor in bound-002 selecting
|
|
58
|
+
Model Service B. Preserve identities, decisions, unresolved identity evidence,
|
|
59
|
+
exact parent references and all previous files. Retain the previous research
|
|
60
|
+
as a dependency. Use the existing safe projection; do not expose private fields
|
|
61
|
+
from instance.json. Saving is authorized. Read back both revisions, compare
|
|
62
|
+
prior-file hashes, return output links and stop.
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
## Useful findings sampled against sources
|
|
66
|
+
|
|
67
|
+
| Product | Finding | Independent source check |
|
|
68
|
+
| --- | --- | --- |
|
|
69
|
+
| LibreChat | Model integrations are documented possibilities, not an observed request path. | README lines 74–99 |
|
|
70
|
+
| LibreChat | Storage uses a Mongoose/MongoDB connection; stored AI data and controls remain unverified. | `api/db/connect.js` lines 1–27 and 43–74 |
|
|
71
|
+
| LibreChat | OIDC discovery and login depend on configuration, including PKCE/nonce settings. | `api/strategies/openidStrategy.js` lines 954–1006 |
|
|
72
|
+
| Open WebUI | Chat/Responses routing sends a POST to the selected connection. | `backend/open_webui/routers/openai.py` lines 1506–1587 |
|
|
73
|
+
| Open WebUI | SQLAlchemy has SQLite/PostgreSQL paths and rejects SQLCipher in async URL conversion. | `backend/open_webui/internal/db.py` lines 230–250, 264–285 and 450–468 |
|
|
74
|
+
| Open WebUI | OAuth identity can affect local authority; effective settings remain unknown. | `backend/open_webui/utils/oauth.py` lines 168–190 and 1618–1650 |
|
|
75
|
+
|
|
76
|
+
These are bounded agent checks of cited claims, not exhaustive source review,
|
|
77
|
+
runtime proof or security certification. The developing agent did not author
|
|
78
|
+
the Researcher's findings or saved bundles.
|
|
79
|
+
|
|
80
|
+
## Observed result
|
|
81
|
+
|
|
82
|
+
Both products passed the bounded agent rehearsal on the unchanged 0.1.11 skill.
|
|
83
|
+
No product-specific code or new skill implementation was needed.
|
|
84
|
+
|
|
85
|
+
| Independent check | LibreChat | Open WebUI |
|
|
86
|
+
| --- | --- | --- |
|
|
87
|
+
| Capability, Bound and successor helper readback | Exit 0, checked | Exit 0, checked |
|
|
88
|
+
| Model target revision | A to B | A to B |
|
|
89
|
+
| Database target | A retained | A retained |
|
|
90
|
+
| Conflicting identity declarations | A and B retained | A and B retained |
|
|
91
|
+
| Scripted reviewer decision | Exactly preserved | Exactly preserved |
|
|
92
|
+
| Research/Surface parent references and Capability pin | Exact match | Exact match |
|
|
93
|
+
| Files present before successful revision | 27 unchanged | 24 unchanged |
|
|
94
|
+
| Source hash comparisons, including repeated dependency references | 19 matched | 28 matched |
|
|
95
|
+
| Files scanned for synthetic sensitive sentinels | 68, no matches | 62, no matches |
|
|
96
|
+
|
|
97
|
+
The owned raw sentinel input was excluded from the leak scan; artifacts,
|
|
98
|
+
projections and execution receipts were included. This is evidence about those
|
|
99
|
+
specific sentinels, not complete secret detection. All supplied public source
|
|
100
|
+
copies remained byte-identical. Successor checks independently compared node and
|
|
101
|
+
relationship identities, role endpoints, decisions, pins and parent digests.
|
|
102
|
+
|
|
103
|
+
The initial harness recorded completed *turns* before checking whether outputs
|
|
104
|
+
existed. That was not artifact success: independent readback found both missing
|
|
105
|
+
successors. The retry requires successful helper readback before recording success.
|
|
106
|
+
The blocked attempts and corrected requests remain in the private evidence.
|
|
107
|
+
|
|
108
|
+
Private evidence retains the installation inventory, source identities/hashes,
|
|
109
|
+
requests, execution receipts, before/after inventories, independent helper output
|
|
110
|
+
and per-product verification reports. Saved briefs, HTML and JSON were read back;
|
|
111
|
+
no screenshot or desktop visual acceptance is claimed.
|
|
112
|
+
|
|
113
|
+
## Human acceptance and completion boundary
|
|
114
|
+
|
|
115
|
+
**Agent rehearsal: Pass. User acceptance: accepted on 8 November 2026.** The user accepted the skill verification and authorized PRs #54 and #58 to be
|
|
116
|
+
made ready to merge. The test-reviewer correction was scripted input, not a PM
|
|
117
|
+
decision about a deployment or a separately observed human rerun. There is no remaining
|
|
118
|
+
skill-code blocker identified by this bounded check. Broader research coverage,
|
|
119
|
+
runtime use, CLI integration and Labs compatibility have not been established.
|
|
120
|
+
No further presentation polish or additional product trials are required by this
|
|
121
|
+
increment. Review and merge of PR #54 remain separate decisions.
|
|
122
|
+
|
|
123
|
+
The accepted evidence can be reviewed using the two final `bound-002/brief.md` links supplied with
|
|
124
|
+
this run's handoff; no installation or new research is necessary:
|
|
125
|
+
|
|
126
|
+
- [ ] Read each brief. It must explain Model Service B, the retained database
|
|
127
|
+
target, the unresolved identity disagreement, evidence and unknowns.
|
|
128
|
+
- [ ] Open its research HTML or JSON if a claim needs inspection. The reviewer
|
|
129
|
+
preference must remain separate from source evidence and actual runtime use.
|
|
130
|
+
- [ ] Report **Pass / Fail / Blocked** for usefulness. For Fail, name the unclear
|
|
131
|
+
or unsupported claim; for Blocked, name the inaccessible file or missing context.
|
|
132
|
+
|
|
133
|
+
User acceptance completes the usefulness decision for this scope. Readiness
|
|
134
|
+
authorization does not authorize merge, publication or deployment. Keep #52 WIP while its
|
|
135
|
+
implementing PRs await review/merge, following the repository's issue policy.
|
|
@@ -0,0 +1,359 @@
|
|
|
1
|
+
# Installed Researcher workspace acceptance
|
|
2
|
+
|
|
3
|
+
Issue [#43](https://github.com/injectionator/n8r-surface/issues/43) binds only the
|
|
4
|
+
first three [canonical scenarios](../../tests/acceptance/skill-workspace.feature)
|
|
5
|
+
to Cucumber: create, repeat setup and fresh-session reopen. The other scenarios
|
|
6
|
+
remain manual. Ordinary `npm test` is offline and never starts inference.
|
|
7
|
+
|
|
8
|
+
## Scope and interface
|
|
9
|
+
|
|
10
|
+
The runner uses **Codex CLI 0.166.0 app-server over stdio**, not the desktop UI
|
|
11
|
+
and not the standalone n8r command. It requires an enabled installed Researcher
|
|
12
|
+
**0.1.9**, byte-identical to source commit
|
|
13
|
+
`3cb08f59a4a932121bdd1c742fbac724d12a6d74`. The default installation is the normal
|
|
14
|
+
Codex plugin cache. `--installed PATH --revision FULL_COMMIT` can select another
|
|
15
|
+
already-installed, byte-verified 0.1.9 candidate. The runner never installs or
|
|
16
|
+
replaces a plugin. See the [installation procedure](workspace-installation.md).
|
|
17
|
+
|
|
18
|
+
The [documented skill input](https://developers.openai.com/codex/app-server#skills)
|
|
19
|
+
loads the discovered installed SKILL.md. The harness supplies only the selected
|
|
20
|
+
folder and scenario request. The model reads the skill reference and chooses the
|
|
21
|
+
installed helper invocation; the harness never invokes the helper or writes
|
|
22
|
+
workspace identity metadata. Its sole content write is the synthetic user note
|
|
23
|
+
between create and repeat.
|
|
24
|
+
|
|
25
|
+
One ephemeral thread holds create and repeat. Reopen starts another app-server
|
|
26
|
+
process and empty ephemeral thread, with no copied transcript, expected identity
|
|
27
|
+
or answer. The first process is closed; the second must fail to read the old
|
|
28
|
+
thread through `thread/read`. The
|
|
29
|
+
[permission profile](https://learn.chatgpt.com/docs/permissions) grants writes only
|
|
30
|
+
to the disposable workspace and reads to the installed plugin and runtime paths.
|
|
31
|
+
It denies access to retained evidence; a sandbox command checks allowed skill
|
|
32
|
+
reads and denied evidence reads before each session, including the actual setup
|
|
33
|
+
receipt before reopen. This technical probe never invokes the workspace helper.
|
|
34
|
+
|
|
35
|
+
Memories, project instructions, apps, configured MCP servers, browser/computer
|
|
36
|
+
access, remote control, multi-agent/history features and web search are disabled.
|
|
37
|
+
The normal sign-in is used by the host, never copied into fixtures or receipts.
|
|
38
|
+
Local commands receive a restricted environment. Command network access is off;
|
|
39
|
+
Codex authentication and model service traffic still need network access.
|
|
40
|
+
|
|
41
|
+
These controls establish the tested CLI boundary, not a general sandbox security
|
|
42
|
+
certification. Runtime paths remain readable. The Codex service processes the
|
|
43
|
+
conversation. No claim is made about desktop behavior, other hosts, or complete
|
|
44
|
+
system-wide file auditing.
|
|
45
|
+
|
|
46
|
+
## Run the three scenarios
|
|
47
|
+
|
|
48
|
+
Maintainer prerequisites: Node.js 22, 24 or 26; Git; `npm ci`; Codex CLI 0.166.0
|
|
49
|
+
signed in with ChatGPT; the identified plugin installed/enabled; a configured
|
|
50
|
+
available OpenAI model. The runner records the selected model and refuses a
|
|
51
|
+
provider fallback. A live run consumes account inference allowance and may incur
|
|
52
|
+
cost under the account's terms. It is opt-in and is not scheduled in CI.
|
|
53
|
+
|
|
54
|
+
From the repository root, on the branch under test:
|
|
55
|
+
|
|
56
|
+
```sh
|
|
57
|
+
npm run acceptance:skill -- --live --evidence .local/skill-acceptance/run-1
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
The evidence directory must not already exist. Create its parent first if needed:
|
|
61
|
+
`mkdir -p .local/skill-acceptance`. Keep it private and ignored. A second run uses
|
|
62
|
+
a new evidence directory and automatically creates a new disposable workspace:
|
|
63
|
+
|
|
64
|
+
```sh
|
|
65
|
+
npm run acceptance:skill -- --live --evidence .local/skill-acceptance/run-2
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
There are at most three user turns, one per scenario, each limited to 120 seconds,
|
|
69
|
+
and a 600-second suite limit. No simulated user, retries or automatic conversation
|
|
70
|
+
repair are used. A missing capability, unexpected host question or unanswered
|
|
71
|
+
setup question is a failure/blocker; dependent scenarios cannot pass.
|
|
72
|
+
|
|
73
|
+
The runner reports **automated** results separately from full Gherkin acceptance.
|
|
74
|
+
Independent checks compare the saved format, UUID, folder-derived name, supplied
|
|
75
|
+
purpose, complete entry lists, hashes and file timestamps. They compare helper
|
|
76
|
+
receipts and assistant identity/location claims with filesystem readback. This
|
|
77
|
+
detects changed-note and wrong-identity evidence without a model grader.
|
|
78
|
+
|
|
79
|
+
Semantic criteria, such as useful continuation, accurate attachment claims and
|
|
80
|
+
absence of unnecessary questions, remain **pending human review**. Each scenario
|
|
81
|
+
runs its automated checks before an After hook marks it pending. Cucumber's
|
|
82
|
+
passing intermediate steps are not full scenario acceptance. No pending criterion
|
|
83
|
+
is counted as passed by default.
|
|
84
|
+
|
|
85
|
+
Exit codes:
|
|
86
|
+
|
|
87
|
+
- **1:** at least one automated scenario or final installation check failed.
|
|
88
|
+
- **2:** blocked prerequisites, or automated checks passed but human review is
|
|
89
|
+
pending. Inspect `results.json` and `blocked.json` to distinguish these cases.
|
|
90
|
+
- **0:** successful owned-fixture cleanup. Offline report and review modes have
|
|
91
|
+
their own exit meanings below. A live run does not claim full human
|
|
92
|
+
acceptance and therefore does not return 0 in this increment.
|
|
93
|
+
|
|
94
|
+
## Evidence and human verification
|
|
95
|
+
|
|
96
|
+
The evidence directory contains run/install identity once (`metadata.json`), raw
|
|
97
|
+
app-server requests/events, each turn, before/after inventories, final metadata,
|
|
98
|
+
Cucumber JSON and individual results with outstanding review criteria. Failed
|
|
99
|
+
attempts get their own directory; rerunning never overwrites them. Paths, model
|
|
100
|
+
responses and tool receipts are private: do not commit these files or paste raw
|
|
101
|
+
receipts into public issues/PRs. Review and sanitize any published summary.
|
|
102
|
+
|
|
103
|
+
Tester steps, after the maintainer has prepared the prerequisites above:
|
|
104
|
+
|
|
105
|
+
- [ ] Run the first command from the repository root. Expect three automated
|
|
106
|
+
results and exit 2 when all automated checks pass with human review pending.
|
|
107
|
+
An automated failure, skipped prerequisite or `blocked.json` is not a pass.
|
|
108
|
+
- [ ] Open `results.json`, `1-turn.json`, `2-turn.json` and `3-turn.json` in the
|
|
109
|
+
selected private evidence directory. For each result, inspect the listed
|
|
110
|
+
`reviewRequired` criteria against actual responses and tool activity.
|
|
111
|
+
Expect creation only, recognition on repeat, and useful readiness orientation
|
|
112
|
+
on reopen. No research, credentials request, attachment claim or repeated
|
|
113
|
+
setup permission question should occur. Use the
|
|
114
|
+
[existing experience rubric](workspace-human-check.md#independent-evidence-and-report).
|
|
115
|
+
- [ ] Open the inventory files and `final-record.json`. Expect only identity
|
|
116
|
+
metadata after create, then that metadata plus the synthetic note, unchanged
|
|
117
|
+
through repeat/reopen. Inspect `setup.jsonl` and `reopen.jsonl` for the installed
|
|
118
|
+
helper path, denied-evidence probes and distinct empty ephemeral sessions.
|
|
119
|
+
- [ ] Run again with `run-2`. Expect the same observable outcomes with a different
|
|
120
|
+
disposable destination, UUID, session IDs and timings. These values may vary;
|
|
121
|
+
preservation within one journey must not vary.
|
|
122
|
+
- [ ] Report **Pass / Fail / Blocked** per scenario, including the reviewed run
|
|
123
|
+
directory and criterion, without publishing private transcripts. An automated
|
|
124
|
+
pass with unreviewed criteria is **Pending**, not human acceptance.
|
|
125
|
+
|
|
126
|
+
After preserving/reviewing evidence, clean only the owned disposable fixture:
|
|
127
|
+
|
|
128
|
+
```sh
|
|
129
|
+
npm run acceptance:skill -- --cleanup --evidence .local/skill-acceptance/run-1
|
|
130
|
+
```
|
|
131
|
+
|
|
132
|
+
Expected: exit 0 and “Removed owned fixture only; evidence retained.” Cleanup
|
|
133
|
+
requires the recorded ownership token and final inventory, refuses other roots,
|
|
134
|
+
and leaves the evidence, normal installation and earlier trials untouched.
|
|
135
|
+
|
|
136
|
+
## Offline checker tests
|
|
137
|
+
|
|
138
|
+
```sh
|
|
139
|
+
node --test tests/skill-acceptance.test.mjs
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
These tests deliberately substitute an incorrect identity, alter a synthetic note
|
|
143
|
+
and timestamp, add a file, remove/fail the installed helper receipt, and fail a
|
|
144
|
+
scenario prerequisite. They validate the assertions; they are not installed-skill
|
|
145
|
+
acceptance runs. They also run in the normal offline `npm test` suite.
|
|
146
|
+
|
|
147
|
+
## Development evidence — 8 November 2026
|
|
148
|
+
|
|
149
|
+
Initial live rehearsals used macOS, Node 26.13.0, Codex CLI 0.166.0 and the
|
|
150
|
+
configured `gpt-6-astra` model (medium effort). The normal installed Researcher
|
|
151
|
+
0.1.9 matched all 14 files at the pinned revision before use. No installation
|
|
152
|
+
replacement, desktop claim, merge or release was involved.
|
|
153
|
+
|
|
154
|
+
Failed attempts are retained privately:
|
|
155
|
+
|
|
156
|
+
| Attempt | Observed result |
|
|
157
|
+
| --- | --- |
|
|
158
|
+
| Initial interface probe | Installed skill selected its helper; Node could not read its system OpenSSL configuration. Destination remained empty. |
|
|
159
|
+
| Permission probe launch | Nested config was incorrectly serialized as JSON rather than TOML; server did not start. |
|
|
160
|
+
| First runner preflight | The harness incorrectly rejected the standard configured endpoint; no inference. An offline regression test now covers it. |
|
|
161
|
+
| First full journey | Create passed. Repeat preserved files/identity but the checker wrongly demanded the absolute path again rather than allowing “selected folder.” Reopen was blocked by that failed prerequisite. The corrected assertion retains the create location requirement. |
|
|
162
|
+
|
|
163
|
+
Two later fresh-fixture journeys passed all three automated scenarios:
|
|
164
|
+
|
|
165
|
+
| Private run label | Create | Repeat | Fresh reopen | Full Gherkin result |
|
|
166
|
+
| --- | --- | --- | --- | --- |
|
|
167
|
+
| run-3 | 17.842 s | 11.881 s | 16.574 s | Pending human review |
|
|
168
|
+
| run-4 | 21.449 s | 12.872 s | 18.257 s | Pending human review |
|
|
169
|
+
|
|
170
|
+
These are developing-agent rehearsals, not PM acceptance. They used a dirty
|
|
171
|
+
checkout during implementation; run-4 records complete harness file digests.
|
|
172
|
+
A subsequent startup-deadline fix has an offline regression test; the committed
|
|
173
|
+
rehearsals below include that fix. The maintained feature text is
|
|
174
|
+
unchanged apart from selection tags. Checker tests demonstrate rejection of
|
|
175
|
+
incorrect identity, altered notes, ambiguous isolation failures, malformed output
|
|
176
|
+
and unsafe cleanup. The runner leaves human criteria pending instead of treating
|
|
177
|
+
these tests or filesystem results as full conversational acceptance.
|
|
178
|
+
|
|
179
|
+
### Committed-code repetitions
|
|
180
|
+
|
|
181
|
+
Both runs below exercised clean commit
|
|
182
|
+
`d9c99c4e9853ca407264f126a9dd2aa6b8599df4`, with identical harness digests and
|
|
183
|
+
the environment/installed revision above. Each used a fresh disposable fixture,
|
|
184
|
+
a different generated identity, and separate setup/reopen processes. Both exited
|
|
185
|
+
2 with all automated checks passed and all full scenarios pending human review.
|
|
186
|
+
|
|
187
|
+
| Private run label | Create | Repeat | Fresh reopen |
|
|
188
|
+
| --- | --- | --- | --- |
|
|
189
|
+
| run-5 | 18.251 s | 8.713 s | 17.801 s |
|
|
190
|
+
| run-6 | 17.966 s | 8.431 s | 14.814 s |
|
|
191
|
+
|
|
192
|
+
Independent readback confirmed unchanged repeat/reopen inventories, global
|
|
193
|
+
settings fingerprints and installed bytes. Both reopen receipts retain the exact
|
|
194
|
+
missing-thread error and denied evidence reads. Agent inspection of the messages
|
|
195
|
+
found orientation, no repeated initialization question, honest attachment/access
|
|
196
|
+
boundaries and no research activity. This is an agent assessment, not PM acceptance;
|
|
197
|
+
the runner correctly leaves those human criteria pending. The documented cleanup
|
|
198
|
+
command then removed both owned fixtures, exited 0 and retained all evidence.
|
|
199
|
+
|
|
200
|
+
A deliberately missing installation exited 2 before inference and marked all
|
|
201
|
+
three scenarios blocked. All 153 offline tests passed, including 14 acceptance
|
|
202
|
+
checker/adapter tests. The package check passed locally on Node 26.13.0. CI for
|
|
203
|
+
`d9c99c4` passed Node 22/24/26, fixtures/tooling, Markdown and local links. Local
|
|
204
|
+
Markdown lint, lychee 0.24.2 offline links/anchors, citation validation and diff
|
|
205
|
+
checks also passed. No live inference was run by CI.
|
|
206
|
+
|
|
207
|
+
### Human acceptance — 8 November 2026
|
|
208
|
+
|
|
209
|
+
The user ran the documented command from clean commit
|
|
210
|
+
`77b1bc49108c2741e012cf1d6b5ca43e3cee97ab` on macOS with Node 22.23.2,
|
|
211
|
+
Codex CLI 0.166.0 and `gpt-6-astra`. The installed Researcher remained 0.1.9
|
|
212
|
+
at the pinned revision above. All three automated scenarios passed.
|
|
213
|
+
|
|
214
|
+
After reviewing the actual final responses and relevant tool activity presented
|
|
215
|
+
from that run, the user explicitly accepted **create, repeat setup and
|
|
216
|
+
fresh-session reopen: Pass**. The user also accepted the reopen turn's denied
|
|
217
|
+
attempt to read the unrelated Superpowers skill; its read returned exit 1 and
|
|
218
|
+
the Researcher inspection subsequently succeeded. Recorded metadata/note
|
|
219
|
+
inventories and global-settings fingerprints were unchanged through the
|
|
220
|
+
applicable checks. Scenario durations were 16.998, 10.737 and 19.492 seconds.
|
|
221
|
+
|
|
222
|
+
This is human acceptance of the reviewed run, separate from the earlier agent
|
|
223
|
+
rehearsals. A private acceptance receipt accompanies the original run evidence;
|
|
224
|
+
the raw Cucumber pending results remain unchanged as the historical automated
|
|
225
|
+
output. Future runs still require their own human review. The user authorized
|
|
226
|
+
PR #44 to become ready for review, not merge or release.
|
|
227
|
+
|
|
228
|
+
## Readable reports and recorded human verdicts
|
|
229
|
+
|
|
230
|
+
The follow-up in [#45](https://github.com/injectionator/n8r-surface/issues/45)
|
|
231
|
+
adds a private `review.md` after live-run finalization. It contains each actual
|
|
232
|
+
final response, user input, commentary, tool commands and results, including
|
|
233
|
+
failed or denied operations. Successful tool outputs and technical session
|
|
234
|
+
receipts are expandable in Markdown viewers supporting HTML details; their full
|
|
235
|
+
text remains in the file. Failed outputs remain visible. Missing evidence is
|
|
236
|
+
explicit. No model generates the report or judges the interaction.
|
|
237
|
+
|
|
238
|
+
The runner prints the report path and a SHA-256 **run fingerprint**. The companion
|
|
239
|
+
`review-manifest.json` binds the evidence file names/bytes and the report bytes.
|
|
240
|
+
These files contain private machine paths and transcripts: retain them with the
|
|
241
|
+
ignored run directory, not in public commits. Existing reports are verified,
|
|
242
|
+
never silently regenerated over different evidence.
|
|
243
|
+
|
|
244
|
+
To generate or verify a report for a retained run without inference:
|
|
245
|
+
|
|
246
|
+
```sh
|
|
247
|
+
npm run acceptance:skill -- --report --evidence .local/skill-acceptance/run-1
|
|
248
|
+
```
|
|
249
|
+
|
|
250
|
+
Expected: exit 0 when the report is successfully generated or verified, with the
|
|
251
|
+
report path, fingerprint and automated status. Exit 0 here means reporting worked,
|
|
252
|
+
not that the scenario passed. A blocked run can have a valid readable report.
|
|
253
|
+
Missing required evidence prevents an overall Pass. Once a verdict receipt exists,
|
|
254
|
+
this command also shows the three human verdicts and combined overall status.
|
|
255
|
+
|
|
256
|
+
### Human review task list
|
|
257
|
+
|
|
258
|
+
Prerequisite: a completed run from the command above or the live command. No new
|
|
259
|
+
Codex sign-in, network access or inference is needed for report/review commands.
|
|
260
|
+
From the repository root:
|
|
261
|
+
|
|
262
|
+
- [ ] Run `--report` as shown above and open the printed `review.md` path. Read
|
|
263
|
+
each scenario's human criteria, actual final response and tool activity.
|
|
264
|
+
Inspect any failed commands and preservation warnings. Expect the same actual
|
|
265
|
+
responses as the retained turn receipts; there are no invented summaries.
|
|
266
|
+
- [ ] Copy the report's run fingerprint. Choose **Pass**, **Fail** or **Blocked**
|
|
267
|
+
independently for create, repeat and reopen. The command below shows all Pass
|
|
268
|
+
only as an example; replace each verdict with your actual judgment, replace
|
|
269
|
+
the fingerprint and reviewer, and use the optional comment for any caveats:
|
|
270
|
+
|
|
271
|
+
```sh
|
|
272
|
+
npm run acceptance:skill -- --review \
|
|
273
|
+
--evidence .local/skill-acceptance/run-1 \
|
|
274
|
+
--run COPY_THE_64_CHARACTER_RUN_FINGERPRINT \
|
|
275
|
+
--reviewer 'Your name' \
|
|
276
|
+
--create Pass --repeat Pass --reopen Pass \
|
|
277
|
+
--comment 'Reviewed the actual responses and tool activity.'
|
|
278
|
+
```
|
|
279
|
+
|
|
280
|
+
Expect a separate `human-review.json` containing the run fingerprint, report
|
|
281
|
+
digest, reviewer, date/time, each verdict and comment. The terminal shows
|
|
282
|
+
automated status, human verdicts and overall status together. **Pass → exit 0;
|
|
283
|
+
Fail → exit 1; Blocked → exit 2.** A human Pass cannot override an automated
|
|
284
|
+
failure, final preservation error, missing evidence or blocked prerequisite.
|
|
285
|
+
- [ ] Run `--report` again. Expect verification of the unchanged evidence/report
|
|
286
|
+
and the recorded human verdicts. Original Cucumber JSON and pending results
|
|
287
|
+
remain untouched; a verdict receipt does not retroactively alter execution.
|
|
288
|
+
- [ ] Report **Pass / Fail / Blocked** for this review workflow with the private
|
|
289
|
+
run reference and any failed step. If commands refuse modified evidence, an
|
|
290
|
+
incorrect fingerprint or a duplicate review, that is the expected protection;
|
|
291
|
+
retain the refusal when reporting a problem. Do not publish raw private files.
|
|
292
|
+
|
|
293
|
+
Only one review receipt is supported per run in this increment. Duplicate
|
|
294
|
+
submissions refuse to overwrite it. A correction workflow, reviewer identity
|
|
295
|
+
verification and signed audit records are out of scope. A reviewer name is a
|
|
296
|
+
self-attribution, not authentication. Fingerprints detect drift relative to the
|
|
297
|
+
retained manifest; they do not make files tamper-proof against someone able to
|
|
298
|
+
rewrite both evidence and manifest. Evidence must be quiescent while reporting
|
|
299
|
+
and reviewing; concurrent writers are unsupported.
|
|
300
|
+
|
|
301
|
+
Owned-fixture cleanup still uses `--cleanup`. It retains reports and verdicts;
|
|
302
|
+
its later `cleanup.json` receipt is excluded from the evidence fingerprint. The
|
|
303
|
+
three generated review files are also excluded from that fingerprint, with
|
|
304
|
+
report bytes bound separately. All other evidence files are included. Symlinks,
|
|
305
|
+
nested entries and files over 64 MiB are refused. Historical manually written
|
|
306
|
+
acceptance receipts are preserved, not automatically imported as new approval.
|
|
307
|
+
|
|
308
|
+
Offline report/review checks:
|
|
309
|
+
|
|
310
|
+
```sh
|
|
311
|
+
node --test tests/skill-review.test.mjs
|
|
312
|
+
```
|
|
313
|
+
|
|
314
|
+
These tests use synthetic reviewer names and verdicts. They do not establish
|
|
315
|
+
human acceptance of any real run. Live execution retains its existing pending
|
|
316
|
+
exit behavior; `--review` is the separate acceptance step.
|
|
317
|
+
|
|
318
|
+
### Report/review development evidence — 8 November 2026
|
|
319
|
+
|
|
320
|
+
The observed baseline was the user's accepted #43 run: an agent had to extract
|
|
321
|
+
responses/tools from JSON for human review. The new offline report command was
|
|
322
|
+
exercised against that retained real run and a retained missing-installation run;
|
|
323
|
+
they reported automated Pass and Blocked respectively without new inference or
|
|
324
|
+
rewriting raw evidence. The earlier manual acceptance receipt was preserved and
|
|
325
|
+
was not imported as a new command-recorded review.
|
|
326
|
+
|
|
327
|
+
A fresh installed-skill run on macOS, Node 26.13.0, Codex CLI 0.166.0 and the
|
|
328
|
+
configured `gpt-6-astra` model passed all three automated scenarios and generated
|
|
329
|
+
the report automatically. The installed 0.1.9 plugin matched the same pinned
|
|
330
|
+
revision before and after. This was a development checkout with harness file
|
|
331
|
+
hashes retained privately, not a clean-commit or human acceptance claim.
|
|
332
|
+
|
|
333
|
+
The agent checked every final response and command against the readable report,
|
|
334
|
+
then exercised the documented review command using reviewer name
|
|
335
|
+
“Codex agent rehearsal (not human acceptance)” and an explicit rehearsal comment.
|
|
336
|
+
It recorded three Pass verdicts, exited 0 and displayed separate automated/human
|
|
337
|
+
results. Offline report verification still succeeded after owned-fixture cleanup;
|
|
338
|
+
raw results and all evidence remained retained. These verdicts test the recording
|
|
339
|
+
workflow; they are not the user's acceptance of this increment.
|
|
340
|
+
|
|
341
|
+
Focused offline checks cover changed/missing evidence, wrong fingerprints,
|
|
342
|
+
modified reports, invalid/duplicate reviews, blocked/failing aggregation, null or
|
|
343
|
+
empty required evidence, and interrupted-turn tool recovery from session receipts.
|
|
344
|
+
The latter two cases were found in fresh review, corrected and regression-tested.
|
|
345
|
+
All 171 offline tests passed, including 18 report/review cases; the installed
|
|
346
|
+
package check also passed on Node 26.13.0. No new dependencies or live CI were added.
|
|
347
|
+
|
|
348
|
+
A second fresh run exercised clean commit `1eb4cae` with the same environment
|
|
349
|
+
and installed skill. All three automated scenarios passed in 52.7 seconds;
|
|
350
|
+
live execution retained its pending-human status. The agent inspected the actual
|
|
351
|
+
responses, tool activity, preservation and isolation receipts, then recorded an
|
|
352
|
+
explicitly labeled agent rehearsal with three Pass verdicts (exit 0). Cleanup
|
|
353
|
+
and offline report verification both exited 0, preserving the exact fingerprint
|
|
354
|
+
and recorded verdicts. This does not establish desktop or human acceptance.
|
|
355
|
+
|
|
356
|
+
A later offline regression caught an evidence file named `__proto__` being omitted
|
|
357
|
+
from a plain JavaScript object's hash map. Null-prototype maps now include that
|
|
358
|
+
filename; changing its contents is detected. The final narrow fix was verified
|
|
359
|
+
by the offline suite rather than another live inference run.
|
|
@@ -0,0 +1,172 @@
|
|
|
1
|
+
# Human verification: bare startup (STARTUP-01)
|
|
2
|
+
|
|
3
|
+
Use a macOS or Linux terminal with Bash or zsh, Node.js **22, 24 or 26**, npm,
|
|
4
|
+
Python **3.9+**, Git and an authenticated GitHub CLI (`gh`). The recorded run used
|
|
5
|
+
macOS; this human procedure has not been rehearsed on Linux. No AI or Lab
|
|
6
|
+
credentials are needed.
|
|
7
|
+
|
|
8
|
+
Run the blocks below **in order, in the same terminal**, starting anywhere inside
|
|
9
|
+
an existing `injectionator/n8r-surface` checkout. Stop if a preparation command
|
|
10
|
+
fails. Installation downloads npm dependencies and creates ignored build files in
|
|
11
|
+
the checkout. The installed package, synthetic home, captured output and snapshot
|
|
12
|
+
all go in one temporary folder. Only the command under test receives that home.
|
|
13
|
+
|
|
14
|
+
## 1. Select PR #25 and run the automated checks
|
|
15
|
+
|
|
16
|
+
First run `git status --short`. **Expected: no output.** If it lists changes, keep
|
|
17
|
+
those changes and use your normal process to finish that work before switching.
|
|
18
|
+
Then run:
|
|
19
|
+
|
|
20
|
+
```sh
|
|
21
|
+
cd "$(git rev-parse --show-toplevel)"
|
|
22
|
+
gh pr checkout 25
|
|
23
|
+
git branch --show-current
|
|
24
|
+
git rev-parse HEAD
|
|
25
|
+
node --version
|
|
26
|
+
npm ci && npm cache add yaml@2.9.1 && npm test && npm run check:package
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
**Expected:** branch `feat/16-bare-startup`; record the printed commit for your
|
|
30
|
+
review. `node --version` must start with `v22.`, `v24.` or `v26.`. The final command
|
|
31
|
+
must exit **0**. Its output must show **43 tests passed, 0 failed, 0 skipped** for
|
|
32
|
+
`npm test`, then **20 tests passed, 0 failed, 0 skipped** for the installed startup
|
|
33
|
+
checks, plus these messages (Node's summary formatting varies by version):
|
|
34
|
+
|
|
35
|
+
```text
|
|
36
|
+
Installed package passed on Node v…; 4 nodes, 6 relationships, exact research references.
|
|
37
|
+
Installed PTY startup passed: overview before question, y and EOF, unchanged files.
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
Those checks drive a terminal automatically. Continue below to see and answer the
|
|
41
|
+
installed command yourself.
|
|
42
|
+
|
|
43
|
+
## 2. Install a disposable copy and prepare the starting state
|
|
44
|
+
|
|
45
|
+
```sh
|
|
46
|
+
n8r_verify=$(mktemp -d "${TMPDIR:-/tmp}/n8r-startup-review.XXXXXX")
|
|
47
|
+
n8r_verify=$(cd "$n8r_verify" && pwd -P)
|
|
48
|
+
npm pack --ignore-scripts --pack-destination "$n8r_verify"
|
|
49
|
+
npm install --prefix "$n8r_verify/install" --offline --ignore-scripts --no-audit --no-fund "$n8r_verify/n8r-surface-0.0.0-experimental.1.tgz"
|
|
50
|
+
mkdir -p "$n8r_verify/home/.n8r" "$n8r_verify/home/n8r/artifacts"
|
|
51
|
+
cat > "$n8r_verify/home/.n8r/n8r-surface.json" <<'JSON'
|
|
52
|
+
{
|
|
53
|
+
"version": 1,
|
|
54
|
+
"artefactRoot": "~/n8r/artifacts",
|
|
55
|
+
"surfaceFormat": "json",
|
|
56
|
+
"private": "PRIVATE_VALUE_MUST_NOT_APPEAR"
|
|
57
|
+
}
|
|
58
|
+
JSON
|
|
59
|
+
cat > "$n8r_verify/home/n8r/artifacts/librechat-uniqueid.json" <<'JSON'
|
|
60
|
+
{
|
|
61
|
+
"contract": "n8r.surface/0.1",
|
|
62
|
+
"id": "urn:surface:librechat-uniqueid",
|
|
63
|
+
"title": "LibreChat",
|
|
64
|
+
"scope": "Synthetic startup input; no research performed",
|
|
65
|
+
"revision": {
|
|
66
|
+
"id": "r1", "parent": null, "createdAt": "2026-11-04T00:00:00Z",
|
|
67
|
+
"author": "Synthetic author", "reason": "Human startup verification"
|
|
68
|
+
},
|
|
69
|
+
"extensions": {}, "kind": "definition", "records": [], "vocabularies": [],
|
|
70
|
+
"nodes": [], "relationships": [], "requirements": [], "coverage": [], "gaps": []
|
|
71
|
+
}
|
|
72
|
+
JSON
|
|
73
|
+
cp -R "$n8r_verify/home" "$n8r_verify/before"
|
|
74
|
+
printf 'Test home: %s\n' "$n8r_verify/home"
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
**Expected:** every preparation command exits **0**. The last line identifies the
|
|
78
|
+
home to expect in the overview. Your real `~/.n8r` and `~/n8r` are not used.
|
|
79
|
+
|
|
80
|
+
## 3. Run with closed input
|
|
81
|
+
|
|
82
|
+
```sh
|
|
83
|
+
n8r_exit=0
|
|
84
|
+
(cd "$n8r_verify" && env HOME="$n8r_verify/home" "$n8r_verify/install/node_modules/.bin/n8r-surface" </dev/null) >"$n8r_verify/closed.txt" 2>"$n8r_verify/stderr.txt" || n8r_exit=$?
|
|
85
|
+
cat "$n8r_verify/closed.txt"
|
|
86
|
+
printf 'Exit code: %s (expected 1)\n' "$n8r_exit"
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
**Expected:** return to the shell within **five seconds**, with no question or input
|
|
90
|
+
needed. If it is still waiting, press Ctrl-C and report a failed check. Output must
|
|
91
|
+
match the following; `<test-home>` is the path printed in step 2, and the positive
|
|
92
|
+
PID and UTC timestamp vary each run. Do not type the angle-bracket placeholders.
|
|
93
|
+
|
|
94
|
+
```text
|
|
95
|
+
Configuration
|
|
96
|
+
File: <test-home>/.n8r/n8r-surface.json
|
|
97
|
+
Settings version: 1
|
|
98
|
+
Artefact folder: <test-home>/n8r/artifacts
|
|
99
|
+
Surface output: JSON
|
|
100
|
+
AI runner: Not configured
|
|
101
|
+
Lab connection: Not configured
|
|
102
|
+
|
|
103
|
+
Processes
|
|
104
|
+
n8r-surface overview: running (PID <number>); observed <UTC timestamp>
|
|
105
|
+
Runner/Lab processes: Status unknown (no live status source)
|
|
106
|
+
|
|
107
|
+
Local artefacts (1)
|
|
108
|
+
LibreChat | Surface definition | <test-home>/n8r/artifacts/librechat-uniqueid.json
|
|
109
|
+
|
|
110
|
+
Setup and AI conversation are not implemented in this build. Research needs a supported AI runner.
|
|
111
|
+
Run in an interactive terminal for read-only setup guidance.
|
|
112
|
+
Exit code: 1 (expected 1)
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
There must be exactly one artefact row. **Exit 1 is expected** because setup and
|
|
116
|
+
conversation are unavailable in this increment; exit 0 or 2 fails this prepared
|
|
117
|
+
case. Check private-value omission and stderr with:
|
|
118
|
+
|
|
119
|
+
```sh
|
|
120
|
+
if grep -Fq 'PRIVATE_VALUE_MUST_NOT_APPEAR' "$n8r_verify/closed.txt" "$n8r_verify/stderr.txt"; then printf 'FAIL: private value leaked\n'; else printf 'PASS: private value absent\n'; fi
|
|
121
|
+
if test -s "$n8r_verify/stderr.txt"; then printf 'FAIL: unexpected stderr\n'; cat "$n8r_verify/stderr.txt"; else printf 'PASS: stderr empty\n'; fi
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
**Expected:** `PASS: private value absent` and `PASS: stderr empty`.
|
|
125
|
+
|
|
126
|
+
## 4. Answer the real terminal question
|
|
127
|
+
|
|
128
|
+
Paste this block, then wait for the question. Do not pipe it through `tee` or
|
|
129
|
+
redirect its input/output: the command must see a terminal.
|
|
130
|
+
|
|
131
|
+
```sh
|
|
132
|
+
n8r_exit=0; (cd "$n8r_verify" && env HOME="$n8r_verify/home" "$n8r_verify/install/node_modules/.bin/n8r-surface") || n8r_exit=$?; printf '\nExit code: %s (expected 1)\n' "$n8r_exit"
|
|
133
|
+
```
|
|
134
|
+
|
|
135
|
+
**Before typing anything**, confirm that the configuration, processes and the one
|
|
136
|
+
LibreChat row have appeared, followed by the unavailable-setup message and then
|
|
137
|
+
`Show setup requirements? [y/N]`. The private marker must be absent. The closed-input
|
|
138
|
+
message about running in an interactive terminal must not appear here.
|
|
139
|
+
|
|
140
|
+
Type **`y` and Enter**. Expect:
|
|
141
|
+
|
|
142
|
+
```text
|
|
143
|
+
Research will require an explicitly selected runner, model and external credential reference. A Lab connection is optional. No settings have been changed.
|
|
144
|
+
|
|
145
|
+
Exit code: 1 (expected 1)
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
Run the same block again. This time press **Ctrl-D at the empty question**. Expect
|
|
149
|
+
an immediate return to the shell, `Exit code: 1 (expected 1)`, and no guidance answer
|
|
150
|
+
or error stack. No setup or conversation is expected to start in either case.
|
|
151
|
+
|
|
152
|
+
## 5. Confirm unchanged files and clean up
|
|
153
|
+
|
|
154
|
+
```sh
|
|
155
|
+
if diff -r "$n8r_verify/before" "$n8r_verify/home"; then printf 'PASS: fixture files and directory contents unchanged\n'; else printf 'FAIL: startup changed the fixture\n'; fi
|
|
156
|
+
```
|
|
157
|
+
|
|
158
|
+
**Expected:** only `PASS: fixture files and directory contents unchanged`, with no
|
|
159
|
+
diff. If any step differs from its expectation, retain the temporary folder while
|
|
160
|
+
you investigate and report the step, commit and Node version; do not call it a pass.
|
|
161
|
+
After inspection, remove only the temporary verification folder:
|
|
162
|
+
|
|
163
|
+
```sh
|
|
164
|
+
rm -r -- "${n8r_verify:?}"
|
|
165
|
+
unset n8r_verify n8r_exit
|
|
166
|
+
git status --short
|
|
167
|
+
```
|
|
168
|
+
|
|
169
|
+
**Expected:** no output. The temporary installation, synthetic home, snapshot and
|
|
170
|
+
captured output are gone. Ignored `node_modules/` and `dist/` from the build remain
|
|
171
|
+
in the checkout. This procedure verifies bare startup only; it does not complete
|
|
172
|
+
\#16 or verify AI-driven research.
|