@nerviq/cli 1.26.0 → 1.27.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +1407 -0
- package/README.md +4 -4
- package/SECURITY.md +82 -0
- package/bin/cli.js +13 -1
- package/contracts/audit-webhook-event.schema.json +138 -0
- package/contracts/pack-contract.schema.json +15 -0
- package/contracts/technique-contract.schema.json +18 -0
- package/docs/ARCHITECTURE.md +74 -0
- package/docs/api-reference.md +356 -0
- package/docs/autofix.md +64 -0
- package/docs/bitbucket-pipe.yml +57 -0
- package/docs/case-studies.md +149 -0
- package/docs/category-definition-kit.md +56 -0
- package/docs/ci-integration.md +127 -0
- package/docs/claude-code-style.md +24 -0
- package/docs/claude-maintainer-ops.md +19 -0
- package/docs/external-validation.md +78 -0
- package/docs/first-tier-integration-gate.md +59 -0
- package/docs/getting-started.md +119 -0
- package/docs/gitlab-ci-template.yml +54 -0
- package/docs/index.html +597 -0
- package/docs/integration-contracts.md +287 -0
- package/docs/license-faq.md +53 -0
- package/docs/maintenance.md +155 -0
- package/docs/methodology.md +236 -0
- package/docs/new-platform-guide.md +202 -0
- package/docs/open-vsx-publishing.md +46 -0
- package/docs/platform-change-ingestion.md +46 -0
- package/docs/plugins.md +101 -0
- package/docs/pre-commit.md +58 -0
- package/docs/security-model.md +63 -0
- package/docs/shallow-risk.md +246 -0
- package/docs/versioning-policy.md +63 -0
- package/docs/why-nerviq.md +82 -0
- package/package.json +7 -2
- package/sdk/README.md +190 -0
- package/src/audit/layers.js +180 -179
- package/src/audit.js +118 -48
- package/src/codex/setup.js +3 -2
- package/src/formatters/csv.js +86 -85
- package/src/formatters/junit.js +123 -103
- package/src/formatters/markdown.js +164 -135
- package/src/gemini/setup.js +3 -2
- package/src/init.js +4 -3
- package/src/opencode/context.js +42 -3
- package/src/opencode/techniques.js +198 -142
- package/src/output-icons.js +44 -0
- package/src/setup/runtime.js +6 -5
- package/src/setup.js +4 -3
- package/src/shallow-risk/index.js +56 -0
- package/src/shallow-risk/patterns/agent-config-cross-platform-drift.js +50 -0
- package/src/shallow-risk/patterns/agent-config-dangerous-autoapprove.js +46 -0
- package/src/shallow-risk/patterns/agent-config-deprecated-keys.js +46 -0
- package/src/shallow-risk/patterns/agent-config-missing-file.js +72 -0
- package/src/shallow-risk/patterns/agent-config-secret-literal.js +49 -0
- package/src/shallow-risk/patterns/agent-config-stack-contradiction.js +34 -0
- package/src/shallow-risk/patterns/hook-script-missing.js +70 -0
- package/src/shallow-risk/patterns/mcp-server-no-allowlist.js +52 -0
- package/src/shallow-risk/shared.js +520 -0
|
@@ -0,0 +1,287 @@
|
|
|
1
|
+
# Integration Contract Pack
|
|
2
|
+
|
|
3
|
+
This document centralizes Nerviq's integration surfaces so external systems can adopt one stable package instead of scraping CLI text output.
|
|
4
|
+
|
|
5
|
+
## Included contract surfaces
|
|
6
|
+
|
|
7
|
+
- `nerviq serve` live OpenAPI contract via `GET /api/openapi.json`
|
|
8
|
+
- `nerviq-mcp` stdio JSON-RPC 2.0 transport for MCP hosts
|
|
9
|
+
- Generic audit webhook event contract via `contracts/audit-webhook-event.schema.json`
|
|
10
|
+
- CI reference patterns for GitHub Actions, GitLab, Bitbucket, and generic shell runners
|
|
11
|
+
- SDK usage examples through `sdk/README.md`
|
|
12
|
+
|
|
13
|
+
## 1. Local REST contract
|
|
14
|
+
|
|
15
|
+
Start the local API server:
|
|
16
|
+
|
|
17
|
+
```bash
|
|
18
|
+
npx @nerviq/cli serve --port 3000
|
|
19
|
+
curl http://127.0.0.1:3000/api/openapi.json > nerviq-openapi.json
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
Current GET endpoints:
|
|
23
|
+
|
|
24
|
+
- `/api/openapi.json`
|
|
25
|
+
- `/api/health`
|
|
26
|
+
- `/api/audit`
|
|
27
|
+
- `/api/harmony`
|
|
28
|
+
- `/api/catalog`
|
|
29
|
+
|
|
30
|
+
The live OpenAPI document is the canonical machine-readable contract for local HTTP consumers.
|
|
31
|
+
|
|
32
|
+
## 2. MCP transport (stdio JSON-RPC 2.0)
|
|
33
|
+
|
|
34
|
+
Nerviq also ships a separate MCP server binary for hosts that speak Model Context Protocol over stdio.
|
|
35
|
+
|
|
36
|
+
Use `nerviq serve` for local HTTP/OpenAPI consumers.
|
|
37
|
+
Use `nerviq-mcp` when the host expects MCP over stdio.
|
|
38
|
+
|
|
39
|
+
Example host registration:
|
|
40
|
+
|
|
41
|
+
```json
|
|
42
|
+
{
|
|
43
|
+
"mcpServers": {
|
|
44
|
+
"nerviq": {
|
|
45
|
+
"command": "npx",
|
|
46
|
+
"args": ["-y", "-p", "@nerviq/cli", "nerviq-mcp"]
|
|
47
|
+
}
|
|
48
|
+
}
|
|
49
|
+
}
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
The runtime implementation lives in `src/mcp-server.js` and speaks JSON-RPC 2.0 over stdin/stdout.
|
|
53
|
+
|
|
54
|
+
## 3. Generic webhook event contract
|
|
55
|
+
|
|
56
|
+
When `audit` is called with `--webhook` and the URL is not a Slack or Discord webhook, Nerviq now emits a stable generic JSON event contract.
|
|
57
|
+
|
|
58
|
+
Schema:
|
|
59
|
+
|
|
60
|
+
- `contracts/audit-webhook-event.schema.json`
|
|
61
|
+
|
|
62
|
+
Example:
|
|
63
|
+
|
|
64
|
+
```json
|
|
65
|
+
{
|
|
66
|
+
"event": "nerviq.audit.completed",
|
|
67
|
+
"schemaVersion": "1.0",
|
|
68
|
+
"generatedAt": "2026-04-09T12:00:00.000Z",
|
|
69
|
+
"platform": "claude",
|
|
70
|
+
"score": 84,
|
|
71
|
+
"passed": 196,
|
|
72
|
+
"failed": 34,
|
|
73
|
+
"results": [],
|
|
74
|
+
"data": {
|
|
75
|
+
"platform": "claude",
|
|
76
|
+
"platformLabel": "Claude",
|
|
77
|
+
"score": 84,
|
|
78
|
+
"scoreType": "live-audit-score",
|
|
79
|
+
"organicScore": 68,
|
|
80
|
+
"passed": 196,
|
|
81
|
+
"failed": 34,
|
|
82
|
+
"skipped": 28,
|
|
83
|
+
"checkCount": 258,
|
|
84
|
+
"topNextActions": [],
|
|
85
|
+
"quickWins": [],
|
|
86
|
+
"scoreCoaching": {
|
|
87
|
+
"currentScore": 84,
|
|
88
|
+
"nextMilestone": 90,
|
|
89
|
+
"pointsNeeded": 6,
|
|
90
|
+
"fixesNeeded": 2
|
|
91
|
+
},
|
|
92
|
+
"suggestedNextCommand": "npx nerviq fix verificationLoop"
|
|
93
|
+
},
|
|
94
|
+
"meta": {
|
|
95
|
+
"cliVersion": "1.27.1",
|
|
96
|
+
"source": "nerviq-cli",
|
|
97
|
+
"webhookFormat": "generic-audit-event"
|
|
98
|
+
}
|
|
99
|
+
}
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
Compatibility note:
|
|
103
|
+
|
|
104
|
+
- Top-level `platform`, `score`, `passed`, `failed`, and `results` remain present for older consumers
|
|
105
|
+
- New consumers should prefer the nested `data` + `meta` contract
|
|
106
|
+
|
|
107
|
+
## 4. CI reference patterns
|
|
108
|
+
|
|
109
|
+
See:
|
|
110
|
+
|
|
111
|
+
- `docs/ci-integration.md`
|
|
112
|
+
- `action/README.md`
|
|
113
|
+
- `docs/gitlab-ci-template.yml`
|
|
114
|
+
- `docs/bitbucket-pipe.yml`
|
|
115
|
+
|
|
116
|
+
Recommended patterns:
|
|
117
|
+
|
|
118
|
+
### Score gate
|
|
119
|
+
|
|
120
|
+
```bash
|
|
121
|
+
npx @nerviq/cli audit --threshold 60
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
### PR drift gate
|
|
125
|
+
|
|
126
|
+
```bash
|
|
127
|
+
npx @nerviq/cli audit --diff-only --drift-mode ci --threshold 60
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
### Fleet rollup artifact
|
|
131
|
+
|
|
132
|
+
```bash
|
|
133
|
+
npx @nerviq/cli org scan ./app ./api ./infra --json --out nerviq-fleet.json
|
|
134
|
+
```
|
|
135
|
+
|
|
136
|
+
## 5. SDK reference
|
|
137
|
+
|
|
138
|
+
See:
|
|
139
|
+
|
|
140
|
+
- `sdk/README.md`
|
|
141
|
+
- `docs/api-reference.md`
|
|
142
|
+
|
|
143
|
+
Recommended SDK surfaces:
|
|
144
|
+
|
|
145
|
+
- Stable: `audit`, `harmonyAudit`, `detectPlatforms`, `getCatalog`
|
|
146
|
+
- Experimental: `synergyReport`, `routeTask`
|
|
147
|
+
|
|
148
|
+
## Why this pack exists
|
|
149
|
+
|
|
150
|
+
- External dashboards can bind to explicit contracts instead of scraping CLI text
|
|
151
|
+
- CI systems can use repeatable patterns for threshold gates, drift gates, and fleet rollups
|
|
152
|
+
- SDK users and HTTP users now share one public integration story instead of separate ad-hoc examples
|
|
153
|
+
|
|
154
|
+
## 6. First-tier integration gate
|
|
155
|
+
|
|
156
|
+
Nerviq does not treat every external surface as equally ready on day one.
|
|
157
|
+
|
|
158
|
+
Before broader distribution surfaces such as GitHub Marketplace listings, JetBrains plugins, or similar first-tier integrations are unblocked, the release bar must be green across:
|
|
159
|
+
|
|
160
|
+
- contract stability
|
|
161
|
+
- public proof density
|
|
162
|
+
- operational reliability
|
|
163
|
+
- clear ownership and support posture
|
|
164
|
+
- category fit
|
|
165
|
+
|
|
166
|
+
See `docs/first-tier-integration-gate.md` for the explicit gate and current posture.
|
|
167
|
+
|
|
168
|
+
## 7. CI output format contracts
|
|
169
|
+
|
|
170
|
+
`nerviq audit --format=<name>` emits machine-readable output shapes for standard
|
|
171
|
+
CI surfaces. All six formats share the same underlying audit result object; they
|
|
172
|
+
differ only in rendering.
|
|
173
|
+
|
|
174
|
+
### markdown (`--format=markdown`)
|
|
175
|
+
|
|
176
|
+
GitHub-flavoured markdown suitable for a PR comment. Contract:
|
|
177
|
+
|
|
178
|
+
- first line is an `## Score: N/100` heading followed by a shields.io badge
|
|
179
|
+
- second block lists `**Platform:**` and `**Checks:**` summary
|
|
180
|
+
- `### Top next actions` section with up to 5 items rendered as a GitHub
|
|
181
|
+
task-list checklist: `- [ ] **[SEVERITY] Title** (` + backtick key + `)`
|
|
182
|
+
optionally suffixed with ` — ` + backtick file:line
|
|
183
|
+
- when `auditResult.shallowRiskHints` is present, a `### Shallow Risk (experimental, opt-in)`
|
|
184
|
+
section appears immediately after `### Top next actions`, with the
|
|
185
|
+
shallow-risk banner rendered as a blockquote and each hint listed with
|
|
186
|
+
severity, key, and file:line
|
|
187
|
+
- `<details><summary>All failed checks (N)</summary>` collapsible block
|
|
188
|
+
containing a GitHub pipe-table with columns
|
|
189
|
+
`key | name | category | rating | file | line`. Pipe characters inside cells
|
|
190
|
+
are backslash-escaped.
|
|
191
|
+
- footer line `Generated by [Nerviq](https://nerviq.net) vX.Y.Z · TIMESTAMP`
|
|
192
|
+
- no raw HTML other than `<details>` / `<summary>`
|
|
193
|
+
|
|
194
|
+
### junit (`--format=junit`)
|
|
195
|
+
|
|
196
|
+
JUnit XML, Jenkins-compatible. Contract:
|
|
197
|
+
|
|
198
|
+
- first line is `<?xml version="1.0" encoding="UTF-8"?>`
|
|
199
|
+
- root element `<testsuites name="nerviq" tests="N" failures="F" skipped="S" time="0">`
|
|
200
|
+
- one `<testsuite>` per check category, with attributes
|
|
201
|
+
`name=category`, `tests`, `failures`, `skipped`, `time`, `timestamp`,
|
|
202
|
+
`package="nerviq.PLATFORM"`
|
|
203
|
+
- when `auditResult.shallowRiskHints` is present, a separate
|
|
204
|
+
`<testsuite name="shallow-risk">` is emitted so CI consumers can
|
|
205
|
+
filter it independently from governance scoring suites
|
|
206
|
+
- one `<testcase>` per check, with `classname=category` and `name=key`
|
|
207
|
+
- failed checks emit `<failure message="..." type="SEVERITY">BODY</failure>`
|
|
208
|
+
where message = `fix || name`, type = severity/impact, body contains
|
|
209
|
+
`name [at file:line] [(sourceUrl)]`
|
|
210
|
+
- skipped checks emit `<skipped/>`
|
|
211
|
+
- all attribute values and text nodes XML-escape `& < > " '`
|
|
212
|
+
|
|
213
|
+
### csv (`--format=csv`)
|
|
214
|
+
|
|
215
|
+
RFC 4180 CSV. Contract:
|
|
216
|
+
|
|
217
|
+
- first row = header `key,id,name,category,layer,rating,severity,passed,file,line,sourceUrl,fix,projectedScoreDelta,projectedScoreAfter`
|
|
218
|
+
- `layer` is one of `governance`, `drift`, `hygiene`, or `shallow-risk` (see §8)
|
|
219
|
+
- one row per check result in `auditResult.results`
|
|
220
|
+
- when `auditResult.shallowRiskHints` is present, those hints are
|
|
221
|
+
appended as additional rows tagged with `layer=shallow-risk`
|
|
222
|
+
- `projectedScoreDelta` / `projectedScoreAfter` are populated only for rows whose
|
|
223
|
+
`key` appears in `auditResult.topNextActions` (the projection is computed per
|
|
224
|
+
top action, not per every check). Other rows leave both columns empty.
|
|
225
|
+
- fields containing comma, double-quote, CR, or LF are wrapped in double-quotes
|
|
226
|
+
- internal double-quotes are escaped by doubling (`""`)
|
|
227
|
+
- no UTF-8 BOM
|
|
228
|
+
- line separator = LF
|
|
229
|
+
|
|
230
|
+
These contracts are the stable consumer API for `--format=markdown|junit|csv`.
|
|
231
|
+
Downstream wrappers (GitHub Actions, Jenkins plugins, GitLab reporters,
|
|
232
|
+
dashboards) should bind to these shapes rather than scraping the text output.
|
|
233
|
+
|
|
234
|
+
## 8. Audit layers (scope taxonomy)
|
|
235
|
+
|
|
236
|
+
Every check in the NERVIQ audit is tagged with exactly one `layer`. The
|
|
237
|
+
layer answers the question "what kind of problem does this check look
|
|
238
|
+
for?" and gives evaluators an explicit, stable map of what NERVIQ covers
|
|
239
|
+
and what it deliberately does not.
|
|
240
|
+
|
|
241
|
+
| Layer | Short definition | Example checks |
|
|
242
|
+
| -------------- | ------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- |
|
|
243
|
+
| `governance` | Agent configuration posture: presence, content, and quality of agent-instruction files and platform settings. "Does my agent know X?" | `claudeMdExists`, `codexAgentsMdSubstantive`, `geminiSettingsExists`, MCP servers, hook presence |
|
|
244
|
+
| `drift` | Cross-platform consistency and declared-vs-actual alignment. "Do two places agree on X?" | Harmony drift, Gemini propagation completeness, "rules consistent across surfaces", pack agreement |
|
|
245
|
+
| `hygiene` | Repo-level cleanliness adjacent to agents — the engineering baseline that makes an agent's job easier. | `.gitignore`, `CHANGELOG`, `SECURITY.md`, `LICENSE`, `.editorconfig`, Node version pinning |
|
|
246
|
+
| `shallow-risk` | Opt-in boundary checks that sit at the agent-config <-> codebase edge and are emitted in parallel to the governance audit. | `agent-config-missing-file`, `mcp-server-no-allowlist`, `agent-config-dangerous-autoapprove` |
|
|
247
|
+
|
|
248
|
+
**Disambiguation rules** (apply when a check could plausibly fit more than one layer):
|
|
249
|
+
|
|
250
|
+
- If the check asks "does my agent know X?" → `governance`.
|
|
251
|
+
- If the check asks "do two places agree on X?" → `drift`.
|
|
252
|
+
- If the check asks "does the repo have standard engineering hygiene that makes the agent's job easier?" → `hygiene`.
|
|
253
|
+
- When in doubt, default to `hygiene`. A mild misclassification is recoverable; a missing tag breaks the coverage contract.
|
|
254
|
+
|
|
255
|
+
**No deep-review lane.** There is intentionally no `deep-review`,
|
|
256
|
+
`security`, or `code-review` layer. NERVIQ audits agent configuration and
|
|
257
|
+
the cleanliness of the repo boundary an agent operates inside. It does
|
|
258
|
+
not perform dataflow analysis, SAST, or general code review — those are
|
|
259
|
+
out of scope and left to dedicated tools. This is the contract that
|
|
260
|
+
lets evaluators know where our claim to ground-truth starts and stops.
|
|
261
|
+
|
|
262
|
+
### 8.1 Shallow-risk as a parallel lane
|
|
263
|
+
|
|
264
|
+
`shallow-risk` is the one layer that does **not** roll into the governance
|
|
265
|
+
score. When enabled via `nerviq audit --shallow-risk`, findings are emitted
|
|
266
|
+
through `auditResult.shallowRiskHints[]` and rendered separately in text,
|
|
267
|
+
Markdown, JUnit, and CSV. They are intentionally excluded from:
|
|
268
|
+
|
|
269
|
+
- `auditResult.score`
|
|
270
|
+
- `auditResult.organicScore`
|
|
271
|
+
- `auditResult.passed` / `failed` / `skipped`
|
|
272
|
+
- `auditResult.topNextActions`
|
|
273
|
+
- `auditResult.layerSummary.*.failed`
|
|
274
|
+
|
|
275
|
+
This keeps the governance pipeline stable while still surfacing the
|
|
276
|
+
agent-config <-> codebase red flags documented in [docs/shallow-risk.md](shallow-risk.md).
|
|
277
|
+
|
|
278
|
+
**Surfaces.** The `layer` field appears in:
|
|
279
|
+
|
|
280
|
+
- `auditResult.results[].layer` (JSON)
|
|
281
|
+
- `auditResult.topNextActions[].layer` (JSON)
|
|
282
|
+
- `auditResult.layerSummary` (JSON) — per-layer `{ total, passed, failed, skipped }`
|
|
283
|
+
- CSV output — column between `category` and `rating`
|
|
284
|
+
- JUnit output — attribute on each `<testcase layer="…">`
|
|
285
|
+
- Markdown output — column in the failed-checks table plus a `_layer: X_` suffix on each top-action checklist item
|
|
286
|
+
- Text output — "Coverage by layer:" summary block and a small `[layer]` prefix on failed-check names
|
|
287
|
+
|
|
@@ -0,0 +1,53 @@
|
|
|
1
|
+
# Nerviq License FAQ
|
|
2
|
+
|
|
3
|
+
Nerviq is licensed under **AGPL-3.0** (GNU Affero General Public License v3.0).
|
|
4
|
+
|
|
5
|
+
This FAQ answers common questions about what you can and can't do.
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## What can I do freely?
|
|
10
|
+
|
|
11
|
+
**Use the CLI locally** — Run `nerviq audit`, `setup`, `fix`, `benchmark`, etc. on your own machine or CI pipeline. No restrictions.
|
|
12
|
+
|
|
13
|
+
**Use it in CI/CD** — Add the GitHub Action or run `npx @nerviq/cli` in your pipeline. The output (scores, reports) is yours.
|
|
14
|
+
|
|
15
|
+
**Fork and modify** — You can fork the repo, modify it, and use your fork internally.
|
|
16
|
+
|
|
17
|
+
**Use the npm package** — Install `@nerviq/cli` globally or as a dev dependency. Normal usage is fine.
|
|
18
|
+
|
|
19
|
+
## What triggers AGPL obligations?
|
|
20
|
+
|
|
21
|
+
**Running Nerviq as a network service for others.** If you modify Nerviq and offer it as a hosted service (e.g., a SaaS dashboard that runs Nerviq audits for external users over a network), you must make your modified source code available to those users under AGPL-3.0.
|
|
22
|
+
|
|
23
|
+
This is the key difference from regular GPL: AGPL extends the source-sharing requirement to users who interact with the software over a network, not just those who receive a copy.
|
|
24
|
+
|
|
25
|
+
## Common scenarios
|
|
26
|
+
|
|
27
|
+
| Scenario | AGPL obligation? |
|
|
28
|
+
|----------|-----------------|
|
|
29
|
+
| Developer runs `nerviq audit` locally | **No** |
|
|
30
|
+
| CI pipeline runs `nerviq audit --threshold 60` | **No** |
|
|
31
|
+
| Team uses nerviq internally on their repos | **No** |
|
|
32
|
+
| Company wraps nerviq in an internal tool for their own devs | **No** |
|
|
33
|
+
| Company offers "nerviq-as-a-service" to external customers | **Yes** — must share modified source |
|
|
34
|
+
| Embedding nerviq checks in a paid SaaS product | **Yes** — must share modified source |
|
|
35
|
+
| Using nerviq output (scores, reports) in your own product | **No** — output is data, not software |
|
|
36
|
+
|
|
37
|
+
## What about the SDK (`@nerviq/sdk`)?
|
|
38
|
+
|
|
39
|
+
The SDK is also AGPL-3.0. If you import it into your application and serve that application to users over a network, the AGPL obligation applies to the combined work.
|
|
40
|
+
|
|
41
|
+
## What if I need a commercial license?
|
|
42
|
+
|
|
43
|
+
Contact us at **hello@nerviq.net** to discuss commercial licensing options. We offer:
|
|
44
|
+
- **Enterprise license** — use Nerviq without AGPL obligations
|
|
45
|
+
- **OEM license** — embed Nerviq in your own product
|
|
46
|
+
|
|
47
|
+
## Key points
|
|
48
|
+
|
|
49
|
+
- **Local CLI usage is always fine** — no obligations
|
|
50
|
+
- **CI usage is always fine** — no obligations
|
|
51
|
+
- **Internal tools are fine** — no obligations
|
|
52
|
+
- **Hosting for others triggers AGPL** — share your source
|
|
53
|
+
- **Output data is yours** — scores, reports, JSON are not covered by AGPL
|
|
@@ -0,0 +1,155 @@
|
|
|
1
|
+
# Knowledge Maintenance Process
|
|
2
|
+
|
|
3
|
+
How to keep Nerviq's knowledge base accurate, current, and trustworthy.
|
|
4
|
+
|
|
5
|
+
## Maintenance Cadences
|
|
6
|
+
|
|
7
|
+
### Daily: Freshness Daemon
|
|
8
|
+
|
|
9
|
+
The freshness system runs automatically and flags checks whose source documentation may have changed.
|
|
10
|
+
|
|
11
|
+
**Canonical source of truth:**
|
|
12
|
+
- `src/platform-change-manifest.js` centralizes tracked sources, review cadence, workflow details, and update triggers for all 8 platforms.
|
|
13
|
+
- `.github/workflows/freshness-check.yml` is the live automation path that runs the daily freshness review and opens stale-source issues.
|
|
14
|
+
|
|
15
|
+
**What to do:**
|
|
16
|
+
- Review `nerviq doctor` output for freshness warnings
|
|
17
|
+
- Review `nerviq doctor` output for broken MCP declarations (missing commands, unreachable URLs, missing env vars)
|
|
18
|
+
- Review `nerviq doctor` output for broken hook runtime health (missing scripts, missing runtimes, or starter hook regressions)
|
|
19
|
+
- Review `.claude/logs/prompt-injection-alerts.log` if the starter `injection-defense` hook reports suspicious external-content patterns during normal development
|
|
20
|
+
- If a check's `sourceUrl` returns 404 or has changed significantly, mark for review
|
|
21
|
+
- Update `confidence` values for any checks with degraded sources
|
|
22
|
+
|
|
23
|
+
**Automation:**
|
|
24
|
+
```bash
|
|
25
|
+
npx @nerviq/cli doctor --verbose # Shows freshness gates + MCP + hook runtime validation
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
**Programmatic manifest:**
|
|
29
|
+
```js
|
|
30
|
+
const { getPlatformChangeManifest, summarizePlatformChangeManifest } = require('@nerviq/cli');
|
|
31
|
+
|
|
32
|
+
console.log(summarizePlatformChangeManifest());
|
|
33
|
+
console.log(getPlatformChangeManifest()[0].trackedSources);
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
### Weekly: Platform Changelog Review
|
|
37
|
+
|
|
38
|
+
Each supported platform publishes updates that may affect Nerviq's checks.
|
|
39
|
+
|
|
40
|
+
**Process:**
|
|
41
|
+
1. Check official changelogs/release notes for all 8 platforms:
|
|
42
|
+
- Claude Code: Anthropic changelog
|
|
43
|
+
- Codex: OpenAI platform updates
|
|
44
|
+
- Gemini CLI: Google AI Studio releases
|
|
45
|
+
- GitHub Copilot: GitHub blog/changelog
|
|
46
|
+
- Cursor: Cursor changelog
|
|
47
|
+
- Windsurf: Codeium/Windsurf releases
|
|
48
|
+
- Aider: GitHub releases
|
|
49
|
+
- OpenCode: GitHub releases
|
|
50
|
+
|
|
51
|
+
2. For each relevant change:
|
|
52
|
+
- Does it add a new config option? → Candidate for new check
|
|
53
|
+
- Does it deprecate a feature? → Update or retire affected checks
|
|
54
|
+
- Does it change file formats? → Update config-parser.js
|
|
55
|
+
|
|
56
|
+
3. Log findings in `research/` with date stamp
|
|
57
|
+
4. If the change affects rollout posture or config semantics, update `src/platform-change-manifest.js` so cadence + triggers stay aligned with reality
|
|
58
|
+
|
|
59
|
+
### Monthly: Cross-Reference Audit
|
|
60
|
+
|
|
61
|
+
Verify internal consistency across the knowledge base.
|
|
62
|
+
|
|
63
|
+
**Process:**
|
|
64
|
+
1. Run the cross-reference audit:
|
|
65
|
+
```bash
|
|
66
|
+
# From the NERVIQ research repo
|
|
67
|
+
python tools/check-trust-drift.py
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
2. Check for:
|
|
71
|
+
- Duplicate check IDs across platforms
|
|
72
|
+
- Checks with confidence below 0.7 (candidates for re-verification)
|
|
73
|
+
- Techniques referenced in research but missing from code
|
|
74
|
+
- Techniques in code with no corresponding research doc
|
|
75
|
+
|
|
76
|
+
3. Reconcile any mismatches:
|
|
77
|
+
- Update source URLs that have moved
|
|
78
|
+
- Re-verify claims that have contradicting evidence
|
|
79
|
+
- Remove techniques that are no longer applicable
|
|
80
|
+
|
|
81
|
+
### Quarterly: Pruning
|
|
82
|
+
|
|
83
|
+
Remove or archive knowledge that is no longer relevant.
|
|
84
|
+
|
|
85
|
+
**Process:**
|
|
86
|
+
1. Query all checks by `lastVerified` date
|
|
87
|
+
2. Identify checks not verified in 90+ days
|
|
88
|
+
3. For each stale check:
|
|
89
|
+
- **Re-verify**: Visit sourceUrl, test the check, update confidence
|
|
90
|
+
- **Archive**: If the feature no longer exists, move to `_archived/`
|
|
91
|
+
- **Retire**: If the platform deprecated the feature, remove the check and note in CHANGELOG
|
|
92
|
+
|
|
93
|
+
4. Review research documents older than 90 days:
|
|
94
|
+
- Still accurate? → Update `lastVerified` date
|
|
95
|
+
- Outdated? → Mark as `status: archived` in frontmatter
|
|
96
|
+
- Contradicted by newer findings? → Create superseding document
|
|
97
|
+
|
|
98
|
+
5. Audit experiment results:
|
|
99
|
+
- Re-run 5 random experiments to confirm results still hold
|
|
100
|
+
- Update ratings if behavior has changed
|
|
101
|
+
|
|
102
|
+
## The 90-Day Rule
|
|
103
|
+
|
|
104
|
+
Every piece of knowledge in Nerviq has a maximum freshness window of 90 days.
|
|
105
|
+
|
|
106
|
+
**What this means:**
|
|
107
|
+
- After 90 days without re-verification, a check's confidence is automatically degraded
|
|
108
|
+
- Research documents older than 90 days without updates are flagged in `nerviq doctor`
|
|
109
|
+
- Stale checks are not removed automatically but are surfaced in audit output with warnings
|
|
110
|
+
|
|
111
|
+
**How to handle stale checks:**
|
|
112
|
+
|
|
113
|
+
1. **Check still valid, source unchanged:**
|
|
114
|
+
Update `lastVerified` timestamp. No code change needed.
|
|
115
|
+
|
|
116
|
+
2. **Check valid but source URL changed:**
|
|
117
|
+
Update `sourceUrl` in the technique definition. Update `lastVerified`.
|
|
118
|
+
|
|
119
|
+
3. **Check partially valid — behavior changed:**
|
|
120
|
+
Update the check function, fix text, confidence level. Create a research note documenting the change.
|
|
121
|
+
|
|
122
|
+
4. **Check no longer valid — feature removed:**
|
|
123
|
+
Remove from `techniques.js`. Add entry to CHANGELOG. If other checks depend on it, update those too.
|
|
124
|
+
|
|
125
|
+
5. **Check cannot be verified — source unavailable:**
|
|
126
|
+
Lower confidence to 0.5. Add a `TODO` comment. Flag for next weekly review.
|
|
127
|
+
|
|
128
|
+
## Freshness Tracking Fields
|
|
129
|
+
|
|
130
|
+
Each technique in Nerviq supports these freshness-related fields:
|
|
131
|
+
|
|
132
|
+
| Field | Type | Purpose |
|
|
133
|
+
|-------|------|---------|
|
|
134
|
+
| `sourceUrl` | string | Primary documentation URL |
|
|
135
|
+
| `confidence` | number (0.0-1.0) | How confident we are the check is correct |
|
|
136
|
+
| `lastVerified` | ISO date string | When the check was last manually verified |
|
|
137
|
+
| `addedVersion` | string | Nerviq version when the check was added |
|
|
138
|
+
|
|
139
|
+
## Emergency Updates
|
|
140
|
+
|
|
141
|
+
When a platform ships a breaking change:
|
|
142
|
+
|
|
143
|
+
1. Create a research doc immediately: `research/{platform}-breaking-change-YYYY-MM-DD.md`
|
|
144
|
+
2. Update affected techniques in the same session
|
|
145
|
+
3. Run full check matrix: `node test/{platform}-check-matrix.js`
|
|
146
|
+
4. Run golden matrix to detect output changes: `node test/{platform}-golden-matrix.js`
|
|
147
|
+
5. Publish patch release if checks produce incorrect results
|
|
148
|
+
|
|
149
|
+
## Metrics to Track
|
|
150
|
+
|
|
151
|
+
- **Freshness coverage**: % of checks verified within 90 days
|
|
152
|
+
- **Confidence distribution**: histogram of confidence values across all checks
|
|
153
|
+
- **Stale check count**: number of checks past 90-day window
|
|
154
|
+
- **Source availability**: % of sourceUrls returning 200
|
|
155
|
+
- **Platform coverage**: checks per platform (target: balanced distribution)
|