archsutra-ai 0.1.2 → 0.1.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +21 -0
- package/README.md +25 -17
- package/dist/audit.js +5 -2
- package/dist/cli.js +116 -12
- package/dist/drift.js +57 -0
- package/dist/imports.js +42 -0
- package/dist/map.js +13 -1
- package/dist/mcp.js +81 -0
- package/dist/scaffold.js +87 -0
- package/dist/suppress.js +51 -0
- package/docs/architecture/overview.html +3 -3
- package/docs/architecture/threat-model.html +3 -3
- package/docs/community/good-first-issues.html +3 -3
- package/docs/compatibility/matrix.html +3 -3
- package/docs/compatibility/smoke-tests/index.html +3 -3
- package/docs/compatibility/smoke-tests/results-2026-11-07.md +17 -0
- package/docs/decisions/ADR-0001-deterministic-cli-plus-ide-skills.html +3 -3
- package/docs/handbook/01-executive-summary.html +5 -5
- package/docs/handbook/01-executive-summary.md +2 -2
- package/docs/handbook/02-functional-specification.html +5 -5
- package/docs/handbook/02-functional-specification.md +2 -2
- package/docs/handbook/03-technical-specification.html +4 -4
- package/docs/handbook/03-technical-specification.md +1 -1
- package/docs/handbook/04-architecture.html +4 -4
- package/docs/handbook/04-architecture.md +1 -1
- package/docs/handbook/05-user-guide.html +6 -6
- package/docs/handbook/05-user-guide.md +3 -3
- package/docs/handbook/06-cli-reference.html +5 -5
- package/docs/handbook/06-cli-reference.md +9 -2
- package/docs/handbook/07-security-privacy-legal.html +4 -4
- package/docs/handbook/07-security-privacy-legal.md +1 -1
- package/docs/handbook/08-testing-and-proof.html +5 -5
- package/docs/handbook/08-testing-and-proof.md +2 -2
- package/docs/handbook/09-compatibility-and-roadmap.html +11 -11
- package/docs/handbook/09-compatibility-and-roadmap.md +25 -14
- package/docs/handbook/10-contributor-guide.html +4 -4
- package/docs/handbook/10-contributor-guide.md +3 -3
- package/docs/handbook/11-glossary-and-faq.html +4 -4
- package/docs/handbook/11-glossary-and-faq.md +1 -1
- package/docs/handbook/12-rule-catalog.html +3 -3
- package/docs/handbook/13-trust-and-assurance.html +5 -5
- package/docs/handbook/13-trust-and-assurance.md +3 -3
- package/docs/handbook/14-real-world-evaluation.html +3 -3
- package/docs/handbook/15-supply-chain-and-release.html +5 -5
- package/docs/handbook/15-supply-chain-and-release.md +7 -5
- package/docs/handbook/16-performance-and-quality-metrics.html +4 -4
- package/docs/handbook/16-performance-and-quality-metrics.md +1 -1
- package/docs/handbook/README.md +2 -2
- package/docs/handbook/index.html +5 -5
- package/docs/index.html +4 -4
- package/docs/legal/legal-review-brief.html +3 -3
- package/docs/legal/trademark-and-name-clearance.html +5 -5
- package/docs/legal/trademark-and-name-clearance.md +3 -1
- package/docs/proof/accessibility.html +3 -3
- package/docs/proof/test-results.txt +40 -35
- package/docs/rfc/0001-archsutra-v0.1.html +5 -5
- package/docs/rfc/0001-archsutra-v0.1.md +2 -2
- package/docs/sitemap.xml +28 -28
- package/llms.txt +6 -3
- package/package.json +7 -3
|
@@ -19,7 +19,7 @@
|
|
|
19
19
|
<link rel="icon" href="../assets/favicon.svg" type="image/svg+xml">
|
|
20
20
|
<link rel="stylesheet" href="../assets/site.css">
|
|
21
21
|
<script>try{var t=localStorage.getItem('as-theme');if(t)document.documentElement.setAttribute('data-theme',t)}catch(e){}</script>
|
|
22
|
-
<script type="application/ld+json">{"@context":"https://schema.org","@type":"TechArticle","headline":"Testing and proof","description":"Test results, CI evidence, coverage, fixtures, sample outputs and what is not proven. Real-world accuracy is in its own chapter.","author":{"@type":"Person","name":"Vatsal Shah"},"inLanguage":"en","isPartOf":{"@type":"WebSite","name":"ArchSutra AI","url":"https://shahvatsalofficial.github.io/archsutra-ai"}}</script>
|
|
22
|
+
<script type="application/ld+json">{"@context":"https://schema.org","@type":"TechArticle","headline":"Testing and proof","description":"Test results, CI evidence, coverage, fixtures, sample outputs and what is not proven. Real-world accuracy is in its own chapter.","author":{"@type":"Person","name":"Vatsal Shah","url":"https://shahvatsal.com"},"inLanguage":"en","isPartOf":{"@type":"WebSite","name":"ArchSutra AI","url":"https://shahvatsalofficial.github.io/archsutra-ai"}}</script>
|
|
23
23
|
</head>
|
|
24
24
|
<body class="doc">
|
|
25
25
|
<a class="skip" href="#main">Skip to content</a>
|
|
@@ -34,13 +34,13 @@
|
|
|
34
34
|
<div class="layout">
|
|
35
35
|
<aside id="sidebar" class="sidebar" aria-label="Documentation"><div class="nav-group"><p class="nav-title">Handbook</p><ul><li><a href="../handbook/index.html">Overview</a></li><li><a href="../handbook/01-executive-summary.html">Executive summary</a></li><li><a href="../handbook/02-functional-specification.html">Functional specification</a></li><li><a href="../handbook/03-technical-specification.html">Technical specification</a></li><li><a href="../handbook/04-architecture.html">Architecture</a></li><li><a href="../handbook/05-user-guide.html">User guide</a></li><li><a href="../handbook/06-cli-reference.html">CLI reference</a></li><li><a href="../handbook/07-security-privacy-legal.html">Security, privacy and legal</a></li><li><a href="../handbook/08-testing-and-proof.html" aria-current="page">Testing and proof</a></li><li><a href="../handbook/09-compatibility-and-roadmap.html">Compatibility and roadmap</a></li><li><a href="../handbook/10-contributor-guide.html">Contributor guide</a></li><li><a href="../handbook/11-glossary-and-faq.html">Glossary and FAQ</a></li><li><a href="../handbook/12-rule-catalog.html">Audit rule catalog</a></li><li><a href="../handbook/13-trust-and-assurance.html">Trust and assurance</a></li><li><a href="../handbook/14-real-world-evaluation.html">Real-world evaluation</a></li><li><a href="../handbook/15-supply-chain-and-release.html">Supply chain and release</a></li><li><a href="../handbook/16-performance-and-quality-metrics.html">Performance and quality metrics</a></li></ul></div><div class="nav-group"><p class="nav-title">Reference</p><ul><li><a href="../architecture/overview.html">Architecture overview</a></li><li><a href="../architecture/threat-model.html">Threat model (ArchSutra CLI and skills)</a></li><li><a href="../compatibility/matrix.html">Compatibility matrix</a></li><li><a href="../rfc/0001-archsutra-v0.1.html">RFC 0001: ArchSutra AI v0.1</a></li><li><a href="../decisions/ADR-0001-deterministic-cli-plus-ide-skills.html">ADR-0001: Deterministic CLI plus IDE-native skills</a></li><li><a href="../compatibility/smoke-tests/index.html">Live smoke tests</a></li><li><a href="../community/good-first-issues.html">Good first issues (ready to file)</a></li><li><a href="../legal/trademark-and-name-clearance.html">Name and trademark clearance</a></li><li><a href="../legal/legal-review-brief.html">Legal review brief</a></li><li><a href="../proof/accessibility.html">Accessibility audit of the documentation site</a></li></ul></div></aside>
|
|
36
36
|
<main id="main" class="content"><article class="prose"><p class="crumbs"><a href="../index.html">ArchSutra AI</a> / Handbook</p><h1 id="testing-and-proof">Testing and proof</h1>
|
|
37
|
-
<p class="lede">Test results, CI evidence, coverage, fixtures, sample outputs and what is not proven. Real-world accuracy is in its own chapter.</p><p class="meta">Version 0.1.
|
|
37
|
+
<p class="lede">Test results, CI evidence, coverage, fixtures, sample outputs and what is not proven. Real-world accuracy is in its own chapter.</p><p class="meta">Version 0.1.4 · Source: <a href="https://github.com/shahvatsalofficial/archsutra-ai/blob/main/docs/handbook/08-testing-and-proof.md" rel="noopener">docs/handbook/08-testing-and-proof.md</a></p>
|
|
38
38
|
<p>Trust should be earned with evidence you can re-run. Everything below is reproducible from a clean clone. For accuracy on real repositories see <a href="14-real-world-evaluation.html">Real-world evaluation</a>; for the register of open trust gaps see <a href="13-trust-and-assurance.html">Trust and assurance</a>.</p>
|
|
39
39
|
<h2 id="1-reproduce-it-yourself">1. Reproduce it yourself<a class="anchor" href="#1-reproduce-it-yourself" aria-label="Link to this section">#</a></h2>
|
|
40
40
|
<div class="code"><button class="copy" type="button" aria-label="Copy code">Copy</button><pre tabindex="0"><code class="language-bash">git clone https://github.com/shahvatsalofficial/archsutra-ai
|
|
41
41
|
cd archsutra-ai
|
|
42
42
|
npm ci
|
|
43
|
-
npm test # builds, then runs
|
|
43
|
+
npm test # builds, then runs 36 tests
|
|
44
44
|
npm run coverage # line, branch, function coverage
|
|
45
45
|
node scripts/eval.mjs # labeled-fixture precision / recall (regression suite)
|
|
46
46
|
node dist/cli.js verify . # validates skills, agents, links, diagrams
|
|
@@ -48,7 +48,7 @@ node dist/cli.js audit . # the tool audits itself
|
|
|
48
48
|
node scripts/bench.mjs # synthetic performance benchmark
|
|
49
49
|
node scripts/contrast.mjs # WCAG contrast of the docs site tokens</code></pre></div>
|
|
50
50
|
<h2 id="2-automated-test-results">2. Automated test results<a class="anchor" href="#2-automated-test-results" aria-label="Link to this section">#</a></h2>
|
|
51
|
-
<p>Captured output (Node 24, Windows 11; stored at <a href="../proof/test-results.txt"><code>docs/proof/test-results.txt</code></a>): <strong>
|
|
51
|
+
<p>Captured output (Node 24, Windows 11; stored at <a href="../proof/test-results.txt"><code>docs/proof/test-results.txt</code></a>): <strong>36 tests, 36 passed, 0 failed.</strong></p>
|
|
52
52
|
<p>Groups:</p>
|
|
53
53
|
<div class="table-wrap" tabindex="0" role="region" aria-label="Table 1: 2. Automated test results"><table><thead><tr><th scope="col">Group</th><th scope="col">What it covers</th></tr></thead><tbody><tr><td>Safety</td><td>secret redaction, path traversal, symlinks not followed, hostile file names, binary files, byte and file budgets, pathological long lines (ReDoS guard)</td></tr><tr><td>Analysis</td><td>module map, cycles, stack detection, seeded findings with evidence, clean project has no high findings</td></tr><tr><td>Precision regressions</td><td>each class of false positive found in the real-world evaluation has a test (comments, docstrings, dummy credentials, vendored files, shell vs SQL, public web ports, JWT <code>verify=False</code>, dev-tooling Dockerfiles)</td></tr><tr><td>Installer</td><td>idempotent install and uninstall for all hosts, conflict protection, backups, user-edited files kept, documented frontmatter only (Claude, Cursor, Copilot)</td></tr><tr><td>CLI</td><td>exit codes, JSON schema validity, <code>--fail-on</code>, export, dry-run, escaping output paths blocked</td></tr><tr><td>Quality gates</td><td>the repository's own skills, agents, links and diagrams verify; labeled fixtures give expected rules</td></tr></tbody></table></div>
|
|
54
54
|
<p>The symlink test returns early (passes without asserting) on systems where creating a symlink or junction is not permitted.</p>
|
|
@@ -76,7 +76,7 @@ node scripts/contrast.mjs # WCAG contrast of the docs site tokens</code></p
|
|
|
76
76
|
</div>
|
|
77
77
|
<footer class="footer">
|
|
78
78
|
<div class="footer-inner">
|
|
79
|
-
<p><strong>ArchSutra AI</strong> v0.1.
|
|
79
|
+
<p><strong>ArchSutra AI</strong> v0.1.4. Design. Audit. Evolve. Created by <a href="https://shahvatsal.com" rel="noopener">Vatsal Shah</a>. Licensed under <a href="https://github.com/shahvatsalofficial/archsutra-ai/blob/main/LICENSE" rel="noopener">Apache-2.0</a>.</p>
|
|
80
80
|
<p class="fine">Review aid only. Not a certification, compliance attestation, security guarantee or legal advice. Not affiliated with Anthropic, OpenAI, Cursor, GitHub or Google; product names belong to their owners. <a href="../handbook/07-security-privacy-legal.html">Legal details</a>.</p>
|
|
81
81
|
</div>
|
|
82
82
|
</footer>
|
|
@@ -10,7 +10,7 @@ Trust should be earned with evidence you can re-run. Everything below is reprodu
|
|
|
10
10
|
git clone https://github.com/shahvatsalofficial/archsutra-ai
|
|
11
11
|
cd archsutra-ai
|
|
12
12
|
npm ci
|
|
13
|
-
npm test # builds, then runs
|
|
13
|
+
npm test # builds, then runs 36 tests
|
|
14
14
|
npm run coverage # line, branch, function coverage
|
|
15
15
|
node scripts/eval.mjs # labeled-fixture precision / recall (regression suite)
|
|
16
16
|
node dist/cli.js verify . # validates skills, agents, links, diagrams
|
|
@@ -21,7 +21,7 @@ node scripts/contrast.mjs # WCAG contrast of the docs site tokens
|
|
|
21
21
|
|
|
22
22
|
## 2. Automated test results
|
|
23
23
|
|
|
24
|
-
Captured output (Node 24, Windows 11; stored at [`docs/proof/test-results.txt`](../proof/test-results.txt)): **
|
|
24
|
+
Captured output (Node 24, Windows 11; stored at [`docs/proof/test-results.txt`](../proof/test-results.txt)): **36 tests, 36 passed, 0 failed.**
|
|
25
25
|
|
|
26
26
|
Groups:
|
|
27
27
|
|
|
@@ -19,7 +19,7 @@
|
|
|
19
19
|
<link rel="icon" href="../assets/favicon.svg" type="image/svg+xml">
|
|
20
20
|
<link rel="stylesheet" href="../assets/site.css">
|
|
21
21
|
<script>try{var t=localStorage.getItem('as-theme');if(t)document.documentElement.setAttribute('data-theme',t)}catch(e){}</script>
|
|
22
|
-
<script type="application/ld+json">{"@context":"https://schema.org","@type":"TechArticle","headline":"Compatibility and roadmap","description":"Per-host support levels, what is verified, and the milestone plan with acceptance tests.","author":{"@type":"Person","name":"Vatsal Shah"},"inLanguage":"en","isPartOf":{"@type":"WebSite","name":"ArchSutra AI","url":"https://shahvatsalofficial.github.io/archsutra-ai"}}</script>
|
|
22
|
+
<script type="application/ld+json">{"@context":"https://schema.org","@type":"TechArticle","headline":"Compatibility and roadmap","description":"Per-host support levels, what is verified, and the milestone plan with acceptance tests.","author":{"@type":"Person","name":"Vatsal Shah","url":"https://shahvatsal.com"},"inLanguage":"en","isPartOf":{"@type":"WebSite","name":"ArchSutra AI","url":"https://shahvatsalofficial.github.io/archsutra-ai"}}</script>
|
|
23
23
|
</head>
|
|
24
24
|
<body class="doc">
|
|
25
25
|
<a class="skip" href="#main">Skip to content</a>
|
|
@@ -34,27 +34,27 @@
|
|
|
34
34
|
<div class="layout">
|
|
35
35
|
<aside id="sidebar" class="sidebar" aria-label="Documentation"><div class="nav-group"><p class="nav-title">Handbook</p><ul><li><a href="../handbook/index.html">Overview</a></li><li><a href="../handbook/01-executive-summary.html">Executive summary</a></li><li><a href="../handbook/02-functional-specification.html">Functional specification</a></li><li><a href="../handbook/03-technical-specification.html">Technical specification</a></li><li><a href="../handbook/04-architecture.html">Architecture</a></li><li><a href="../handbook/05-user-guide.html">User guide</a></li><li><a href="../handbook/06-cli-reference.html">CLI reference</a></li><li><a href="../handbook/07-security-privacy-legal.html">Security, privacy and legal</a></li><li><a href="../handbook/08-testing-and-proof.html">Testing and proof</a></li><li><a href="../handbook/09-compatibility-and-roadmap.html" aria-current="page">Compatibility and roadmap</a></li><li><a href="../handbook/10-contributor-guide.html">Contributor guide</a></li><li><a href="../handbook/11-glossary-and-faq.html">Glossary and FAQ</a></li><li><a href="../handbook/12-rule-catalog.html">Audit rule catalog</a></li><li><a href="../handbook/13-trust-and-assurance.html">Trust and assurance</a></li><li><a href="../handbook/14-real-world-evaluation.html">Real-world evaluation</a></li><li><a href="../handbook/15-supply-chain-and-release.html">Supply chain and release</a></li><li><a href="../handbook/16-performance-and-quality-metrics.html">Performance and quality metrics</a></li></ul></div><div class="nav-group"><p class="nav-title">Reference</p><ul><li><a href="../architecture/overview.html">Architecture overview</a></li><li><a href="../architecture/threat-model.html">Threat model (ArchSutra CLI and skills)</a></li><li><a href="../compatibility/matrix.html">Compatibility matrix</a></li><li><a href="../rfc/0001-archsutra-v0.1.html">RFC 0001: ArchSutra AI v0.1</a></li><li><a href="../decisions/ADR-0001-deterministic-cli-plus-ide-skills.html">ADR-0001: Deterministic CLI plus IDE-native skills</a></li><li><a href="../compatibility/smoke-tests/index.html">Live smoke tests</a></li><li><a href="../community/good-first-issues.html">Good first issues (ready to file)</a></li><li><a href="../legal/trademark-and-name-clearance.html">Name and trademark clearance</a></li><li><a href="../legal/legal-review-brief.html">Legal review brief</a></li><li><a href="../proof/accessibility.html">Accessibility audit of the documentation site</a></li></ul></div></aside>
|
|
36
36
|
<main id="main" class="content"><article class="prose"><p class="crumbs"><a href="../index.html">ArchSutra AI</a> / Handbook</p><h1 id="compatibility-and-roadmap">Compatibility and roadmap</h1>
|
|
37
|
-
<p class="lede">Per-host support levels, what is verified, and the milestone plan with acceptance tests.</p><p class="meta">Version 0.1.
|
|
37
|
+
<p class="lede">Per-host support levels, what is verified, and the milestone plan with acceptance tests.</p><p class="meta">Version 0.1.4 · Source: <a href="https://github.com/shahvatsalofficial/archsutra-ai/blob/main/docs/handbook/09-compatibility-and-roadmap.md" rel="noopener">docs/handbook/09-compatibility-and-roadmap.md</a></p>
|
|
38
38
|
<h2 id="1-support-levels">1. Support levels<a class="anchor" href="#1-support-levels" aria-label="Link to this section">#</a></h2>
|
|
39
39
|
<div class="table-wrap" tabindex="0" role="region" aria-label="Table 1: 1. Support levels"><table><thead><tr><th scope="col">Level</th><th scope="col">Meaning</th></tr></thead><tbody><tr><td>native</td><td>Host loads ArchSutra directly; verified in a live session</td></tr><tr><td>adapter</td><td>Files are written to the vendor-documented layout; <strong>not yet smoke-tested live</strong></td></tr><tr><td>manual</td><td>Copy/paste fallback</td></tr><tr><td>experimental</td><td>Layout unconfirmed for the current host release</td></tr><tr><td>unsupported</td><td>Not supported</td></tr></tbody></table></div>
|
|
40
|
-
<h2 id="2-host-matrix-
|
|
41
|
-
<div class="table-wrap" tabindex="0" role="region" aria-label="Table 2: 2. Host matrix (v0.1.
|
|
40
|
+
<h2 id="2-host-matrix-v013">2. Host matrix (v0.1.3)<a class="anchor" href="#2-host-matrix-v013" aria-label="Link to this section">#</a></h2>
|
|
41
|
+
<div class="table-wrap" tabindex="0" role="region" aria-label="Table 2: 2. Host matrix (v0.1.3)"><table><thead><tr><th scope="col">Host</th><th scope="col">Level</th><th scope="col">Skills path</th><th scope="col">Agents path</th><th scope="col">Verified</th><th scope="col">Not verified</th></tr></thead><tbody><tr><td>Claude Code</td><td>adapter</td><td><code>.claude/skills/<name>/SKILL.md</code></td><td><code>.claude/agents/<name>.md</code></td><td>file generation; idempotent install; frontmatter</td><td>live discovery</td></tr><tr><td>Cursor</td><td>adapter</td><td><code>.cursor/skills/<name>/SKILL.md</code></td><td><code>.cursor/agents/<name>.md</code></td><td>file generation; frontmatter keys</td><td>live discovery</td></tr><tr><td>OpenAI Codex</td><td>adapter</td><td><code>.agents/skills/<name>/SKILL.md</code> + <code>AGENTS.md</code> block</td><td>none</td><td>file generation</td><td>live discovery</td></tr><tr><td>GitHub Copilot</td><td>adapter</td><td><code>.github/skills/<name>/SKILL.md</code></td><td><code>.github/agents/<name>.agent.md</code></td><td>file generation; only <code>description</code>, <code>tools</code></td><td>live behavior</td></tr><tr><td>Gemini CLI</td><td>adapter</td><td><code>.gemini/skills/<name>/SKILL.md</code></td><td>none</td><td>file generation; path confirmed in vendor docs (2026-11-01)</td><td>live discovery; the docs announce Antigravity CLI as successor</td></tr></tbody></table></div>
|
|
42
42
|
<p>All five layouts were checked against the vendors' current documentation on 2026-11-01 (see <a href="../compatibility/smoke-tests/index.html">smoke tests</a>). Cursor, Copilot and Gemini also read <code>.agents/skills</code>, and Cursor and Copilot read <code>.claude/skills</code>: install <strong>one</strong> host per project to avoid duplicates. Layouts change; confirm against current docs.</p>
|
|
43
43
|
<h2 id="3-operating-systems-and-runtimes">3. Operating systems and runtimes<a class="anchor" href="#3-operating-systems-and-runtimes" aria-label="Link to this section">#</a></h2>
|
|
44
|
-
<div class="table-wrap" tabindex="0" role="region" aria-label="Table 3: 3. Operating systems and runtimes"><table><thead><tr><th scope="col">Platform</th><th scope="col">Status</th></tr></thead><tbody><tr><td>Ubuntu (latest), Node 20 and
|
|
44
|
+
<div class="table-wrap" tabindex="0" role="region" aria-label="Table 3: 3. Operating systems and runtimes"><table><thead><tr><th scope="col">Platform</th><th scope="col">Status</th></tr></thead><tbody><tr><td>Ubuntu (latest), Node 20, 22, 24, and the newest Node</td><td>CI passes</td></tr><tr><td>Windows (latest), Node 20, 22, and 24</td><td>CI passes</td></tr><tr><td>macOS</td><td>not in CI. GitHub cancelled macOS jobs before a runner started (no steps ran). No Mac is available to re-check. v0.1.0 did pass on Node 20 and 22 before those cancellations</td></tr></tbody></table></div>
|
|
45
45
|
<h2 id="4-language-support-in-analysis">4. Language support in analysis<a class="anchor" href="#4-language-support-in-analysis" aria-label="Link to this section">#</a></h2>
|
|
46
|
-
<div class="table-wrap" tabindex="0" role="region" aria-label="Table 4: 4. Language support in analysis"><table><thead><tr><th scope="col">Language</th><th scope="col">Detected</th><th scope="col">Import graph</th><th scope="col">Notes</th></tr></thead><tbody><tr><td>TypeScript, JavaScript</td><td>yes</td><td>yes (relative imports)</td><td>dynamic imports and path aliases may be missed</td></tr><tr><td>Python</td><td>yes</td><td>yes (absolute imports matching a module)</td><td>relative imports not graphed</td></tr><tr><td>Java, Kotlin, C#, Go, Rust
|
|
46
|
+
<div class="table-wrap" tabindex="0" role="region" aria-label="Table 4: 4. Language support in analysis"><table><thead><tr><th scope="col">Language</th><th scope="col">Detected</th><th scope="col">Import graph</th><th scope="col">Notes</th></tr></thead><tbody><tr><td>TypeScript, JavaScript</td><td>yes</td><td>yes (relative imports)</td><td>dynamic imports and path aliases may be missed</td></tr><tr><td>Python</td><td>yes</td><td>yes (absolute imports matching a module)</td><td>relative imports not graphed</td></tr><tr><td>Java, Kotlin, C#, Go, Rust</td><td>yes</td><td>yes (import path suffix matched to a module directory)</td><td>stdlib imports that do not match a module are ignored; DI is not graphed</td></tr><tr><td>PHP, Ruby, Swift, Dart</td><td>yes</td><td>no</td><td>reported as a blind spot</td></tr></tbody></table></div>
|
|
47
47
|
<p>Line-based security rules apply to all recognised source extensions where noted in the <a href="12-rule-catalog.html">rule catalog</a>.</p>
|
|
48
48
|
<h2 id="5-roadmap">5. Roadmap<a class="anchor" href="#5-roadmap" aria-label="Link to this section">#</a></h2>
|
|
49
|
-
<div class="table-wrap" tabindex="0" role="region" aria-label="Table 5: 5. Roadmap"><table><thead><tr><th scope="col">Milestone</th><th scope="col">Scope</th><th scope="col">Acceptance test</th></tr></thead><tbody><tr><td>v0.1 (done)</td><td>Skills, agents, CLI, adapters, CI, docs</td><td><code>npm test</code> green
|
|
49
|
+
<div class="table-wrap" tabindex="0" role="region" aria-label="Table 5: 5. Roadmap"><table><thead><tr><th scope="col">Milestone</th><th scope="col">Scope</th><th scope="col">Acceptance test</th></tr></thead><tbody><tr><td>v0.1 (done)</td><td>Skills, agents, CLI, adapters, CI, docs</td><td><code>npm test</code> green; fixtures match expected rules. The written acceptance said 3 OS. macOS left CI in 0.1.3 after GitHub cancelled those jobs before a runner started</td></tr><tr><td>v0.1.1 (released)</td><td>Real-world evaluation and precision fixes, pinned actions, release/SBOM/attestation workflow, Scorecard, DCO, accessibility audit, coverage, benchmarks, governance</td><td>See <a href="13-trust-and-assurance.html">Trust and assurance</a></td></tr><tr><td>v0.1.2 (released, git tag)</td><td>Dual-file reports (Markdown and HTML) for the audit set and for narrative skill output</td><td><code>audit --output</code> writes both formats; <code>html <file.md></code> renders narrative pages. GitHub release has SBOM, checksums, and 2 attestations. Bin name is only <code>archsutra</code></td></tr><tr><td>v0.1.3 (on npm and on <code>main</code>, no git tag)</td><td>Second bin <code>archsutra-ai</code> so Windows <code>npx</code> runs; macOS workflow removed; creator URL <code>https://shahvatsal.com</code></td><td>npm <code>bin</code> lists both commands. npm <code>gitHead</code> is <code>4a7ab27</code>, not the docs commit. No Actions provenance</td></tr><tr><td>v0.1.4</td><td>Suppression file, Go/Java/Kotlin/C#/Rust import suffixes, drift, compare, evidence starters, MCP stdio, npm publish step in the release workflow</td><td>Tests cover each. Starters keep <code>unknown</code> where a model is required. npm provenance still depends on a trusted publisher</td></tr><tr><td>v0.2</td><td>Live smoke tests per host; upgrade verified hosts to <code>native</code>; matrix with versions</td><td>One issue-linked report per host with version and evidence</td></tr><tr><td>v0.3</td><td>More analyzers (Java/Kotlin, C#, Go, Rust import graphs); fitness-function generator; rule suppression file</td><td>New fixtures with expected rules; precision/recall stays 1.00 on fixtures</td></tr><tr><td>v0.4</td><td>Larger, multi-labeler corpus; recall estimate; comparison with other scanners published</td><td>At least 100 labeled findings by two labelers; first <code>comparison.yml</code> run reviewed</td></tr><tr><td>v0.5</td><td>Optional local-model runtime and optional bring-your-own-key server, strictly opt-in, localhost by default</td><td>Security tests: unauthorized request rejected, no data leaves without opt-in</td></tr><tr><td>v1.0</td><td>Stable CLI and schema. npm publish from Actions with provenance, same digest as the GitHub release. <code>archsutra-ai@0.1.2</code> and <code>@0.1.3</code> already exist; <code>0.1.3</code> cannot be republished</td><td>Semantic-versioned contracts; provenance on the npm tarball and the GitHub release</td></tr></tbody></table></div>
|
|
50
50
|
<p>Principles for all future work: default stays keyless and offline; every new capability ships with tests and a documented limitation; no support claim without a verification record.</p>
|
|
51
|
-
<h2 id="6-known-gaps-
|
|
52
|
-
<ul><li>No live host
|
|
53
|
-
<aside class="rail"><nav class="toc" aria-label="On this page"><p class="nav-title">On this page</p><ul><li class="l2"><a href="#1-support-levels">1. Support levels</a></li><li class="l2"><a href="#2-host-matrix-
|
|
51
|
+
<h2 id="6-known-gaps-v013">6. Known gaps (v0.1.3)<a class="anchor" href="#6-known-gaps-v013" aria-label="Link to this section">#</a></h2>
|
|
52
|
+
<ul><li>No live model session inside Claude Code, Cursor, Codex, Copilot, or Gemini CLI. File layouts for all five are written by <code>install</code> and checked by tests. That is not proof the host loaded the skill.</li><li>Suppression file is <code>.archsutra/suppress.json</code> (<code>rule</code>, <code>path</code>, <code>reason</code>, optional <code>expires</code>). Expired rows are ignored. A row without a reason is ignored.</li><li>Module-level graph for JS/TS, Python, Go, Java, Kotlin, C#, and Rust. No function-level analysis. Path aliases, dynamic import, and dependency injection are missed. PHP, Ruby, Swift, and Dart are not graphed.</li><li><code>design</code>, <code>review</code>, <code>threat-model</code>, <code>modernize</code>, and <code>cost</code> write starters. Undecided sections stay <code>unknown</code>.</li><li><code>drift</code> and <code>compare</code> diff two ArchSutra findings files. Semgrep overlap is <code>node scripts/compare.mjs</code> (needs Semgrep on PATH) or the manual <code>comparison.yml</code> workflow. The CLI does not clone repositories.</li><li>MCP stdio server: <code>archsutra mcp</code>. Tools <code>archsutra_audit</code>, <code>archsutra_map</code>, <code>archsutra_drift</code>.</li><li>No <code>v0.1.3</code> git tag. npm <code>0.1.3</code> has no provenance. <code>0.1.4</code> is the next release and publishes from Actions with <code>--provenance</code> when a trusted publisher is configured.</li><li>OpenSSF Scorecard <strong>5.7 / 10</strong> (2026-11-06) until branch protection and reviewed pull requests exist. A solo maintainer cannot satisfy Code-Review without a second account. Force-push can be disabled. Required approving reviews would block direct pushes.</li><li>macOS is not in CI.</li><li>HTML Mermaid loads from <code>cdn.jsdelivr.net</code>. Offline, the diagram source stays visible and is not drawn.</li><li>Coverage is reported and not enforced as a CI threshold.</li><li>No screen-reader pass.</li><li>Formal trademark search and lawyer review are not done.</li></ul></article></main>
|
|
53
|
+
<aside class="rail"><nav class="toc" aria-label="On this page"><p class="nav-title">On this page</p><ul><li class="l2"><a href="#1-support-levels">1. Support levels</a></li><li class="l2"><a href="#2-host-matrix-v013">2. Host matrix (v0.1.3)</a></li><li class="l2"><a href="#3-operating-systems-and-runtimes">3. Operating systems and runtimes</a></li><li class="l2"><a href="#4-language-support-in-analysis">4. Language support in analysis</a></li><li class="l2"><a href="#5-roadmap">5. Roadmap</a></li><li class="l2"><a href="#6-known-gaps-v013">6. Known gaps (v0.1.3)</a></li></ul></nav></aside>
|
|
54
54
|
</div>
|
|
55
55
|
<footer class="footer">
|
|
56
56
|
<div class="footer-inner">
|
|
57
|
-
<p><strong>ArchSutra AI</strong> v0.1.
|
|
57
|
+
<p><strong>ArchSutra AI</strong> v0.1.4. Design. Audit. Evolve. Created by <a href="https://shahvatsal.com" rel="noopener">Vatsal Shah</a>. Licensed under <a href="https://github.com/shahvatsalofficial/archsutra-ai/blob/main/LICENSE" rel="noopener">Apache-2.0</a>.</p>
|
|
58
58
|
<p class="fine">Review aid only. Not a certification, compliance attestation, security guarantee or legal advice. Not affiliated with Anthropic, OpenAI, Cursor, GitHub or Google; product names belong to their owners. <a href="../handbook/07-security-privacy-legal.html">Legal details</a>.</p>
|
|
59
59
|
</div>
|
|
60
60
|
</footer>
|
|
@@ -12,7 +12,7 @@
|
|
|
12
12
|
| experimental | Layout unconfirmed for the current host release |
|
|
13
13
|
| unsupported | Not supported |
|
|
14
14
|
|
|
15
|
-
## 2. Host matrix (v0.1.
|
|
15
|
+
## 2. Host matrix (v0.1.3)
|
|
16
16
|
|
|
17
17
|
| Host | Level | Skills path | Agents path | Verified | Not verified |
|
|
18
18
|
|---|---|---|---|---|---|
|
|
@@ -28,8 +28,8 @@ All five layouts were checked against the vendors' current documentation on 2026
|
|
|
28
28
|
|
|
29
29
|
| Platform | Status |
|
|
30
30
|
|---|---|
|
|
31
|
-
| Ubuntu (latest), Node 20 and
|
|
32
|
-
| Windows (latest), Node 20 and
|
|
31
|
+
| Ubuntu (latest), Node 20, 22, 24, and the newest Node | CI passes |
|
|
32
|
+
| Windows (latest), Node 20, 22, and 24 | CI passes |
|
|
33
33
|
| macOS | not in CI. GitHub cancelled macOS jobs before a runner started (no steps ran). No Mac is available to re-check. v0.1.0 did pass on Node 20 and 22 before those cancellations |
|
|
34
34
|
|
|
35
35
|
## 4. Language support in analysis
|
|
@@ -38,7 +38,8 @@ All five layouts were checked against the vendors' current documentation on 2026
|
|
|
38
38
|
|---|---|---|---|
|
|
39
39
|
| TypeScript, JavaScript | yes | yes (relative imports) | dynamic imports and path aliases may be missed |
|
|
40
40
|
| Python | yes | yes (absolute imports matching a module) | relative imports not graphed |
|
|
41
|
-
| Java, Kotlin, C#, Go, Rust
|
|
41
|
+
| Java, Kotlin, C#, Go, Rust | yes | yes (import path suffix matched to a module directory) | stdlib imports that do not match a module are ignored; DI is not graphed |
|
|
42
|
+
| PHP, Ruby, Swift, Dart | yes | no | reported as a blind spot |
|
|
42
43
|
|
|
43
44
|
Line-based security rules apply to all recognised source extensions where noted in the [rule catalog](12-rule-catalog.md).
|
|
44
45
|
|
|
@@ -46,21 +47,31 @@ Line-based security rules apply to all recognised source extensions where noted
|
|
|
46
47
|
|
|
47
48
|
| Milestone | Scope | Acceptance test |
|
|
48
49
|
|---|---|---|
|
|
49
|
-
| v0.1 (done) | Skills, agents, CLI, adapters, CI, docs | `npm test` green
|
|
50
|
+
| v0.1 (done) | Skills, agents, CLI, adapters, CI, docs | `npm test` green; fixtures match expected rules. The written acceptance said 3 OS. macOS left CI in 0.1.3 after GitHub cancelled those jobs before a runner started |
|
|
50
51
|
| v0.1.1 (released) | Real-world evaluation and precision fixes, pinned actions, release/SBOM/attestation workflow, Scorecard, DCO, accessibility audit, coverage, benchmarks, governance | See [Trust and assurance](13-trust-and-assurance.md) |
|
|
51
|
-
| v0.1.2 | Dual-file reports (Markdown and HTML) for the audit set and for narrative skill output | `audit --output` writes both formats; `html <file.md>` renders narrative pages |
|
|
52
|
+
| v0.1.2 (released, git tag) | Dual-file reports (Markdown and HTML) for the audit set and for narrative skill output | `audit --output` writes both formats; `html <file.md>` renders narrative pages. GitHub release has SBOM, checksums, and 2 attestations. Bin name is only `archsutra` |
|
|
53
|
+
| v0.1.3 (on npm and on `main`, no git tag) | Second bin `archsutra-ai` so Windows `npx` runs; macOS workflow removed; creator URL `https://shahvatsal.com` | npm `bin` lists both commands. npm `gitHead` is `4a7ab27`, not the docs commit. No Actions provenance |
|
|
54
|
+
| v0.1.4 | Suppression file, Go/Java/Kotlin/C#/Rust import suffixes, drift, compare, evidence starters, MCP stdio, npm publish step in the release workflow | Tests cover each. Starters keep `unknown` where a model is required. npm provenance still depends on a trusted publisher |
|
|
52
55
|
| v0.2 | Live smoke tests per host; upgrade verified hosts to `native`; matrix with versions | One issue-linked report per host with version and evidence |
|
|
53
56
|
| v0.3 | More analyzers (Java/Kotlin, C#, Go, Rust import graphs); fitness-function generator; rule suppression file | New fixtures with expected rules; precision/recall stays 1.00 on fixtures |
|
|
54
57
|
| v0.4 | Larger, multi-labeler corpus; recall estimate; comparison with other scanners published | At least 100 labeled findings by two labelers; first `comparison.yml` run reviewed |
|
|
55
58
|
| v0.5 | Optional local-model runtime and optional bring-your-own-key server, strictly opt-in, localhost by default | Security tests: unauthorized request rejected, no data leaves without opt-in |
|
|
56
|
-
| v1.0 | Stable CLI and schema
|
|
59
|
+
| v1.0 | Stable CLI and schema. npm publish from Actions with provenance, same digest as the GitHub release. `archsutra-ai@0.1.2` and `@0.1.3` already exist; `0.1.3` cannot be republished | Semantic-versioned contracts; provenance on the npm tarball and the GitHub release |
|
|
57
60
|
|
|
58
61
|
Principles for all future work: default stays keyless and offline; every new capability ships with tests and a documented limitation; no support claim without a verification record.
|
|
59
62
|
|
|
60
|
-
## 6. Known gaps (v0.1)
|
|
61
|
-
|
|
62
|
-
- No live host
|
|
63
|
-
-
|
|
64
|
-
- Module-level graph
|
|
65
|
-
-
|
|
66
|
-
-
|
|
63
|
+
## 6. Known gaps (v0.1.3)
|
|
64
|
+
|
|
65
|
+
- No live model session inside Claude Code, Cursor, Codex, Copilot, or Gemini CLI. File layouts for all five are written by `install` and checked by tests. That is not proof the host loaded the skill.
|
|
66
|
+
- Suppression file is `.archsutra/suppress.json` (`rule`, `path`, `reason`, optional `expires`). Expired rows are ignored. A row without a reason is ignored.
|
|
67
|
+
- Module-level graph for JS/TS, Python, Go, Java, Kotlin, C#, and Rust. No function-level analysis. Path aliases, dynamic import, and dependency injection are missed. PHP, Ruby, Swift, and Dart are not graphed.
|
|
68
|
+
- `design`, `review`, `threat-model`, `modernize`, and `cost` write starters. Undecided sections stay `unknown`.
|
|
69
|
+
- `drift` and `compare` diff two ArchSutra findings files. Semgrep overlap is `node scripts/compare.mjs` (needs Semgrep on PATH) or the manual `comparison.yml` workflow. The CLI does not clone repositories.
|
|
70
|
+
- MCP stdio server: `archsutra mcp`. Tools `archsutra_audit`, `archsutra_map`, `archsutra_drift`.
|
|
71
|
+
- No `v0.1.3` git tag. npm `0.1.3` has no provenance. `0.1.4` is the next release and publishes from Actions with `--provenance` when a trusted publisher is configured.
|
|
72
|
+
- OpenSSF Scorecard **5.7 / 10** (2026-11-06) until branch protection and reviewed pull requests exist. A solo maintainer cannot satisfy Code-Review without a second account. Force-push can be disabled. Required approving reviews would block direct pushes.
|
|
73
|
+
- macOS is not in CI.
|
|
74
|
+
- HTML Mermaid loads from `cdn.jsdelivr.net`. Offline, the diagram source stays visible and is not drawn.
|
|
75
|
+
- Coverage is reported and not enforced as a CI threshold.
|
|
76
|
+
- No screen-reader pass.
|
|
77
|
+
- Formal trademark search and lawyer review are not done.
|
|
@@ -19,7 +19,7 @@
|
|
|
19
19
|
<link rel="icon" href="../assets/favicon.svg" type="image/svg+xml">
|
|
20
20
|
<link rel="stylesheet" href="../assets/site.css">
|
|
21
21
|
<script>try{var t=localStorage.getItem('as-theme');if(t)document.documentElement.setAttribute('data-theme',t)}catch(e){}</script>
|
|
22
|
-
<script type="application/ld+json">{"@context":"https://schema.org","@type":"TechArticle","headline":"Contributor guide","description":"How to set up, extend and release ArchSutra AI; coding rules and review checklist.","author":{"@type":"Person","name":"Vatsal Shah"},"inLanguage":"en","isPartOf":{"@type":"WebSite","name":"ArchSutra AI","url":"https://shahvatsalofficial.github.io/archsutra-ai"}}</script>
|
|
22
|
+
<script type="application/ld+json">{"@context":"https://schema.org","@type":"TechArticle","headline":"Contributor guide","description":"How to set up, extend and release ArchSutra AI; coding rules and review checklist.","author":{"@type":"Person","name":"Vatsal Shah","url":"https://shahvatsal.com"},"inLanguage":"en","isPartOf":{"@type":"WebSite","name":"ArchSutra AI","url":"https://shahvatsalofficial.github.io/archsutra-ai"}}</script>
|
|
23
23
|
</head>
|
|
24
24
|
<body class="doc">
|
|
25
25
|
<a class="skip" href="#main">Skip to content</a>
|
|
@@ -34,7 +34,7 @@
|
|
|
34
34
|
<div class="layout">
|
|
35
35
|
<aside id="sidebar" class="sidebar" aria-label="Documentation"><div class="nav-group"><p class="nav-title">Handbook</p><ul><li><a href="../handbook/index.html">Overview</a></li><li><a href="../handbook/01-executive-summary.html">Executive summary</a></li><li><a href="../handbook/02-functional-specification.html">Functional specification</a></li><li><a href="../handbook/03-technical-specification.html">Technical specification</a></li><li><a href="../handbook/04-architecture.html">Architecture</a></li><li><a href="../handbook/05-user-guide.html">User guide</a></li><li><a href="../handbook/06-cli-reference.html">CLI reference</a></li><li><a href="../handbook/07-security-privacy-legal.html">Security, privacy and legal</a></li><li><a href="../handbook/08-testing-and-proof.html">Testing and proof</a></li><li><a href="../handbook/09-compatibility-and-roadmap.html">Compatibility and roadmap</a></li><li><a href="../handbook/10-contributor-guide.html" aria-current="page">Contributor guide</a></li><li><a href="../handbook/11-glossary-and-faq.html">Glossary and FAQ</a></li><li><a href="../handbook/12-rule-catalog.html">Audit rule catalog</a></li><li><a href="../handbook/13-trust-and-assurance.html">Trust and assurance</a></li><li><a href="../handbook/14-real-world-evaluation.html">Real-world evaluation</a></li><li><a href="../handbook/15-supply-chain-and-release.html">Supply chain and release</a></li><li><a href="../handbook/16-performance-and-quality-metrics.html">Performance and quality metrics</a></li></ul></div><div class="nav-group"><p class="nav-title">Reference</p><ul><li><a href="../architecture/overview.html">Architecture overview</a></li><li><a href="../architecture/threat-model.html">Threat model (ArchSutra CLI and skills)</a></li><li><a href="../compatibility/matrix.html">Compatibility matrix</a></li><li><a href="../rfc/0001-archsutra-v0.1.html">RFC 0001: ArchSutra AI v0.1</a></li><li><a href="../decisions/ADR-0001-deterministic-cli-plus-ide-skills.html">ADR-0001: Deterministic CLI plus IDE-native skills</a></li><li><a href="../compatibility/smoke-tests/index.html">Live smoke tests</a></li><li><a href="../community/good-first-issues.html">Good first issues (ready to file)</a></li><li><a href="../legal/trademark-and-name-clearance.html">Name and trademark clearance</a></li><li><a href="../legal/legal-review-brief.html">Legal review brief</a></li><li><a href="../proof/accessibility.html">Accessibility audit of the documentation site</a></li></ul></div></aside>
|
|
36
36
|
<main id="main" class="content"><article class="prose"><p class="crumbs"><a href="../index.html">ArchSutra AI</a> / Handbook</p><h1 id="contributor-guide">Contributor guide</h1>
|
|
37
|
-
<p class="lede">How to set up, extend and release ArchSutra AI; coding rules and review checklist.</p><p class="meta">Version 0.1.
|
|
37
|
+
<p class="lede">How to set up, extend and release ArchSutra AI; coding rules and review checklist.</p><p class="meta">Version 0.1.4 · Source: <a href="https://github.com/shahvatsalofficial/archsutra-ai/blob/main/docs/handbook/10-contributor-guide.md" rel="noopener">docs/handbook/10-contributor-guide.md</a></p>
|
|
38
38
|
<p>Short rules are in <a href="https://github.com/shahvatsalofficial/archsutra-ai/blob/main/CONTRIBUTING.md" rel="noopener">CONTRIBUTING.md</a>. This is the detailed version.</p>
|
|
39
39
|
<h2 id="1-setup">1. Setup<a class="anchor" href="#1-setup" aria-label="Link to this section">#</a></h2>
|
|
40
40
|
<div class="code"><button class="copy" type="button" aria-label="Copy code">Copy</button><pre tabindex="0"><code class="language-bash">git clone https://github.com/shahvatsalofficial/archsutra-ai
|
|
@@ -62,7 +62,7 @@ npm test</code></pre></div>
|
|
|
62
62
|
<h2 id="8b-pull-request-checklist">8b. Pull request checklist<a class="anchor" href="#8b-pull-request-checklist" aria-label="Link to this section">#</a></h2>
|
|
63
63
|
<ul><li>[ ] Commits are signed off (<code>git commit -s</code>)</li><li>[ ] <code>npm test</code> passes locally</li><li>[ ] <code>npm run verify</code> passes</li><li>[ ] New behavior has tests, including a negative case</li><li>[ ] No secrets, no realistic secret-like strings committed</li><li>[ ] Docs updated (handbook, matrix, rule catalog, changelog)</li><li>[ ] Support claims match verification evidence</li></ul>
|
|
64
64
|
<h2 id="9-release-process-maintainers">9. Release process (maintainers)<a class="anchor" href="#9-release-process-maintainers" aria-label="Link to this section">#</a></h2>
|
|
65
|
-
<ol><li>Update <code>CHANGELOG.md</code> and <code>package.json</code> version (semantic versioning).</li><li><code>npm test</code>, <code>npm run verify</code>, <code>npm run coverage</code>, <code>node scripts/eval.mjs</code>, <code>node scripts/labeled-recall.mjs <workdir></code>, <code>node scripts/bench.mjs</code>, <code>node scripts/build-docs.mjs</code>.</li><li>Regenerate proof files under <code>docs/proof/</code> and sample outputs.</li><li>Commit, push, wait for the
|
|
65
|
+
<ol><li>Update <code>CHANGELOG.md</code> and <code>package.json</code> version (semantic versioning).</li><li><code>npm test</code>, <code>npm run verify</code>, <code>npm run coverage</code>, <code>node scripts/eval.mjs</code>, <code>node scripts/labeled-recall.mjs <workdir></code>, <code>node scripts/bench.mjs</code>, <code>node scripts/build-docs.mjs</code>.</li><li>Regenerate proof files under <code>docs/proof/</code> and sample outputs.</li><li>Commit, push, wait for CI (Ubuntu and Windows, Node 20, 22, and 24, plus Ubuntu on the newest Node).</li><li>Push the tag <code>vX.Y.Z</code>; the <code>Release</code> workflow creates the release with tarball, SBOMs, checksums and a GitHub attestation (<a href="15-supply-chain-and-release.html">Supply chain and release</a>).</li><li>npm: <code>archsutra-ai@0.1.2</code> and <code>@0.1.3</code> are already published from a laptop. Do not republish <code>0.1.3</code>. The next version needs <code>npm publish --provenance</code> from Actions. The name <code>archsutra</code> (no <code>-ai</code>) was still unclaimed on 2026-11-06; do not publish the parent-folder stub.</li></ol>
|
|
66
66
|
<h2 id="10-regenerating-the-documentation-site">10. Regenerating the documentation site<a class="anchor" href="#10-regenerating-the-documentation-site" aria-label="Link to this section">#</a></h2>
|
|
67
67
|
<p>Handbook Markdown in <code>docs/handbook/</code> is the source. Run:</p>
|
|
68
68
|
<div class="code"><button class="copy" type="button" aria-label="Copy code">Copy</button><pre tabindex="0"><code class="language-bash">node scripts/build-docs.mjs</code></pre></div>
|
|
@@ -73,7 +73,7 @@ npm test</code></pre></div>
|
|
|
73
73
|
</div>
|
|
74
74
|
<footer class="footer">
|
|
75
75
|
<div class="footer-inner">
|
|
76
|
-
<p><strong>ArchSutra AI</strong> v0.1.
|
|
76
|
+
<p><strong>ArchSutra AI</strong> v0.1.4. Design. Audit. Evolve. Created by <a href="https://shahvatsal.com" rel="noopener">Vatsal Shah</a>. Licensed under <a href="https://github.com/shahvatsalofficial/archsutra-ai/blob/main/LICENSE" rel="noopener">Apache-2.0</a>.</p>
|
|
77
77
|
<p class="fine">Review aid only. Not a certification, compliance attestation, security guarantee or legal advice. Not affiliated with Anthropic, OpenAI, Cursor, GitHub or Google; product names belong to their owners. <a href="../handbook/07-security-privacy-legal.html">Legal details</a>.</p>
|
|
78
78
|
</div>
|
|
79
79
|
</footer>
|
|
@@ -86,9 +86,9 @@ Every commit needs `Signed-off-by` (`git commit -s`). See [DCO.md](../../DCO.md)
|
|
|
86
86
|
1. Update `CHANGELOG.md` and `package.json` version (semantic versioning).
|
|
87
87
|
2. `npm test`, `npm run verify`, `npm run coverage`, `node scripts/eval.mjs`, `node scripts/labeled-recall.mjs <workdir>`, `node scripts/bench.mjs`, `node scripts/build-docs.mjs`.
|
|
88
88
|
3. Regenerate proof files under `docs/proof/` and sample outputs.
|
|
89
|
-
4. Commit, push, wait for the
|
|
90
|
-
5. Push the tag `vX.Y.Z`; the `Release` workflow creates the release with tarball, SBOMs, checksums and
|
|
91
|
-
6.
|
|
89
|
+
4. Commit, push, wait for CI (Ubuntu and Windows, Node 20, 22, and 24, plus Ubuntu on the newest Node).
|
|
90
|
+
5. Push the tag `vX.Y.Z`; the `Release` workflow creates the release with tarball, SBOMs, checksums and a GitHub attestation ([Supply chain and release](15-supply-chain-and-release.md)).
|
|
91
|
+
6. npm: `archsutra-ai@0.1.2` and `@0.1.3` are already published from a laptop. Do not republish `0.1.3`. The next version needs `npm publish --provenance` from Actions. The name `archsutra` (no `-ai`) was still unclaimed on 2026-11-06; do not publish the parent-folder stub.
|
|
92
92
|
|
|
93
93
|
## 10. Regenerating the documentation site
|
|
94
94
|
|
|
@@ -19,7 +19,7 @@
|
|
|
19
19
|
<link rel="icon" href="../assets/favicon.svg" type="image/svg+xml">
|
|
20
20
|
<link rel="stylesheet" href="../assets/site.css">
|
|
21
21
|
<script>try{var t=localStorage.getItem('as-theme');if(t)document.documentElement.setAttribute('data-theme',t)}catch(e){}</script>
|
|
22
|
-
<script type="application/ld+json">{"@context":"https://schema.org","@type":"TechArticle","headline":"Glossary and FAQ","description":"Definitions of terms and direct answers to the most common questions.","author":{"@type":"Person","name":"Vatsal Shah"},"inLanguage":"en","isPartOf":{"@type":"WebSite","name":"ArchSutra AI","url":"https://shahvatsalofficial.github.io/archsutra-ai"}}</script>
|
|
22
|
+
<script type="application/ld+json">{"@context":"https://schema.org","@type":"TechArticle","headline":"Glossary and FAQ","description":"Definitions of terms and direct answers to the most common questions.","author":{"@type":"Person","name":"Vatsal Shah","url":"https://shahvatsal.com"},"inLanguage":"en","isPartOf":{"@type":"WebSite","name":"ArchSutra AI","url":"https://shahvatsalofficial.github.io/archsutra-ai"}}</script>
|
|
23
23
|
</head>
|
|
24
24
|
<body class="doc">
|
|
25
25
|
<a class="skip" href="#main">Skip to content</a>
|
|
@@ -34,7 +34,7 @@
|
|
|
34
34
|
<div class="layout">
|
|
35
35
|
<aside id="sidebar" class="sidebar" aria-label="Documentation"><div class="nav-group"><p class="nav-title">Handbook</p><ul><li><a href="../handbook/index.html">Overview</a></li><li><a href="../handbook/01-executive-summary.html">Executive summary</a></li><li><a href="../handbook/02-functional-specification.html">Functional specification</a></li><li><a href="../handbook/03-technical-specification.html">Technical specification</a></li><li><a href="../handbook/04-architecture.html">Architecture</a></li><li><a href="../handbook/05-user-guide.html">User guide</a></li><li><a href="../handbook/06-cli-reference.html">CLI reference</a></li><li><a href="../handbook/07-security-privacy-legal.html">Security, privacy and legal</a></li><li><a href="../handbook/08-testing-and-proof.html">Testing and proof</a></li><li><a href="../handbook/09-compatibility-and-roadmap.html">Compatibility and roadmap</a></li><li><a href="../handbook/10-contributor-guide.html">Contributor guide</a></li><li><a href="../handbook/11-glossary-and-faq.html" aria-current="page">Glossary and FAQ</a></li><li><a href="../handbook/12-rule-catalog.html">Audit rule catalog</a></li><li><a href="../handbook/13-trust-and-assurance.html">Trust and assurance</a></li><li><a href="../handbook/14-real-world-evaluation.html">Real-world evaluation</a></li><li><a href="../handbook/15-supply-chain-and-release.html">Supply chain and release</a></li><li><a href="../handbook/16-performance-and-quality-metrics.html">Performance and quality metrics</a></li></ul></div><div class="nav-group"><p class="nav-title">Reference</p><ul><li><a href="../architecture/overview.html">Architecture overview</a></li><li><a href="../architecture/threat-model.html">Threat model (ArchSutra CLI and skills)</a></li><li><a href="../compatibility/matrix.html">Compatibility matrix</a></li><li><a href="../rfc/0001-archsutra-v0.1.html">RFC 0001: ArchSutra AI v0.1</a></li><li><a href="../decisions/ADR-0001-deterministic-cli-plus-ide-skills.html">ADR-0001: Deterministic CLI plus IDE-native skills</a></li><li><a href="../compatibility/smoke-tests/index.html">Live smoke tests</a></li><li><a href="../community/good-first-issues.html">Good first issues (ready to file)</a></li><li><a href="../legal/trademark-and-name-clearance.html">Name and trademark clearance</a></li><li><a href="../legal/legal-review-brief.html">Legal review brief</a></li><li><a href="../proof/accessibility.html">Accessibility audit of the documentation site</a></li></ul></div></aside>
|
|
36
36
|
<main id="main" class="content"><article class="prose"><p class="crumbs"><a href="../index.html">ArchSutra AI</a> / Handbook</p><h1 id="glossary-and-faq">Glossary and FAQ</h1>
|
|
37
|
-
<p class="lede">Definitions of terms and direct answers to the most common questions.</p><p class="meta">Version 0.1.
|
|
37
|
+
<p class="lede">Definitions of terms and direct answers to the most common questions.</p><p class="meta">Version 0.1.4 · Source: <a href="https://github.com/shahvatsalofficial/archsutra-ai/blob/main/docs/handbook/11-glossary-and-faq.md" rel="noopener">docs/handbook/11-glossary-and-faq.md</a></p>
|
|
38
38
|
<h2 id="glossary">Glossary<a class="anchor" href="#glossary" aria-label="Link to this section">#</a></h2>
|
|
39
39
|
<div class="table-wrap" tabindex="0" role="region" aria-label="Table 1: Glossary"><table><thead><tr><th scope="col">Term</th><th scope="col">Meaning</th></tr></thead><tbody><tr><td>Agent Skill</td><td>A folder with a <code>SKILL.md</code> (name, description, instructions) that an AI assistant loads on demand. Specification: agentskills.io</td></tr><tr><td>Agent role / sub-agent</td><td>A scoped persona with its own contract (for example Sentinel for security) that the host can delegate to</td></tr><tr><td>ArchSutra Prime</td><td>The orchestrating agent: scopes, delegates, synthesizes</td></tr><tr><td>Adapter</td><td>Code that renders canonical skills/agents into a host's folder layout</td></tr><tr><td>Evidence</td><td>A reference to where a claim comes from: <code>path:line</code> or a named document</td></tr><tr><td>observed / inferred / proposed / unknown</td><td>Claim labels. Observed = seen in source. Inferred = reasoned from observations. Proposed = recommended future design. Unknown = not determinable</td></tr><tr><td>Finding</td><td>A reported issue with ID, severity, confidence, evidence, fix</td></tr><tr><td>Severity</td><td>Impact if the finding is real: critical, high, medium, low, info</td></tr><tr><td>Confidence</td><td>How sure the rule is that it is real: high, medium, low</td></tr><tr><td>Scorecard</td><td>Ratings per quality area; <code>not assessed</code> when evidence is missing</td></tr><tr><td>Module</td><td>Top-level unit used for dependency analysis (see <a href="03-technical-specification.html">technical spec</a>)</td></tr><tr><td>Cycle</td><td>Modules that depend on each other directly or indirectly</td></tr><tr><td>C4 model</td><td>Context, Container, Component, Code diagram hierarchy</td></tr><tr><td>ADR</td><td>Architecture Decision Record</td></tr><tr><td>HLD / LLD</td><td>High-level / low-level design</td></tr><tr><td>STRIDE</td><td>Spoofing, Tampering, Repudiation, Information disclosure, Denial of service, Elevation of privilege</td></tr><tr><td>SARIF</td><td>Static Analysis Results Interchange Format, used by GitHub code scanning</td></tr><tr><td>Fitness function</td><td>An automated check that an architectural property still holds</td></tr><tr><td>BYOK</td><td>Bring your own key (user-supplied model provider credentials)</td></tr><tr><td>Prompt injection</td><td>Hostile text in data that tries to steer an AI agent</td></tr><tr><td>Blind spot</td><td>A known limit of what the analysis can see</td></tr></tbody></table></div>
|
|
40
40
|
<h2 id="faq">FAQ<a class="anchor" href="#faq" aria-label="Link to this section">#</a></h2>
|
|
@@ -52,13 +52,13 @@
|
|
|
52
52
|
<p><strong>Does it certify compliance (GDPR, SOC 2, ...)?</strong> No. Never. It produces review aids only.</p>
|
|
53
53
|
<p><strong>How do I add a rule or a language?</strong> See the <a href="10-contributor-guide.html">contributor guide</a>.</p>
|
|
54
54
|
<p><strong>Where are the test results?</strong> <a href="08-testing-and-proof.html">Testing and proof</a> and <code>docs/proof/</code>.</p>
|
|
55
|
-
<p><strong>Is it on npm?</strong>
|
|
55
|
+
<p><strong>Is it on npm?</strong> Yes. Package name <code>archsutra-ai</code>, versions <code>0.1.2</code> and <code>0.1.3</code>. Use <code>npx --yes archsutra-ai@0.1.3</code>. The <code>0.1.2</code> package does not install a command named <code>archsutra-ai</code>, so that invocation fails on Windows. There is no <code>v0.1.3</code> git tag and no npm provenance. The name <code>archsutra</code> without <code>-ai</code> was still unclaimed on 2026-11-06.</p>
|
|
56
56
|
<p><strong>Who made it?</strong> Created by Vatsal Shah, with Claude assisting in implementation. Contributors are credited in git history.</p></article></main>
|
|
57
57
|
<aside class="rail"><nav class="toc" aria-label="On this page"><p class="nav-title">On this page</p><ul><li class="l2"><a href="#glossary">Glossary</a></li><li class="l2"><a href="#faq">FAQ</a></li></ul></nav></aside>
|
|
58
58
|
</div>
|
|
59
59
|
<footer class="footer">
|
|
60
60
|
<div class="footer-inner">
|
|
61
|
-
<p><strong>ArchSutra AI</strong> v0.1.
|
|
61
|
+
<p><strong>ArchSutra AI</strong> v0.1.4. Design. Audit. Evolve. Created by <a href="https://shahvatsal.com" rel="noopener">Vatsal Shah</a>. Licensed under <a href="https://github.com/shahvatsalofficial/archsutra-ai/blob/main/LICENSE" rel="noopener">Apache-2.0</a>.</p>
|
|
62
62
|
<p class="fine">Review aid only. Not a certification, compliance attestation, security guarantee or legal advice. Not affiliated with Anthropic, OpenAI, Cursor, GitHub or Google; product names belong to their owners. <a href="../handbook/07-security-privacy-legal.html">Legal details</a>.</p>
|
|
63
63
|
</div>
|
|
64
64
|
</footer>
|
|
@@ -73,7 +73,7 @@ See the [contributor guide](10-contributor-guide.md).
|
|
|
73
73
|
[Testing and proof](08-testing-and-proof.md) and `docs/proof/`.
|
|
74
74
|
|
|
75
75
|
**Is it on npm?**
|
|
76
|
-
|
|
76
|
+
Yes. Package name `archsutra-ai`, versions `0.1.2` and `0.1.3`. Use `npx --yes archsutra-ai@0.1.3`. The `0.1.2` package does not install a command named `archsutra-ai`, so that invocation fails on Windows. There is no `v0.1.3` git tag and no npm provenance. The name `archsutra` without `-ai` was still unclaimed on 2026-11-06.
|
|
77
77
|
|
|
78
78
|
**Who made it?**
|
|
79
79
|
Created by Vatsal Shah, with Claude assisting in implementation. Contributors are credited in git history.
|
|
@@ -19,7 +19,7 @@
|
|
|
19
19
|
<link rel="icon" href="../assets/favicon.svg" type="image/svg+xml">
|
|
20
20
|
<link rel="stylesheet" href="../assets/site.css">
|
|
21
21
|
<script>try{var t=localStorage.getItem('as-theme');if(t)document.documentElement.setAttribute('data-theme',t)}catch(e){}</script>
|
|
22
|
-
<script type="application/ld+json">{"@context":"https://schema.org","@type":"TechArticle","headline":"Audit rule catalog","description":"All 19 audit rules: what each detects, severity, confidence, false-positive notes and how to fix.","author":{"@type":"Person","name":"Vatsal Shah"},"inLanguage":"en","isPartOf":{"@type":"WebSite","name":"ArchSutra AI","url":"https://shahvatsalofficial.github.io/archsutra-ai"}}</script>
|
|
22
|
+
<script type="application/ld+json">{"@context":"https://schema.org","@type":"TechArticle","headline":"Audit rule catalog","description":"All 19 audit rules: what each detects, severity, confidence, false-positive notes and how to fix.","author":{"@type":"Person","name":"Vatsal Shah","url":"https://shahvatsal.com"},"inLanguage":"en","isPartOf":{"@type":"WebSite","name":"ArchSutra AI","url":"https://shahvatsalofficial.github.io/archsutra-ai"}}</script>
|
|
23
23
|
</head>
|
|
24
24
|
<body class="doc">
|
|
25
25
|
<a class="skip" href="#main">Skip to content</a>
|
|
@@ -34,7 +34,7 @@
|
|
|
34
34
|
<div class="layout">
|
|
35
35
|
<aside id="sidebar" class="sidebar" aria-label="Documentation"><div class="nav-group"><p class="nav-title">Handbook</p><ul><li><a href="../handbook/index.html">Overview</a></li><li><a href="../handbook/01-executive-summary.html">Executive summary</a></li><li><a href="../handbook/02-functional-specification.html">Functional specification</a></li><li><a href="../handbook/03-technical-specification.html">Technical specification</a></li><li><a href="../handbook/04-architecture.html">Architecture</a></li><li><a href="../handbook/05-user-guide.html">User guide</a></li><li><a href="../handbook/06-cli-reference.html">CLI reference</a></li><li><a href="../handbook/07-security-privacy-legal.html">Security, privacy and legal</a></li><li><a href="../handbook/08-testing-and-proof.html">Testing and proof</a></li><li><a href="../handbook/09-compatibility-and-roadmap.html">Compatibility and roadmap</a></li><li><a href="../handbook/10-contributor-guide.html">Contributor guide</a></li><li><a href="../handbook/11-glossary-and-faq.html">Glossary and FAQ</a></li><li><a href="../handbook/12-rule-catalog.html" aria-current="page">Audit rule catalog</a></li><li><a href="../handbook/13-trust-and-assurance.html">Trust and assurance</a></li><li><a href="../handbook/14-real-world-evaluation.html">Real-world evaluation</a></li><li><a href="../handbook/15-supply-chain-and-release.html">Supply chain and release</a></li><li><a href="../handbook/16-performance-and-quality-metrics.html">Performance and quality metrics</a></li></ul></div><div class="nav-group"><p class="nav-title">Reference</p><ul><li><a href="../architecture/overview.html">Architecture overview</a></li><li><a href="../architecture/threat-model.html">Threat model (ArchSutra CLI and skills)</a></li><li><a href="../compatibility/matrix.html">Compatibility matrix</a></li><li><a href="../rfc/0001-archsutra-v0.1.html">RFC 0001: ArchSutra AI v0.1</a></li><li><a href="../decisions/ADR-0001-deterministic-cli-plus-ide-skills.html">ADR-0001: Deterministic CLI plus IDE-native skills</a></li><li><a href="../compatibility/smoke-tests/index.html">Live smoke tests</a></li><li><a href="../community/good-first-issues.html">Good first issues (ready to file)</a></li><li><a href="../legal/trademark-and-name-clearance.html">Name and trademark clearance</a></li><li><a href="../legal/legal-review-brief.html">Legal review brief</a></li><li><a href="../proof/accessibility.html">Accessibility audit of the documentation site</a></li></ul></div></aside>
|
|
36
36
|
<main id="main" class="content"><article class="prose"><p class="crumbs"><a href="../index.html">ArchSutra AI</a> / Handbook</p><h1 id="audit-rule-catalog">Audit rule catalog</h1>
|
|
37
|
-
<p class="lede">All 19 audit rules: what each detects, severity, confidence, false-positive notes and how to fix.</p><p class="meta">Version 0.1.
|
|
37
|
+
<p class="lede">All 19 audit rules: what each detects, severity, confidence, false-positive notes and how to fix.</p><p class="meta">Version 0.1.4 · Source: <a href="https://github.com/shahvatsalofficial/archsutra-ai/blob/main/docs/handbook/12-rule-catalog.md" rel="noopener">docs/handbook/12-rule-catalog.md</a></p>
|
|
38
38
|
<p>Severity and confidence are the defaults emitted by the rule. IDs are stable; the <code>#NNN</code> suffix in a finding is positional per run.</p>
|
|
39
39
|
<div class="table-wrap" tabindex="0" role="region" aria-label="Table 1: Audit rule catalog"><table><thead><tr><th scope="col">Rule</th><th scope="col">Title</th><th scope="col">Severity</th><th scope="col">Confidence</th><th scope="col">Category</th></tr></thead><tbody><tr><td>AS-SEC-001</td><td>Possible hard-coded credential</td><td>critical</td><td>medium</td><td>security</td></tr><tr><td>AS-SEC-002</td><td>Private key material committed</td><td>critical</td><td>high</td><td>security</td></tr><tr><td>AS-SEC-003</td><td>Environment file present in repository tree</td><td>high</td><td>medium</td><td>security</td></tr><tr><td>AS-SEC-004</td><td>Dynamic code or shell execution</td><td>high</td><td>medium</td><td>security</td></tr><tr><td>AS-SEC-005</td><td>SQL built by string concatenation or interpolation</td><td>high</td><td>medium</td><td>security</td></tr><tr><td>AS-SEC-006</td><td>TLS certificate verification disabled</td><td>high</td><td>high</td><td>security</td></tr><tr><td>AS-IAC-001</td><td>Over-permissive infrastructure configuration</td><td>medium</td><td>medium</td><td>infrastructure</td></tr><tr><td>AS-ARC-001</td><td>Circular dependency between modules</td><td>medium</td><td>high</td><td>architecture</td></tr><tr><td>AS-ARC-002</td><td>Lower layer depends on a higher layer</td><td>medium</td><td>medium</td><td>architecture</td></tr><tr><td>AS-ARC-003</td><td>Very large source files</td><td>low</td><td>medium</td><td>maintainability</td></tr><tr><td>AS-OPS-001</td><td>No CI pipeline configuration found</td><td>medium</td><td>medium</td><td>operability</td></tr><tr><td>AS-OPS-002</td><td>No test files found</td><td>medium</td><td>medium</td><td>maintainability</td></tr><tr><td>AS-OPS-003</td><td>Container runs as root</td><td>medium</td><td>high</td><td>security</td></tr><tr><td>AS-SUP-001</td><td>Dependency manifest without lockfile</td><td>medium</td><td>high</td><td>supply-chain</td></tr><tr><td>AS-SUP-002</td><td>Unpinned dependency ranges (<code>latest</code>, <code>*</code>)</td><td>medium</td><td>high</td><td>supply-chain</td></tr><tr><td>AS-SUP-003</td><td>Container base image not pinned</td><td>low</td><td>medium</td><td>supply-chain</td></tr><tr><td>AS-AI-001</td><td>LLM call co-located with dynamic code execution</td><td>high</td><td>low</td><td>ai-security</td></tr><tr><td>AS-DOC-001</td><td>No README at repository root</td><td>low</td><td>high</td><td>developer-experience</td></tr><tr><td>AS-DOC-002</td><td>No architecture decision records (only if more than 200 source files)</td><td>info</td><td>medium</td><td>governance</td></tr></tbody></table></div>
|
|
40
40
|
<h2 id="details">Details<a class="anchor" href="#details" aria-label="Link to this section">#</a></h2>
|
|
@@ -80,7 +80,7 @@
|
|
|
80
80
|
</div>
|
|
81
81
|
<footer class="footer">
|
|
82
82
|
<div class="footer-inner">
|
|
83
|
-
<p><strong>ArchSutra AI</strong> v0.1.
|
|
83
|
+
<p><strong>ArchSutra AI</strong> v0.1.4. Design. Audit. Evolve. Created by <a href="https://shahvatsal.com" rel="noopener">Vatsal Shah</a>. Licensed under <a href="https://github.com/shahvatsalofficial/archsutra-ai/blob/main/LICENSE" rel="noopener">Apache-2.0</a>.</p>
|
|
84
84
|
<p class="fine">Review aid only. Not a certification, compliance attestation, security guarantee or legal advice. Not affiliated with Anthropic, OpenAI, Cursor, GitHub or Google; product names belong to their owners. <a href="../handbook/07-security-privacy-legal.html">Legal details</a>.</p>
|
|
85
85
|
</div>
|
|
86
86
|
</footer>
|
|
@@ -19,7 +19,7 @@
|
|
|
19
19
|
<link rel="icon" href="../assets/favicon.svg" type="image/svg+xml">
|
|
20
20
|
<link rel="stylesheet" href="../assets/site.css">
|
|
21
21
|
<script>try{var t=localStorage.getItem('as-theme');if(t)document.documentElement.setAttribute('data-theme',t)}catch(e){}</script>
|
|
22
|
-
<script type="application/ld+json">{"@context":"https://schema.org","@type":"TechArticle","headline":"Trust and assurance","description":"Every question a skeptical reader should ask, the evidence we have, and what is still open, with no rounding up.","author":{"@type":"Person","name":"Vatsal Shah"},"inLanguage":"en","isPartOf":{"@type":"WebSite","name":"ArchSutra AI","url":"https://shahvatsalofficial.github.io/archsutra-ai"}}</script>
|
|
22
|
+
<script type="application/ld+json">{"@context":"https://schema.org","@type":"TechArticle","headline":"Trust and assurance","description":"Every question a skeptical reader should ask, the evidence we have, and what is still open, with no rounding up.","author":{"@type":"Person","name":"Vatsal Shah","url":"https://shahvatsal.com"},"inLanguage":"en","isPartOf":{"@type":"WebSite","name":"ArchSutra AI","url":"https://shahvatsalofficial.github.io/archsutra-ai"}}</script>
|
|
23
23
|
</head>
|
|
24
24
|
<body class="doc">
|
|
25
25
|
<a class="skip" href="#main">Skip to content</a>
|
|
@@ -34,11 +34,11 @@
|
|
|
34
34
|
<div class="layout">
|
|
35
35
|
<aside id="sidebar" class="sidebar" aria-label="Documentation"><div class="nav-group"><p class="nav-title">Handbook</p><ul><li><a href="../handbook/index.html">Overview</a></li><li><a href="../handbook/01-executive-summary.html">Executive summary</a></li><li><a href="../handbook/02-functional-specification.html">Functional specification</a></li><li><a href="../handbook/03-technical-specification.html">Technical specification</a></li><li><a href="../handbook/04-architecture.html">Architecture</a></li><li><a href="../handbook/05-user-guide.html">User guide</a></li><li><a href="../handbook/06-cli-reference.html">CLI reference</a></li><li><a href="../handbook/07-security-privacy-legal.html">Security, privacy and legal</a></li><li><a href="../handbook/08-testing-and-proof.html">Testing and proof</a></li><li><a href="../handbook/09-compatibility-and-roadmap.html">Compatibility and roadmap</a></li><li><a href="../handbook/10-contributor-guide.html">Contributor guide</a></li><li><a href="../handbook/11-glossary-and-faq.html">Glossary and FAQ</a></li><li><a href="../handbook/12-rule-catalog.html">Audit rule catalog</a></li><li><a href="../handbook/13-trust-and-assurance.html" aria-current="page">Trust and assurance</a></li><li><a href="../handbook/14-real-world-evaluation.html">Real-world evaluation</a></li><li><a href="../handbook/15-supply-chain-and-release.html">Supply chain and release</a></li><li><a href="../handbook/16-performance-and-quality-metrics.html">Performance and quality metrics</a></li></ul></div><div class="nav-group"><p class="nav-title">Reference</p><ul><li><a href="../architecture/overview.html">Architecture overview</a></li><li><a href="../architecture/threat-model.html">Threat model (ArchSutra CLI and skills)</a></li><li><a href="../compatibility/matrix.html">Compatibility matrix</a></li><li><a href="../rfc/0001-archsutra-v0.1.html">RFC 0001: ArchSutra AI v0.1</a></li><li><a href="../decisions/ADR-0001-deterministic-cli-plus-ide-skills.html">ADR-0001: Deterministic CLI plus IDE-native skills</a></li><li><a href="../compatibility/smoke-tests/index.html">Live smoke tests</a></li><li><a href="../community/good-first-issues.html">Good first issues (ready to file)</a></li><li><a href="../legal/trademark-and-name-clearance.html">Name and trademark clearance</a></li><li><a href="../legal/legal-review-brief.html">Legal review brief</a></li><li><a href="../proof/accessibility.html">Accessibility audit of the documentation site</a></li></ul></div></aside>
|
|
36
36
|
<main id="main" class="content"><article class="prose"><p class="crumbs"><a href="../index.html">ArchSutra AI</a> / Handbook</p><h1 id="trust-and-assurance">Trust and assurance</h1>
|
|
37
|
-
<p class="lede">Every question a skeptical reader should ask, the evidence we have, and what is still open, with no rounding up.</p><p class="meta">Version 0.1.
|
|
37
|
+
<p class="lede">Every question a skeptical reader should ask, the evidence we have, and what is still open, with no rounding up.</p><p class="meta">Version 0.1.4 · Source: <a href="https://github.com/shahvatsalofficial/archsutra-ai/blob/main/docs/handbook/13-trust-and-assurance.md" rel="noopener">docs/handbook/13-trust-and-assurance.md</a></p>
|
|
38
38
|
<p>An independent-style review of this project listed 15 gaps that stop people from trusting a young open-source tool. This page tracks each one. Status values: <strong>Done</strong> (evidence in the repository), <strong>Partly</strong> (some evidence, limits stated), <strong>Pending</strong> (cannot be done by the maintainer alone or needs a first run), <strong>Not possible yet</strong> (needs time or third parties).</p>
|
|
39
|
-
<div class="table-wrap" tabindex="0" role="region" aria-label="Table 1: Trust and assurance"><table><thead><tr><th scope="col">#</th><th scope="col">Gap</th><th scope="col">Status</th><th scope="col">Evidence and honest limits</th></tr></thead><tbody><tr><td>1</td><td>Real-world accuracy</td><td><strong>Partly</strong></td><td>52 public repositories, 98 labeled findings, three batches, precision with confidence intervals: <a href="14-real-world-evaluation.html">Real-world evaluation</a>. Single labeler, small samples, no recall on real code</td></tr><tr><td>2</td><td>Live proof in AI IDEs</td><td><strong>Pending</strong></td><td>Layouts checked against vendor docs on 2026-11-01; protocol and empty results log: <a href="../compatibility/smoke-tests/index.html">smoke tests</a>. Needs the owner or community with access to each IDE</td></tr><tr><td>3</td><td>Head-to-head comparison</td><td><strong>Pending</strong></td><td>Workflow and script ready (<a href="https://github.com/shahvatsalofficial/archsutra-ai/blob/main/.github/workflows/comparison.yml" rel="noopener"><code>comparison.yml</code></a>, <code>scripts/compare.mjs</code>); first run needs the repository on GitHub with Actions. A plain-AI-prompt comparison is deliberately not offered (not reproducible)</td></tr><tr><td>4</td><td>CI confirmed on all platforms</td><td><strong>Partly</strong></td><td>Ubuntu and Windows pass (Node 20, 22, 24, plus Ubuntu on the newest Node). macOS is not in CI: GitHub cancelled those jobs before a runner started, and there is no Mac available to re-check. Dependabot is limited and grouped to stop queue floods</td></tr><tr><td>5</td><td>Signed releases, provenance, SBOM</td><td><strong>Partly</strong></td><td>
|
|
39
|
+
<div class="table-wrap" tabindex="0" role="region" aria-label="Table 1: Trust and assurance"><table><thead><tr><th scope="col">#</th><th scope="col">Gap</th><th scope="col">Status</th><th scope="col">Evidence and honest limits</th></tr></thead><tbody><tr><td>1</td><td>Real-world accuracy</td><td><strong>Partly</strong></td><td>52 public repositories, 98 labeled findings, three batches, precision with confidence intervals: <a href="14-real-world-evaluation.html">Real-world evaluation</a>. Single labeler, small samples, no recall on real code</td></tr><tr><td>2</td><td>Live proof in AI IDEs</td><td><strong>Pending</strong></td><td>Layouts checked against vendor docs on 2026-11-01; protocol and empty results log: <a href="../compatibility/smoke-tests/index.html">smoke tests</a>. Needs the owner or community with access to each IDE</td></tr><tr><td>3</td><td>Head-to-head comparison</td><td><strong>Pending</strong></td><td>Workflow and script ready (<a href="https://github.com/shahvatsalofficial/archsutra-ai/blob/main/.github/workflows/comparison.yml" rel="noopener"><code>comparison.yml</code></a>, <code>scripts/compare.mjs</code>); first run needs the repository on GitHub with Actions. A plain-AI-prompt comparison is deliberately not offered (not reproducible)</td></tr><tr><td>4</td><td>CI confirmed on all platforms</td><td><strong>Partly</strong></td><td>Ubuntu and Windows pass (Node 20, 22, 24, plus Ubuntu on the newest Node). macOS is not in CI: GitHub cancelled those jobs before a runner started, and there is no Mac available to re-check. Dependabot is limited and grouped to stop queue floods</td></tr><tr><td>5</td><td>Signed releases, provenance, SBOM</td><td><strong>Partly</strong></td><td>GitHub release <code>v0.1.2</code> has the tarball, two CycloneDX SBOMs, <code>SHA256SUMS</code>, and 2 GitHub attestations on digest <code>sha256:4a7f9124501f646dad77799e5c0d3ed6533b9884775904285bc050f77c989018</code>. Scorecard Signed-Releases still scores 0 and does not credit those attestations. npm <code>archsutra-ai@0.1.2</code> and <code>@0.1.3</code> are published from a laptop: no npm provenance, no <code>v0.1.3</code> tag, and <code>0.1.3</code> <code>gitHead</code> is <code>4a7ab27</code> (a commit whose <code>package.json</code> is still 0.1.2)</td></tr><tr><td>6</td><td>OpenSSF Scorecard</td><td><strong>Partly</strong></td><td>Public score <strong>5.7 / 10</strong> on 2026-11-06 for commit <code>06c87c4</code> (Scorecard v5.6.0). Checks at 10 include Security-Policy, Binary-Artifacts, Dependency-Update-Tool, SAST, Token-Permissions, Dangerous-Workflow, Vulnerabilities, and License. Zeros: Branch-Protection, Code-Review, Signed-Releases, Maintained (repository younger than 90 days), CII-Best-Practices, Fuzzing, Contributors. Packaging is -1 (no Actions npm publish). Pinned-Dependencies is 7 (fixture Dockerfiles and the Semgrep pip install are not hash-pinned). CI-Tests is -1 because Scorecard saw no pull request; tests do run on push</td></tr><tr><td>7</td><td>Pinned install and pinned actions</td><td><strong>Done</strong></td><td>README and user guide show tag-pinned <code>npx</code> usage; every GitHub Action in all workflows is pinned to a commit SHA with a version comment, kept fresh by Dependabot</td></tr><tr><td>8</td><td>Performance data</td><td><strong>Done</strong></td><td><a href="../proof/benchmarks.md">Benchmarks</a> (synthetic, 200 to 20 000 files) and real-repository timings in <code>docs/proof/realworld/results*.json</code>. Found and fixed a gap: added a total-bytes budget and cache cap so memory is bounded</td></tr><tr><td>9</td><td>Test coverage</td><td><strong>Done</strong></td><td>92% lines, 81% branches, 91% functions over <code>dist/</code> (<a href="../proof/coverage.txt">report</a>); CI prints coverage on every run. No threshold is enforced yet</td></tr><tr><td>10</td><td>Third-party review</td><td><strong>Not possible yet</strong></td><td>No external review exists. Added: pinned CodeQL, Scorecard, DCO, <code>security.txt</code>, ReDoS and hostile-input tests, and an open invitation in <a href="https://github.com/shahvatsalofficial/archsutra-ai/blob/main/SECURITY.md" rel="noopener">SECURITY.md</a>. Counts as done only after someone independent reports</td></tr><tr><td>11</td><td>Accessibility audit</td><td><strong>Partly</strong></td><td>axe-core on 19 pages x 2 themes: 0 violations after fixing 3 found issues; 22 contrast checks pass (<a href="../proof/accessibility.html">accessibility</a>). No screen-reader test; no formal conformance claim</td></tr><tr><td>12</td><td>Project maturity signals</td><td><strong>Not possible yet</strong></td><td>Added governance, support, templates, CODEOWNERS, <a href="../community/good-first-issues.html">good-first-issue backlog</a>, Discussions. Stars, users and track record take time and cannot be manufactured</td></tr><tr><td>13</td><td>Trademark clearance</td><td><strong>Pending</strong></td><td>Informal checks recorded, no formal search: <a href="../legal/trademark-and-name-clearance.html">name clearance</a>. Needs an attorney</td></tr><tr><td>14</td><td>Contribution policy</td><td><strong>Done</strong></td><td>DCO adopted (<a href="https://github.com/shahvatsalofficial/archsutra-ai/blob/main/DCO.md" rel="noopener">DCO.md</a>) with a CI check that every PR commit is signed off</td></tr><tr><td>15</td><td>Lawyer review</td><td><strong>Pending</strong></td><td>Review-ready brief prepared: <a href="../legal/legal-review-brief.html">legal review brief</a>. Owner must engage counsel</td></tr></tbody></table></div>
|
|
40
40
|
<h2 id="what-a-reader-can-verify-in-five-minutes">What a reader can verify in five minutes<a class="anchor" href="#what-a-reader-can-verify-in-five-minutes" aria-label="Link to this section">#</a></h2>
|
|
41
|
-
<ol><li><code>git clone</code> and <code>npm ci && npm test</code> (
|
|
41
|
+
<ol><li><code>git clone</code> and <code>npm ci && npm test</code> (36 tests).</li><li><code>node scripts/realworld.mjs report holdout2</code> reproduces the headline accuracy table from the committed samples and labels.</li><li>Read ten labels in <code>docs/proof/realworld/labels-holdout2.json</code> and open the cited lines.</li><li><code>npm ls --omit=dev</code> shows no runtime dependencies.</li><li>Check the Actions page for the matrix, CodeQL and (after first run) Scorecard.</li></ol>
|
|
42
42
|
<h2 id="claims-we-deliberately-do-not-make">Claims we deliberately do not make<a class="anchor" href="#claims-we-deliberately-do-not-make" aria-label="Link to this section">#</a></h2>
|
|
43
43
|
<ul><li>No certification of security, compliance or accuracy.</li><li>No claim that "no findings" means "no problems".</li><li>No claim of support in any AI IDE beyond "files written in the documented layout".</li><li>No Scorecard number, star count or user count.</li><li>No comparison win against other scanners.</li></ul>
|
|
44
44
|
<h2 id="what-would-change-these-statuses">What would change these statuses<a class="anchor" href="#what-would-change-these-statuses" aria-label="Link to this section">#</a></h2>
|
|
@@ -47,7 +47,7 @@
|
|
|
47
47
|
</div>
|
|
48
48
|
<footer class="footer">
|
|
49
49
|
<div class="footer-inner">
|
|
50
|
-
<p><strong>ArchSutra AI</strong> v0.1.
|
|
50
|
+
<p><strong>ArchSutra AI</strong> v0.1.4. Design. Audit. Evolve. Created by <a href="https://shahvatsal.com" rel="noopener">Vatsal Shah</a>. Licensed under <a href="https://github.com/shahvatsalofficial/archsutra-ai/blob/main/LICENSE" rel="noopener">Apache-2.0</a>.</p>
|
|
51
51
|
<p class="fine">Review aid only. Not a certification, compliance attestation, security guarantee or legal advice. Not affiliated with Anthropic, OpenAI, Cursor, GitHub or Google; product names belong to their owners. <a href="../handbook/07-security-privacy-legal.html">Legal details</a>.</p>
|
|
52
52
|
</div>
|
|
53
53
|
</footer>
|