loadout-ai 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (125) hide show
  1. package/LICENSE +21 -0
  2. package/MASTER_PLAN.md +1711 -0
  3. package/README.md +453 -0
  4. package/SECURITY.md +18 -0
  5. package/SIMPLE_PLAN.md +44 -0
  6. package/catalog/discovered.json +10469 -0
  7. package/catalog/packages.json +1150 -0
  8. package/dashboard/app.js +607 -0
  9. package/dashboard/index.html +249 -0
  10. package/dashboard/styles.css +384 -0
  11. package/dist/src/cli.js +2735 -0
  12. package/dist/src/core/active-policy.js +233 -0
  13. package/dist/src/core/active-set.js +382 -0
  14. package/dist/src/core/adapters.js +254 -0
  15. package/dist/src/core/adopt.js +92 -0
  16. package/dist/src/core/agent-health-score.js +475 -0
  17. package/dist/src/core/agent-inspection.js +152 -0
  18. package/dist/src/core/agent-versions.js +131 -0
  19. package/dist/src/core/api.js +90 -0
  20. package/dist/src/core/atomic-file.js +22 -0
  21. package/dist/src/core/audit.js +174 -0
  22. package/dist/src/core/benchmark-campaign.js +496 -0
  23. package/dist/src/core/benchmark-evidence.js +458 -0
  24. package/dist/src/core/benchmark-fixtures.js +1157 -0
  25. package/dist/src/core/benchmark-runner.js +301 -0
  26. package/dist/src/core/benchmark-trust.js +795 -0
  27. package/dist/src/core/canary.js +77 -0
  28. package/dist/src/core/candidate-intelligence.js +661 -0
  29. package/dist/src/core/catalog-coverage.js +85 -0
  30. package/dist/src/core/catalog-install.js +186 -0
  31. package/dist/src/core/catalog-release.js +286 -0
  32. package/dist/src/core/catalog.js +327 -0
  33. package/dist/src/core/codex-mcp.js +122 -0
  34. package/dist/src/core/community.js +86 -0
  35. package/dist/src/core/compatibility-intelligence.js +465 -0
  36. package/dist/src/core/completion.js +160 -0
  37. package/dist/src/core/components.js +138 -0
  38. package/dist/src/core/conversion.js +56 -0
  39. package/dist/src/core/credentials.js +377 -0
  40. package/dist/src/core/demo.js +136 -0
  41. package/dist/src/core/diff.js +83 -0
  42. package/dist/src/core/discovery-connector.js +165 -0
  43. package/dist/src/core/doctor.js +95 -0
  44. package/dist/src/core/ecosystem-import.js +1072 -0
  45. package/dist/src/core/evaluate.js +72 -0
  46. package/dist/src/core/file-lock.js +140 -0
  47. package/dist/src/core/freshness-alerts.js +230 -0
  48. package/dist/src/core/github-discovery.js +97 -0
  49. package/dist/src/core/github.js +148 -0
  50. package/dist/src/core/head-to-head.js +437 -0
  51. package/dist/src/core/health-score-evidence.js +169 -0
  52. package/dist/src/core/health.js +129 -0
  53. package/dist/src/core/improve.js +223 -0
  54. package/dist/src/core/install.js +233 -0
  55. package/dist/src/core/intelligence-feed-build.js +59 -0
  56. package/dist/src/core/intelligence-feed.js +353 -0
  57. package/dist/src/core/loadout-badge.js +55 -0
  58. package/dist/src/core/loadout-card.js +87 -0
  59. package/dist/src/core/manifest.js +307 -0
  60. package/dist/src/core/mcp-recipes.js +419 -0
  61. package/dist/src/core/mcp-registry-discovery.js +455 -0
  62. package/dist/src/core/mcp.js +287 -0
  63. package/dist/src/core/model-config.js +102 -0
  64. package/dist/src/core/observations.js +87 -0
  65. package/dist/src/core/outcomes.js +164 -0
  66. package/dist/src/core/package.js +225 -0
  67. package/dist/src/core/paths.js +214 -0
  68. package/dist/src/core/portable.js +130 -0
  69. package/dist/src/core/private-discovery.js +45 -0
  70. package/dist/src/core/profiles.js +255 -0
  71. package/dist/src/core/provenance.js +278 -0
  72. package/dist/src/core/ranking.js +104 -0
  73. package/dist/src/core/recommend.js +181 -0
  74. package/dist/src/core/registry-api.js +97 -0
  75. package/dist/src/core/registry.js +316 -0
  76. package/dist/src/core/release-claims.js +175 -0
  77. package/dist/src/core/remove.js +122 -0
  78. package/dist/src/core/review-queue.js +200 -0
  79. package/dist/src/core/runtime-tool-recipe.js +315 -0
  80. package/dist/src/core/runtime-tools.js +453 -0
  81. package/dist/src/core/safety.js +234 -0
  82. package/dist/src/core/sandbox.js +78 -0
  83. package/dist/src/core/scheduler.js +264 -0
  84. package/dist/src/core/share-report.js +85 -0
  85. package/dist/src/core/signing.js +115 -0
  86. package/dist/src/core/skill-compare.js +239 -0
  87. package/dist/src/core/skill-inventory.js +193 -0
  88. package/dist/src/core/skill-security.js +340 -0
  89. package/dist/src/core/skills-sh-discovery.js +336 -0
  90. package/dist/src/core/skills.js +208 -0
  91. package/dist/src/core/snapshot.js +210 -0
  92. package/dist/src/core/source.js +305 -0
  93. package/dist/src/core/state.js +300 -0
  94. package/dist/src/core/sync.js +348 -0
  95. package/dist/src/core/terminal.js +53 -0
  96. package/dist/src/core/transaction.js +211 -0
  97. package/dist/src/core/update-watch.js +39 -0
  98. package/dist/src/core/update.js +216 -0
  99. package/dist/src/core/upgrade.js +207 -0
  100. package/dist/src/dashboard.js +418 -0
  101. package/dist/src/shared/schemas.js +336 -0
  102. package/dist/src/shared/types.js +1 -0
  103. package/docs/ACTIVE_SET.md +53 -0
  104. package/docs/ACTIVE_SET_POLICY.md +45 -0
  105. package/docs/CANDIDATE_INTELLIGENCE.md +139 -0
  106. package/docs/CATALOG.md +78 -0
  107. package/docs/CATALOG_POLICY.md +41 -0
  108. package/docs/COMMUNITY_DISCOVERY.md +14 -0
  109. package/docs/COMPATIBILITY_POLICY.md +22 -0
  110. package/docs/CONVERSION_AND_SANDBOX.md +27 -0
  111. package/docs/CREDENTIAL_AND_UPDATE_POLICY.md +31 -0
  112. package/docs/DISCOVERED.md +280 -0
  113. package/docs/EVALUATION_PROTOCOL.md +40 -0
  114. package/docs/EVALUATION_PROTOCOL_V1.md +300 -0
  115. package/docs/FEATURE_TEST_MATRIX.md +802 -0
  116. package/docs/GITHUB_AUTHORIZATION.md +38 -0
  117. package/docs/HEAD_TO_HEAD_EVALUATION.md +79 -0
  118. package/docs/PROVENANCE_AND_COMPARISON.md +72 -0
  119. package/docs/PROVIDER_CONFIGURATION.md +45 -0
  120. package/docs/RELEASE_REVIEW.md +117 -0
  121. package/docs/SAFE_UPDATE_DEMO.md +25 -0
  122. package/docs/SCHEMA_DECISIONS.md +25 -0
  123. package/docs/TEAM_POLICY.md +18 -0
  124. package/docs/TESTING.md +190 -0
  125. package/package.json +82 -0
@@ -0,0 +1,38 @@
1
+ # Optional GitHub authorization
2
+
3
+ Public discovery remains anonymous. Private discovery is opt-in and begins only
4
+ when the user explicitly connects GitHub from the local CLI or loopback dashboard.
5
+
6
+ ## Authorization model
7
+
8
+ Use a GitHub App installation, not a broad OAuth `repo` token. The app asks for
9
+ read-only **Contents** access, is installed on repositories selected by the user,
10
+ and uses the user-to-server OAuth flow solely to identify the user's installation.
11
+ The resulting short-lived installation token is scoped by GitHub to that selected
12
+ repository set. Metadata is read-only. No organization administration, workflow,
13
+ issues, pull-request write, or repository-write permission is requested.
14
+
15
+ The app client ID, redirect URI, and application slug may be committed as public
16
+ configuration. The private key and client secret cannot: they belong to a hosted
17
+ broker or a user-controlled local credential manager. Loadout never asks the user to
18
+ paste a token into a manifest, catalog, log, or dashboard URL.
19
+
20
+ ## Local flow and failure modes
21
+
22
+ 1. `loadout connect github` opens a browser using PKCE and a loopback callback.
23
+ 2. The local process verifies `state`, PKCE verifier, expiration, and callback host.
24
+ 3. It stores only an OS-keychain reference to the refresh/session material.
25
+ 4. `loadout discover --private` obtains a short-lived installation token in memory,
26
+ lists only selected repositories, and discards it after the request.
27
+ 5. `loadout disconnect github` deletes the keychain item and local connection record.
28
+
29
+ If no registered GitHub App/client and secure credential backend are configured,
30
+ private discovery stays unavailable and public discovery keeps working. A token that
31
+ cannot be refreshed or no longer has access is reported as disconnected; it is never
32
+ silently retried with a broader scope.
33
+
34
+ ## Non-goals
35
+
36
+ This does not clone private repositories, execute private source, or expose private
37
+ repository names in shared exports. A user must still explicitly add and plan a
38
+ repository before Loadout fetches it.
@@ -0,0 +1,79 @@
1
+ # Head-to-head capability evaluation design
2
+
3
+ This protocol compares two reviewed skill instruction sets on one declared task
4
+ family. It does not rank repositories globally, execute candidate code, or silently
5
+ change an active set.
6
+
7
+ ## Evidence envelope
8
+
9
+ Every run stores:
10
+
11
+ - evaluation schema and harness version;
12
+ - category and fixture ids plus the fixture/rubric SHA-256;
13
+ - candidate package, skill path, reviewed commit, and instruction SHA-256;
14
+ - agent surface, model provider/model/version, sampling settings, and tool policy;
15
+ - per-trial rubric dimensions, grader rationale, raw aggregate, variance, and trial
16
+ count;
17
+ - input/output token counts, reported provider cost or 'unknown', duration, failures,
18
+ and retries;
19
+ - deterministic grader version, human audit status, uncertainty statement, timestamp,
20
+ and machine-verifiable signature.
21
+
22
+ Candidate instructions are data. The harness never runs their scripts, shell commands,
23
+ hooks, MCP servers, network requests, or tool calls. Fixtures use disposable synthetic
24
+ repositories and redact secrets. A model request is permitted only through an explicit
25
+ provider selection and cost ceiling.
26
+
27
+ ## Initial categories
28
+
29
+ ### Workflow adherence
30
+
31
+ Fixtures ask for a change under explicit constraints: inspect before editing, preserve
32
+ unrelated work, write a focused test, verify, and report uncertainty. Rubric:
33
+ constraint recall (25), plan-to-action consistency (20), safe scope (20), verification
34
+ quality (20), and honest completion report (15). A hidden forbidden action produces a
35
+ blocking safety failure rather than a negative quality point.
36
+
37
+ ### Code-review coverage
38
+
39
+ Fixtures contain synthetic diffs with seeded correctness, security, concurrency,
40
+ compatibility, and test-gap defects plus harmless distractors. Rubric: weighted seeded
41
+ defect recall (45), precision (20), severity calibration (15), actionable file/line
42
+ evidence (10), and regression-test advice (10). Findings not grounded in the fixture
43
+ count against precision.
44
+
45
+ ### Documentation retrieval
46
+
47
+ Fixtures provide versioned local documentation and deliberately stale alternatives.
48
+ Rubric: correct source/version choice (30), supported API facts (30), citation
49
+ traceability (20), uncertainty handling (10), and absence of invented APIs (10).
50
+ Network access is disabled so retrieval quality is not confused with internet access.
51
+
52
+ ### Browser-test planning
53
+
54
+ Fixtures provide an HTML/accessibility snapshot, user journey, and failure evidence.
55
+ Rubric: critical-path coverage (25), stable locator strategy (20), accessibility and
56
+ responsive cases (20), isolation/setup (15), failure diagnostics (10), and avoidance
57
+ of brittle timing (10). The first harness plans tests only; browser execution is a
58
+ separate sandboxed verification stage.
59
+
60
+ ## Trial and decision rules
61
+
62
+ - Minimum five trials per candidate/fixture/model; alternate candidate order.
63
+ - Identical prompts, context budget, model version, temperature, tool policy, and
64
+ timeout within a comparison.
65
+ - Report mean, median, standard deviation, failures, and 95% bootstrap interval. Fewer
66
+ than five successful trials is 'insufficient-evidence'.
67
+ - A result is only 'preferred-for-this-category' when the interval clears the declared
68
+ practical-effect threshold and no safety failure exists. Otherwise it is 'tie',
69
+ 'mixed', or 'insufficient-evidence'.
70
+ - One fixture, model, or judge cannot promote a global default. Active-set replacement
71
+ always requires a preview and explicit user approval.
72
+
73
+ ## Signed snapshot boundary
74
+
75
+ The harness writes canonical JSON, hashes fixtures and candidate inputs, and signs the
76
+ evidence envelope with Loadout's existing Ed25519 catalog-signing primitive. Private
77
+ keys stay outside the repository. Verification must fail on any changed score,
78
+ metadata, candidate hash, or rubric. Unsigned local experiments may be displayed but
79
+ cannot influence shared ranking evidence.
@@ -0,0 +1,72 @@
1
+ # Provenance and comparison contract
2
+
3
+ Loadout must answer “where did this skill come from?” and “is there a better reviewed
4
+ alternative?” without turning weak signals into facts. Both workflows are read-only.
5
+
6
+ ## Provenance confidence
7
+
8
+ From strongest to weakest:
9
+
10
+ 1. `catalog-exact` / `exact`: the installed `SKILL.md` SHA-256 is byte-identical to a
11
+ skill indexed from a catalog repository at its reviewed immutable commit.
12
+ 2. `loadout-managed` / `high`: Loadout state owns the installed path. The file may
13
+ still be reported separately as drifted if the user edited it after installation.
14
+ 3. `embedded-source` / `medium`: the skill text contains a GitHub repository that is in
15
+ the reviewed index, but the instruction fingerprint differs. A documentation link
16
+ or fork can create this signal, so it is evidence rather than proof.
17
+ 4. `catalog-name-candidate` / `low`: only the normalized skill name matches. Names are
18
+ not unique and cannot establish authorship, equivalence, or quality.
19
+ 5. `unknown` / `unknown`: no supported evidence matched. Loadout preserves this result
20
+ rather than guessing from a directory name.
21
+
22
+ `loadout scan` uses only an existing local reviewed index. It never performs network
23
+ access unless `--refresh-provenance` is passed. Refresh fetches only catalog repositories
24
+ with reviewed skill evidence, checks out their exact 40-character commit, fingerprints
25
+ their `SKILL.md` files, and writes a local index below `LOADOUT_HOME/provenance`.
26
+
27
+ ## Relationship classification
28
+
29
+ `loadout compare` uses deterministic relationships:
30
+
31
+ - `exact-copy`: identical instruction fingerprint;
32
+ - `divergent-same-name`: same normalized name, different instructions;
33
+ - `overlapping-capability`: at least 30% Jaccard overlap after conservative
34
+ name/description token normalization; and
35
+ - `same-category-candidate`: reviewed catalog category matches but textual overlap is
36
+ weak.
37
+
38
+ Two additional capability-family states are deliberately not inferred from text:
39
+
40
+ - `complementary-capability` requires an explicit reviewed catalog relationship showing
41
+ that the tools solve different parts of a workflow; and
42
+ - `verified-hard-conflict` requires a catalog conflict-family record with hard severity
43
+ and blocks installation until resolved.
44
+
45
+ If neither reviewed relationship exists, Loadout leaves the relationship unclassified
46
+ rather than calling two vaguely related tools complementary or incompatible.
47
+
48
+ Cross-agent installations with the same name and fingerprint are mirrors, not duplicate
49
+ problems. Divergent same-name installations require `--agent` disambiguation. Existing
50
+ catalog conflict families remain the source of verified soft/hard package conflicts.
51
+
52
+ ## Comparison evidence and guardrails
53
+
54
+ Alternatives show repository, reviewed commit, tier, license, deterministic relationship,
55
+ text similarity, and the existing catalog evidence score. Adoption, momentum,
56
+ maintenance, compatibility, permission, and evaluation evidence are printed separately;
57
+ missing evidence is stated instead of being silently treated as good. The score orders
58
+ available catalog evidence; it does not measure model-output quality.
59
+
60
+ The comparison recommendation follows these rules:
61
+
62
+ - strongly attributable installed skills remain in place by default;
63
+ - same-name or overlapping candidates require head-to-head evaluation or explicit user
64
+ experience before replacement;
65
+ - weak same-category candidates are exploration only; and
66
+ - no command changes agent files, activates a skill, or removes an unknown skill.
67
+
68
+ Limitations are explicit: `SKILL.md` fingerprinting does not prove that referenced assets
69
+ are identical, GitHub URLs embedded in instructions do not prove provenance, semantic
70
+ token overlap is not an embedding or behavioral evaluation, and current comparison does
71
+ not claim universal “best.” Phase 12 task evaluations and local human outcomes are
72
+ required before guided optimization may apply a replacement.
@@ -0,0 +1,45 @@
1
+ # Provider-neutral model configuration
2
+
3
+ Loadout can validate and share a model-selection document before it has a
4
+ provider adapter. The schema is provider-neutral: a selection identifies a
5
+ provider, model, HTTPS endpoint, optional credential _reference_, and optional
6
+ target agent IDs. `openrouter` is an ordinary provider identifier, so an
7
+ OpenRouter selection needs no special schema or stored integration.
8
+
9
+ ```json
10
+ {
11
+ "schemaVersion": 1,
12
+ "selections": [
13
+ {
14
+ "id": "coding",
15
+ "provider": "openrouter",
16
+ "model": "anthropic/claude-sonnet-4",
17
+ "endpoint": "https://openrouter.ai/api/v1",
18
+ "credential": { "kind": "environment", "name": "OPENROUTER_API_KEY" },
19
+ "targetAgents": ["codex", "claude-code"]
20
+ }
21
+ ]
22
+ }
23
+ ```
24
+
25
+ ## Credential boundary
26
+
27
+ The document contains only a credential location, never a credential value.
28
+ The accepted references are an environment-variable **name** or an OS-keychain
29
+ service/account reference. It has no `apiKey`, `token`, `authorization`, or
30
+ arbitrary-header fields; strict runtime validation rejects them. Endpoints must
31
+ be credential-free HTTPS URLs (no userinfo, query string, or fragment), and
32
+ secret-looking values are rejected in model/identifier fields.
33
+
34
+ Loadout does not resolve the reference, write it to state/lockfiles, log it,
35
+ or transmit it in this design. A future provider adapter may read the named
36
+ environment variable or keychain entry only at execution time, and must redact
37
+ it from errors, plans, snapshots, logs, exports, and telemetry. That adapter is
38
+ explicitly outside P11-05.
39
+
40
+ ## Deliberate scope
41
+
42
+ This schema declares intent; it does not configure Codex, Claude, Cursor, or
43
+ any provider, call OpenRouter, select a model automatically, or store provider
44
+ headers. Keeping those side effects out of the schema layer makes the document
45
+ portable and safe to review in version control.
@@ -0,0 +1,117 @@
1
+ # Release review — 2026-07-15
2
+
3
+ This review covers the current Loadout implementation, not an aspirational
4
+ roadmap. It was performed after the transaction, source-fetch, dashboard, and
5
+ adapter test suites passed locally.
6
+
7
+ ## P4-08: atomic-commit review — accepted with explicit durability boundary
8
+
9
+ Mutation metadata that can make a completed install unrecoverable now uses a
10
+ temporary sibling file followed by `rename`:
11
+
12
+ - `~/.loadout/state.json`
13
+ - `loadout.lock`
14
+ - edits to an existing `loadout.json`
15
+ - MCP JSON/TOML config writers already used the same-directory temporary-file
16
+ pattern.
17
+
18
+ The replacement prevents readers from observing a partially written JSON file
19
+ on local filesystems with atomic same-directory rename support. Snapshots are
20
+ created before mutation and restoration is exercised by tests. CI run
21
+ `29401149042` executed the atomic-file and transaction suites on Node 20 and
22
+ 22 for Windows, macOS, and Linux. The decision is therefore accepted for the
23
+ supported local-filesystem scope.
24
+
25
+ This is not a claim of durable, power-loss-safe multi-file transactions: a
26
+ process or system failure can still leave either the old or new version of an
27
+ individual file. Transaction journals recover interrupted multi-file work before
28
+ the next synchronization; users needing database-grade durability should not rely
29
+ on filesystem rename alone.
30
+
31
+ ## P7-15: product and security review — pass with stated boundaries
32
+
33
+ The reviewed release flow is appropriate for a hackathon demo and local use:
34
+
35
+ - Plans are read-only until an explicit apply command or dashboard action.
36
+ - Install/update safety requires approval for scripts, hooks, binaries, new
37
+ domains, environment references, suspicious instructions, and MCP changes.
38
+ - Snapshots are taken before managed mutations; failures restore them.
39
+ - Repository cloning does not run package lifecycle scripts. Generic Git URLs
40
+ with embedded credentials, query strings, or fragments are rejected so a
41
+ failed Git command cannot echo a token.
42
+ - The API is loopback-only. The dashboard also validates the actual socket peer
43
+ instead of trusting a spoofable `Host` header; mutation endpoints require a
44
+ random per-process session token and reject cross-origin requests.
45
+ - The dashboard and CLI summaries avoid emitting MCP environment values.
46
+
47
+ Known boundaries remain intentional: Loadout does not execute third-party
48
+ install scripts, silently replace existing Codex TOML MCP tables, or support
49
+ private repository credentials. It should not be marketed as a full arbitrary
50
+ plugin executor.
51
+
52
+ ## P9-07: cross-platform go/no-go — bounded go for native skills
53
+
54
+ CI run `29401149042` planed, installed, byte-verified, and removed a real
55
+ `SKILL.md` through every declared agent-owned skill layout on Windows, macOS,
56
+ and Linux, using disposable native home and state directories. **Current
57
+ decision:** go for the bounded native-skill-directory claim. The no-go remains
58
+ for plugins, hooks, executables, and arbitrary MCP runtimes.
59
+
60
+ ## Local verification
61
+
62
+ On 2026-07-15 the corrective audit also verified:
63
+
64
+ ```text
65
+ npm run format:check
66
+ npm run lint
67
+ npm run typecheck
68
+ npm test -- --run # 55 files, 169 tests
69
+ npm run build
70
+ npm run test:e2e # Chromium first-run preview/apply flow
71
+ ```
72
+
73
+ The audit fixed these release blockers and added regressions for them:
74
+
75
+ - Vitest now assigns every test file disposable Loadout and user homes; a CRLF
76
+ fixture can no longer write into the developer's real `~/.loadout`.
77
+ - A stale refresh cache overlays mutable metadata onto the bundled catalog instead
78
+ of hiding newer bundled records or retaining deleted cache-only packages.
79
+ - The Maximum profile references only catalog packages that actually exist.
80
+ - Dashboard Installed and Updates routes have unique container/list IDs and are
81
+ exercised through navigation in Playwright.
82
+ - Demo output distinguishes planned skill directories from tracked files instead of
83
+ printing an impossible installed/total fraction.
84
+ - Multi-file synchronization now writes a durable journal and recovers interrupted
85
+ work even when the next synchronization is otherwise empty.
86
+ - The npm tarball exposes the `loadout` executable, carries runtime code/assets but
87
+ not compiled tests, and resolves its catalog/dashboard independently of the current
88
+ working directory.
89
+
90
+ An outside-checkout 72-file tarball smoke test launched version `0.1.0`, read all 20
91
+ catalog records, and served the packaged dashboard. The live isolated demo fetched
92
+ `obra/superpowers`, planned 14 skill directories and 48 files, then verified rollback
93
+ and removed its temporary profile. The npm package is named `loadout-ai`; the shorter
94
+ `loadout` registry name is owned by an unrelated package.
95
+
96
+ ## CLI-first product correction
97
+
98
+ The dashboard is now explicitly secondary. Running `loadout` in an interactive terminal
99
+ starts Maximum/Stable/Custom onboarding, while `loadout setup --mode maximum` provides
100
+ a scriptable read-only preview and `--yes --approve-risk` applies the reviewed result.
101
+
102
+ Catalog setup now:
103
+
104
+ - filters out MCP/executable-only records before cloning;
105
+ - fetches up to four repositories concurrently with visible progress;
106
+ - fetches the catalog's exact reviewed commit rather than mutable default-branch HEAD;
107
+ - reuses only a clean cache whose Git HEAD matches that commit;
108
+ - resolves duplicate skill targets by retaining the higher-ranked reviewed source and
109
+ reporting every lower-ranked duplicate;
110
+ - installs all remaining packages through one transaction and one restore point;
111
+ - keeps local health checks network-free unless `--updates` is explicitly requested.
112
+
113
+ A disposable real Maximum Boost run prepared all 31 skill-bearing repositories from the
114
+ 50-record catalog, deferred 19 explicit MCP-only setups, exposed 1,219 skill directories,
115
+ and resolved 48 duplicate target directories. The same isolated flow exercised library
116
+ inspection, project optimization, apply, and rollback successfully before removing its
117
+ test home. No real user agent directory was used.
@@ -0,0 +1,25 @@
1
+ # Safe update demonstration
2
+
3
+ Loadout treats an update as a reviewable filesystem transaction; it never runs
4
+ repository scripts, hooks, package-manager lifecycle commands, or MCP servers.
5
+
6
+ The regression demo uses only the inert local repositories in
7
+ `tests/fixtures/update-safety`:
8
+
9
+ 1. `benign-v1` to `benign-v2` is a documentation-only skill update. Loadout
10
+ copies it, validates the copied `SKILL.md`, and records the new commit.
11
+ 2. `risky-v2` adds a hook-shaped shell file and an external domain. Static
12
+ inspection marks it as requiring explicit approval. Without that approval,
13
+ Loadout writes review metadata to its quarantine directory and leaves the
14
+ installed files and state untouched.
15
+ 3. A simulated post-copy static verification failure restores the exact
16
+ transaction snapshot, including the previous skill bytes and install state.
17
+
18
+ Run the evidence locally:
19
+
20
+ ```bash
21
+ npm test -- --reporter=verbose tests/update-safety-demo.test.ts
22
+ ```
23
+
24
+ The hook fixture contains a marker command solely to prove that inspection does
25
+ not execute it. The test asserts that the marker is absent.
@@ -0,0 +1,25 @@
1
+ # Runtime schema decision
2
+
3
+ Loadout treats catalog JSON, shared manifests and lockfiles, persisted install
4
+ state, and inter-module install plans as untrusted data. `src/shared/schemas.ts`
5
+ is the single runtime-validation boundary for those structures. It uses Zod so
6
+ the parsed output has inferred TypeScript types as well as a path-aware runtime
7
+ error for the CLI and tests.
8
+
9
+ The schemas intentionally model identifiers, source provenance, agent IDs,
10
+ component kinds, hashes, and safety levels. They do not define fields for MCP
11
+ environment values, API keys, tokens, or other secret values. MCP environment
12
+ values remain in their agent-owned configuration and are only summarized using
13
+ counts/names elsewhere in the product.
14
+
15
+ Compatibility and failure handling:
16
+
17
+ - New manifest, lockfile, and state writers always emit schema version 1.
18
+ - Lockfiles produced before `generatedAt` existed remain readable as the
19
+ explicit legacy value `unknown`; newly written lockfiles always include the
20
+ timestamp.
21
+ - Unknown catalog and policy fields are retained where required for existing
22
+ portable-export secret scanning. They are never interpreted as executable
23
+ configuration by the schema layer.
24
+ - Invalid data fails before install, audit, or update processing with the exact
25
+ field path (for example, `installs.0.targetAgents.0`).
@@ -0,0 +1,18 @@
1
+ # Team policy and audit schema
2
+
3
+ A shared Loadout is a versioned declarative manifest plus a signed catalog reference;
4
+ it never contains credentials or private repository names by default. Team policy has
5
+ four independent controls:
6
+
7
+ - allowlist: package IDs/repository identities eligible for planning;
8
+ - denylist: identities, domains, component types, or risk findings that block;
9
+ - required approvals: roles required for risky changes; and
10
+ - audit events: timestamp, actor label, manifest/catalog hashes, decision, and reason.
11
+
12
+ The serverless/local MVP stores policy in a project-controlled JSON file and validates
13
+ it before a plan. There is no hidden admin override. A later hosted product can add
14
+ identity and retention without changing the portable policy semantics.
15
+
16
+ The current manifest implementation enforces package/repository allowlists and
17
+ denylists before synchronization, alongside the existing blocked-domain and
18
+ blocked-command rules. Violations are included in the dry-run plan and stop apply.
@@ -0,0 +1,190 @@
1
+ # Testing Loadout as a product
2
+
3
+ The primary product is the CLI. Use this walkthrough before installing into your real
4
+ Codex, Claude Code, or other agent directories.
5
+
6
+ The automated product journey runs the built CLI against disposable user, state, and
7
+ project directories. It performs a real scan and offline provenance comparison, then
8
+ previews and applies project optimization, verifies the installed bytes, and rolls the
9
+ snapshot back:
10
+
11
+ ```bash
12
+ npm run test:e2e:cli
13
+ ```
14
+
15
+ This test does not use the dashboard, network, mock command output, or any real agent
16
+ profile. It is a required CI gate on Ubuntu; the manual cross-platform workflow runs
17
+ the broader native filesystem suite.
18
+
19
+ ## 1. Build the exact npm package entry point
20
+
21
+ ```bash
22
+ cd /path/to/loadout
23
+ npm ci
24
+ npm run build
25
+ npx . --help
26
+ ```
27
+
28
+ `npx .` runs the same `loadout` executable that `npx loadout-ai` will run after npm
29
+ publication.
30
+
31
+ Before creating a disposable profile, the only recommended real-profile command is the
32
+ read-only inventory:
33
+
34
+ ```bash
35
+ npx . scan
36
+ ```
37
+
38
+ It reports actual `SKILL.md` capabilities, Loadout ownership, duplicates, and capacity
39
+ warnings. It does not treat unmanaged content as unsafe and does not change any agent.
40
+
41
+ To build the local reviewed provenance index from exact catalog commits and compare a
42
+ reported skill:
43
+
44
+ ```bash
45
+ npx . scan --refresh-provenance
46
+ npx . compare <skill-name>
47
+ ```
48
+
49
+ The first command writes only Loadout's local cache. `compare` is read-only and uses
50
+ `--offline` when a test must forbid network fallback. A same-name match is a candidate,
51
+ not proof of provenance or quality.
52
+
53
+ To inspect and exercise the managed active set after installing a package:
54
+
55
+ ```bash
56
+ npx . library
57
+ npx . disable <managed-package> # dry-run
58
+ npx . disable <managed-package> --yes # cache, verify, deactivate
59
+ npx . enable <managed-package> # dry-run
60
+ npx . enable <managed-package> --yes # verify and reactivate
61
+ ```
62
+
63
+ Copy the snapshot id printed by either applied command to test
64
+ `npx . rollback --snapshot <id>`. These commands refuse unmanaged packages, drifted
65
+ managed content, incomplete library copies, quarantined entries, and occupied enable
66
+ targets.
67
+
68
+ ## 2. Create a completely disposable agent profile
69
+
70
+ macOS or Linux:
71
+
72
+ ```bash
73
+ TEST_HOME="$(mktemp -d)"
74
+ export LOADOUT_USER_HOME="$TEST_HOME/user"
75
+ export LOADOUT_HOME="$TEST_HOME/state"
76
+ mkdir -p "$LOADOUT_USER_HOME/.codex" "$LOADOUT_USER_HOME/.claude"
77
+ ```
78
+
79
+ PowerShell:
80
+
81
+ ```powershell
82
+ $TestHome = Join-Path $env:TEMP ("loadout-test-" + [guid]::NewGuid())
83
+ $env:LOADOUT_USER_HOME = Join-Path $TestHome "user"
84
+ $env:LOADOUT_HOME = Join-Path $TestHome "state"
85
+ New-Item -ItemType Directory -Force (Join-Path $env:LOADOUT_USER_HOME ".codex") | Out-Null
86
+ New-Item -ItemType Directory -Force (Join-Path $env:LOADOUT_USER_HOME ".claude") | Out-Null
87
+ ```
88
+
89
+ Those empty directories make Loadout detect virtual Codex and Claude Code profiles.
90
+ Every write stays below the disposable path.
91
+
92
+ ## 3. Preview Power, then inspect Maximum Library
93
+
94
+ Power is the broad daily-use profile:
95
+
96
+ ```bash
97
+ npx . setup --mode power
98
+ ```
99
+
100
+ Expect a curated skill-level set across the strongest reviewed collections. Stable
101
+ remains the smallest Superpowers + Context7 option. Maximum prepares the full library:
102
+
103
+ ```bash
104
+ npx . setup --mode maximum
105
+ ```
106
+
107
+ This is read-only. Expect the CLI to show:
108
+
109
+ - detected agents;
110
+ - reviewed catalog repositories fetched at pinned commits;
111
+ - actual skill-directory targets, not only repository count;
112
+ - lower-ranked duplicate skills that were deferred;
113
+ - MCP-only repositories requiring explicit configuration;
114
+ - safety categories requiring a separate approval.
115
+
116
+ No agent skill directory or Loadout install state is created by preview.
117
+
118
+ ## 4. Download the reviewed library
119
+
120
+ ```bash
121
+ npx . setup --mode maximum --yes --approve-risk
122
+ ```
123
+
124
+ `--approve-risk` acknowledges the displayed scripts, domains, environment names, or
125
+ instruction findings inside the reviewed skill content. Loadout copies skills into its
126
+ disabled library but does not execute repository installation or lifecycle scripts.
127
+
128
+ The command should finish with one snapshot identifier. The library download is one
129
+ transaction: a failure restores every library/state target rather than leaving a
130
+ half-installed profile.
131
+
132
+ ## 5. Inspect the installed product
133
+
134
+ ```bash
135
+ npx . list
136
+ npx . status
137
+ npx . health
138
+ npx . library
139
+ npx . optimize --project .
140
+ npx . optimize --project . --yes
141
+ ```
142
+
143
+ The first optimize command is a dry run. The second activates only reviewed,
144
+ project-relevant skill units, verifies their hashes, and prints the exact snapshot
145
+ rollback command.
146
+
147
+ `health` is local and fast by default. Use `npx . health --updates` or
148
+ `npx . update` when you intentionally want live network update checks.
149
+
150
+ You can also inspect the disposable files directly:
151
+
152
+ ```bash
153
+ find "$LOADOUT_USER_HOME/.agents/skills" -name SKILL.md | wc -l
154
+ find "$LOADOUT_USER_HOME/.claude/skills" -name SKILL.md | wc -l
155
+ ```
156
+
157
+ ## 6. Verify rollback
158
+
159
+ ```bash
160
+ npx . rollback
161
+ npx . list
162
+ ```
163
+
164
+ The second command should report that no Loadout-managed packages are installed. Files
165
+ that existed before setup must remain byte-identical.
166
+
167
+ ## 7. Test the real interactive experience
168
+
169
+ After the disposable run succeeds, open a fresh terminal without `LOADOUT_HOME` or
170
+ `LOADOUT_USER_HOME` overrides and run:
171
+
172
+ ```bash
173
+ npx .
174
+ ```
175
+
176
+ Choose Maximum, Stable, or Custom, review the plan, and confirm only when you want to
177
+ write to the detected real agent profiles. After npm publication, replace `npx .` with
178
+ `npx loadout-ai`.
179
+
180
+ ## Optional dashboard
181
+
182
+ The dashboard is a secondary inspection surface, not the onboarding requirement:
183
+
184
+ ```bash
185
+ npx . dashboard
186
+ ```
187
+
188
+ Open the printed loopback URL. CLI setup, updates, removal, discovery, and rollback all
189
+ work without it. Browser automation is also optional and runs only when manually
190
+ dispatched in CI; locally, use `npm run test:e2e:dashboard`.