akm-cli 0.10.261012-beta → 0.10.261014-beta

package/docs/architecture/internals/functional-contract-patterns.md

@@ -1,340 +0,0 @@
-# Functional Contract Patterns
-
-Quick reference for the repeated design patterns used in akm's refactor away
-from type-centric behavior.
-
----
-
-## 1. Core Rule
-
-Model behavior around core processes, not around one large `asset type` object.
-
-Use:
-
-- `type` as data
-- small contracts as extension seams
-- fixed-stage pipelines as orchestrators
-
-Do not use:
-
-- one mega interface with many optional methods
-- open-ended plugin graphs
-- process logic hidden behind global type switches
-
----
-
-## 2. Repeated Pattern
-
-For each process:
-
-1. define a small context object
-2. define a narrow contributor interface
-3. register ordered contributors
-4. let each contributor decide `appliesTo(...)`
-5. keep one central orchestrator for stage order
-
-This is the standard pattern to repeat across the codebase.
-
-Contributor invariants:
-
-1. registration is static and in-process only
-2. ordering is deterministic
-3. composition semantics are declared once per seam
-4. execution is sequential by default
-5. contributors do not dispatch into other registries directly
-6. if a seam needs more machinery than this, it is probably too abstract
-
----
-
-## 3. Pattern Catalog
-
-## 3.1 Fixed-Stage Pipeline
-
-Use when a process already has a clear top-down flow.
-
-Examples:
-
-- search
-- improve
-- indexing
-
-Shape:
-
-```ts
-for (const stage of stages) {
-  for (const contributor of contributorsFor(stage)) {
-    if (!contributor.appliesTo(ctx)) continue;
-    contributor.run(ctx);
-  }
-}
-```
-
-Why:
-
-- preserves readability
-- makes order explicit
-- prevents plugin soup
-
----
-
-## 3.2 Ordered Contributor Registry
-
-Use when one stage needs several isolated policies.
-
-Examples:
-
-- ranking signals
-- proposal validators
-
-Shape:
-
-```ts
-interface Contributor<TContext> {
-  name: string;
-  order?: number;
-  appliesTo?(ctx: TContext): boolean;
-}
-```
-
-Why:
-
-- testable in isolation
-- easy to add behavior without editing the orchestrator
-
----
-
-## 3.3 Structural Contract
-
-Use for stable physical concerns such as refs and paths.
-
-Example:
-
-- `PathResolver`
-
-Why:
-
-- storage layout is real and durable
-- should not be mixed with ranking, rendering, or validation
-
-Rule:
-
-- keep structural contracts small and boring
-
----
-
-## 3.4 Classification As Facts
-
-Classification should produce facts, not choose downstream behavior too early.
-
-Good output:
-
-- `type`
-- `specificity`
-- annotations or reasons
-
-Avoid:
-
-- coupling classification directly to renderer names or search policy
-
----
-
-## 3.5 Process-Local Validation
-
-Validation belongs to the process that needs it.
-
-Examples:
-
-- proposal validation
-- lint validation
-- improve preflight validation
-
-Avoid one shared validator that tries to know every process.
-
----
-
-## 3.6 One Pipeline, Many Signals
-
-Search must remain one pipeline.
-
-Use contributors for:
-
-- exact-match boosts
-- type preference boosts
-- belief-state boosts
-- graph boosts
-- utility boosts
-
-Do not create separate per-type scoring pipelines.
-
----
-
-## 3.7 Adapters Before Rewrites
-
-First move existing behavior behind the new seam.
-
-Only after parity is proven should behavior be reorganized.
-
-Why:
-
-- lower risk
-- easier regression testing
-- architectural progress without feature churn
-
----
-
-## 3.8 Refactor-Only Safety Rule
-
-When the work is explicitly architectural cleanup:
-
-- no functionality changes
-- all tests must pass with the same validation expectations
-- the only allowed test edits are import-path or symbol-import updates caused by file moves
-- do not rewrite assertions, fixtures, or expected outputs to accommodate the refactor
-
-This keeps architectural cleanup separate from feature or bug-fix work.
-
----
-
-## 3.8a Non-goals
-
-This pattern guide is **not** permission to:
-
-1. build a framework
-2. add complexity for its own sake
-3. create registries everywhere by default
-4. replace simple code with abstract dispatch when no real hotspot exists
-5. introduce dynamic plugin systems or runtime discovery
-6. change runtime behavior under the banner of refactoring
-
-The purpose of these patterns is to remove concrete duplication and switchboard logic while keeping the system simpler to reason about than it is today.
-
----
-
-## 3.9 Execution Pipeline Contract
-
-Every execution crosses the same plain functions:
-
-1. `resolveExecution()` selects the engine (nearest layer, `defaults.engine`,
-   then the `opencode-sdk` fallback), merges the engine's defaults with the
-   caller's layers (nearest wins), expands a `models.json` alias once,
-   authorizes tools against `execution.allowedTools`, and returns the
-   `ResolvedExecutionRequestV1`, the engine's `RunnerSpec` with the request
-   applied, and per-field provenance. Credentials stay symbolic.
-2. `buildExecution()` hands the request to the harness's own builder (derived
-   from `HARNESS_REGISTRY`), or builds chat messages for a direct LLM. A field
-   the transport cannot carry becomes a secret-free notice; a tool policy it
-   cannot enforce, or a denied tool selection, is a pre-dispatch error.
-3. `buildExecutionFromWire()` does the same from a journaled
-   `{ request, runner }` without reading config, aliases, environment
-   variables, or credentials — workflow resume.
-4. `runExecution()` reads credentials for the call, runs the agent CLI, the
-   OpenCode SDK, or the chat transport (an exhaustive switch over
-   `llm | agent | sdk`), and redacts every secret the child could have seen.
-
-A prompt-free interactive native-agent launch (`akm agent`) resolves its
-engine directly; it carries no request to build.
-
-Diagnostic provenance is field metadata (`field`, `layer`, `kind`, `via`), not
-resolved values or content. Lowering notices are secret-free records with a
-fixed public vocabulary; arbitrary authored inference keys are represented by
-wildcard field names rather than copied into diagnostics.
-
----
-
-## 3.10 Session Log Harness Contract
-
-Use one narrow raw-event ingestion seam for harness logs and session histories.
-
-Rule:
-
-1. harness adapters discover files and parse raw events
-2. shared AKM logic performs normalization, fingerprinting, aggregation, and de-duplication
-3. new harnesses should not require edits to shared unions or duplicated aggregation logic
-
----
-
-## 4. Implemented Contracts
-
-**[0.9.0 change, ruled Q-06/Q-16]** This section previously listed twelve
-"recommended" contracts; seven had no implementation anywhere in `src/`
-(`PathResolver`, `MatchContributor`, `MetadataContributor`,
-`LintContributor`, `ImproveContributor`, `IndexPostProcessor`,
-`AgentRunner`) and were aspirational, not a contract any code followed.
-Trimmed to the contributor contracts below. `SearchHitEnricher` and
-`ActionContributor` were later folded into direct calls
-(`src/indexer/search/search-hit-enrichers.ts`, `buildLocalAction` in
-`src/indexer/search/db-search.ts`) — each had one implementation. The engine-owned
-`AgentRequestLowerer` is the separate structural boundary documented in
-§3.9, not a revival of the removed generic `AgentRunner`. See the [drift
-register](../specs/0.9.0-docs-code-drift-register.md#q-16--functional-contract-patternsmd-4-aspiration-or-contract)
-if a removed entry needs reviving; add it back only alongside a real
-implementation.
-
-## 4.1 `RankingContributor`
-
-For search score adjustments and explanations.
-
-## 4.2 `ProposalValidator`
-
-For proposal acceptance checks.
-
-## 4.3 `SessionLogHarness`
-
-For raw session-log or history ingestion from external harnesses.
-
----
-
-## 5. What Stays Centralized
-
-- source/provider model
-- CLI command contracts
-- DB schema and persistence boundaries
-- write-target resolution
-- search stage order
-- final score normalization and clamping
-- proposal storage
-
-If these become pluggable, the system becomes harder to reason about.
-
----
-
-## 6. What Gets Delegated
-
-- heuristics
-- enrichment
-- validation rules
-- process-specific side-effects
-- applicability logic
-- search explanations
-
-If these stay centralized, switchboards keep growing.
-
----
-
-## 7. Smells To Avoid
-
-1. One large interface with many optional methods
-2. Registries keyed only by `type`
-3. Matching logic that also chooses presentation names
-4. Renderers that also own indexing and search policy
-5. Ad hoc per-command path lookup logic
-6. Open-ended capability graphs with unclear precedence
-7. Hidden special cases spread across callers instead of isolated behind one seam
-8. Architectural cleanup that changes behavior or rewrites tests to fit the refactor
-
----
-
-## 8. Review Checklist
-
-When adding a new seam, ask:
-
-1. Is this a stable process boundary?
-2. Can the contributor be tested in isolation?
-3. Does orchestration remain readable top-down?
-4. Is ordering explicit?
-5. Is `type` only an input, not the behavior object?
-6. Can existing behavior be adapted before rewriting it?
-7. Is this a refactor-only change with behavior preserved?
-8. Does onboarding a new external harness require only one narrow adapter?
-
-If the answer is no to several of these, the abstraction is probably too large.

package/docs/architecture/internals/registry-network-boundary.md

@@ -1,57 +0,0 @@
-# Registry network boundary
-
-All HTTP(S) traffic used to discover or install registry content passes
-through `fetchRegistry` / `fetchRegistryJson` in `src/registry/network.ts`.
-The boundary is plain `fetch()` plus a request timeout, bounded retries on
-network failure / timeout / 429 / 5xx (honoring a capped `Retry-After`), a
-byte cap on the response body, and classified errors: a request that fails
-after retries, or whose body can't be used, always leaves as an `AkmError` —
-never a raw, unclassified exit 70.
-
-- Not found / unusable response (HTTP 404/410, or a body that isn't valid
-  JSON / isn't shaped as expected) -> `NotFoundError` (exit 1).
-- Every other non-2xx status, and connection failure / timeout after retries
-  are exhausted -> `TransientError` (`REGISTRY_UNREACHABLE`, exit 75).
-- A malformed registry URL, an embedded `user:pass@`, or a non-http(s) scheme
-  -> `ConfigError` (`REGISTRY_URL_INVALID`, exit 78).
-
-Callers: `src/registry/providers/static-index.ts`, `providers/skills-sh.ts`,
-`src/registry/resolve.ts` (npm and GitHub metadata), `src/setup/registry-stash-loader.ts`,
-and `src/sources/providers/provider-utils.ts`'s `downloadArchive` (npm tarball
-download, used by `sources/providers/npm.ts`). Credentials (a resolved GitHub
-token, etc.) are injected by the caller as ordinary `headers` on the request —
-this module does not resolve credentials itself. HTTPS is not required by
-`fetchRegistry` itself: `--allow-insecure-transport`/HTTPS enforcement happens
-once, where a URL enters configuration (`akm registry add`, `akm bundle add`),
-per `AGENTS.md`'s "Unsafe CLI overrides" policy.
-
-## What this boundary does *not* do
-
-Earlier releases pinned every registry request's DNS resolution: they
-resolved and classified the hostname's A/AAAA answers, rejected loopback /
-private / link-local / cloud-metadata / reserved addresses, connected to the
-one validated numeric address (never re-resolving at connect time), and
-re-validated on every retry and redirect hop. Because Bun's fetch client
-cannot pin a TLS connection to a literal address while retaining SNI, that
-DNS-pinned transport spawned a short-lived Node child process per request to
-do the actual connect, over a bounded stdin/stdout framing protocol.
-
-That pinning, and the subprocess that implemented it
-(`pinned-transport.ts`, `pinned-request-helper.ts`), were deleted as
-over-engineering for this codebase's actual threat model: registry URLs are
-either the built-in registry or a URL an operator explicitly configured
-(`akm registry add`, `AKM_REGISTRY_URL`, `AKM_NPM_REGISTRY`) — a deliberate
-human command, not third-party input akm fetches on an untrusted caller's
-behalf (see `AGENTS.md`'s "Defensive Code" test 3). The subprocess-per-request
-design was also the direct cause of registry network failures surfacing as an
-unclassified exit 70 instead of a typed error, a symptom fixed repeatedly
-before this simplification (see CHANGELOG history around the registry HTTP
-stack). `src/core/network-policy.ts`'s address classifier is unrelated to this
-boundary now; it still backs the `website` source provider's own SSRF guard
-(`src/sources/snapshot-fetchers/host-guard.ts`), which fetches arbitrary
-operator-pasted URLs and keeps its own justification for that guard.
-
-Registry-derived `source: git` entries are still rejected before an install
-ref can reach `git ls-remote` or clone — that check lives in
-`src/registry/providers/static-index.ts` and is independent of the network
-boundary.

package/docs/architecture/internals/search.md

@@ -1,248 +0,0 @@
-# Search Architecture
-
-Search uses a multi-signal scoring pipeline combining lexical matching, semantic
-similarity, and relevance boosts to find the most useful assets for a query.
-
-## Pipeline Overview
-
-```
-Query
-  │
-  ├─ FTS5 (lexical) ──────────────┐
-  │   Multi-column BM25 with      │
-  │   field weighting              │   Normalize + combine
-  │                                ├──────────────────────── Score + Boost ── Sort ── Return
-  └─ Vector (semantic) ───────────┘        0.7 FTS          │
-      Cosine similarity, exact             0.3 Vec          │
-      scan of stored vectors                                │
-                                                            │
-                                              Boosts applied:
-                                              • exact name match
-                                              • type relevance
-                                              • tag/alias/hint match
-                                              • description match
-                                              • metadata quality
-                                              • usage history (M-2)
-```
-
-## Indexed Search (primary)
-
-When an index exists (`~/.local/share/akm/index.db`), local search uses two
-ranking signals:
-
-### 1. FTS5 (lexical)
-
-SQLite full-text search with Porter stemming, using **multi-column field
-weighting**. The FTS5 table has separate columns for different metadata fields,
-each weighted differently in the BM25 scoring:
-
-| Column | BM25 Weight | Contents |
-| --- | --- | --- |
-| `name` | 10.0 | Asset name |
-| `description` | 5.0 | Description text |
-| `tags` | 3.0 | Tags and aliases |
-| `hints` | 2.0 | Search hints and examples |
-| `content` | 1.0 | bounded body projection, TOC headings, parameters |
-
-A name match is weighted 10x higher than a content match. This ensures that
-searching "docker" ranks the `docker-homelab` skill above a knowledge doc that
-merely mentions Docker in its table of contents.
-
-**Progressive fallback:** Search runs strict AND, then prefix-AND, then one
-OR/prefix-OR recovery only if both strict forms return no candidates. Unicode
-letter/number tokens are deduplicated and capped; callers do not strip
-stopwords or maintain separate result collections.
-
-**`matchStage`:** Each hit that has an FTS component reports which rung of
-the ladder produced it via `matchStage`: `"exact"` (strict AND), `"prefix"`
-(prefix AND), or `"relaxed"` (OR/prefix-OR recovery). The field is omitted
-for hits with no FTS contribution (e.g. a pure-semantic hybrid match) and for
-hits that never go through the lexical ladder (registry/browse hits). Because
-the ladder stops at the first non-empty stage, all FTS-only hits in a single
-response share one stage; in `hybrid` ranking mode a response can legitimately
-mix hits that carry `matchStage` with hits that omit it.
-
-### 2. Semantic (vector)
-
-Cosine similarity between query embedding and stored entry embeddings.
-Requires an embedding model — either the declared external
-`@huggingface/transformers` dependency (default model
-`bge-small-en-v1.5`) or a remote OpenAI-compatible endpoint. AKM imports the
-package directly and does not copy, wrap, alias, or hash a second runtime under
-`src/` or `dist/`. Dependency installation and platform support remain the
-upstream package manager's responsibility. Users who need a different runtime
-or native/GPU-class throughput can configure a remote embedding endpoint.
-
-An LRU cache (100 entries) avoids redundant embedding computation for repeated
-queries.
-
-Query-time semantic failure is not conflated with configured keyword search.
-FTS still serves the request, while `searchMode: "fts-fallback"` and one
-sanitized warning name the normalized endpoint without URL userinfo, query
-parameters, fragments, API keys, or raw runtime error text. Curate deduplicates
-that warning across its internal fallback queries. `searchMode: "keyword"`
-means no ready semantic attempt failed for this request.
-
-### Score Normalization
-
-FTS5 BM25 scores are negative (lower = better match). They are normalized to
-a **0.3–1.0 range** using min-max normalization across the result set:
-
-- Best FTS match → 1.0
-- Worst FTS match → 0.3 (floor prevents zero-score entries)
-
-When vector search is also active, scores are combined with weighted addition:
-
-```
-base_score = 0.7 × normalized_bm25 + 0.3 × cosine_similarity
-```
-
-**Design decision — why not RRF?** An earlier version used Reciprocal Rank
-Fusion (RRF) to merge FTS and vector results. RRF uses rank positions instead
-of raw scores, which avoids scale mismatch but destroys score differentiation.
-With RRF (K=60), the best result scores 1/61 = 0.0164 and the 5th result
-scores 1/65 = 0.0154 — a 6% difference that makes all results look equally
-relevant. Normalized BM25 + weighted combination preserves the actual relevance
-signal: the best match might score 1.0 while the 5th scores 0.45 — a 55%
-difference that enables meaningful ranking. See the
-[Scoring Design](#scoring-design) section below for the full rationale.
-
-## Scoring Design
-
-After normalization, multiplicative boosts are applied in a single pass:
-
-```
-final_score = base_score × (1 + sum_of_boosts)
-```
-
-### Boost signals (ordered by strength)
-
-| Signal | Boost | When it fires |
-| --- | --- | --- |
-| Exact name match | +2.0 | Query exactly equals the asset's base name |
-| Alias exact match | +1.5 | Query exactly matches a defined alias |
-| Near-exact name | +1.0 | Query is a substring of the name or vice versa |
-| Name token overlap | +0.3/token (max 0.9) | Individual query tokens found in name |
-| Type: skill | +0.4 | Asset is a skill |
-| Type: command / workflow | +0.35 | Asset is a command or workflow |
-| Type: agent | +0.3 | Asset is an agent |
-| Type: knowledge / fact / instruction | +0.22 | Asset is a knowledge doc, fact, or project instruction file |
-| Type: script | +0.2 | Asset is a script |
-| Type: memory | -0.02 | Asset is a memory (slight demotion) |
-| Tag exact match | +0.15/tag (max 0.3) | Query token exactly matches a tag |
-| All-token description | +0.25 | Every query token appears in description |
-| Partial description | +0.1 | Some query tokens in description |
-| Search hint match | +0.12/hint (max 0.24) | Query token found in a search hint |
-| Alias token match | +0.3 | Query token found in any alias |
-| Curated metadata | +0.05 | Non-generated metadata (quality signal) |
-| Confidence | up to +0.05 | Based on metadata source reliability |
-| Usage history (M-2) | up to +0.5 (capped 1.5×) | Utility score from usage telemetry |
-
-### Design principles
-
-**Exact name match is the strongest signal.** If a user types "docker-homelab",
-the asset named `docker-homelab` must rank first with a decisive score gap.
-This is the single most predictable and important ranking behavior.
-
-**Actionable assets rank above reference material.** Skills, commands, and
-agents are things you can execute or dispatch. Knowledge docs are reference
-material — valuable but secondary when a user is searching for something to
-use. The type boost encodes this hierarchy.
-
-**Author-curated signals are stronger than auto-generated ones.** Tags,
-aliases, and search hints are explicitly chosen by the asset author. They
-carry more intent than terms that happen to appear in file content.
-
-**Boosts are multiplicative on the base score.** This means a high base score
-(strong FTS match) amplified by boosts produces a much larger gap than a low
-base score with the same boosts. Highly relevant assets separate clearly from
-marginally relevant ones.
-
-### Worked example
-
-Query: `"docker homelab"` → asset: `skills/docker-homelab`
-
-```
-Base BM25 (normalized):  1.0    (best FTS match in result set)
-+ Name token overlap:    +0.6   (2 tokens match: "docker", "homelab")
-+ Type boost (skill):    +0.4
-+ Tag exact match:       +0.3   (tags "docker" and "homelab" both match)
-+ Alias token match:     +0.3   (alias "docker-compose" contains "docker")
-+ Search hint match:     +0.24  (hints contain "docker" and "homelab")
-+ All-token description: +0.25  (all query tokens in description)
-+ Curated quality:       +0.05
-─────────────────────────────
-BoostSum:                2.14
-Final score:             1.0 × (1 + 2.14) = 3.14
-```
-
-Compare to a sub-reference knowledge doc for the same query:
-
-```
-Base BM25 (normalized):  0.85   (good FTS match but not the best)
-+ Name token overlap:    +0.3   (1 token "homelab" in path-derived name)
-+ Type boost:            +0.0   (knowledge docs get no type boost)
-+ Tag match:             +0.15  (1 tag matches)
-─────────────────────────────
-BoostSum:                0.45
-Final score:             0.85 × (1 + 0.45) = 1.23
-```
-
-The skill scores 3.14 vs the sub-reference at 1.23 — a 2.6× gap that clearly
-communicates which result is more useful.
-
-## Result Merging
-
-There is one scoring pipeline because there is one data store. All sources —
-filesystem, git, website, npm — are materialised to local directories and
-indexed together. The indexer holds every searchable asset, so source results
-do not need cross-merging.
-
-### Source + registry results (`--from all`)
-
-When registry results (static-index, skills.sh) are included:
-
-- Source hits and registry hits stay in **separate response fields**:
-  `hits` and `registryHits`
-- Registry results are never rank-merged into `hits`
-- Registry raw scores (which may be on a 0-100 scale) do not leak through
-
-## Substring Fallback
-
-When no index is available, search falls back to scanning the primary bundle
-and installed bundle roots, filtering by substring match. This ensures search
-always works, even before `akm index` has been run.
-
-## Output Modes
-
-### Standard output
-
-By default (`--format json`, `--detail brief`), search emits minimal fields:
-`type`, `name`, `ref`, `action`, and `estimatedTokens`.
-
-`--detail normal` emits `type`, `name`, `description`, `action`, `score`, and
-`estimatedTokens`, plus optional warning, quality, or env-key fields.
-`--detail full` adds `whyMatched`, `origin`, `path`, and timing data.
-
-### Agent-optimized output
-
-For each materialized local hit, `--detail agent` includes `name`, canonical
-`ref`, `type`, absolute `path`, current-policy `editable`, `description`,
-`action`, `score`, and optional `estimatedTokens`/env keys. `editHint` appears
-only when `editable` is `false`. The hint is supplemental: `action` remains the
-normal show, run, or use action. Registry-only hits do not acquire local access
-fields.
-
-`--format jsonl` outputs one JSON object per line for streaming consumption.
-
-## Explainability
-
-`whyMatched` explains which signals contributed to a hit's ranking. Examples:
-
-- `"exact name match"` — query matched the asset name exactly
-- `"skill type boost"` — asset is a skill (actionable)
-- `"matched tags"` — query token matched a tag
-- `"near-exact name match"` — query is a substring of the name
-- `"usage history boost"` — utility score from M-2 telemetry
-
-Visible in `--detail full` output.