@skein-code/cli 0.3.5 → 0.3.7

package/docs/ARCHITECTURE.md

@@ -14,8 +14,8 @@
                │                      │
 ┌──────────────▼─────────────┐  ┌─────▼────────────────────────┐
 │       Context fabric       │  │         Trust layer          │
-│ ContextEngine CLI          │  │ workspace boundary           │
-│ local BM25/path/symbol     │  │ allow / ask / deny           │
+│ local BM25/path/symbol     │  │ workspace boundary           │
+│ language-aware chunks      │  │ allow / ask / deny           │
 │ @file resolver             │  │ command policy · hooks       │
 │ token-budgeted packer      │  │ checkpoints · audit events   │
 └──────────────┬─────────────┘  └─────┬────────────────────────┘
@@ -34,7 +34,7 @@
 ## Agent turn
 
 1. Resolve `@path` mentions inside configured workspace roots.
-2. Ask the selected context engine for task-relevant spans under the configured
+2. Ask the local context engine for task-relevant spans under the configured
    token budget.
 3. Combine product rules, project rules, retrieved spans, mentions, current plan,
    and conversation history.
@@ -62,36 +62,37 @@ This lets providers reuse stable prompt prefixes while ensuring a changed plan
 or newly retrieved file is visible on the next turn. Every retrieved or generated
 state block is marked as untrusted context and cannot authorize a tool call.
 
-## Context selection
+## Local context selection
 
-`context.engine: auto` is the recommended setting.
+The context boundary is deliberately in-process and local:
 
 ```text
-              contextengine executable healthy?
-                         /          \
-                       yes          no
-                       /              \
-        hybrid external retrieval    local incremental index
-        symbols + vectors + graph     BM25 + path + symbol boosts
-                       \              /
-                        token-budget pack
+workspace files
+      |
+manifest + freshness checks
+      |
+language-aware chunks + BM25/path/symbol/CJK signals
+      |
+verified, diverse spans under token budget
+      |
+untrusted evidence block for the model
 ```
 
-The fallback is intentional. A missing database, embedding endpoint, or external
-binary should reduce retrieval quality, not make the coding agent unusable.
+Index state is persisted in the active project namespace. Search results are
+revalidated against current files before packing, so a stale index reduces
+recall rather than silently injecting old code.
 
 ## Storage
 
 Project-local data is kept in `.mosaic/` and ignored by default:
 
 - `config.json` — project overrides;
-- `index.json` — local fallback index;
+- `index.json` — local retrieval index;
 - `sessions/` — auditable conversation and tool state;
 - `checkpoints/` — pre-mutation file snapshots and manifests.
 
 No source content is sent anywhere except the model endpoint selected by the
-user. With local-compatible model and local retrieval endpoints, the complete
-stack can remain self-hosted.
+user. With a local-compatible model, the complete stack can remain self-hosted.
 
 Durable memory uses SQLite in WAL mode with FTS5 and bounded lexical fallback;
 it does not require a hosted vector service. Records carry scope, kind,
@@ -131,3 +132,33 @@ useful without silently accumulating guesses.
   `PATH` entries, and reports non-zero exits as failed tool results. Operations
   that can invoke transport, signing, merge, or checkout helpers also require
   shell permission.
+
+## Capability extension policy
+
+Skein keeps a small built-in tool kernel: read, list, search, write, patch,
+shell, Git, tasks, and working memory. Built-ins use closed input schemas,
+explicit permission categories, workspace-root resolution, bounded inputs and
+outputs, cancellation, checkpoints for known writes, and persisted audit
+events. New built-ins must satisfy the same contract; a convenience wrapper is
+not enough reason to enlarge the kernel.
+
+Skills and workflows are the preferred plugin surface for reusable guidance.
+They are data-only prompt additions, carry their source and trust state, and
+cannot grant permissions or execute code by being loaded. Arbitrary in-process
+JavaScript plugins are intentionally unsupported because they would share the
+CLI's full filesystem, environment, and process privileges.
+
+MCP is the interoperability boundary for external executable capabilities. It
+is disabled by default, removed from untrusted project configuration, exposes
+namespaced tools, treats server annotations as untrusted, and applies argument,
+schema, result, server-count, timeout, and transport limits. Every MCP call
+currently requires the network permission category. A configured stdio server
+is still an external program with the user's operating-system privileges; cwd
+and environment validation reduce accidental exposure but are not a sandbox.
+Only reviewed user-owned configuration should enable one.
+
+Before any marketplace-style plugin support, add a declarative capability
+manifest, first-run review, lazy tool-schema activation, per-server permission
+scopes, and an optional process sandbox. Plugin packages should compose Skills,
+workflows, and MCP servers rather than load arbitrary code into the Skein
+process.

package/docs/NEXT_STEPS.md

@@ -9,12 +9,12 @@ one of the milestones below.
 
 - Product name: `Skein`; primary executable: `skein`.
 - Compatibility executables: `mosaic` and `mosaic-code`.
-- Current release: `0.3.0`.
+- Current repository version: `0.3.5`.
 - Runtime requirement: Node.js `>=22.16.0` (the runtime uses unflagged
   `node:sqlite` with FTS5, and current CLI/build dependencies require this
   Node 22 baseline).
-- Retrieval: local BM25/path/symbol index with automatic ContextEngine-plugin
-  detection and fallback.
+- Retrieval: local BM25/path/symbol index with freshness validation and bounded
+  packing; no retrieval service is required.
 - Agent: provider-agnostic multi-turn runner for OpenAI, Anthropic, Gemini, and
   OpenAI-compatible endpoints; built-in tools, permissions, checkpoints,
   workflows, Skills, MCP, expert profiles, sessions, and memory are present.
@@ -47,7 +47,7 @@ archive it describes.
 The final verification included a fresh install and real PTY interaction for
 all three executable aliases, `/about`, a permission prompt, denial, and clean
 Ctrl+C exit. PTY coverage included 20, 24 ASCII, 40, 80, 120 columns and a
-40x10 short-height case. The current suite contains 34 test files and 317 tests.
+40x10 short-height case. The current 35-file test suite passes the full check.
 
 ## Recommended Order
 
@@ -140,53 +140,29 @@ Implementation progress:
   concrete paths involved. `skein doctor` surfaces this as `legacyCompatibility`
   so users see the removal timeline before aliases disappear.
 
-### P1: ContextEngine-Plugin Production Adapter
+### P1: Local Context Engine Reliability And Benchmarking
 
-Exercise the adapter against a real ContextEngine-plugin fixture, not only the
-local fallback. Cover capability negotiation, index progress, search/context
-packing, unavailable PostgreSQL/pgvector, stale indexes, multi-root workspaces,
-and a useful degraded-mode explanation in the TUI and headless output.
+Keep retrieval local and measurable as the repository grows. The next slice
+should add content hashes to index entries, a small generation-keyed query cache,
+overlap-aware packing, and language adapters for common declaration styles.
 
 Definition of done:
 
-- `auto` selects the external engine only when its health contract is valid.
-- A failed external query falls back without losing the user request.
-- Results preserve source paths, symbols, line ranges, scores, and token caps.
-- An integration fixture runs in CI without requiring a developer database.
+- Editing a file without changing its size cannot leave a stale hit in a prompt.
+- New, deleted, renamed, binary, symlinked, and out-of-root files are covered by
+  regression tests.
+- A benchmark reports Recall@5/10/20, MRR, stale-hit rate, useful-token ratio,
+  and cold/incremental/warm latency for curated multilingual queries.
+- `npm run check` and `npm run test:pty` pass without a service or downloaded
+  model.
 
 Implementation progress:
 
-- `auto` now negotiates the real 0.4 CLI boundary using `--version`, required
-  help flags, exit behavior, and strict required response fields. Capability
-  probes are coalesced, cached for ten seconds after completion, refreshable,
-  and recover when an executable is installed while Skein is running.
-- Compatible but unindexed workspaces use ContextEngine only for `index`;
-  `search` and `context` fall back locally. Explicit `contextengine` mode makes
-  missing, incompatible, unhealthy, and unindexed states hard failures, and
-  `doctor` treats that explicit requirement as required.
-- External index progress is parsed from the current CR/ANSI human stream and
-  finalized only after the JSON result validates. Multi-root aliases use
-  `main`, `workspace2`, and later numbered roots; single-root directories with
-  those literal names remain ordinary paths.
-- Search/context hits are schema-checked, realpath-bound, hash-checked, and
-  compared with current file lines. A stale or invalid hit rejects the entire
-  external response; empty auto-mode results are cross-checked locally so new
-  files are not missed. Synthetic commit-lineage hits are reconstructed with a
-  constrained read-only Git command. Skein ignores external `packedText` and
-  repacks verified current-file or commit-summary bytes under its own top-K and
-  token limits.
-- External processes use a private temporary working directory and a minimal
-  `CONTEXTENGINE_*`/proxy/certificate environment. Repository `.env` files and
-  generic chat-model credentials are not inherited. External failures and
-  unknown index fields are redacted or stripped before reaching output.
-- Structured degradation now reaches the TUI, headless text/final JSON,
-  `skein search/context`, and `doctor`; narrow TUI layouts show a separate
-  fallback reason and remediation row.
-- `test/fixtures/contextengine-cli.mjs` is a faithful CLI-boundary fixture, not
-  an embedded fake database. It covers unavailable PostgreSQL/pgvector,
-  unindexed and malformed contracts, stale/current/empty hits, commit-lineage
-  verification, multi-root mapping, oversized progress, credential redaction,
-  cache recovery, and degraded channels without requiring PostgreSQL in CI.
+- The public `pack/search/index/status` boundary is now a pure local façade.
+- Legacy external configuration is stripped at the config schema boundary and
+  no external executable or database is probed by the CLI.
+- Remaining work is content-hash freshness, cache invalidation, structured
+  chunking, and the reproducible benchmark.
 
 ### P1: Multi-Agent Scheduler And Team UX
 
@@ -284,6 +260,10 @@ Add a first-run catalog and inspection flow for bundled capabilities. Keep
 installation explicit and reviewable: show source, requested tools, filesystem
 scope, network scope, and trust state before activation. Add fixture servers and
 Skills that exercise timeout, malformed schema, disconnect, and version drift.
+Introduce lazy MCP schema discovery and declarative capability manifests before
+any marketplace. Do not load arbitrary plugin JavaScript in-process; package
+reusable extensions as data-only Skills/workflows plus explicitly trusted MCP
+servers, with an optional subprocess sandbox for stdio servers.
 
 ### P2: Memory Quality And User Control
 
@@ -323,8 +303,8 @@ deprecation window is complete.
   responsive panels.
 - `src/agent/runner.ts` — model/tool loop, context events, verification, and
   delegation boundaries.
-- `src/context/context-engine.ts` — external/local retrieval selection.
-- `src/context/local-index.ts` — offline fallback index.
+- `src/context/context-engine.ts` — local retrieval façade.
+- `src/context/local-index.ts` — persisted local index, scoring, and packing.
 - `src/mcp/manager.ts` — MCP lifecycle and tool registration.
 - `src/skills/catalog.ts` — Skills discovery and activation.
 - `src/memory/store.ts` and `src/tools/working-memory.ts` — durable and

package/docs/PRODUCT.md

@@ -47,27 +47,14 @@ Sources:
 - [Sandboxing](https://geminicli.com/docs/cli/sandbox/)
 - [CLI reference](https://geminicli.com/docs/cli/cli-reference/)
 
-### ContextEngine-plugin
-
-ContextEngine-plugin provides the strongest open integration path for code
-retrieval: PostgreSQL FTS, symbols, paths, optional pgvector embeddings, graph
-expansion, reranking, and token-budgeted MMR packing. Its CLI contract maps well
-to an agent: `index`, `search`, `context`, and `status`.
-
-Sources:
-
-- [ContextEngine-plugin](https://github.com/lixiang12345/ContextEngine-plugin)
-- [Architecture](https://github.com/lixiang12345/ContextEngine-plugin/blob/main/ARCHITECTURE.md)
-- [Comparison with Augment](https://github.com/lixiang12345/ContextEngine-plugin/blob/main/COMPARISON.md)
-
 ## Product principles
 
 1. **Context before tools.** Every task begins with retrieval and explicit file
    mentions are merged into the evidence packet.
 2. **Open by default.** Headless automation is a core feature, not an enterprise
    entitlement.
-3. **Local first, not local only.** The built-in index works without a service;
-   ContextEngine is selected automatically when installed.
+3. **Local by default.** Retrieval is inspectable, offline-capable, and does not
+   require a database, daemon, embedding download, or external executable.
 4. **Bring your own model.** OpenAI, Anthropic, Gemini, and OpenAI-compatible
    endpoints share one agent contract.
 5. **Trust is visible.** Tool intent, permission category, result, changed files,
@@ -81,7 +68,7 @@ Sources:
 
 | Surface | Capability |
 |---|---|
-| Context | ContextEngine adapter, built-in BM25/path/symbol index, token packing, multi-root, `@file` mentions |
+| Context | Local BM25/path/symbol index, freshness checks, token packing, multi-root, `@file` mentions |
 | Models | OpenAI, Anthropic, Gemini, OpenAI-compatible endpoints |
 | Agent | Multi-turn tool loop, task plan, automatic verification, ask-only mode |
 | Tools | Read, list, search, write, patch, shell, Git, context search, task updates |

package/docs/PRODUCT_BENCHMARK.md

@@ -20,7 +20,7 @@ product surfaces.
 
 | Area | Mainstream signal | Skein today | Product implication |
 | --- | --- | --- | --- |
-| Context | Auggie automatically indexes projects and offers context-aware interactive and print modes. Its MCP Tool Search avoids loading every remote schema up front. See [Auggie overview](https://docs.augmentcode.com/cli/overview) and [integrations](https://docs.augmentcode.com/cli/integrations). | Local BM25/path/symbol retrieval plus a behavior-negotiated ContextEngine adapter, verified current-file/commit packing, empty-result cross-checks, observable fallback, progressive Skills, and hard token caps. Filesystem freshness remains `unknown` until each hit is consumed; doctor reports semantic readiness and query telemetry reports degraded channels. MCP schemas are registered eagerly. | Make MCP tool discovery lazy and measurable, and add proactive channel state to the TUI without pretending status alone proves freshness. |
+| Context | Auggie automatically indexes projects and offers context-aware interactive and print modes. Its MCP Tool Search avoids loading every remote schema up front. See [Auggie overview](https://docs.augmentcode.com/cli/overview) and [integrations](https://docs.augmentcode.com/cli/integrations). | Local BM25/path/symbol retrieval with language-aware chunks, current-file freshness checks, diversity-aware token packing, progressive Skills, and hard token caps. MCP schemas are registered eagerly. | Benchmark local recall and latency by language, then make MCP tool discovery lazy and measurable. |
 | Workflow modes | Claude Code documents isolated subagents, agent teams, hooks, code intelligence, Skills, MCP, and plugins in one extension model. See [Claude Code extensions](https://code.claude.com/docs/en/features-overview). Copilot CLI exposes Plan and Autopilot modes. See [Copilot CLI](https://github.com/features/copilot/cli). | Ask and Build modes exist; Ask is read-only but does not produce a named approval-ready plan. | Add an explicit Plan mode. Keep it read-only and require approval before Build. |
 | Code intelligence | Claude Code advertises language-server-backed symbol navigation and live type errors. | Retrieval is lexical/BM25/path/symbol index based; no LSP diagnostics or rename graph. | Add an optional LSP adapter after storage and scheduler foundations. |
 | Parallel work | Claude documents isolated subagents and agent teams; Copilot CLI offers background delegation and fleet-style parallel work. | Routed multi-model councils share bounded reports, observe/guard/strict task-budget policies, reviewer acceptance/revision, and a responsive Team Cockpit. The main agent remains the only writer. | Add worktree-isolated writers, deterministic integration, and conflict/rollback UX without weakening the visible review gate. |
@@ -48,7 +48,7 @@ product surfaces.
    reviewer loop, then add cancellation propagation, deterministic integration,
    and single-writer/worktree boundaries.
 6. **P1 code intelligence:** add optional LSP diagnostics and symbol actions
-   without making the local index or external ContextEngine mandatory.
+   without making the local index mandatory.
 7. **P2 trust and sharing:** first-run capability review, sandbox adapters,
    redacted review bundles, and explicit privacy/export/delete controls.
 

package/examples/config.yaml

@@ -8,10 +8,8 @@ model:
   maxTokens: 8192
 
 context:
-  engine: auto
   maxTokens: 12000
   topK: 12
-  contextEngineCommand: contextengine
 
 permissions:
   read: allow

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@skein-code/cli",
-  "version": "0.3.5",
+  "version": "0.3.7",
   "description": "A context-first, model-agnostic coding agent with an auditable terminal workspace.",
   "type": "module",
   "license": "MIT",
@@ -16,8 +16,9 @@
   },
   "skein": {
     "releaseNotes": [
-      "New ANSI Shadow \"SKEIN\" logotype gives the entry banner a real product wordmark",
-      "Wordmark carries a subtle theme-aware gradient; narrow and ASCII terminals fall back cleanly"
+      "Terminal startup no longer leaks Kitty keyboard probes or wraps long model names into stray rows",
+      "The fresh-session summary is denser and verified across narrow, wide, and short terminal layouts",
+      "Prompt assembly now keeps optional orchestration guidance out of simple runs and strengthens language, user-work, and verification guardrails"
     ]
   },
   "bin": {
@@ -43,6 +44,7 @@
     "test:watch": "vitest",
     "pretest:pty": "npm run build",
     "test:pty": "sh test/pty/run-visual.sh",
+    "benchmark:context": "tsx scripts/benchmark-local-index.ts",
     "verify:package": "node scripts/verify-package.mjs",
     "release:verify": "npm run check && npm run verify:package --",
     "typecheck": "tsc --noEmit",