@heytherevibin/skillforge 0.8.0 → 0.10.1

package/RELEASING.md

@@ -5,7 +5,7 @@
 | Workflow | File | When it runs |
 |----------|------|----------------|
 | **CI** | [.github/workflows/ci.yml](.github/workflows/ci.yml) | Every **push** and **pull request** to `main`; also **`workflow_dispatch`** (run manually from the Actions tab) |
-| **Skillforge release** | [.github/workflows/release.yml](.github/workflows/release.yml) | When a **`v*`** tag is **pushed** to the repository (e.g. `v0.2.1`) |
+| **Skillforge release** | [.github/workflows/release.yml](.github/workflows/release.yml) | When a **`v*`** tag is **pushed** to the repository (e.g. `v0.10.1`) |
 
 In the GitHub UI, open **Actions** and look for **CI** and **Skillforge release** (not the older “publish to npm” name).
 
@@ -28,14 +28,14 @@ You need the **`NPM_TOKEN`** repository secret.
 
 Create one at [npm → Access Tokens](https://www.npmjs.com/settings/~/tokens) (**Generate New Token** → **Granular Access Token**). Optionally explore [**trusted publishing** (OIDC)](https://docs.npmjs.com/trusted-publishers/) later to avoid long-lived tokens.
 
-1. On `main`, set **`version`** in `package.json` to the version you are releasing (must be **unused** on npm — e.g. `0.2.1`).
-2. Match **MCP** `serverInfo.version` in `python/app/mcp_server.py` and add a **`CHANGELOG.md`** section for that version.
+1. On `main`, set **`version`** in `package.json` to the version you are releasing (must be **unused** on npm — e.g. `0.10.1`).
+2. Match **MCP** `serverInfo.version` in `python/app/mcp_server.py` (must equal `package.json`) and add a **`CHANGELOG.md`** section for that version.
 3. Commit and **`git push origin main`**. Wait for **CI** to pass.
 4. Create a tag whose name is **`v` + that exact version**:  
-   `git tag v0.2.1 && git push origin v0.2.1`
+   `git tag v0.10.1 && git push origin v0.10.1`
 5. Open **Actions → Skillforge release**. The job will **fail the version check** if the tag does not match `package.json`.
 6. Confirm on npm: `npm view @heytherevibin/skillforge version`  
-   Confirm the **GitHub Release** exists with title **`Skillforge <tag>`** (e.g. **`Skillforge v0.2.1`**) and the `.tgz` asset.
+   Confirm the **GitHub Release** exists with title **`Skillforge <tag>`** (e.g. **`Skillforge v0.10.1`**) and the `.tgz` asset.
 
 Scoped packages require a **public** publish; the workflow already runs `npm publish --access public`.
 
@@ -67,6 +67,10 @@ Typical causes: the tag was created before **Skillforge release** existed on `ma
 1. Bump `version` in `package.json`, commit, push `main`.
 2. Push a **new** tag: `v` + new version.
 
+## Bundled skills gate
+
+The **minimum** number of **`skills/**/**/SKILL.md`** files required in CI is **`ci/bundle-gate.json`** → **`minSkillMdFiles`**. Change that value when your distribution policy for the vendored catalog changes (do not edit the number inline in **`ci.yml`**).
+
 ## Local sanity checks (before push)
 
 ```bash
@@ -74,10 +78,18 @@ node --check bin/cli.js && node --check lib/packs.js
 npm test
 ```
 
-Python (syntax only):
+Python (syntax only)—the **authoritative** module list is in **`.github/workflows/ci.yml`** (step “Check Python syntax”). Example:
 
 ```bash
-for f in python/app/main.py python/app/mcp_server.py python/app/events_cli.py python/app/materialize.py python/app/db_paths.py python/app/route_cli.py python/app/mcp_contract.py python/app/chunking.py python/app/project_index.py python/app/index_cli.py python/app/context_fusion.py python/app/redaction.py python/app/route_policies.py python/app/routing_signals.py; do python3 -m py_compile "$f"; done
+for f in \
+  python/app/main.py python/app/mcp_server.py python/app/events_cli.py python/app/materialize.py \
+  python/app/db_paths.py python/app/route_cli.py python/app/mcp_contract.py python/app/chunking.py \
+  python/app/project_index.py python/app/index_cli.py python/app/context_fusion.py python/app/redaction.py \
+  python/app/route_policies.py python/app/routing_signals.py python/app/route_quality.py \
+  python/app/route_eval_harness.py python/app/eval_cli.py python/app/health_cli.py \
+  python/app/feedback_meta.py python/app/weights_cli.py; do
+  python3 -m py_compile "$f"
+done
 ```
 
 ## Troubleshooting: `EOTP` / one-time password in CI

package/SECURITY.md

@@ -2,30 +2,78 @@
 
 ## Supported versions
 
-Security fixes are applied to the **latest** published minor release on the `main` branch. Older tags may not receive backports; upgrade to the current release when possible.
+Security fixes are applied to the **latest published release** on the default branch (**`main`**). Older release lines may not receive backports; upgrade to the current release when advisories apply.
 
 ## Reporting a vulnerability
 
-Please **do not** open a public GitHub issue for undisclosed security vulnerabilities.
+Please **do not** open a **public** GitHub issue for undisclosed security vulnerabilities.
 
 **Preferred:** use [GitHub Security Advisories](https://github.com/heytherevibin/skillforge/security/advisories/new) for this repository (if you have access).
 
-**Alternative:** open a **private** report via GitHub **Security → Report a vulnerability** on the repository, or contact the maintainer through a channel they publish on their GitHub profile.
+**Alternative:** **Security → Report a vulnerability** on the repository, or contact maintainers through channels published on their GitHub profiles.
 
 Include:
 
-- Description of the issue and impact
-- Steps to reproduce (if known)
-- Affected version or commit (if known)
+- Description of the issue and assessed impact  
+- Steps to reproduce (if known)  
+- Affected **version**, **tag**, or **commit** (if known)  
 
-We aim to acknowledge valid reports within a few business days.
+We aim to **acknowledge** valid reports within a few **business days**. Resolution timelines depend on severity and validation effort.
 
-## npm and supply chain
+---
 
-- For **`NPM_TOKEN`** in GitHub Actions, use a [**granular access token**](https://docs.npmjs.com/about-access-tokens) with **read and write** to your scope and **Bypass 2FA** enabled, so **`npm publish`** does not return **`EOTP`**. Legacy automation/classic tokens are no longer available on npm as of 2025.
-- Keep **2FA** enabled on the npm account that owns the `@heytherevibin` scope.
-- Prefer pinning action versions or reviewing Dependabot PRs before merge.
+## Threat model (summary)
 
-## Runtime security
+| Component | Exposure |
+|-----------|-----------|
+| **MCP stdio server** | Runs locally; accepts JSON-RPC from the parent **MCP host** process. Treat the host as part of your trust boundary. |
+| **CLI** | Spawns Python modules with environment inherited from the shell; avoid untrusted **`PATH`** or wrapper scripts. |
+| **SQLite databases** | Contain prompts (redacted where configured), events, embeddings metadata, and learning weights. Protect filesystem permissions and backups. |
+| **Optional Anthropic API** | Outbound HTTPS when **`ANTHROPIC_API_KEY`** is set; prompts and router context may leave the machine per Anthropic’s policies. |
 
-Skillforge’s published surface is **local**: **stdio MCP** and **CLI** commands. Keep your **MCP host** and **`ANTHROPIC_API_KEY`** (if used) within a trusted environment. Do not commit secrets.
+Skillforge does **not** implement network listening for the published OSS package (no HTTP API in current releases).
+
+---
+
+## Data handling & privacy
+
+- **Redaction** (`SKILLFORGE_REDACT_*`) is **pattern-based** and **best-effort**—not a substitute for data-classification or DLP controls.  
+- **Project routing notes** (`project_notes` / aliases in policies) are **prefixed** to the internal routing query only when **`project_root`** is set—reducing accidental global application from org-wide env JSON.  
+- Operators should assume **route events** and **`_meta`** may contain **sensitive path fragments** unless redaction is enabled and effective for their workload.
+
+---
+
+## Secrets & configuration
+
+- **Never commit** `ANTHROPIC_API_KEY`, npm **`NPM_TOKEN`**, or workspace tokens.  
+- Prefer **OS secret stores**, **CI secrets**, or MCP host **env injection**.  
+- Review **`policies.json`** and **`SKILLFORGE_ROUTE_POLICIES`** for **prompt injection** risk: notes influence embeddings for the declared project only, but can still steer retrieval.
+
+---
+
+## Supply chain & distribution
+
+### npm
+
+- Use **granular access tokens** with the minimum scope required; enable **Bypass 2FA** only on dedicated automation tokens when CI publishing requires it (see [RELEASING.md](RELEASING.md)).  
+- Keep **2FA** enabled on maintainer accounts that own the **`@heytherevibin`** scope.  
+- Review **Dependabot** / dependency updates before merge for typosquatting and breaking changes.
+
+### CI
+
+- Workflows run with **`contents: read`** on the default **CI** job; verify **`release.yml`** permissions when extending publish steps.  
+- Prefer **pinned** major versions of third-party Actions or follow your org’s pinning policy.
+
+---
+
+## Runtime hardening (recommended)
+
+- Run Skillforge on **managed workstations** or **locked-down CI agents** appropriate to your data classification.  
+- Separate **production** and **development** orchestrator databases if weights or events must not mix.  
+- Periodically run **`skillforge health`** and **`skillforge route-eval`** after upgrades to validate embeddings and bundled catalog integrity in your environment.
+
+---
+
+## Coordinated disclosure
+
+If you are preparing **public** research or a conference talk that names Skillforge, please contact maintainers **before** the embargo date so fixes or guidance can ship alongside disclosures when appropriate.

package/STRATEGY.md

@@ -1,24 +1,50 @@
-# Skillforge strategy (Cursor-first)
+# Skillforge — product strategy
 
-## Product intent
+## Mission
 
-Skillforge is an **npm-packaged skill orchestrator**: it routes tasks to a small set of **SKILL.md** documents (embeddings + optional Haiku) and returns their bodies for agent context. **Per-project** learning and telemetry live under **`<workspace>/.skillforge/`** when **`project_root`** (or **`SKILLFORGE_PROJECT_ROOT`**) is set; otherwise the global DB under **`~/.skillforge/data/`** is used.
+Skillforge is a **local, MCP-first orchestration layer** that selects a **small, relevant** subset of **`SKILL.md`** (and optional indexed project) context for each user task—so agent hosts can stay **grounded** without loading entire catalogs.
 
-## Surfaces (today)
+## Principles
 
-- **MCP** (`skillforge mcp`): primary — `route_skills`, `list_skills`, feedback tools, `materialize_project`, `skillforge_bootstrap`.
-- **Terminal**: `skillforge events --watch` with optional **`--project-root`**; `skillforge route`, `skillforge index`.
+| Principle | Implication |
+|-----------|-------------|
+| **Local-first** | Default data plane is **SQLite** and **stdio MCP** on the operator’s machine. |
+| **Explicit trust** | Optional **LLM** stages run only when keys and modes allow; **host** mode delegates final skill choice to the MCP client. |
+| **Observable** | Versioned **`_meta`**, route **events**, and operator CLIs (**`events`**, **`health`**, **`route-eval`**) support enterprise debugging and CI gates. |
+| **Portable learning** | Weights and feedback are **exportable**—suitable for backup, migration, and controlled restore. |
+| **Governable** | **Regex policies** and **project overlays** (exclude, boost, notes) let platform teams steer routing without forking the catalog. |
 
-## Cursor reality
+## Primary surfaces
 
-Native **`/skillforge`** in editor chat is **not** registered by this npm package. The practical pattern is **Cursor rules** (e.g. materialized **`skillforge.mdc`**) instructing the agent to call MCP tools; users may *say* `/skillforge` as shorthand.
+| Surface | Role |
+|---------|------|
+| **MCP (`skillforge mcp`)** | Integration for Claude, Cursor, Claude Code, and other JSON-RPC MCP hosts. |
+| **CLI** | Parity routing, indexing, observability, preflight, eval harness, weights I/O. |
 
-## Near-term backlog
+## Integration matrix
 
-- Shared **`orchestrate()`** API for MCP + CLI parity.
-- Tests: MCP handshake + `resolve_orchestrator_db` behavior.
+| Host capability | Skillforge pattern |
+|-----------------|---------------------|
+| **Project workspace** | Pass **`project_root`** so state and optional RAG live under **`.skillforge/`**. |
+| **Strict context budgets** | Tune **`SKILLFORGE_CONTEXT_*`**, **`SKILLFORGE_ROUTE_MAX_CHARS`**, and project RAG caps. |
+| **No LLM keys on router** | Use **`SKILLFORGE_ROUTER_MODE=embedding`** (or omit **`ANTHROPIC_API_KEY`** in auto mode). |
+| **Human-in-the-loop picks** | **`SKILLFORGE_ROUTER_MODE=host`**: shortlist first, then **`picked_names`**. |
+| **Org policy** | Central **`SKILLFORGE_ROUTE_POLICIES`** / file + per-repo **`policies.json`** with overlay keys. |
 
-## Non-goals (v1)
+## Near-term themes
 
-- Auto-scanning every AI agent installed on the host.
-- VS Code extension (unless added as a separate track).
+- Continue tightening **parity** between MCP tools and CLI (**routing**, **meta**, **policies**).
+- Expand **fixture library** for **`route-eval`** as catalog and hybrid modes evolve.
+- Optional: richer **enterprise** packaging (e.g. signed SBOM, pinned base images) as downstream consumers require—tracked outside this doc when scope is agreed.
+
+## Non-goals
+
+- Replacing the host model or owning end-user billing for conversations.
+- Implicit network egress except **optional** model APIs (Anthropic) and **pack** git clones invoked explicitly by operators.
+- A hosted multi-tenant SaaS as part of the core OSS package.
+
+## References
+
+- **Operator documentation:** [README.md](README.md)  
+- **Security posture:** [SECURITY.md](SECURITY.md)  
+- **Shipping checklist:** [RELEASING.md](RELEASING.md)

package/bin/cli.js

@@ -8,7 +8,12 @@
  *   skillforge events [--watch] [--limit=N]     Print SQLite routing events
  *   skillforge route [words…] [--prompt=…]     Same routing as MCP route_skills (terminal)
  *   skillforge index --project-root=…            Chunk/embed repo files for project RAG
- *   skillforge install               One-time Python venv + deps
+ *   skillforge health [--quick] [--json]         Preflight: paths, catalog, optional router load
+ *   skillforge route-eval --fixture=…            Embedding-only regression cases (CI-friendly)
+ *   skillforge weights export|import             Snapshot learned weights (JSON)
+ *   skillforge install               One-time Python venv + deps (+ Cursor /skillforge when detected)
+ *   skillforge hosts init [--force]  Install global /skillforge for Cursor + Claude Code (no Python setup)
+ *   skillforge cursor init [--force]  Same as hosts init (alias)
  *   skillforge skills … / pack … / reset
  */
 
@@ -19,7 +24,9 @@ const os = require('os');
 const packs = require('../lib/packs');
 
 const PKG_ROOT = path.resolve(__dirname, '..');
-const NPM_PKG_NAME = require(path.join(PKG_ROOT, 'package.json')).name;
+const PKG = require(path.join(PKG_ROOT, 'package.json'));
+const NPM_PKG_NAME = PKG.name;
+const PKG_VERSION = PKG.version || '0.0.0';
 const CONFIG_DIR = path.join(os.homedir(), '.skillforge');
 const VENV_DIR = path.join(CONFIG_DIR, 'venv');
 const DATA_DIR = path.join(CONFIG_DIR, 'data');
@@ -143,7 +150,23 @@ function runSetup() {
   }
   ok('Python dependencies installed');
 
-  // 4. Mark setup complete
+  // 4. Cursor / MCP host hooks (no Python required)
+  try {
+    const hostSetup = require('../lib/host-setup');
+    hostSetup.reportHostsAndInstallAgentCommands({
+      force: process.argv.includes('--force-cursor'),
+      pkgRoot: PKG_ROOT,
+      pkgVersion: PKG_VERSION,
+      log,
+      ok,
+      err,
+      dim: s => log(c.dim(s)),
+    });
+  } catch (e) {
+    err(`Host integration failed: ${/** @type {Error} */ (e).message}`);
+  }
+
+  // 5. Mark setup complete
   fs.writeFileSync(SETUP_MARKER, new Date().toISOString());
   ok('Setup complete\n');
 }
@@ -237,6 +260,41 @@ function runIndexCmd() {
   proc.on('exit', (code) => process.exit(code ?? 0));
 }
 
+function runHealthCmd() {
+  setupIfNeeded();
+  const sub = args.slice(1);
+  const proc = spawn(venvPython(), ['-m', 'app.health_cli', ...sub], {
+    stdio: 'inherit',
+    env: buildEnv(),
+  });
+  proc.on('exit', (code) => process.exit(code ?? 0));
+}
+
+function runRouteEvalCmd() {
+  setupIfNeeded();
+  const sub = args.slice(1);
+  const proc = spawn(venvPython(), ['-m', 'app.eval_cli', ...sub], {
+    stdio: 'inherit',
+    env: buildEnv(),
+  });
+  proc.on('exit', (code) => process.exit(code ?? 0));
+}
+
+function runWeightsCmd() {
+  setupIfNeeded();
+  const sub = args.slice(1);
+  if (sub.length === 0 || sub[0] === '--help' || sub[0] === '-h') {
+    log(c.dim('Usage: skillforge weights export [-o file] [--user-id=] [--project-root=]'));
+    log(c.dim('       skillforge weights import <file.json> [--user-id=] [--replace-user] [--project-root=]'));
+    process.exit(sub.length === 0 ? 1 : 0);
+  }
+  const proc = spawn(venvPython(), ['-m', 'app.weights_cli', ...sub], {
+    stdio: 'inherit',
+    env: buildEnv(),
+  });
+  proc.on('exit', (code) => process.exit(code ?? 0));
+}
+
 // ---- skill management ----
 function skillsAdd(srcPath) {
   if (!srcPath) {
@@ -306,6 +364,19 @@ function reset() {
   }
 }
 
+function runHostsInit() {
+  const hostSetup = require('../lib/host-setup');
+  hostSetup.reportHostsAndInstallAgentCommands({
+    force: args.includes('--force') || args.includes('--force-cursor'),
+    pkgRoot: PKG_ROOT,
+    pkgVersion: PKG_VERSION,
+    log,
+    ok,
+    err,
+    dim: s => log(c.dim(s)),
+  });
+}
+
 function showHelp() {
   log(`
 ${c.bold('skillforge')} — skill orchestrator co-tool for Claude (MCP-first)
@@ -317,6 +388,8 @@ ${c.bold('Run modes:')}
   skillforge events [--watch] [--limit=N] [--verbose] [--user=…]   Live routing log + usage (see --help)
   skillforge route [words…] [--project-root=…] [--include-project-rag]   Route a prompt (see skillforge route --help)
   skillforge index --project-root=… [--reset] [--stats-only]   Index repo text for include_project_rag
+  skillforge health [--quick] [--json] [--project-root=…]   Paths + SKILL.md counts; omit --quick to load the embedder
+  skillforge route-eval --fixture=path/to/cases.json [--router-mode=embedding]   Run routing eval cases
 
 ${c.bold('Skills:')}
   skillforge skills list           List bundled and user skills
@@ -331,10 +404,15 @@ ${c.bold('Skill packs (install from git):')}
 
 ${c.bold('Maintenance:')}
   skillforge reset                 Wipe learned state and event log
-  skillforge install               Re-run setup (auto-runs on first launch)
+  skillforge weights export        Dump learned weights JSON (see skillforge weights export --help)
+  skillforge weights import        Restore weights snapshot (see skillforge weights import --help)
+  skillforge install               Re-run setup (auto-runs on first launch; installs editor /skillforge when detected)
+  skillforge install --force-cursor  Replace managed host command files even if present
+  skillforge hosts init [--force]    Write ~/.cursor/commands + ~/.claude/commands /skillforge (no Python)
+  skillforge cursor init [--force]   Alias for hosts init
   skillforge --help                This message
 
-${c.bold('First run:')} ${c.cyan('skillforge install')} (auto on first command). Primary use: add MCP config (below). ${c.cyan('skillforge mcp')} needs no API key for embedding-only routing.
+${c.bold('First run:')} ${c.cyan('skillforge install')} (auto on first command) or ${c.cyan('npx -y')} ${NPM_PKG_NAME} ${c.cyan('install')}. Detects ${c.cyan('Cursor')} and ${c.cyan('Claude Code')} and installs managed **/skillforge** under ${c.cyan('~/.cursor/commands')} and ${c.cyan('~/.claude/commands')} (skip: ${c.dim('SKILLFORGE_SKIP_CURSOR_SETUP')}, ${c.dim('SKILLFORGE_SKIP_CLAUDE_CODE_SETUP')}; force paths: ${c.dim('SKILLFORGE_CURSOR_GLOBAL_COMMAND')} / ${c.dim('SKILLFORGE_CLAUDE_CODE_GLOBAL_COMMAND')}). ${c.cyan('skillforge mcp')} needs no API key for embedding-only routing.
 ${c.bold('Config dir:')} ${CONFIG_DIR}
 
 ${c.bold('MCP integration:')}
@@ -366,6 +444,15 @@ async function main() {
     case 'index':
       runIndexCmd();
       break;
+    case 'health':
+      runHealthCmd();
+      break;
+    case 'route-eval':
+      runRouteEvalCmd();
+      break;
+    case 'weights':
+      runWeightsCmd();
+      break;
     case 'mcp':
       if (args[1] === 'config') {
         printMcpConfig();
@@ -376,6 +463,26 @@ async function main() {
     case 'install':
       runSetup();
       break;
+    case 'hosts': {
+      const sub = args[1];
+      if (sub === 'init') {
+        runHostsInit();
+        break;
+      }
+      err('Unknown hosts subcommand.');
+      log(c.dim('  Try: skillforge hosts init [--force]'));
+      process.exit(1);
+    }
+    case 'cursor': {
+      const sub = args[1];
+      if (sub === 'init') {
+        runHostsInit();
+        break;
+      }
+      err('Unknown cursor subcommand.');
+      log(c.dim('  Try: skillforge cursor init [--force] (alias: hosts init)'));
+      process.exit(1);
+    }
     case 'reset':
       reset();
       break;

package/ci/bundle-gate.json

@@ -0,0 +1,4 @@
+{
+  "description": "CI gate for the vendored skills tree: at least this many SKILL.md files must be present under skills/. Bump minSkillMdFiles only when intentionally changing bundle size policy.",
+  "minSkillMdFiles": 200
+}