@heytherevibin/skillforge 0.10.0 → 0.11.7

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (58) hide show
  1. package/CHANGELOG.md +53 -0
  2. package/CONTRIBUTING.md +5 -3
  3. package/README.md +37 -345
  4. package/RELEASING.md +8 -7
  5. package/STRATEGY.md +2 -2
  6. package/bin/cli.js +297 -52
  7. package/ci/test-user-env-profile.cjs +65 -0
  8. package/docs/README.md +14 -0
  9. package/docs/architecture-and-data.md +90 -0
  10. package/docs/cli-reference.md +57 -0
  11. package/docs/environment-and-configuration.md +76 -0
  12. package/docs/getting-started.md +88 -0
  13. package/docs/mcp-integration.md +75 -0
  14. package/docs/troubleshooting.md +50 -0
  15. package/lib/templates/claude-code-skillforge-global.md +3 -3
  16. package/lib/templates/cursor-skillforge-global.md +6 -2
  17. package/lib/user-env-profile.js +141 -0
  18. package/package.json +3 -2
  19. package/python/app/agent_cli.py +334 -0
  20. package/python/app/explain_route.py +170 -0
  21. package/python/app/health_cli.py +13 -0
  22. package/python/app/main.py +131 -48
  23. package/python/app/materialize.py +150 -68
  24. package/python/app/mcp_contract.py +2 -1
  25. package/python/app/mcp_operator.py +252 -0
  26. package/python/app/mcp_server.py +290 -118
  27. package/python/app/npm_pkg_version.py +38 -0
  28. package/python/app/pick_diversify.py +51 -0
  29. package/python/app/replay_cli.py +145 -0
  30. package/python/app/route_cli.py +251 -87
  31. package/python/app/route_cli_pick.py +35 -0
  32. package/python/app/route_policies.py +18 -3
  33. package/python/app/route_quality.py +70 -1
  34. package/python/app/router_llm.py +85 -0
  35. package/python/app/router_mode.py +21 -0
  36. package/python/app/routing_signals.py +7 -1
  37. package/python/app/skill_manifest.py +67 -0
  38. package/python/app/skills_author_cli.py +117 -0
  39. package/python/app/tips_cli.py +37 -0
  40. package/python/app/tools_cli.py +276 -0
  41. package/python/fixtures/route_eval/smoke.json +5 -0
  42. package/python/requirements.txt +1 -0
  43. package/python/tests/test_capabilities_bundle.py +33 -0
  44. package/python/tests/test_materialize_hosts.py +108 -0
  45. package/python/tests/test_mcp_contract.py +1 -1
  46. package/python/tests/test_mcp_initialize_clientinfo.py +26 -0
  47. package/python/tests/test_mcp_operator.py +84 -0
  48. package/python/tests/test_npm_pkg_version.py +21 -0
  49. package/python/tests/test_pick_diversify.py +47 -0
  50. package/python/tests/test_replay_cli.py +31 -0
  51. package/python/tests/test_route_cli_pick.py +25 -0
  52. package/python/tests/test_route_policies.py +29 -0
  53. package/python/tests/test_route_quality.py +72 -0
  54. package/python/tests/test_router_llm.py +63 -0
  55. package/python/tests/test_router_mode_env.py +21 -0
  56. package/python/tests/test_routing_signals.py +20 -0
  57. package/python/tests/test_skill_manifest.py +48 -0
  58. package/python/tests/test_tools_cli.py +69 -0
package/CHANGELOG.md CHANGED
@@ -1,5 +1,58 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.11.7
4
+
5
+ - **Python router:** Restore **`Router.__init__`** so skill embeddings, **`_by_name`**, hybrid/BM25, and chunk indexing initialize correctly (fixes **`AttributeError: 'Router' object has no attribute '_by_name'`** in **`route-eval`** / **`run_route_turn`** when router setup was mistakenly unreachable behind the **`anthropic`** accessor).
6
+ - **Docs:** **`README`**, **`docs/`** guides, and **`RELEASING`** prose align release line **0.11.7** with **`package.json`**; MCP **`serverInfo.version`** sourcing is documented as **`python/app/npm_pkg_version.py`** (**`published_package_version()`**), not a duplicate field in **`mcp_server.py`**.
7
+
8
+ ## 0.11.6
9
+
10
+ - **Documentation hub:** Added [`docs/`](docs/) with instructional guides (**[docs/README.md](docs/README.md)** lists them).
11
+ - **README refreshed:** Repo root **`README.md`** is a compact hub (**npm**, **GitHub release**, **`package.json` on main**, CI, licence badges; links into **`docs/`**).
12
+ - **npm package manifest:** **`docs/`** is included under **`files`** so published tarballs bundle the guides.
13
+ - **Hygiene:** Added repository **`.gitignore`** covering **`__pycache__`**, **`*.pyc`**, **`.pytest_cache/`**.
14
+
15
+ ## 0.11.5
16
+
17
+ - **`skillforge config validate`:** Lint **`~/.skillforge/env`** (errors exit **1**, missing profile exits **0**). Parser extracted to **`lib/user-env-profile.js`** (shared semantics with merge). **`npm test`** runs **`node --test ci/test-user-env-profile.cjs`**, **`skillforge config validate`**, plus **`skillforge --help`**.
18
+ - **`skillforge mcp config --with-env`:** MCP snippet includes **`entry.env`** with **`SKILLFORGE_ROUTER_MODE=host`** (non-secret scaffold). **`--with-anthropic`** still replaces the **`env`** object entirely when both are passed.
19
+ - **`SKILLFORGE_ROUTE_POLICIES` / file policies:** **`stderr`** warning when JSON is invalid (policies ignored); tests in **`python/tests/test_route_policies.py`**.
20
+ - **README:** document **`validate`**, MCP **`--with-env`**, and that **`python -m app.*`** skips Node profile loading unless you replicate **`buildEnv`** yourself.
21
+ - **CLI:** **`node bin/cli.js --help`** (and **`-h`** as the first argument) prints the same banner as **`skillforge --help`**.
22
+ - **`capabilities` MCP bundle:** **`user_env_profile`** object with **`path_command`**, **`init_command`**, **`validate_command`**, **`file`** ( **`~/.skillforge/env`** ).
23
+
24
+ ## 0.11.4
25
+
26
+ - **Operator env profile:** optional **`~/.skillforge/env`** dotenv-style file merged before **`process.env`** when spawning Python (**`skillforge config path`**, **`skillforge config init [--force]`**). Bootstrap **`SKILLFORGE_*_SKILLS`**, **`SKILLFORGE_DB_PATH`**, and **`PYTHONPATH`** still finalize last (**`bin/cli.js`**).
27
+ - **`skillforge health`** reports **`user_env_profile`** (whether **`~/.skillforge/env`** exists). **`skillforge tips`** mentions **`skillforge config`** and the README configuration section.
28
+
29
+ ## 0.11.3
30
+
31
+ - **MCP operator tools (read-only):** **`get_router_status`** — env + loaded router snapshot; **`project_index_status`** — project chunk counts / last index metadata (**`project_root`** required); **`weights_snapshot`** — same JSON shape as **`skillforge weights export`**; **`events_recent`** — recent SQLite events for **`user_id`** with optional **`event_type`** filter (**`_meta.rows`** capped at 100). Implementations in **`app/mcp_operator.py`**.
32
+
33
+ ## 0.11.2
34
+
35
+ - **Routing calibration (`route_quality`):** Bump inner schema to **`route_quality/2`**. **`shortlist`** adds **`ambiguous`**, **`confidence_tier`** (`high`/`medium`/`low`), **`routing_score_margin`**, **`second_routing_score`**, and **`cosine_leader_matches_routing_top`** (alias of **`top1_dense_and_fused_agree`**). Tunables: **`SKILLFORGE_ROUTE_AMBIGUITY_COS_MARGIN`** (default `0.012`), **`SKILLFORGE_ROUTE_AMBIGUITY_ROUTE_MARGIN`** (default `0.018`), **`SKILLFORGE_ROUTE_AMBIGUITY_DISABLE`**. **`router.pick_diversify`** records optional per-source thinning (below).
36
+ - **Pick diversify (opt-in):** When **`SKILLFORGE_PICK_DIVERSIFY=1`**, cap picks per **`source`** (**`bundled`** / **`user`**) via **`SKILLFORGE_PICK_MAX_PER_SOURCE`** (default **`2`**) **before** regex policy **`include`** merge. Applies to **`run_route_turn`** (MCP + CLI **`route`**) and **`explain_route`**.
37
+ - **MCP contract:** **`MCP_RESPONSE_SCHEMA_VERSION` 1.8** (additive **`_meta`** semantics; embedded **`route_quality`** v2).
38
+
39
+ ## 0.11.1
40
+
41
+ - **MCP `materialize_project` / `skillforge_bootstrap`:** Default **`hosts`** is **`auto`**. Resolution order when **`hosts`** is **`auto`** or omitted: **`SKILLFORGE_MATERIALIZE_HOSTS`** (**`both`**, **`cursor`**, or **`claude_code`**) if set, else MCP **`initialize`** **`clientInfo`** name/title (substring **`cursor`** → **`cursor`**, **`claude`** → **`claude_code`**), optional **`CURSOR_AGENT`** / **`CURSOR_TRACE_ID`** hints, else **`both`**. Explicit **`hosts: cursor`**, **`claude_code`**, or **`both`** on the tool always wins. Responses add **`hosts_resolution`** (**`explicit`**, **`environment`**, or **`inferred`**) plus **`hosts_requested`**, **`mcp_client_name`**, **`mcp_client_title`** in **`materialize`** **`_meta`**.
42
+
43
+ ## 0.11.0
44
+
45
+ - **Breaking (routing default):** When **`SKILLFORGE_ROUTER_MODE` is unset**, Skillforge now defaults to **`host`** (two-step **`route_skills`**: shortlist, then **`picked_names`**). Restore the previous **auto** behavior (**Haiku in-process when **`ANTHROPIC_API_KEY`** is set**, else embedding-first) with **`SKILLFORGE_ROUTER_MODE=auto`** or an empty value. Use **`embedding`** or **`full`** as before.
46
+ - **MCP config:** **`skillforge mcp config --with-anthropic`** now sets **`SKILLFORGE_ROUTER_MODE=auto`** together with the **`ANTHROPIC_API_KEY`** placeholder so the key is not ignored (default **host** mode does not call Anthropic).
47
+ - **Refactor:** **`app/router_mode.py`** — **`normalise_skillforge_router_mode`**; unit tests in **`python/tests/test_router_mode_env.py`**.
48
+ - **Docs / MCP tool text:** Describe default **host** routing and how to override (**README**, **`route_skills`** tool description).
49
+ - **Global/project `/skillforge` command:** YAML **`description`** in frontmatter (**`cursor-skillforge-global.md`**, **`claude-code-skillforge-global.md`**, **`materialize_project`** Cursor command); **`<!-- skillforge-managed … -->`** moved to EOF so Composer no longer uses the HTML marker as tooltip text.
50
+ - **`materialize_project`:** Optional **`hosts`**: **`cursor`**, **`claude_code`**, or **`both`** (default) — scaffold only the IDE folders you ask for instead of always writing **`.cursor/`** + **`.claude/`**.
51
+
52
+ ## 0.10.1
53
+
54
+ - **README:** Clarify that **npm** **`latest`** and **`npm view`** are authoritative for semver; note CDN/browser cache can make the shields **npm** badge lag briefly after a publish. **Badge:** add **`cacheSeconds`** so the image URL refreshes sooner.
55
+
3
56
  ## 0.10.0
4
57
 
5
58
  - **CI:** Minimum bundled **`SKILL.md`** count is configured via **`ci/bundle-gate.json`** (`minSkillMdFiles`); **`.github/workflows/ci.yml`** reads that file. The **`ci/`** directory is included in the published package **`files`** list for transparency. Health + **route-eval** step chains both commands under a single **`cd python`** so the second command does not run from **`python/python`**.
package/CONTRIBUTING.md CHANGED
@@ -6,10 +6,12 @@ Thank you for improving Skillforge.
6
6
 
7
7
  1. **Fork** the repository and create a branch from **`main`**.
8
8
  2. Keep changes **focused**; follow patterns in surrounding code.
9
- 3. Run **local checks** before opening a PR from the **package root** (the directory that contains **`package.json`**):
9
+ 3. Operator / user-facing doc changes belong in **`docs/`** (`README.md` should stay the short hub + badges). Bump **`package.json`** / **`CHANGELOG.md`** when shipping user-visible doc releases.
10
+
11
+ 4. Run **local checks** before opening a PR from the **package root** (the directory that contains **`package.json`**):
10
12
 
11
13
  ```bash
12
- node --check bin/cli.js && node --check lib/packs.js
14
+ node --check bin/cli.js && node --check lib/packs.js && node --check lib/user-env-profile.js
13
15
  npm test
14
16
  ```
15
17
 
@@ -23,7 +25,7 @@ Thank you for improving Skillforge.
23
25
 
24
26
  If you **intentionally** shrink or grow the bundled **`skills/`** tree below/above the current CI minimum, update **`ci/bundle-gate.json`** (`minSkillMdFiles`) and note it in the PR—see **[RELEASING.md](RELEASING.md)**.
25
27
 
26
- 4. Open a **pull request** into **`main`** with:
28
+ 5. Open a **pull request** into **`main`** with:
27
29
  - What changed and **why**
28
30
  - How you **verified** it (tests, manual MCP smoke, etc.)
29
31
 
package/README.md CHANGED
@@ -1,378 +1,70 @@
1
1
  # Skillforge
2
2
 
3
3
  <p align="left">
4
- <a href="https://www.npmjs.com/package/@heytherevibin/skillforge"><img src="https://img.shields.io/npm/v/@heytherevibin/skillforge?label=npm&color=blue" alt="npm version" /></a>
4
+ <a href="https://www.npmjs.com/package/@heytherevibin/skillforge"><img src="https://img.shields.io/npm/v/@heytherevibin/skillforge.svg?label=npm&logo=npm&logoColor=white&color=blue" alt="npm registry version" /></a>
5
+ <a href="https://github.com/heytherevibin/skillforge/releases/latest"><img src="https://img.shields.io/github/v/release/heytherevibin/skillforge?sort=semver&label=github%20release&logo=github&color=purple" alt="Latest GitHub release tag" /></a>
6
+ <a href="https://github.com/heytherevibin/skillforge/blob/main/package.json"><img src="https://img.shields.io/github/package-json/v/heytherevibin/skillforge?label=package.json%20%28main%29&logo=github" alt="package.json semver on GitHub default branch" /></a>
5
7
  <a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-green.svg" alt="License: MIT" /></a>
6
- <a href="https://github.com/heytherevibin/skillforge/actions/workflows/ci.yml"><img src="https://github.com/heytherevibin/skillforge/actions/workflows/ci.yml/badge.svg" alt="CI" /></a>
8
+ <a href="https://github.com/heytherevibin/skillforge/actions/workflows/ci.yml"><img src="https://github.com/heytherevibin/skillforge/actions/workflows/ci.yml/badge.svg" alt="GitHub Actions CI status" /></a>
7
9
  </p>
8
10
 
9
- **Skillforge** is a **local-first orchestration layer** for agent workflows: it maintains a catalog of **`SKILL.md`** documents, **routes** a small subset per task using **embedding-first retrieval**, optional **hybrid** sparse signals and **LLM** stages, optional **project-scoped** policies and notes, and returns **structured context** for downstream models. The **primary integration** is **stdio MCP**; the **CLI** provides parity, operations, and automation hooks.
11
+ **Skillforge** is a **local-first** SKILL.md orchestration layer: embeddings pick a **small routed set** per task (optional hybrid + LLM stages), SQLite stores **sessions, learned weights, and events**, optional **project RAG** augments prompts, and the **production surface** is **stdio MCP**. A **Node** CLI (**`skillforge`**) bootstraps a managed **Python venv** under **`~/.skillforge/venv`**, merges **`~/.skillforge/env`**, mirrors MCP behaviours in **`skillforge route`**, **`skillforge tools`**, **`skillforge agent`**, and exposes operator CLIs (**`health`**, **`events`**, **`weights`**).
10
12
 
11
- **Published version:** see **`package.json`** and the npm badge above (they should match after each release). **Change history:** [CHANGELOG.md](CHANGELOG.md). **Product direction:** [STRATEGY.md](STRATEGY.md). **Vulnerability reporting:** [SECURITY.md](SECURITY.md). **Release process:** [RELEASING.md](RELEASING.md).
13
+ **Semantic versions** should align across **`package.json`**, git tags (**`vX.Y.Z`**), MCP **`initialize.serverInfo.version`**, **npm tarball**, and the **GitHub Release** artifact—see [`RELEASING.md`](RELEASING.md). **Published line on `main`:** **`0.11.7`** (same value in **[`package.json` `version`](package.json#L3)** and **[`CHANGELOG`](CHANGELOG.md)** top section). The **`package.json`** shield tracks **`main`**; **`npm`** / **`release`** shields track **published** artefacts and may briefly lag immediately after tagging.
12
14
 
13
15
  ---
14
16
 
15
- ## Table of contents
17
+ ## Documentation (start here)
16
18
 
17
- - [What Skillforge provides](#what-skillforge-provides)
18
- - [Architecture at a glance](#architecture-at-a-glance)
19
- - [Requirements](#requirements)
20
- - [Quick start](#quick-start)
21
- - [Installation](#installation)
22
- - [Operational interfaces](#operational-interfaces)
23
- - [Model Context Protocol (MCP)](#model-context-protocol-mcp)
24
- - [MCP response contract](#mcp-response-contract)
25
- - [Routing pipeline](#routing-pipeline)
26
- - [Route policies and project overlay](#route-policies-and-project-overlay)
27
- - [Project RAG](#project-rag)
28
- - [Learning, weights, and portability](#learning-weights-and-portability)
29
- - [Skills and packs](#skills-and-packs)
30
- - [Configuration](#configuration)
31
- - [Local data and paths](#local-data-and-paths)
32
- - [Security](#security)
33
- - [Contributing and governance](#contributing-and-governance)
34
- - [License](#license)
19
+ | Guide | Audience |
20
+ |-------|-----------|
21
+ | [docs/README.md — index](docs/README.md) | Choose your path |
22
+ | [Getting started](docs/getting-started.md) | Install MCP + sanity checks |
23
+ | [Environment & configuration](docs/environment-and-configuration.md) | `~/.skillforge/env`, MCP host `entry.env`, and the full SKILLFORGE variable matrix |
24
+ | [MCP integration](docs/mcp-integration.md) | Router modes (**`host`**, **`auto`**, …), tools, **`_meta`** |
25
+ | [CLI reference](docs/cli-reference.md) | Subcommands (**`route`**, **`tools`**, **`agent`**, …) |
26
+ | [Architecture & data](docs/architecture-and-data.md) | Pipeline, SQLite, policies, indexing |
27
+ | [Troubleshooting](docs/troubleshooting.md) | Tools missing, npm **`EOTP`**, bad policy JSON |
35
28
 
36
- ---
37
-
38
- ## What Skillforge provides
39
-
40
- | Area | Capability |
41
- |------|------------|
42
- | **Context control** | Returns only **relevant** skill (and optional project) chunks instead of an entire catalog. |
43
- | **Routing** | Dense embeddings on skill **cards** (title, description, optional triggers); optional **keyword / BM25** fusion; optional **LLM** rerank and final pick—or **embedding-only** or **host-delegated** selection. |
44
- | **Conversation-aware retrieval** | Recent turns can influence the **shortlist query** when enabled via environment (see [Configuration](#configuration)). |
45
- | **Governance** | Regex **policies** to append skills after routing; **project overlay** for excludes, score boosts, and **project notes** (notes require a declared **project root**). |
46
- | **Adaptation** | Per-user **SQLite** statistics and explicit feedback adjust routing over time (portable via **export/import**). |
47
- | **Project grounding** | Optional **index** of repository text into the same SQLite DB used for sessions (**project RAG**). |
48
- | **Observability** | Versioned **`_meta`** on MCP responses; route **events** in SQLite; **`skillforge events`** for operators. |
49
- | **Reliability hooks** | **`skillforge health`** (preflight) and **`skillforge route-eval`** (fixture-driven smoke checks; used in CI). |
50
-
51
- ---
52
-
53
- ## Architecture at a glance
54
-
55
- ```
56
- Host (Claude / Cursor / Claude Code / …)
57
- │ MCP JSON-RPC (stdio)
58
-
59
- ┌───────────────────────────────────────────────────────────┐
60
- │ skillforge mcp → Python: embed → shortlist → optional │
61
- │ LLM stages → policies/overlay → context assembly │
62
- └───────────────────────────────────────────────────────────┘
63
-
64
- ├── SQLite (global ~/.skillforge or <project>/.skillforge)
65
- └── Optional: Anthropic API (Haiku) in-process when enabled
66
- ```
67
-
68
- **Trust boundary:** Skillforge runs **on the operator’s machine** (or your CI runner). Prompts and retrieved text should be handled per your org’s data policy. See [Security](#security).
69
-
70
- ---
71
-
72
- ## Requirements
73
-
74
- | Dependency | Notes |
75
- |------------|--------|
76
- | **Node.js** | **≥ 18** (CLI bootstrap). CI validates on **Node 22** (see `.github/workflows/ci.yml`). |
77
- | **Python** | **≥ 3.10**; the CLI creates **`~/.skillforge/venv`** and installs `python/requirements.txt`. |
78
- | **Anthropic API** | **Optional.** Without **`ANTHROPIC_API_KEY`**, routing stays **embedding-first** unless you delegate picks to the host. |
79
-
80
- **First run** installs the virtualenv and Python dependencies and may download the default sentence-transformer model once; subsequent starts are typically fast.
81
-
82
- ---
83
-
84
- ## Quick start
85
-
86
- ```bash
87
- npx --yes @heytherevibin/skillforge --help
88
- ```
89
-
90
- Configure **MCP** in your host (see [Model Context Protocol](#model-context-protocol-mcp)). **Embedding-only** operation does not require an Anthropic key.
91
-
92
- **Operator visibility:**
93
-
94
- ```bash
95
- skillforge events --watch
96
- ```
97
-
98
- **Preflight (after install):**
99
-
100
- ```bash
101
- skillforge health --quick
102
- ```
29
+ **Project meta:** [`CHANGELOG.md`](CHANGELOG.md) · [`STRATEGY.md`](STRATEGY.md) · [`SECURITY.md`](SECURITY.md) · [`CONTRIBUTING.md`](CONTRIBUTING.md) · [`CODE_OF_CONDUCT.md`](CODE_OF_CONDUCT.md) · [`RELEASING.md`](RELEASING.md)
103
30
 
104
31
  ---
105
32
 
106
- ## Installation
107
-
108
- **Evaluate without global install**
33
+ ## TL;DR — try it now
109
34
 
110
35
  ```bash
111
36
  npx --yes @heytherevibin/skillforge --help
37
+ skillforge install # provisions ~/.skillforge/venv when needed
38
+ skillforge mcp config # stdout JSON snippet → paste into ~/.cursor/mcp.json, then restart IDE
39
+ skillforge tips && skillforge health --quick
40
+ skillforge config init # optional ~/.skillforge/env template · skillforge config validate
112
41
  ```
113
42
 
114
- **Global install**
115
-
116
- ```bash
117
- npm install -g @heytherevibin/skillforge
118
- skillforge --help
119
- ```
120
-
121
- - **npm:** [@heytherevibin/skillforge](https://www.npmjs.com/package/@heytherevibin/skillforge)
122
- - **Source / issues:** [github.com/heytherevibin/skillforge](https://github.com/heytherevibin/skillforge)
123
-
124
- ---
125
-
126
- ## Operational interfaces
127
-
128
- Skillforge is organized around a small **CLI** surface (implemented in **Node** spawning **Python** modules). Use **`skillforge <command> --help`** for flags.
129
-
130
- | Group | Commands | Purpose |
131
- |-------|----------|--------|
132
- | **Core** | `mcp`, `route`, `events`, `index` | Primary routing, logs, project indexing. |
133
- | **Reliability** | `health`, `route-eval` | Preflight checks; embedding-mode fixture evaluation (CI uses both). |
134
- | **Learning portability** | `weights export`, `weights import` | Snapshot / restore **`skill_weights`** rows (JSON). |
135
- | **Catalog** | `skills`, `pack` | User skills and git-backed **packs**. |
136
- | **Setup** | `install`, `hosts init`, `reset` | Bootstrap venv, global `/skillforge` commands, wipe local DB state. |
137
-
138
- **MCP config snippet (stdout):**
139
-
140
- ```bash
141
- skillforge mcp config
142
- # Optional: --local (checkout), --with-anthropic (env placeholder)
143
- ```
144
-
145
- **Important:** MCP requires a **clean stdout** stream (JSON-RPC). Logs belong on **stderr**. If tools do not appear in the host, update the package and fully restart the host after setup.
146
-
147
- ---
148
-
149
- ## Model Context Protocol (MCP)
150
-
151
- ### Router modes (`SKILLFORGE_ROUTER_MODE`)
152
-
153
- | Mode / default | Anthropic key | Behavior |
154
- |----------------|---------------|----------|
155
- | *(unset)* **auto** | optional | Embedding-first when key absent; full LLM routing when key present. |
156
- | `embedding` | ignored for routing | No in-process LLM pick; top candidates drive selection. |
157
- | `full` | recommended | LLM final pick; falls back per implementation on errors. |
158
- | `host` | optional for pick | **Two-step:** first `route_skills` returns shortlist; second call passes **`picked_names`**. |
159
-
160
- ### MCP tools (summary)
161
-
162
- | Tool | Role |
163
- |------|------|
164
- | `route_skills` | Main routing: **prompt**, optional **conversation**, **`project_root`**, **`include_project_rag`**, **`session_id`**, **`user_id`**, **`picked_names`** (host or override). |
165
- | `search_skills` | Embedding shortlist for a **query** (read-only). |
166
- | `explain_route` | Diagnostics: shortlist + picks + policy audit **without** writing sessions. |
167
- | `get_skill`, `list_skills` | Catalog access. |
168
- | `skill_feedback`, `skill_referenced`, `disable_skill` | Learning loop and toggles. |
169
- | `materialize_project`, `skillforge_bootstrap` | Project file materialization (bootstrap **errors** in `host` mode by design—use two-step routing + materialize). |
170
-
171
- Full argument lists: tool definitions in **`python/app/mcp_server.py`** (source of truth).
172
-
173
- ---
174
-
175
- ## MCP response contract
176
-
177
- Successful **`route_skills`** responses include **`_meta`** built in **`python/app/mcp_contract.py`**. The **`schema_version`** string tracks additive JSON shape changes (hosts may rely on it for parsing).
178
-
179
- **Authoritative version:** constant **`MCP_RESPONSE_SCHEMA_VERSION`** in **`app/mcp_contract.py`** (do not rely on this README if the two drift).
180
-
181
- **Notable `_meta` fields (non-exhaustive):**
182
-
183
- | Field | Description |
184
- |-------|-------------|
185
- | `schema_version` | Contract version string. |
186
- | `sources`, `budget` | Chunk citations and size accounting. |
187
- | `fusion` | Present when MMR-style fusion ran. |
188
- | `context_redaction` | Redaction hit counts when enabled. |
189
- | `route_quality` | Shortlist / router / policy / session telemetry for calibration. |
190
- | `feedback_effect` | Per-pick **learned weight** snapshot (uses / thumbs / reference rate). |
191
- | `routing_overlay` | Audit of **exclude** / **boost** / **project notes** application when configured. |
192
- | `host_pick_shortlist`, `host_pick_candidates` | Host-pick phase payloads. |
193
-
194
- Structured errors (e.g. empty prompt) return **`isError`: true** with **`_meta.error`** and **`schema_version`**.
195
-
196
- ---
197
-
198
- ## Routing pipeline
199
-
200
- ```
201
- User prompt (+ optional conversation-aware routing query)
202
- → Encode routing query (skill cards + optional hybrid sparse signal)
203
- → Fuse scores + per-user weights + optional project overlay boosts
204
- → Shortlist (top-K)
205
- → Optional LLM rerank / final pick (or embedding / host selection)
206
- → Assemble context (skill chunks ± project chunks, optional fusion)
207
- → Return markdown + _meta; optional SQLite events
208
- ```
209
-
210
- Re-routing when the active skill set changes significantly is controlled by **`SKILLFORGE_REROUTE_THRESHOLD`** (see [Configuration](#configuration)).
211
-
212
- ---
213
-
214
- ## Route policies and project overlay
215
-
216
- ### Regex policies (post-pick merge)
217
-
218
- Rules match the user **`prompt`** with **`re.search`** (**`re.DOTALL`**). Matched **`include`** skills append after the router, capped by **`SKILLFORGE_MAX_ACTIVE`**. Audit lands on route **events** under **`policy`**.
219
-
220
- **Load order:** **`SKILLFORGE_ROUTE_POLICIES`** (inline JSON) → **`SKILLFORGE_ROUTE_POLICIES_FILE`** → **`<project_root>/.skillforge/policies.json`** → **`<project_root>/skillforge-policies.json`**.
221
-
222
- ### Project routing overlay (same JSON document)
223
-
224
- Optional keys alongside **`rules`**:
225
-
226
- | Key | Aliases | Purpose |
227
- |-----|---------|--------|
228
- | `exclude_skills` | `host_exclude`, `denylist` | Remove skills from the embedding shortlist. |
229
- | `routing_boosts` | `skill_boosts` | Additive score delta after learned weight (clamped; see **route_policies** module). |
230
- | `project_notes` | `routing_notes`, `rag_notes` | Free text **prepended** to the internal routing query when **`project_root`** is set (not applied without a project root—mitigates accidental global injection from shared policy files). |
231
-
232
- **Example fragment** (illustrative—adjust skill ids to your catalog):
233
-
234
- ```json
235
- {
236
- "rules": [
237
- {
238
- "if_text_matches": "(?i)(auth|oauth|jwt)",
239
- "include": ["security-review"]
240
- }
241
- ],
242
- "project_notes": "Service stack and conventions for this repo (short, factual).",
243
- "routing_boosts": { "python-testing": 0.15 },
244
- "exclude_skills": ["legacy-skill-id"]
245
- }
246
- ```
247
-
248
- ---
249
-
250
- ## Project RAG
251
-
252
- 1. **Index** repository text into **`<project>/.skillforge/orchestrator.db`**:
253
-
254
- ```bash
255
- skillforge index --project-root=/path/to/repo
256
- ```
257
-
258
- 2. Call **`route_skills`** with **`project_root`** and **`include_project_rag`** (or CLI **`--include-project-rag`**) when embeddings and schema match (see **`project_index.py`** for model/dimension guards).
259
-
260
- Chunk caps and ignore rules are **environment-driven** (see configuration table).
261
-
262
- ---
263
-
264
- ## Learning, weights, and portability
265
-
266
- - **Signals:** route **`uses`**, **`skill_referenced`**, **`skill_feedback`** (thumbs), and **`disable_skill`** feed **SQLite** **`skill_weights`**.
267
- - **Transparency:** **`_meta.feedback_effect`** summarizes per-pick weight context after the route’s **`uses`** increment.
268
- - **Portability:**
269
-
270
- ```bash
271
- skillforge weights export -o weights.json
272
- skillforge weights import weights.json
273
- ```
274
-
275
- Use **`--project-root`** / **`--user-id`** / **`--replace-user`** as documented in **`skillforge weights --help`** (implemented in **`python/app/weights_cli.py`**).
276
-
277
- ---
278
-
279
- ## Skills and packs
280
-
281
- **Bundled catalog** ships inside the npm package. **CI** enforces a **minimum** bundled **`SKILL.md`** count so releases cannot silently ship an empty tree—the threshold lives in **`ci/bundle-gate.json`** (`minSkillMdFiles`); **`.github/workflows/ci.yml`** reads that file at build time.
282
-
283
- **Custom skills:** directory with **`SKILL.md`** and YAML frontmatter (`name`, `description`; optional **`triggers`** / **`anti_triggers`**).
284
-
285
- ```bash
286
- skillforge skills add ./path/to/skill
287
- ```
288
-
289
- **Packs:** repositories with **`skillforge.json`**:
290
-
291
- ```bash
292
- skillforge pack install <org/repo>
293
- skillforge pack list
294
- ```
295
-
296
- ---
297
-
298
- ## Configuration
299
-
300
- Environment variables tune routing, context budgets, redaction, MCP defaults, and file watchers. **Authoritative defaults and parsing** live in **`python/app/main.py`** and related modules—treat the table below as **operator reference**, not a legal spec.
301
-
302
- | Variable | Role |
303
- |----------|------|
304
- | `ANTHROPIC_API_KEY` | Enables in-process **Haiku** routing / rerank when configured. |
305
- | `SKILLFORGE_ROUTER_MODE` | `full` · `embedding` · `host` · auto. |
306
- | `SKILLFORGE_EMBED_MODEL`, `SKILLFORGE_ROUTER_MODEL` | Model identifiers for embeddings / routing LLM. |
307
- | `SKILLFORGE_TOP_K`, `SKILLFORGE_MAX_ACTIVE` | Shortlist size and max simultaneous skills. |
308
- | `SKILLFORGE_REROUTE_THRESHOLD` | Re-route sensitivity (Jaccard distance). |
309
- | `SKILLFORGE_ROUTER_CONV_MAX_TURNS`, `SKILLFORGE_ROUTER_CONV_MSG_CHARS` | Conversation-aware routing query. |
310
- | `SKILLFORGE_ROUTER_HYBRID`, `SKILLFORGE_ROUTER_HYBRID_ALPHA` | Hybrid sparse/dense fusion. |
311
- | `SKILLFORGE_HAIKU_RERANK`, `SKILLFORGE_HAIKU_RERANK_MAX`, `SKILLFORGE_HAIKU_RERANK_MODEL` | Optional rerank stage. |
312
- | `SKILLFORGE_CONTEXT_MODE`, `SKILLFORGE_ROUTE_MAX_CHARS`, chunk envs | Skill body chunking vs full-body legacy. |
313
- | `SKILLFORGE_CONTEXT_FUSION`, `SKILLFORGE_CONTEXT_BUDGET_CHARS`, `SKILLFORGE_CONTEXT_MMR_LAMBDA`, pool sizes | Skill + project **MMR** fusion. |
314
- | `SKILLFORGE_PROJECT_RAG_MAX_CHARS`, `SKILLFORGE_PROJECT_RAG_MAX_CHUNKS` | Project chunk retrieval caps. |
315
- | `SKILLFORGE_PROJECT_NOTES_MAX_CHARS` | Cap for **`project_notes`** prepended to routing query. |
316
- | `SKILLFORGE_REDACT_CONTEXT`, `SKILLFORGE_REDACT_HOME_IN_PATHS` | Output redaction behavior. |
317
- | `SKILLFORGE_MCP_USER_ID`, `SKILLFORGE_PROJECT_ROOT` | MCP defaults for user scoping and DB resolution. |
318
- | `SKILLFORGE_ROUTE_POLICIES`, `SKILLFORGE_ROUTE_POLICIES_FILE` | Inline or file-backed policy JSON. |
319
- | `SKILLFORGE_HOST_PICK_MAX`, `SKILLFORGE_HOST_PICK_LINE_CHARS` | Host-mode shortlist sizing / formatting. |
320
- | Hot reload | `SKILLFORGE_SKILL_HOT_RELOAD`, `SKILLFORGE_WATCH_SKILLS_INTERVAL`, `SKILLFORGE_MCP_LIST_CHANGED`. |
321
- | Install hooks | `SKILLFORGE_SKIP_*`, `SKILLFORGE_*_GLOBAL_COMMAND`, etc. |
43
+ **Default MCP routing (**`SKILLFORGE_ROUTER_MODE` unset ⇒ **`host`**) is two-step:** first **`route_skills`** shortlist · second finalize with **`picked_names`**.
322
44
 
323
45
  ---
324
46
 
325
- ## Local data and paths
47
+ ## What ships in this repository
326
48
 
327
- **Per-project** (when **`project_root`** / **`SKILLFORGE_PROJECT_ROOT`** is used):
49
+ | Path | Purpose |
50
+ |------|---------|
51
+ | `bin/cli.js` | Node entry (`skillforge`); merges env (**`buildEnv`**) and spawns Python |
52
+ | `lib/` | Host setup, **`user-env-profile`** parser (`config validate`) |
53
+ | `python/app/` | Router, MCP server, SQLite, CLIs, contracts |
54
+ | `skills/` | Bundled SKILL.md corpus (CI minimum via `ci/bundle-gate.json`) |
55
+ | `ci/` | Node tests (**`test-user-env-profile.cjs`**), bundle gate JSON |
56
+ | `docs/` | Human-oriented guides (mirrors published npm tarball **`files`** list) |
328
57
 
329
- ```
330
- <workspace>/.skillforge/
331
- ├── orchestrator.db # SQLite: sessions, weights, events, project_chunks (after index)
332
- ├── policies.json # Optional policies + overlay (or repo-root skillforge-policies.json)
333
- └── last_route.json # Last CLI route snapshot (when applicable)
334
- ```
335
-
336
- **Global default:**
337
-
338
- ```
339
- ~/.skillforge/
340
- ├── venv/
341
- ├── data/orchestrator.db
342
- ├── skills/ # User skills
343
- └── packs/ # Pack working copies
344
- ```
345
-
346
- | Command | Role |
347
- |---------|------|
348
- | `skillforge events` | Usage + recent **`route`** / **`feedback`** rows (`--watch`, `--project-root`, `--user`, `--verbose`). |
349
- | `skillforge index` | (Re)build **`project_chunks`**. |
350
- | `skillforge reset` | Clears learning + events in the target DB. |
351
- | `skillforge health` | Validates paths, catalog discovery, optional deep router load. |
352
- | `skillforge route-eval` | Runs JSON fixtures (CI). |
58
+ **Tests:** **`npm test`** (Node **`--check`**) + **`cd python && pytest tests/`** in CI (**`.github/workflows/ci.yml`**).
353
59
 
354
60
  ---
355
61
 
356
- ## Security
62
+ ## NPM package
357
63
 
358
- - **Redaction is best-effort.** Do not treat scrubbed output as a certified wipe of secrets.
359
- - **Secrets:** keep **`ANTHROPIC_API_KEY`** and workspace tokens out of VCS; inject via host or OS secret stores.
360
- - **Project notes** intentionally **do not apply** without **`project_root`** to reduce cross-talk from global policy config.
361
- - **Disclosure:** follow **[SECURITY.md](SECURITY.md)** (private channels for undisclosed issues).
64
+ Scoped package **`@heytherevibin/skillforge`**: **`npm install -g`** or **`npx -y`** (see badges above).
362
65
 
363
66
  ---
364
67
 
365
- ## Contributing and governance
366
-
367
- | Document | Purpose |
368
- |----------|---------|
369
- | [CONTRIBUTING.md](CONTRIBUTING.md) | Workflow and local checks. |
370
- | [CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md) | Community standards. |
371
- | [SECURITY.md](SECURITY.md) | Reporting vulnerabilities. |
372
- | [RELEASING.md](RELEASING.md) | Tags, **`NPM_TOKEN`**, npm **2FA** / granular tokens, CI vs release workflows. |
373
- | [CHANGELOG.md](CHANGELOG.md) | Version-by-version changes. |
374
- | [STRATEGY.md](STRATEGY.md) | Product direction and non-goals. |
375
-
376
- **Continuous integration:** `.github/workflows/ci.yml` (**push** / **PR** to **`main`**).
68
+ ## License
377
69
 
378
- **License:** [LICENSE](LICENSE) (MIT).
70
+ [MIT License](LICENSE) © contributors.
package/RELEASING.md CHANGED
@@ -5,7 +5,7 @@
5
5
  | Workflow | File | When it runs |
6
6
  |----------|------|----------------|
7
7
  | **CI** | [.github/workflows/ci.yml](.github/workflows/ci.yml) | Every **push** and **pull request** to `main`; also **`workflow_dispatch`** (run manually from the Actions tab) |
8
- | **Skillforge release** | [.github/workflows/release.yml](.github/workflows/release.yml) | When a **`v*`** tag is **pushed** to the repository (e.g. `v0.10.0`) |
8
+ | **Skillforge release** | [.github/workflows/release.yml](.github/workflows/release.yml) | When a **`v*`** tag is **pushed** to the repository (e.g. `v0.10.1`) |
9
9
 
10
10
  In the GitHub UI, open **Actions** and look for **CI** and **Skillforge release** (not the older “publish to npm” name).
11
11
 
@@ -28,14 +28,15 @@ You need the **`NPM_TOKEN`** repository secret.
28
28
 
29
29
  Create one at [npm → Access Tokens](https://www.npmjs.com/settings/~/tokens) (**Generate New Token** → **Granular Access Token**). Optionally explore [**trusted publishing** (OIDC)](https://docs.npmjs.com/trusted-publishers/) later to avoid long-lived tokens.
30
30
 
31
- 1. On `main`, set **`version`** in `package.json` to the version you are releasing (must be **unused** on npm e.g. `0.10.0`).
32
- 2. Match **MCP** `serverInfo.version` in `python/app/mcp_server.py` (must equal `package.json`) and add a **`CHANGELOG.md`** section for that version.
31
+ 1. On `main`, set **`version`** in **`package.json`** to the version you are releasing (must be **unused** on npm). Bump **[`CHANGELOG.md`](CHANGELOG.md)** with a **`## X.Y.Z`** section for that number. Reflect that **semver** anywhere user-facing prose names the shipping line (**[`README`](README.md)**, **[`docs/README`](docs/README.md)**, **[`getting-started`](docs/getting-started.md)**, **[`docs/mcp-integration`](docs/mcp-integration.md)**, …). Shield URLs track **published** artefacts and **`main`** automatically; only **plain-text version numbers** need a manual bump.
32
+ 2. **MCP `serverInfo.version`** matches **`package.json`** automatically (**[`python/app/npm_pkg_version.py`](python/app/npm_pkg_version.py)** **`published_package_version()`** used from **`python/app/mcp_server.py`**). Operators can override via **`SKILLFORGE_MCP_SERVER_VERSION`**; do **not** hand-edit a second copy of semver in Python.
33
33
  3. Commit and **`git push origin main`**. Wait for **CI** to pass.
34
- 4. Create a tag whose name is **`v` + that exact version**:
35
- `git tag v0.10.0 && git push origin v0.10.0`
34
+ 4. Tag **`v` + the value of **`package.json` `version`**, push to **`origin`**, e.g.:
35
+ `git tag v0.11.7 && git push origin v0.11.7`
36
+ (**Replace **`v0.11.7`** with your semver** when releasing a newer number.)
36
37
  5. Open **Actions → Skillforge release**. The job will **fail the version check** if the tag does not match `package.json`.
37
38
  6. Confirm on npm: `npm view @heytherevibin/skillforge version`
38
- Confirm the **GitHub Release** exists with title **`Skillforge <tag>`** (e.g. **`Skillforge v0.10.0`**) and the `.tgz` asset.
39
+ Confirm the **GitHub Release** exists with title **`Skillforge <tag>`** (e.g. **`Skillforge v0.11.7`**) and the `.tgz` asset.
39
40
 
40
41
  Scoped packages require a **public** publish; the workflow already runs `npm publish --access public`.
41
42
 
@@ -74,7 +75,7 @@ The **minimum** number of **`skills/**/**/SKILL.md`** files required in CI is **
74
75
  ## Local sanity checks (before push)
75
76
 
76
77
  ```bash
77
- node --check bin/cli.js && node --check lib/packs.js
78
+ node --check bin/cli.js && node --check lib/packs.js && node --check lib/user-env-profile.js
78
79
  npm test
79
80
  ```
80
81
 
package/STRATEGY.md CHANGED
@@ -27,8 +27,8 @@ Skillforge is a **local, MCP-first orchestration layer** that selects a **small,
27
27
  |-----------------|---------------------|
28
28
  | **Project workspace** | Pass **`project_root`** so state and optional RAG live under **`.skillforge/`**. |
29
29
  | **Strict context budgets** | Tune **`SKILLFORGE_CONTEXT_*`**, **`SKILLFORGE_ROUTE_MAX_CHARS`**, and project RAG caps. |
30
- | **No LLM keys on router** | Use **`SKILLFORGE_ROUTER_MODE=embedding`** (or omit **`ANTHROPIC_API_KEY`** in auto mode). |
31
- | **Human-in-the-loop picks** | **`SKILLFORGE_ROUTER_MODE=host`**: shortlist first, then **`picked_names`**. |
30
+ | **No LLM keys on router** | Omit **`ANTHROPIC_API_KEY`** (**`host`** is the default when **`ROUTER_MODE`** is unset) or set **`SKILLFORGE_ROUTER_MODE=embedding`**. Legacy **auto** + key: **`SKILLFORGE_ROUTER_MODE=auto`**. |
31
+ | **Human-in-the-loop picks** | **`SKILLFORGE_ROUTER_MODE=host`** (default when unset): shortlist first, then **`picked_names`**. |
32
32
  | **Org policy** | Central **`SKILLFORGE_ROUTE_POLICIES`** / file + per-repo **`policies.json`** with overlay keys. |
33
33
 
34
34
  ## Near-term themes