pi-crew 0.9.47 → 0.9.49

package/docs/decisions/2026-07-24-oidc-trusted-publishing.md

@@ -0,0 +1,112 @@
+# ADR: OIDC Trusted Publishing for npm
+
+**Date**: 2026-07-24
+**Status**: PROPOSED — not yet active (requires npm-side configuration)
+**Distilled from**: oh-my-pi@c84e9c020 `CONTRIBUTING.md:313-317` + `.github/workflows/release.yml` (GPL-3.0 — pattern only, not code)
+
+## Context
+
+pi-crew currently publishes to npm manually via `npm publish` using a long-lived `NPM_TOKEN` (npm user `bom0792`). This has two supply-chain weaknesses:
+
+1. **Token theft surface**: a long-lived npm token stored as a GitHub secret can be exfiltrated via a compromised dependency or workflow `pull_request_target` misuse. The token grants full publish access until manually rotated.
+2. **No build attestation**: published packages carry no cryptographic proof of *what CI built and published them*. Consumers cannot verify a `pi-crew@x.y.z` tarball came from this repo's CI.
+
+The v0.9.3 mid-CI-publish incident class (publishing mid-run rather than from a dedicated gated workflow) is a third motivator: a dedicated release workflow moves publishing out of the general CI run and into a controlled, test-gated path.
+
+## Decision
+
+Adopt **npm Trusted Publishing (OIDC)** + **provenance attestation**, following the pattern oh-my-pi@c84e9c020 uses (`id-token: write`, no `NPM_TOKEN`, `--provenance`):
+
+1. Eliminate the long-lived `NPM_TOKEN`. Instead, GitHub Actions exchanges an OIDC token for a short-lived npm publish credential at publish time. No token to steal.
+2. Attach **provenance** (Sigstore-signed build attestation) to every published tarball. Consumers can verify origin with `npm audit signatures`.
+
+## Implementation steps (external + code)
+
+### A. npm-side (manual, one-time — cannot be done from code)
+
+1. Go to `https://www.npmjs.com/package/pi-crew/access`
+2. Under **Publishing access**, switch to **Require two-factor authentication and disallow tokens** (or set up Trusted Publishing if available for the account).
+3. Alternatively, use `npm token create --cidr` to scope a publish-only token to GitHub Actions IPs (weaker than OIDC but simpler).
+4. For full OIDC: configure the npm-side Trusted Publishing link to `baphuongna/pi-crew` + the `release.yml` workflow.
+
+### B. Code-side (when OIDC is configured)
+
+1. Add `.github/workflows/release.yml` (sample below) — **manual trigger only** (`workflow_dispatch`) until proven stable.
+2. Do **NOT** add `provenance=true` to a global `.npmrc` — that breaks local `npm publish` with "requires CI environment". Provenance must be a `--provenance` flag inside the CI workflow only.
+
+### Sample `release.yml` (MIT-licensed, pi-crew style — not copied from oh-my-pi)
+
+```yaml
+name: Release
+
+on:
+  workflow_dispatch:
+    inputs:
+      version:
+        description: 'Version to publish (must match package.json)'
+        required: true
+        type: string
+
+permissions:
+  contents: write
+  id-token: write  # OIDC for trusted publishing
+
+concurrency:
+  group: release-${{ github.ref }}
+  cancel-in-progress: false
+
+jobs:
+  publish:
+    runs-on: ubuntu-latest
+    environment: release  # requires manual approval via GitHub environment protection
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+
+      - uses: actions/setup-node@v4
+        with:
+          node-version: '22'
+          registry-url: 'https://registry.npmjs.org'
+
+      - run: npm ci
+
+      - name: Verify version matches input
+        run: |
+          PKG_VERSION=$(node -p "require('./package.json').version")
+          if [ "$PKG_VERSION" != "${{ inputs.version }}" ]; then
+            echo "Version mismatch: package.json=$PKG_VERSION input=${{ inputs.version }}"
+            exit 1
+          fi
+
+      - run: npm run typecheck
+      - run: npm run test:critical
+      - run: npm run build:bundle
+
+      - name: Publish with provenance
+        run: npm publish --provenance --access public
+        # No NPM_TOKEN env — OIDC trusted publishing exchanges the job's
+        # id-token for a short-lived credential automatically.
+
+      - name: Create GitHub release
+        uses: softprops/action-gh-release@v2
+        with:
+          tag_name: v${{ inputs.version }}
+          generate_release_notes: true
+```
+
+## What this prevents
+
+- **Token exfiltration**: no long-lived `NPM_TOKEN` secret to steal.
+- **Unverified publishes**: every tarball carries a signed attestation linking it to this repo's CI.
+- **Accidental publishes**: `workflow_dispatch` + `environment: release` (protected) requires manual approval before publish runs.
+
+## Why not active yet
+
+- npm-side Trusted Publishing / token-lockdown must be configured at `npmjs.com/package/pi-crew/access` first.
+- The `environment: release` protection rule (required reviewers) must be configured in GitHub repo settings.
+- Until both are done, keep the manual `npm publish` + GitHub release flow.
+
+## Trade-off
+
+OIDC trusted publishing is slightly less convenient than a long-lived token (each publish goes through a workflow dispatch + approval). The supply-chain hardening is worth it for a package with 700+ files shipping to public npm.

package/docs/publishing.md

@@ -71,8 +71,11 @@ pi install npm:pi-crew
 1. Builds the ESM bundle (`scripts/build-bundle.mjs`) — best-effort, falls
    back to strip-types loading if esbuild is missing.
 2. Installs the bundled `crew-vibes.ttf` font into the user fonts directory.
-3. Copies every `skills/<name>/` dir to `~/.pi/agent/skills/` so pi-crew's
-   skills are available globally (not just inside the pi-crew project).
+3. Runs `cleanupStaleSkillCopies()` — a one-time migration that removes
+   stale skill copies from `~/.pi/agent/skills/` left by v0.9.47's (now-removed)
+   `copySkills()` step. Only removes **byte-identical** copies; user-customized
+   skills are preserved. Pi discovers skills natively from the npm package's
+   own `skills/` dir, so copies are never needed.
 
 All three steps are best-effort and never fail the install.
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pi-crew",
-  "version": "0.9.47",
+  "version": "0.9.49",
   "description": "Pi extension for coordinated AI teams, workflows, worktrees, and async task orchestration",
   "author": "baphuongna",
   "license": "MIT",
@@ -36,7 +36,7 @@
     "agents/",
     "teams/",
     "workflows/",
-    "skills/**/SKILL.md",
+    "skills/**",
     "themes/",
     "assets/crew-vibes.ttf",
     "assets/runner-spritesheet.png",

package/scripts/postinstall.mjs

@@ -6,13 +6,17 @@
  *      let Pi fall back to strip-types loading).
  *   2. Install the bundled crew-vibes.ttf into the user fonts directory so
  *      the crew-vibes speed/capacity PUA glyphs render.
- *   3. Copy bundled skills to ~/.pi/agent/skills/ so they are available
- *      globally (not just inside the pi-crew project).
+ *   3. Remove stale skill copies from ~/.pi/agent/skills/ (migration from
+ *      the v0.9.47 copySkills() bug — see cleanupStaleSkillCopies()).
  *
  * Replaces the old `postinstall` shell chain so the font install runs on
  * every platform without relying on shell-specific chaining (`;`/`&&`).
+ *
+ * NOTE: skills are NOT copied here. Pi discovers skills natively from the
+ * npm package dir (`~/.pi/agent/npm/node_modules/pi-crew/skills/`); copying
+ * them to ~/.pi/agent/skills/ only creates duplicate "collision" warnings.
  */
-import { cpSync, existsSync, mkdirSync, readdirSync } from "node:fs";
+import { existsSync, readFileSync, readdirSync, rmSync } from "node:fs";
 import { spawnSync } from "node:child_process";
 import { dirname, join } from "node:path";
 import { fileURLToPath } from "node:url";
@@ -29,32 +33,53 @@ function run(scriptRel) {
 }
 
 /**
- * Copy each skill directory from pi-crew/skills/ to ~/.pi/agent/skills/.
- * Best-effort: must never fail the install.
+ * Remove stale skill copies from ~/.pi/agent/skills/ that are byte-identical
+ * to the bundled skills/ — left over from the v0.9.47 copySkills() postinstall
+ * (which redundantly copied all skills, causing "collision" warnings).
+ *
+ * Pi discovers skills natively from the npm package dir, so these copies only
+ * create noise AND can shadow newer package versions on upgrade.
+ *
+ * SAFE: only removes EXACT (byte-identical) copies. A skill the user has
+ * customized (differs from the package) is preserved. Best-effort, never
+ * fails the install.
  */
-function copySkills() {
+function cleanupStaleSkillCopies() {
 	const srcSkillsDir = join(root, "skills");
 	if (!existsSync(srcSkillsDir)) return;
 
 	const destSkillsDir = join(homedir(), ".pi", "agent", "skills");
-	try {
-		mkdirSync(destSkillsDir, { recursive: true });
-	} catch {
-		return; // can't create dest — skip silently
-	}
+	if (!existsSync(destSkillsDir)) return;
 
+	let removed = 0;
+	let kept = 0;
 	for (const entry of readdirSync(srcSkillsDir, { withFileTypes: true })) {
 		if (!entry.isDirectory()) continue;
-		const skillMd = join(srcSkillsDir, entry.name, "SKILL.md");
-		if (!existsSync(skillMd)) continue;
+		const srcSkillMd = join(srcSkillsDir, entry.name, "SKILL.md");
+		if (!existsSync(srcSkillMd)) continue;
+		const destSkillDir = join(destSkillsDir, entry.name);
+		const destSkillMd = join(destSkillDir, "SKILL.md");
+		if (!existsSync(destSkillMd)) continue;
 		try {
-			cpSync(join(srcSkillsDir, entry.name), join(destSkillsDir, entry.name), {
-				recursive: true,
-			});
+			const srcContent = readFileSync(srcSkillMd, "utf8");
+			const destContent = readFileSync(destSkillMd, "utf8");
+			if (srcContent === destContent) {
+				rmSync(destSkillDir, { recursive: true, force: true });
+				removed++;
+			} else {
+				// User customized this skill — preserve it.
+				kept++;
+			}
 		} catch {
 			// best-effort: skip this skill on error
 		}
 	}
+	if (removed > 0) {
+		const note = kept > 0 ? ` (${kept} customized copy(ies) preserved)` : "";
+		console.warn(
+			`[pi-crew] postinstall: removed ${removed} stale skill cop${removed === 1 ? "y" : "ies"} from ~/.pi/agent/skills/${note}.`,
+		);
+	}
 }
 
 function main() {
@@ -74,8 +99,8 @@ function main() {
 		}
 		// Font install is best-effort and must never fail the install.
 		run("scripts/install-crew-vibes-font.mjs");
-		// Copy bundled skills to ~/.pi/agent/skills/ for global availability.
-		copySkills();
+		// Migration cleanup: remove stale skill copies from v0.9.47's copySkills().
+		cleanupStaleSkillCopies();
 	} catch (err) {
 		// Postinstall must NEVER fail the install (SEC-M2).
 		console.warn("[pi-crew] postinstall: best-effort step failed:", err instanceof Error ? err.message : err);

package/skills/distill-persona/BUILD-NOTES.md

@@ -0,0 +1,55 @@
+# distill-persona — Build Notes (decision traceability)
+
+> Why this pi-native port makes each choice. Traces every design decision to a finding from the review/experiment cycle.
+
+## What this is
+A pi-skill port of nuwa's (女娲) distillation engine, incorporating: the nuwa methodology (researched), the rigorous review (NUWA-SKILL-REVIEW), the variant reviews (x-mastery-mentor, mrbeast), the empirical fidelity experiment (f2-experiment), AND **two self-distillation passes** over the 3 source projects (dogfooded — see below).
+
+## Self-distillation log (dogfooded the skill on its own sources)
+The skill was applied to the 3 source projects (nuwa + 2 awesome-lists), topic flavor. Three rounds:
+- **Pass 1** (`references/distillation-field-synthesis.md`): triple-verified 13 candidates → 3 NEW field models (M-F1 dissemination flywheel, M-F2 target taxonomy, M-F3 ethics spectrum) integrated into SKILL.md Phase 0.
+- **Pass 2** (`references/distillation-field-synthesis-pass2.md`): independent re-derivation → **core 8/8 reproduced (100%)** + 1 standout new model **M-F4 (field over-claims its own fidelity)** — elevates the F2' finding from a skill-internal fix to a field-level meta-model.
+- **R1 exhaustive sweep** (user correction: distillation must sweep EVERY part over many rounds, not gestalt): upgraded Phase 1 to add **project/topic exhaustive-sweep mode** (coverage manifest + diminishing-returns gate + self-correction meta-loop). Then RAN it — 4 parallel batches swept the 9 uncovered nuwa examples + scripts + 76KB README + 5 test files (`references/research/r1-{a,b,c,d}.md`). **3 self-corrections integrated**: (1) **M-F2 empirically corrected** — gestalt spectrum was wrong (84% public-figures, bimodal; +meta-tier, living-creator, commemorative-split); (2) **+M-F7** (structural invariants = index-scale quality gate, from the test sweep); (3) **M-F4 strengthened** (quality_check.py structural-pass ≠ behavioral edge-honesty → never trust structural-only auto-pass). This is the self-correction meta-loop firing: the sweep found the skill's own M-F2 was gestalt-inaccurate and fixed it.
+- **R1.5 extraction verification** (user correction: distillation must verify it's optimal+effective, not "chưng cất bừa làm rác"): added **Phase 2.6 Extraction verification gate** (V1 signal-not-persona-content · V2 non-redundant · V3 effective · V4 optimal · reject principle + post-integration delta check). RAN it on R1 output (`references/research/r1-verification.md`): audited ~30 candidates → **5 field models kept** (all pass V1-V4), **~22 persona-content REJECTED** (V1 fail — idiot-index/desire-as-contract/etc. are subject content, not distillation method), 4 reclassified. Over-extraction was already contained (garbage stayed in research files); now made *principled & repeatable*. Skill carries 10 models total (5 core M1-M5 + 5 field M-F1..F4/M-F7) — sharp, not bloated.
+- **Reproducibility (F10 dogfood)**: core model-set overlap 8/8 — methodology is stable (not coin-flip); pass 2 is a strict superset (deeper pass enlarges surface). R1 then enlarged surface further (M-F7) AND corrected an existing model (M-F2). Caveat: same mind across passes — suggestive, not proof.
+
+## Decision → finding trace
+
+| Decision in the skill | Driven by |
+|---|---|
+| pi frontmatter (`name` + 1-line `description` + `triggers:` list) | F4/F5 — pi's explicit-trigger convention avoids nuwa's keyword-stuffed descriptions |
+| Detect `<skill-dirs>`; don't hardcode `.claude/skills/` | **F1** — environment lock-in (the #1 reason nuwa won't run in pi) |
+| Dispatch research via `team action='parallel'` / background `Agent`, with **serial + single-agent fallback as documented default** | **F1** + nuwa's own failure table ("runtime without background tasks hangs") |
+| Guard tool availability (WebSearch/rg/git/pi-langsrv "if present") | **F1** — don't assume named external skills exist |
+| Cost tier quoted in Phase 0, context-window segmentation guidance front-loaded | **F6** — nuwa buries this in a failure table; default standard tier is the one that blows windows |
+| Triple-verification with exclusivity emphasized as anti-bloat | **S2** + SOFTWARE-DISTILLATION-DEEP-DIVE §D |
+| Honest boundaries ≥3 + `distilled:` staleness date mandatory | **S3** + software staleness anchor |
+| Fallback tree + anti-example blacklist required in output | **S5** |
+| Contradictions preserved (≥2 tensions, never flattened) | **S7** |
+| **The F2' third inference category** ("in-field-but-never-addressed → flag as inference") in the Agentic Protocol | **f2-experiment finding** — the empirical result; this is the single most important addition over nuwa |
+| Phase 4 mandates a **NOVEL in-domain edge question** not covered by research files | **F2'/f2-experiment** — nuwa's edge questions sat too close to documented material; that's why its 97/100 didn't reproduce (got 71-72) |
+| Scorer MAY read the skill file (kept nuwa's design) | **F2 REFUTED** — experiment showed skill-file access does NOT inflate scores (71 vs 72 converged); so the "blind scoring" fix proposed earlier is dropped |
+| Operational scripts wired INTO the Agentic Protocol Step 2 | **F13** — mrbeast scripts were orphaned showpieces |
+| Engine embeds methodology inline (self-contained) | **F9** — nuwa's engine depends on its own references/ at runtime; outputs are portable but the engine isn't |
+| Three flavors (person / topic / software) with pointers | x-mastery-mentor (topic works, S8/S9/S10) + SOFTWARE-DISTILLATION-DEEP-DIVE |
+
+## What was deliberately NOT copied from nuwa
+- Keyword-stuffed descriptions (F4) → pi triggers list instead.
+- The published 97/100 fidelity scores as ground truth (F2') → re-run on novel edge questions.
+- Chinese-source blacklist as a *universal* rule (F11) → scoped to "Chinese figures only, quality reason".
+- Default = the expensive standard tier in one session (F6) → segmentation front-loaded.
+
+## Known gaps in this port (honest) — status after the gap-closing pass
+
+- ✅ **F2' validation — CLOSED.** Ran the experiment on 2 more nuwa skills (ilya, munger): published 94/97 → blind 76/67; edge-honesty 20→13/7. F2' generalizes at n=3. Refined mechanism: **framework-answerable** novel edges are the drift zone; fact-demanding edges give false passes. See `source/f2-experiment/validation-conclusion.md`. The skill's F2' rule + Phase 4 now specifically require a framework-answerable edge.
+- ✅ **Phase 4 automation + reproducibility test — CLOSED.** Shipped `scripts/fidelity_eval.py` (stdlib-only): `validate` (spec must have 3 known + 1 framework-answerable edge + ground-truth + edge-note) · `runbook` (emits runtime-agnostic answerer+scorer prompts) · `parse` (5-dim scorecard → grade, with the F2' edge-honesty gate: edge <14 = NO-SHIP regardless of total) · `repro` (Jaccard/name-overlap on mental models, <60% = non-reproducible). Tested against all 3 experiment scorecards + self/diff-persona repro.
+- ⚠️ **All 3 nuwa skills grade NO-SHIP** under the validated gate (karpathy edge 6, ilya 13, munger 7 — all <14). Strong confirmation the gate catches what it should: none of nuwa's published skills reliably flag framework-derived inference on novel in-domain questions.
+- 🔲 **No operational DNA script shipped yet.** The engine is methodology; a software-flavor distillation should add per-target `scripts/<persona>_dna.py` (per the mrbeast pattern / F13 — wired into the Agentic Protocol, not orphaned). Build per-target.
+- 🔲 **Topic-flavor user-data persistence** (S9) referenced but not implemented — add when building a stateful topic skill.
+- 🔲 **pi-crew specialization adapter** not written — the base is runtime-agnostic by design; a `distill-persona-pi-crew` specialization would pin `team action='parallel'` + pi skill-dirs + pi-langsrv. Defer until the base is battle-tested.
+
+## Suggested next slices (in priority order)
+1. **Scale F2' validation** to 5+ skills × 3-5 framework-answerable edges each, via `fidelity_eval.py runbook`. The harness makes this cheap. Confirms the 60% repro floor and the edge-honesty gate are calibrated.
+2. **Build `distill-software`** as a sibling (the SOFTWARE-DISTILLATION-DEEP-DIVE playbook, pi-langsrv-native; ships a `<engineer>_dna.py` code-Expression-DNA script wired into its Agentic Protocol).
+3. **Write the pi-crew specialization adapter** (pins the runtime; reuses the validated base unchanged).
+4. **Implement topic-flavor user-data persistence** (S9) when the first topic-skill use case appears.