@zalom/plastic 1.0.0-alpha.2 → 1.0.0-alpha.20

package/skills/doctor/SKILL.md

@@ -0,0 +1,116 @@
+---
+name: plastic-doctor
+description: Use when diagnosing Plastic installation health, after updates, or when something seems broken. Runs checks and reports findings with fix options.
+---
+
+# Doctor — Plastic Health Check
+
+## When to Use
+
+- User invokes `/plastic-doctor`
+- After `plastic-update` completes (automatically)
+- When hooks aren't firing, skills aren't loading, or something seems broken
+- When the user says "check plastic", "diagnose", "what's wrong with plastic"
+
+## Procedure
+
+### Step 1: Run the diagnostic script
+
+```bash
+ruby ~/.plastic/scripts/doctor.rb --agent claude
+```
+
+Replace `claude` with the current agent type if known (`codex`, `hermes`).
+
+Parse the JSON output from stdout. The script is read-only and never modifies
+files. Errors go to stderr.
+
+Exit codes indicate check results, not script failure:
+- `0` — all checks passed
+- `1` — warnings found
+- `2` — failures found
+
+All three exit codes mean the script ran successfully. Do not treat non-zero
+as an error.
+
+### Step 2: Determine overall status
+
+Read the `status` field from the JSON root:
+
+| Status | Meaning |
+|--------|---------|
+| `pass` | Everything is healthy |
+| `warn` | Warnings found but Plastic works normally |
+| `fail` | Blocking issues that prevent Plastic from operating |
+
+### Step 3: Fill the report template
+
+Read `report.md` from the same directory as this SKILL.md
+(`~/.plastic/skills/doctor/report.md` at runtime, or the plugin source
+`skills/doctor/report.md` during development).
+
+Group checks by category. For each category, list the checks with their
+status icon and message. If a check has `details`, list them as sub-items.
+
+Present the filled template to the user.
+
+### Step 4: Offer fixes (if applicable)
+
+If any checks have `fixable: true` AND status is not `pass`:
+
+1. Group fixable items by category
+2. Show what each fix would do (from `fix_hint`)
+3. Ask the user: **"Fix all / Select individually / Skip"**
+
+If no fixable issues exist, skip this step.
+
+### Step 5: Apply fixes
+
+Use the `fix_hint` value to determine the correct action:
+
+| Fix hint pattern | Agent action |
+|---|---|
+| "chmod +x on the listed files" | Run `chmod +x` on each file listed in `details` |
+| "Create missing directory" | Run `mkdir -p` on the path |
+| "Create INDEX.md with required sections" | Write INDEX.md with the 5 sections: Active, Future, Clusters, Abandoned, Completed |
+| "Add missing entries to INDEX.md" | Add orphaned intents to the appropriate INDEX.md section |
+| "Remove stale references from INDEX.md" | Edit INDEX.md to remove ghost references |
+| "Re-run installer" | Run `npx @zalom/plastic@latest --agent` |
+
+For fixes the agent cannot handle automatically, explain what the user needs
+to do manually.
+
+### Step 6: Verify
+
+After applying fixes, re-run the diagnostic script:
+
+```bash
+ruby ~/.plastic/scripts/doctor.rb --agent claude
+```
+
+Show the updated results.
+
+- If all checks pass: announce success.
+- If issues remain: explain what is still wrong and what the user can do.
+
+## Post-Update Mode
+
+When invoked from `plastic-update` (not directly by the user):
+
+1. Run the diagnostic script as in Step 1.
+2. If all checks pass: show a single line — **"Health check: all clear."**
+3. If issues are found: show the full report (Steps 3-6).
+
+This keeps the update flow clean when nothing is wrong.
+
+## Important Notes
+
+- The script is **read-only**. It inspects but never modifies files.
+  All fixes are performed by the agent using standard tools.
+- The script outputs JSON to stdout. Any diagnostic errors go to stderr.
+- Non-zero exit codes mean "issues found", not "script crashed".
+  Always parse stdout regardless of exit code.
+
+## References
+
+- Read `references/gates-stuck-detection.md` for the full gate enforcement table, bridge file pattern, and stuck detection thresholds when diagnosing gate failures or stuck agents

package/skills/doctor/references/gates-stuck-detection.md

@@ -0,0 +1,38 @@
+# Gate Enforcement and Stuck Detection
+
+## Bridge File Pattern
+
+`/tmp/plastic-{session}.json` is the hot cache; the filesystem is the authority.
+SessionStart rebuilds the bridge file from the filesystem — the bridge is always disposable.
+
+## Gate Taxonomy
+
+| Gate type | Purpose |
+|---|---|
+| **Pre-flight** | Prerequisites exist? |
+| **Revision** | New info invalidated prior work? |
+| **Escalation** | Blocked items surface to user |
+| **Abort** | Inconsistent state detected |
+
+## Full Gate Enforcement
+
+| Gate | Blocked action | Required prerequisite |
+|---|---|---|
+| Pre-flight | Cannot write `plan.md` | `spec.md` must exist |
+| Pre-flight | Cannot create `actions/` | `spec.md` must exist |
+| Pre-flight | Cannot write `outcome.md` | `checklist.md` must exist with all items checked |
+| Revision | Cannot proceed to Exec | Context changed after spec.md — re-derive spec |
+| Abort | Cannot complete intent | `outcome.md` exists but checklist has unchecked items |
+
+Hard blocking — hooks exit with code 2 when gates fail.
+
+## Stuck Detection
+
+| Condition | Threshold | Action |
+|---|---|---|
+| Consecutive gate failures | 3+ | Warning |
+| Consecutive gate failures | 5+ | Force savepoint + escalate |
+| No activity | 5+ min | Warning |
+| No activity | 10+ min | Force savepoint + escalate |
+| Context pressure | 80% | Warning |
+| Context pressure | 90% | Force savepoint |

package/skills/doctor/report.md

@@ -0,0 +1,96 @@
+# Plastic Doctor Report
+
+<!-- =======================================================================
+  AGENT INSTRUCTIONS -- How to fill this template
+  =========================================================================
+  1. Run the doctor script. It outputs JSON with check results.
+  2. Replace every {{placeholder}} below with the corresponding JSON value.
+  3. For the category sections: the template shows ONE example section.
+     Repeat that pattern for each unique category in the checks array.
+     The six known categories and their display names are:
+       global_store       -> "Global Store"
+       conventions        -> "Conventions"
+       agent_registration -> "Agent Registration"
+       core_files         -> "Core Files"
+       project_stores     -> "Project Stores"
+       deprecations       -> "Deprecations"
+  4. For each check within a category, emit one line with the status icon
+     and the check message. If the check has non-empty details, list them
+     as indented sub-items.
+  5. The "Fixable Issues" section should ONLY appear if at least one check
+     has fixable=true AND status is not "pass". Omit the entire section
+     otherwise.
+  6. Status icons (plain text, no emoji):
+       pass -> [PASS]
+       warn -> [WARN]
+       fail -> [FAIL]
+  7. The overall status icon in the header uses the same mapping.
+  8. After filling, remove all HTML comments -- they are instructions only.
+  ======================================================================= -->
+
+## {{overall_status_icon}} Overall: {{status}} -- Plastic v{{version}}
+
+Checked at: {{timestamp}}
+
+### Summary
+
+{{pass}} passed, {{warn}} warnings, {{fail}} failed -- {{total}} checks total
+
+---
+
+<!-- =====================================================================
+  CATEGORY SECTIONS
+  ======================================================================
+  Repeat the block below ONCE PER CATEGORY present in the checks array.
+  Group checks by their "category" field. Use the display name mapping
+  above for the heading. Within each category, list every check as a
+  single line: status icon + message. If a check has non-empty "details",
+  list each detail as an indented bullet beneath.
+
+  Example category section (for agent_registration with two checks):
+  ===================================================================== -->
+
+### Agent Registration
+
+- [PASS] Claude Code adapter registered
+- [FAIL] 2 hook scripts not executable
+  - ~/.claude/hooks/plastic-session-start
+  - ~/.claude/hooks/plastic-gate-check
+
+<!-- =====================================================================
+  Repeat the above pattern for each category found in the checks array.
+  Only include categories that have at least one check.
+  Order categories as they appear in the checks array.
+  ===================================================================== -->
+
+---
+
+<!-- =====================================================================
+  FIXABLE ISSUES SECTION
+  ======================================================================
+  Include this section ONLY if one or more checks have fixable=true AND
+  status is "warn" or "fail". If no fixable issues exist, omit everything
+  from the "Fixable Issues" heading through the end of the horizontal
+  rule that follows the table.
+
+  For each fixable check that is not "pass", emit one table row:
+    | status_icon | check_message | fix_hint |
+  ===================================================================== -->
+
+### Fixable Issues
+
+| Status | Issue | Fix |
+|--------|-------|-----|
+| [FAIL] | 2 hook scripts not executable | chmod +x on the listed files |
+
+<!-- =====================================================================
+  Repeat one row per fixable non-pass check.
+  ===================================================================== -->
+
+---
+
+<!-- =====================================================================
+  FOOTER -- always include this line exactly as written.
+  ===================================================================== -->
+
+Run `plastic doctor --fix` to auto-fix all fixable issues, or ask me to fix them now.

package/skills/evaluating-skills/SKILL.md

@@ -0,0 +1,140 @@
+---
+name: plastic-evaluating-skills
+description: >
+  Evaluate Plastic skills for correctness, convention compliance, and
+  progressive disclosure. Use when testing whether a skill produces good
+  outputs, verifying convention compliance after changes, running evals
+  against skills or instructions, creating evals for a new or updated
+  skill, checking if a description triggers correctly, or assessing
+  whether a skill is still needed. Also use when the user says "evaluate",
+  "test the skill", "run evals", "check conventions", or "write evals".
+---
+
+# Evaluating Skills
+
+Eval methodology for Plastic skills, based on
+[agentskills.io](https://agentskills.io/skill-creation/evaluating-skills)
+and [Anthropic's eval guide](https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents).
+
+## Gotchas
+
+- Assertions written before observing output are almost always wrong — run
+  the eval first, observe actual output, THEN write assertions
+- Near-miss negative test cases are the most valuable — prompts that share
+  keywords with should-trigger cases but need a different skill entirely
+- Select the best skill iteration by validation pass rate, not the last one
+- Grade outcomes, not execution paths — if the agent solved the task via an
+  unexpected route but produced correct output, that is a pass
+- Same skill can behave differently across agent frameworks — test on each
+  target agent (Claude Code, Hermes, OpenClaw, Codex)
+
+## Procedure
+
+### Step 1: Choose eval scope
+
+Determine what you are evaluating:
+
+- **Description triggering** — does the agent activate the right skill for
+  a given prompt? Tests the description field effectiveness.
+- **Output quality** — does the skill produce correct results when activated?
+  Tests the skill body and references.
+- **Convention compliance** — does the output follow Plastic conventions?
+  Read `references/convention-checks.md` for the full assertion library.
+
+Multiple scopes can apply to the same skill. Start with the scope that
+addresses your immediate concern, add others as needed.
+
+### Step 2: Design test cases
+
+Create `evals/evals.json` in the skill being evaluated. Copy the starter
+template from `assets/eval-template.json` in this skill.
+
+**For description triggering:**
+- Write ~20 queries: 8-10 should-trigger, 8-10 should-not-trigger
+- Split 60/40 into train and validation sets (proportional mix in each)
+- Include near-miss negatives that share keywords but need a different skill
+- In `expected_output`, describe whether the skill should or should not activate
+  and why
+
+**For output quality:**
+- Start with 2-3 test cases, expand after first results
+- Use realistic user prompts with varied phrasing, detail level, and formality
+- In `expected_output`, describe what correct output looks like — not exact text
+- Use `files` array for any input files the test needs
+
+**For convention compliance:**
+- Start with 2-3 test cases targeting specific convention areas
+- In `expected_output`, describe which conventions must be met
+- Read `references/convention-checks.md` for the full assertion library
+
+Leave `assertions` arrays empty. They are populated after Step 4.
+
+### Step 3: Run paired evals
+
+Dispatch a subagent per test case to ensure clean context — no leakage
+between test runs. Run each case twice:
+
+1. **With skill** — the skill is available and loaded
+2. **Without skill** — the skill is not available (baseline comparison)
+
+The delta between with-skill and without-skill measures what the skill adds.
+If the delta is negligible, the skill may not be adding value for that case.
+
+Grade outcomes, not paths. An unexpected tool-call sequence that produces
+correct output is still a pass.
+
+### Step 4: Write assertions after observing
+
+Review actual outputs from Step 3. Write specific, verifiable assertions
+based on what you observed — not what you expected beforehand.
+
+Choose the grader type that fits each assertion:
+- **Code-based** — structure, file existence, format validity, counts
+- **LLM-as-judge** — quality, completeness, tone, semantic correctness
+- **Human** — edge cases, calibration, judgment calls
+
+Read `references/eval-methodology.md` for the full grader taxonomy and
+LLM-judge calibration protocol.
+
+Good assertions are specific, verifiable, and countable:
+"Output includes at least 3 concrete recommendations."
+
+Weak assertions are vague: "Output is good."
+Brittle assertions use exact phrase matching.
+
+Require concrete evidence for PASS. No benefit of the doubt.
+
+### Step 5: Grade and iterate
+
+Compute pass rates per test case and aggregate across the eval suite.
+
+Track two metrics separately:
+- **pass@k** — succeeded at least once in k trials (capability)
+- **pass^k** — succeeded every time in k trials (reliability)
+
+Read `references/eval-methodology.md` for formulas and interpretation.
+
+Three signal sources for skill improvement:
+1. **Failed assertions** — specific gaps in the skill
+2. **Human feedback** — broader quality issues not captured by assertions
+3. **Execution transcripts** — reveals WHY things went wrong
+
+Feed all three plus the current SKILL.md to propose targeted changes.
+Iterate on the train set only. Check the validation set for generalization.
+5 iterations is usually enough. If not improving after 5, the test cases
+themselves may be the problem — revisit Step 2.
+
+### Step 6: Graduate and monitor
+
+Once a skill hits ~100% on capability evals (pass@k = 1.0 for 3+
+consecutive runs):
+
+1. **Graduate** capability evals into regression tests — run them on every
+   skill change to protect against backsliding
+2. **Monitor** the with/without delta over time — if it shrinks to zero
+   across 3+ runs, the model may have internalized the skill
+3. **Retire** cautiously — archive the skill, do not delete. Re-test after
+   model updates in case capabilities regress.
+
+Read `references/eval-methodology.md` for graduation criteria, retirement
+detection, and the three-layer eval taxonomy.

package/skills/evaluating-skills/assets/eval-template.json

@@ -0,0 +1,12 @@
+{
+  "skill_name": "",
+  "evals": [
+    {
+      "id": 1,
+      "prompt": "",
+      "expected_output": "",
+      "files": [],
+      "assertions": []
+    }
+  ]
+}

package/skills/evaluating-skills/evals/evals.json

@@ -0,0 +1,75 @@
+{
+  "skill_name": "evaluating-skills",
+  "evals": [
+    {
+      "id": 1,
+      "prompt": "I want to evaluate whether my creating-intent skill follows Plastic conventions",
+      "expected_output": "The skill should activate and guide the user through convention compliance evaluation: choose eval scope, design test cases using convention-checks reference, run paired evals, write assertions after observing.",
+      "files": [],
+      "assertions": []
+    },
+    {
+      "id": 2,
+      "prompt": "Run evals on the brainstorming skill to see if the description triggers correctly",
+      "expected_output": "The skill should activate and guide through description triggering evaluation: design ~20 queries with should-trigger and near-miss negatives, 60/40 train/validation split, compute trigger rates.",
+      "files": [],
+      "assertions": []
+    },
+    {
+      "id": 3,
+      "prompt": "Create evals for a new skill I just wrote for database migrations",
+      "expected_output": "The skill should activate, help create evals/evals.json in the migration skill directory using the template, guide through choosing eval scope (likely output quality), and design initial test cases.",
+      "files": [],
+      "assertions": []
+    },
+    {
+      "id": 4,
+      "prompt": "Check if my SKILL.md is under the token budget and follows progressive disclosure",
+      "expected_output": "The skill should activate and guide through convention compliance evaluation focused on progressive disclosure checks: body under 500 lines/5000 tokens, references have conditional triggers, description under 1024 chars.",
+      "files": [],
+      "assertions": []
+    },
+    {
+      "id": 5,
+      "prompt": "My skill's description isn't triggering on the right prompts, how do I fix it?",
+      "expected_output": "The skill should activate and guide through description triggering evaluation: create should-trigger and should-not-trigger queries, run paired evals, measure trigger rate, iterate on description wording.",
+      "files": [],
+      "assertions": []
+    },
+    {
+      "id": 6,
+      "prompt": "Write unit tests for my Ruby model that validates email addresses",
+      "expected_output": "The skill should NOT trigger. This is a code testing task, not a skill evaluation task. Near-miss negative — shares 'test' keyword but needs a different skill.",
+      "files": [],
+      "assertions": []
+    },
+    {
+      "id": 7,
+      "prompt": "Fix the bug in my login controller where sessions aren't persisting",
+      "expected_output": "The skill should NOT trigger. This is a debugging task with no relation to skill evaluation.",
+      "files": [],
+      "assertions": []
+    },
+    {
+      "id": 8,
+      "prompt": "Review this pull request for code quality issues",
+      "expected_output": "The skill should NOT trigger. Code review is different from skill evaluation. Near-miss negative — shares 'review/evaluate' concept.",
+      "files": [],
+      "assertions": []
+    },
+    {
+      "id": 9,
+      "prompt": "I updated my skill and want to make sure it still works correctly",
+      "expected_output": "The skill should activate and guide through regression evaluation: run existing evals against the updated skill, compare pass rates with previous version, check for backsliding.",
+      "files": [],
+      "assertions": []
+    },
+    {
+      "id": 10,
+      "prompt": "How do I know if a skill is still needed or if the model has learned the behavior?",
+      "expected_output": "The skill should activate and guide through skill retirement detection: monitor with/without delta over time, if delta approaches zero the model has internalized the skill.",
+      "files": [],
+      "assertions": []
+    }
+  ]
+}

package/skills/evaluating-skills/references/convention-checks.md

@@ -0,0 +1,76 @@
+# Plastic Convention Checks
+
+Assertion library for convention compliance evals on Plastic skills.
+Load when running convention compliance evals on a Plastic skill.
+
+## Directory Structure
+
+| Check | Pass criteria |
+|-------|--------------|
+| SKILL.md exists | File present at skill root |
+| Name matches directory | `name` in frontmatter equals directory name |
+| Standard directories only | Only scripts/, references/, assets/, evals/ at root level (all optional) |
+| References one level deep | No nested directories inside references/ |
+| No orphan files | Every file in the skill dir is referenced from SKILL.md or another skill file |
+| evals/ has evals.json | If evals/ exists, it contains evals.json |
+
+## Frontmatter
+
+| Check | Pass criteria |
+|-------|--------------|
+| `name` present | Non-empty, 1-64 chars |
+| `name` format | Lowercase alphanumeric + hyphens, no leading/trailing/consecutive hyphens |
+| `name` matches dir | Exact match with skill directory name |
+| `description` present | Non-empty, 1-1024 chars |
+| `description` phrasing | Starts with imperative verb or "Use when" pattern |
+| `description` triggers | Mentions at least one trigger context ("Use when...") |
+| `description` edge cases | Includes at least one indirect trigger (user doesn't name the domain) |
+| No unknown required fields | Only uses fields from agentskills.io spec: name, description, license, compatibility, metadata, allowed-tools |
+
+## Progressive Disclosure
+
+| Check | Pass criteria |
+|-------|--------------|
+| Body line count | SKILL.md body (excluding frontmatter) under 500 lines |
+| Body token count | SKILL.md body under 5000 tokens (estimate: lines * 10) |
+| Deep detail in references | Content exceeding activation budget lives in references/ |
+| Conditional reference triggers | Every reference file mentioned in SKILL.md has a "when" condition |
+| No generic references | No "see references/ for more" — each reference has specific load trigger |
+| Description token budget | Description stays under ~100 tokens for discovery stage |
+
+## Content Quality
+
+| Check | Pass criteria |
+|-------|--------------|
+| Gotchas are concrete | Each gotcha is a specific correction, not general advice |
+| Gotchas positioned early | Gotchas section appears before or near the top of procedures |
+| Defaults, not menus | Skill picks one approach; alternatives mentioned briefly if at all |
+| Procedures over declarations | Instructions teach HOW to approach, not WHAT to produce |
+| Reasoning over rigid directives | Uses "Do X because Y" pattern, not "ALWAYS/NEVER" without rationale |
+| No redundant knowledge | Every instruction passes "would the agent get this wrong without it?" |
+| No explaining basics | Does not explain HTTP, JSON, what a migration is, etc. |
+
+## Eval Quality (when evals/ exists)
+
+| Check | Pass criteria |
+|-------|--------------|
+| Standard format | evals.json follows agentskills.io eval structure |
+| Prompts are realistic | Each prompt reads like a real user message |
+| Varied phrasing | Prompts use different wording, detail levels, formality |
+| Near-miss negatives | At least 2 should-not-trigger cases that share keywords |
+| Expected output descriptive | expected_output describes success, not exact text |
+| Assertions (if populated) | Each assertion is specific, verifiable, and countable |
+
+## Intent Structure (when evaluating intent compliance)
+
+| Check | Pass criteria |
+|-------|--------------|
+| Intent file exists | `{ID}--{slug}.md` present in intent directory |
+| Frontmatter complete | id, intent, sources, chain, created, author, tags present |
+| ID format correct | Follows Luhmann alternating: digits and letters alternate |
+| Directory name matches | `{ID}--{slug}` format, 3-5 word slug |
+| Lifecycle artifacts | Present artifacts match the intent's lifecycle stage |
+| spec.md gate | plan.md only exists if spec.md exists |
+| plan.md gate | checklist.md only exists if plan.md exists |
+| outcome.md gate | outcome.md only exists if all checklist items checked |
+| Insights append-only | `## Insights` section only grows, never shrinks |