dsh-compound-engineering 0.0.0-stage → 0.1.1

package/skills/ce-compound-refresh/references/report.md

@@ -0,0 +1,23 @@
+# The refresh report
+
+**Print the full report as markdown — it is the deliverable, not an internal summary.** After processing the scope:
+
+```text
+Compound Refresh Summary
+========================
+Scanned: N learnings
+
+Kept: X
+Updated: Y
+Consolidated: C
+Replaced: Z
+Deleted: W
+Skipped: V
+Marked stale: S
+
+CONCEPTS.md: <scanned, no qualifying terms | created with N entries (M seeded) | updated — N added, N refined, N reconciled, N folded, N retired, N deleted, N scrubbed | repo-wide map created with N entries>
+```
+
+Then, for EVERY file processed: path, classification, evidence found (tag memory-sourced findings "(auto memory [claude])"), and the action taken or recommended; for Consolidate, which doc was canonical, what was merged, what was deleted. Group Keeps under a reviewed-without-edits section.
+
+In non-interactive mode the report is the sole deliverable — self-contained, never abbreviated — and actions split into two sections. **Applied:** writes that succeeded, with the same per-file detail. **Recommended:** writes that failed (with enough context for a human to apply them), plus everything that never runs unattended — relocations that failed the four-condition gate (doc, target, failing condition), splits (doc, proposed fragment boundaries), category-shape observations, potential product regressions where code violates independently supported guidance, guidance files a learning names that contradict it, and the discoverability recommendation if any. If no writes succeed, the report is a maintenance plan. If `_archived/` exists, list its files and recommend disposition (restore, delete, or consolidate).

package/skills/ce-compound-refresh/references/schema.yaml

@@ -0,0 +1,236 @@
+# Documentation schema for learnings written by ce-compound
+# Treat this as the canonical frontmatter contract for compounded learnings.
+#
+# The schema has two tracks based on problem_type:
+#   Bug track  — problem_type is a defect or failure (build_error, test_failure, etc.)
+#   Knowledge track — problem_type is guidance or practice (best_practice, workflow_issue, etc.)
+#
+# Both tracks share the same required core fields. The tracks differ in which
+# additional fields are required vs optional (see track_rules below).
+
+# Contract version for tools that load this file at a pinned ref (compound-cli).
+# Bump only when the file's shape changes, not when values are added.
+schema_version: 1
+
+# --- Track classification ---------------------------------------------------
+tracks:
+  bug:
+    description: "Defects, failures, and errors that were diagnosed and fixed"
+    problem_types:
+      - build_error
+      - test_failure
+      - runtime_error
+      - performance_issue
+      - database_issue
+      - security_issue
+      - ui_bug
+      - integration_issue
+      - logic_error
+  knowledge:
+    description: "Practices, patterns, conventions, decisions, workflow improvements, and documentation"
+    problem_types:
+      - best_practice
+      - documentation_gap
+      - workflow_issue
+      - developer_experience
+      - architecture_pattern
+      - design_pattern
+      - tooling_decision
+      - convention
+
+# --- Fields required by BOTH tracks -----------------------------------------
+required_fields:
+  module:
+    type: string
+    description: "Module or area affected"
+
+  date:
+    type: string
+    pattern: '^\d{4}-\d{2}-\d{2}$'
+    description: "Date documented (YYYY-MM-DD)"
+
+  problem_type:
+    type: enum
+    values:
+      - build_error
+      - test_failure
+      - runtime_error
+      - performance_issue
+      - database_issue
+      - security_issue
+      - ui_bug
+      - integration_issue
+      - logic_error
+      - developer_experience
+      - workflow_issue
+      - best_practice
+      - documentation_gap
+      - architecture_pattern
+      - design_pattern
+      - tooling_decision
+      - convention
+    description: "Primary category — determines track (bug vs knowledge). Prefer the narrowest applicable value; best_practice is the fallback when no narrower knowledge-track value fits."
+
+  component:
+    type: string
+    suggested_values:
+      - data_model
+      - api_layer
+      - service_layer
+      - background_job
+      - database
+      - frontend
+      - messaging
+      - infrastructure
+      - observability
+      - authentication
+      - payments
+      - development_workflow
+      - testing_framework
+      - documentation
+      - tooling
+    description: "Component or area involved. Open vocabulary: choose it per the corpus-first rule in yaml-schema.md (corpus value for this area, most-used spelling when the corpus disagrees, suggested_values only when no doc covers the area; corpus values reused as spelled). A newly chosen suggested_values fallback is lowercase, underscore-separated."
+
+  severity:
+    type: enum
+    values:
+      - critical
+      - high
+      - medium
+      - low
+    description: "Impact severity"
+
+# --- Track-specific rules ----------------------------------------------------
+track_rules:
+  bug:
+    required:
+      symptoms:
+        type: array[string]
+        min_items: 1
+        max_items: 5
+        description: "Observable symptoms such as errors or broken behavior"
+      root_cause:
+        type: string
+        suggested_values:
+          - wrong_api
+          - data_integrity
+          - concurrency
+          - async_timing
+          - memory_leak
+          - config_error
+          - logic_error
+          - test_isolation
+          - missing_validation
+          - missing_permission
+          - missing_workflow_step
+          - inadequate_documentation
+          - missing_tooling
+          - incomplete_setup
+        description: "Fundamental technical cause of the problem. Open vocabulary, corpus-first rule as for component, matched by the cause itself rather than the area."
+      resolution_type:
+        type: enum
+        values:
+          - code_fix
+          - migration
+          - config_change
+          - test_fix
+          - dependency_update
+          - environment_setup
+          - workflow_improvement
+          - documentation_update
+          - tooling_addition
+          - seed_data_update
+        description: "Type of fix applied"
+
+  knowledge:
+    optional:
+      applies_when:
+        type: array[string]
+        max_items: 5
+        description: "Conditions or situations where this guidance applies"
+      symptoms:
+        type: array[string]
+        max_items: 5
+        description: "Observable gaps or friction that prompted this guidance (optional for knowledge track)"
+      root_cause:
+        type: string
+        suggested_values:
+          - wrong_api
+          - data_integrity
+          - concurrency
+          - async_timing
+          - memory_leak
+          - config_error
+          - logic_error
+          - test_isolation
+          - missing_validation
+          - missing_permission
+          - missing_workflow_step
+          - inadequate_documentation
+          - missing_tooling
+          - incomplete_setup
+        description: "Underlying cause, if there is a specific one (optional for knowledge track). Open vocabulary, corpus-first rule as for component, matched by the cause itself rather than the area."
+      resolution_type:
+        type: enum
+        values:
+          - code_fix
+          - migration
+          - config_change
+          - test_fix
+          - dependency_update
+          - environment_setup
+          - workflow_improvement
+          - documentation_update
+          - tooling_addition
+          - seed_data_update
+        description: "Type of change, if applicable (optional for knowledge track)"
+
+# --- Fields optional for BOTH tracks ----------------------------------------
+optional_fields:
+  related_components:
+    type: array[string]
+    description: "Other components involved"
+
+  tags:
+    type: array[string]
+    max_items: 8
+    description: "Search keywords, lowercase and hyphen-separated"
+
+  retire_when:
+    type: string
+    description: "The change outside the repo that would retire this learning, and how a refresh can check for it without changing the repo, e.g. \"example-org/sdk#1234 closed as fixed in a released sdk version; check the issue and the sdk release notes\". Set it only when the guidance holds just while something outside the repo holds: an open upstream bug, a tool or platform version, a dependency's behavior. Omit it otherwise. A verification that needs a repo change, such as removing a workaround and rerunning CI, belongs in the body as the step before removal."
+
+# --- Fields optional for bug track only -------------------------------------
+bug_optional_fields:
+  framework_version:
+    type: string
+    description: "Framework or runtime name and version the bug was observed on, e.g. \"rails 7.1.2\" or \"node 22.4.0\". Only relevant for bug-track docs."
+
+# --- Backward compatibility --------------------------------------------------
+# Docs created before the track system was introduced may have bug-track
+# fields (symptoms, root_cause, resolution_type) on knowledge-type
+# problem_types. These are valid legacy docs:
+#   - Bug-track fields present on a knowledge-track doc are harmless. Do not
+#     strip them during refresh unless the doc is being rewritten for other reasons.
+#   - When creating NEW docs, follow the track rules above.
+# Docs written before component/root_cause became open vocabulary may carry
+# values from the earlier closed list (rails_*, hotwire_turbo, missing_include,
+# ...) or a rails_version field. They stay valid; the corpus-first rule means a
+# repo whose docs consistently use those values keeps using them.
+
+# --- Validation rules --------------------------------------------------------
+validation_rules:
+  - "Determine track from problem_type using the tracks section above"
+  - "All shared required_fields must be present"
+  - "Bug-track required fields (symptoms, root_cause, resolution_type) must be present on bug-track docs"
+  - "Knowledge-track docs have no additional required fields beyond the shared ones"
+  - "Bug-track fields on existing knowledge-track docs are harmless (see backward compatibility note)"
+  - "Track-specific optional fields may be included but are not required"
+  - "Enum fields (problem_type, severity, resolution_type) must match allowed values exactly"
+  - "Open-vocabulary fields (component, root_cause) follow the corpus-first rule in yaml-schema.md: component is matched by area (the value existing docs under <root>/solutions/ use for that area, most-used spelling when existing docs disagree), root_cause is matched by the cause itself (the value existing docs use for that same cause, most-used spelling when existing docs disagree), suggested_values only when no existing doc covers the area (component) or names the cause (root_cause). Do not coin a near-synonym of a value the corpus already uses"
+  - "Array fields must respect min_items/max_items when specified"
+  - "date must match YYYY-MM-DD format"
+  - "framework_version, if provided, only applies to bug-track docs"
+  - "retire_when is optional on both tracks; when present it is one string, and a doc without it stays valid"
+  - "tags should be lowercase and hyphen-separated"
+  - "Array-of-strings frontmatter items (symptoms, applies_when, tags, related_components, or any future array field) must be wrapped in double quotes when the value starts with a YAML reserved indicator (`, [, *, &, !, |, >, %, @, ?) or contains the substring `: ` — otherwise strict YAML parsers reject the file"

package/skills/ce-compound-refresh/references/scope.md

@@ -0,0 +1,18 @@
+# Scope selection and triage
+
+Find all `.md` files under `<root>/solutions/`, excluding `README.md` files and anything under `_archived/` (if `_archived/` exists, flag it in the report as legacy to clean up). READMEs are excluded as review *candidates* only: whenever an action deletes, renames, moves, consolidates, or replaces a doc a catalog README lists, update that README's rows mechanically as part of the action's cleanup.
+
+Words that describe the run rather than name a doc, category, module, or keyword are not a scope hint: an operation word (`refresh`, `audit`, `check`), a worth-lens intent (SKILL.md, Worth lens), or a mode token. Remove them first; the remainder is the hint, and an empty remainder means the whole store.
+
+If a scope argument was provided, narrow with the first strategy that produces results: subdirectory name → frontmatter (`module`/`component`/`tags`) → filename → content keyword. No matches: ask the user to clarify (interactive) or report the miss and exit (non-interactive).
+
+If the store is empty, report:
+
+```text
+No candidate docs found in <root>/solutions/.
+Run /ce-compound after solving problems to start building your knowledge base.
+```
+
+For a broad sweep (9+ docs), triage before deep investigation: read all frontmatter, cluster by module/category, spot-check whether primary referenced files still exist, and start with the highest-impact cluster (interactive: confirm the starting area with the user; non-interactive: process all clusters in impact order). Review individual learning docs before the pattern docs that depend on them — stale learnings make a pattern look more valid than it is. If the user named a pattern doc, you may start there, but inspect its supporting learnings before changing it.
+
+Render that last line for the active harness: `/ce-compound` by default, `$ce-compound` on Codex or another host documenting dollar-prefixed skill invocation. Print one form only.

package/skills/ce-compound-refresh/references/worth-audit.md

@@ -0,0 +1,35 @@
+# Worth audit
+
+Read this only after the user confirmed the worth lens (SKILL.md, Worth lens). It adds one test to classification; the five outcomes, the investigation roles, and the per-action flows are unchanged.
+
+## The bar
+
+<!-- ce-durable-bar:start -->
+A learning earns its place only when it holds durable project reasoning that is not readily recoverable from the final code, tests, types, comments, or existing documentation, and losing it would plausibly cause recurrence, material risk, or substantial rediscovery. Apply this counterfactual: if the learning document disappeared, would a future engineer reading the final implementation still be likely to repeat the mistake or redo substantial investigation? Completion, effort, and diff size do not establish eligibility.
+<!-- ce-durable-bar:end -->
+
+This is the bar `ce-compound` applies before writing a learning. The worth lens applies it to learnings that were written before the bar existed, or that accreted past it.
+
+## Recoverability needs positive evidence
+
+A doc's reasoning is recoverable only when a named in-repo artifact states it in its own text: a test whose assertion or comment carries the rule, a code comment beside the mechanism, the project's always-loaded instructions, a skill reference, or another surviving learning. Topical overlap is not coverage; verify the specific reasoning is present and quote it. The unverifiable-is-not-false rule in `references/classify.md` still holds: a claim no artifact corroborates is kept, never treated as recoverable. A doc describing a mechanism that no longer exists is judged by the ordinary Delete gate, not by this lens.
+
+Add this clause to every investigation subagent prompt on this path, verbatim:
+
+> For each claim the learning makes that a reader would need in order to avoid repeating a mistake or redoing an investigation, search the repository for an in-repo artifact whose own text states that reasoning (a test, a code comment, the root agent-instructions file, a skill reference, another learning). Return, per claim: recoverable with the artifact path and the quoted sentence, or not recoverable. Do not infer coverage from a related file name or topic.
+
+## Routing
+
+Apply the accuracy classification first, then the worth test to docs that survived it:
+
+- **Every claim recoverable, citations absent or decorative:** Delete, with the citation cleanup the Delete flow already owns. The quoted artifacts are the doc's evidence bullets.
+- **Some claims recoverable:** Update. Cut the recoverable content and keep the measured facts, cross-file invariants, and rejected alternatives nothing else records. Point at the artifact in one line where a reader would otherwise look for the cut material. Narrow the frontmatter to match.
+- **Nothing recoverable:** Keep.
+
+A doc whose only remaining content is a session narrative, a list of what was changed, or a restatement of a contract another file owns has no non-recoverable claim and routes to Delete.
+
+**Interactive:** the run-start confirmation authorized the lens. Batch broad sweeps and confirm continuation between batches as `references/classify.md` already directs; present each worth-based Delete or Update with its quoted artifacts as the evidence bullets. **Non-interactive:** worth-based Delete and Update verdicts are never applied; record each under **Recommended** with the quoted artifacts so a human can apply the batch. Accuracy-based actions apply as usual.
+
+## Report
+
+The summary block carries one line, `Worth lens: confirmed | recommended-only (non-interactive) | off`, and each worth-based verdict lists the recovering artifacts it rests on.

package/skills/ce-compound-refresh/references/yaml-schema.md

@@ -0,0 +1,130 @@
+# YAML Frontmatter Schema
+
+`schema.yaml` in this directory is the canonical contract for `<root>/solutions/` frontmatter written by `ce-compound`.
+
+Use this file as the quick reference for:
+- required fields
+- enum values and open-vocabulary defaults
+- validation expectations
+- category mapping
+- track classification (bug vs knowledge)
+
+## Corpus-First Vocabulary
+
+`component` and `root_cause` are open vocabulary, and the category directories are a default layout, not a fixed one. Repos that already hold learnings usually have their own self-consistent values and directory names, and their retrieval (agent instructions, greps, tooling) is keyed on those. A doc written to this file's defaults instead of the repo's vocabulary is a doc the repo cannot find, which defeats the point of writing it.
+
+So, before classifying: when `<root>/solutions/` already contains docs, sample their frontmatter (`component`, `root_cause`, `problem_type`, `tags`) and directory names for this area. For `component`, use the value the corpus's existing docs use for this area (the component/directory grouping this change belongs to). For `root_cause`, match by the cause itself rather than the area: use the value existing docs use for this same underlying cause, wherever in the corpus those docs live — an area's typical root_cause does not carry over to a doc whose verified cause is different. For either field, when the corpus has more than one spelling for the same area (component) or the same cause (root_cause), use the spelling the most docs use (tie: the most recently dated doc); use the schema's suggested default only when no existing doc covers the area (component) or names the cause (root_cause). Reuse corpus values as spelled — do not coin a near-synonym of a value the corpus already uses. Directory choice is a separate condition: file the doc in the existing directory that covers this area; use the Category Mapping below only when no existing directory covers it. `problem_type`, `severity`, and `resolution_type` remain closed enums (`problem_type` drives track selection).
+
+## Tracks
+
+The `problem_type` determines which **track** applies. Each track has different required and optional fields.
+
+| Track | problem_types | Description |
+|-------|--------------|-------------|
+| **Bug** | `build_error`, `test_failure`, `runtime_error`, `performance_issue`, `database_issue`, `security_issue`, `ui_bug`, `integration_issue`, `logic_error` | Defects and failures that were diagnosed and fixed |
+| **Knowledge** | `best_practice`, `documentation_gap`, `workflow_issue`, `developer_experience`, `architecture_pattern`, `design_pattern`, `tooling_decision`, `convention` | Practices, patterns, conventions, decisions, workflow improvements, and documentation. Prefer the narrowest applicable value; `best_practice` is the fallback. |
+
+## Required Fields (both tracks)
+
+- **module**: Module or area affected
+- **date**: ISO date in `YYYY-MM-DD`
+- **problem_type**: One of the values listed in the Tracks table above
+- **component**: Component or area involved — open vocabulary (see Corpus-First Vocabulary). Suggested defaults when the corpus has no value for the area: `data_model`, `api_layer`, `service_layer`, `background_job`, `database`, `frontend`, `messaging`, `infrastructure`, `observability`, `authentication`, `payments`, `development_workflow`, `testing_framework`, `documentation`, `tooling`
+- **severity**: One of `critical`, `high`, `medium`, `low`
+
+## Bug Track Fields
+
+Required:
+- **symptoms**: YAML array with 1-5 observable symptoms (errors, broken behavior)
+- **root_cause**: Fundamental technical cause — open vocabulary (see Corpus-First Vocabulary). Suggested defaults: `wrong_api`, `data_integrity`, `concurrency`, `async_timing`, `memory_leak`, `config_error`, `logic_error`, `test_isolation`, `missing_validation`, `missing_permission`, `missing_workflow_step`, `inadequate_documentation`, `missing_tooling`, `incomplete_setup`
+- **resolution_type**: One of `code_fix`, `migration`, `config_change`, `test_fix`, `dependency_update`, `environment_setup`, `workflow_improvement`, `documentation_update`, `tooling_addition`, `seed_data_update`
+
+## Knowledge Track Fields
+
+No additional required fields beyond the shared ones. All fields below are optional:
+
+- **applies_when**: Conditions or situations where this guidance applies
+- **symptoms**: Observable gaps or friction that prompted this guidance
+- **root_cause**: Underlying cause, if there is a specific one
+- **resolution_type**: Type of change, if applicable
+
+## Optional Fields (both tracks)
+
+- **related_components**: Other components involved
+- **tags**: Search keywords, lowercase and hyphen-separated
+- **retire_when**: The change outside the repo that would retire this learning, and how a refresh can check for it without changing the repo. Set it only when the guidance holds just while something outside the repo holds: an open upstream bug, a tool or platform version, a dependency's behavior. Omit it otherwise. A verification that needs a repo change, such as removing a workaround and rerunning CI, belongs in the body as the step before removal. It is one string; quote the whole value when it contains ` #` or `: `.
+
+## Optional Fields (bug track only)
+
+- **framework_version**: Framework or runtime name and version the bug was observed on, e.g. `rails 7.1.2` or `node 22.4.0`
+
+## Backward Compatibility
+
+Docs created before the track system may have `symptoms`/`root_cause`/`resolution_type` on knowledge-type problem_types. These are valid legacy docs:
+
+- Bug-track fields present on a knowledge-track doc are harmless. Do not strip them during refresh unless the doc is being rewritten for other reasons.
+- Docs written before `component`/`root_cause` became open vocabulary may carry values from the earlier closed list or the earlier `rails_version` field (now `framework_version`). They stay valid, and a corpus that consistently uses them keeps using them.
+- When creating **new** docs, follow the track rules above.
+
+## Category Mapping
+
+Default layout for a repo with no existing learnings. When `<root>/solutions/` already has an established directory taxonomy, place the doc in the existing directory that covers this area (the corpus-first rule above) rather than creating a new directory from this table that nothing else uses.
+
+- `build_error` -> `<root>/solutions/build-errors/`
+- `test_failure` -> `<root>/solutions/test-failures/`
+- `runtime_error` -> `<root>/solutions/runtime-errors/`
+- `performance_issue` -> `<root>/solutions/performance-issues/`
+- `database_issue` -> `<root>/solutions/database-issues/`
+- `security_issue` -> `<root>/solutions/security-issues/`
+- `ui_bug` -> `<root>/solutions/ui-bugs/`
+- `integration_issue` -> `<root>/solutions/integration-issues/`
+- `logic_error` -> `<root>/solutions/logic-errors/`
+- `developer_experience` -> `<root>/solutions/developer-experience/`
+- `workflow_issue` -> `<root>/solutions/workflow-issues/`
+- `best_practice` -> `<root>/solutions/best-practices/`
+- `documentation_gap` -> `<root>/solutions/documentation-gaps/`
+- `architecture_pattern` -> `<root>/solutions/architecture-patterns/`
+- `design_pattern` -> `<root>/solutions/design-patterns/`
+- `tooling_decision` -> `<root>/solutions/tooling-decisions/`
+- `convention` -> `<root>/solutions/conventions/`
+
+## Validation Rules
+
+1. Determine the track from `problem_type` using the Tracks table.
+2. All shared required fields must be present.
+3. Bug-track required fields (`symptoms`, `root_cause`, `resolution_type`) must be present on bug-track docs.
+4. Knowledge-track docs have no additional required fields beyond the shared ones.
+5. Bug-track fields on existing knowledge-track docs are harmless (see Backward Compatibility).
+6. Enum fields (`problem_type`, `severity`, `resolution_type`) must match the allowed values exactly.
+7. Open-vocabulary fields (`component`, `root_cause`) follow the Corpus-First Vocabulary rule above.
+8. Array fields must respect min/max item counts.
+9. `date` must match `YYYY-MM-DD`.
+10. `framework_version`, if present, only applies to bug-track docs.
+11. `retire_when` is optional on both tracks. When present it is one string, and a doc without it stays valid.
+
+## YAML Safety Rules
+
+Strict YAML 1.2 parsers (`yq`, `js-yaml` strict, PyYAML) reject array items
+that start with a reserved indicator character as unquoted scalars. When
+writing items for any array-of-strings field (`symptoms`, `applies_when`,
+`tags`, `related_components`, or any future array field), wrap the value in
+double quotes if it starts with any of:
+
+`` ` ``, `[`, `*`, `&`, `!`, `|`, `>`, `%`, `@`, `?`
+
+Also quote if the value contains the substring `": "` — that punctuation
+confuses flow-style parsers.
+
+Example — before (breaks strict YAML):
+
+    symptoms:
+      - `sudo dscacheutil -flushcache` does not restore in-container mDNS
+
+Example — after (parses cleanly):
+
+    symptoms:
+      - "`sudo dscacheutil -flushcache` does not restore in-container mDNS"
+
+This rule applies to all array-of-strings frontmatter fields. Scalar string
+fields like `description:` have their own quoting rules (see plugin
+`AGENTS.md` under "YAML Frontmatter").