opensdd 0.1.0

package/opensdd/sdd-generate.md

@@ -0,0 +1,209 @@
+# SDD Generate
+
+> Guides an AI agent through analyzing a repository and generating a behavioral spec in the OpenSDD format.
+
+## Overview
+
+Handles large repos via a multi-pass, artifact-driven strategy that survives context window clears. This document defines the workflow that the sdd-generate skill teaches agents to follow. The skill is installed into each supported coding agent's configuration directory via `opensdd init`.
+
+## Prerequisites
+
+Before starting, you need:
+
+1. A **target repository** — a GitHub URL or local path provided by the user.
+2. A **scope** — which capability, module, or function to spec. If the user provides a whole-repo URL without scoping, ask them to narrow it before proceeding. A spec for "the entire lodash library" is not useful. A spec for "lodash's string slugification" is.
+3. The **spec-format reference** — read [spec-format.md](spec-format.md) to understand the required output structure.
+4. A **working directory** — confirm with the user where the generated spec should be written. If the project has `opensdd.json`, use the directory specified by `specs_dir` (default: `opensdd/`). If the project is not yet initialized, ask the user where to output the spec — a common default is `opensdd/` in the current project root. The agent does not need `opensdd.json` to exist before generating a spec.
+
+## Output
+
+A spec written to the working directory:
+
+```
+opensdd/
+  spec.md           # Behavioral contract
+```
+
+The spec author MAY include additional files in the output directory (e.g., supplementary schema definitions in a `references/` subdirectory). The protocol does not mandate any particular structure beyond `spec.md` — organization of supplementary files is at the author's discretion. If you create supplementary files, use relative markdown hyperlinks from `spec.md` to reference them so the spec is self-navigating — an agent starts at `spec.md` and follows links to discover everything it needs.
+
+If `opensdd.json` exists, add or update the `publish` object:
+
+```json
+{
+  "publish": {
+    "name": "{name}",
+    "version": "{semver}",
+    "description": "{one-line description}",
+    "spec_format": "0.1.0",
+    "dependencies": []
+  }
+}
+```
+
+## Strategy: Multi-Pass Artifact-Driven Generation
+
+Large repositories exceed a single context window. The strategy is to **write to disk as you go** — not at the end. Each pass reads from the repo and writes to the output spec directory or to scratch notes. If context clears, read your own artifacts to resume.
+
+### Working artifacts
+
+Maintain a `_notes/` scratch directory alongside the output spec in your working directory during generation:
+
+```
+{working-directory}/
+  _notes/
+    scope.md        # What we're spec'ing, which files matter
+    inventory.md    # Public API surface, function signatures, types
+    examples.md     # Extracted inline examples from tests
+    gaps.md         # Behaviors not yet covered, open questions
+  spec.md           # Built up incrementally
+```
+
+`_notes/` is your working memory across context clears. Delete it before finalizing.
+
+### Context recovery
+
+If you are resuming after a context clear:
+
+1. Read `_notes/scope.md` to understand what you're spec'ing and where you are.
+2. Read `_notes/gaps.md` to see what remains.
+3. Read the current `spec.md` draft to see what's been written.
+4. Continue from the next incomplete pass.
+
+Always update `_notes/gaps.md` at the end of each pass with what still needs to be done, so a fresh context can pick up cleanly.
+
+### Pass 1: Reconnaissance
+
+**Goal:** Understand what the repo does and map the territory for the scoped capability.
+
+**Read (in this order):**
+- README and top-level documentation
+- Package manifest (package.json, pyproject.toml, Cargo.toml, go.mod) for dependencies, version, description
+- Directory structure (just the tree, not file contents)
+- Changelog or release notes if present (for understanding version history)
+
+**Write:**
+- `_notes/scope.md` — what the repo is, what capability we're spec'ing, which directories/files are relevant to that capability, which are not. Include the spec name, version (use the repo's version or `0.1.0`), and description for later use in the `opensdd.json` `publish` object.
+- `opensdd/spec.md` — Header (H1 + blockquote) and `## Overview` only
+
+**Do NOT read** source code or tests yet. The goal is orientation, not comprehension.
+
+### Pass 2: Surface Area
+
+**Goal:** Identify the public API — what the outside world interacts with.
+
+**Read (in this order):**
+- Type definitions, interfaces, and exported symbols for the scoped capability
+- Public function/method signatures with their parameter and return types
+- Configuration types, options objects, builder patterns
+- If the repo has OpenAPI, protobuf, GraphQL, or JSON Schema files relevant to the scope, read those
+
+**Write:**
+- `_notes/inventory.md` — complete list of public API surface: every exported function, class, type, constant. Include signatures. This is your checklist — every item here must appear in the spec's behavioral contract.
+- `spec.md` — add `## Behavioral Contract` skeleton with H3 subsection headers for each logical grouping of API surface. No behavioral descriptions yet, just the structure.
+- If formal schema files are found (OpenAPI, protobuf, JSON Schema, etc.), consider copying them into the spec directory as supplementary reference files (e.g., in a `references/` subdirectory).
+
+### Pass 3: Behavior Extraction
+
+**Goal:** Extract concrete behaviors from tests. Tests are the richest source of behavioral truth — they show what the code actually does with real inputs and outputs.
+
+**Read:**
+- Test files for the scoped capability, starting with unit tests, then integration tests
+- For each test, identify: what input is given, what output or behavior is asserted, what edge case is being covered
+
+**Write:**
+- `_notes/examples.md` — every concrete input/output pair and scenario extracted from tests, grouped by the behavioral subsection it belongs to
+- `spec.md` — fill in Behavioral Contract subsections with:
+  - Behavioral descriptions (what happens, not how)
+  - Inline examples extracted from tests (converted to spec format)
+  - `## Edge Cases` section with concrete examples from edge case tests
+- `_notes/gaps.md` — which inventory items have no test coverage, which behavioral subsections still need examples
+
+**Guidance on extracting behavior from tests:**
+- A test assertion like `expect(slugify("Hello World")).toBe("hello-world")` becomes the inline example `slugify("Hello World")` MUST return `"hello-world"`
+- A test that mocks internal implementation details is telling you about a dependency boundary, not a behavior to spec
+- A test that checks error throwing tells you about an error path to include
+- Property-based tests translate directly into Invariants
+- Parameterized tests are dense sources of edge cases
+
+### Pass 4: Gap Filling
+
+**Goal:** Cover behaviors not captured by tests. Read source code only now, and only for gaps.
+
+**Read:**
+- Source code for behaviors listed in `_notes/gaps.md`
+- Error handling paths (try/catch, error returns, validation logic)
+- Default values and configuration handling
+- Any inline comments describing "why" (these often reveal behavioral intent)
+
+**Write:**
+- `spec.md` — fill remaining Behavioral Contract gaps, add any discovered edge cases, add `## Options / Configuration` if the capability has configurable parameters
+- `_notes/gaps.md` — update with any remaining unknowns
+
+**Guidance on separating what from how:**
+- Source says `str.toLowerCase().replace(/[^a-z0-9]+/g, '-').replace(/(^-|-$)/g, '')` — spec says "converts to lowercase, replaces non-alphanumeric characters with hyphens, strips leading/trailing hyphens"
+- Source uses a trie for prefix matching — spec says "matches the longest matching prefix"
+- Source caches results in a Map — this is implementation detail, do NOT spec it (mention in NOT Specified or Implementation Hints)
+- Source validates input with a regex — spec says "MUST reject inputs not matching pattern X" (the constraint matters, the mechanism doesn't)
+
+### Pass 5: Completion
+
+**Goal:** Fill in recommended sections, self-validate, finalize.
+
+**Read:**
+- The full `spec.md` draft
+- `_notes/inventory.md` — verify every public API item is covered
+- `_notes/gaps.md` — verify no critical gaps remain
+- [spec-format.md](spec-format.md) — verify structural compliance
+
+**Write:**
+- `spec.md` — add or complete:
+  - `## NOT Specified (Implementation Freedom)` — list implementation choices observed in the source that the spec intentionally leaves open (data structures, algorithms, caching, internal architecture)
+  - `## Invariants` — universal properties extracted from property tests, type constraints, or behavioral patterns (e.g., idempotency, commutativity, output format guarantees)
+  - `## Implementation Hints` (optional) — guidance on performance, concurrency, or common pitfalls observed in the source, only if genuinely useful. If the spec targets a specific platform, hints MAY be platform-specific; otherwise they SHOULD be language-agnostic.
+- If `opensdd.json` exists, add or update the `publish` object with name, version, description, spec_format, and dependencies from `_notes/scope.md`.
+
+**Validate the output:**
+- MUST have H1 header with blockquote summary and `## Behavioral Contract`
+- Behavioral Contract subsections MAY include inline examples or narrative scenarios where the behavior benefits from concrete demonstration
+- If an Edge Cases section is present, it SHOULD have concrete examples (not just descriptions)
+- NOT Specified and Invariants sections SHOULD be present — they are recommended but not required
+- Invariants SHOULD be expressed as testable assertions when present
+- No implementation details (specific algorithms, data structures, internal architecture) in the Behavioral Contract
+
+**Clean up:**
+- Delete `_notes/` directory entirely
+- Verify the output directory is a valid spec: contains `spec.md`
+
+## Specs with Subcomponents
+
+If the scoped capability has natural subcomponents (e.g., multiple provider integrations, platform adapters), consider organizing the spec with supplementary files for each component. For example:
+
+```
+opensdd/
+  spec.md              # Shared behavioral contract
+  components/
+    {component-a}.md   # Component-specific contract
+    {component-b}.md
+```
+
+This is an organizational pattern, not a protocol requirement — the protocol only mandates `spec.md`. Choose whatever file organization makes the spec easiest to read and maintain. `spec.md` MUST link to all component files using relative markdown hyperlinks so the spec is self-navigating.
+
+Run passes 2-4 for the shared contract first, then for each component. The shared `spec.md` should capture behavior common to all components. Component-level files capture only what differs.
+
+Update `_notes/scope.md` to track which components have been completed so context clears don't cause rework.
+
+## Handling Very Large Repos
+
+For repos where even a single pass exceeds context:
+
+- **Narrow the scope further.** Ask the user to target a specific module or subdirectory rather than a broad capability.
+- **Split passes into sub-passes.** For Pass 3 (behavior extraction), process one test file at a time, writing to `_notes/examples.md` after each. The notes file accumulates across sub-passes.
+- **Use `_notes/gaps.md` as a work queue.** Write remaining file paths to process. Pick up the next unprocessed file after each context clear.
+- **Prioritize depth over breadth.** A thorough spec of a smaller scope is more valuable than a shallow spec of a large scope.
+
+## What This Skill Does NOT Do
+
+- Publish specs to a registry (use `opensdd publish` to validate and publish)
+- Generate implementations from specs (that's the sdd-manager skill)
+- Modify the source repository in any way
+- Make judgments about code quality — the spec captures behavior as-is, not as it should be

package/opensdd/sdd-manager.md

@@ -0,0 +1,134 @@
+# SDD Manager
+
+> Teaches agents how to implement, update, and verify installed dependency specs in an OpenSDD-compliant project.
+
+## Overview
+
+The sdd-manager skill is installed once per project via `opensdd init` alongside the sdd-generate skill, into each supported coding agent's configuration directory. It teaches agents four workflows: implementing a spec, processing a spec update, checking conformance, and creating deviations. It also defines universal implementation defaults, the project conventions check, and the verification protocol that apply to all spec implementations.
+
+This skill is the required entry point whenever an agent reads an installed OpenSDD dependency spec to make changes to the project — whether that is a first implementation, an incremental update, a conformance check, or a deviation. The agent MUST NOT implement or modify code based on an OpenSDD spec outside of the workflows defined here.
+
+## Spec as Source of Truth
+
+The dependency spec (`spec.md`) is the authoritative description of what to build. It is already a carefully structured behavioral contract with precise language, edge cases, and constraints. The agent MUST treat it as the primary reference throughout all workflows and MUST NOT replace it with a self-generated substitute.
+
+**Do not rewrite the spec into a plan.** The agent MUST NOT translate spec requirements into its own planning format (todo lists, step-by-step plans, internal summaries, etc.) as a substitute for the spec itself. Such translations are inherently lossy — they flatten nuance, drop edge cases, and shift intent. The spec's behavioral contract already defines what to build; duplicating it in another format adds no value and introduces drift.
+
+**Re-read the spec directly.** During implementation, the agent MUST re-read relevant sections of `spec.md` directly rather than working from a self-generated summary or plan. If the agent's context window requires chunking work across multiple passes, it MUST chunk by spec section and re-read each section from the file before implementing it — not from memory or prior notes.
+
+**Plans are for additive context only.** If the agent uses planning tools (todo lists, scratchpads, plan mode, etc.), those plans MUST be limited to information that is _not_ in the spec: project-specific decisions (file paths, module structure, integration points), target language and framework details, implementation ordering, and deviations. Plans SHOULD reference spec sections by name rather than restating their content.
+
+## Workflows
+
+### Implement
+
+1. **Read:** Read the dependency spec from `.opensdd.deps/<name>/` (`spec.md` and any supplementary files), `deviations.md` (if it exists), and other dependency specs it depends on.
+
+2. **Clarify (pre-implementation Q&A):** Before writing any code, the agent MUST walk the user through the spec's scope and structure and solicit clarifications. The agent MUST:
+   - Present the spec's behavioral sections and their scope (key behaviors, components, options) — referencing the spec's own structure rather than generating a lossy re-summary.
+   - List any ambiguities, underspecified areas, or decisions that require user input (e.g., where to place files, which optional behaviors to include, how to integrate with existing code).
+   - Ask whether the user wants to deviate from any behaviors upfront. Present the spec's major behavioral groups and ask: "Do you want to skip, modify, or narrow any of these?" If the user identifies deviations, create `deviations.md` entries before proceeding to implementation.
+   - The agent MUST NOT proceed to implementation until the user has responded. A response of "no clarifications, proceed" or equivalent is sufficient.
+
+3. **Conventions check:** Perform the project conventions check (see Project Conventions Check section below).
+
+4. **Implement:** Generate the implementation applying universal defaults (see Universal Implementation Defaults section below).
+
+5. **Verify:** Execute the full verification protocol (see Verification Protocol section below): generate test suite → run tests until all pass (or SHOULD bail after 50 attempts) → dispatch subagent for spec compliance audit → fix any findings → re-run tests.
+
+6. **Record:** Update `opensdd.json` `dependencies` entry with `implementation` path, `tests` path, and `has_deviations` if applicable.
+
+7. **Report:** Report results with spec coverage summary.
+
+### Update
+
+1. Read `changeset.md` from `.opensdd.deps/.updates/<name>/` → identify which behavioral sections changed.
+2. Read `deviations.md` (if it exists in `.opensdd.deps/<name>/`) and flag any deviations that reference changed sections as potentially stale.
+3. Present the changes to the user: summarize what changed, flag stale deviations, and ask whether the user wants to adjust any deviations before proceeding.
+4. Patch implementation to conform to the new behavioral contract.
+5. Regenerate affected tests → run until all pass → dispatch subagent for spec compliance audit scoped to the changed sections → fix any findings → re-run tests.
+6. Tell the user to run `opensdd update apply <name>` to finalize the update in `opensdd.json`. The agent MUST always specify the spec name explicitly — it MUST NOT suggest or use the no-args batch form (`opensdd update apply` without a name).
+
+### Check Conformance
+
+Run existing test suite → report pass/fail. If test suite is missing or stale, regenerate from spec and re-run. After tests pass, dispatch subagent for spec compliance audit → report any compliance issues found.
+
+### Create Deviation
+
+Determine affected spec section → classify type → create/append to `deviations.md` in `.opensdd.deps/<name>/` → update test suite to skip affected tests → update `has_deviations` in `opensdd.json`.
+
+## Universal Implementation Defaults
+
+Quality floors that apply to every spec implementation, regardless of language or project. These exist to maximize the chance of a correct implementation on the first attempt. The sdd-manager skill MUST instruct the agent to follow these defaults.
+
+**Typing and type safety:**
+- The agent MUST use the strongest type system available in the target language. In Python, this means type annotations with strict mypy-compatible types. In JavaScript projects, the agent SHOULD prefer TypeScript if the project supports it.
+- All public function signatures MUST have fully typed parameters and return types.
+- The agent SHOULD use narrow types over broad ones (e.g., `str` over `Any`, specific union types over generic ones).
+- The agent MUST NOT use type suppression features (`# type: ignore`, `@ts-ignore`, `// nolint`, `as any`, etc.) unless there is no type-safe alternative. If used, the agent MUST include a comment explaining why.
+
+**Error handling:**
+- All error paths defined in the spec MUST be handled explicitly. The agent MUST NOT silently swallow errors.
+- Errors SHOULD use the language's idiomatic error mechanism (exceptions in Python/JS/Java, Result types in Rust, error returns in Go).
+- Public boundary inputs MUST be validated. Internal calls between trusted functions MAY skip validation.
+
+**Code structure:**
+- The agent SHOULD prefer pure functions where the spec does not require state or side effects.
+- The agent MUST NOT introduce global mutable state unless the spec explicitly requires it.
+- The implementation SHOULD be contained in as few files as makes sense for the project's conventions.
+
+**Defensiveness:**
+- The agent MUST handle null/nil/undefined inputs gracefully when the spec defines edge case behavior for them.
+- The agent SHOULD prefer standard library over third-party dependencies where both satisfy the spec equivalently.
+- The agent MUST NOT introduce dependencies not already present in the project without flagging this to the user.
+
+## Project Conventions Check
+
+Before implementing a spec, the agent MUST determine the target language and check whether the project has sufficient coding conventions defined (in CLAUDE.md, cursor rules, AGENTS.md, or equivalent): target language/version, code style, module organization, testing framework, and error handling patterns.
+
+If the spec declares a target language, runtime, or framework, the agent MUST use that as the starting point. If the spec does not declare a target platform, the agent infers it from the project context.
+
+If the project lacks clear conventions, the agent MUST prompt the user to specify preferences or explicitly opt for best-judgment inference. The agent MUST NOT proceed until the user responds.
+
+## Verification Protocol
+
+The agent MUST verify its implementation through two complementary methods: a generated test suite and a spec compliance audit performed by a subagent.
+
+### Test Suite
+
+The agent MUST generate a persisted test file in the project's testing framework that thoroughly covers all behaviors described in the spec, including edge cases, error conditions, and invariants. The spec itself defines what constitutes thorough coverage — the agent should use its judgment to ensure all described behaviors are tested.
+
+The agent MUST run the tests after implementation, fix and re-run until all pass, and report spec coverage. The agent SHOULD abandon the fix-and-rerun cycle after 50 attempts and report the remaining failures to the user. The test file path MUST be tracked in `opensdd.json` under the dependency's `tests` field and be runnable via the project's standard test command.
+
+When `deviations.md` exists, the agent MUST skip tests for deviated behaviors and note the skips in the test file referencing the deviation.
+
+### Spec Compliance Audit
+
+After the test suite passes, the agent MUST dispatch a subagent to perform an independent spec compliance audit. The audit catches classes of issues that tests alone miss — incorrect ordering of operations, missing guards, wrong defaults, incomplete error handling, and subtle deviations from the spec's behavioral contract.
+
+The agent MUST dispatch the subagent with these instructions:
+
+1. **Read the full spec.** Read `spec.md` and all supplementary files linked from it. Read `deviations.md` if it exists.
+2. **Read the full implementation.** Read every implementation file produced for this spec.
+3. **Walk each behavioral contract section.** For each section in the spec's `## Behavioral Contract`, verify:
+   - Every MUST requirement is satisfied in the implementation
+   - Every SHOULD requirement is either satisfied or has a justified reason for omission
+   - The ordering of operations matches the spec when the spec prescribes a specific order
+   - Error handling matches the spec's defined error paths and exit conditions
+   - Edge cases enumerated in the spec are handled
+4. **Check invariants.** For each invariant listed in the spec's `## Invariants` section, verify the implementation upholds it.
+5. **Check omissions.** Verify that behaviors the spec says MUST NOT happen are not present in the implementation (e.g., the implementation does not create files the spec forbids, does not modify data the spec says is read-only).
+6. **Skip deviated behaviors.** If `deviations.md` exists, do not flag deviated behaviors as compliance issues.
+7. **Report findings.** Return a structured list of compliance issues found, each with:
+   - The spec section it relates to
+   - What the spec requires
+   - What the implementation does instead
+   - Severity (violation of MUST vs. SHOULD vs. minor discrepancy)
+
+The subagent MUST NOT modify any files — it is read-only. It reports findings back to the primary agent.
+
+Upon receiving the audit results, the primary agent MUST:
+- Fix all MUST-level violations
+- Evaluate SHOULD-level issues and fix or document as appropriate
+- Re-run the test suite after any fixes to confirm no regressions
+- If fixes were made, dispatch the subagent for one additional audit pass to verify the fixes (a single re-audit is sufficient — do not loop indefinitely)