@arjunkhera/atlas 0.3.7 → 0.3.9

package/skills/sdlc-task/design/parts/why.md

@@ -0,0 +1,22 @@
+# Why one initiative
+
+It holds: Why these designs belong together.
+
+Needs: Decide: none, Flow: none, Contract: none, Initiative: needed.
+
+Rules:
+
+1. Say why these designs belong together.
+2. Name the one goal they serve.
+3. Say what would go wrong if each went alone.
+
+Example, in a file of its own:
+
+````markdown
+---
+part: why
+---
+# Why one initiative
+
+Each file type needs its own reader, but all share one viewer and one queue. Alone, each would build its own queue.
+````

package/skills/sdlc-task/design/parts/words.md

@@ -0,0 +1,29 @@
+# Your words
+
+It holds: The owner's words, word for word. Claude's reading stays apart.
+
+Needs: Decide: needed, Flow: needed, Contract: needed, Initiative: needed.
+
+Rules:
+
+1. Quote the owner word for word, in a block quote, with the date and the place.
+2. Put your own reading after the quote, as a short list headed "What we take from it".
+3. Never change a word inside the quote.
+
+Example, in a file of its own:
+
+````markdown
+---
+part: words
+---
+# Your words
+
+> "I want tests to run before the merge, not after."
+
+The owner, 9 October, in the project thread.
+
+What we take from it:
+
+- A red test stops the merge.
+- The owner sees the failing check, not a log.
+````

package/skills/sdlc-task/design/parts/yardstick.md

@@ -0,0 +1,25 @@
+# Yardstick
+
+It holds: The tests every option must pass, set before the options.
+
+Needs: Decide: needed, Flow: none, Contract: none, Initiative: none.
+
+Rules:
+
+1. Set the tests every option must pass before you list the options.
+2. Give each test a short name, so the scorecard can use it.
+3. Keep it to five tests or fewer.
+
+Example, in a file of its own:
+
+````markdown
+---
+part: yardstick
+---
+# Yardstick
+
+| Test | What it asks |
+|---|---|
+| Stops red | Does a red test stop the merge? |
+| Fast | Does the check finish in ten minutes? |
+````

package/skills/sdlc-task/design/parts.yaml

@@ -0,0 +1,306 @@
+# The parts of an Atlas design, and which kind needs each one.
+#
+# A design is a folder. design.yaml names its kinds. Each part is one
+# markdown file in the folder, with `part: <id>` in its front matter.
+# The design check reads this file to find a missing or empty part.
+# The page builder reads it for the order and the groups of the rail.
+# The guide in design/README.md shows the same table; a test keeps the two equal.
+#
+# Each part says what each kind needs:
+#   needed    the part must be in the folder and must not be empty
+#   optional  write it when it applies
+#   none      the kind does not use it
+#   shared    needed when design.yaml says `shared: true`, optional when not
+#
+# A design keeps no state. The tracker parts at the end are read from the
+# tracker at build time. No part file may hold them.
+
+kinds:
+  - id: decide
+    name: Decide
+    pick: There are several ways, and one must win.
+  - id: flow
+    name: Flow
+    pick: The behaviour is the hard part.
+  - id: contract
+    name: Contract
+    pick: An interface, a schema or a command changes.
+  - id: initiative
+    name: Initiative
+    pick: Many designs serve one goal.
+
+groups:
+  - id: start
+    name: Start here
+  - id: decide
+    name: Decide
+  - id: flow
+    name: Flow
+  - id: contract
+    name: Contract
+  - id: initiative
+    name: Initiative
+  - id: prove
+    name: Prove it
+  - id: tracker
+    name: From the tracker
+
+parts:
+  - id: summary
+    name: Summary
+    group: start
+    holds: The answer in a few lines, and one picture.
+    decide: needed
+    flow: needed
+    contract: needed
+    initiative: needed
+  - id: words
+    name: Your words
+    group: start
+    holds: The owner's words, word for word. Claude's reading stays apart.
+    decide: needed
+    flow: needed
+    contract: needed
+    initiative: needed
+  - id: goals
+    name: Goals and non-goals
+    group: start
+    holds: What it is for, what it is not, and how we know it worked.
+    decide: needed
+    flow: needed
+    contract: needed
+    initiative: needed
+
+  - id: problem
+    name: Problem today
+    group: decide
+    holds: What happens today, with a real example.
+    decide: needed
+    flow: optional
+    contract: optional
+    initiative: none
+  - id: yardstick
+    name: Yardstick
+    group: decide
+    holds: The tests every option must pass, set before the options.
+    decide: needed
+    flow: none
+    contract: none
+    initiative: none
+  - id: proposal
+    name: Proposal
+    group: decide
+    holds: The option we pick, and how it works.
+    decide: needed
+    flow: none
+    contract: none
+    initiative: none
+  - id: alternatives
+    name: Alternatives
+    group: decide
+    holds: The other options, and why each one lost.
+    decide: needed
+    flow: optional
+    contract: optional
+    initiative: none
+  - id: scorecard
+    name: Scorecard
+    group: decide
+    holds: Each option against each yardstick test.
+    decide: needed
+    flow: none
+    contract: none
+    initiative: none
+  - id: build
+    name: Build list
+    group: decide
+    holds: The pieces to build, in order.
+    decide: needed
+    flow: optional
+    contract: optional
+    initiative: none
+
+  - id: actors
+    name: Who and what
+    group: flow
+    holds: The people, agents and systems in the flow.
+    decide: optional
+    flow: needed
+    contract: optional
+    initiative: none
+  - id: stories
+    name: Stories
+    group: flow
+    holds: One walkthrough for each story, step by step.
+    decide: optional
+    flow: needed
+    contract: optional
+    initiative: none
+  - id: edges
+    name: Edge cases
+    group: flow
+    holds: What happens when a step fails or comes in a strange order.
+    decide: optional
+    flow: needed
+    contract: optional
+    initiative: none
+  - id: records
+    name: Records
+    group: flow
+    holds: The records each step reads and writes.
+    decide: none
+    flow: needed
+    contract: optional
+    initiative: none
+  - id: states
+    name: States
+    group: flow
+    holds: The states a thing moves through, and what moves it.
+    decide: none
+    flow: optional
+    contract: none
+    initiative: none
+
+  - id: routes
+    name: Routes and interface
+    group: contract
+    holds: Every route, exported function, command, tool or schema object that is added, changed or removed.
+    decide: none
+    flow: none
+    contract: needed
+    initiative: none
+  - id: calls
+    name: Calls and answers
+    group: contract
+    holds: A real call and its real answer for each entry of the interface. A schema repo shows a real query; a plugin shows a tool call.
+    decide: none
+    flow: none
+    contract: needed
+    initiative: none
+  - id: data
+    name: Data change
+    group: contract
+    holds: The schema or data change. Write "No change" and why, when there is none, as a library or a CLI often does.
+    decide: none
+    flow: none
+    contract: needed
+    initiative: none
+  - id: migration
+    name: Migration and undo
+    group: contract
+    holds: How old data moves to the new shape, and how the data change is undone.
+    decide: none
+    flow: none
+    contract: needed
+    initiative: none
+  - id: security
+    name: Security
+    group: contract
+    holds: Who may call it, what it may reach, and what it never leaks.
+    decide: optional
+    flow: optional
+    contract: needed
+    initiative: none
+  - id: rollout
+    name: Rollout
+    group: contract
+    holds: The order of release, and how we watch it.
+    decide: optional
+    flow: optional
+    contract: needed
+    initiative: none
+
+  - id: why
+    name: Why one initiative
+    group: initiative
+    holds: Why these designs belong together.
+    decide: none
+    flow: none
+    contract: none
+    initiative: needed
+  - id: shared-decisions
+    name: Shared decisions
+    group: initiative
+    holds: The decisions every design in the family follows, and why.
+    decide: none
+    flow: none
+    contract: none
+    initiative: needed
+  - id: order
+    name: Order and dependencies
+    group: initiative
+    holds: Which design comes first, and what each one waits on.
+    decide: none
+    flow: none
+    contract: none
+    initiative: needed
+
+  - id: risks
+    name: Risks
+    group: prove
+    holds: What breaks, and what we do. "Accepted" with a reason is valid.
+    decide: needed
+    flow: needed
+    contract: needed
+    initiative: needed
+  - id: proof
+    name: Test and proof plan
+    group: prove
+    holds: How each piece proves itself.
+    decide: needed
+    flow: needed
+    contract: needed
+    initiative: none
+  - id: repos
+    name: Each kind of repo
+    group: prove
+    holds: What the design means for a service, a library, a schema, a CLI and a plugin.
+    decide: shared
+    flow: shared
+    contract: shared
+    initiative: none
+  - id: change
+    name: Change and undo
+    group: prove
+    holds: The kind of change, its impact, how to undo it, and each existing test it changes.
+    decide: needed
+    flow: needed
+    contract: needed
+    initiative: optional
+  - id: done
+    name: Done line
+    group: prove
+    holds: Lines someone else can check. The approval lives in the tracker.
+    decide: needed
+    flow: needed
+    contract: needed
+    initiative: needed
+  - id: key
+    name: Key
+    group: prove
+    holds: Every word and code the design uses.
+    decide: needed
+    flow: needed
+    contract: needed
+    initiative: needed
+
+# Read from the tracker at build time. The page shows them read only.
+tracker:
+  - id: questions
+    name: Questions
+    holds: Each question to the owner, open or answered, with the answer.
+  - id: decisions
+    name: Decisions
+    holds: Each decision, in the owner's words, with its date.
+  - id: approval
+    name: Approval and lock
+    holds: The owner's approval and the lock, with the words and the date.
+  - id: since
+    name: Since approval
+    holds: Each change after approval, the part it touched, and the owner's words.
+  - id: family
+    name: Family
+    holds: The designs linked to this one, with their state.
+  - id: rollup
+    name: Roll-up
+    holds: How far the family has come. Initiative designs only.

package/skills/sdlc-task/lifecycle.yaml

@@ -25,7 +25,7 @@ tripwires: [definition_of_done, security, schema_migrations, money_cost]
 tiers: [hotfix, standard, initiative]
 tier_escalators: [migrations, auth_rls_surface, new_external_dependency, money_cost]
 
-# Both templates travel with the package, beside the kernel.
+# The design format and the resume anchor travel with the package, beside the kernel.
 templates:
-  design_doc: templates/design-doc.md
+  design_doc: design/README.md
   resume_anchor: templates/resume-anchor.md

package/skills/tests/SKILL.md

@@ -0,0 +1,234 @@
+---
+name: tests
+description: >
+  Scenario tests for an area of a repo. Trigger when the owner asks to put tests on an area,
+  to review the words of a scenario, to write or convert a test from a scenario, when a test
+  is red, or to prove that tests catch faults. It holds five procedures: onboard an area,
+  review words, convert, sort a red test, and plant faults.
+---
+
+# Tests
+
+A scenario is a Markdown file. It says what must be true. A test is code
+that checks it. Atlas ships the code that joins the two: the contract
+library, the drivers, the stand-in helper and the link check. This skill
+holds the steps to use them.
+
+## The files of an area
+
+| File | What it holds | Who changes it |
+|---|---|---|
+| `atlas/tests.yaml` | How to start the product, the ways in, the stand-ins, the personas, the guards | You, at onboarding. The owner approves its guard parts |
+| `scenarios/<id>.md` | The words: start, steps, waits, case tables, assertions | The design. The builder copies the approved words |
+| `scenarios/map.md` | How to reach each thing: calls, page parts, error forms, gaps | The builder, as it learns facts |
+| `test/atlas/` | The Atlas kit, copied byte for byte | Only `atlas tests write` |
+| `test/scenarios/` | One test file for each scenario | The builder |
+| `test/support/` | The stand-ins, the actions, the makers, the adapter | The builder |
+| `test/evidence/` | The evidence of each run | The library. Git ignores it |
+
+In this skill, "you" is the agent that runs the skill. It is never the owner.
+
+Never edit a file in `test/atlas/`. A hand edit stops the next
+`atlas tests write`, and `atlas tests check` reports it. A person merges
+every file that command writes.
+
+## The three commands
+
+1. `atlas tests write --root <repo>` puts the kit in `test/atlas/`. It
+   refuses to replace a file that you edited, and it refuses to write
+   through a link. It writes `test/evidence/.gitignore`. It adds a `.env.test`
+   line to the `.gitignore` at the repo root. It says what it added.
+2. `atlas tests check --root <repo>` runs the link check. It prints one line
+   for each finding, as `file:line  code  words`. Run it before you ask for a merge.
+3. `atlas tests proof --root <repo>` prints the proof table of one run.
+
+The link check has three halves. Run the first two before the tests and the
+third after them:
+
+1. `--halves static,lint` reads the scenarios, `tests.yaml` and the code. It
+   also checks each file in `test/atlas/` against `manifest.json`. The manifest
+   must match a kit that a release of Atlas shipped.
+2. `--halves run --run <id>` reads the evidence of that one run. With no
+   `--run`, it reads the newest run of each scenario, and `atlas tests proof`
+   does the same. It fails on an assertion that the run did not check. It fails
+   on evidence of an older version of a scenario file.
+3. `--guards-from <file>` takes `tests.yaml` from the main branch. A guard
+   part that differs is a finding. CI must run the check from the main branch.
+
+The environment variable `ATLAS_GUARDS_FROM` names that file for both the
+check and the scenario run. A scenario run whose guard parts differ from
+that file ends `blocked` and starts nothing. A local run without it uses the branch file.
+
+## Run the tests in CI
+
+CI runs the tests from the main branch, so a pull request cannot change its own judge.
+
+1. Fetch the main branch.
+2. Run the copy of `<area>/test/atlas/` that main holds. If main has none
+   yet, run the copy of the branch, and print a warning that says so.
+3. Set `ATLAS_GUARDS_FROM` to `atlas/tests.yaml` of the main branch.
+4. Run the scenarios with the runner command of `tests.yaml`. Set `ATLAS_RUN_ID`
+   or let CI make the run id.
+5. Run the link check with `--run <that run id>`.
+6. Upload `<area>/test/evidence/` for 14 days.
+
+## Onboard an area
+
+Path 1 is for an area that has integration tests which call its routes,
+and which people keep using. Old unit tests that call functions do not make path 1.
+Use path 2: the area has no integration test that calls its routes. If it has
+route tests that people keep, stop. Tell the owner that path 1 comes later.
+
+1. Send the `atlas:code-explorer` crew to map the area. Ask for routes,
+   tools, pages, storage, calls to outside services and existing tests.
+2. Run `atlas tests write --root <repo>`.
+3. Draft `atlas/tests.yaml`. Give it `local` and `ci`, each way in, a persona
+   for each caller, and `guards.hosts` with local addresses only.
+4. Give the product no real secret. Write `{run.secret}` for each secret
+   of the product. The library makes it new for each run.
+5. Write a stand-in for each outside service. Use `serveStandIn` from
+   `test/atlas/stand-in.mjs`. Give the product a fake key for the real service.
+6. Write the contract scenario of each stand-in (kind `contract`). It proves
+   that the stand-in answers like the real service.
+7. Put two personas on one secret only with `same-identity` on both.
+8. Draft `scenarios/map.md` from the code digest and the docs of the repo.
+9. Write two or three first scenarios. Start with the smallest.
+10. Review the words (next section). Run `atlas ste` on each scenario.
+11. Convert each scenario into a test.
+12. Run `atlas tests check --root <repo> --halves static,lint`, then the tests,
+    then `atlas tests check --root <repo> --halves run`.
+13. Ask the `atlas:verifier` crew to start each environment once and run every scenario.
+14. Plant a fault for each assertion (last section).
+15. Open one pull request. Show the owner the first scenarios, and the guard
+    parts of `tests.yaml` as a short list in plain words.
+
+Worked run, on a meal planner with a web page, an MCP server and storage in
+a note service:
+
+1. Input: the code digest names the routes, the MCP server and the page. It
+   finds 11 tests that call functions and none that call a route.
+2. Process: path 2. Draft `tests.yaml` with two environments, three ways in,
+   a stand-in for the note service and four personas. Write the stand-in
+   and two scenarios.
+3. Output: one pull request. The owner reads two scenarios and three lines of guards.
+
+## Write assertions
+
+Use these steps when you write or change an assertion. They come from
+sections 9.2 and 6.10 of the design.
+
+1. Write the definition of done as assertions. A feature gets full scenarios.
+   A small fix gets a short card with its assertions.
+2. Name each assertion that you add, change or remove by its full id in the design.
+3. Write one outcome in each assertion. Then one planted fault points at one assertion.
+4. Mark it `exact` when code checks one right answer. Mark it `judged` only
+   when it is about meaning, and say why in the design.
+5. Never let a judged assertion decide an access rule alone.
+6. Mark `gate` on an assertion that later steps need.
+7. Start a "when" assertion with the step that makes it true: "step 3", or a kept result.
+8. Name what it reads in backticks.
+9. Write a "not" assertion so that a blank answer fails it.
+10. Add `repeat` only when the subject varies by design, such as a model. Say why.
+
+## Review the words
+
+A separate agent reads each new or changed scenario before any code exists.
+It proposes words. It never edits. It asks ten questions of each assertion:
+
+| Problem | The question |
+|---|---|
+| two-readings | Can two careful agents write different tests from it? |
+| not-exact | Does an "exact" assertion have one right answer? |
+| many-claims | Does it hold more than one outcome? |
+| duplicate | Does another assertion check the same thing? |
+| weak | How could the product pass it while the claim is false? |
+| hidden-precondition | Does it need a state that `Start with` does not name? |
+| missing-data | Does it need a value that the scenario does not give? |
+| shared-data | Can another run change the data it reads? |
+| empty-when | Does a "when" assertion name a step that makes it true? |
+| loose-not | Does a "not" assertion pass on a blank or broken answer? |
+
+Then run `atlas ste` on the scenario. The owner reads these pages.
+
+Rules for the words:
+
+1. One outcome for each assertion. Mark `exact` or `judged`.
+2. Mark `gate` on an assertion that later steps need.
+3. A wait ends with "Give up after <time>". The link check refuses it otherwise.
+4. A "when" assertion names its step: "step 3", or a result that a step keeps.
+5. Put each name in backticks. A name is a persona, way in, data rule, setup, fixture, table or kept result.
+6. An access rule is always exact. A judge never decides it alone.
+
+## Convert a scenario into a test
+
+1. Read the scenario, the map, the actions and one worked example. Do not
+   read the product source if the map covers the scenario.
+2. Write one test file for each scenario, with `scenario(path, body, { adapter })`
+   from `test/atlas/contract.mjs`. The library runs the body once for each way in.
+3. Name each assertion by its full id, as a plain string:
+   `run.check('<scenario>/e1#<fingerprint>', () => assert...)`.
+   Use `run.gate` for a `gate` assertion. Get the fingerprints from the failure message of `atlas tests check`.
+4. Use an action, or add one. An action for a new call also adds its row to the map.
+5. Make data with `run.fresh(rule)`. Read a case table with `run.table`, `run.cells` or `run.cases`.
+6. Wait with `run.wait` and a limit. Never sleep for a fixed time.
+7. Use `run.notExercised(id, why)` for a "when" assertion that the run never made true.
+8. Mark each guess in a comment. List the guesses in the pull request.
+9. Never change the words to make a test pass. Only the design changes the words.
+10. Run the test before the change. A test for new behaviour must fail.
+
+The adapter binds the library to the product. All its parts are optional:
+`standIns`, `warmUp`, `drivers`, `functions`, `loadModule`, `makers`, `signIns`,
+`data` and `actions`. The header of `test/atlas/contract.mjs` describes each one.
+
+A result is `pass`, `fail`, `not checked`, `not exercised`, `blocked` or
+`not here`. A test is red when any assertion is not `pass`. The one
+exception is `not here`: the scenario does not run in that environment, it is
+not counted, and the test passes.
+
+After the product is ready, a lost connection, a server exit or a time-out is
+`fail`. `blocked` is for a guard refusal, a start-up fault and the first-call check.
+
+## Sort a red test
+
+Do not edit a red test before you know why it is red. There are four causes:
+
+| Cause | Sign | What happens | What may change |
+|---|---|---|---|
+| The product broke | The words still hold. The product does not | Fix the code | Product code only |
+| The behaviour changed on purpose | The approved design names the assertion | Change the assertion and its test | The assertions that the design names, and their tests |
+| The test is out of date | A page part moved, or the form of an answer changed. The verifier's own check passes | Repair the actions, the map or the finders | Actions, map and finders. No assertion and no assert statement |
+| The environment failed | The result is `blocked`: the product did not start, or a stand-in is down | Fix the setup | `tests.yaml` without its guard parts, and the support code |
+
+Two checks follow:
+
+1. A repair that changes an assertion or an assert statement is not a repair. Refuse it unless the design names the assertion.
+2. A repair must keep the verifier's own check green. The verifier does not use the actions.
+
+A `blocked` result because a guard refused, or because the stand-in saw no
+first call from the product, is a finding, not noise.
+Find out why the product reached for something else.
+
+## Plant faults
+
+A fault proves that an assertion can go red. Plant one for each new or
+changed assertion, and for each assertion at onboarding.
+
+1. Commit first. A clone holds committed files only, so a change that is not
+   committed is not in the clone. Then clone the worktree to a throwaway
+   folder outside it, with `git clone --no-hardlinks`.
+2. Remove every remote from the clone. Run `git remote` there. It must print nothing.
+3. Plant faults in product code only. Never touch `tests.yaml`, the support code or the kit.
+4. Use the cheap model first: Haiku. After two failed tries on one assertion, use Sonnet.
+5. Give the planter one assertion and the product code. Ask for one fault that makes only that claim false.
+6. Run the scenario tests from the real worktree. Set `ATLAS_PRODUCT_ROOT` to
+   the area folder in the clone, and `ATLAS_FAULT_RUN` to a label. The product,
+   the MCP server and the module of the `function` driver start in the clone.
+   The guards, the kit and `tests.yaml` load from the worktree.
+7. The test for that assertion must go red.
+8. Ask the `atlas:verifier` crew for its own check of that assertion, on the same clone. It uses a raw driver, not the actions.
+9. A fault is valid only when the verifier's own check also fails. Discard any other fault and try again.
+10. Report each assertion as caught, missed, or "no fault of its own" after Sonnet also fails.
+11. Delete the clone. Run `git status` in the worktree. It must show no change from the faults.
+
+Evidence from a fault run carries the label. The link check ignores it, so it never counts as proof.
+Never push from the clone.