letsdo 0.3.0 → 0.5.0

data/docs/prompts.md

@@ -0,0 +1,209 @@
+# letsdo — prompt-authoring guide
+
+An agent in letsdo is exactly one file: `agents/<name>.md`. The file is the
+agent's whole identity — role, rules, workflow. The assignee handle is
+derived from the name (`agents/developer.md` ⇄ `@developer`), so the agent
+automatically works on the tasks already assigned to it. Adding a file
+creates an agent; no code, no schemas, no setup.
+
+This guide covers what a good agent prompt contains, what to avoid, and two
+worked examples from this repository's own agents.
+
+- [How letsdo uses the prompt](#how-letsdo-uses-the-prompt)
+- [Optional: per-agent configuration](#optional-per-agent-configuration)
+- [Must-haves](#must-haves)
+- [Anti-patterns](#anti-patterns)
+- [Worked examples](#worked-examples)
+- [Checklist](#checklist)
+
+## How letsdo uses the prompt
+
+- The prompt is read on every run from `<LETSDO_ROOT>/agents/<name>.md` and
+  handed to pi as the agent's system prompt (`pi --mode json`).
+- Before the prompt is handed over, letsdo prepends the agent's identity —
+  its name and backlog assignee handle (`Config#assignee_handle`, default
+  `@<name>`) — to it. The injection always happens, whatever the template
+  contains, so a template never has to state the name or handle.
+- When the file is missing, the agent runs on the built-in default prompt
+  (process-only instructions); letsdo announces where it looked and how to
+  create a prompt (`letsdo <name> --init`).
+- The pipeline is deterministic per run: one run = one agent invocation =
+  one task. The prompt defines HOW the agent executes that task.
+
+## Optional: per-agent configuration
+
+A prompt file can open with a YAML front-matter block that sets launch
+parameters for that one agent. The block is optional — a file without it
+behaves exactly as before.
+
+```markdown
+---
+model: anthropic/claude-sonnet-4-5
+---
+
+# Developer agent (developer)
+
+You are a developer agent named developer.
+...
+```
+
+What to know:
+
+- The block must be the very first thing in the file (`---`, the keys, a
+  closing `---`). It is stripped before the prompt reaches the agent, so it
+  never shows up in the system prompt.
+- `model` is the only key used today: it is passed to pi as
+  `--model <name>` for this agent. Leave it out and pi uses its own default
+  model.
+- A `--model` in `LETSDO_PI_FLAGS`/`AGENT_PI_FLAGS` wins over the file, so
+  keep `--model` out of the global flags when you want to choose the model
+  per agent.
+- The block is extensible: unknown keys are parsed and ignored, so new
+  parameters can be added later without breaking existing prompts.
+
+This is the only configuration a *prompt file* carries; everything else is
+an environment variable (see the [configuration reference](config.md)).
+
+## Must-haves
+
+A robust agent prompt has these parts:
+
+1. **Identity.** The agent's identity — its name and backlog assignee
+   handle — is injected by letsdo at the top of every prompt, so the
+   template does not have to state it. A hardcoded handle is a bad idea:
+   it drifts from `AGENT_ASSIGNEE_HANDLE` and the backlog assignment. You
+   can still describe the role in prose; keep the name in the prompt
+   matching the file name.
+
+2. **Exactly one task per run.** The main rule. The agent picks up and
+   completes exactly one assigned task, then stops. This is what keeps an
+   orchestrator loop honest: no context-switching, no runaway churn.
+
+   ```
+   ## Main rule: exactly one task per run
+   In a single run you pick up and complete exactly one task assigned to
+   you, then stop. You take the next task only in the next run.
+   ```
+
+3. **What to do when there are no tasks.** The corollary of the one-task
+   rule: an agent must *not* invent work or create tasks itself.
+
+   ```
+   If there are no tasks assigned to you — do not invent work and do not
+   create tasks yourself. End the run with a message that there are no
+   tasks.
+   ```
+
+4. **How to choose a task.** Be explicit about the selection order, because
+   the agent will face multiple open tasks:
+
+   ```
+   Take the highest priority task assigned to you in order:
+   0. if a task is already in progress
+   1. priority
+   2. order (ordinal), if priorities are equal
+   If the chosen task is currently blocked, take the task that blocks it.
+   ```
+
+5. **An execution protocol.** Step-by-step instruction for the lifecycle of
+   a task: read the instructions, start (status + assign to yourself), plan,
+   record progress as you go, verify and finish, commit. Reference the
+   project's actual commands and guides (`backlog instructions
+   task-execution`, `backlog instructions task-finalization`, ...) — the
+   agent executes them literally.
+
+6. **Deliverables.** What the final result must look like — concrete
+   artifacts: implementation tasks for the developer with acceptance
+   criteria, documentation files, recorded decisions. Name file paths and
+   commands.
+
+7. **Style and language rules.** Project conventions the agent must follow
+   in every artifact it writes (for example English-only task tracking).
+   State them as a hard rule, not a suggestion.
+
+8. **Prohibitions.** What the agent must never do — explicitly. The most
+   valuable one in a multi-agent system: *do not take work that is not
+   assigned to you* and *do not complete several tasks in one run*.
+
+## Anti-patterns
+
+- **Multi-task runs.** "Work through all open tasks" defeats the loop
+  design: one task per run is the contract that makes progress observable
+  and stops possible.
+- **Inventing work.** No task → done, not "let me also fix wiring/". A
+  worker without an explicit no-invent rule will drift off the backlog.
+- **Vague instructions.** "Improve the code" — improve what, where, how
+  will we know it worked? Name files, components, commands and exit
+  criteria. The agent's job is execution, not guessing intent.
+- **No stop condition.** A prompt without a finish ("end the run with a
+  message when there are no tasks") produces an agent that never knows it
+  is done.
+- **Keeping state only in the head.** Long tasks fail when intermediate
+  results live only in the conversation. The prompt must require recording
+  (task notes, comments) as it goes.
+- **Untestable acceptance criteria.** "Works well" cannot be verified.
+  Prefer testable, objective criteria ("`rake test` green, 0 failures",
+  "the created task is assigned to @developer").
+- **Missing identity/role.** Without a role the agent cannot tell what it
+  owns. letsdo injects the name and handle, but the role and rules are the
+  template's job.
+- **Mixing languages in tracked artifacts.** In this project the convention
+  is English for tasks, notes, decisions and docs (TASK-35/48) even though
+  conversations may be in any language. State the convention; do not rely
+  on osmosis.
+
+## Worked examples
+
+Both examples are real agents of this repository: `agents/developer.md` and
+`agents/analyst.md`.
+
+### agents/developer.md — the implementer
+
+Structure: identity → main rule → task selection → execution protocol →
+prohibitions.
+
+- **Identity + main rule:** "You are a developer agent named developer",
+  then exactly-one-task-per-run with the no-tasks rule.
+- **Task selection:** the priority → ordinal ladder, with the blocked-task
+  rule (take the blocker into work).
+- **Execution protocol:** read `backlog instructions task-execution` first;
+  move the task to In Progress and assign it to yourself; draft a plan and
+  record it; implement in short iterations with progress in task notes;
+  on completion verify every acceptance criterion with objective evidence,
+  write a final summary and move the task to Done; commit the changes
+  including the backlog folder.
+- **Prohibitions:** no unassigned work, no several tasks in one run.
+
+### agents/analyst.md — the designer
+
+Same skeleton, different deliverables: identity, main rule, task selection,
+then an analysis protocol (start → plan → research → record rationale in
+task notes/comments → produce deliverables) and a finalization protocol
+(verify each acceptance criterion with objective evidence, mark completed
+items, final summary, Done, commit). The analyst's "deliverables" step is
+the interesting part — it says documentation deliverables are written into
+the repo (`backlog doc/decision`), reference them from the task, and
+implementation work is handed to the developer as new tasks: clear title,
+description, testable acceptance criteria, affected components. Both agents
+end with the same prohibitions and an explicit language/style section.
+
+The pattern to copy: **identity → one task per run → selection → protocol
+with concrete commands → deliverables → style rule → prohibitions.**
+
+## Checklist
+
+Before writing `agents/<name>.md`, check:
+
+- [ ] Role described in prose (the name/handle identity block is injected
+      by letsdo).
+- [ ] Main rule: exactly one task per run.
+- [ ] No-tasks rule: do not invent work, do not create tasks.
+- [ ] Task selection: in progress → priority → ordinal; blocked → take the
+      blocker.
+- [ ] Execution protocol with the project's real commands and guides.
+- [ ] Deliverables: concrete artifacts, paths, acceptance criteria.
+- [ ] Style/language conventions for tracked artifacts.
+- [ ] Prohibitions (unassigned work, several tasks per run, ...).
+- [ ] Prompt matches the file name (`<name>.md` ⇄ `@<name>`).
+- [ ] (Optional) Front-matter `model:` is set only when you want a
+      per-agent model — a global `--model` flag overrides it.

data/docs/task-selection.md

@@ -0,0 +1,239 @@
+# letsdo — task selection: feasibility and requirements
+
+This document is the outcome of TASK-86: can the next task an agent should
+pick up be selected programmatically and algorithmically? It records how
+selection works today, which criteria an algorithm would use, what data is
+missing, the risks and edge cases, a sketch of a possible approach, and the
+recommendation.
+
+- [Summary](#summary)
+- [How selection works today](#how-selection-works-today)
+- [Selection criteria](#selection-criteria)
+- [Missing data](#missing-data)
+- [Risks, limitations, edge cases](#risks-limitations-edge-cases)
+- [Possible approaches](#possible-approaches)
+- [Recommendation](#recommendation)
+- [Out of scope](#out-of-scope)
+
+## Summary
+
+Selection today is **LLM-driven, not algorithmic**. `Letsdo::Loop` receives a
+batch of open tasks and runs the agent once per batch element, but the task
+object it holds is discarded before the run: the agent re-queries the backlog
+inside its own prompt and decides which task to work. letsdo never enforces
+the batch order it just read.
+
+The backlog CLI already exposes the primitives needed for deterministic
+selection (`--sort priority`, `--ready`, `--type`, `--labels`, `--limit`), and
+`Letsdo::Providers::Backlog` uses none of them. The recommendation is to build
+a **small, scoped deterministic-ordering change** (scope A below): request a
+priority-sorted, ready-only batch and keep the fields the selector needs. Do
+**not** build a cross-agent scheduler (scope C); task injection into the
+prompt (scope B) is a documented follow-up that needs its own decision because
+it changes the agent contract.
+
+**Status: scope A is implemented (TASK-95).** The provider now requests
+`--ready --sort priority` and sorts the normalized batch in the adapter (In
+Progress first, then priority, then ordinal, then id), so the run order it
+hands to the loop is the authoritative one. The analysis below is kept as the
+record; scopes B and C remain out of scope.
+
+## How selection works today
+
+End-to-end, one `letsdo <name>` session:
+
+1. `Letsdo::Providers::Backlog` runs
+   `backlog task list --assignee <handle> --exclude-status Done --ready
+   --sort priority --json` (`lib/letsdo/providers/backlog.rb`,
+   `#command_line`). `--ready` drops tasks whose dependencies are not all
+   done; `--sort priority` orders by priority then ordinal as a first pass.
+2. The adapter projects each raw task onto `TASK_FIELDS =
+   %w[id title status priority assignees ordinal type labels milestone]` and
+   drops `reporter`, `parentTaskId`, `createdAt`, `updatedAt` and everything
+   else. It then sorts the batch itself, because the CLI does not put In
+   Progress first: In Progress first, then priority High > Medium > Low, then
+   ordinal ascending, then id ascending. The batch the provider returns is
+   therefore the authoritative run order (TASK-95).
+3. `Letsdo::Loop#run_batch` (`lib/letsdo/loop.rb`) iterates the batch once and
+   calls `@run_task.call(task)` for each element. The provider is **not**
+   re-polled between the runs of one batch.
+4. `Letsdo::AgentLoop#run_one_task` (`lib/letsdo/agent_loop/tasks.rb`) logs
+   the task label and calls the runner. Its default runner is
+   `->(_task) { @agent.run }` (`lib/letsdo/agent_loop.rb`, `#assign_opts`) —
+   **the task object is thrown away**.
+5. `Letsdo::Agent#run` (`lib/letsdo/agent.rb`) takes no task argument. It
+   prepends the injected identity and hands the prompt to pi. The prompt is
+   what chooses the task: `agents/analyst.md` and `agents/developer.md`
+   instruct the agent to run
+   `backlog task list --assignee @<name> --exclude-status Done --sort priority --plain`
+   and take the first task, with the "already In Progress first" and "if
+   blocked, take the blocker" exceptions.
+6. Retry state lives in `Letsdo::RetryPolicy` and is keyed on the **batch
+   element** passed to `run_one_task` (`@last_attempted[task_key(task)]`),
+   then reconciled against the next provider batch. It is not keyed on the
+   task the agent actually worked.
+
+So the authority over "which task next" sits in the prompt, and letsdo's own
+batch is only a run counter. The order letsdo read does not govern the work.
+
+## Selection criteria
+
+A deterministic selector needs these inputs, in this order:
+
+1. **Eligibility**
+   - assignee contains the agent's handle (`Config#assignee_handle`, default
+     `@<name>`); other agents' and `@human` tasks are never auto-run.
+   - `status != Done`.
+   - not in retry cooldown and not given up for this session
+     (`RetryPolicy#cooldown?` / `#gave_up?`).
+   - dependencies satisfied (all blocking tasks done).
+2. **Ordering**
+   - `In Progress` first: a task already started by this agent resumes before
+     a new task starts (the prompt's existing rule 3).
+   - `priority`: High > Medium > Low.
+   - **tie-break: `ordinal` ascending, then `id` ascending.** Equal priority
+     is the common case (all three open tasks in this repo are Medium), so the
+     tie-break is not optional — without it the order is unstable. The backlog
+     CLI's `--sort priority` already orders same-priority tasks by ordinal
+     ascending (verified: TASK-86 ordinal 75000 before TASK-89 ordinal 78000).
+3. **Optional routing inputs** (only if per-agent routing is wanted later)
+   - `type` (`spike`, `bug`, `feature`, `chore`, `docs`, `task`) — e.g. the
+     analyst takes spikes, the developer takes bugs/features.
+   - `labels` / `milestone` — team-defined tags usable as capability filters.
+   - `parentTaskId` — subtask ordering, if subtasks ever become selectable
+     units.
+4. **Load**
+   - number of open tasks runnable by this agent, or per-agent concurrency.
+     This criterion is **not implementable in the current design**: each agent
+     runs as its own process and the backlog is the only shared state, so no
+     process can see another's queue or in-flight work.
+
+## Missing data
+
+| Gap | Where | Impact |
+| --- | --- | --- |
+| No sort requested | `Providers::Backlog#command_line` | Resolved in TASK-95: the command requests `--sort priority` and the adapter applies the full In-Progress-first tie-break. |
+| No readiness filter | same | Resolved in TASK-95: the command requests `--ready`, so blocked tasks never reach the loop. |
+| `type`, `ordinal`, `labels`, `milestone` dropped | `TASK_FIELDS` | Resolved in TASK-95: all four are normalized onto `Task`. |
+| `dependencies` absent from list JSON | backlog `task list --json` | Blocked state is not a field; the provider relies on the `--ready` filter instead of N per-task `backlog task <id> --json` calls. |
+| Task object discarded before the run | `AgentLoop#assign_opts` + `Agent#run` | letsdo cannot force the task it selected; the LLM re-selects and can diverge from the batch element (see risks). |
+| No machine-readable agent capabilities | `agents/<name>.md` front matter (only `model` is consumed, `lib/letsdo/agent.rb`) | Type/skill routing exists only as prose in the prompt, not as data. The front-matter block is the natural place to add `types:` / `skills:`. |
+| No claim/lock | backlog only | Two agents (or two processes of the same agent) can race for the same unassigned task; a status update is not atomic. |
+| No selection telemetry | `Metrics` records provider counts and run outcomes | There is no record of which task was selected, on which basis, or why a task was skipped. |
+
+## Risks, limitations, edge cases
+
+- **Equal priority.** Must be resolved deterministically. Use `ordinal`, then
+  `id`; do not rely on the raw CLI order.
+- **Blocked top task.** If the highest-priority task is blocked, the selector
+  must skip it. The prompt's current rule ("take the blocker") is ambiguous
+  for an algorithm: if the blocker is assigned to this agent, selecting the
+  blocker is correct; if it is another agent's task, that would violate the
+  "do not take work that is not assigned to you" rule, so the selector must
+  fall through to the next runnable task and surface the skip.
+- **Transitive and stale dependencies.** `--ready` is documented as
+  "unblocked tasks with all dependencies completed" — rely on the CLI rather
+  than reimplementing graph traversal locally. A dangling dependency id
+  (blocker deleted/archived) may make a task permanently "not ready"; the
+  session should make that visible (a `--ready`/blocked count in the log)
+  instead of silently never running it.
+- **Batch order vs. prompt order (the real defect).** The loop runs the agent
+  once per batch element without re-polling. If the batch is
+  `[B(Low), A(High)]` and run 1 does not close `A`, run 2 runs for batch
+  element `B` while the agent again selects `A`. `B` starves and the retry
+  accounting misattributes a failure to `B` (which was never worked) while
+  `A` is not cooled down. Aligning the provider order with the prompt order
+  fixes the common case; only task injection (scope B) fixes it absolutely.
+- **New work during a batch.** Because the provider is polled once per batch,
+  a High task created mid-batch cannot preempt it; it is picked up after the
+  current batch drains. Acceptable for short runs; note it if runs get long.
+- **Race conditions.** Unassigned tasks are a coordination gap: two agents can
+  pick the same task. The design's answer is assignment by a human
+  (`@developer`, `@analyst`, `@human`), not a lock. A lock/claim mechanism
+  would require shared mutable state and belongs to scope C, not here.
+- **Retry interaction.** Any selector must compose with `RetryPolicy`: a task
+  in cooldown or given up must not be re-offered, and the outcome must be
+  recorded against the task that was actually selected.
+- **Scope creep.** "Algorithmic selection" can silently grow into a central
+  scheduler with a queue, locks, load balancing and preemption. That is a
+  different architecture (shared coordinator) from letsdo's "one process per
+  agent, backlog is the only shared state" model and should not be smuggled
+  into this change.
+
+## Possible approaches
+
+### A. Provider-side deterministic ordering (small)
+
+Make the provider return the batch in authoritative order and keep the fields
+a selector needs:
+
+- request `--ready --sort priority` in
+  `Providers::Backlog#command_line`;
+- extend `TASK_FIELDS` with `type`, `ordinal`, `labels`, `milestone`;
+- order in the selector/provider as: In Progress first, then priority, then
+  ordinal, then id (the CLI sorts by priority+ordinal but does not put In
+  Progress first — verified: `--ready --sort priority` returned the In
+  Progress task first here only because its ordinal is lower);
+- `Letsdo::Loop` stays generic; no prompt or `Agent` signature change.
+
+Pros: small, no agent-contract change, testable against the fake backlog, uses
+documented CLI features. Cons: the agent still re-selects inside its prompt,
+so letsdo's batch order is only advisory and the failure/accounting defect in
+the risks section narrows but is not eliminated.
+
+### B. Selector + task injection (medium)
+
+Build on A: a pure `Letsdo::Selection` object computes the next runnable task,
+and letsdo passes it into the run so the agent does not re-select:
+
+- `Agent#run(task:)` injects "your task for this run is `<id>` — `<title>`"
+  into the prompt;
+- the prompts' "Choosing a task" section is replaced by "work the task letsdo
+  handed you" (the blocker/readiness logic moves into the selector);
+- `AgentLoop` records retry outcomes against the injected task.
+
+Pros: one run = one task becomes enforced, not advisory; retry accounting and
+starvation are fixed; per-agent `types:`/`skills:` front matter routing becomes
+possible. Cons: changes the agent contract and both prompt files; needs its own
+decision, prompt tests and a transition for custom prompts that still
+self-select.
+
+### C. Central scheduler (large, not recommended)
+
+A coordinator process with a shared queue, atomic claim/lock, per-agent load
+balancing and preemption.
+
+Pros: true multi-agent scheduling. Cons: introduces shared mutable state,
+concurrency, a new failure domain and an operational component; contradicts the
+project's stated design ("multiple agents run as separate processes ... they
+coordinate through the shared backlog — nothing else in common", README). Not
+justified by any observed failure.
+
+## Recommendation
+
+**Build it — to scope A.** Add deterministic ordering and readiness to the
+provider: `--ready --sort priority`, the missing normalized fields, and the
+explicit In-Progress→priority→ordinal→id order. It is small, low-risk, uses
+existing backlog CLI features, needs no prompt or agent-contract change, and
+removes the common divergence between the order letsdo reads and the order the
+agent picks. **Implemented in TASK-95** (`Providers::Backlog#command_line` and
+`#sort`, `Providers::Task` fields).
+
+Record scopes **B and C as out of scope for now**. B (task injection into the
+prompt) is the correct way to fully enforce the batch and fix retry
+misattribution, but it changes the agent contract and both prompt files and
+deserves a separate decision once A is in use. C (central scheduler) is
+explicitly rejected: it conflicts with the backlog-only, one-process-per-agent
+architecture and is not justified by current pain.
+
+The follow-up implementation task for scope A is tracked as TASK-95
+(`Deterministic task selection: priority-sorted, ready-only provider batch`).
+
+## Out of scope
+
+- Task injection into the agent prompt and the prompt rewrite (scope B).
+- Per-agent `types:`/`skills:` front-matter routing (needs B's injection to be
+  enforceable).
+- Cross-agent load balancing, shared queue, atomic claim/lock, preemption
+  (scope C).
+- Subtask-level selection and milestone-driven scheduling.