razzou-harness 0.1.1 → 0.1.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +40 -0
- package/README.md +341 -30
- package/dist/design/schemas.js +1 -1
- package/dist/setup/doctor.js +2 -2
- package/dist/setup/init.js +1 -1
- package/package.json +24 -2
package/CHANGELOG.md
ADDED
|
@@ -0,0 +1,40 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
The notable changes in each release of razzou-harness. Before 1.0, a new minor version
|
|
4
|
+
may change how a project is set up, and its notes say how.
|
|
5
|
+
|
|
6
|
+
## 0.1.2 (2026-10-11)
|
|
7
|
+
|
|
8
|
+
- The source is public, on GitHub, and the package links to it.
|
|
9
|
+
- Releases are published to npm from GitHub Actions, with provenance: a GitHub release
|
|
10
|
+
tagged `v<version>` publishes that version.
|
|
11
|
+
- A user guide in the README: your first module, the concepts, the commands, the
|
|
12
|
+
configuration, the AWS setup, security, usage, and troubleshooting.
|
|
13
|
+
- The README states what GitHub requires: an organization for the App, and a paid plan
|
|
14
|
+
for rulesets on private repositories.
|
|
15
|
+
- `harness init` and `harness doctor` no longer point to a private setup repository for
|
|
16
|
+
AWS.
|
|
17
|
+
- A security policy, in `SECURITY.md`.
|
|
18
|
+
|
|
19
|
+
## 0.1.1 (2026-10-11)
|
|
20
|
+
|
|
21
|
+
- The dashboard ships with `THIRD-PARTY-LICENSES.md`, the licenses of the libraries
|
|
22
|
+
bundled into it.
|
|
23
|
+
- The command is also installed as `razzou-harness`, for machines where another
|
|
24
|
+
`harness` comes first on the PATH.
|
|
25
|
+
- The package has no source maps any more: they pointed to sources it doesn't contain.
|
|
26
|
+
- A new description: your AI dev team, with you in charge.
|
|
27
|
+
|
|
28
|
+
## 0.1.0 (2026-10-11)
|
|
29
|
+
|
|
30
|
+
The first release:
|
|
31
|
+
- an Architect per module that designs it with you, Developers that build it one feature
|
|
32
|
+
at a time, and a Manager that reviews their work, each in its own OpenShell sandbox;
|
|
33
|
+
- a daemon that runs the loop from an event log, and picks up where it stopped after a
|
|
34
|
+
restart;
|
|
35
|
+
- one pull request per plan, opened by a GitHub App with no access to workflows, and CI
|
|
36
|
+
rules that protect the design and its checks;
|
|
37
|
+
- read-only AWS for the agents, Terraform plans against the real state, applies only in
|
|
38
|
+
CI, and post-deploy checks;
|
|
39
|
+
- the dashboard, with the Architect's terminal and live transcripts;
|
|
40
|
+
- `harness init`, `harness doctor` with its probe, and `harness recover`.
|
package/README.md
CHANGED
|
@@ -9,13 +9,27 @@ feature is checked in a fresh one, and nothing reaches `main` until you merge it
|
|
|
9
9
|
request. A small daemon runs the loop, following Anthropic's
|
|
10
10
|
[Effective harnesses for long-running agents](https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents).
|
|
11
11
|
|
|
12
|
+
- [How it works](#how-it-works)
|
|
13
|
+
- [Requirements](#requirements)
|
|
14
|
+
- [Install and set up a project](#install-and-set-up-a-project)
|
|
15
|
+
- [Your first module](#your-first-module)
|
|
16
|
+
- [Concepts](#concepts)
|
|
17
|
+
- [The dashboard](#the-dashboard)
|
|
18
|
+
- [Commands](#commands)
|
|
19
|
+
- [Configuration](#configuration)
|
|
20
|
+
- [AWS and Terraform](#aws-and-terraform)
|
|
21
|
+
- [Security](#security)
|
|
22
|
+
- [Usage and costs](#usage-and-costs)
|
|
23
|
+
- [Troubleshooting](#troubleshooting)
|
|
24
|
+
- [Development](#development)
|
|
25
|
+
|
|
12
26
|
## How it works
|
|
13
27
|
|
|
14
|
-
|
|
15
|
-
interactive) in its terminal. The Architect
|
|
16
|
-
|
|
17
|
-
reach `main` as an `Arch:` pull request that
|
|
18
|
-
module, feature by feature:
|
|
28
|
+
You start a module in the dashboard, and talk with its Architect (Claude Code,
|
|
29
|
+
interactive) in its terminal. The Architect asks you questions until the need is clear,
|
|
30
|
+
writes proposals until you accept one, then turns it into plans, features, checks and
|
|
31
|
+
owner actions. These reach `main` as an `Arch:` pull request that you merge. Then the
|
|
32
|
+
daemon builds the module, feature by feature:
|
|
19
33
|
- it starts a Developer (Claude Code) in a fresh sandbox, with a 2-attempt budget and a
|
|
20
34
|
deadline;
|
|
21
35
|
- it checks the result in another fresh sandbox, which has no agent;
|
|
@@ -27,8 +41,7 @@ module, feature by feature:
|
|
|
27
41
|
and the Manager submits it;
|
|
28
42
|
- the daemon pushes the plan branch and opens one pull request for it, as a GitHub App
|
|
29
43
|
with no access to workflows. CI checks the PR with the rules of the base branch. If CI
|
|
30
|
-
fails, the Manager diagnoses it and a Developer fixes it, twice at most.
|
|
31
|
-
merges.
|
|
44
|
+
fails, the Manager diagnoses it and a Developer fixes it, twice at most. You merge.
|
|
32
45
|
|
|
33
46
|
Infrastructure features take the same path. Agents see AWS through a read-only role
|
|
34
47
|
whose requests the OpenShell proxy signs, so they plan against the real state but never
|
|
@@ -44,13 +57,23 @@ without pull requests.
|
|
|
44
57
|
|
|
45
58
|
## Requirements
|
|
46
59
|
|
|
47
|
-
- Linux, Node 24 or later, git, Docker, and `gh` logged in as
|
|
48
|
-
|
|
49
|
-
-
|
|
50
|
-
|
|
60
|
+
- Linux, Node 24 or later, git, Docker, and `gh` logged in as an admin of your
|
|
61
|
+
project's repository.
|
|
62
|
+
- OpenShell 0.1.2 with a running gateway. The harness is tested with this version only,
|
|
63
|
+
and `harness doctor` refuses others.
|
|
64
|
+
- Claude Code on this machine, with a Claude subscription: its binary goes into the
|
|
65
|
+
sandbox image, and `claude setup-token` gives the agents' token. Access with an API
|
|
66
|
+
key is untested.
|
|
51
67
|
- For terminals in the dashboard: `python3`, `make` and `g++`, to build node-pty at
|
|
52
68
|
install. Without them, everything else works, and `harness attach` opens the
|
|
53
69
|
Architect's terminal.
|
|
70
|
+
- On GitHub, for pull requests:
|
|
71
|
+
- **An organization** that owns your project's repository: `harness github-app
|
|
72
|
+
create` creates the harness's App in an organization. A free one works.
|
|
73
|
+
- **Rulesets** protect `main`. GitHub offers them on private repositories only with a
|
|
74
|
+
paid plan, such as Team for an organization; public repositories get them for free.
|
|
75
|
+
Without them, nothing on GitHub makes code wait for your review before it reaches
|
|
76
|
+
`main`, and `harness doctor --probe` reports it.
|
|
54
77
|
|
|
55
78
|
## Install and set up a project
|
|
56
79
|
|
|
@@ -73,7 +96,7 @@ ones, and prints the setup checklist:
|
|
|
73
96
|
3. The GitHub App, once per machine: `harness github-app create --org <org>`, then install
|
|
74
97
|
it on the project repositories only.
|
|
75
98
|
4. The repository rules: `harness github-setup --repo <org>/<repo>`, with your own `gh`.
|
|
76
|
-
5. AWS, for infrastructure: see
|
|
99
|
+
5. AWS, for infrastructure: see [AWS and Terraform](#aws-and-terraform).
|
|
77
100
|
6. Review the files `init` wrote, commit them, and push them to `main` yourself: the App
|
|
78
101
|
may not touch workflows.
|
|
79
102
|
7. `harness doctor --probe`.
|
|
@@ -81,7 +104,72 @@ ones, and prints the setup checklist:
|
|
|
81
104
|
`harness doctor` checks each step, and says what to do about what fails. `--probe` tries,
|
|
82
105
|
from a sandbox like an agent's and from outside, everything that must be refused: AWS
|
|
83
106
|
writes and secret reads, other projects' state, `git push`, unlisted hosts and the
|
|
84
|
-
metadata address, a workflow pushed by the App, a merge without
|
|
107
|
+
metadata address, a workflow pushed by the App, a merge without your approval.
|
|
108
|
+
|
|
109
|
+
`harness init` lets the sandboxes reach only the `core` network bundle. Add your
|
|
110
|
+
stack's bundles, such as `node` for npm, in `harness/config.json` (see
|
|
111
|
+
[Configuration](#configuration)).
|
|
112
|
+
|
|
113
|
+
## Your first module
|
|
114
|
+
|
|
115
|
+
With `harness up` running and the dashboard open:
|
|
116
|
+
|
|
117
|
+
1. **Start it.** In **Project**, give the module an id and a line about what it is for,
|
|
118
|
+
then **Start**. An Architect starts in its own sandbox. From a terminal:
|
|
119
|
+
`harness module new greeter "A greeting command-line tool"`.
|
|
120
|
+
2. **Talk it through.** In **Agents**, open the Architect's terminal and say what you
|
|
121
|
+
need. It reads the repository, asks its questions, then writes a proposal: the
|
|
122
|
+
problem, the design, the plans it expects, and what only you can do. From a terminal:
|
|
123
|
+
`harness attach greeter`, and detach with Ctrl-P, then Ctrl-Q.
|
|
124
|
+
3. **Accept a proposal.** **Needs you** shows it. Accept it, or ask for changes in the
|
|
125
|
+
terminal, and the Architect writes the next version. Once you accept one, the
|
|
126
|
+
dashboard prompts the Architect, which turns it into plans, features and checks, and
|
|
127
|
+
the daemon opens an `Arch:` pull request with them. Its new checks run once first: they should fail,
|
|
128
|
+
since nothing is built yet, and any that pass are flagged.
|
|
129
|
+
4. **Merge the design.** Review the Arch PR on GitHub, and merge it with a merge commit
|
|
130
|
+
once CI passes. The build starts.
|
|
131
|
+
5. **Watch the build.** In **Agents**, each Developer's session and the Manager's are
|
|
132
|
+
live transcripts. **Project** shows each feature's attempts, checks and commits.
|
|
133
|
+
6. **Answer what comes up.** A question from the Manager, or a feature that ran out of
|
|
134
|
+
attempts, appears in **Needs you** with a form or buttons.
|
|
135
|
+
7. **Merge the plan.** When all of a plan's features are accepted and its checks pass
|
|
136
|
+
again, the daemon opens one `Plan:` pull request. Merge it with a merge commit once
|
|
137
|
+
CI passes. The next plan starts.
|
|
138
|
+
|
|
139
|
+
You can also write a module's design by hand, on `main`: the daemon builds whatever
|
|
140
|
+
plans it finds there (see [Configuration](#configuration)).
|
|
141
|
+
|
|
142
|
+
## Concepts
|
|
143
|
+
|
|
144
|
+
- **Module:** a unit of design, with one Architect and one accepted proposal, then its
|
|
145
|
+
plans. Modules are built one after another.
|
|
146
|
+
- **Proposal:** the Architect's design document, in versions v1, v2 and so on. You
|
|
147
|
+
accept one.
|
|
148
|
+
- **Plan:** features that ship together as one pull request. A module's plans are built
|
|
149
|
+
in order.
|
|
150
|
+
- **Feature:** one change that a Developer can make in one attempt, with at least one
|
|
151
|
+
check. Its definition is frozen once its design is merged.
|
|
152
|
+
- **Check:** a shell command, run from the repository root, that exits 0 when the feature
|
|
153
|
+
works. `pre-merge` checks run on every attempt and in CI; `post-deploy` checks run
|
|
154
|
+
after an infrastructure deploy.
|
|
155
|
+
- **Attempt:** one Developer session on one feature, in a fresh sandbox, with a deadline
|
|
156
|
+
(90 minutes) and a budget (2 attempts per feature). A failed attempt's successor
|
|
157
|
+
starts with the Manager's guidance. Once the budget is spent, the feature is escalated
|
|
158
|
+
to you.
|
|
159
|
+
- **Owner actions:** what only you can do, such as creating an account, approving a
|
|
160
|
+
policy or setting a secret. A **gate** holds back the plans or features it blocks
|
|
161
|
+
until you mark it done; a **task** holds nothing back.
|
|
162
|
+
- **Protected files:** the design and its checks, CI, and what agents read as
|
|
163
|
+
instructions: `harness/`, `.github/`, `.claude/`, `CLAUDE.md`, `AGENTS.md`,
|
|
164
|
+
`ARCHITECTURE.md`, `init.sh`, tool configs and the npm scripts that checks run. A
|
|
165
|
+
`Plan:` or `Fix:` pull request may not change them: CI's `verify` job, run with the
|
|
166
|
+
base branch's rules, fails it. They change through an `Arch:` pull request, or your own
|
|
167
|
+
commits. `harness/lock.json` holds their hashes.
|
|
168
|
+
- **`progress.txt`:** one line per accepted feature and satisfied gate, committed with
|
|
169
|
+
the work. It only grows.
|
|
170
|
+
- **Brakes:** counters and deadlines, not judgment. Three failed attempts in a row, a
|
|
171
|
+
Manager that leaves an item waiting for 40 minutes, or CI still failing after two fix
|
|
172
|
+
rounds pause the pipeline until you resume it.
|
|
85
173
|
|
|
86
174
|
## The dashboard
|
|
87
175
|
|
|
@@ -95,38 +183,256 @@ behind a token. Its views:
|
|
|
95
183
|
transcripts, a message box for the Manager, and a shell in any agent's sandbox.
|
|
96
184
|
- **Events:** the event log, live, with a filter.
|
|
97
185
|
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
186
|
+
The header pauses and resumes the pipeline: a pause lets what runs finish, and starts
|
|
187
|
+
nothing new.
|
|
188
|
+
|
|
189
|
+
## Commands
|
|
101
190
|
|
|
102
|
-
|
|
191
|
+
Run them in the project's folder, or add `--dir <project>`. `harness --help` lists them
|
|
192
|
+
too.
|
|
103
193
|
|
|
194
|
+
| Command | What it does |
|
|
195
|
+
|---|---|
|
|
196
|
+
| `harness init` | Writes the harness's files into the repository, and prints the setup checklist |
|
|
197
|
+
| `harness doctor [--build-images] [--probe]` | Checks the machine and the project, builds the agents' image, and tries what agents must not do |
|
|
198
|
+
| `harness github-app create --org <org>` | Creates the harness's GitHub App, once per machine |
|
|
199
|
+
| `harness github-app check --repo <org>/<repo>` | Checks that the App can reach the repository |
|
|
200
|
+
| `harness github-setup --repo <org>/<repo>` | Sets merge commits only, and the ruleset on `main`, with your own `gh` |
|
|
201
|
+
| `harness install-ci [--owner <login>]` | Writes the workflow, its CI script and `CODEOWNERS` again, after an upgrade |
|
|
202
|
+
| `harness lock` | Writes `harness/lock.json`, the hashes of the protected files |
|
|
203
|
+
| `harness up`, `harness down` | Starts the daemon and the dashboard, or stops the daemon; sandboxes keep running |
|
|
204
|
+
| `harness status` | Plans, features, modules and open questions, in the terminal |
|
|
205
|
+
| `harness module new <id> <title>` | Starts a module and its Architect |
|
|
206
|
+
| `harness attach <module>` | Opens the Architect's terminal here; detach with Ctrl-P, then Ctrl-Q |
|
|
207
|
+
| `harness accept <module> <vN>` | Accepts the Architect's proposal vN |
|
|
208
|
+
| `harness actions`, `harness done <module>/<id>` | Lists the owner actions, or marks one done |
|
|
209
|
+
| `harness answer <request-id> <text>` | Answers a question from the Manager |
|
|
210
|
+
| `harness message <text>` | Writes to the Manager of the module being built |
|
|
211
|
+
| `harness retry <module>/<plan>/<feature>` | Gives an escalated feature one more attempt; `--note` adds guidance |
|
|
212
|
+
| `harness resume` | Lifts a pause |
|
|
213
|
+
| `harness recover` | Rebuilds a lost `.harness/` from `main` and GitHub |
|
|
214
|
+
|
|
215
|
+
## Configuration
|
|
216
|
+
|
|
217
|
+
`harness/config.json` holds the project's settings. It is a protected file: it changes
|
|
218
|
+
through an `Arch:` pull request, or a commit of your own on `main`.
|
|
219
|
+
|
|
220
|
+
```json
|
|
221
|
+
{
|
|
222
|
+
"harnessVersion": "0.1.1",
|
|
223
|
+
"project": { "name": "shop", "repo": "<org>/shop", "bot": "<org>-harness[bot]" },
|
|
224
|
+
"roles": {
|
|
225
|
+
"architect": { "model": "claude-opus-5-5", "effort": "max" },
|
|
226
|
+
"manager": { "model": "claude-opus-5-5", "effort": "high" },
|
|
227
|
+
"developer": { "model": "claude-opus-5-5", "effort": "xhigh", "attemptTimeoutMinutes": 90, "maxAttempts": 2 }
|
|
228
|
+
},
|
|
229
|
+
"network": { "bundles": ["core", "node"], "extraHosts": [] },
|
|
230
|
+
"checks": [{ "id": "unit", "run": "npm test", "phase": "pre-merge" }]
|
|
231
|
+
}
|
|
104
232
|
```
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
233
|
+
|
|
234
|
+
| Field | Default | What it sets |
|
|
235
|
+
|---|---|---|
|
|
236
|
+
| `project.name` | the folder's name, from `init` | Names the sandboxes and the dashboard |
|
|
237
|
+
| `project.repo`, `project.bot` | none | The GitHub repository, and the App's bot login. Without them, the project stays local. |
|
|
238
|
+
| `roles.<role>.model` | `claude-opus-5-5` | The model of the Architect, the Manager and the Developers |
|
|
239
|
+
| `roles.<role>.effort` | `max`, `high`, `xhigh` | Claude Code's `--effort`; `null` for a model without effort levels, such as Haiku |
|
|
240
|
+
| `roles.developer.attemptTimeoutMinutes` | 90 | An attempt's deadline |
|
|
241
|
+
| `roles.developer.maxAttempts` | 2 | Attempts per feature before it is escalated to you |
|
|
242
|
+
| `network.bundles` | `["core"]` | The hosts sandboxes reach, by bundle: `core`, `node`, `python`, `aws`, `terraform`, `go`, `rust`, `jvm`, `opentofu`, `browsers`. The Architect gets them all. |
|
|
243
|
+
| `network.extraHosts` | `[]` | More hosts, read-only |
|
|
244
|
+
| `checks` | `[]` | Checks that every feature must keep passing, such as `npm run lint` |
|
|
245
|
+
| `protect.files`, `protect.jsonFields` | none | More protected files, and protected JSON fields by file, like `{"tsconfig.json": ["/compilerOptions/strict"]}` |
|
|
246
|
+
| `sandbox.image` | `razzou-harness/agent:0.1.0` | The agents' image, which `harness doctor --build-images` builds |
|
|
247
|
+
| `sandbox.claudeProvider` | `claude-sub` | The OpenShell provider that holds your Claude token |
|
|
248
|
+
| `sandbox.env` | `{}` | Environment variables for every sandbox |
|
|
249
|
+
| `github.requiredChecks` | `["verify", "checks"]` | The CI jobs a pull request must pass; `tf-plan` is added with Terraform |
|
|
250
|
+
| `brakes.consecutiveFailures` | 3 | Failed attempts in a row that pause the pipeline |
|
|
251
|
+
| `brakes.managerTurnMinutes` | 20 | How long an item waits for the Manager before a reminder; as long again, and the pipeline pauses |
|
|
252
|
+
| `brakes.ciFixRounds`, `brakes.deployFixRounds` | 2, 2 | Fix rounds after CI fails, and `Fix:` pull requests after a deploy fails |
|
|
253
|
+
| `dashboard.port` | 4777 | The dashboard's port, or the next free one |
|
|
254
|
+
| `aws`, `terraform` | none | See [AWS and Terraform](#aws-and-terraform) |
|
|
255
|
+
|
|
256
|
+
A module's design lives on `main` too, whether the Architect wrote it or you did:
|
|
257
|
+
|
|
258
|
+
```
|
|
259
|
+
harness/modules/<module>/plans/P1.json a plan and its features
|
|
260
|
+
harness/modules/<module>/checks/... the scripts its checks run, with synthetic data
|
|
261
|
+
harness/modules/<module>/owner-actions.json gates and tasks for you, if any
|
|
262
|
+
```
|
|
263
|
+
|
|
264
|
+
A plan, `harness/modules/greeter/plans/P1.json`:
|
|
265
|
+
|
|
266
|
+
```json
|
|
267
|
+
{
|
|
268
|
+
"id": "P1",
|
|
269
|
+
"module": "greeter",
|
|
270
|
+
"title": "Greeting CLI",
|
|
271
|
+
"goal": "A command that greets a name.",
|
|
272
|
+
"features": [
|
|
273
|
+
{
|
|
274
|
+
"id": "F01",
|
|
275
|
+
"title": "Greet by name",
|
|
276
|
+
"description": "src/greet.js prints \"Hello, <name>!\" for its first argument, and a usage line with exit code 1 without one.",
|
|
277
|
+
"expected": "`node src/greet.js Ada` prints Hello, Ada!",
|
|
278
|
+
"checks": [
|
|
279
|
+
{ "id": "F01-greet", "run": "node --test harness/modules/greeter/checks/F01.test.mjs", "phase": "pre-merge" }
|
|
280
|
+
]
|
|
281
|
+
}
|
|
282
|
+
]
|
|
283
|
+
}
|
|
108
284
|
```
|
|
109
285
|
|
|
286
|
+
- **Features** may also have `kind` (`app` or `infra`), `dependsOn` (earlier features of
|
|
287
|
+
the same plan), `gates`, and `resources` (where the Developer should look).
|
|
288
|
+
- **Checks** have `run` (a shell command from the repository root, never a `/sandbox/`
|
|
289
|
+
path), `phase` (`pre-merge` or `post-deploy`), `timeoutSec` (600 by default), and
|
|
290
|
+
`needs`: `tfplan` to get a Terraform plan's JSON in `$HARNESS_PLAN_JSON`, `aws` to use
|
|
291
|
+
the read-only AWS role.
|
|
292
|
+
- **Owner actions** look like this, and a `verify` command, if set, runs in a fresh
|
|
293
|
+
sandbox before the action counts as done:
|
|
294
|
+
|
|
295
|
+
```json
|
|
296
|
+
[{ "id": "G1", "kind": "gate", "title": "Approve the retention policy", "blocks": ["P2"],
|
|
297
|
+
"instructions": "Markdown for you.", "verify": null }]
|
|
298
|
+
```
|
|
299
|
+
|
|
110
300
|
## AWS and Terraform
|
|
111
301
|
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
302
|
+
Infrastructure is optional: skip this section for a project without it. With it, agents
|
|
303
|
+
see your AWS account and plan Terraform against the real state, but never change
|
|
304
|
+
anything. They work through a read-only role whose requests the OpenShell proxy signs,
|
|
305
|
+
and Terraform applies only in CI, after you merge.
|
|
306
|
+
|
|
307
|
+
**What you need in AWS.** The harness doesn't create it:
|
|
308
|
+
1. **A state bucket** for Terraform, with this project's state under its own key.
|
|
309
|
+
2. **CI roles, trusted through GitHub's OIDC:** a plan role for pull requests, and an
|
|
310
|
+
apply role trusted only from a GitHub environment named `staging`, which only `main`
|
|
311
|
+
may use.
|
|
312
|
+
3. **The agents' read-only role**, named `harness-<project>-view`: `harness doctor`
|
|
313
|
+
expects the `harness-` prefix and the `-view` suffix. It has:
|
|
314
|
+
- the AWS managed policies `ViewOnlyAccess` and `SecurityAudit`;
|
|
315
|
+
- list access to the state bucket, read access to this project's state object, and
|
|
316
|
+
`kms:Decrypt` on the state's key through S3;
|
|
317
|
+
- an explicit deny on data and secrets, which no update to the managed policies can
|
|
318
|
+
undo: `s3:GetObject*` on anything but the state object; the DynamoDB item reads
|
|
319
|
+
(`GetItem`, `BatchGetItem`, `Query`, `Scan`, the PartiQL calls, `GetRecords`);
|
|
320
|
+
`secretsmanager:GetSecretValue` and `BatchGetSecretValue`; the SSM parameter reads;
|
|
321
|
+
`kms:Decrypt` on any other key; the CloudWatch Logs content reads (`GetLogEvents`,
|
|
322
|
+
`FilterLogEvents`, `StartQuery`, `GetQueryResults`, `StartLiveTail`);
|
|
323
|
+
`sqs:ReceiveMessage`, `kinesis:GetRecords`, `athena:GetQueryResults` and
|
|
324
|
+
`rds-data:*`; the Cognito user reads; the SNS subscription reads, which hold emails
|
|
325
|
+
and phone numbers; `ec2:GetPasswordData` and `GetConsoleOutput`; and the ECR image
|
|
326
|
+
content (`BatchGetImage`, `GetDownloadUrlForLayer`);
|
|
327
|
+
- a trust policy that lets only the gateway user assume it.
|
|
328
|
+
4. **A gateway user**, whose only permission is `sts:AssumeRole` on the view roles. Its
|
|
329
|
+
access key goes into OpenShell, never into a repository or a sandbox.
|
|
330
|
+
5. **A permissions boundary** on the CI apply role that denies changes to the
|
|
331
|
+
`harness-*` roles and to the gateway user, so your project's own Terraform can't widen
|
|
332
|
+
the agents' access.
|
|
333
|
+
|
|
334
|
+
**Then, in the project:**
|
|
335
|
+
1. In `harness/config.json`, add `aws` and `terraform`:
|
|
117
336
|
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
337
|
+
```json
|
|
338
|
+
"aws": { "region": "us-east-1", "provider": "aws-view-shop", "services": ["sts", "s3", "iam", "sqs"] },
|
|
339
|
+
"terraform": { "dir": "iac", "backendConfig": "backend-configs/staging.hcl", "varFile": "env/staging.tfvars" }
|
|
340
|
+
```
|
|
341
|
+
|
|
342
|
+
`services` lists the AWS services the project uses: their endpoints become reachable,
|
|
343
|
+
and signed. Keep the `terraform` paths as they are: the CI workflow uses the same
|
|
344
|
+
ones. `harness github-setup` then requires `tf-plan` as well.
|
|
122
345
|
2. Set the repository variables the workflow reads: `AWS_REGION_STAGING`,
|
|
123
346
|
`AWS_ROLE_PLAN_ARN_STAGING` and `AWS_ROLE_APPLY_ARN_STAGING`.
|
|
124
|
-
3. `harness doctor` creates the OpenShell AWS provider: it asks for the
|
|
347
|
+
3. `harness doctor` creates the OpenShell AWS provider: it asks for the view role's ARN,
|
|
125
348
|
and for the gateway user's key at a hidden prompt. Only OpenShell holds the key.
|
|
349
|
+
4. `harness doctor --probe` checks the result: AWS must answer as the view role, and
|
|
350
|
+
refuse a write, a secret and another project's state.
|
|
126
351
|
|
|
127
352
|
A check with `"needs": ["tfplan"]` finds the plan's JSON in `$HARNESS_PLAN_JSON`. A
|
|
128
353
|
`post-deploy` check with `"needs": ["aws"]` runs after the apply, as the read-only role.
|
|
129
354
|
|
|
355
|
+
Agents can read the Terraform state, so keep secrets out of it: Terraform creates a
|
|
356
|
+
secret, but you or a rotation set its value; a database manages its own password (for
|
|
357
|
+
RDS, `manage_master_user_password`); anything else uses write-only arguments.
|
|
358
|
+
|
|
359
|
+
## Security
|
|
360
|
+
|
|
361
|
+
What agents can't do, and what stops them:
|
|
362
|
+
- **Hold credentials.** Sandboxes hold placeholders. The OpenShell proxy adds the Claude
|
|
363
|
+
token on `api.anthropic.com` only, and signs AWS requests with the read-only role.
|
|
364
|
+
- **Reach the host.** Each agent runs in its own container, with no host folder mounted;
|
|
365
|
+
code moves in and out as git bundles. The host never runs code or git hooks that an
|
|
366
|
+
agent wrote.
|
|
367
|
+
- **Write to GitHub.** Only the daemon talks to GitHub, as an App with no access to
|
|
368
|
+
workflows or settings. Sandboxes may clone and fetch, never push.
|
|
369
|
+
- **Change the rules.** Checks, CI and the agents' instructions are protected files: a
|
|
370
|
+
Developer's work can't change them, and CI checks that with the base branch's rules.
|
|
371
|
+
They change only through an Arch PR you merge, or your own commits.
|
|
372
|
+
- **Change AWS.** A read-only role with explicit denies, and applies only in CI after
|
|
373
|
+
your merge.
|
|
374
|
+
- **Go anywhere online.** Sandboxes reach listed hosts, read-only. Other pages go
|
|
375
|
+
through `web_read`: GET only, public addresses only, checked again after every
|
|
376
|
+
redirect.
|
|
377
|
+
- **Decide alone.** Code runs the loop: deadlines, attempt budgets, checks in fresh
|
|
378
|
+
sandboxes and your merges are enforced by the daemon, not left to the agents.
|
|
379
|
+
|
|
380
|
+
What remains possible, by design:
|
|
381
|
+
- an agent could send code out in the URL of an allowed GET request;
|
|
382
|
+
- the read-only role shows infrastructure metadata, S3 object names and Lambda
|
|
383
|
+
environment variables: keep user identifiers out of object names, and secrets out of
|
|
384
|
+
Lambda environment variables;
|
|
385
|
+
- the Architect writes the checks: review Arch PRs. Checks that pass before anything is
|
|
386
|
+
built are flagged in them.
|
|
387
|
+
|
|
388
|
+
`harness doctor --probe` tries, from sandboxes set up like the agents', what must be
|
|
389
|
+
refused. Run it after setup, and after upgrading the harness, OpenShell or Claude Code.
|
|
390
|
+
To report a vulnerability, see `SECURITY.md` in the source repository.
|
|
391
|
+
|
|
392
|
+
## Usage and costs
|
|
393
|
+
|
|
394
|
+
- The agents run on your Claude subscription, so their usage counts against your plan's
|
|
395
|
+
limits. When Claude Code reports that a limit is reached, the attempt under way
|
|
396
|
+
doesn't count, and the pipeline pauses for an hour, then goes on.
|
|
397
|
+
- By default, all three roles run Opus 5.5 at a high effort. To try the harness, or to
|
|
398
|
+
use less of your plan, set each role's `model` to Haiku 4.5
|
|
399
|
+
(`claude-haiku-4-5-20251001`) and its `effort` to `null`.
|
|
400
|
+
- With Haiku 4.5 in our test runs, a Developer attempt took under 90 seconds and 17 to
|
|
401
|
+
23 turns: 150,000 to 310,000 input tokens, mostly read from the cache, and 2,000 to
|
|
402
|
+
6,000 output tokens. Claude Code estimated $0.08 to $0.12 per attempt at API prices.
|
|
403
|
+
The Architect and the Manager use more, over their long sessions.
|
|
404
|
+
- The dashboard shows each session's model calls, tokens and Claude Code's cost estimate.
|
|
405
|
+
|
|
406
|
+
## Troubleshooting
|
|
407
|
+
|
|
408
|
+
`harness doctor` names what fails and what to do. The usual cases:
|
|
409
|
+
|
|
410
|
+
| `harness doctor` says | What to do |
|
|
411
|
+
|---|---|
|
|
412
|
+
| `Docker: … permission denied` | Add yourself to the `docker` group, then log in again |
|
|
413
|
+
| `Tailscale: an exit node is active`, or `container network` fails | An exit node can cut the containers' network: turn it off |
|
|
414
|
+
| `OpenShell: … pinned to 0.1.2` | Install OpenShell 0.1.2 |
|
|
415
|
+
| `OpenShell gateway` fails | `openshell gateway start` |
|
|
416
|
+
| `image: … has Claude Code X, this machine Y` | `harness doctor --build-images`, after each Claude Code update |
|
|
417
|
+
| `terminals: node-pty is missing` | Install `python3`, `make` and `g++`, then reinstall with `--allow-scripts=node-pty`. `harness attach` works meanwhile. |
|
|
418
|
+
| `Claude provider: … is missing` | `claude setup-token`, then `harness doctor` in a terminal: it asks for the token |
|
|
419
|
+
| `GitHub App: … 404` | Install the App on the repository, from the link `harness github-app create` printed, or the organization's settings |
|
|
420
|
+
| `ruleset: …` fails | `harness github-setup --repo <org>/<repo>`. A private repository needs a paid GitHub plan for rulesets. |
|
|
421
|
+
| `.github/… differs from this version of the harness` | `harness install-ci`, then commit the files yourself |
|
|
422
|
+
|
|
423
|
+
While the daemon runs:
|
|
424
|
+
- **The pipeline is paused.** **Needs you** says why: attempts that failed in a row, a
|
|
425
|
+
Manager that didn't act, CI still failing after its fix rounds, a usage limit, or a
|
|
426
|
+
broken environment. Fix the cause, then **Resume**.
|
|
427
|
+
- **A feature is escalated.** It used its attempts. In **Needs you**, give it one more
|
|
428
|
+
attempt with your guidance, send it to the Architect (the design may change), or pause.
|
|
429
|
+
- **The dashboard asks for its token.** Each `harness up` prints a new link: open that
|
|
430
|
+
one.
|
|
431
|
+
- **A merge takes minutes to show up.** The daemon polls GitHub: every minute, then every
|
|
432
|
+
five once nothing has changed for ten minutes.
|
|
433
|
+
- **Closing the terminal stopped the daemon.** Run `harness up` again: it picks up from
|
|
434
|
+
its event log, and the agents kept working meanwhile.
|
|
435
|
+
|
|
130
436
|
## Development
|
|
131
437
|
|
|
132
438
|
```sh
|
|
@@ -138,6 +444,11 @@ npm pack # a package of these sources: npm install -g ./razzou-harnes
|
|
|
138
444
|
npm run harness -- up --dir path/to/project # from the sources; run `npm run build` once for the dashboard
|
|
139
445
|
```
|
|
140
446
|
|
|
447
|
+
A GitHub release tagged with the package's version, such as `v0.1.2`, publishes it to
|
|
448
|
+
npm. The release workflow checks the tag against `package.json`, runs the typecheck and
|
|
449
|
+
the tests, then publishes with npm's trusted publishing, so each version shows where it
|
|
450
|
+
was built.
|
|
451
|
+
|
|
141
452
|
## License
|
|
142
453
|
|
|
143
454
|
MIT
|
package/dist/design/schemas.js
CHANGED
|
@@ -112,7 +112,7 @@ export const ConfigSchema = z.object({
|
|
|
112
112
|
*/
|
|
113
113
|
aws: z
|
|
114
114
|
.object({
|
|
115
|
-
region: z.string().regex(/^[a-z]{2}(-[a-z]+)+-\d$/, 'an AWS region, like
|
|
115
|
+
region: z.string().regex(/^[a-z]{2}(-[a-z]+)+-\d$/, 'an AWS region, like us-east-1'),
|
|
116
116
|
/** The OpenShell provider holding the view role, like "aws-view-shop". */
|
|
117
117
|
provider: z.string().min(1),
|
|
118
118
|
/** The services the project uses: their endpoints are reachable, and signed. */
|
package/dist/setup/doctor.js
CHANGED
|
@@ -60,7 +60,7 @@ async function machine(options, image, report) {
|
|
|
60
60
|
report(git?.code === 0 ? ok('git', git.stdout.trim()) : fail('git', 'not found', 'install git'));
|
|
61
61
|
const docker = await run('docker', ['info', '--format', '{{.ServerVersion}}']).catch(() => undefined);
|
|
62
62
|
report(docker?.code === 0 ? ok('Docker', `server ${docker.stdout.trim()}`) : fail('Docker', firstLine(docker?.stderr) || 'not found', 'install Docker, and add yourself to the docker group'));
|
|
63
|
-
// A
|
|
63
|
+
// A VPN exit node, such as Tailscale's, can cut the containers' network (§17.3).
|
|
64
64
|
const tailscale = await run('tailscale', ['status', '--json']).catch(() => undefined);
|
|
65
65
|
if (tailscale?.code === 0 && JSON.parse(tailscale.stdout).ExitNodeStatus) {
|
|
66
66
|
report(warn('Tailscale', 'an exit node is active', 'containers can lose their network: turn the exit node off if sandboxes can\'t connect'));
|
|
@@ -172,7 +172,7 @@ async function aws(options, config, report) {
|
|
|
172
172
|
if (imported.code !== 0)
|
|
173
173
|
return report(fail('AWS profile', firstLine(imported.stderr), `import ${AWS_PROFILE_URL}`));
|
|
174
174
|
}
|
|
175
|
-
console.log(`AWS provider ${provider} is missing.
|
|
175
|
+
console.log(`AWS provider ${provider} is missing. It needs the project's read-only role, and the gateway user's key (see the README's AWS section).`);
|
|
176
176
|
const role = await ask('The project\'s view role ARN: ');
|
|
177
177
|
const keyId = await ask('The harness-gateway user\'s AccessKeyId: ');
|
|
178
178
|
const secret = await askSecret('Its SecretAccessKey (hidden): ');
|
package/dist/setup/init.js
CHANGED
|
@@ -12,7 +12,7 @@ export const CHECKLIST = `Setup checklist (\`harness doctor\` checks it):
|
|
|
12
12
|
3. GitHub App: \`harness github-app create --org <org>\` once per machine, then install it
|
|
13
13
|
on this repository only.
|
|
14
14
|
4. Repository rules: \`harness github-setup --repo <owner/name>\`, with your own gh login.
|
|
15
|
-
5. AWS, for infrastructure:
|
|
15
|
+
5. AWS, for infrastructure: create the roles the README lists, add \`aws\` and
|
|
16
16
|
\`terraform\` to harness/config.json, then \`harness doctor\` stores the gateway key.
|
|
17
17
|
6. Project files: review what \`harness init\` wrote, commit it and push it to main.
|
|
18
18
|
7. Probe: \`harness doctor --probe\`.
|
package/package.json
CHANGED
|
@@ -1,8 +1,29 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "razzou-harness",
|
|
3
|
-
"version": "0.1.
|
|
3
|
+
"version": "0.1.2",
|
|
4
4
|
"description": "Your AI dev team, with you in charge: sandboxed Claude Code agents design, build and review your project, and you merge.",
|
|
5
5
|
"license": "MIT",
|
|
6
|
+
"author": "razzou",
|
|
7
|
+
"homepage": "https://github.com/razzou-labs/razzou-harness#readme",
|
|
8
|
+
"repository": {
|
|
9
|
+
"type": "git",
|
|
10
|
+
"url": "git+https://github.com/razzou-labs/razzou-harness.git"
|
|
11
|
+
},
|
|
12
|
+
"bugs": {
|
|
13
|
+
"url": "https://github.com/razzou-labs/razzou-harness/issues"
|
|
14
|
+
},
|
|
15
|
+
"keywords": [
|
|
16
|
+
"claude",
|
|
17
|
+
"claude-code",
|
|
18
|
+
"ai-agents",
|
|
19
|
+
"coding-agents",
|
|
20
|
+
"agent-harness",
|
|
21
|
+
"openshell",
|
|
22
|
+
"sandbox",
|
|
23
|
+
"terraform",
|
|
24
|
+
"aws",
|
|
25
|
+
"github-app"
|
|
26
|
+
],
|
|
6
27
|
"type": "module",
|
|
7
28
|
"engines": {
|
|
8
29
|
"node": ">=24"
|
|
@@ -18,7 +39,8 @@
|
|
|
18
39
|
"shim",
|
|
19
40
|
"images",
|
|
20
41
|
"templates",
|
|
21
|
-
"src/ci/harness-ci.ts"
|
|
42
|
+
"src/ci/harness-ci.ts",
|
|
43
|
+
"CHANGELOG.md"
|
|
22
44
|
],
|
|
23
45
|
"scripts": {
|
|
24
46
|
"build": "node -e \"require('node:fs').rmSync('dist', { recursive: true, force: true })\" && tsc -p tsconfig.build.json && vite build --config dashboard/vite.config.ts",
|