razzou-harness 0.4.0 → 0.4.2

package/CHANGELOG.md

@@ -3,6 +3,55 @@
 The notable changes in each release of razzou-harness. Before 1.0, a new minor version
 may change how a project is set up, and its notes say how.
 
+## 0.4.2 (2026-11-08)
+
+**Upgrading from 0.4.1:** upgrade OpenShell to 0.1.7 first, from its Debian package
+(`sudo dpkg -i openshell_0.1.7-1_amd64.deb`); the gateway keeps its providers and
+profiles. Then install the new version, and rebuild the agents' image with `harness
+doctor --build-images`: its tag follows the harness's version. A project's AWS services
+stay as its `harness/config.json` lists them; for every one, set `"services": ["all"]`
+under `aws`.
+
+- **OpenShell 0.1.7.** The harness is pinned to it, and tested with it: the probe passes
+  with every bundle and every AWS service, and the Architect's terminal and the shell
+  in a sandbox work. The doctor's and setup's fix for a stopped gateway now starts its
+  systemd user service: OpenShell's CLI has no command for it.
+- **Every AWS service by default.** `"all"` in `aws.services` is every service the AWS
+  SDK knows, about 420 endpoints in a region with the global ones, each signed with the
+  view role under its own signing name, as OpenShell requires. It is the default, and
+  `harness setup` no longer asks which services the project uses: the view role alone
+  decides what agents read. The table comes from the AWS SDK's own data
+  (`scripts/aws-services.ts` writes it).
+- **AWS reads with `GET` work.** OpenShell's automatic `sigv4` mode signs a request
+  with neither a body nor `x-amz-content-sha256` as `UNSIGNED-PAYLOAD`, which only S3
+  accepts ([NVIDIA/OpenShell#5375](https://github.com/NVIDIA/OpenShell/issues/5375)):
+  every service that reads with `GET`, such as Lambda, API Gateway, EKS, Route 53,
+  CloudFront and Bedrock, answered `SignatureDoesNotMatch` in the sandboxes, and a
+  `terraform plan` there failed on their resources. Every AWS endpoint but S3's is now
+  signed with `sigv4:body`, the hash of the request's body, empty for a `GET`: the AWS
+  CLI's calls to those services and a `terraform plan` reading them work.
+
+## 0.4.1 (2026-11-08)
+
+**Upgrading a project from 0.4.0:** install the new version, then rebuild the agents'
+image with `harness doctor --build-images`: its tag follows the harness's version. A
+project's network stays as its `harness/config.json` lists it; for every bundle, set
+`"bundles": ["all"]`.
+
+- **Every network bundle by default.** `"all"` in `network.bundles` stands for every
+  bundle, those later versions add included, and it is the default: `harness init`
+  writes it, and `harness setup` no longer asks which bundles the stack needs. The
+  sandboxes of every role reach every bundle's package registries and documentation,
+  read-only. A project narrows it by listing bundles. An unknown bundle now fails when
+  the config is read, not when a sandbox is made.
+- **Third-party APIs**, in `network.apis`: the hosts the code may call with any method,
+  like a payment provider's test API. With `"key": "STRIPE_API_KEY"`, `harness doctor`
+  asks for the API's key at a hidden prompt and stores it in an OpenShell provider of
+  its own; every sandbox gets it, and holds a placeholder in that variable, which
+  OpenShell replaces with the key on requests to that host alone, in headers and URLs,
+  not in request bodies. While a key isn't stored on the machine, no new attempt
+  starts: the pipeline pauses and says so.
+
 ## 0.4.0 (2026-11-08)
 
 **Upgrading a project from 0.3.x:** install the new version, then rebuild the agents'

package/README.md

@@ -106,7 +106,7 @@ without pull requests.
 
 - Linux, Node 24 or later, git, Docker, and `gh` logged in as an admin of your
   project's repository.
-- OpenShell 0.1.2 with a running gateway. The harness is tested with this version only,
+- OpenShell 0.1.7 with a running gateway. The harness is tested with this version only,
   and `harness doctor` refuses others.
 - Claude Code on this machine, with a Claude subscription: its binary goes into the
   sandbox image, and `claude setup-token` gives the agents' token. Access with an API
@@ -134,9 +134,9 @@ such as the Harness.io CLI, comes first on the PATH.
 
 `harness setup` does every step it can and asks only for what is yours: a `gh` login,
 the token from `claude setup-token` at a hidden prompt, two clicks on GitHub for the
-App, since GitHub creates and installs Apps only in the browser, three questions (your
-stack's network bundles, AWS or not, the hours of your time), and, with AWS, the role
-ARNs from your bootstrap and the gateway key. In order, it logs you in to `gh`; creates
+App, since GitHub creates and installs Apps only in the browser, two questions (AWS or
+not, the hours of your time), and, with AWS, the role ARNs from your bootstrap and the
+gateway key. In order, it logs you in to `gh`; creates
 the repository on GitHub if the project has no remote; creates the App from its
 manifest when this machine has none, and waits for you to install it; writes the
 harness's files; asks the questions and writes `harness/config.json`, with the
@@ -169,9 +169,10 @@ from a sandbox like an agent's and from outside, everything that must be refused
 writes and secret reads, other projects' state, `git push`, unlisted hosts and the
 metadata address, a workflow pushed by the App, a merge without your approval.
 
-`harness init` lets the sandboxes reach only the `core` network bundle. Add your
-stack's bundles, such as `node` for npm, in `harness/config.json` (see
-[Configuration](#configuration)).
+`harness init` lets the sandboxes reach every network bundle: the package registries
+and documentation of each stack, read-only. To narrow it, list fewer bundles in
+`harness/config.json`; to let the code call a third-party API, add it under
+`network.apis` (see [Configuration](#configuration)).
 
 ## Your first module
 
@@ -432,14 +433,14 @@ through an `Arch:` pull request, or a commit of your own on `main`.
 
 ```json
 {
-  "harnessVersion": "0.4.0",
+  "harnessVersion": "0.4.2",
   "project": { "name": "shop", "repo": "<org>/shop", "bot": "<org>-harness[bot]" },
   "roles": {
     "architect": { "model": "claude-opus-5-5", "effort": "max" },
     "manager": { "model": "claude-opus-5-5", "effort": "high" },
     "developer": { "model": "claude-opus-5-5", "effort": "xhigh", "attemptTimeoutMinutes": 90, "maxAttempts": 2 }
   },
-  "network": { "bundles": ["core", "node"], "extraHosts": [] },
+  "network": { "bundles": ["all"], "apis": [{ "host": "api.stripe.com", "key": "STRIPE_API_KEY" }] },
   "checks": [{ "id": "unit", "run": "npm test", "phase": "pre-merge" }]
 }
 ```
@@ -452,8 +453,9 @@ through an `Arch:` pull request, or a commit of your own on `main`.
 | `roles.<role>.effort` | `max`, `high`, `xhigh` | Claude Code's `--effort`; `null` for a model without effort levels, such as Haiku |
 | `roles.developer.attemptTimeoutMinutes` | 120 | An attempt's deadline |
 | `roles.developer.maxAttempts` | 2 | Attempts per feature before it is escalated to you |
-| `network.bundles` | `["core"]` | The hosts sandboxes reach, by bundle: `core`, `node`, `python`, `aws`, `terraform`, `go`, `rust`, `jvm`, `opentofu`, `browsers`. The Architect gets them all. |
+| `network.bundles` | `["all"]` | The hosts sandboxes reach directly, read-only, by bundle: `core`, `node`, `python`, `aws`, `terraform`, `go`, `rust`, `jvm`, `opentofu`, `browsers`, or `all` for every one, those later versions add included. The Architect gets them all. |
 | `network.extraHosts` | `[]` | More hosts, read-only |
+| `network.apis` | `[]` | Third-party APIs the code may call with any method, like a payment provider's test API: `{ "host": "api.stripe.com" }`, with `"key": "STRIPE_API_KEY"` for one that needs a key. `harness doctor` asks for the key at a hidden prompt and stores it in OpenShell; in a sandbox the variable holds a placeholder, which OpenShell replaces with the key on requests to that host alone, in headers and URLs, not in request bodies. CI gets no key. |
 | `checks` | `[]` | Checks that every feature must keep passing, such as `npm run lint` |
 | `protect.files`, `protect.jsonFields` | none | More protected files, as paths or globs (`scripts/*.sh`, `docs/`, `**/Makefile`), and protected JSON fields by file, like `{"tsconfig.json": ["/compilerOptions/strict"]}` |
 | `sandbox.image` | `razzou-harness/agent:<version>` | The agents' image, tagged with the harness's version, which `harness doctor --build-images` builds |
@@ -556,13 +558,18 @@ and Terraform applies only in CI, after you merge.
 1. In `harness/config.json`, add `aws` and `terraform`:
 
    ```json
-   "aws": { "region": "us-east-1", "provider": "aws-view-<org>-shop", "services": ["sts", "s3", "iam", "sqs"] },
+   "aws": { "region": "us-east-1", "provider": "aws-view-<org>-shop", "services": ["all"] },
    "terraform": { "dir": "iac", "backendConfig": "backend-configs/staging.hcl", "varFile": "env/staging.tfvars" }
    ```
 
-   `services` lists the AWS services the project uses: their endpoints become reachable,
-   and signed. Keep the `terraform` paths as they are: the CI workflow uses the same
-   ones. `harness github-setup` then requires `tf-plan` as well.
+   `services` lists the AWS services whose endpoints the sandboxes reach, signed with
+   the view role. `all`, the default, is every service the AWS SDK knows, so the view
+   role alone decides what agents read; a list of names narrows it. Every service but S3
+   is signed with the hash of the request's body, empty for a `GET`: OpenShell's
+   automatic mode signs a bodiless `GET` in a way only S3 accepts
+   ([NVIDIA/OpenShell#5375](https://github.com/NVIDIA/OpenShell/issues/5375)). Keep the `terraform` paths
+   as they are: the CI workflow uses the same ones. `harness github-setup` then requires
+   `tf-plan` as well.
 2. Set the repository variables the workflow reads: `AWS_REGION_STAGING`,
    `AWS_ROLE_PLAN_ARN_STAGING` and `AWS_ROLE_APPLY_ARN_STAGING`.
 3. `harness doctor` creates the OpenShell AWS provider: it asks for the view role's ARN,
@@ -581,7 +588,8 @@ RDS, `manage_master_user_password`); anything else uses write-only arguments.
 
 What agents can't do, and what stops them:
 - **Hold credentials.** Sandboxes hold placeholders. The OpenShell proxy adds the Claude
-  token on `api.anthropic.com` only, and signs AWS requests with the read-only role.
+  token on `api.anthropic.com` only, each key of `network.apis` on its API's host only,
+  and signs AWS requests with the read-only role.
 - **Reach the host.** Each agent runs in its own container, with no host folder mounted;
   code moves in and out as git bundles. The host never runs code or git hooks that an
   agent wrote.
@@ -594,7 +602,8 @@ What agents can't do, and what stops them:
   They change only through an Arch PR you merge, or your own commits.
 - **Change AWS.** A read-only role with explicit denies, and applies only in CI after
   your merge.
-- **Go anywhere online.** Sandboxes reach listed hosts, read-only. Other pages go
+- **Go anywhere online.** Sandboxes reach listed hosts, read-only, and the APIs in
+  `network.apis` with any method; OpenShell has no rule for every host. Other pages go
   through `web_read`: GET only, public addresses only, never this machine's own
   addresses or the networks it is on (IPv6 included), checked again after every
   redirect, within 75 seconds and 2 MB, decompressed.
@@ -639,8 +648,8 @@ To report a vulnerability, see `SECURITY.md` in the source repository.
 |---|---|
 | `Docker: … permission denied` | Add yourself to the `docker` group, then log in again |
 | `Tailscale: an exit node is active`, or `container network` fails | An exit node can cut the containers' network: turn it off |
-| `OpenShell: … pinned to 0.1.2` | Install OpenShell 0.1.2 |
-| `OpenShell gateway` fails | `openshell gateway start` |
+| `OpenShell: … pinned to 0.1.7` | Install OpenShell 0.1.7, from its Debian package |
+| `OpenShell gateway` fails | `systemctl --user start openshell-gateway`: the gateway is a systemd user service |
 | `image: … has Claude Code X, this machine Y` | `harness doctor --build-images`, after each Claude Code update |
 | `terminals: node-pty is missing` | Install `python3`, `make` and `g++`, then reinstall with `--allow-scripts=node-pty`. `harness attach` works meanwhile. |
 | `Claude provider: … is missing` | `claude setup-token`, then `harness doctor` in a terminal: it asks for the token |

package/dist/daemon/items.js

@@ -263,7 +263,7 @@ function refusedLines(attempt, failed) {
         ...shown.map((d) => `- ${d.target} ${d.program ? `from ${d.program}` : '(a DNS query)'}, ${d.count} time${d.count === 1 ? '' : 's'}: ${d.reason.replace(/_/g, ' ')}`),
         ...(refused.length > shown.length ? [`- and ${refused.length - shown.length} more`] : []),
         ...(failed
-            ? ["A sandbox reaches only the hosts of network.bundles and network.extraHosts in harness/config.json, and the next attempt meets the same policy. If the feature can do without a host, give the way around it in your guidance. If it can't, ask_owner to allow the host, with blocking: true when no attempt can succeed without it: only the owner widens the policy."]
+            ? ["A sandbox reaches only the hosts of network.bundles, network.extraHosts and network.apis in harness/config.json, and the next attempt meets the same policy. If the feature can do without a host, give the way around it in your guidance. If it can't, ask_owner to allow the host, with blocking: true when no attempt can succeed without it: only the owner widens the policy."]
             : []),
     ];
 }

package/dist/daemon/orchestrator.js

@@ -15,7 +15,7 @@ import { rulesProblems } from '../github/setup.js';
 import { toolsManifest } from '../roles.js';
 import { PROJECT_LABEL } from '../sandbox/driver.js';
 import { BUNDLES } from '../sandbox/allowlist.js';
-import { buildPolicy } from '../sandbox/policy.js';
+import { apiProviderName, buildPolicy } from '../sandbox/policy.js';
 import { Denials, LOG_LINES } from '../sandbox/refusals.js';
 import { EventStore } from '../state/events.js';
 import { activeAttempts, apply, countedAttempts, featureKey, interruptions, isActive, managersOf, planKey, planMerged, project, writeSnapshot, } from '../state/state.js';
@@ -75,6 +75,8 @@ export class Orchestrator {
     stepErrors = { key: '', count: 0 };
     /** When the left-over sandboxes are next deleted: at the first pass, then every hour. */
     nextSweep = 0;
+    /** The gateway's providers, read at most once a minute, for the keys of the project's APIs. */
+    providerNames;
     /** Errors in a row of each plan's step, and when it is tried again: one plan that keeps failing never stops the others. */
     planErrors = new Map();
     /** The heads of Arch and Fix PRs already given the reviewers' statuses as "not reviewed". */
@@ -262,10 +264,11 @@ export class Orchestrator {
         this.record({ type: 'PIPELINE_PAUSED', payload: { reason, ...(until ? { until } : {}) } });
         return true;
     }
-    /** The owner lifts a pause. */
+    /** The owner lifts a pause. An API key stored meanwhile counts at once. */
     resume() {
         if (!this.state.pipeline.paused)
             return false;
+        this.providerNames = undefined;
         this.record({ type: 'PIPELINE_RESUMED', payload: {} });
         this.kick();
         return true;
@@ -430,6 +433,8 @@ export class Orchestrator {
             return 'progress';
         if (paused)
             return 'wait';
+        if (await this.apiKeysMissing())
+            return 'wait';
         // Manual builds one feature at a time, and looks at nothing else while it runs. Swarm starts
         // more while others run, up to its limit, and follows its other plans meanwhile (v3 §6.2).
         const swarm = this.state.mode === 'swarm';
@@ -438,6 +443,32 @@ export class Orchestrator {
             return 'wait';
         return this.scheduleNext(active.length, limit);
     }
+    /**
+     * An API of the design whose key isn't stored on this machine pauses the pipeline, with
+     * what to do: every new sandbox carries the API's provider, and OpenShell refuses one
+     * that doesn't exist. A design change that adds an API leads here.
+     */
+    async apiKeysMissing() {
+        const keyed = this.design.config.network.apis.filter((api) => api.key);
+        if (keyed.length === 0)
+            return false;
+        if (!this.providerNames || this.now() - this.providerNames.at >= 60_000) {
+            try {
+                this.providerNames = { at: this.now(), names: await this.driver.providers() };
+            }
+            catch (error) {
+                // The gateway's trouble shows elsewhere; this check doesn't hold the pipeline for it.
+                this.log(`the providers could not be listed: ${error.message}`);
+                return false;
+            }
+        }
+        const project = this.design.config.project.name;
+        const missing = keyed.filter((api) => !this.providerNames.names.includes(apiProviderName(project, api.host)));
+        if (missing.length === 0)
+            return false;
+        this.pausePipeline(`no key stored on this machine for ${missing.map((api) => api.host).join(', ')}: run \`harness doctor\` in a terminal, which asks for it, then resume`);
+        return true;
+    }
     /** One attempt's next step. An error counts against this attempt alone, and interrupts it past the limit. */
     async stepAttempt(feature, attempt) {
         try {
@@ -1724,12 +1755,16 @@ export class Orchestrator {
      * A sandbox's providers, policy and settings. Agents get Claude; with AWS in the
      * project, they and the checks that need it also get the view role, which OpenShell
      * holds and signs with (§12.1). A provider is only attached where its endpoints are.
+     * Every sandbox gets the keys of the project's APIs, for the code it runs.
      */
     access(options) {
         const aws = options.aws ? this.design.config.aws : undefined;
         const limits = this.state.sandboxLimits;
+        const project = this.design.config.project.name;
+        // The keys of the project's APIs, which OpenShell adds on requests to their hosts alone.
+        const apiKeys = this.design.config.network.apis.flatMap((api) => (api.key ? [apiProviderName(project, api.host)] : []));
         return {
-            providers: [...(options.claude ? [this.design.config.sandbox.claudeProvider] : []), ...(aws ? [aws.provider] : [])],
+            providers: [...(options.claude ? [this.design.config.sandbox.claudeProvider] : []), ...(aws ? [aws.provider] : []), ...apiKeys],
             policyFile: this.policyFile({ allBundles: options.allBundles ?? false, aws: aws !== undefined }),
             env: this.sandboxEnv(),
             // Every sandbox the daemon makes says whose it is: the dashboard tells them from other projects'.

package/dist/design/schemas.js

@@ -1,5 +1,6 @@
 import { readFileSync } from 'node:fs';
 import { z } from 'zod';
+import { ALL_BUNDLES, BUNDLES } from '../sandbox/allowlist.js';
 import { canonicalJson, sha256 } from '../util.js';
 /** The agents' image is tagged with the harness's version, which may change what goes into it: `harness doctor --build-images` builds it. */
 const AGENT_IMAGE = `razzou-harness/agent:${JSON.parse(readFileSync(new URL('../../package.json', import.meta.url), 'utf8')).version}`;
@@ -71,6 +72,14 @@ export function featureDependency(planId, dep) {
     const [a, b] = dep.split('/');
     return b === undefined ? { plan: planId, feature: a } : { plan: a, feature: b };
 }
+/** Variables the harness, OpenShell or the system set in a sandbox: an API's key can't take one of their names. */
+const RESERVED_VARIABLES = /^(AWS_|CLAUDE_|OPENSHELL_|HARNESS_|SSL_|NODE_|NPM_)|^(PATH|HOME|USER|SHELL|LANG|TERM|PWD|TMPDIR|REQUESTS_CA_BUNDLE|CURL_CA_BUNDLE|DISABLE_AUTOUPDATER|CHECKPOINT_DISABLE|MCP_TOOL_TIMEOUT)$/;
+/** A third-party API of `network.apis`: an exact host, and the variable its key is read from, if it has one. */
+const ApiSchema = z.object({
+    host: z.string().regex(/^[a-z0-9]([a-z0-9-]*[a-z0-9])?(\.[a-z0-9]([a-z0-9-]*[a-z0-9])?)+$/, 'a host name, like api.stripe.com'),
+    key: z.string().regex(/^[A-Z][A-Z0-9_]*$/, 'a variable name in capitals, like STRIPE_API_KEY')
+        .refine((name) => !RESERVED_VARIABLES.test(name), 'a variable the harness, OpenShell or the system sets; pick another name, like STRIPE_API_KEY').optional(),
+}).strict();
 export const ConfigSchema = z.object({
     harnessVersion: z.string(),
     project: z.object({
@@ -117,8 +126,18 @@ export const ConfigSchema = z.object({
         .prefault({}),
     network: z
         .object({
-        bundles: z.array(z.string()).default(['core']),
+        /** The hosts sandboxes reach directly, read-only, by bundle; `all`, the default, is every bundle. */
+        bundles: z.array(z.string().refine((name) => name === ALL_BUNDLES || name in BUNDLES, `a bundle: ${ALL_BUNDLES}, ${Object.keys(BUNDLES).join(', ')}`)).default([ALL_BUNDLES]),
         extraHosts: z.array(z.string()).default([]),
+        /**
+         * Third-party APIs sandboxes may call with any method, like a payment provider's test
+         * API. `key` names the variable the project's code reads the API's key from: in a
+         * sandbox it holds a placeholder, which OpenShell replaces with the key on requests to
+         * the host alone. `harness doctor` stores the key, at a hidden prompt.
+         */
+        apis: z.array(ApiSchema).default([])
+            .refine((apis) => new Set(apis.map((a) => a.host)).size === apis.length, 'each host once')
+            .refine((apis) => { const keys = apis.flatMap((a) => (a.key ? [a.key] : [])); return new Set(keys).size === keys.length; }, 'each key once'),
     })
         .prefault({}),
     checks: z.array(CheckSchema).default([]),
@@ -146,8 +165,8 @@ export const ConfigSchema = z.object({
         region: z.string().regex(/^[a-z]{2}(-[a-z]+)+-\d$/, 'an AWS region, like us-east-1'),
         /** The OpenShell provider holding the view role, like "aws-view-shop". */
         provider: z.string().min(1),
-        /** The services the project uses: their endpoints are reachable, and signed. */
-        services: z.array(z.string().regex(/^[a-z0-9-]+$/)).default(['sts', 's3', 'iam']),
+        /** The services whose endpoints are reachable, and signed; `all`, the default, is every one, so the view role alone decides. */
+        services: z.array(z.string().regex(/^[a-z0-9-]+$/)).default(['all']),
     })
         .optional(),
     /** Where the project's Terraform lives, and how a plan is made against the real state (§12.4). */

package/dist/sandbox/allowlist.js

@@ -19,6 +19,12 @@ export const BUNDLES = {
     opentofu: read('registry.opentofu.org', 'opentofu.org'),
     browsers: read('cdn.playwright.dev', 'playwright.download.prss.microsoft.com'),
 };
+/** Every bundle, those later versions add included: what a project gets unless it lists fewer. */
+export const ALL_BUNDLES = 'all';
+/** The bundles a project's list names, `all` standing for every one. */
+export function bundleNames(names) {
+    return names.includes(ALL_BUNDLES) ? Object.keys(BUNDLES) : [...names];
+}
 /** Throws on a host OpenShell would reject, so a bad entry fails at load time rather than at sandbox creation. */
 export function checkHost(host) {
     const labels = host.split('.');