@skyramp/testbot-fault-plan 0.0.0-pr676.175

package/LICENSE.md

@@ -0,0 +1 @@
+© Skyramp Inc. All rights reserved. Use is subject to the Skyramp Terms of Service: https://docs.skyramp.dev/terms-of-service

package/README.md

@@ -0,0 +1,53 @@
+# @skyramp/testbot-fault-plan
+
+Test support for Skyramp Testbot (SKYR-5196, Area G gap G1). It is not part of the engine: `@skyramp/testbot-core` uses it only in its tests, and the Testbot web app takes it as a devDependency.
+
+It holds one idea in one place:
+
+- **`FaultPlan`**: which calls of a test fake fail, and how.
+- **`faultyFetch`**: the plan on an HTTP fake (any `fetch`).
+- **`recorded`**: the real error answers of GitHub, Atlassian, Anthropic, Amazon Bedrock and the workflow steps, with where each was seen.
+
+Adapters for one seam only stay in the repository that has that seam, built on this package: `faultyMethods` (the web app's interface fakes of GitHub, in its integration test support) and `faultySessions` (the engine's fake Claude session, in `packages/core/src/__tests__/support`).
+
+## How it works
+
+A fake asks the plan before each call, with a key that the fake chooses. `faultyFetch` uses `METHOD /path` (no query string). The plan answers a fault, or nothing. With nothing, the fake gives its normal answer. The plan never changes the normal answers of a fake. A fake's older ad hoc failure switches (for example `mintError` in the web app's runner tests, or `outage` in its fake Atlassian) stay as they are, and the tests that use them do not change; new tests use the plan.
+
+```ts
+import { FaultKind, FaultPlan, fault, faultyFetch, recorded } from "@skyramp/testbot-fault-plan";
+
+const plan = new FaultPlan()
+  .on(/^PUT \/api\/v1\/runner\/attempts\/[^/]+\/report$/, fault.drop())          // the next upload: the app saves it, the answer is lost
+  .on("POST /repos/acme/api/dispatches", fault.recorded(recorded.github.secondaryRateLimit), { nth: 2 })
+  .down(/^GET \/ex\/confluence\//, fault.status(503));                          // an outage, until plan.up(...)
+const fetch = faultyFetch(plan, fakeApp.fetch);
+// ... run the code under test ...
+expect(plan.seen).toContainEqual({ key: expect.stringMatching(/report$/), kind: FaultKind.Drop });
+```
+
+| Fault | What the caller sees | The fake |
+| --- | --- | --- |
+| `fault.status(status, body?, headers?)` | that answer | not called |
+| `fault.recorded(error)` | a recorded real answer. A recorded text (`recorded.model.*`, a CLI line) has no HTTP answer: `faultyFetch` refuses it, and a session adapter prints it | not called |
+| `fault.drop()` | `TypeError: fetch failed` | called first: the service did the work |
+| `fault.timeout()` | `DOMException` named `TimeoutError`, at once, as the caller's `AbortSignal.timeout` gives | not called |
+| `fault.hang()` | nothing until the caller's signal aborts, then its reason. Refused for a call with no signal | not called |
+
+`on(match, fault, { nth, times })` fails the `nth` matching call from now (default 1), `times` calls in a row (default 1). `down(match, fault)` fails every matching call until `up(match)`. `match` is an exact key or a `RegExp` without the `g` or `y` flag. Every rule that matches a call counts it; the first rule whose turn it is gives the fault.
+
+`plan.seen` lists each fault the plan gave. **Assert that your fault is in it.** A test whose fault never happened is "not exercised", and must not pass as a recovery test.
+
+There is no separate "connection refused" fault: none of these clients reads a rejection before the answer differently from a timeout, so `fault.timeout()` covers it.
+
+`timeout` rejects at once instead of waiting: the clients under test (the engine's runner client, 30 s; the web app's GitHub client, 10 s; its Atlassian client) read any rejection before the answer as a lost answer, so a real wait would only make the test slow.
+
+## How to add a fault
+
+1. **A new answer of a service:** add it to `src/recorded-errors.ts`, with `provenance` (where and when it was seen: a drill, a run id, a log) and `provisional: true` unless a drill or run showed that exact answer. Never put a service's error text in a test file.
+2. **A new way to fail:** add a `FaultKind` only when no existing kind gives the client the same input. Handle it in `faultyFetch` and in every adapter (`faultyMethods` in the web app, `faultySessions` in the engine), and add a test here.
+3. **A new fake:** wrap it. An HTTP fake: `faultyFetch(plan, fake)`. Another seam: an adapter in that repository's test support that asks `plan.take(key)` and answers each `FaultKind` as that seam's real client would.
+
+## Release
+
+The package publishes with `@skyramp/testbot-core`, in the same `release-engine.yml` run, with the same version and dist-tag (for example `0.13.0-v3.<run>` under `next-v3`). Pin an exact version.

package/dist/fault-plan.d.ts

@@ -0,0 +1,97 @@
+/**
+ * A fault plan: which calls of a test fake fail, and how. A fake asks the plan before each call with a key it
+ * chooses (for an HTTP fake, `METHOD /path`), and the plan answers the fault to give, or nothing. The fake keeps its
+ * own normal answers; the plan only adds failures. See README.md.
+ */
+/** How a call fails. */
+export declare enum FaultKind {
+    /** Answer an error status, with an optional body and headers. */
+    Status = "status",
+    /** Answer a recorded real error of the service (recorded-errors.ts). */
+    Recorded = "recorded",
+    /** Do the work, then lose the answer: the caller sees a broken connection after the service changed its state. */
+    Drop = "drop",
+    /** Reject at once as the caller's own time limit (`AbortSignal.timeout`) would, before the work. */
+    Timeout = "timeout",
+    /** Never answer: reject only when the caller's signal aborts. For callers with a short, injected time limit. */
+    Hang = "hang"
+}
+/** A real error answer of an HTTP service, as recorded where it was seen. */
+export interface RecordedHttpError {
+    service: string;
+    status: number;
+    headers?: Record<string, string>;
+    body: unknown;
+    /** Where and when the answer was seen, or what it was built from. */
+    provenance: string;
+    /** True when no drill or run has shown this exact answer yet. */
+    provisional: boolean;
+}
+/** A line a tool printed, such as the Claude Code CLI's `API Error:` result or a workflow step's log. */
+export interface RecordedText {
+    service: string;
+    text: string;
+    /** Where and when the text was seen, or what it was built from. */
+    provenance: string;
+    /** True when no drill or run has shown this exact text yet. */
+    provisional: boolean;
+}
+/** A recorded answer that is a printed line, not an HTTP answer. */
+export declare const isRecordedText: (error: RecordedHttpError | RecordedText) => error is RecordedText;
+export type Fault = {
+    kind: FaultKind.Status;
+    status: number;
+    body?: unknown;
+    headers?: Record<string, string>;
+} | {
+    kind: FaultKind.Recorded;
+    error: RecordedHttpError | RecordedText;
+} | {
+    kind: FaultKind.Drop;
+} | {
+    kind: FaultKind.Timeout;
+} | {
+    kind: FaultKind.Hang;
+};
+/** The faults, for short test code: `plan.on("PUT /report", fault.drop())`. */
+export declare const fault: {
+    status: (status: number, body?: unknown, headers?: Record<string, string>) => Fault;
+    recorded: (error: RecordedHttpError | RecordedText) => Fault;
+    drop: () => Fault;
+    timeout: () => Fault;
+    hang: () => Fault;
+};
+/** Which matching calls fail: from the `nth` (1 = the next one), `times` calls in a row (Infinity: every one after). */
+export interface FaultOptions {
+    nth?: number;
+    times?: number;
+}
+/** A call key, exact, or a pattern. */
+export type FaultMatch = string | RegExp;
+/** A fault the plan gave. */
+export interface FaultSeen {
+    key: string;
+    kind: FaultKind;
+}
+export declare class FaultPlan {
+    private rules;
+    /** Each fault the plan gave, in order. A test asserts its fault is here, so a fault never reached fails the test. */
+    readonly seen: FaultSeen[];
+    /** Fails matching calls: from the `nth` after this one is added (default 1), `times` in a row (default 1). */
+    on(match: FaultMatch, failure: Fault, options?: FaultOptions): this;
+    /** An outage: every matching call fails with `fault` until `up(match)`. */
+    down(match: FaultMatch, failure: Fault): this;
+    /** Ends the faults added with this match. */
+    up(match: FaultMatch): this;
+    /** The fault for this call, or undefined to answer it normally. Every rule that matches counts the call. */
+    take(key: string): Fault | undefined;
+}
+/** What a caller sees when the connection broke after the service did the work (undici's `fetch failed`). */
+export declare function lostAnswer(): TypeError;
+/** What a caller's `AbortSignal.timeout` gives when its time limit passes. */
+export declare function timeoutError(): DOMException;
+/**
+ * Wraps an HTTP fake (a `fetch`): each call asks `plan` with the key `METHOD /path` (no query), and a fault replaces or
+ * follows the fake's answer. A call the plan does not fail goes to the fake unchanged.
+ */
+export declare function faultyFetch(plan: FaultPlan, inner: typeof fetch): typeof fetch;

package/dist/fault-plan.js

@@ -0,0 +1,130 @@
+/**
+ * A fault plan: which calls of a test fake fail, and how. A fake asks the plan before each call with a key it
+ * chooses (for an HTTP fake, `METHOD /path`), and the plan answers the fault to give, or nothing. The fake keeps its
+ * own normal answers; the plan only adds failures. See README.md.
+ */
+/** How a call fails. */
+export var FaultKind;
+(function (FaultKind) {
+    /** Answer an error status, with an optional body and headers. */
+    FaultKind["Status"] = "status";
+    /** Answer a recorded real error of the service (recorded-errors.ts). */
+    FaultKind["Recorded"] = "recorded";
+    /** Do the work, then lose the answer: the caller sees a broken connection after the service changed its state. */
+    FaultKind["Drop"] = "drop";
+    /** Reject at once as the caller's own time limit (`AbortSignal.timeout`) would, before the work. */
+    FaultKind["Timeout"] = "timeout";
+    /** Never answer: reject only when the caller's signal aborts. For callers with a short, injected time limit. */
+    FaultKind["Hang"] = "hang";
+})(FaultKind || (FaultKind = {}));
+/** A recorded answer that is a printed line, not an HTTP answer. */
+export const isRecordedText = (error) => "text" in error;
+/** The faults, for short test code: `plan.on("PUT /report", fault.drop())`. */
+export const fault = {
+    status: (status, body, headers) => ({
+        kind: FaultKind.Status, status, ...(body === undefined ? {} : { body }), ...(headers === undefined ? {} : { headers }),
+    }),
+    recorded: (error) => ({ kind: FaultKind.Recorded, error }),
+    drop: () => ({ kind: FaultKind.Drop }),
+    timeout: () => ({ kind: FaultKind.Timeout }),
+    hang: () => ({ kind: FaultKind.Hang }),
+};
+const matches = (match, key) => (typeof match === "string" ? match === key : match.test(key));
+const sameMatch = (a, b) => typeof a === "string" || typeof b === "string" ? a === b : a.source === b.source && a.flags === b.flags;
+export class FaultPlan {
+    rules = [];
+    /** Each fault the plan gave, in order. A test asserts its fault is here, so a fault never reached fails the test. */
+    seen = [];
+    /** Fails matching calls: from the `nth` after this one is added (default 1), `times` in a row (default 1). */
+    on(match, failure, options = {}) {
+        const from = options.nth ?? 1;
+        const times = options.times ?? 1;
+        if (!Number.isInteger(from) || from < 1)
+            throw new Error(`nth must be a whole number from 1, got ${from}`);
+        if (!(times >= 1))
+            throw new Error(`times must be 1 or more, got ${times}`);
+        // A global or sticky RegExp keeps lastIndex between test() calls, so it would match every other call.
+        if (typeof match !== "string" && (match.global || match.sticky))
+            throw new Error(`Use a RegExp without the g or y flag: ${match}`);
+        this.rules.push({ match, fault: failure, from, until: from + times - 1, calls: 0 });
+        return this;
+    }
+    /** An outage: every matching call fails with `fault` until `up(match)`. */
+    down(match, failure) {
+        return this.on(match, failure, { times: Infinity });
+    }
+    /** Ends the faults added with this match. */
+    up(match) {
+        this.rules = this.rules.filter(rule => !sameMatch(rule.match, match));
+        return this;
+    }
+    /** The fault for this call, or undefined to answer it normally. Every rule that matches counts the call. */
+    take(key) {
+        let chosen;
+        for (const rule of this.rules) {
+            if (!matches(rule.match, key))
+                continue;
+            rule.calls += 1;
+            if (!chosen && rule.calls >= rule.from && rule.calls <= rule.until)
+                chosen = rule.fault;
+        }
+        if (chosen)
+            this.seen.push({ key, kind: chosen.kind });
+        return chosen;
+    }
+}
+/** What a caller sees when the connection broke after the service did the work (undici's `fetch failed`). */
+export function lostAnswer() {
+    return new TypeError("fetch failed", { cause: new Error("other side closed: the answer was lost after the service did the work (fault plan)") });
+}
+/** What a caller's `AbortSignal.timeout` gives when its time limit passes. */
+export function timeoutError() {
+    return new DOMException("The operation was aborted due to timeout (fault plan)", "TimeoutError");
+}
+function errorResponse(status, body, headers = {}) {
+    if (body === undefined)
+        return new Response(null, { status, headers });
+    if (typeof body === "string")
+        return new Response(body, { status, headers });
+    return new Response(JSON.stringify(body), { status, headers: { "content-type": "application/json", ...headers } });
+}
+/** Rejects when `signal` aborts, with its reason. */
+function hang(signal) {
+    if (!signal)
+        return Promise.reject(new Error("A hang fault needs the caller's signal to end it; this call has none, so use fault.timeout()"));
+    return new Promise((_resolve, reject) => {
+        if (signal.aborted)
+            reject(signal.reason);
+        else
+            signal.addEventListener("abort", () => reject(signal.reason), { once: true });
+    });
+}
+/**
+ * Wraps an HTTP fake (a `fetch`): each call asks `plan` with the key `METHOD /path` (no query), and a fault replaces or
+ * follows the fake's answer. A call the plan does not fail goes to the fake unchanged.
+ */
+export function faultyFetch(plan, inner) {
+    return (async (input, init) => {
+        const request = input instanceof Request ? input : undefined;
+        const url = new URL(request ? request.url : String(input));
+        const method = (init?.method ?? request?.method ?? "GET").toUpperCase();
+        const chosen = plan.take(`${method} ${url.pathname}`);
+        if (!chosen)
+            return inner(input, init);
+        switch (chosen.kind) {
+            case FaultKind.Status:
+                return errorResponse(chosen.status, chosen.body, chosen.headers);
+            case FaultKind.Recorded:
+                if (isRecordedText(chosen.error))
+                    throw new Error(`A recorded text (${chosen.error.service}) has no HTTP answer: use it with a session adapter`);
+                return errorResponse(chosen.error.status, chosen.error.body, chosen.error.headers);
+            case FaultKind.Drop:
+                await inner(input, init);
+                throw lostAnswer();
+            case FaultKind.Timeout:
+                throw timeoutError();
+            case FaultKind.Hang:
+                return hang(init?.signal ?? request?.signal);
+        }
+    });
+}

package/dist/index.d.ts

@@ -0,0 +1,2 @@
+export { FaultKind, FaultPlan, fault, faultyFetch, isRecordedText, lostAnswer, timeoutError, type Fault, type FaultMatch, type FaultOptions, type FaultSeen, type RecordedHttpError, type RecordedText, } from "./fault-plan.js";
+export * as recorded from "./recorded-errors.js";

package/dist/index.js

@@ -0,0 +1,2 @@
+export { FaultKind, FaultPlan, fault, faultyFetch, isRecordedText, lostAnswer, timeoutError, } from "./fault-plan.js";
+export * as recorded from "./recorded-errors.js";

package/dist/recorded-errors.d.ts

@@ -0,0 +1,170 @@
+/** GitHub REST API answers. */
+export declare const github: {
+    /** A secondary rate limit, which GitHub answers with 403, never a refusal (SKYR-5191). */
+    secondaryRateLimit: {
+        service: string;
+        status: number;
+        body: {
+            message: string;
+            documentation_url: string;
+        };
+        provenance: string;
+        provisional: true;
+    };
+    /** A workflow dispatch to a disabled workflow (drill G-03). */
+    dispatchRefusedDisabledWorkflow: {
+        service: string;
+        status: number;
+        body: {
+            message: string;
+            documentation_url: string;
+        };
+        provenance: string;
+        provisional: true;
+    };
+    /** An App that may not do this, such as a permission removed from the installation. */
+    resourceNotAccessible: {
+        service: string;
+        status: number;
+        body: {
+            message: string;
+            documentation_url: string;
+        };
+        provenance: string;
+        provisional: true;
+    };
+    /** A token request for permissions the installation no longer grants. */
+    permissionsNotGranted: {
+        service: string;
+        status: number;
+        body: {
+            message: string;
+            documentation_url: string;
+        };
+        provenance: string;
+        provisional: true;
+    };
+};
+/** Atlassian (Jira, Confluence and the OAuth token endpoint) answers. */
+export declare const atlassian: {
+    /** Jira's rate limit, with the wait it asks for. */
+    jiraRateLimit: {
+        service: string;
+        status: number;
+        headers: {
+            "retry-after": string;
+        };
+        body: {
+            errorMessages: string[];
+            errors: {};
+        };
+        provenance: string;
+        provisional: true;
+    };
+    /** A product API outage. */
+    outage: {
+        service: string;
+        status: number;
+        body: string;
+        provenance: string;
+        provisional: true;
+    };
+    /** The token endpoint's rate limit. */
+    tokenRateLimit: {
+        service: string;
+        status: number;
+        body: {
+            error: string;
+        };
+        provenance: string;
+        provisional: true;
+    };
+};
+/**
+ * The Claude Code CLI's result line for a model provider error, which a staged session ends with (exit 1). The engine
+ * reads it in `isApiErrorCrash`. Sorted by the pass rule's kinds (Area G, SKYR-5193): temporary, then permanent.
+ */
+export declare const model: {
+    anthropicOverloaded: {
+        service: string;
+        text: string;
+        provenance: string;
+        provisional: true;
+    };
+    anthropicServiceUnavailable: {
+        service: string;
+        text: string;
+        provenance: string;
+        provisional: true;
+    };
+    anthropicInternalError: {
+        service: string;
+        text: string;
+        provenance: string;
+        provisional: true;
+    };
+    connectionError: {
+        service: string;
+        text: string;
+        provenance: string;
+        provisional: true;
+    };
+    connectionClosedMidResponse: {
+        service: string;
+        text: string;
+        provenance: string;
+        provisional: true;
+    };
+    anthropicTokensPerDay: {
+        service: string;
+        text: string;
+        provenance: string;
+        provisional: false;
+    };
+    anthropicTooManyRequests: {
+        service: string;
+        text: string;
+        provenance: string;
+        provisional: true;
+    };
+    bedrockUnexpectedError: {
+        service: string;
+        text: string;
+        provenance: string;
+        provisional: false;
+    };
+    usageLimit: {
+        service: string;
+        text: string;
+        provenance: string;
+        provisional: false;
+    };
+    bedrockExpiredToken: {
+        service: string;
+        text: string;
+        provenance: string;
+        provisional: false;
+    };
+    anthropicInvalidApiKey: {
+        service: string;
+        text: string;
+        provenance: string;
+        provisional: true;
+    };
+    bedrockAccessDenied: {
+        service: string;
+        text: string;
+        provenance: string;
+        provisional: true;
+    };
+};
+/** Workflow step log lines. */
+export declare const workflow: {
+    /** The "Configure AWS credentials (Bedrock)" step with a role that does not exist (drill G-08). */
+    awsAssumeRoleFailed: {
+        service: string;
+        text: string;
+        provenance: string;
+        provisional: false;
+    };
+};

package/dist/recorded-errors.js

@@ -0,0 +1,131 @@
+/** GitHub REST API answers. */
+export const github = {
+    /** A secondary rate limit, which GitHub answers with 403, never a refusal (SKYR-5191). */
+    secondaryRateLimit: {
+        service: "github", status: 403,
+        body: { message: "You have exceeded a secondary rate limit. Please wait a few minutes before you try again.", documentation_url: "https://docs.github.com/rest/overview/rate-limits-for-the-rest-api" },
+        provenance: "GitHub REST API documentation wording, as testbot-installer-app#667 (SKYR-5191) tests it; no drill recorded the body yet",
+        provisional: true,
+    },
+    /** A workflow dispatch to a disabled workflow (drill G-03). */
+    dispatchRefusedDisabledWorkflow: {
+        service: "github", status: 404,
+        body: { message: "Not Found", documentation_url: "https://docs.github.com/rest/actions/workflows#create-a-workflow-dispatch-event" },
+        provenance: "Drill G-03, staging 2026-10-29: the Worker log shows `GitHub refused it: NOT_FOUND`; the body was not logged, so it is GitHub's usual 404 body",
+        provisional: true,
+    },
+    /** An App that may not do this, such as a permission removed from the installation. */
+    resourceNotAccessible: {
+        service: "github", status: 403,
+        body: { message: "Resource not accessible by integration", documentation_url: "https://docs.github.com/rest" },
+        provenance: "GitHub's answer to an App without the permission, as testbot-installer-app#667 tests it",
+        provisional: true,
+    },
+    /** A token request for permissions the installation no longer grants. */
+    permissionsNotGranted: {
+        service: "github", status: 422,
+        body: { message: "The permissions requested are not granted to this installation.", documentation_url: "https://docs.github.com/rest/apps/apps#create-an-installation-access-token-for-an-app" },
+        provenance: "GitHub REST API documentation of the installation token route; no drill recorded it yet",
+        provisional: true,
+    },
+};
+/** Atlassian (Jira, Confluence and the OAuth token endpoint) answers. */
+export const atlassian = {
+    /** Jira's rate limit, with the wait it asks for. */
+    jiraRateLimit: {
+        service: "atlassian", status: 429, headers: { "retry-after": "30" },
+        body: { errorMessages: ["Rate limit exceeded."], errors: {} },
+        provenance: "Jira Cloud REST documentation (rate limiting); no drill recorded it yet",
+        provisional: true,
+    },
+    /** A product API outage. */
+    outage: {
+        service: "atlassian", status: 503, body: "upstream error",
+        provenance: "The shape fake-atlassian.ts answers for an outage; no drill recorded it yet",
+        provisional: true,
+    },
+    /** The token endpoint's rate limit. */
+    tokenRateLimit: {
+        service: "atlassian", status: 429, body: { error: "too_many_requests" },
+        provenance: "The shape fake-atlassian.ts answers; no drill recorded it yet",
+        provisional: true,
+    },
+};
+/**
+ * The Claude Code CLI's result line for a model provider error, which a staged session ends with (exit 1). The engine
+ * reads it in `isApiErrorCrash`. Sorted by the pass rule's kinds (Area G, SKYR-5193): temporary, then permanent.
+ */
+export const model = {
+    // Temporary: the provider is overloaded, rate-limited, unreachable or stops in mid-answer.
+    anthropicOverloaded: {
+        service: "anthropic", text: 'API Error: 529 {"type":"overloaded_error"}',
+        provenance: "Built from Anthropic's documented 529 overloaded_error (agent-failure-reason.test.ts), not from a captured log",
+        provisional: true,
+    },
+    anthropicServiceUnavailable: {
+        service: "anthropic", text: "API Error: 503 Service Unavailable",
+        provenance: "Built from the API docs' 5xx (agent-failure-reason.test.ts)",
+        provisional: true,
+    },
+    anthropicInternalError: {
+        service: "anthropic", text: "API Error: 500 Internal server error.",
+        provenance: "execute-agent.test.ts result entry",
+        provisional: true,
+    },
+    connectionError: {
+        service: "anthropic", text: "API Error: Connection error.",
+        provenance: "Built for a stream dropped before any status (agent-failure-reason.test.ts)",
+        provisional: true,
+    },
+    connectionClosedMidResponse: {
+        service: "anthropic", text: "API Error: Connection closed mid-response. The response above may be incomplete.",
+        provenance: "execute-agent.test.ts result entry of a stream cut after it started",
+        provisional: true,
+    },
+    anthropicTokensPerDay: {
+        service: "anthropic", text: "API Error: Request rejected (429) · Too many tokens per day, please wait before trying again.",
+        provenance: "Verbatim from eval run 33602596506 (agent-failure-reason.test.ts)",
+        provisional: false,
+    },
+    anthropicTooManyRequests: {
+        service: "anthropic", text: "API Error: Request rejected (429) · Too many requests, please wait before trying again.",
+        provenance: "agent-failure-reason.test.ts, same form as eval run 33602596506",
+        provisional: true,
+    },
+    bedrockUnexpectedError: {
+        service: "bedrock", text: "API Error: The system encountered an unexpected error during processing. Try your request again.",
+        provenance: "Five of 40 eval fixture runs on 2026-10-07 (run-stage.ts isApiErrorCrash)",
+        provisional: false,
+    },
+    // Permanent: no permission, a bad key or credential, or a usage limit.
+    usageLimit: {
+        service: "bedrock", text: "API Error: 400 You have reached your specified workspace API usage limits. You will regain access on 2026-09-01 at 00:00 UTC.",
+        provenance: "Real string from a Bedrock run (agent-failure-reason.test.ts)",
+        provisional: false,
+    },
+    bedrockExpiredToken: {
+        service: "bedrock", text: "Failed to authenticate. API Error: 403 The security token included in the request is expired",
+        provenance: "Verbatim from eval agent-log.ndjson entries, eval-29726542076 and 9 other runs (agent-failure-reason.test.ts)",
+        provisional: false,
+    },
+    anthropicInvalidApiKey: {
+        service: "anthropic", text: "Invalid API key · Please run /login",
+        provenance: "The CLI's answer to a bad key (staged-api-error-retry.test.ts); not an `API Error:` line",
+        provisional: true,
+    },
+    bedrockAccessDenied: {
+        service: "bedrock",
+        text: "API Error: 403 User: arn:aws:sts::000000000000:assumed-role/testbot-drill-no-invoke/testbot is not authorized to perform: bedrock:InvokeModel on resource: arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-sonnet-4-5",
+        provenance: "PLACEHOLDER built from AWS's AccessDeniedException wording. Drill G-09 (SKYR-5212) records the real text; replace it then",
+        provisional: true,
+    },
+};
+/** Workflow step log lines. */
+export const workflow = {
+    /** The "Configure AWS credentials (Bedrock)" step with a role that does not exist (drill G-08). */
+    awsAssumeRoleFailed: {
+        service: "aws", text: "Could not assume role with OIDC: The web identity token provided could not be validated.",
+        provenance: "Drill G-08, staging 2026-10-29, archit/quote-service run 40928283162",
+        provisional: false,
+    },
+};

package/package.json

@@ -0,0 +1,43 @@
+{
+  "name": "@skyramp/testbot-fault-plan",
+  "version": "0.0.0-pr676.175",
+  "description": "Test support for Skyramp Testbot: a fault plan for test fakes, and the real error answers of the services Testbot calls. Not part of the engine.",
+  "type": "module",
+  "main": "dist/index.js",
+  "types": "dist/index.d.ts",
+  "exports": {
+    ".": {
+      "types": "./dist/index.d.ts",
+      "default": "./dist/index.js"
+    }
+  },
+  "//files": "An explicit allowlist, as for @skyramp/testbot-core: no glob, no sourcemaps, no TypeScript source.",
+  "files": [
+    "dist/index.js",
+    "dist/index.d.ts",
+    "dist/fault-plan.js",
+    "dist/fault-plan.d.ts",
+    "dist/recorded-errors.js",
+    "dist/recorded-errors.d.ts"
+  ],
+  "engines": {
+    "node": ">=22"
+  },
+  "scripts": {
+    "build": "tsc -p tsconfig.build.json",
+    "test": "vitest run",
+    "typecheck": "tsc --noEmit",
+    "check:pack": "node scripts/check-pack.mjs"
+  },
+  "repository": {
+    "type": "git",
+    "url": "https://github.com/letsramp/testbot",
+    "directory": "packages/fault-plan"
+  },
+  "author": "Skyramp Inc",
+  "license": "SEE LICENSE IN LICENSE.md",
+  "devDependencies": {
+    "@types/node": "^25.2.3",
+    "vitest": "^5.0.0"
+  }
+}