ax 0.10.3 → 0.11.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -18,7 +18,7 @@
18
18
  npx ax audit https://stripe.com
19
19
  ```
20
20
 
21
- **No account, no API key.** Everything below works out of the box. The exceptions are [`journey`](#for-ora-partners) and `audit --tunnel ora`, which need an ora platform key, and `deep-journey --task`, which needs a partner key.
21
+ **No account, no API key.** Everything below works out of the box. The exceptions are [`journey`](#for-ora-partners) on a goal of your own and `audit --tunnel ora`, which need an ora platform key; a goal of your own on the free path needs a partner key instead.
22
22
 
23
23
  <p align="center">
24
24
  <img src="https://raw.githubusercontent.com/ora/ax/main/.github/assets/demo.gif" width="880" alt="ax audit https://stripe.com — live progress, then the layered report and ora's ranked top fixes" />
@@ -28,8 +28,8 @@ Six commands:
28
28
 
29
29
  - **`audit <url>`** — run ora's hosted agent-readiness audit against a site: live progress, then a layered report of what passed, what's broken, and ora's ranked list of the highest-impact fixes (or `--json` for the raw contract payload). No account or API key needed. Gate CI with `--min-score`.
30
30
  - **`webmcp-audit <url>`** — audit a page's `document.modelContext` tools in a real browser. Works on any http(s) URL, localhost included, so you can check a surface before you publish it. Measures the page the way ora's capture worker does and has ora score it; nothing is stored, published, or ranked.
31
- - **`deep-journey <url>`** — run a real AI agent at a site on one of ora's curated tasks through the public journey API: no key, no workspace. A partner API key unlocks free-text tasks (`--task`) and a larger allowance.
32
- - **`journey "<intent>"`** — send a real AI agent (claude-code, codex, …) at a site and watch its navigation live as a boxed node-graph, then get the scored insight, tokens, and cost. Partner-only: needs an `ORA_API_KEY` (see [For ora partners](#for-ora-partners)).
31
+ - **`journey --domain <site>`** — run a real AI agent at a site on one of ora's curated tasks through the public journey API: no key, no workspace. A partner API key unlocks a goal of your own and a larger allowance. `ax deep-journey <url>` is the older name and still works.
32
+ - **`journey "<goal>"`** — send a real AI agent (claude-code, codex, …) at a site and watch its navigation live as a boxed node-graph, then get the scored insight, tokens, and cost. Partner-only: needs an `ORA_API_KEY` (see [For ora partners](#for-ora-partners)).
33
33
  - **`skill [name]`** — list, print, or install ora's agent skills, digest-verified from the public registry.
34
34
  - **`generate-files <domain>`** — draft the files agents look for on a site (`ard.json`, `ai-catalog.json`, `poppy.json`, `llms.txt`, a `robots.txt` edit, an API catalog, MCP server cards), filled in from ora's last scan of the domain.
35
35
 
@@ -158,7 +158,7 @@ present in order to call your tools.
158
158
  passes and the same probe code ora's worker runs, plus a viewport screenshot at ora's 1280×720. As
159
159
  ora does, it loads the page with images, media and webfonts refused — stylesheets still apply — so
160
160
  the screenshot the grader sees is the one ora would have taken. That capture is
161
- posted to ora, which runs its 19 checks, two model hops and the tool-selection simulation, and
161
+ posted to ora, which runs its 20 checks, two model hops and the tool-selection simulation, and
162
162
  streams back a score. The CLI never calls your tools, so the three checks that need a tool call
163
163
  (`invoke-success-rate`, `visible-effect`, `hint-vs-observed`) read `Not applicable` on a local
164
164
  audit. The CLI scores nothing itself (design decision #1) — grade, findings,
@@ -211,17 +211,22 @@ page did not load, or a capture ora would refuse) · `3` API, a failed capture,
211
211
  returned. The gate reads the score, not the grade — a withheld grade would otherwise let
212
212
  everything through.
213
213
 
214
- ## deep-journey
214
+ <a id="deep-journey"></a>
215
+
216
+ ## journey, free on a curated task
215
217
 
216
218
  ```
217
- ax deep-journey <url> [--intent id | --task text] [--agent id] [--api-key k] [--no-stream] [--json]
219
+ ax journey --domain <site> [--intent id] [--agent id] [--api-key k] [--no-stream] [--json]
220
+ ax journey "<goal>" --domain <site> --api-key k [--agent id] [--no-stream] [--json]
218
221
  ```
219
222
 
220
223
  ```
221
- ax deep-journey stripe.com --intent pricing
222
- ax deep-journey zapier.com --task "Find how to build a Slack → Sheets zap" # needs a partner key
224
+ ax journey --domain stripe.com --intent pricing
225
+ ax journey "Find how to build a Slack → Sheets zap" --domain zapier.com --api-key k # needs a partner key
223
226
  ```
224
227
 
228
+ `ax deep-journey <url>` is the older name for this command. It keeps its flags (`--task` included) and exit codes, prints the `ax journey` call it stands for on stderr, and will be removed in a future major release.
229
+
225
230
  Runs a real AI agent at a site through ora's **public** journey API — no key, no workspace needed for curated tasks. The agent's trajectory streams live as progress lines; the terminal output is ora's verdict, the billable step count, and the generated insight:
226
231
 
227
232
  ```
@@ -238,22 +243,22 @@ Runs a real AI agent at a site through ora's **public** journey API — no key,
238
243
 
239
244
  Anonymous callers run curated intents (`pricing`, `signup`, `api-docs`, `integrate`, `support`, …) — list them with `GET https://ora.ai/api/journey/intents`, and the publicly offered agents with `GET https://ora.ai/api/journey/agents`. The public caps are 100 runs per rolling 24h per target and task, and 20 runs per 24h per IP (plus a 20/min burst limit and a shared ceiling of 3,000 anonymous runs a day across all callers); a target at its cap answers with the most recent stored run (`rate_limited: true`) instead of an error, so repeat CI invocations are cheap. Verdict and step count come from ora's contract as-is — the CLI never re-judges a run.
240
245
 
241
- **Keyed tier:** an ora-issued partner API key (`--api-key`, or `ORA_PARTNER_API_KEY` / `ORA_SCAN_API_KEY` in the environment) unlocks `--task` — a free-text task of 4–300 characters that replaces the curated intent — and moves the caller to an allowance of 1000 runs per rolling 24h per key with no per-target cap and no burst guard. Keys are issued manually by ora (no self-serve signup). A wrong or unrecognized key with a *curated* intent silently degrades to the anonymous tier; `--task` without a recognized key is an error.
246
+ **Keyed tier:** an ora-issued partner API key (`--api-key`, or `ORA_PARTNER_API_KEY` / `ORA_SCAN_API_KEY` in the environment) unlocks a goal of your own — a free-text task of 4–300 characters that replaces the curated intent — and moves the caller to an allowance of 1000 runs per rolling 24h per key with no per-target cap and no burst guard. Keys are issued manually by ora (no self-serve signup). A wrong or unrecognized key with a *curated* intent silently degrades to the anonymous tier; a goal of your own without a recognized key is an error. A goal with only a partner key set (no `ORA_API_KEY`) takes this path; with `ORA_API_KEY` set too, it runs on ora's platform unless you add a flag of this path (`--intent`, `--agent`, `--api-key`, `--no-stream`).
242
247
 
243
248
  The CLI always follows the run's stream to the end, `--no-stream` included, because ora saves a finished run from inside the stream. If the stream drops or goes quiet (no frames for 120s), the CLI reopens it, which resumes the same run without re-running the agent. A stream `429` is waited out (up to 60s) before reopening; any other HTTP error from the stream (a `404`, an auth error, a `5xx`) ends the command at once. Only after five failed reopens does it fall back to polling the run detail, within a 15-minute budget shared with the stream.
244
249
 
245
250
  | Flag | Effect |
246
251
  |---|---|
247
252
  | `--intent <id>` | Curated task id (server default when omitted) |
248
- | `--task <text>` | Free-text task — needs a partner API key; mutually exclusive with `--intent` |
253
+ | `"<goal>"` | A goal of your own; needs a partner API key and cannot be combined with `--intent` |
249
254
  | `--api-key <k>` | ora partner API key; also read from `ORA_PARTNER_API_KEY` (then `ORA_SCAN_API_KEY`) |
250
255
  | `--agent <id>` | Agent from the public roster (default: `cas-haiku`, Claude Code on the Haiku model ora's roster lists) |
251
256
  | `--no-stream` | Hide the live trajectory view and print only the final result |
252
257
  | `--json` | Print the terminal run detail (verdict, step_count, result) as JSON |
253
258
 
254
- deep-journey exit codes: `0` run succeeded (any verdict) · `1` run failed · `2` usage error · `3` API unreachable / rate-limited.
259
+ Exit codes: `0` run succeeded (any verdict) · `1` run failed · `2` usage error · `3` API unreachable / rate-limited.
255
260
 
256
- This differs from `ax journey` (below), which drives the authenticated platform with free-text goals and renders the full trajectory graph; `deep-journey` is the zero-setup public path.
261
+ Mixing these flags with `--harness` or `--model`, which are for a goal of your own on ora's platform ([below](#journey)), is a usage error (exit 2) naming both. This is the zero-setup public path; the platform path renders the full trajectory graph.
257
262
 
258
263
  ## skill
259
264
 
@@ -279,7 +284,7 @@ Drafts the files agents look for on a site, filled in from ora's last scan of th
279
284
  | File | Written to | What it is |
280
285
  |---|---|---|
281
286
  | `ard.json`, `ai-catalog.json` | `.well-known/` | An [Agentic Resource Discovery](https://agenticresourcediscovery.org/) catalog (the same document at the [AI Catalog Standard](https://ai-catalog.io/)'s path). Every entry is anchored to your domain, not to ora |
282
- | `poppy.json` | `.well-known/` | A [Personal Agent Protocol](https://personalagentprotocol.org/docs/spec) file for signed-out browsing; the OAuth issuer and APIs ora saw are printed as notes, because naming them is only correct once they implement the protocol |
287
+ | `poppy.json` | `.well-known/` | A [Personal Agent Protocol](https://personalagentprotocol.org/docs/1.0/spec) (version 1.0) file for signed-out browsing; the OAuth issuer and APIs ora saw are printed as notes, because naming them is only correct once they implement the protocol |
283
288
  | `llms.txt` | web root | An index of the docs, APIs and agent endpoints ora found; skipped when you already publish one |
284
289
  | `robots.txt` | web root | An edit of your current robots.txt: every line kept, a group added for AI search crawlers and user-requested agents your `User-agent: *` rules block (copying your other rules), and a `Sitemap:` line. Training crawlers and crawlers you block by name are never touched |
285
290
  | `api-catalog` | `.well-known/` | An [RFC 9727](https://www.rfc-editor.org/rfc/rfc9727) API catalog pointing at your OpenAPI description |
@@ -302,7 +307,7 @@ console.log(result.score, result.grade, result.topFixes);
302
307
 
303
308
  `result` is the raw versioned audit payload (`AuditScanResult` / `AuditScoreResult`, generated from ora's OpenAPI spec). `fetchSkill` / `fetchSkillIndex` expose the digest-verified skill registry, and `fetchAgentFiles` the agent files generator.
304
309
 
305
- The public journey client is exported too — the same no-key path as `ax deep-journey`:
310
+ The public journey client is exported too — the same no-key path as `ax journey --domain`:
306
311
 
307
312
  ```ts
308
313
  import { deepJourney, fetchJourneyAgents } from "ax";
@@ -321,12 +326,12 @@ console.log(detail.verdict, detail.step_count, detail.result?.insight?.summary);
321
326
 
322
327
  ## For ora partners
323
328
 
324
- Everything above runs without an account. The commands in this section are for teams working with ora directly, and there is no self-serve signup: keys are issued by ora (contact [hello@ora.ai](mailto:hello@ora.ai)). Two kinds of key exist. `journey` (and `audit --tunnel ora`) needs an ora platform key (`ora_sk_…`). A separate partner or scan key, passed with `--api-key`, lifts the anonymous rate limits on `audit` and `deep-journey`, and the daily limit on `webmcp-audit`.
329
+ Everything above runs without an account. The commands in this section are for teams working with ora directly, and there is no self-serve signup: keys are issued by ora (contact [hello@ora.ai](mailto:hello@ora.ai)). Two kinds of key exist. `journey` (and `audit --tunnel ora`) needs an ora platform key (`ora_sk_…`). A separate partner or scan key, passed with `--api-key`, lifts the anonymous rate limits on `audit` and the free `journey`, and the daily limit on `webmcp-audit`.
325
330
 
326
331
  ### journey
327
332
 
328
333
  ```
329
- ax journey "<intent>" [--domain d] [--harness h] [--model m] [--json]
334
+ ax journey "<goal>" [--domain d] [--harness h] [--model m] [--json]
330
335
  ```
331
336
 
332
337
  ```
@@ -392,16 +397,18 @@ A `.env` in the working directory is read on startup — copy `.env.example` and
392
397
 
393
398
  | Var | Used by | Default | Purpose |
394
399
  |---|---|---|---|
395
- | `ORA_API_URL` | audit, webmcp-audit, deep-journey, skill, generate-files | `https://ora.ai` | Public API base (no auth); also the ingest base for `webmcp-audit` |
400
+ | `ORA_API_URL` | audit, webmcp-audit, journey (free), skill, generate-files | `https://ora.ai` | Public API base (no auth); also the ingest base for `webmcp-audit` |
396
401
  | `ORA_PLATFORM_URL` | journey, audit `--tunnel ora` | `https://api.agentfront.sh` | Authenticated platform API base |
397
402
  | `ORA_TUNNEL` | audit | — (optional) | `ora` opens ora's own tunnel for a local target (same as `--tunnel ora`) |
398
403
  | `ORA_TUNNEL_CMD` | audit | — (optional) | Tunnel command for a local target (same as `--tunnel-cmd`); wins over `ORA_TUNNEL` |
399
404
  | `ORA_API_KEY` | journey, audit `--tunnel ora` | — (required for both) | ora platform key (`ora_sk_…`); exchanged for a bearer token (the tunnel needs the `tunnels:write` + `tunnels:connect` scopes) |
400
- | `ORA_SCAN_API_KEY` | audit, deep-journey, webmcp-audit | — (optional) | ora-issued scan API key; lifts every scan rate limit (issued manually by ora). deep-journey and webmcp-audit accept it as a partner-key fallback |
401
- | `ORA_PARTNER_API_KEY` | deep-journey, webmcp-audit | — (optional) | ora-issued partner API key; unlocks `--task` and the 1000/24h keyed allowance on deep-journey, and lifts webmcp-audit's daily limit |
405
+ | `ORA_SCAN_API_KEY` | audit, journey (free), webmcp-audit | — (optional) | ora-issued scan API key; lifts every scan rate limit (issued manually by ora). the free journey and webmcp-audit accept it as a partner-key fallback |
406
+ | `ORA_PARTNER_API_KEY` | journey (free), webmcp-audit | — (optional) | ora-issued partner API key; unlocks a goal of your own and the 1000/24h keyed allowance on the free journey, and lifts webmcp-audit's daily limit |
402
407
  | `CHROME_PATH` | webmcp-audit | — (auto-detected) | The Chrome to launch, when yours is not in a standard location |
403
408
  | `AX_WEBMCP_NO_SANDBOX` | webmcp-audit | — (unset) | `1` launches Chrome without its sandbox. Containers only: the audited page is untrusted and the sandbox contains it |
404
409
 
410
+ `ORA_API_URL` and `ORA_PLATFORM_URL` decide where your keys go, so ax checks them before it sends one. Their defaults always work. Any other host gets a key only over https (plain http only to this machine, such as `localhost`), and only when you set the variable in your shell: a `.env` belongs to whatever directory you run in, so a host named there is refused before anything is sent (`ax journey` exits 2 with `KEY_DESTINATION_REFUSED`). The exceptions are `localhost`, `127.0.0.1` and `[::1]`. Requests that carry no key are not checked.
411
+
405
412
  ## Contract versioning
406
413
 
407
414
  The audit types are generated from ora's served OpenAPI spec (`pnpm contract:gen`); the build is pinned to a contract version and CI fails when production drifts from the checked-in types. At runtime, the CLI prints one stderr warning when ora reports a newer contract than the build — upgrade with `npm i -g ax@latest`.
@@ -29,7 +29,7 @@ export interface AgentFilesBundle {
29
29
  /**
30
30
  * Any failure to get the files from ora. `status` is the HTTP status when ora
31
31
  * answered (400 = not a public domain), and `payload` the error body exactly as
32
- * served, which is what `--json` prints.
32
+ * served, whose `errorCode` is the code `--json` reports.
33
33
  */
34
34
  export declare class AgentFilesApiError extends Error {
35
35
  readonly status: number | null;
@@ -1,6 +1,32 @@
1
- import type { JourneyRun, JourneyRunDetail, JourneyTrajectoryStep } from "../contract";
2
- /** Any failure to obtain a run from ora: network, HTTP, rate limit, stream error. */
1
+ import type { JourneyNextAction, JourneyRun, JourneyRunDetail, JourneyTrajectoryStep } from "../contract";
2
+ /** Where a run is started: `ax journey --domain <site>` is the ax call that sends this POST. */
3
+ export declare const JOURNEY_RUNS_PATH = "/api/journey/runs";
4
+ export type DeepJourneyErrorCode = "NEEDS_KEY" | "RATE_LIMITED" | "USAGE_LIMIT_REACHED" | "ENGINE_UNAVAILABLE" | "RUN_STILL_RUNNING";
5
+ /**
6
+ * The `next_action` a journey body carries (a run detail or an error body),
7
+ * when it has one in the REST shape; anything else carries none.
8
+ */
9
+ export declare function servedNextAction(body: unknown): JourneyNextAction | undefined;
10
+ /**
11
+ * Any failure to obtain a run from ora: network, HTTP, rate limit, stream error.
12
+ * `code` names the failures a caller answers differently (a key to fix, a wait,
13
+ * a run to read rather than start again) and is null for the rest; `runUrl` is
14
+ * the run's detail URL when the run exists and is still executing; `payload`
15
+ * is the error body exactly as ora served it when ora answered with JSON
16
+ * (what `--json` prints), and null or undefined otherwise; `nextAction` is the
17
+ * request ora says to send next, from that body or from the run's detail.
18
+ */
3
19
  export declare class DeepJourneyApiError extends Error {
20
+ readonly code: DeepJourneyErrorCode | null;
21
+ readonly runUrl: string | undefined;
22
+ readonly payload: unknown;
23
+ readonly nextAction: JourneyNextAction | undefined;
24
+ constructor(message: string, options?: {
25
+ code?: DeepJourneyErrorCode | null;
26
+ runUrl?: string;
27
+ payload?: unknown;
28
+ nextAction?: JourneyNextAction;
29
+ });
4
30
  }
5
31
  export interface JourneyIntentOption {
6
32
  id: string;
@@ -81,6 +107,8 @@ export interface DeepJourneyOutcome {
81
107
  /** The stream's terminal error message, when the run failed mid-stream. */
82
108
  engineError?: string;
83
109
  }
110
+ /** The origin journey requests go to: the override, else $ORA_API_URL, else https://ora.ai. */
111
+ export declare function apiBase(baseUrl?: string): string;
84
112
  /** The curated intents a public run can execute. */
85
113
  export declare function fetchJourneyIntents(baseUrl?: string): Promise<{
86
114
  intents: JourneyIntentOption[];
@@ -0,0 +1,19 @@
1
+ /** The variables that move where ax sends a key. */
2
+ export type KeyBaseVariable = "ORA_PLATFORM_URL" | "ORA_API_URL";
3
+ /** ax refused to send a key to the host a base URL names. */
4
+ export declare class KeyDestinationError extends Error {
5
+ constructor(message: string);
6
+ }
7
+ /**
8
+ * Why ax will not send a key to `base`, or undefined when it may.
9
+ *
10
+ * The default host is always fine. Any other host gets the key only over https
11
+ * (plain http only to this machine), and only when the user chose it: a host
12
+ * that came from a `.env` in the current directory is refused, because that
13
+ * file may belong to whatever repository ax happens to run in.
14
+ */
15
+ export declare function keyDestinationProblem(base: string, variable: KeyBaseVariable, defaultBase: string): string | undefined;
16
+ /** Throws `KeyDestinationError` when ax must not send a key to `base`. */
17
+ export declare function assertKeyDestination(base: string, variable: KeyBaseVariable, defaultBase: string): void;
18
+ /** The `why` of the retry line printed after a refused key destination. */
19
+ export declare const KEY_DESTINATION_WHY = "once ORA_PLATFORM_URL is fixed as said above";
@@ -1,5 +1,19 @@
1
1
  /** The platform API base: explicit override, else $ORA_PLATFORM_URL, else prod. */
2
2
  export declare function platformBase(override?: string): string;
3
+ /** Why the key must not go to the platform `platformBase()` names, checked before any call. */
4
+ export declare function platformKeyProblem(): string | undefined;
5
+ /**
6
+ * The platform answered with an HTTP error. `payload` is the error body
7
+ * exactly as served when it was JSON (what `--json` prints), null otherwise.
8
+ */
9
+ export declare class PlatformApiError extends Error {
10
+ readonly status: number;
11
+ readonly payload: unknown;
12
+ constructor(message: string, options: {
13
+ status: number;
14
+ payload?: unknown;
15
+ });
16
+ }
3
17
  export interface RunSummary {
4
18
  run_id: string;
5
19
  domain?: string;
@@ -1,4 +1,15 @@
1
+ export type SkillErrorCode = "REGISTRY_UNREACHABLE" | "REGISTRY_ERROR" | "UNKNOWN_SKILL" | "DIGEST_MISMATCH";
2
+ /**
3
+ * Any failure to get a skill from the registry; `code` says which. `payload`
4
+ * is the registry's error body exactly as served when it was JSON (what
5
+ * `--json` prints), null or undefined otherwise.
6
+ */
1
7
  export declare class SkillApiError extends Error {
8
+ readonly code: SkillErrorCode;
9
+ readonly payload: unknown;
10
+ constructor(message: string, code: SkillErrorCode, options?: {
11
+ payload?: unknown;
12
+ });
2
13
  }
3
14
  export interface SkillEntry {
4
15
  name: string;
@@ -23,3 +34,11 @@ export interface FetchedSkill {
23
34
  * for.
24
35
  */
25
36
  export declare function fetchSkill(name: string, baseUrl?: string): Promise<FetchedSkill>;
37
+ /**
38
+ * The section of a SKILL.md under the heading `title` (any level, matched
39
+ * case-insensitively), heading included, up to the next heading of the same
40
+ * or a higher level. A `#` line inside a fenced code block is content, not a
41
+ * heading: ora's skills carry example reports that start with `## `. Null when
42
+ * the skill has no such heading.
43
+ */
44
+ export declare function skillSection(markdown: string, title: string): string | null;
@@ -10,6 +10,7 @@ export type JourneyCappedRun = components["schemas"]["JourneyCappedRun"];
10
10
  export type JourneyRunDetail = components["schemas"]["JourneyRunDetail"];
11
11
  export type JourneyRunResult = components["schemas"]["JourneyRunResult"];
12
12
  export type JourneyTrajectoryStep = components["schemas"]["JourneyTrajectoryStep"];
13
+ export type JourneyNextAction = components["schemas"]["JourneyNextAction"];
13
14
  export { BUILT_AGAINST };
14
15
  /**
15
16
  * True when the server reports a newer contract (major or minor) than the one