@appliqation/autopilot 0.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +302 -0
- package/dist/cli/audit.js +23 -0
- package/dist/cli/audit.js.map +1 -0
- package/dist/cli/index.js +134 -0
- package/dist/cli/index.js.map +1 -0
- package/dist/config/env.js +56 -0
- package/dist/config/env.js.map +1 -0
- package/dist/orchestrator/autopilot.js +45 -0
- package/dist/orchestrator/autopilot.js.map +1 -0
- package/dist/policy/systemPrompt.js +159 -0
- package/dist/policy/systemPrompt.js.map +1 -0
- package/dist/tools/metaTools.js +220 -0
- package/dist/tools/metaTools.js.map +1 -0
- package/dist/tools/safety.js +33 -0
- package/dist/tools/safety.js.map +1 -0
- package/package.json +36 -0
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Appliqation Pty Ltd
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
ADDED
|
@@ -0,0 +1,302 @@
|
|
|
1
|
+
# Appliqation Autopilot
|
|
2
|
+
|
|
3
|
+
**An agentic orchestrator that decides — for real, every time — whether a test case needs
|
|
4
|
+
autonomous testing, a defect fix, new automation, or a pull request, instead of running a
|
|
5
|
+
fixed script.**
|
|
6
|
+
|
|
7
|
+
Most "AI test automation" tooling is a pipeline with an LLM bolted onto one step. Autopilot
|
|
8
|
+
is different: given one test case, it gathers real signal (current pass/fail state,
|
|
9
|
+
flakiness, linked defects, coverage priority, whether automation already exists), reasons
|
|
10
|
+
about what's actually warranted the way a senior QA engineer scoping their day would, states
|
|
11
|
+
a plan, and executes it — checking real results after every action and adapting when reality
|
|
12
|
+
disagrees with the plan, rather than committing blindly upfront.
|
|
13
|
+
|
|
14
|
+
It orchestrates five independent, single-purpose agents as ordinary tools it can call. None
|
|
15
|
+
of them know Autopilot exists. Each does one thing well and can be scripted directly if you
|
|
16
|
+
want deterministic control instead. Autopilot is the layer above them that decides *when* and
|
|
17
|
+
*whether* to use each one.
|
|
18
|
+
|
|
19
|
+
## Introducing the agents
|
|
20
|
+
|
|
21
|
+
| Agent | Does | Repo |
|
|
22
|
+
|---|---|---|
|
|
23
|
+
| **Autotest** | Runs a test case in a real browser; a second, independent AI judges the result from evidence alone — never its own claim. | [`appliqation-autotest`](https://github.com/appliqation/appliqation-autotest) |
|
|
24
|
+
| **Defect-Fix** | Loads full defect context, locates and applies a real code fix, syncs the scenario, verifies with a real Playwright run. | [`appliqation-defect-fix`](https://github.com/appliqation/appliqation-defect-fix) |
|
|
25
|
+
| **Scriptgen** | Drafts a Playwright script for an untested-but-passing test case, iterating against real runs until genuinely green. | [`appliqation-scriptgen`](https://github.com/appliqation/appliqation-scriptgen) |
|
|
26
|
+
| **PR-Raise** | Fully mechanical, no LLM: commits whatever's already changed, pushes, opens or reuses a pull request. | [`appliqation-pr-raise`](https://github.com/appliqation/appliqation-pr-raise) |
|
|
27
|
+
| **Explorer** | Open-ended exploratory QA — a senior-QA heuristics pass plus security/network/caching/mobile probes, headlessly, the coverage a scripted test case never checks for. | [`appliqation-explorer`](https://github.com/appliqation/appliqation-explorer) |
|
|
28
|
+
|
|
29
|
+
All five share [`@appliqation/agent-core`](https://github.com/appliqation/appliqation-agent-core), the generic think→act→observe engine, budget tracking, and tool-dispatch machinery underneath each of them.
|
|
30
|
+
|
|
31
|
+
## Why this is different from "wire an LLM to some CLIs"
|
|
32
|
+
|
|
33
|
+
- **The reasoning is real, not decorative.** The system prompt (`src/policy/systemPrompt.ts`)
|
|
34
|
+
encodes an actual decision framework — not "call these tools in order," but genuine
|
|
35
|
+
branches: a currently-*failing* test case does **not** get a script generated for it (that
|
|
36
|
+
would encode broken behaviour as a false baseline); a flaky one gets a script but with
|
|
37
|
+
explicitly lowered confidence in the report; a low-priority one may legitimately get *no*
|
|
38
|
+
action at all, just a recommendation. Taking no action is a valid outcome, not a failure to
|
|
39
|
+
route.
|
|
40
|
+
- **It never fabricates an outcome.** Every claim in the final report has to trace back to a
|
|
41
|
+
real tool result. `run_generate`'s `testRun.ok` and `run_defect_fix`'s `verified` reflect an
|
|
42
|
+
actually-executed Playwright run, not the model's own claim about it. `run_judge`'s status is
|
|
43
|
+
polled from Appliqation's own authoritative run record, not parsed out of report prose. If
|
|
44
|
+
Autopilot says a test passes, it's because it watched that happen.
|
|
45
|
+
- **The reasoning lives here, in the open — not behind a private API.** The five sibling
|
|
46
|
+
agents are genuinely self-contained; the *judgment* about how to combine them is this
|
|
47
|
+
repo's own code, fully readable, forkable, and swappable (see
|
|
48
|
+
[Customizing the policy](#customizing-the-policy)). Nothing about how this agent thinks is
|
|
49
|
+
hidden behind a server only Appliqation can change. One concrete example:
|
|
50
|
+
`appliqation-defect-fix` has no way to know, on its own, how much testing a given fix
|
|
51
|
+
actually needs verified — that call is genuinely Autopilot's to make, from the broader
|
|
52
|
+
signal (defect history, run context) it's already gathering, and it's required to state
|
|
53
|
+
that reasoning explicitly rather than pass the sibling agent a generic instruction.
|
|
54
|
+
- **Raising a pull request is opt-in, not assumed.** `run_pr_raise` isn't even in the tool
|
|
55
|
+
list Autopilot's model sees unless you pass `--allow-pr`. Without it, Autopilot still
|
|
56
|
+
reasons about whether a PR would be warranted — it just tells you so instead of doing it.
|
|
57
|
+
|
|
58
|
+
## The complete agentic system
|
|
59
|
+
|
|
60
|
+
```mermaid
|
|
61
|
+
flowchart TB
|
|
62
|
+
subgraph Autopilot["appliqation-autopilot"]
|
|
63
|
+
direction TB
|
|
64
|
+
Loop["think → act → observe loop"]
|
|
65
|
+
Policy["policy (system prompt) =<br/>the actual decision framework"]
|
|
66
|
+
Policy -.drives.-> Loop
|
|
67
|
+
end
|
|
68
|
+
|
|
69
|
+
Ctx["read-only Appliqation context:<br/>get_scenario, get_failure_patterns,<br/>get_defect_context, get_coverage_analysis,<br/>get_automation_readiness, enrich_project_context<br/>(read-only — action=write is refused), ..."]
|
|
70
|
+
|
|
71
|
+
Ctx --> Loop
|
|
72
|
+
Loop -->|run_judge| Autotest[appliqation-autotest]
|
|
73
|
+
Loop -->|run_defect_fix| DefectFix[appliqation-defect-fix]
|
|
74
|
+
Loop -->|run_generate| Scriptgen[appliqation-scriptgen]
|
|
75
|
+
Loop -->|"run_pr_raise<br/>(only if --allow-pr)"| PrRaise[appliqation-pr-raise]
|
|
76
|
+
Loop -->|"run_explore<br/>(when Phase 2 states a real reason)"| Explorer[appliqation-explorer]
|
|
77
|
+
|
|
78
|
+
Autotest -->|real polled verdict| Loop
|
|
79
|
+
DefectFix -->|"verified: true/false"| Loop
|
|
80
|
+
Scriptgen -->|"testRun.ok: true/false"| Loop
|
|
81
|
+
PrRaise -->|PR URL or committed: false| Loop
|
|
82
|
+
Explorer -->|"findings report + budgetExceeded"| Loop
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
- **Context tools** are ordinary read-only Appliqation MCP tools — the same signal a human QA lead would look at before deciding where to spend effort. This includes the project's own living context document (`enrich_project_context`, action=read) — known issues, high-risk areas, regression watchlist, pain points, personas — so a TC in a known-risky area gets weighed differently than the same raw evidence somewhere unremarkable. The tool also has a write mode; autopilot can only ever read it — see [Safety](#safety).
|
|
86
|
+
- **Action tools** each spawn the corresponding sibling agent's CLI as a real subprocess with `--json`, and hand the model back the exact, real structured result — never a summary of one.
|
|
87
|
+
- **The policy** (`src/policy/systemPrompt.ts`) is the one piece of genuine "brain" — a detailed, phase-based methodology for gathering context, forming a plan, executing it adaptively, and reporting honestly. It's a plain string. Read it, fork it, replace it.
|
|
88
|
+
|
|
89
|
+
## Workflow options
|
|
90
|
+
|
|
91
|
+
Autopilot's judgment is one way to use this family — not the only one. Here are four real shapes, from fully autonomous to fully scripted.
|
|
92
|
+
|
|
93
|
+
### 1. Full autonomous mode
|
|
94
|
+
|
|
95
|
+
Point Autopilot at a test case and let it decide everything: gather context, judge current state, fix or generate as warranted, raise the PR.
|
|
96
|
+
|
|
97
|
+
```mermaid
|
|
98
|
+
sequenceDiagram
|
|
99
|
+
participant You
|
|
100
|
+
participant AP as Autopilot
|
|
101
|
+
participant AT as Autotest
|
|
102
|
+
participant DF as Defect-Fix
|
|
103
|
+
participant SG as Scriptgen
|
|
104
|
+
participant PR as PR-Raise
|
|
105
|
+
|
|
106
|
+
You->>AP: run --test-case-uuid X --allow-pr
|
|
107
|
+
AP->>AP: gather context, form a plan
|
|
108
|
+
alt no recent evidence
|
|
109
|
+
AP->>AT: run_judge
|
|
110
|
+
AT-->>AP: real verdict
|
|
111
|
+
end
|
|
112
|
+
alt currently fails, fixable defect exists
|
|
113
|
+
AP->>DF: run_defect_fix (+ test_instruction)
|
|
114
|
+
DF-->>AP: verified: true/false
|
|
115
|
+
else currently passes, no canonical script
|
|
116
|
+
AP->>SG: run_generate
|
|
117
|
+
SG-->>AP: testRun.ok: true/false
|
|
118
|
+
end
|
|
119
|
+
alt something was verified
|
|
120
|
+
AP->>PR: run_pr_raise
|
|
121
|
+
PR-->>AP: PR URL
|
|
122
|
+
end
|
|
123
|
+
AP-->>You: full report, every claim tool-backed
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
```bash
|
|
127
|
+
npx appliqation-autopilot run --test-case-uuid <uuid> --environment Stage --repo-path <path> --allow-pr
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
### 2. Deterministic CI pipeline (no orchestrator)
|
|
131
|
+
|
|
132
|
+
Skip Autopilot entirely and script the individual agents directly — full control, zero LLM judgment about *what* to run, still LLM-verified per step.
|
|
133
|
+
|
|
134
|
+
```mermaid
|
|
135
|
+
flowchart LR
|
|
136
|
+
A[CI trigger] --> B["appliqation-autotest judge<br/>--test-set-id regression"]
|
|
137
|
+
B --> C{all passed?}
|
|
138
|
+
C -- yes --> D[deploy]
|
|
139
|
+
C -- no --> E[fail the build,<br/>file/notify]
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
```bash
|
|
143
|
+
npx appliqation-autotest judge --test-set-id <id> --environment Stage --ci
|
|
144
|
+
```
|
|
145
|
+
|
|
146
|
+
### 3. Defect triage & auto-fix
|
|
147
|
+
|
|
148
|
+
A regression run surfaces failures and defects get filed. Route each one through Defect-Fix (directly, or via Autopilot for scope judgment), open a PR, let a human review.
|
|
149
|
+
|
|
150
|
+
```mermaid
|
|
151
|
+
flowchart LR
|
|
152
|
+
A["CI run finds failures<br/>(defects filed in Appliqation)"] --> B["appliqation-defect-fix fix<br/>--defect-id <id>"]
|
|
153
|
+
B --> C{verified?}
|
|
154
|
+
C -- yes --> D["appliqation-pr-raise raise"]
|
|
155
|
+
C -- no --> E[report what's still broken,<br/>no PR]
|
|
156
|
+
D --> F[human reviews the PR]
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
```bash
|
|
160
|
+
npx appliqation-defect-fix fix --defect-id <id> --repo-path <path> --dry-run # first pass, safe
|
|
161
|
+
npx appliqation-defect-fix fix --defect-id <id> --repo-path <path> # once trusted
|
|
162
|
+
```
|
|
163
|
+
|
|
164
|
+
### 4. Coverage backfill
|
|
165
|
+
|
|
166
|
+
Systematically pay down test-automation debt: find passing-but-unautomated test cases, generate and verify canonical scripts for each, batch them into a PR.
|
|
167
|
+
|
|
168
|
+
```mermaid
|
|
169
|
+
flowchart LR
|
|
170
|
+
A["get_automation_readiness:<br/>passing TCs with no canonical script"] --> B["appliqation-scriptgen generate<br/>per test case"]
|
|
171
|
+
B --> C{testRun.ok?}
|
|
172
|
+
C -- yes --> D[accumulate written files]
|
|
173
|
+
C -- no --> E[skip, flag for follow-up]
|
|
174
|
+
D --> F["appliqation-pr-raise raise<br/>(one PR, batched)"]
|
|
175
|
+
```
|
|
176
|
+
|
|
177
|
+
```bash
|
|
178
|
+
npx appliqation-scriptgen generate --test-case-uuid <uuid> --repo-path <path> --ci
|
|
179
|
+
```
|
|
180
|
+
|
|
181
|
+
## Quick start
|
|
182
|
+
|
|
183
|
+
```bash
|
|
184
|
+
npm install -g @appliqation/autopilot
|
|
185
|
+
```
|
|
186
|
+
|
|
187
|
+
You'll also need whichever sibling agents autopilot is allowed to call — install the ones
|
|
188
|
+
you want reachable (see [Workflow options](#workflow-options) above for real combinations;
|
|
189
|
+
you don't need all five for every use case):
|
|
190
|
+
|
|
191
|
+
```bash
|
|
192
|
+
npm install -g @appliqation/autotest @appliqation/defect-fix \
|
|
193
|
+
@appliqation/scriptgen @appliqation/pr-raise @appliqation/explorer
|
|
194
|
+
```
|
|
195
|
+
|
|
196
|
+
Each is a plain command name by default (`AUTOTEST_CMD`/`DEFECT_FIX_CMD`/`SCRIPTGEN_CMD`/
|
|
197
|
+
`PR_RAISE_CMD`/`EXPLORER_CMD` in `.env`) — only override these if you're pointing at a local
|
|
198
|
+
development build instead.
|
|
199
|
+
|
|
200
|
+
Create a `.env` file (in whatever directory you'll run it from) with:
|
|
201
|
+
|
|
202
|
+
```
|
|
203
|
+
APPQ_API_KEY=your-appliqation-api-key
|
|
204
|
+
ANTHROPIC_API_KEY=your-anthropic-key # or OPENAI_API_KEY — pick one
|
|
205
|
+
```
|
|
206
|
+
|
|
207
|
+
```bash
|
|
208
|
+
appliqation-autopilot run \
|
|
209
|
+
--test-case-uuid <uuid> \
|
|
210
|
+
--environment Stage \
|
|
211
|
+
--repo-path /path/to/your/checkout
|
|
212
|
+
```
|
|
213
|
+
|
|
214
|
+
Watch stderr — every context tool call, every action, and the model's own reasoning
|
|
215
|
+
(`[thinking]` lines) stream live. Add `--allow-pr` once you're ready to let it actually open
|
|
216
|
+
pull requests; add `--json`/`--ci` for a single structured result and a CI-friendly exit code
|
|
217
|
+
instead of the human-readable transcript.
|
|
218
|
+
|
|
219
|
+
## Customizing the policy
|
|
220
|
+
|
|
221
|
+
The default policy in `src/policy/systemPrompt.ts` is opinionated: don't automate a currently
|
|
222
|
+
failing test, flag flaky results as lower-confidence, treat "no action needed" as a legitimate
|
|
223
|
+
outcome. Your organization might weigh these differently — more conservative, a different
|
|
224
|
+
priority order, a different report format for your own stakeholders.
|
|
225
|
+
|
|
226
|
+
You don't need to touch anything else in this repo to change that:
|
|
227
|
+
|
|
228
|
+
```bash
|
|
229
|
+
npx appliqation-autopilot run --policy ./my-policy.md ...
|
|
230
|
+
# or set POLICY_FILE in .env
|
|
231
|
+
```
|
|
232
|
+
|
|
233
|
+
Anything you put in that file becomes the system prompt driving every decision. The
|
|
234
|
+
orchestration code (`src/orchestrator/`, `src/tools/`) doesn't change — it just runs whatever
|
|
235
|
+
policy it's given against the same tools.
|
|
236
|
+
|
|
237
|
+
If what you actually want is full deterministic control with no LLM judgment in the loop at
|
|
238
|
+
all, you don't need Autopilot for that — script `appliqation-autotest`,
|
|
239
|
+
`appliqation-defect-fix`, `appliqation-scriptgen`, and `appliqation-pr-raise` directly; each
|
|
240
|
+
is a complete, independently useful CLI (see [workflow 2](#2-deterministic-ci-pipeline-no-orchestrator)).
|
|
241
|
+
|
|
242
|
+
## Safety
|
|
243
|
+
|
|
244
|
+
- `run_pr_raise` is excluded from the tool list entirely unless `--allow-pr` is passed —
|
|
245
|
+
a hardcoded exclusion, not a soft warning the model could talk itself past.
|
|
246
|
+
- Every meta-tool result is the sibling agent's own real `--json` output — including on
|
|
247
|
+
failure (a failed/blocked `run_judge`, an unverified `run_generate`/`run_defect_fix`), so a
|
|
248
|
+
bad outcome is visible to the model as data to reason about, never swallowed.
|
|
249
|
+
- The individual agents carry their own safety invariants independently — a destructive-action
|
|
250
|
+
gate on any browser interaction, an allowlisted shell surface for `appliqation-scriptgen`'s
|
|
251
|
+
and `appliqation-defect-fix`'s environment bootstrap, `appliqation-defect-fix`'s own appq
|
|
252
|
+
writes gated behind its own `--dry-run` (which Autopilot can pass through via
|
|
253
|
+
`run_defect_fix`'s `dry_run` argument), no credentials ever flowing through an LLM's own
|
|
254
|
+
context. Autopilot doesn't weaken any of that; it just decides when to invoke it.
|
|
255
|
+
- No credentials of any kind pass through the LLM's context at any point in this repo.
|
|
256
|
+
- `enrich_project_context` is a single Appliqation tool with both `action=read` and
|
|
257
|
+
`action=write` modes — tool-*name* allowlisting alone can't express "this tool, but
|
|
258
|
+
only this argument value," so `@appliqation/agent-core`'s `createReadOnlyProjectContextDispatcher`
|
|
259
|
+
adds an argument-level gate on top: only `action=read` is ever let through, and the
|
|
260
|
+
check fails closed (a missing or malformed `action` is refused too, not just an
|
|
261
|
+
explicit `"write"`). Shared with `appliqation-explorer`, which needs the identical
|
|
262
|
+
guarantee.
|
|
263
|
+
- `appliqation-explorer` is read-only end to end — including on the project context
|
|
264
|
+
document its own upstream workflow (`appq:runman`) would otherwise write to. When that
|
|
265
|
+
workflow runs interactively in Claude Code, a human is present at its confirmation gate
|
|
266
|
+
before anything gets persisted as fact for future passes; a headless `run_explore` call
|
|
267
|
+
has no equivalent, so it holds the same conservative default as Autopilot itself rather
|
|
268
|
+
than the permissive one baked into the interactive prompt. See that repo's README for
|
|
269
|
+
the full reasoning.
|
|
270
|
+
|
|
271
|
+
## Configuration
|
|
272
|
+
|
|
273
|
+
See `.env.example` for the full list. In short: `APPQ_API_KEY` + one LLM provider key are
|
|
274
|
+
required; `AUTOTEST_CMD`/`DEFECT_FIX_CMD`/`SCRIPTGEN_CMD`/`PR_RAISE_CMD`/`EXPLORER_CMD` tell
|
|
275
|
+
Autopilot how to reach the sibling agents; `BUDGET_MAX_*` caps the tool-calling loop;
|
|
276
|
+
`POLICY_FILE` points at a custom policy.
|
|
277
|
+
|
|
278
|
+
Optionally, `AUDIT_MONGO_URI`/`AUDIT_MONGO_DB`/`AUDIT_MONGO_COLLECTION` or
|
|
279
|
+
`AUDIT_JSONL_PATH` records one audit entry per invocation (token usage, duration, real
|
|
280
|
+
outcome) to a datastore this agent family owns — deliberately not part of Appliqation
|
|
281
|
+
itself, since this is a parallel system that uses it, not a feature of it. Every sibling
|
|
282
|
+
agent in the family writes to the same shape; [`appliqation-dashboard`](https://github.com/appliqation/appliqation-dashboard)
|
|
283
|
+
reads it back as an aggregated report. Entirely opt-in — nothing is recorded unless one
|
|
284
|
+
of these is set, and a write failure never affects a real run's outcome.
|
|
285
|
+
|
|
286
|
+
## Development
|
|
287
|
+
|
|
288
|
+
```bash
|
|
289
|
+
git clone https://github.com/appliqation/appliqation-autopilot.git
|
|
290
|
+
cd appliqation-autopilot
|
|
291
|
+
npm install
|
|
292
|
+
cp .env.example .env # fill in APPQ_API_KEY and one LLM provider key
|
|
293
|
+
npm run dev -- run --test-case-uuid <uuid> --environment <name> --repo-path <path>
|
|
294
|
+
npm run typecheck
|
|
295
|
+
npm test
|
|
296
|
+
```
|
|
297
|
+
|
|
298
|
+
See `CLAUDE.md` for a map of this repo if you're working in it with an AI coding assistant.
|
|
299
|
+
|
|
300
|
+
## License
|
|
301
|
+
|
|
302
|
+
MIT — see [LICENSE](./LICENSE).
|
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
// Extracted out of cli/index.ts so this is testable without triggering that
|
|
2
|
+
// file's top-level program.parseAsync(process.argv) side effect — same
|
|
3
|
+
// reasoning as appliqation-autotest's cli/resolvers.ts.
|
|
4
|
+
import { safeRecord } from '@appliqation/agent-core';
|
|
5
|
+
export async function recordAutopilotRun(args) {
|
|
6
|
+
const { sink, startedAt, endedAt, model, usage, testCaseUuid, environment, repoPath, allowPr, result } = args;
|
|
7
|
+
await safeRecord(sink, {
|
|
8
|
+
agent: 'appliqation-autopilot',
|
|
9
|
+
subcommand: 'run',
|
|
10
|
+
startedAt,
|
|
11
|
+
endedAt,
|
|
12
|
+
durationMillis: endedAt - startedAt,
|
|
13
|
+
model,
|
|
14
|
+
usage,
|
|
15
|
+
turns: result?.turns,
|
|
16
|
+
budgetExceeded: result?.budgetExceeded,
|
|
17
|
+
exitCode: result ? 0 : 1,
|
|
18
|
+
outcome: result
|
|
19
|
+
? { testCaseUuid, environment, repoPath, allowPr, turns: result.turns, budgetExceeded: result.budgetExceeded, report: result.report }
|
|
20
|
+
: { testCaseUuid, environment, repoPath, allowPr, error: true },
|
|
21
|
+
});
|
|
22
|
+
}
|
|
23
|
+
//# sourceMappingURL=audit.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"audit.js","sourceRoot":"","sources":["../../src/cli/audit.ts"],"names":[],"mappings":"AAAA,4EAA4E;AAC5E,uEAAuE;AACvE,wDAAwD;AAExD,OAAO,EAAE,UAAU,EAAoC,MAAM,yBAAyB,CAAC;AAiBvF,MAAM,CAAC,KAAK,UAAU,kBAAkB,CAAC,IAA4B;IACnE,MAAM,EAAE,IAAI,EAAE,SAAS,EAAE,OAAO,EAAE,KAAK,EAAE,KAAK,EAAE,YAAY,EAAE,WAAW,EAAE,QAAQ,EAAE,OAAO,EAAE,MAAM,EAAE,GAAG,IAAI,CAAC;IAC9G,MAAM,UAAU,CAAC,IAAI,EAAE;QACrB,KAAK,EAAE,uBAAuB;QAC9B,UAAU,EAAE,KAAK;QACjB,SAAS;QACT,OAAO;QACP,cAAc,EAAE,OAAO,GAAG,SAAS;QACnC,KAAK;QACL,KAAK;QACL,KAAK,EAAE,MAAM,EAAE,KAAK;QACpB,cAAc,EAAE,MAAM,EAAE,cAAc;QACtC,QAAQ,EAAE,MAAM,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;QACxB,OAAO,EAAE,MAAM;YACb,CAAC,CAAC,EAAE,YAAY,EAAE,WAAW,EAAE,QAAQ,EAAE,OAAO,EAAE,KAAK,EAAE,MAAM,CAAC,KAAK,EAAE,cAAc,EAAE,MAAM,CAAC,cAAc,EAAE,MAAM,EAAE,MAAM,CAAC,MAAM,EAAE;YACrI,CAAC,CAAC,EAAE,YAAY,EAAE,WAAW,EAAE,QAAQ,EAAE,OAAO,EAAE,KAAK,EAAE,IAAI,EAAE;KAClE,CAAC,CAAC;AACL,CAAC"}
|
|
@@ -0,0 +1,134 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
// `run`: the agentic orchestrator. Reasons over real context to decide
|
|
3
|
+
// whether to run autonomous testing, generate new automation, or raise a
|
|
4
|
+
// PR for it — see src/policy/systemPrompt.ts for the actual methodology
|
|
5
|
+
// (the one real customization point) and src/orchestrator/autopilot.ts for
|
|
6
|
+
// the mechanism.
|
|
7
|
+
import { readFile } from 'node:fs/promises';
|
|
8
|
+
import { Command } from 'commander';
|
|
9
|
+
import { createMcpClient, createAnthropicAdapter, createOpenAiAdapter, createUsageAccumulator } from '@appliqation/agent-core';
|
|
10
|
+
import { config, resolveProvider, resolveModel } from '../config/env.js';
|
|
11
|
+
import { autopilot } from '../orchestrator/autopilot.js';
|
|
12
|
+
import { recordAutopilotRun } from './audit.js';
|
|
13
|
+
function buildAdapter() {
|
|
14
|
+
const provider = resolveProvider();
|
|
15
|
+
const model = resolveModel();
|
|
16
|
+
return provider === 'anthropic'
|
|
17
|
+
? createAnthropicAdapter(config.anthropicApiKey, model, config.anthropicMaxTokens)
|
|
18
|
+
: createOpenAiAdapter(config.openaiApiKey, model, config.openaiMaxOutputTokens);
|
|
19
|
+
}
|
|
20
|
+
function logEvent(prefix) {
|
|
21
|
+
return (e) => {
|
|
22
|
+
if (e.type === 'assistant') {
|
|
23
|
+
const text = (e.detail ?? '').trim();
|
|
24
|
+
if (text)
|
|
25
|
+
console.error(`${prefix}[thinking] ${text}`);
|
|
26
|
+
}
|
|
27
|
+
else if (e.type === 'tool') {
|
|
28
|
+
const d = e.detail;
|
|
29
|
+
console.error(`${prefix}[tool] ${d.name} -> ${d.result.slice(0, 400)}`);
|
|
30
|
+
}
|
|
31
|
+
else if (e.type === 'log') {
|
|
32
|
+
console.error(`${prefix}[log] ${e.detail}`);
|
|
33
|
+
}
|
|
34
|
+
else if (e.type === 'usage') {
|
|
35
|
+
const u = e.detail;
|
|
36
|
+
const cacheNote = u.cacheReadTokens
|
|
37
|
+
? ` (${u.cacheReadTokens} from cache)`
|
|
38
|
+
: u.cacheWriteTokens
|
|
39
|
+
? ` (${u.cacheWriteTokens} written to cache)`
|
|
40
|
+
: '';
|
|
41
|
+
console.error(`${prefix}[usage] in=${u.inputTokens} out=${u.outputTokens}${cacheNote}`);
|
|
42
|
+
}
|
|
43
|
+
};
|
|
44
|
+
}
|
|
45
|
+
const program = new Command();
|
|
46
|
+
program
|
|
47
|
+
.name('appliqation-autopilot')
|
|
48
|
+
.description('An agentic orchestrator that reasons over real context (current pass/fail state, flakiness, defects, ' +
|
|
49
|
+
'coverage priority) to decide whether to run autonomous testing, fix a defect, generate new automation, ' +
|
|
50
|
+
"or raise a PR for it — a real decision every time, not a fixed script. See README.md for the full story.");
|
|
51
|
+
program
|
|
52
|
+
.command('run')
|
|
53
|
+
.description('Route one test case. Gathers context, states a plan, executes it adaptively (run_judge/run_defect_fix/' +
|
|
54
|
+
'run_generate, and run_pr_raise if --allow-pr), re-checking real results at every step. Never claims an ' +
|
|
55
|
+
"outcome it did not actually observe via a tool call — see the policy's own non-negotiable rule.")
|
|
56
|
+
.requiredOption('--test-case-uuid <uuid>', 'test case UUID to route')
|
|
57
|
+
.requiredOption('--environment <name>', 'environment name — passed to run_judge/run_generate')
|
|
58
|
+
.requiredOption('--repo-path <path>', 'local repo checkout run_generate/run_pr_raise operate in')
|
|
59
|
+
.option('--allow-pr', 'authorize run_pr_raise — without this flag, that tool is not even offered to the model')
|
|
60
|
+
.option('--policy <path>', 'override the bundled decision policy with your own system prompt file')
|
|
61
|
+
.option('--max-turns <n>', 'override BUDGET_MAX_TURNS for this run')
|
|
62
|
+
.option('--json', 'print a single structured JSON result instead of the human-readable report')
|
|
63
|
+
.option('--ci', 'shorthand for --json')
|
|
64
|
+
.action(async (opts) => {
|
|
65
|
+
const json = (opts.json ?? false) || (opts.ci ?? false);
|
|
66
|
+
const client = createMcpClient({ origin: config.appqOrigin, apiKey: config.appqApiKey() });
|
|
67
|
+
const adapter = buildAdapter();
|
|
68
|
+
const allowPr = opts.allowPr ?? false;
|
|
69
|
+
const policyPath = opts.policy ?? config.policyFile;
|
|
70
|
+
const systemPromptOverride = policyPath ? await readFile(policyPath, 'utf-8') : undefined;
|
|
71
|
+
const budget = { ...config.budget, ...(opts.maxTurns ? { maxTurns: Number(opts.maxTurns) } : {}) };
|
|
72
|
+
if (allowPr) {
|
|
73
|
+
console.error('[setup] run_pr_raise is AUTHORIZED for this invocation.');
|
|
74
|
+
}
|
|
75
|
+
else {
|
|
76
|
+
console.error('[setup] run_pr_raise is not authorized (pass --allow-pr to enable it).');
|
|
77
|
+
}
|
|
78
|
+
const startedAt = Date.now();
|
|
79
|
+
const usage = createUsageAccumulator();
|
|
80
|
+
const baseLog = logEvent('');
|
|
81
|
+
let result;
|
|
82
|
+
try {
|
|
83
|
+
result = await autopilot({
|
|
84
|
+
client,
|
|
85
|
+
adapter,
|
|
86
|
+
testCaseUuid: opts.testCaseUuid,
|
|
87
|
+
environment: opts.environment,
|
|
88
|
+
repoPath: opts.repoPath,
|
|
89
|
+
budget,
|
|
90
|
+
metaTools: {
|
|
91
|
+
autotestCmd: config.autotestCmd,
|
|
92
|
+
scriptgenCmd: config.scriptgenCmd,
|
|
93
|
+
prRaiseCmd: config.prRaiseCmd,
|
|
94
|
+
defectFixCmd: config.defectFixCmd,
|
|
95
|
+
explorerCmd: config.explorerCmd,
|
|
96
|
+
commandTimeoutMs: config.commandTimeoutMs,
|
|
97
|
+
allowPr,
|
|
98
|
+
},
|
|
99
|
+
systemPromptOverride,
|
|
100
|
+
onEvent: (e) => {
|
|
101
|
+
baseLog(e);
|
|
102
|
+
if (e.type === 'usage')
|
|
103
|
+
usage.onUsage(e.detail);
|
|
104
|
+
},
|
|
105
|
+
});
|
|
106
|
+
}
|
|
107
|
+
finally {
|
|
108
|
+
// Audit write happens whether the run succeeded or threw — see
|
|
109
|
+
// @appliqation/agent-core's audit/sink.ts: safeRecord() (used
|
|
110
|
+
// inside recordAutopilotRun) never lets a failed/unreachable audit
|
|
111
|
+
// sink affect this process's real outcome.
|
|
112
|
+
await recordAutopilotRun({
|
|
113
|
+
sink: config.auditSink,
|
|
114
|
+
startedAt,
|
|
115
|
+
endedAt: Date.now(),
|
|
116
|
+
model: resolveModel(),
|
|
117
|
+
usage: usage.totals(),
|
|
118
|
+
testCaseUuid: opts.testCaseUuid,
|
|
119
|
+
environment: opts.environment,
|
|
120
|
+
repoPath: opts.repoPath,
|
|
121
|
+
allowPr,
|
|
122
|
+
result,
|
|
123
|
+
});
|
|
124
|
+
}
|
|
125
|
+
if (json) {
|
|
126
|
+
console.log(JSON.stringify({ report: result.report, turns: result.turns, budgetExceeded: result.budgetExceeded }, null, 2));
|
|
127
|
+
return;
|
|
128
|
+
}
|
|
129
|
+
console.log('\n=== Report ===\n');
|
|
130
|
+
console.log(result.report);
|
|
131
|
+
console.error(`\n(${result.turns} turns, budget exceeded: ${result.budgetExceeded})`);
|
|
132
|
+
});
|
|
133
|
+
program.parseAsync(process.argv);
|
|
134
|
+
//# sourceMappingURL=index.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"index.js","sourceRoot":"","sources":["../../src/cli/index.ts"],"names":[],"mappings":";AACA,uEAAuE;AACvE,yEAAyE;AACzE,wEAAwE;AACxE,2EAA2E;AAC3E,iBAAiB;AAEjB,OAAO,EAAE,QAAQ,EAAE,MAAM,kBAAkB,CAAC;AAC5C,OAAO,EAAE,OAAO,EAAE,MAAM,WAAW,CAAC;AACpC,OAAO,EAAE,eAAe,EAAE,sBAAsB,EAAE,mBAAmB,EAAE,sBAAsB,EAAE,MAAM,yBAAyB,CAAC;AAE/H,OAAO,EAAE,MAAM,EAAE,eAAe,EAAE,YAAY,EAAE,MAAM,kBAAkB,CAAC;AACzE,OAAO,EAAE,SAAS,EAAE,MAAM,8BAA8B,CAAC;AACzD,OAAO,EAAE,kBAAkB,EAAE,MAAM,YAAY,CAAC;AAEhD,SAAS,YAAY;IACnB,MAAM,QAAQ,GAAG,eAAe,EAAE,CAAC;IACnC,MAAM,KAAK,GAAG,YAAY,EAAE,CAAC;IAC7B,OAAO,QAAQ,KAAK,WAAW;QAC7B,CAAC,CAAC,sBAAsB,CAAC,MAAM,CAAC,eAAgB,EAAE,KAAK,EAAE,MAAM,CAAC,kBAAkB,CAAC;QACnF,CAAC,CAAC,mBAAmB,CAAC,MAAM,CAAC,YAAa,EAAE,KAAK,EAAE,MAAM,CAAC,qBAAqB,CAAC,CAAC;AACrF,CAAC;AAED,SAAS,QAAQ,CAAC,MAAc;IAC9B,OAAO,CAAC,CAAqC,EAAE,EAAE;QAC/C,IAAI,CAAC,CAAC,IAAI,KAAK,WAAW,EAAE,CAAC;YAC3B,MAAM,IAAI,GAAG,CAAE,CAAC,CAAC,MAAiB,IAAI,EAAE,CAAC,CAAC,IAAI,EAAE,CAAC;YACjD,IAAI,IAAI;gBAAE,OAAO,CAAC,KAAK,CAAC,GAAG,MAAM,cAAc,IAAI,EAAE,CAAC,CAAC;QACzD,CAAC;aAAM,IAAI,CAAC,CAAC,IAAI,KAAK,MAAM,EAAE,CAAC;YAC7B,MAAM,CAAC,GAAG,CAAC,CAAC,MAA0C,CAAC;YACvD,OAAO,CAAC,KAAK,CAAC,GAAG,MAAM,UAAU,CAAC,CAAC,IAAI,OAAO,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE,CAAC,CAAC;QAC1E,CAAC;aAAM,IAAI,CAAC,CAAC,IAAI,KAAK,KAAK,EAAE,CAAC;YAC5B,OAAO,CAAC,KAAK,CAAC,GAAG,MAAM,SAAS,CAAC,CAAC,MAAM,EAAE,CAAC,CAAC;QAC9C,CAAC;aAAM,IAAI,CAAC,CAAC,IAAI,KAAK,OAAO,EAAE,CAAC;YAC9B,MAAM,CAAC,GAAG,CAAC,CAAC,MAA4G,CAAC;YACzH,MAAM,SAAS,GAAG,CAAC,CAAC,eAAe;gBACjC,CAAC,CAAC,KAAK,CAAC,CAAC,eAAe,cAAc;gBACtC,CAAC,CAAC,CAAC,CAAC,gBAAgB;oBAClB,CAAC,CAAC,KAAK,CAAC,CAAC,gBAAgB,oBAAoB;oBAC7C,CAAC,CAAC,EAAE,CAAC;YACT,OAAO,CAAC,KAAK,CAAC,GAAG,MAAM,cAAc,CAAC,CAAC,WAAW,QAAQ,CAAC,CAAC,YAAY,GAAG,SAAS,EAAE,CAAC,CAAC;QAC1F,CAAC;IACH,CAAC,CAAC;AACJ,CAAC;AAED,MAAM,OAAO,GAAG,IAAI,OAAO,EAAE,CAAC;AAC9B,OAAO;KACJ,IAAI,CAAC,uBAAuB,CAAC;KAC7B,WAAW,CACV,uGAAuG;IACrG,yGAAyG;IACzG,0GAA0G,CAC7G,CAAC;AAEJ,OAAO;KACJ,OAAO,CAAC,KAAK,CAAC;KACd,WAAW,CACV,wGAAwG;IACtG,yGAAyG;IACzG,iGAAiG,CACpG;KACA,cAAc,CAAC,yBAAyB,EAAE,yBAAyB,CAAC;KACpE,cAAc,CAAC,sBAAsB,EAAE,qDAAqD,CAAC;KAC7F,cAAc,CAAC,oBAAoB,EAAE,0DAA0D,CAAC;KAChG,MAAM,CAAC,YAAY,EAAE,wFAAwF,CAAC;KAC9G,MAAM,CAAC,iBAAiB,EAAE,uEAAuE,CAAC;KAClG,MAAM,CAAC,iBAAiB,EAAE,wCAAwC,CAAC;KACnE,MAAM,CAAC,QAAQ,EAAE,4EAA4E,CAAC;KAC9F,MAAM,CAAC,MAAM,EAAE,sBAAsB,CAAC;KACtC,MAAM,CACL,KAAK,EAAE,IASN,EAAE,EAAE;IACH,MAAM,IAAI,GAAG,CAAC,IAAI,CAAC,IAAI,IAAI,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,IAAI,KAAK,CAAC,CAAC;IACxD,MAAM,MAAM,GAAG,eAAe,CAAC,EAAE,MAAM,EAAE,MAAM,CAAC,UAAU,EAAE,MAAM,EAAE,MAAM,CAAC,UAAU,EAAE,EAAE,CAAC,CAAC;IAC3F,MAAM,OAAO,GAAG,YAAY,EAAE,CAAC;IAC/B,MAAM,OAAO,GAAG,IAAI,CAAC,OAAO,IAAI,KAAK,CAAC;IAEtC,MAAM,UAAU,GAAG,IAAI,CAAC,MAAM,IAAI,MAAM,CAAC,UAAU,CAAC;IACpD,MAAM,oBAAoB,GAAG,UAAU,CAAC,CAAC,CAAC,MAAM,QAAQ,CAAC,UAAU,EAAE,OAAO,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC;IAE1F,MAAM,MAAM,GAAG,EAAE,GAAG,MAAM,CAAC,MAAM,EAAE,GAAG,CAAC,IAAI,CAAC,QAAQ,CAAC,CAAC,CAAC,EAAE,QAAQ,EAAE,MAAM,CAAC,IAAI,CAAC,QAAQ,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,CAAC;IAEnG,IAAI,OAAO,EAAE,CAAC;QACZ,OAAO,CAAC,KAAK,CAAC,yDAAyD,CAAC,CAAC;IAC3E,CAAC;SAAM,CAAC;QACN,OAAO,CAAC,KAAK,CAAC,wEAAwE,CAAC,CAAC;IAC1F,CAAC;IAED,MAAM,SAAS,GAAG,IAAI,CAAC,GAAG,EAAE,CAAC;IAC7B,MAAM,KAAK,GAAG,sBAAsB,EAAE,CAAC;IACvC,MAAM,OAAO,GAAG,QAAQ,CAAC,EAAE,CAAC,CAAC;IAC7B,IAAI,MAA8B,CAAC;IACnC,IAAI,CAAC;QACH,MAAM,GAAG,MAAM,SAAS,CAAC;YACvB,MAAM;YACN,OAAO;YACP,YAAY,EAAE,IAAI,CAAC,YAAY;YAC/B,WAAW,EAAE,IAAI,CAAC,WAAW;YAC7B,QAAQ,EAAE,IAAI,CAAC,QAAQ;YACvB,MAAM;YACN,SAAS,EAAE;gBACT,WAAW,EAAE,MAAM,CAAC,WAAW;gBAC/B,YAAY,EAAE,MAAM,CAAC,YAAY;gBACjC,UAAU,EAAE,MAAM,CAAC,UAAU;gBAC7B,YAAY,EAAE,MAAM,CAAC,YAAY;gBACjC,WAAW,EAAE,MAAM,CAAC,WAAW;gBAC/B,gBAAgB,EAAE,MAAM,CAAC,gBAAgB;gBACzC,OAAO;aACR;YACD,oBAAoB;YACpB,OAAO,EAAE,CAAC,CAAC,EAAE,EAAE;gBACb,OAAO,CAAC,CAAC,CAAC,CAAC;gBACX,IAAI,CAAC,CAAC,IAAI,KAAK,OAAO;oBAAE,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,MAA4G,CAAC,CAAC;YACxJ,CAAC;SACF,CAAC,CAAC;IACL,CAAC;YAAS,CAAC;QACT,+DAA+D;QAC/D,8DAA8D;QAC9D,mEAAmE;QACnE,2CAA2C;QAC3C,MAAM,kBAAkB,CAAC;YACvB,IAAI,EAAE,MAAM,CAAC,SAAS;YACtB,SAAS;YACT,OAAO,EAAE,IAAI,CAAC,GAAG,EAAE;YACnB,KAAK,EAAE,YAAY,EAAE;YACrB,KAAK,EAAE,KAAK,CAAC,MAAM,EAAE;YACrB,YAAY,EAAE,IAAI,CAAC,YAAY;YAC/B,WAAW,EAAE,IAAI,CAAC,WAAW;YAC7B,QAAQ,EAAE,IAAI,CAAC,QAAQ;YACvB,OAAO;YACP,MAAM;SACP,CAAC,CAAC;IACL,CAAC;IAED,IAAI,IAAI,EAAE,CAAC;QACT,OAAO,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,EAAE,MAAM,EAAE,MAAM,CAAC,MAAM,EAAE,KAAK,EAAE,MAAM,CAAC,KAAK,EAAE,cAAc,EAAE,MAAM,CAAC,cAAc,EAAE,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,CAAC;QAC5H,OAAO;IACT,CAAC;IACD,OAAO,CAAC,GAAG,CAAC,oBAAoB,CAAC,CAAC;IAClC,OAAO,CAAC,GAAG,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC;IAC3B,OAAO,CAAC,KAAK,CAAC,MAAM,MAAM,CAAC,KAAK,4BAA4B,MAAM,CAAC,cAAc,GAAG,CAAC,CAAC;AACxF,CAAC,CACF,CAAC;AAEJ,OAAO,CAAC,UAAU,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC"}
|
|
@@ -0,0 +1,56 @@
|
|
|
1
|
+
import 'dotenv/config';
|
|
2
|
+
import { DEFAULT_ANTHROPIC_MODEL, DEFAULT_OPENAI_MODEL } from '@appliqation/agent-core/providers';
|
|
3
|
+
import { required, optional } from '@appliqation/agent-core/config';
|
|
4
|
+
import { resolveAuditSink } from '@appliqation/agent-core/audit';
|
|
5
|
+
export const config = {
|
|
6
|
+
appqOrigin: optional('APPQ_ORIGIN') ?? 'https://appq.appliqation.io',
|
|
7
|
+
appqApiKey: () => required('APPQ_API_KEY'),
|
|
8
|
+
anthropicApiKey: optional('ANTHROPIC_API_KEY'),
|
|
9
|
+
openaiApiKey: optional('OPENAI_API_KEY'),
|
|
10
|
+
anthropicModel: optional('ANTHROPIC_MODEL'),
|
|
11
|
+
openaiModel: optional('OPENAI_MODEL'),
|
|
12
|
+
anthropicMaxTokens: Number(optional('ANTHROPIC_MAX_TOKENS') ?? 8192),
|
|
13
|
+
openaiMaxOutputTokens: Number(optional('OPENAI_MAX_OUTPUT_TOKENS') ?? 8192),
|
|
14
|
+
// Generous relative to the other agents in this family — a single
|
|
15
|
+
// autopilot run legitimately needs several rounds: gather context from
|
|
16
|
+
// several tools, reason, act, re-evaluate the real result, possibly act
|
|
17
|
+
// again. Cutting this short would force shallow, single-shot decisions.
|
|
18
|
+
budget: {
|
|
19
|
+
maxCalls: Number(optional('BUDGET_MAX_CALLS') ?? 40),
|
|
20
|
+
maxPages: 999_999, // this agent never drives a browser directly
|
|
21
|
+
maxMillis: Number(optional('BUDGET_MAX_MILLIS') ?? 30 * 60 * 1000),
|
|
22
|
+
maxTurns: Number(optional('BUDGET_MAX_TURNS') ?? 30),
|
|
23
|
+
},
|
|
24
|
+
// How to invoke the five sibling agents — never a filesystem/private-npm
|
|
25
|
+
// dependency (this repo is meant to be cloned standalone), just a command
|
|
26
|
+
// string split on whitespace into [command, ...baseArgs]. Defaults assume
|
|
27
|
+
// the real packages are installed and on PATH; override to point at a
|
|
28
|
+
// local dev build (e.g. "node /path/to/appliqation-autotest/dist/cli/index.js").
|
|
29
|
+
autotestCmd: optional('AUTOTEST_CMD') ?? 'appliqation-autotest',
|
|
30
|
+
scriptgenCmd: optional('SCRIPTGEN_CMD') ?? 'appliqation-scriptgen',
|
|
31
|
+
prRaiseCmd: optional('PR_RAISE_CMD') ?? 'appliqation-pr-raise',
|
|
32
|
+
defectFixCmd: optional('DEFECT_FIX_CMD') ?? 'appliqation-defect-fix',
|
|
33
|
+
explorerCmd: optional('EXPLORER_CMD') ?? 'appliqation-explorer',
|
|
34
|
+
commandTimeoutMs: Number(optional('COMMAND_TIMEOUT_MS') ?? 20 * 60 * 1000),
|
|
35
|
+
// The one real customization point — see src/policy/systemPrompt.ts.
|
|
36
|
+
policyFile: optional('POLICY_FILE'),
|
|
37
|
+
// Observability, entirely opt-in — see @appliqation/agent-core's audit/sink.ts.
|
|
38
|
+
auditSink: resolveAuditSink({
|
|
39
|
+
auditMongoUri: optional('AUDIT_MONGO_URI'),
|
|
40
|
+
auditMongoDb: optional('AUDIT_MONGO_DB'),
|
|
41
|
+
auditMongoCollection: optional('AUDIT_MONGO_COLLECTION'),
|
|
42
|
+
auditJsonlPath: optional('AUDIT_JSONL_PATH'),
|
|
43
|
+
}),
|
|
44
|
+
};
|
|
45
|
+
export function resolveProvider() {
|
|
46
|
+
if (config.anthropicApiKey)
|
|
47
|
+
return 'anthropic';
|
|
48
|
+
if (config.openaiApiKey)
|
|
49
|
+
return 'openai';
|
|
50
|
+
throw new Error('Set ANTHROPIC_API_KEY or OPENAI_API_KEY');
|
|
51
|
+
}
|
|
52
|
+
export function resolveModel() {
|
|
53
|
+
const provider = resolveProvider();
|
|
54
|
+
return provider === 'anthropic' ? (config.anthropicModel ?? DEFAULT_ANTHROPIC_MODEL) : (config.openaiModel ?? DEFAULT_OPENAI_MODEL);
|
|
55
|
+
}
|
|
56
|
+
//# sourceMappingURL=env.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"env.js","sourceRoot":"","sources":["../../src/config/env.ts"],"names":[],"mappings":"AAAA,OAAO,eAAe,CAAC;AACvB,OAAO,EAAE,uBAAuB,EAAE,oBAAoB,EAAE,MAAM,mCAAmC,CAAC;AAClG,OAAO,EAAE,QAAQ,EAAE,QAAQ,EAAE,MAAM,gCAAgC,CAAC;AACpE,OAAO,EAAE,gBAAgB,EAAE,MAAM,+BAA+B,CAAC;AAEjE,MAAM,CAAC,MAAM,MAAM,GAAG;IACpB,UAAU,EAAE,QAAQ,CAAC,aAAa,CAAC,IAAI,6BAA6B;IACpE,UAAU,EAAE,GAAG,EAAE,CAAC,QAAQ,CAAC,cAAc,CAAC;IAC1C,eAAe,EAAE,QAAQ,CAAC,mBAAmB,CAAC;IAC9C,YAAY,EAAE,QAAQ,CAAC,gBAAgB,CAAC;IACxC,cAAc,EAAE,QAAQ,CAAC,iBAAiB,CAAC;IAC3C,WAAW,EAAE,QAAQ,CAAC,cAAc,CAAC;IACrC,kBAAkB,EAAE,MAAM,CAAC,QAAQ,CAAC,sBAAsB,CAAC,IAAI,IAAI,CAAC;IACpE,qBAAqB,EAAE,MAAM,CAAC,QAAQ,CAAC,0BAA0B,CAAC,IAAI,IAAI,CAAC;IAC3E,kEAAkE;IAClE,uEAAuE;IACvE,wEAAwE;IACxE,wEAAwE;IACxE,MAAM,EAAE;QACN,QAAQ,EAAE,MAAM,CAAC,QAAQ,CAAC,kBAAkB,CAAC,IAAI,EAAE,CAAC;QACpD,QAAQ,EAAE,OAAO,EAAE,6CAA6C;QAChE,SAAS,EAAE,MAAM,CAAC,QAAQ,CAAC,mBAAmB,CAAC,IAAI,EAAE,GAAG,EAAE,GAAG,IAAI,CAAC;QAClE,QAAQ,EAAE,MAAM,CAAC,QAAQ,CAAC,kBAAkB,CAAC,IAAI,EAAE,CAAC;KACrD;IACD,yEAAyE;IACzE,0EAA0E;IAC1E,0EAA0E;IAC1E,sEAAsE;IACtE,iFAAiF;IACjF,WAAW,EAAE,QAAQ,CAAC,cAAc,CAAC,IAAI,sBAAsB;IAC/D,YAAY,EAAE,QAAQ,CAAC,eAAe,CAAC,IAAI,uBAAuB;IAClE,UAAU,EAAE,QAAQ,CAAC,cAAc,CAAC,IAAI,sBAAsB;IAC9D,YAAY,EAAE,QAAQ,CAAC,gBAAgB,CAAC,IAAI,wBAAwB;IACpE,WAAW,EAAE,QAAQ,CAAC,cAAc,CAAC,IAAI,sBAAsB;IAC/D,gBAAgB,EAAE,MAAM,CAAC,QAAQ,CAAC,oBAAoB,CAAC,IAAI,EAAE,GAAG,EAAE,GAAG,IAAI,CAAC;IAC1E,qEAAqE;IACrE,UAAU,EAAE,QAAQ,CAAC,aAAa,CAAC;IAEnC,gFAAgF;IAChF,SAAS,EAAE,gBAAgB,CAAC;QAC1B,aAAa,EAAE,QAAQ,CAAC,iBAAiB,CAAC;QAC1C,YAAY,EAAE,QAAQ,CAAC,gBAAgB,CAAC;QACxC,oBAAoB,EAAE,QAAQ,CAAC,wBAAwB,CAAC;QACxD,cAAc,EAAE,QAAQ,CAAC,kBAAkB,CAAC;KAC7C,CAAC;CACH,CAAC;AAEF,MAAM,UAAU,eAAe;IAC7B,IAAI,MAAM,CAAC,eAAe;QAAE,OAAO,WAAW,CAAC;IAC/C,IAAI,MAAM,CAAC,YAAY;QAAE,OAAO,QAAQ,CAAC;IACzC,MAAM,IAAI,KAAK,CAAC,yCAAyC,CAAC,CAAC;AAC7D,CAAC;AAED,MAAM,UAAU,YAAY;IAC1B,MAAM,QAAQ,GAAG,eAAe,EAAE,CAAC;IACnC,OAAO,QAAQ,KAAK,WAAW,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,cAAc,IAAI,uBAAuB,CAAC,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,WAAW,IAAI,oBAAoB,CAAC,CAAC;AACtI,CAAC"}
|
|
@@ -0,0 +1,45 @@
|
|
|
1
|
+
// Ties it together: the read-only appq context tools, the three meta-tools
|
|
2
|
+
// wrapping the sibling agents, and the policy (system prompt) that actually
|
|
3
|
+
// drives judgment — run through @appliqation/agent-core's generic
|
|
4
|
+
// think->act->observe loop directly. No appq prompt fetch at all: the
|
|
5
|
+
// system prompt is local (src/policy/systemPrompt.ts or a --policy
|
|
6
|
+
// override), so runLoop() is used directly rather than runWorkflow()'s
|
|
7
|
+
// appq-fetch indirection, which has nothing to fetch here.
|
|
8
|
+
import { runLoop, fetchAppqToolDefs, createGatedAppqDispatcher, createReadOnlyProjectContextDispatcher, PROJECT_CONTEXT_TOOL } from '@appliqation/agent-core';
|
|
9
|
+
import { READONLY_CONTEXT_TOOLS } from '../tools/safety.js';
|
|
10
|
+
import { metaToolDefs, createMetaToolDispatch } from '../tools/metaTools.js';
|
|
11
|
+
import { buildSystemPrompt } from '../policy/systemPrompt.js';
|
|
12
|
+
export async function autopilot(opts) {
|
|
13
|
+
const contextToolAllowlist = new Set([...READONLY_CONTEXT_TOOLS, PROJECT_CONTEXT_TOOL]);
|
|
14
|
+
const appqToolDefs = await fetchAppqToolDefs(opts.client, contextToolAllowlist);
|
|
15
|
+
// Argument-level gate applied outermost — see @appliqation/agent-core's tools/projectContext.ts —
|
|
16
|
+
// so a write attempt is refused before anything else decides what to do
|
|
17
|
+
// with the call, same ordering reasoning as judgeTc.ts's browser-label
|
|
18
|
+
// correction sitting outside its dry-run interceptor.
|
|
19
|
+
const gatedAppq = createReadOnlyProjectContextDispatcher(createGatedAppqDispatcher(opts.client, contextToolAllowlist));
|
|
20
|
+
const metaDispatch = createMetaToolDispatch(opts.metaTools);
|
|
21
|
+
const metaDefs = metaToolDefs(opts.metaTools);
|
|
22
|
+
const metaNames = new Set(metaDefs.map((t) => t.name));
|
|
23
|
+
const dispatch = async (name, args) => {
|
|
24
|
+
if (metaNames.has(name))
|
|
25
|
+
return metaDispatch(name, args);
|
|
26
|
+
return gatedAppq(name, args);
|
|
27
|
+
};
|
|
28
|
+
const system = opts.systemPromptOverride ?? buildSystemPrompt(opts.metaTools.allowPr);
|
|
29
|
+
const seedMessage = [
|
|
30
|
+
`Test case UUID: ${opts.testCaseUuid}`,
|
|
31
|
+
`Environment: ${opts.environment}`,
|
|
32
|
+
`Repo path (for run_generate/run_pr_raise): ${opts.repoPath}`,
|
|
33
|
+
'Begin now — start with get_scenario.',
|
|
34
|
+
].join('\n');
|
|
35
|
+
return runLoop({
|
|
36
|
+
adapter: opts.adapter,
|
|
37
|
+
system,
|
|
38
|
+
seedMessage,
|
|
39
|
+
tools: [...appqToolDefs, ...metaDefs],
|
|
40
|
+
dispatch,
|
|
41
|
+
budget: opts.budget,
|
|
42
|
+
onEvent: opts.onEvent,
|
|
43
|
+
});
|
|
44
|
+
}
|
|
45
|
+
//# sourceMappingURL=autopilot.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"autopilot.js","sourceRoot":"","sources":["../../src/orchestrator/autopilot.ts"],"names":[],"mappings":"AAAA,2EAA2E;AAC3E,4EAA4E;AAC5E,kEAAkE;AAClE,sEAAsE;AACtE,mEAAmE;AACnE,uEAAuE;AACvE,2DAA2D;AAE3D,OAAO,EAAE,OAAO,EAAE,iBAAiB,EAAE,yBAAyB,EAAE,sCAAsC,EAAE,oBAAoB,EAAE,MAAM,yBAAyB,CAAC;AAE9J,OAAO,EAAE,sBAAsB,EAAE,MAAM,oBAAoB,CAAC;AAC5D,OAAO,EAAE,YAAY,EAAE,sBAAsB,EAAE,MAAM,uBAAuB,CAAC;AAE7E,OAAO,EAAE,iBAAiB,EAAE,MAAM,2BAA2B,CAAC;AAe9D,MAAM,CAAC,KAAK,UAAU,SAAS,CAAC,IAAsB;IACpD,MAAM,oBAAoB,GAAG,IAAI,GAAG,CAAC,CAAC,GAAG,sBAAsB,EAAE,oBAAoB,CAAC,CAAC,CAAC;IACxF,MAAM,YAAY,GAAG,MAAM,iBAAiB,CAAC,IAAI,CAAC,MAAM,EAAE,oBAAoB,CAAC,CAAC;IAChF,kGAAkG;IAClG,wEAAwE;IACxE,uEAAuE;IACvE,sDAAsD;IACtD,MAAM,SAAS,GAAG,sCAAsC,CAAC,yBAAyB,CAAC,IAAI,CAAC,MAAM,EAAE,oBAAoB,CAAC,CAAC,CAAC;IACvH,MAAM,YAAY,GAAG,sBAAsB,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;IAC5D,MAAM,QAAQ,GAAG,YAAY,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;IAC9C,MAAM,SAAS,GAAG,IAAI,GAAG,CAAC,QAAQ,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC;IAEvD,MAAM,QAAQ,GAAmB,KAAK,EAAE,IAAI,EAAE,IAAI,EAAE,EAAE;QACpD,IAAI,SAAS,CAAC,GAAG,CAAC,IAAI,CAAC;YAAE,OAAO,YAAY,CAAC,IAAI,EAAE,IAAI,CAAC,CAAC;QACzD,OAAO,SAAS,CAAC,IAAI,EAAE,IAAI,CAAC,CAAC;IAC/B,CAAC,CAAC;IAEF,MAAM,MAAM,GAAG,IAAI,CAAC,oBAAoB,IAAI,iBAAiB,CAAC,IAAI,CAAC,SAAS,CAAC,OAAO,CAAC,CAAC;IACtF,MAAM,WAAW,GAAG;QAClB,mBAAmB,IAAI,CAAC,YAAY,EAAE;QACtC,gBAAgB,IAAI,CAAC,WAAW,EAAE;QAClC,8CAA8C,IAAI,CAAC,QAAQ,EAAE;QAC7D,sCAAsC;KACvC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IAEb,OAAO,OAAO,CAAC;QACb,OAAO,EAAE,IAAI,CAAC,OAAO;QACrB,MAAM;QACN,WAAW;QACX,KAAK,EAAE,CAAC,GAAG,YAAY,EAAE,GAAG,QAAQ,CAAC;QACrC,QAAQ;QACR,MAAM,EAAE,IAAI,CAAC,MAAM;QACnB,OAAO,EAAE,IAAI,CAAC,OAAO;KACtB,CAAC,CAAC;AACL,CAAC"}
|
|
@@ -0,0 +1,159 @@
|
|
|
1
|
+
// The actual decision-making methodology. This is THE customization point
|
|
2
|
+
// for this agent — see --policy/POLICY_FILE in src/cli/index.ts. Fork this
|
|
3
|
+
// file (or point --policy at your own) to change how autopilot reasons:
|
|
4
|
+
// more conservative, a different priority order, a different report shape.
|
|
5
|
+
// Nothing about the orchestration code around it (src/orchestrator/,
|
|
6
|
+
// src/tools/) needs to change for that — the tool-calling loop is generic;
|
|
7
|
+
// this string is what actually drives its judgment.
|
|
8
|
+
//
|
|
9
|
+
// Deliberately NOT fetched from a server. The individual capabilities this
|
|
10
|
+
// agent orchestrates (autotest, scriptgen, pr-raise) are each self-
|
|
11
|
+
// contained tools that do one thing well; the judgment about how and when
|
|
12
|
+
// to use them belongs here, visible and forkable, not hidden behind a
|
|
13
|
+
// private API only Appliqation can change.
|
|
14
|
+
export function buildSystemPrompt(allowPr) {
|
|
15
|
+
const prToolNote = allowPr
|
|
16
|
+
? '`run_pr_raise` is available for this invocation.'
|
|
17
|
+
: '`run_pr_raise` is NOT available for this invocation — this is expected and normal, not an error. ' +
|
|
18
|
+
'Plan around its absence: recommend raising a PR in your final report instead of attempting it.';
|
|
19
|
+
return `You are an autonomous quality engineering lead deciding where effort is actually warranted for one \
|
|
20
|
+
test case — not a script executing a fixed sequence. Your two responsibilities, in order: (1) figure out, \
|
|
21
|
+
from real evidence, what should happen; (2) make it happen, checking real results at every step rather than \
|
|
22
|
+
assuming your plan survives contact with reality.
|
|
23
|
+
|
|
24
|
+
${prToolNote}
|
|
25
|
+
|
|
26
|
+
**Non-negotiable:** every claim in your final report must cite a real tool result. If you did not actually \
|
|
27
|
+
call run_judge, you have no basis to say the test passes or fails. If you did not actually call run_generate \
|
|
28
|
+
and see testRun.ok in its result, you have no basis to say a script is verified. If you did not actually call \
|
|
29
|
+
run_defect_fix and see verified: true in its result, you have no basis to say a defect is fixed. Never \
|
|
30
|
+
paraphrase a hoped-for outcome as an observed one.
|
|
31
|
+
|
|
32
|
+
## Phase 0 — Prerequisites
|
|
33
|
+
|
|
34
|
+
Check your own tool list for \`run_pr_raise\`. Its absence just means this invocation wasn't authorized to \
|
|
35
|
+
open pull requests — not something to work around or a reason to stop.
|
|
36
|
+
|
|
37
|
+
## Phase 1 — Gather context like a senior QA lead would
|
|
38
|
+
|
|
39
|
+
Call \`get_scenario\` first (scenario intent, sibling test cases, this TC's own steps/expected_results), then \
|
|
40
|
+
pull every signal that would actually change your decision:
|
|
41
|
+
|
|
42
|
+
- \`get_automation_readiness\` — does a canonical script already exist? If so, run_generate would be \
|
|
43
|
+
redundant; consider whether the existing coverage is enough instead.
|
|
44
|
+
- \`get_failure_patterns\` — this TC's own pass_rate/is_flaky, and its siblings'. A flaky TC changes your \
|
|
45
|
+
confidence in any result you get, not just a fact to mention.
|
|
46
|
+
- \`get_defect_context\` for any defect linked to this TC or surfaced above — a known, unresolved root cause \
|
|
47
|
+
changes what a failure *means* (a known bug, not new information) and whether generating a script now would \
|
|
48
|
+
just encode broken behaviour as a passing baseline. This is also the trigger signal for \`run_defect_fix\`: \
|
|
49
|
+
its \`defect_history\`, \`run_context\`, and \`routes_visited\` are exactly what you need to judge both whether \
|
|
50
|
+
the root cause looks fixable and — per Phase 2 below — how much verification the fix will actually need.
|
|
51
|
+
- \`get_coverage_analysis\` / \`get_quality_context\` — is this TC/feature area actually a priority right \
|
|
52
|
+
now, or is effort better spent elsewhere? A routing decision includes deciding effort isn't warranted.
|
|
53
|
+
- \`get_evidence_summary\` / \`get_run_evidence\` / \`get_execution_evidence\` / \`get_test_results\` — has \
|
|
54
|
+
this TC been exercised recently, by a human or agentically, and what did that show?
|
|
55
|
+
- \`enrich_project_context\` (action=read) — the project's own living context document: \
|
|
56
|
+
\`known_issues\`, \`high_risk_areas\`, \`regression_watchlist\`, \`pain_points\`, \`critical_features\`, \
|
|
57
|
+
\`personas\`. This is business/risk context no other tool here carries. Weigh it into Phase 2, don't just \
|
|
58
|
+
fetch it and move on: a TC sitting in a \`high_risk_area\` or matching a \`known_issue\` is a stronger signal \
|
|
59
|
+
for action than the same raw evidence would be in an unremarkable area. This tool also has a write mode — \
|
|
60
|
+
you don't have access to it; only action=read is available to you, enforced below the prompt level, so there's \
|
|
61
|
+
nothing to avoid here beyond calling it the normal way.
|
|
62
|
+
|
|
63
|
+
Don't stop at the first signal that seems to answer the question — weigh several of these against each \
|
|
64
|
+
other, the way a real engineer would before committing effort.
|
|
65
|
+
|
|
66
|
+
## Phase 2 — Form and state a plan
|
|
67
|
+
|
|
68
|
+
Before acting, write out your assessment and the plan it leads to — this is not a formality, it's the actual \
|
|
69
|
+
reasoning a reviewer should be able to audit later. Cover:
|
|
70
|
+
|
|
71
|
+
- **Current state**: is this TC known to currently pass, currently fail, or unknown (no recent evidence of \
|
|
72
|
+
any kind)?
|
|
73
|
+
- **Decision**, and why —
|
|
74
|
+
- No recent evidence exists at all → run_judge first. Never generate a script for a TC whose current \
|
|
75
|
+
behaviour you haven't actually confirmed.
|
|
76
|
+
- Evidence shows the TC currently **fails** → do not generate a script now — that would lock in broken \
|
|
77
|
+
behaviour as a false baseline. If \`get_defect_context\` surfaced an open defect for this TC with a \
|
|
78
|
+
root cause that looks fixable from the evidence you already have (routes_visited, console/network errors, \
|
|
79
|
+
defect_history), \`run_defect_fix\` is warranted. If no such defect exists, or the root cause isn't clear \
|
|
80
|
+
from available evidence, recommend the failure be investigated/fixed first instead of guessing.
|
|
81
|
+
- **Calling \`run_defect_fix\`**: you MUST compose its \`test_instruction\` yourself from what Phase 1 \
|
|
82
|
+
already told you — never pass a vague or generic instruction. \`defect_history\` showing this component has \
|
|
83
|
+
failed before (even if since resolved) signals fragility → instruct a broader, scenario-level re-test, not \
|
|
84
|
+
just the one reproducing TC. An isolated, first-time defect with no such history → the single reproducing TC \
|
|
85
|
+
is genuinely sufficient, say so explicitly. This is the actual point of routing through you instead of \
|
|
86
|
+
appliqation-defect-fix directly: it has no way to know how much verification is warranted on its own — that \
|
|
87
|
+
judgment is yours to make and state.
|
|
88
|
+
- Evidence shows the TC currently **passes** and no canonical script exists → a strong candidate for \
|
|
89
|
+
run_generate, to lock in regression coverage while behaviour is known-good.
|
|
90
|
+
- \`is_flaky\` is true → still worth generating, but say so explicitly and lower your stated confidence — \
|
|
91
|
+
flag it as needing a stabilisation pass, don't present it as equivalent to a stable TC's result.
|
|
92
|
+
- Coverage/quality signals suggest this isn't actually a priority right now → it's entirely valid to \
|
|
93
|
+
conclude no autonomous action is warranted and only report a recommendation. Taking no action is a \
|
|
94
|
+
legitimate decision, not a failure to route.
|
|
95
|
+
- A verified script exists, or run_defect_fix returned verified: true (from this run or already) and \
|
|
96
|
+
run_pr_raise is available → raising the PR is the natural next step; if it's not available, say so and \
|
|
97
|
+
recommend it as a manual follow-up.
|
|
98
|
+
- **\`run_explore\`** is a genuinely separate axis from everything above, weighed alongside whichever \
|
|
99
|
+
branch you land on, not instead of it. The question isn't "does this TC match one of a few known patterns" — \
|
|
100
|
+
it's "does what I've *actually* gathered about this specific TC give me a real, statable reason to suspect \
|
|
101
|
+
the literal expected_result won't be enough." Illustrative, not exhaustive: a defect fix on a page \
|
|
102
|
+
high_risk_areas/defect_history already flags as fragile, or on a TC marked is_flaky, is a reasonable moment \
|
|
103
|
+
to ask whether the surrounding page deserves a broader look before run_pr_raise — but only if that fragility \
|
|
104
|
+
signal is actually present for *this* TC, not by default on every fix. enrich_project_context's known_issues \
|
|
105
|
+
showing a recurring category (e.g. several past localStorage issues) that this TC's feature also touches is \
|
|
106
|
+
another. A new UI/visual component is another — run_judge and run_explore answer different questions and \
|
|
107
|
+
either can be warranted, together or alone. The bar is a specific reason drawn from this TC's own gathered \
|
|
108
|
+
context, stated here like every other routing decision — not routine practice, and not "more testing is \
|
|
109
|
+
always better." Calling it reflexively burns budget without adding signal, which defeats the actual goal \
|
|
110
|
+
(better testing, not more testing); not calling it when the context clearly warrants it misses exactly the \
|
|
111
|
+
kind of gap this exists to catch. Absence of a stated reason means don't call it, the same as every other \
|
|
112
|
+
tool in this policy. Compose \`prompt\` yourself from that stated reason — a generic "explore this page" \
|
|
113
|
+
instruction wastes the specificity you just reasoned your way to, the same discipline run_defect_fix's \
|
|
114
|
+
test_instruction already requires of you.
|
|
115
|
+
- **Planned sequence**: the specific tool calls you intend to make, in order — understanding you will \
|
|
116
|
+
re-evaluate after each real result in Phase 3, not execute this blindly.
|
|
117
|
+
|
|
118
|
+
## Phase 3 — Execute adaptively
|
|
119
|
+
|
|
120
|
+
Carry out the plan, but treat every step's result as new information, not a checkbox:
|
|
121
|
+
|
|
122
|
+
- After run_judge: if its real, polled status is failed/blocked, stop and do not proceed to run_generate — \
|
|
123
|
+
return to Phase 2's reasoning with this new evidence (usually: report the failure, don't automate it, or \
|
|
124
|
+
route to run_defect_fix if a fixable root cause exists).
|
|
125
|
+
- After run_defect_fix: only treat it as a success if its result says \`verified: true\` — the same \
|
|
126
|
+
non-negotiable discipline as run_generate's \`testRun.ok\`, derived from a real, independently-executed \
|
|
127
|
+
Playwright run against the testing scope you yourself specified, never the model's own claim from inside \
|
|
128
|
+
that run. Only proceed to run_pr_raise once you see \`verified: true\`; \`verified: false\` means stop and \
|
|
129
|
+
report what's still broken, not attempt a second guess at a fix in the same pass.
|
|
130
|
+
- After run_generate: only treat it as a success if its result says \`testRun.ok: true\` — that field is \
|
|
131
|
+
derived from actually executing the generated script, never from the model's own claim inside that run. \
|
|
132
|
+
\`testRun.ran: true, testRun.ok: false\` means it ran and failed for real — do not proceed to run_pr_raise \
|
|
133
|
+
with an unverified or failing script.
|
|
134
|
+
- After run_pr_raise (if called): confirm it actually returned a PR URL before reporting one — \
|
|
135
|
+
\`committed: false\` means there was nothing to raise a PR for at all.
|
|
136
|
+
- After run_explore (if called): its result is a report to read and weigh, not a pass/fail verdict — \
|
|
137
|
+
\`budgetExceeded: true\` means the pass ended early and the report may be incomplete, say so if it happened. \
|
|
138
|
+
A finding it surfaces can change your Phase 2 plan (e.g. it turns up a real issue that changes whether a fix \
|
|
139
|
+
or a PR is still warranted as originally planned) — treat it as new evidence like any other real result, not \
|
|
140
|
+
a side quest that doesn't feed back into the rest of your reasoning.
|
|
141
|
+
- If reality diverges from your Phase 2 plan at any point, adapt and say so — a plan that survives \
|
|
142
|
+
unchanged despite contradicting evidence is a red flag, not diligence.
|
|
143
|
+
|
|
144
|
+
## Phase 4 — Final report
|
|
145
|
+
|
|
146
|
+
Structure it plainly:
|
|
147
|
+
|
|
148
|
+
- **Plan & reasoning** — what you decided and why, from Phase 2.
|
|
149
|
+
- **Evidence gathered** — the signals from Phase 1 that actually drove the decision (cite specifics: pass \
|
|
150
|
+
rates, defect IDs, coverage gaps — not vague summaries).
|
|
151
|
+
- **Actions taken** — each tool you actually called and its real result (verdict/status from run_judge, \
|
|
152
|
+
verified and the testing scope you specified from run_defect_fix, testRun.ok and the written file path from \
|
|
153
|
+
run_generate, PR URL from run_pr_raise, findings and budgetExceeded from run_explore — plus the reason you \
|
|
154
|
+
called it, from Phase 2). If you took no action, say that plainly and why.
|
|
155
|
+
- **Authorization notes** — if run_pr_raise wasn't available and a PR would otherwise have been warranted, \
|
|
156
|
+
say so explicitly as a recommendation, not a silent gap.
|
|
157
|
+
- **Recommendation** — what, if anything, a human should do next.`;
|
|
158
|
+
}
|
|
159
|
+
//# sourceMappingURL=systemPrompt.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"systemPrompt.js","sourceRoot":"","sources":["../../src/policy/systemPrompt.ts"],"names":[],"mappings":"AAAA,0EAA0E;AAC1E,2EAA2E;AAC3E,wEAAwE;AACxE,2EAA2E;AAC3E,qEAAqE;AACrE,2EAA2E;AAC3E,oDAAoD;AACpD,EAAE;AACF,2EAA2E;AAC3E,oEAAoE;AACpE,0EAA0E;AAC1E,sEAAsE;AACtE,2CAA2C;AAE3C,MAAM,UAAU,iBAAiB,CAAC,OAAgB;IAChD,MAAM,UAAU,GAAG,OAAO;QACxB,CAAC,CAAC,kDAAkD;QACpD,CAAC,CAAC,mGAAmG;YACnG,gGAAgG,CAAC;IAErG,OAAO;;;;;EAKP,UAAU;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;kEAqIsD,CAAC;AACnE,CAAC"}
|
|
@@ -0,0 +1,220 @@
|
|
|
1
|
+
// Wraps the five sibling agents (appliqation-autotest, appliqation-scriptgen,
|
|
2
|
+
// appliqation-defect-fix, appliqation-pr-raise, appliqation-explorer) as ordinary
|
|
3
|
+
// LLM-callable tools. Never a filesystem or
|
|
4
|
+
// private-npm dependency — each is invoked as a configured command string
|
|
5
|
+
// (see src/config/env.ts), spawned via child_process.execFile with an
|
|
6
|
+
// explicit argv array (never a shell string), consuming the CLI's own
|
|
7
|
+
// --json output. The model sees the REAL structured result every time —
|
|
8
|
+
// verified/testRun.ok/a real PR URL — never a paraphrase of one.
|
|
9
|
+
import { execFile } from 'node:child_process';
|
|
10
|
+
// Same hand-rolled wrapper used by appliqation-scriptgen/appliqation-pr-raise,
|
|
11
|
+
// for the same reason: util.promisify(execFile) only resolves {stdout,
|
|
12
|
+
// stderr} via an internal Node symbol a mocked module in tests won't carry.
|
|
13
|
+
function execFileAsync(command, args, options) {
|
|
14
|
+
return new Promise((resolvePromise, rejectPromise) => {
|
|
15
|
+
execFile(command, args, options, (error, stdout, stderr) => {
|
|
16
|
+
if (error) {
|
|
17
|
+
const failure = error;
|
|
18
|
+
failure.stdout = String(stdout ?? '');
|
|
19
|
+
failure.stderr = String(stderr ?? '');
|
|
20
|
+
rejectPromise(failure);
|
|
21
|
+
}
|
|
22
|
+
else {
|
|
23
|
+
resolvePromise({ stdout: String(stdout ?? ''), stderr: String(stderr ?? '') });
|
|
24
|
+
}
|
|
25
|
+
});
|
|
26
|
+
});
|
|
27
|
+
}
|
|
28
|
+
/** Splits a configured command string ("node /path/to/cli.js") into [command, ...baseArgs]. */
|
|
29
|
+
export function parseCommand(cmd) {
|
|
30
|
+
const parts = cmd.trim().split(/\s+/).filter(Boolean);
|
|
31
|
+
if (parts.length === 0)
|
|
32
|
+
throw new Error(`Empty command string: "${cmd}"`);
|
|
33
|
+
return [parts[0], parts.slice(1)];
|
|
34
|
+
}
|
|
35
|
+
export function metaToolDefs(cfg) {
|
|
36
|
+
const defs = [
|
|
37
|
+
{
|
|
38
|
+
name: 'run_judge',
|
|
39
|
+
description: 'Run autonomous testing (a real executor + validator pass) for one test case against a live ' +
|
|
40
|
+
"environment. Returns the real, appq-polled outcome. This is the ONLY way to know a test case's " +
|
|
41
|
+
'current pass/fail state — never assume it from prior context alone.',
|
|
42
|
+
inputSchema: {
|
|
43
|
+
type: 'object',
|
|
44
|
+
properties: {
|
|
45
|
+
test_case_uuid: { type: 'string' },
|
|
46
|
+
environment: { type: 'string' },
|
|
47
|
+
dry_run: {
|
|
48
|
+
type: 'boolean',
|
|
49
|
+
description: 'Suppress writeback to Appliqation — use when you only need to observe current behaviour, not record a verdict.',
|
|
50
|
+
},
|
|
51
|
+
},
|
|
52
|
+
required: ['test_case_uuid', 'environment'],
|
|
53
|
+
},
|
|
54
|
+
},
|
|
55
|
+
{
|
|
56
|
+
name: 'run_generate',
|
|
57
|
+
description: 'Draft and REALLY verify a Playwright script for one test case — it actually runs the generated ' +
|
|
58
|
+
"script; the result's testRun.ok reflects a real, independently-checked outcome, never the model's " +
|
|
59
|
+
'own claim from inside that run. Only call this once you know (via run_judge or existing evidence) ' +
|
|
60
|
+
'that the test case currently passes — generating a script for a currently-failing test case would ' +
|
|
61
|
+
'encode broken behaviour as a false baseline.',
|
|
62
|
+
inputSchema: {
|
|
63
|
+
type: 'object',
|
|
64
|
+
properties: {
|
|
65
|
+
test_case_uuid: { type: 'string' },
|
|
66
|
+
environment: { type: 'string', description: 'Optional — offered to the generator as base-URL context.' },
|
|
67
|
+
repo_path: { type: 'string' },
|
|
68
|
+
},
|
|
69
|
+
required: ['test_case_uuid', 'repo_path'],
|
|
70
|
+
},
|
|
71
|
+
},
|
|
72
|
+
{
|
|
73
|
+
name: 'run_defect_fix',
|
|
74
|
+
description: 'Fix a defect: loads full defect context, locates and applies a real code fix, syncs the Appliqation ' +
|
|
75
|
+
'scenario, and verifies the fix by actually running Playwright — the result\'s verified field reflects ' +
|
|
76
|
+
'a real, independently-checked outcome, never the model\'s own claim from inside that run. ' +
|
|
77
|
+
'test_instruction is REQUIRED — you must state, from your own gathered evidence (defect_history, ' +
|
|
78
|
+
'run_context, is_flaky), what testing scope this fix actually needs verified beyond just the single ' +
|
|
79
|
+
'reproducing test case (e.g. "also re-run the whole scenario — this component has a history of ' +
|
|
80
|
+
'regressions" or "the single reproducing test case is sufficient — this is an isolated, one-off defect"). ' +
|
|
81
|
+
'Never call this with a vague or empty instruction.',
|
|
82
|
+
inputSchema: {
|
|
83
|
+
type: 'object',
|
|
84
|
+
properties: {
|
|
85
|
+
defect_id: { type: 'string' },
|
|
86
|
+
repo_path: { type: 'string' },
|
|
87
|
+
test_instruction: {
|
|
88
|
+
type: 'string',
|
|
89
|
+
description: 'Required — your own assessment of the testing scope this fix needs, from Phase 1 evidence.',
|
|
90
|
+
},
|
|
91
|
+
dry_run: {
|
|
92
|
+
type: 'boolean',
|
|
93
|
+
description: 'Suppress the Appliqation scenario/run writeback — use when you only need to observe whether a fix is achievable, not commit one.',
|
|
94
|
+
},
|
|
95
|
+
},
|
|
96
|
+
required: ['defect_id', 'repo_path', 'test_instruction'],
|
|
97
|
+
},
|
|
98
|
+
},
|
|
99
|
+
{
|
|
100
|
+
name: 'run_explore',
|
|
101
|
+
description: 'Run a headless exploratory-QA pass (appq:runman) against a live target — open-ended senior-QA ' +
|
|
102
|
+
'heuristics, accessibility, security/network/caching probes, the kind of coverage a scripted test case ' +
|
|
103
|
+
'never checks for. Not a routine step: call it only when your own gathered context gives you a real, ' +
|
|
104
|
+
'statable reason to suspect the literal test case in front of you is not enough on its own (see the ' +
|
|
105
|
+
'policy\'s Phase 2 for how to judge that). prompt should state the exploration intent in plain English ' +
|
|
106
|
+
'— what you actually want covered, not a generic instruction.',
|
|
107
|
+
inputSchema: {
|
|
108
|
+
type: 'object',
|
|
109
|
+
properties: {
|
|
110
|
+
prompt: { type: 'string', description: 'Plain-English exploration intent — what to cover and why, from your own gathered context.' },
|
|
111
|
+
project_id: { type: 'integer' },
|
|
112
|
+
site_url: { type: 'string' },
|
|
113
|
+
},
|
|
114
|
+
required: ['prompt'],
|
|
115
|
+
},
|
|
116
|
+
},
|
|
117
|
+
];
|
|
118
|
+
// Hardcoded exclusion, not a soft warning — matches every other
|
|
119
|
+
// non-negotiable safety boundary in this agent family. If --allow-pr
|
|
120
|
+
// wasn't passed to this CLI invocation, this tool simply isn't in the
|
|
121
|
+
// list the model ever sees; there's no way for it to attempt this.
|
|
122
|
+
if (cfg.allowPr) {
|
|
123
|
+
defs.push({
|
|
124
|
+
name: 'run_pr_raise',
|
|
125
|
+
description: "Commit whatever run_generate already wrote in repo_path, push, and open (or reuse) a pull " +
|
|
126
|
+
'request. Only call this after run_generate reported testRun.ok: true for the same repo_path — ' +
|
|
127
|
+
'never raise a PR for a script you have not seen independently verified.',
|
|
128
|
+
inputSchema: {
|
|
129
|
+
type: 'object',
|
|
130
|
+
properties: {
|
|
131
|
+
project_id: { type: 'integer' },
|
|
132
|
+
repo_path: { type: 'string' },
|
|
133
|
+
branch_name: { type: 'string' },
|
|
134
|
+
pr_title: { type: 'string' },
|
|
135
|
+
pr_body: { type: 'string' },
|
|
136
|
+
},
|
|
137
|
+
required: ['project_id', 'repo_path', 'branch_name', 'pr_title'],
|
|
138
|
+
},
|
|
139
|
+
});
|
|
140
|
+
}
|
|
141
|
+
return defs;
|
|
142
|
+
}
|
|
143
|
+
async function runCliJson(baseCmd, subcommand, args, timeoutMs) {
|
|
144
|
+
const [command, baseArgs] = parseCommand(baseCmd);
|
|
145
|
+
const fullArgs = [...baseArgs, subcommand, ...args, '--json'];
|
|
146
|
+
try {
|
|
147
|
+
const { stdout } = await execFileAsync(command, fullArgs, { timeout: timeoutMs, maxBuffer: 20 * 1024 * 1024 });
|
|
148
|
+
return { ok: true, text: stdout.trim() || '{}' };
|
|
149
|
+
}
|
|
150
|
+
catch (err) {
|
|
151
|
+
const e = err;
|
|
152
|
+
// Every sibling CLI prints its --json summary to stdout BEFORE setting
|
|
153
|
+
// a non-zero exit code on a real failure (judge failed/blocked,
|
|
154
|
+
// generate unverified) — recover it; it's real signal, not noise.
|
|
155
|
+
const out = (e.stdout ?? '').trim();
|
|
156
|
+
if (out)
|
|
157
|
+
return { ok: false, text: out };
|
|
158
|
+
return { ok: false, text: `${baseCmd} ${subcommand} failed: ${e.stderr || e.message}` };
|
|
159
|
+
}
|
|
160
|
+
}
|
|
161
|
+
export function createMetaToolDispatch(cfg) {
|
|
162
|
+
return async (name, args) => {
|
|
163
|
+
switch (name) {
|
|
164
|
+
case 'run_judge': {
|
|
165
|
+
const cliArgs = ['--test-case-uuid', String(args.test_case_uuid), '--environment', String(args.environment)];
|
|
166
|
+
if (args.dry_run)
|
|
167
|
+
cliArgs.push('--dry-run');
|
|
168
|
+
return runCliJson(cfg.autotestCmd, 'judge', cliArgs, cfg.commandTimeoutMs);
|
|
169
|
+
}
|
|
170
|
+
case 'run_generate': {
|
|
171
|
+
const cliArgs = ['--test-case-uuid', String(args.test_case_uuid), '--repo-path', String(args.repo_path)];
|
|
172
|
+
if (args.environment)
|
|
173
|
+
cliArgs.push('--environment', String(args.environment));
|
|
174
|
+
return runCliJson(cfg.scriptgenCmd, 'generate', cliArgs, cfg.commandTimeoutMs);
|
|
175
|
+
}
|
|
176
|
+
case 'run_defect_fix': {
|
|
177
|
+
const cliArgs = [
|
|
178
|
+
'--defect-id',
|
|
179
|
+
String(args.defect_id),
|
|
180
|
+
'--repo-path',
|
|
181
|
+
String(args.repo_path),
|
|
182
|
+
'--test-instruction',
|
|
183
|
+
String(args.test_instruction),
|
|
184
|
+
];
|
|
185
|
+
if (args.dry_run)
|
|
186
|
+
cliArgs.push('--dry-run');
|
|
187
|
+
return runCliJson(cfg.defectFixCmd, 'fix', cliArgs, cfg.commandTimeoutMs);
|
|
188
|
+
}
|
|
189
|
+
case 'run_explore': {
|
|
190
|
+
const cliArgs = ['--prompt', String(args.prompt)];
|
|
191
|
+
if (args.project_id !== undefined)
|
|
192
|
+
cliArgs.push('--project-id', String(args.project_id));
|
|
193
|
+
if (args.site_url)
|
|
194
|
+
cliArgs.push('--site-url', String(args.site_url));
|
|
195
|
+
return runCliJson(cfg.explorerCmd, 'explore', cliArgs, cfg.commandTimeoutMs);
|
|
196
|
+
}
|
|
197
|
+
case 'run_pr_raise': {
|
|
198
|
+
if (!cfg.allowPr) {
|
|
199
|
+
return { ok: false, text: 'run_pr_raise is not authorized for this invocation (--allow-pr was not set on appliqation-autopilot).' };
|
|
200
|
+
}
|
|
201
|
+
const cliArgs = [
|
|
202
|
+
'--project-id',
|
|
203
|
+
String(args.project_id),
|
|
204
|
+
'--repo-path',
|
|
205
|
+
String(args.repo_path),
|
|
206
|
+
'--branch-name',
|
|
207
|
+
String(args.branch_name),
|
|
208
|
+
'--pr-title',
|
|
209
|
+
String(args.pr_title),
|
|
210
|
+
];
|
|
211
|
+
if (args.pr_body)
|
|
212
|
+
cliArgs.push('--pr-body', String(args.pr_body));
|
|
213
|
+
return runCliJson(cfg.prRaiseCmd, 'raise', cliArgs, cfg.commandTimeoutMs);
|
|
214
|
+
}
|
|
215
|
+
default:
|
|
216
|
+
return { ok: false, text: `Unknown meta tool "${name}"` };
|
|
217
|
+
}
|
|
218
|
+
};
|
|
219
|
+
}
|
|
220
|
+
//# sourceMappingURL=metaTools.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"metaTools.js","sourceRoot":"","sources":["../../src/tools/metaTools.ts"],"names":[],"mappings":"AAAA,8EAA8E;AAC9E,kFAAkF;AAClF,4CAA4C;AAC5C,0EAA0E;AAC1E,sEAAsE;AACtE,sEAAsE;AACtE,wEAAwE;AACxE,iEAAiE;AAEjE,OAAO,EAAE,QAAQ,EAAE,MAAM,oBAAoB,CAAC;AAc9C,+EAA+E;AAC/E,uEAAuE;AACvE,4EAA4E;AAC5E,SAAS,aAAa,CACpB,OAAe,EACf,IAAc,EACd,OAA+C;IAE/C,OAAO,IAAI,OAAO,CAAC,CAAC,cAAc,EAAE,aAAa,EAAE,EAAE;QACnD,QAAQ,CAAC,OAAO,EAAE,IAAI,EAAE,OAAO,EAAE,CAAC,KAAK,EAAE,MAAM,EAAE,MAAM,EAAE,EAAE;YACzD,IAAI,KAAK,EAAE,CAAC;gBACV,MAAM,OAAO,GAAG,KAAoB,CAAC;gBACrC,OAAO,CAAC,MAAM,GAAG,MAAM,CAAC,MAAM,IAAI,EAAE,CAAC,CAAC;gBACtC,OAAO,CAAC,MAAM,GAAG,MAAM,CAAC,MAAM,IAAI,EAAE,CAAC,CAAC;gBACtC,aAAa,CAAC,OAAO,CAAC,CAAC;YACzB,CAAC;iBAAM,CAAC;gBACN,cAAc,CAAC,EAAE,MAAM,EAAE,MAAM,CAAC,MAAM,IAAI,EAAE,CAAC,EAAE,MAAM,EAAE,MAAM,CAAC,MAAM,IAAI,EAAE,CAAC,EAAE,CAAC,CAAC;YACjF,CAAC;QACH,CAAC,CAAC,CAAC;IACL,CAAC,CAAC,CAAC;AACL,CAAC;AAED,+FAA+F;AAC/F,MAAM,UAAU,YAAY,CAAC,GAAW;IACtC,MAAM,KAAK,GAAG,GAAG,CAAC,IAAI,EAAE,CAAC,KAAK,CAAC,KAAK,CAAC,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC;IACtD,IAAI,KAAK,CAAC,MAAM,KAAK,CAAC;QAAE,MAAM,IAAI,KAAK,CAAC,0BAA0B,GAAG,GAAG,CAAC,CAAC;IAC1E,OAAO,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,KAAK,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC;AACpC,CAAC;AAaD,MAAM,UAAU,YAAY,CAAC,GAAoB;IAC/C,MAAM,IAAI,GAAiB;QACzB;YACE,IAAI,EAAE,WAAW;YACjB,WAAW,EACT,6FAA6F;gBAC7F,iGAAiG;gBACjG,qEAAqE;YACvE,WAAW,EAAE;gBACX,IAAI,EAAE,QAAQ;gBACd,UAAU,EAAE;oBACV,cAAc,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;oBAClC,WAAW,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;oBAC/B,OAAO,EAAE;wBACP,IAAI,EAAE,SAAS;wBACf,WAAW,EAAE,gHAAgH;qBAC9H;iBACF;gBACD,QAAQ,EAAE,CAAC,gBAAgB,EAAE,aAAa,CAAC;aAC5C;SACF;QACD;YACE,IAAI,EAAE,cAAc;YACpB,WAAW,EACT,iGAAiG;gBACjG,oGAAoG;gBACpG,oGAAoG;gBACpG,oGAAoG;gBACpG,8CAA8C;YAChD,WAAW,EAAE;gBACX,IAAI,EAAE,QAAQ;gBACd,UAAU,EAAE;oBACV,cAAc,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;oBAClC,WAAW,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE,WAAW,EAAE,0DAA0D,EAAE;oBACxG,SAAS,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;iBAC9B;gBACD,QAAQ,EAAE,CAAC,gBAAgB,EAAE,WAAW,CAAC;aAC1C;SACF;QACD;YACE,IAAI,EAAE,gBAAgB;YACtB,WAAW,EACT,sGAAsG;gBACtG,wGAAwG;gBACxG,4FAA4F;gBAC5F,kGAAkG;gBAClG,qGAAqG;gBACrG,gGAAgG;gBAChG,2GAA2G;gBAC3G,oDAAoD;YACtD,WAAW,EAAE;gBACX,IAAI,EAAE,QAAQ;gBACd,UAAU,EAAE;oBACV,SAAS,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;oBAC7B,SAAS,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;oBAC7B,gBAAgB,EAAE;wBAChB,IAAI,EAAE,QAAQ;wBACd,WAAW,EAAE,4FAA4F;qBAC1G;oBACD,OAAO,EAAE;wBACP,IAAI,EAAE,SAAS;wBACf,WAAW,EAAE,kIAAkI;qBAChJ;iBACF;gBACD,QAAQ,EAAE,CAAC,WAAW,EAAE,WAAW,EAAE,kBAAkB,CAAC;aACzD;SACF;QACD;YACE,IAAI,EAAE,aAAa;YACnB,WAAW,EACT,gGAAgG;gBAChG,wGAAwG;gBACxG,sGAAsG;gBACtG,qGAAqG;gBACrG,wGAAwG;gBACxG,8DAA8D;YAChE,WAAW,EAAE;gBACX,IAAI,EAAE,QAAQ;gBACd,UAAU,EAAE;oBACV,MAAM,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE,WAAW,EAAE,2FAA2F,EAAE;oBACpI,UAAU,EAAE,EAAE,IAAI,EAAE,SAAS,EAAE;oBAC/B,QAAQ,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;iBAC7B;gBACD,QAAQ,EAAE,CAAC,QAAQ,CAAC;aACrB;SACF;KACF,CAAC;IAEF,gEAAgE;IAChE,qEAAqE;IACrE,sEAAsE;IACtE,mEAAmE;IACnE,IAAI,GAAG,CAAC,OAAO,EAAE,CAAC;QAChB,IAAI,CAAC,IAAI,CAAC;YACR,IAAI,EAAE,cAAc;YACpB,WAAW,EACT,4FAA4F;gBAC5F,gGAAgG;gBAChG,yEAAyE;YAC3E,WAAW,EAAE;gBACX,IAAI,EAAE,QAAQ;gBACd,UAAU,EAAE;oBACV,UAAU,EAAE,EAAE,IAAI,EAAE,SAAS,EAAE;oBAC/B,SAAS,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;oBAC7B,WAAW,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;oBAC/B,QAAQ,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;oBAC5B,OAAO,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;iBAC5B;gBACD,QAAQ,EAAE,CAAC,YAAY,EAAE,WAAW,EAAE,aAAa,EAAE,UAAU,CAAC;aACjE;SACF,CAAC,CAAC;IACL,CAAC;IAED,OAAO,IAAI,CAAC;AACd,CAAC;AAED,KAAK,UAAU,UAAU,CAAC,OAAe,EAAE,UAAkB,EAAE,IAAc,EAAE,SAAiB;IAC9F,MAAM,CAAC,OAAO,EAAE,QAAQ,CAAC,GAAG,YAAY,CAAC,OAAO,CAAC,CAAC;IAClD,MAAM,QAAQ,GAAG,CAAC,GAAG,QAAQ,EAAE,UAAU,EAAE,GAAG,IAAI,EAAE,QAAQ,CAAC,CAAC;IAC9D,IAAI,CAAC;QACH,MAAM,EAAE,MAAM,EAAE,GAAG,MAAM,aAAa,CAAC,OAAO,EAAE,QAAQ,EAAE,EAAE,OAAO,EAAE,SAAS,EAAE,SAAS,EAAE,EAAE,GAAG,IAAI,GAAG,IAAI,EAAE,CAAC,CAAC;QAC/G,OAAO,EAAE,EAAE,EAAE,IAAI,EAAE,IAAI,EAAE,MAAM,CAAC,IAAI,EAAE,IAAI,IAAI,EAAE,CAAC;IACnD,CAAC;IAAC,OAAO,GAAG,EAAE,CAAC;QACb,MAAM,CAAC,GAAG,GAAkB,CAAC;QAC7B,uEAAuE;QACvE,gEAAgE;QAChE,kEAAkE;QAClE,MAAM,GAAG,GAAG,CAAC,CAAC,CAAC,MAAM,IAAI,EAAE,CAAC,CAAC,IAAI,EAAE,CAAC;QACpC,IAAI,GAAG;YAAE,OAAO,EAAE,EAAE,EAAE,KAAK,EAAE,IAAI,EAAE,GAAG,EAAE,CAAC;QACzC,OAAO,EAAE,EAAE,EAAE,KAAK,EAAE,IAAI,EAAE,GAAG,OAAO,IAAI,UAAU,YAAY,CAAC,CAAC,MAAM,IAAI,CAAC,CAAC,OAAO,EAAE,EAAE,CAAC;IAC1F,CAAC;AACH,CAAC;AAED,MAAM,UAAU,sBAAsB,CAAC,GAAoB;IACzD,OAAO,KAAK,EAAE,IAAY,EAAE,IAA6B,EAAuB,EAAE;QAChF,QAAQ,IAAI,EAAE,CAAC;YACb,KAAK,WAAW,CAAC,CAAC,CAAC;gBACjB,MAAM,OAAO,GAAG,CAAC,kBAAkB,EAAE,MAAM,CAAC,IAAI,CAAC,cAAc,CAAC,EAAE,eAAe,EAAE,MAAM,CAAC,IAAI,CAAC,WAAW,CAAC,CAAC,CAAC;gBAC7G,IAAI,IAAI,CAAC,OAAO;oBAAE,OAAO,CAAC,IAAI,CAAC,WAAW,CAAC,CAAC;gBAC5C,OAAO,UAAU,CAAC,GAAG,CAAC,WAAW,EAAE,OAAO,EAAE,OAAO,EAAE,GAAG,CAAC,gBAAgB,CAAC,CAAC;YAC7E,CAAC;YACD,KAAK,cAAc,CAAC,CAAC,CAAC;gBACpB,MAAM,OAAO,GAAG,CAAC,kBAAkB,EAAE,MAAM,CAAC,IAAI,CAAC,cAAc,CAAC,EAAE,aAAa,EAAE,MAAM,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC;gBACzG,IAAI,IAAI,CAAC,WAAW;oBAAE,OAAO,CAAC,IAAI,CAAC,eAAe,EAAE,MAAM,CAAC,IAAI,CAAC,WAAW,CAAC,CAAC,CAAC;gBAC9E,OAAO,UAAU,CAAC,GAAG,CAAC,YAAY,EAAE,UAAU,EAAE,OAAO,EAAE,GAAG,CAAC,gBAAgB,CAAC,CAAC;YACjF,CAAC;YACD,KAAK,gBAAgB,CAAC,CAAC,CAAC;gBACtB,MAAM,OAAO,GAAG;oBACd,aAAa;oBACb,MAAM,CAAC,IAAI,CAAC,SAAS,CAAC;oBACtB,aAAa;oBACb,MAAM,CAAC,IAAI,CAAC,SAAS,CAAC;oBACtB,oBAAoB;oBACpB,MAAM,CAAC,IAAI,CAAC,gBAAgB,CAAC;iBAC9B,CAAC;gBACF,IAAI,IAAI,CAAC,OAAO;oBAAE,OAAO,CAAC,IAAI,CAAC,WAAW,CAAC,CAAC;gBAC5C,OAAO,UAAU,CAAC,GAAG,CAAC,YAAY,EAAE,KAAK,EAAE,OAAO,EAAE,GAAG,CAAC,gBAAgB,CAAC,CAAC;YAC5E,CAAC;YACD,KAAK,aAAa,CAAC,CAAC,CAAC;gBACnB,MAAM,OAAO,GAAG,CAAC,UAAU,EAAE,MAAM,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC,CAAC;gBAClD,IAAI,IAAI,CAAC,UAAU,KAAK,SAAS;oBAAE,OAAO,CAAC,IAAI,CAAC,cAAc,EAAE,MAAM,CAAC,IAAI,CAAC,UAAU,CAAC,CAAC,CAAC;gBACzF,IAAI,IAAI,CAAC,QAAQ;oBAAE,OAAO,CAAC,IAAI,CAAC,YAAY,EAAE,MAAM,CAAC,IAAI,CAAC,QAAQ,CAAC,CAAC,CAAC;gBACrE,OAAO,UAAU,CAAC,GAAG,CAAC,WAAW,EAAE,SAAS,EAAE,OAAO,EAAE,GAAG,CAAC,gBAAgB,CAAC,CAAC;YAC/E,CAAC;YACD,KAAK,cAAc,CAAC,CAAC,CAAC;gBACpB,IAAI,CAAC,GAAG,CAAC,OAAO,EAAE,CAAC;oBACjB,OAAO,EAAE,EAAE,EAAE,KAAK,EAAE,IAAI,EAAE,uGAAuG,EAAE,CAAC;gBACtI,CAAC;gBACD,MAAM,OAAO,GAAG;oBACd,cAAc;oBACd,MAAM,CAAC,IAAI,CAAC,UAAU,CAAC;oBACvB,aAAa;oBACb,MAAM,CAAC,IAAI,CAAC,SAAS,CAAC;oBACtB,eAAe;oBACf,MAAM,CAAC,IAAI,CAAC,WAAW,CAAC;oBACxB,YAAY;oBACZ,MAAM,CAAC,IAAI,CAAC,QAAQ,CAAC;iBACtB,CAAC;gBACF,IAAI,IAAI,CAAC,OAAO;oBAAE,OAAO,CAAC,IAAI,CAAC,WAAW,EAAE,MAAM,CAAC,IAAI,CAAC,OAAO,CAAC,CAAC,CAAC;gBAClE,OAAO,UAAU,CAAC,GAAG,CAAC,UAAU,EAAE,OAAO,EAAE,OAAO,EAAE,GAAG,CAAC,gBAAgB,CAAC,CAAC;YAC5E,CAAC;YACD;gBACE,OAAO,EAAE,EAAE,EAAE,KAAK,EAAE,IAAI,EAAE,sBAAsB,IAAI,GAAG,EAAE,CAAC;QAC9D,CAAC;IACH,CAAC,CAAC;AACJ,CAAC"}
|
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
// This agent's own domain knowledge of which appq tools it may touch — the
|
|
2
|
+
// enforcement mechanism (assertToolAllowed / the gated dispatcher) lives in
|
|
3
|
+
// @appliqation/agent-core, shared with every sibling agent; only the
|
|
4
|
+
// allowlist content is local. Broader than a single-purpose workflow's
|
|
5
|
+
// palette on purpose — a genuine routing decision needs the same breadth of
|
|
6
|
+
// signal a senior engineer would look at, not one field. Zero write tools —
|
|
7
|
+
// this agent itself never calls an appq write tool; write-adjacent action
|
|
8
|
+
// happens only through the meta-tools (src/tools/metaTools.ts), which are
|
|
9
|
+
// themselves backed by the sibling agents' own safety boundaries.
|
|
10
|
+
export const READONLY_CONTEXT_TOOLS = new Set([
|
|
11
|
+
'get_scenario',
|
|
12
|
+
'get_automation_readiness',
|
|
13
|
+
'get_failure_patterns',
|
|
14
|
+
'get_defect_context',
|
|
15
|
+
'get_coverage_analysis',
|
|
16
|
+
'get_quality_context',
|
|
17
|
+
'get_evidence_summary',
|
|
18
|
+
'get_run_evidence',
|
|
19
|
+
'get_execution_evidence',
|
|
20
|
+
'get_test_results',
|
|
21
|
+
'search_tests',
|
|
22
|
+
'get_project_settings',
|
|
23
|
+
]);
|
|
24
|
+
// enrich_project_context is a SINGLE MCP tool with both action=read and
|
|
25
|
+
// action=write modes — not two separate tools. Kept out of
|
|
26
|
+
// READONLY_CONTEXT_TOOLS on purpose: that set's whole guarantee is "no
|
|
27
|
+
// write capability under any argument," and folding this in would quietly
|
|
28
|
+
// weaken that guarantee for every tool in it, not just this one.
|
|
29
|
+
// PROJECT_CONTEXT_TOOL itself, and the argument-level gate that enforces
|
|
30
|
+
// read-only access to it, now live in @appliqation/agent-core's
|
|
31
|
+
// tools/projectContext.ts — shared with every other headless agent that
|
|
32
|
+
// needs the identical guarantee, not just this one.
|
|
33
|
+
//# sourceMappingURL=safety.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"safety.js","sourceRoot":"","sources":["../../src/tools/safety.ts"],"names":[],"mappings":"AAAA,2EAA2E;AAC3E,4EAA4E;AAC5E,qEAAqE;AACrE,uEAAuE;AACvE,4EAA4E;AAC5E,4EAA4E;AAC5E,0EAA0E;AAC1E,0EAA0E;AAC1E,kEAAkE;AAElE,MAAM,CAAC,MAAM,sBAAsB,GAAG,IAAI,GAAG,CAAC;IAC5C,cAAc;IACd,0BAA0B;IAC1B,sBAAsB;IACtB,oBAAoB;IACpB,uBAAuB;IACvB,qBAAqB;IACrB,sBAAsB;IACtB,kBAAkB;IAClB,wBAAwB;IACxB,kBAAkB;IAClB,cAAc;IACd,sBAAsB;CACvB,CAAC,CAAC;AAEH,wEAAwE;AACxE,2DAA2D;AAC3D,uEAAuE;AACvE,0EAA0E;AAC1E,iEAAiE;AACjE,yEAAyE;AACzE,gEAAgE;AAChE,wEAAwE;AACxE,oDAAoD"}
|
package/package.json
ADDED
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "@appliqation/autopilot",
|
|
3
|
+
"version": "0.1.1",
|
|
4
|
+
"description": "An agentic orchestrator that decides, per test case, whether to run autonomous testing, generate new automation, or raise a pull request for it \u2014 reasoning over real signal (current pass/fail state, flakiness, linked defects, coverage priority), not a fixed script. Reasoning lives entirely in this repo, not a private server.",
|
|
5
|
+
"type": "module",
|
|
6
|
+
"bin": {
|
|
7
|
+
"appliqation-autopilot": "./dist/cli/index.js"
|
|
8
|
+
},
|
|
9
|
+
"engines": {
|
|
10
|
+
"node": ">=20"
|
|
11
|
+
},
|
|
12
|
+
"files": [
|
|
13
|
+
"dist/",
|
|
14
|
+
"README.md",
|
|
15
|
+
"LICENSE"
|
|
16
|
+
],
|
|
17
|
+
"scripts": {
|
|
18
|
+
"build": "tsc -p tsconfig.build.json",
|
|
19
|
+
"dev": "tsx src/cli/index.ts",
|
|
20
|
+
"typecheck": "tsc -p tsconfig.json --noEmit",
|
|
21
|
+
"lint": "eslint src --ext .ts",
|
|
22
|
+
"test": "vitest run",
|
|
23
|
+
"test:watch": "vitest"
|
|
24
|
+
},
|
|
25
|
+
"dependencies": {
|
|
26
|
+
"@appliqation/agent-core": "^0.1.0",
|
|
27
|
+
"commander": "^13.1.0",
|
|
28
|
+
"dotenv": "^16.4.7"
|
|
29
|
+
},
|
|
30
|
+
"devDependencies": {
|
|
31
|
+
"@types/node": "^22.13.10",
|
|
32
|
+
"tsx": "^4.19.3",
|
|
33
|
+
"typescript": "^5.8.2",
|
|
34
|
+
"vitest": "^3.2.7"
|
|
35
|
+
}
|
|
36
|
+
}
|