@miller-tech/uap 2.36.4 → 2.39.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/config/model-profiles/qwen38.json +3 -3
- package/config/model-registry.json +96 -6
- package/dist/.tsbuildinfo +1 -1
- package/dist/bin/cli.js +57 -0
- package/dist/bin/cli.js.map +1 -1
- package/dist/cli/deliver.d.ts +46 -0
- package/dist/cli/deliver.d.ts.map +1 -1
- package/dist/cli/deliver.js +155 -36
- package/dist/cli/deliver.js.map +1 -1
- package/dist/cli/models.d.ts +11 -1
- package/dist/cli/models.d.ts.map +1 -1
- package/dist/cli/models.js +164 -3
- package/dist/cli/models.js.map +1 -1
- package/dist/cli/workflow.d.ts +36 -0
- package/dist/cli/workflow.d.ts.map +1 -0
- package/dist/cli/workflow.js +246 -0
- package/dist/cli/workflow.js.map +1 -0
- package/dist/delivery/run-state.d.ts +1 -1
- package/dist/delivery/run-state.d.ts.map +1 -1
- package/dist/delivery/run-state.js +1 -1
- package/dist/delivery/run-state.js.map +1 -1
- package/dist/delivery/team-solve-deliver.d.ts +83 -0
- package/dist/delivery/team-solve-deliver.d.ts.map +1 -0
- package/dist/delivery/team-solve-deliver.js +191 -0
- package/dist/delivery/team-solve-deliver.js.map +1 -0
- package/dist/inference/model-files.d.ts +111 -0
- package/dist/inference/model-files.d.ts.map +1 -0
- package/dist/inference/model-files.js +172 -0
- package/dist/inference/model-files.js.map +1 -0
- package/dist/placement/registry.d.ts +650 -0
- package/dist/placement/registry.d.ts.map +1 -1
- package/dist/placement/registry.js +21 -0
- package/dist/placement/registry.js.map +1 -1
- package/dist/workflow/loader.d.ts +34 -0
- package/dist/workflow/loader.d.ts.map +1 -0
- package/dist/workflow/loader.js +232 -0
- package/dist/workflow/loader.js.map +1 -0
- package/dist/workflow/runner.d.ts +90 -0
- package/dist/workflow/runner.d.ts.map +1 -0
- package/dist/workflow/runner.js +416 -0
- package/dist/workflow/runner.js.map +1 -0
- package/dist/workflow/types.d.ts +400 -0
- package/dist/workflow/types.d.ts.map +1 -0
- package/dist/workflow/types.js +147 -0
- package/dist/workflow/types.js.map +1 -0
- package/docs/INDEX.md +1 -0
- package/docs/architecture/adr/0013-declarative-workflow-layer.md +99 -0
- package/docs/architecture/adr/0014-loop-steps-and-team-solve-deliver.md +142 -0
- package/docs/benchmarks/LCB_DROID_SMOKE.md +111 -0
- package/docs/guides/DELIVER.md +3 -1
- package/docs/guides/WORKFLOWS.md +179 -0
- package/docs/plans/model-performance-proof-options.md +39 -0
- package/docs/reference/CLI.md +33 -0
- package/docs/reference/CONFIGURATION.md +1 -0
- package/docs/reference/FEATURES.md +18 -0
- package/examples/auto-coordinator-example.js +127 -0
- package/examples/demo-order-solid.ts +19 -0
- package/examples/order-processing-strategy-pattern.ts +186 -0
- package/examples/order-processor-refactored.ts +473 -0
- package/examples/order-solid-refactor.ts +830 -0
- package/examples/refactored-order-processing.ts +106 -0
- package/examples/run-order-processing.ts +48 -0
- package/examples/solid-order-processing.ts +324 -0
- package/examples/strategy-pattern-example.ts +100 -0
- package/examples/workflows/team-solve/README.md +85 -0
- package/examples/workflows/team-solve/check-samples.sh +50 -0
- package/examples/workflows/team-solve/sample/task.md +22 -0
- package/examples/workflows/team-solve/sample/tests/1.in +2 -0
- package/examples/workflows/team-solve/sample/tests/1.out +1 -0
- package/examples/workflows/team-solve/sample/tests/2.in +2 -0
- package/examples/workflows/team-solve/sample/tests/2.out +1 -0
- package/examples/workflows/team-solve/sample/tests/3.in +2 -0
- package/examples/workflows/team-solve/sample/tests/3.out +1 -0
- package/examples/workflows/team-solve/sample/tests/4.in +2 -0
- package/examples/workflows/team-solve/sample/tests/4.out +1 -0
- package/examples/workflows/team-solve/sample/tests/5.in +1 -0
- package/examples/workflows/team-solve/sample/tests/5.out +1 -0
- package/examples/workflows/team-solve/steps/ideate-a.md +26 -0
- package/examples/workflows/team-solve/steps/ideate-b.md +28 -0
- package/examples/workflows/team-solve/steps/plan.md +20 -0
- package/examples/workflows/team-solve/steps/repair.md +25 -0
- package/examples/workflows/team-solve/steps/review.md +29 -0
- package/examples/workflows/team-solve/steps/solve.md +26 -0
- package/examples/workflows/team-solve/workflow.yaml +78 -0
- package/examples/workflows/team-solve-deliver/README.md +84 -0
- package/examples/workflows/team-solve-deliver/steps/ideate-a.md +25 -0
- package/examples/workflows/team-solve-deliver/steps/ideate-b.md +28 -0
- package/examples/workflows/team-solve-deliver/steps/plan.md +25 -0
- package/examples/workflows/team-solve-deliver/steps/repair.md +20 -0
- package/examples/workflows/team-solve-deliver/steps/review.md +31 -0
- package/examples/workflows/team-solve-deliver/steps/solve.md +28 -0
- package/examples/workflows/team-solve-deliver/steps/verify.md +9 -0
- package/examples/workflows/team-solve-deliver/workflow.yaml +74 -0
- package/package.json +2 -1
- package/src/policies/enforcers/__pycache__/_common.cpython-311.pyc +0 -0
- package/src/policies/enforcers/__pycache__/expert_review_required.cpython-311.pyc +0 -0
- package/src/policies/enforcers/__pycache__/quality_metrics_gate.cpython-311.pyc +0 -0
- package/templates/hooks/__pycache__/deliver_autoroute.cpython-311.pyc +0 -0
- package/tools/agents/scripts/__pycache__/toolcall_path_normalizer.cpython-311.pyc +0 -0
- package/tools/agents/scripts/anthropic_proxy.py +147 -14
|
@@ -0,0 +1,142 @@
|
|
|
1
|
+
# ADR 0014: Loop step kind + team-solve-deliver — the team directs the convergence loop
|
|
2
|
+
|
|
3
|
+
- Status: Accepted
|
|
4
|
+
- Date: 2026-11-03
|
|
5
|
+
- Spec: `uap deliver --team-solve` (src/cli/deliver.ts), the `loop` step kind
|
|
6
|
+
(src/workflow/), examples/workflows/team-solve-deliver/,
|
|
7
|
+
docs/guides/DELIVER.md, docs/guides/WORKFLOWS.md.
|
|
8
|
+
- Extends: ADR 0013 (the declarative workflow layer). Amends its decision #2
|
|
9
|
+
(three step kinds → four) and #7 (O7 now measures the shipped posture).
|
|
10
|
+
|
|
11
|
+
## Context
|
|
12
|
+
|
|
13
|
+
UAP has two engines that had never been combined:
|
|
14
|
+
|
|
15
|
+
1. **team-solve** (ADR 0013): a static role DAG — planner, parallel
|
|
16
|
+
ideators, solver, reviewer, repairer — fresh contexts, files as the only
|
|
17
|
+
channel, deterministic verdict. Measured shape (GVS5H/Memdoor prior:
|
|
18
|
+
9.8/10 LiveCodeBench Hard with a 27B local model). But its bundled form
|
|
19
|
+
is contest-shaped: everything happens in the run directory and a command
|
|
20
|
+
step is the verdict. A REPO mission needs tools and the project's real
|
|
21
|
+
gates, which the workflow engine did not have.
|
|
22
|
+
2. **deliver**: the convergence loop — a tool-using executor iterating
|
|
23
|
+
against the project's real completion gates (tiered ladder), with
|
|
24
|
+
posture escalation (single → phased → orchestrated → epics). But its
|
|
25
|
+
planning is a single model's inline reasoning; there is no role
|
|
26
|
+
structure, no parallel ideation, no fresh-context review.
|
|
27
|
+
|
|
28
|
+
The request: combine them — the convergence loop DIRECTED BY team-solve for
|
|
29
|
+
superior results — and make the behavior automatic in UAP, not a manual
|
|
30
|
+
incantation.
|
|
31
|
+
|
|
32
|
+
## Decision
|
|
33
|
+
|
|
34
|
+
1. **A fourth step kind: `loop`.** The engine stays the graph authority and
|
|
35
|
+
gains a convergence-loop seam: it assembles the instruction (role prompt
|
|
36
|
+
or `goal:`, direct-dependency summaries, inlined `reads:`) and hands it
|
|
37
|
+
to a caller-injected `LoopStepExecutor` which owns the iteration and
|
|
38
|
+
reports `{ success, summary, turns }`. A loop step declares no
|
|
39
|
+
`writes:` — it works the PROJECT tree; its DeliveryResult is the record.
|
|
40
|
+
Loop-only fields: `turns:` (bound, passed through) and `verifyOnly:`
|
|
41
|
+
(gates with no executor turns — a verdict, not a builder). A standalone
|
|
42
|
+
`uap workflow run` fails closed at run start: no provider is injected.
|
|
43
|
+
The loader enforces kind/field separation exactly as for the other kinds.
|
|
44
|
+
2. **Role steps ride the caller's executor rails.** `RunWorkflowOptions`
|
|
45
|
+
gains `completeStep` (per-step role-call seam) which takes precedence
|
|
46
|
+
over the raw `client`, and `client` becomes optional when it covers
|
|
47
|
+
every agent step. This is what lets deliver drive the roles through its
|
|
48
|
+
judge/generator rails (plan + review on the cool deterministic judge;
|
|
49
|
+
ideation on the generator) instead of re-implementing model choice.
|
|
50
|
+
3. **The bundled `team-solve-deliver` workflow** composes the two engines:
|
|
51
|
+
plan → ideate-a/ideate-b (parallel, different engineering families) →
|
|
52
|
+
solve (loop) → review (fresh-context judge) → repair (loop, review-
|
|
53
|
+
seeded) → verify (loop, verifyOnly). The run directory is the role
|
|
54
|
+
ledger; the project tree is the work surface.
|
|
55
|
+
4. **`uap deliver` injects itself as the seams** (src/delivery/
|
|
56
|
+
team-solve-deliver.ts — deps-injected and unit-tested, the epic/phased/
|
|
57
|
+
orchestrated extraction precedent). solve = the mission dispatch itself —
|
|
58
|
+
the SAME posture decision as any other mission (epics → orchestrated →
|
|
59
|
+
phased → single, verbatim extraction) with the team's approach enriching
|
|
60
|
+
the instruction; repair = a bounded ConvergenceLoop with the reviewer's
|
|
61
|
+
brief as instruction, fresh checkpoint state, and a declared `turns:`
|
|
62
|
+
bound honored as its cap (a NO REPAIR NEEDED verdict collapses to the
|
|
63
|
+
loop's own baseline check — zero turns, tree untouched); verify = the
|
|
64
|
+
tiered ladder with no executor turns, checked BEFORE any step-id
|
|
65
|
+
dispatch so a gates-only verdict can never run executor turns whatever
|
|
66
|
+
the id says. The final DeliveryResult aggregates honestly: verify pass
|
|
67
|
+
AND workflow status decide success; histories concatenate; the role
|
|
68
|
+
ledger path lands in finalFeedback. The role ledger's source directory
|
|
69
|
+
is surfaced at engagement (a project `.uap/workflows/` copy shadows the
|
|
70
|
+
bundled one — the operator must see which ran).
|
|
71
|
+
5. **Engagement is a posture, tier-gated AUTO by default.**
|
|
72
|
+
`resolveTeamSolveWanted` (pure, tested): explicit flag
|
|
73
|
+
(`--team-solve`/`--no-team-solve`) > env kill (`UAP_DELIVER_TEAM_SOLVE=0`)
|
|
74
|
+
> persistent config (`.uap.json` `deliver.teamSolve` on/off) > AUTO,
|
|
75
|
+
which engages for high/critical complexity AND a non-single-file scope
|
|
76
|
+
(`looksSingleFileScoped`: exactly one named file + short instruction
|
|
77
|
+
stands down — the classifier's keyword ladder promotes "optimize"/
|
|
78
|
+
"refactor" regardless of scope, which is right for cheap model routing
|
|
79
|
+
and wrong for a 4-6× pipeline multiplier; an operator who names one file
|
|
80
|
+
has already disambiguated the scope) — the reliability ladder's "hardest
|
|
81
|
+
tier → full loop + escalation" row now resolves to this shape.
|
|
82
|
+
Stand-downs: self-gate and acceptance-primary (no objective ladder for
|
|
83
|
+
the roles to converge against — the verify pass would be vacuous),
|
|
84
|
+
lazy-solved (nothing left to direct), and resumes under AUTO (the
|
|
85
|
+
interrupted run continues on its recorded posture; an explicit
|
|
86
|
+
`--team-solve` resume re-runs the team wrapper over the resumed mission).
|
|
87
|
+
If the workflow is missing or its step shape has drifted from what the
|
|
88
|
+
seam mapping is keyed to, the run falls back to the standard dispatch
|
|
89
|
+
with a warning — auto engagement must never make deliver worse.
|
|
90
|
+
6. **Composition, not replacement.** team-solve composes with epics and
|
|
91
|
+
orchestration rather than competing with them: the solve step dispatches
|
|
92
|
+
through the same chain, so a massive mission still sequences as epics —
|
|
93
|
+
each epic's loop now directed by the team's approach. One nuance: for
|
|
94
|
+
PHASED/ORCHESTRATED missions the phase plan is produced from the
|
|
95
|
+
un-enriched instruction BEFORE the solve step runs, so the team directs
|
|
96
|
+
per-phase execution but not the decomposition itself; epics are the
|
|
97
|
+
exception (their planning happens inside the solve dispatch, after the
|
|
98
|
+
enrichment).
|
|
99
|
+
7. **The solve step is not engine-retried.** The bundled workflow sets
|
|
100
|
+
`maxRepairsPerStep: 0`: the solve loop already converges internally
|
|
101
|
+
(until-delivered + escalation inside the mission dispatch), and a
|
|
102
|
+
workflow-level retry would re-run the ENTIRE dispatch after it burned
|
|
103
|
+
its turn budget — doubling the hardest-tier spend and bypassing this
|
|
104
|
+
workflow's own review→repair discipline. The repair loop is the retry.
|
|
105
|
+
|
|
106
|
+
## Alternatives considered
|
|
107
|
+
|
|
108
|
+
- **Make deliver read workflow.yaml directly (no loop kind)** — rejected:
|
|
109
|
+
the engine would need to know about convergence loops, gates, and
|
|
110
|
+
DeliveryResult; the seam keeps the workflow layer generic and testable
|
|
111
|
+
with stub loops.
|
|
112
|
+
- **A second engine (a "team deliver" runner beside ConvergenceLoop)** —
|
|
113
|
+
rejected: duplicates gate logic, posture routing, resume bookkeeping;
|
|
114
|
+
the wrapper composes the existing, already-tested machinery.
|
|
115
|
+
- **Auto-engage on every mission** — rejected: role overhead is real tokens
|
|
116
|
+
(4+ completions before the first tool call); the ladder escalates with
|
|
117
|
+
MEASURED difficulty, not anxiety. Simple/moderate missions keep the
|
|
118
|
+
cheap posture.
|
|
119
|
+
- **Team-solve replaces the epics/orchestration postures** — rejected:
|
|
120
|
+
they answer different questions (context scaling vs approach quality);
|
|
121
|
+
the composition answers both.
|
|
122
|
+
|
|
123
|
+
## Consequences
|
|
124
|
+
|
|
125
|
+
- `runState.runnerKind` gains `'teamsolve'` (transient: the solve step's
|
|
126
|
+
dispatch records the inner posture, which is what a resume re-enters; a
|
|
127
|
+
crash mid-roles records 'teamsolve' and the resume warns + falls to the
|
|
128
|
+
inner path).
|
|
129
|
+
- Review visibility is v1-honest and PROMPT-MATCHED: the reviewer receives
|
|
130
|
+
the plan file plus a one-line solve report (gate status, turns,
|
|
131
|
+
changed-tree flag, feedback tail) and `steps/review.md` is written to
|
|
132
|
+
judge exactly on that — no file list, no diff. A richer review surface
|
|
133
|
+
(diff inlining) is a tracked follow-up.
|
|
134
|
+
- Loader tightening (fail-closed by design): stray `turns:`/`verifyOnly:`
|
|
135
|
+
on a non-loop step previously zod-stripped silently; it now exits 3.
|
|
136
|
+
- O7's arm C is now the shipped auto posture (`--no-team-solve` vs
|
|
137
|
+
`--team-solve` on the same tasks) — the experiment validates the
|
|
138
|
+
product, not a bespoke rig.
|
|
139
|
+
- Known follow-ups (tracked, deliberate): diff-surface for the reviewer;
|
|
140
|
+
role-pin awareness inside completeStep (rails own model choice today);
|
|
141
|
+
ledger-aware resume for an explicitly `--team-solve` resumed run (roles
|
|
142
|
+
re-run; the ledger files persist for inspection).
|
|
@@ -0,0 +1,111 @@
|
|
|
1
|
+
# LCB-Droid Smoke — 10 problems × 1 pass, arm 2 (UAP-in-Droid)
|
|
2
|
+
|
|
3
|
+
**Result: 10/10 = 100% pass@1 on hidden tests. All 10 public gates green. All 10 scorer-control references green. 9/10 agents self-stopped; 1/10 hit the wall cap with a gate-green solution collected at cap.**
|
|
4
|
+
|
|
5
|
+
Run: 2026-10-27, started ~01:21 local, 284 min serial wall time (~4.7 h).
|
|
6
|
+
Model: `abliterated-iq3_s` (DavidAU Qwen3.8-Flash-Next IQ3_S abliterated)
|
|
7
|
+
served by strata on `:8080`. Chain:
|
|
8
|
+
`droid exec` → `uap-anthropic-proxy` `:4000` → strata. Arm 2 only: the
|
|
9
|
+
UAP operating protocol (AGENTS.md) with restricted tools
|
|
10
|
+
(`Read,Glob,Grep,LS,Create,Edit,Execute`), 60-min wall cap with
|
|
11
|
+
collect-at-cap, concurrency 1. Scoring: the vendored LCB fork's
|
|
12
|
+
`codegen_metrics` (hidden tests, k=1, `release_v6`), each attempt scored
|
|
13
|
+
alongside the published passing reference as a per-problem control.
|
|
14
|
+
|
|
15
|
+
## Caveats — read before comparing any number below
|
|
16
|
+
|
|
17
|
+
1. **The abliteration delta is unmeasured and is the strongest confound.**
|
|
18
|
+
The local model is the *abliterated* IQ3_S quant, not the pristine
|
|
19
|
+
Flash Next the published numbers describe. Abliteration can move
|
|
20
|
+
code-generation reasoning in either direction; no local A/B measures it.
|
|
21
|
+
2. **Local quant vs remote full precision.** GVS5H's published Flash Next
|
|
22
|
+
ran on a pinned remote backend (full-precision API model,
|
|
23
|
+
`reasoning=on`, 128k cap). The local arm is IQ3_S.
|
|
24
|
+
3. **Different mechanism.** GVS5H's orchestrated arm is ledger-based
|
|
25
|
+
self-orchestration in their scaffold; arm 2 is the UAP protocol inside
|
|
26
|
+
the Droid harness (droid exec + gates + tool restrictions). Same
|
|
27
|
+
benchmark, different agent machinery.
|
|
28
|
+
4. **This subset is published-solved.** The published orchestrated arm
|
|
29
|
+
scores 10/10 on pass 1 over these exact 10 problems (9 of them 5/5).
|
|
30
|
+
A local 10/10 therefore **validates the pipeline and reaches the subset
|
|
31
|
+
ceiling — it does not establish parity or superiority on the full 100**,
|
|
32
|
+
where the published orchestrated mean is 93.0 and single-call is 84.2.
|
|
33
|
+
5. **n = 1 pass.** No variance measured. Published per-pass spread on the
|
|
34
|
+
full 100 is 92–94 (orchestrated) and 82–86 (single).
|
|
35
|
+
6. **One wall-capped problem.** `abc399_e` ran to the 60-min cap; its
|
|
36
|
+
collected-at-cap solution passes both public and hidden tests, but the
|
|
37
|
+
session never self-terminated (result doc killed before emission, so
|
|
38
|
+
whether it kept working past a green gate is unobservable).
|
|
39
|
+
7. **Token usage is unmeasurable on this chain** — the local proxy reports
|
|
40
|
+
zero usage counters, so no cost/efficiency comparison is possible.
|
|
41
|
+
|
|
42
|
+
## Per-problem results
|
|
43
|
+
|
|
44
|
+
| Problem | Style | Wall | Turns | Stop | Public gate | Hidden | Pub. orchestrated p1 | Pub. single p1 |
|
|
45
|
+
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|
|
46
|
+
| arc196_b | stdin | 19.1 min | 60 | done | PASS | **PASS** | True (5/5) | True |
|
|
47
|
+
| arc196_c | stdin | 56.4 min | 81 | done | PASS | **PASS** | True (5/5) | True |
|
|
48
|
+
| arc196_a | stdin | 15.3 min | 53 | done | PASS | **PASS** | True (5/5) | True |
|
|
49
|
+
| arc196_d | stdin | 57.3 min | 43 | done | PASS | **PASS** | True (5/5) | True |
|
|
50
|
+
| 3777 | leetcode | 8.9 min | 81 | done | PASS | **PASS** | True (4/5) | True |
|
|
51
|
+
| abc400_e | stdin | 5.9 min | 43 | done | PASS | **PASS** | True (5/5) | True |
|
|
52
|
+
| abc400_g | stdin | 48.5 min | 81 | done | PASS | **PASS** | True (5/5) | True |
|
|
53
|
+
| 3717 | leetcode | 5.1 min | 43 | done | PASS | **PASS** | True (5/5) | **False** |
|
|
54
|
+
| 3765 | leetcode | 7.6 min | 81 | done | PASS | **PASS** | True (5/5) | True |
|
|
55
|
+
| abc399_e | stdin | 60 min (cap) | — | wall-cap | PASS | **PASS** | True (5/5) | True |
|
|
56
|
+
|
|
57
|
+
Smoke pass@1: **10/10 (100%)** on this subset. Published orchestrated p1
|
|
58
|
+
on the same subset: 10/10. Published single-call p1 on the same subset:
|
|
59
|
+
9/10 (missed 3717, which the local arm solved).
|
|
60
|
+
|
|
61
|
+
## Anchors (published, full 100-hardest, release_v6, pass@1)
|
|
62
|
+
|
|
63
|
+
| Arm | Published pass@1 (full 100) |
|
|
64
|
+
| --- | --- |
|
|
65
|
+
| Qwen3.8 Flash Next single-call | 84.2 |
|
|
66
|
+
| Qwen3.8 Flash Next orchestrated | **93.0** |
|
|
67
|
+
| Claude Fable 5 single-call | 90.4 |
|
|
68
|
+
| Qwen3.8-27B single / orchestrated | 66.8 / 92.4 |
|
|
69
|
+
|
|
70
|
+
The local smoke subset (10 problems) is not comparable to these full-100
|
|
71
|
+
means; it is listed here because these are the anchors any full run will
|
|
72
|
+
be judged against.
|
|
73
|
+
|
|
74
|
+
## Interpretation
|
|
75
|
+
|
|
76
|
+
The smoke's purpose (per spec) was end-to-end pipeline validation, and
|
|
77
|
+
that succeeded completely: rendering, verbatim prompt fidelity, public-test
|
|
78
|
+
gates, UAP stop discipline (9/10 self-stops plus one collect-at-cap
|
|
79
|
+
rescue), served-model snapshots, resumable records, and hidden-test
|
|
80
|
+
scoring with per-problem reference controls all worked with zero
|
|
81
|
+
infrastructure blockers.
|
|
82
|
+
|
|
83
|
+
The 10/10 is at the subset ceiling — consistent with the published
|
|
84
|
+
orchestrated arm, but this subset cannot discriminate arms. Any claim
|
|
85
|
+
about local-vs-published performance requires the full 100 (or a
|
|
86
|
+
randomly drawn, not published-solved, subset).
|
|
87
|
+
|
|
88
|
+
Runtime scale for follow-on decisions (measured, serial, concurrency 1):
|
|
89
|
+
mean 28.4 min/problem, total 284 min for 10. Extrapolations: 50×3 pilot
|
|
90
|
+
≈ 71 h; full 100×5 ≈ 237 h (~10 days). Two problems ran near the 60-min
|
|
91
|
+
cap, so a full run should expect some wall-caps and keep collect-at-cap.
|
|
92
|
+
|
|
93
|
+
## Artifacts
|
|
94
|
+
|
|
95
|
+
- `benchmarks/lcb-droid/results/smoke-10/` — `summary.json`,
|
|
96
|
+
`records.jsonl` (per-attempt records with served-model snapshots),
|
|
97
|
+
`smoke-scored.jsonl` (official verdicts + controls), `solutions/*.py`
|
|
98
|
+
(the 10 collected model outputs), `droid-results/*.json` (droid result
|
|
99
|
+
docs per completed session; abc399_e's was killed at cap).
|
|
100
|
+
- `benchmarks/lcb-droid/render_tasks.py`, `run_smoke.py`, `score_smoke.py`
|
|
101
|
+
— reproduce with the recipe in `benchmarks/lcb-droid/README.md`.
|
|
102
|
+
- `benchmarks/lcb-droid/anchors.json` — published anchors and per-problem
|
|
103
|
+
verdicts (GVS5H @ 707e2129).
|
|
104
|
+
|
|
105
|
+
## Decision gate (requires explicit go)
|
|
106
|
+
|
|
107
|
+
1. **Stop** — smoke goal met, archive as-is.
|
|
108
|
+
2. **Pilot 50×3** — ~71 h serial; adds variance signal.
|
|
109
|
+
3. **Full 100×5** — ~237 h serial; directly comparable to published means.
|
|
110
|
+
4. **Add arm 1** (no AGENTS.md protocol) — isolates the protocol's
|
|
111
|
+
contribution on the same chain; any of the above can be combined with it.
|
package/docs/guides/DELIVER.md
CHANGED
|
@@ -58,6 +58,7 @@ For a mission too large to hold in one context, `deliver` scales down the contex
|
|
|
58
58
|
- **`--decompose`** splits the mission into sequential phases, each converged by its own loop (auto for long, complex instructions; `--no-decompose` forbids it).
|
|
59
59
|
- **`--orchestrate`** runs those phases through the **blackboard orchestrator** with *minimal* per-task context — each task sees only its goal + its direct-dependency outputs. Implies `--decompose`.
|
|
60
60
|
- **`--epics`** runs a massive mission as a sequence of **epics, each a fresh session** — only prior epics' summaries are injected, never their full code — looped until each is accepted (auto for very long missions; `--no-epics` disables).
|
|
61
|
+
- **`--team-solve`** runs the mission through the **team-solve-deliver workflow**: team roles direct the convergence loop. A planner and two parallel ideators (minimal-change vs structural) produce an approach; the **solve** step runs the SAME posture decision as any other mission (epics → orchestrated → phased → single) with that approach enriching the instruction; a **fresh-context reviewer** judges the outcome against the plan; a review-seeded **repair** loop applies only what the review named; and a **gates-only verify** pass — the tiered ladder with no executor turns — decides the mission. **AUTO by default**: engages for high/critical-complexity missions only (the reliability ladder's hardest-tier row), stands down on self-gate, lazy-solved, and resumed runs. `--no-team-solve`, `UAP_DELIVER_TEAM_SOLVE=0`, or `.uap.json` `deliver.teamSolve: "off"` disable; `deliver.teamSolve: "on"` always engages. The role ledger (plan, proposals, repair brief) lands in `.uap/workflow-runs/<runId>/`. See [Workflows](WORKFLOWS.md) for the workflow format.
|
|
61
62
|
|
|
62
63
|
**Contracts-first** (on by default inside epic/decompose runs; `UAP_DELIVER_CONTRACTS=0` disables): the first epic plans the shared types, interfaces, and registry APIs the rest build against. Once accepted, its files are **locked read-only** so later epics build against frozen signatures and can't drift them. The public surface is *extracted and verified against emitted source* — only names proven present are recorded — which is what makes cross-module composition reliable without holding every module in context.
|
|
63
64
|
|
|
@@ -202,13 +203,14 @@ Long autonomous runs fail in predictable ways; `deliver` has a guard for each:
|
|
|
202
203
|
| `--ci-passes <n>` | Max CI re-converge passes on failure (1–10, default `2`) |
|
|
203
204
|
| `--ci-timeout <minutes>` | CI watch budget in minutes (1–120, default `20`) |
|
|
204
205
|
|
|
205
|
-
**Big builds (decompose → orchestrate → epics)**
|
|
206
|
+
**Big builds (decompose → orchestrate → epics → team-solve)**
|
|
206
207
|
|
|
207
208
|
| Flag | Purpose |
|
|
208
209
|
|---|---|
|
|
209
210
|
| `--decompose` / `--no-decompose` | Split the mission into sequential phases, each converged by its own loop (auto for long complex tasks) |
|
|
210
211
|
| `--orchestrate` / `--no-orchestrate` | Run decomposed tasks through the blackboard orchestrator with MINIMAL per-task context (implies `--decompose`) |
|
|
211
212
|
| `--epics` / `--no-epics` | Run a massive mission as a sequence of fresh-session epics until each is accepted (auto for very long missions) |
|
|
213
|
+
| `--team-solve` / `--no-team-solve` | Run the mission through the team-solve-deliver workflow: roles plan/ideate/review, solve+repair are gates-driven convergence loops, verify is gates-only (AUTO on high/critical complexity) |
|
|
212
214
|
|
|
213
215
|
**Exploration & quality aids**
|
|
214
216
|
|
|
@@ -0,0 +1,179 @@
|
|
|
1
|
+
# Workflows — static step DAGs over the blackboard orchestrator
|
|
2
|
+
|
|
3
|
+
A workflow is **data, not a plan**: one `workflow.yaml` plus role-prompt files
|
|
4
|
+
that describe a step graph — agent roles, deterministic commands, approval
|
|
5
|
+
gates. The runner executes it on the same machinery `uap deliver` uses
|
|
6
|
+
(task-orchestrator: dependency-aware dispatch, bounded repair,
|
|
7
|
+
resume-at-the-step-boundary), so the graph semantics are the validated
|
|
8
|
+
delivery semantics.
|
|
9
|
+
|
|
10
|
+
The shape this buys is the measured one from GVS5H/Memdoor: several small
|
|
11
|
+
roles — planner, ideators, solver, reviewer, repairer — each in a FRESH
|
|
12
|
+
context, communicating only through files, with a command step's exit code
|
|
13
|
+
(rather than any model's claim) as the verdict. Five Qwen3.8-27B roles on
|
|
14
|
+
this shape matched the authors' own harness on LiveCodeBench Hard (9.8/10
|
|
15
|
+
vs 9.6, Fable 5 single-call at 8.0 — their runs, their caveats; see
|
|
16
|
+
`examples/workflows/team-solve/README.md`).
|
|
17
|
+
|
|
18
|
+
## Quickstart
|
|
19
|
+
|
|
20
|
+
```bash
|
|
21
|
+
uap workflow list # what exists, from where
|
|
22
|
+
uap workflow show team-solve # the validated plan + pins
|
|
23
|
+
uap workflow run team-solve \
|
|
24
|
+
--input examples/workflows/team-solve/sample # run the LIS smoke task
|
|
25
|
+
uap workflow show team-solve-deliver # the loop-step variant (deliver-driven)
|
|
26
|
+
uap workflow status # every run + its verdict
|
|
27
|
+
uap workflow resume <runId> # continue at the boundary
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
Exit codes: `0` success · `1` a step failed · `2` blocked at a gate
|
|
31
|
+
(resumable) · `3` the workflow file is invalid. Cron-safe by construction.
|
|
32
|
+
|
|
33
|
+
Two bundled workflows: **team-solve** (contest-shaped: everything happens in
|
|
34
|
+
the run directory, a command step is the verdict — runs standalone) and
|
|
35
|
+
**team-solve-deliver** (repo missions: solve/repair are convergence loops
|
|
36
|
+
against the project's real gates, verify is gates-only — driven by
|
|
37
|
+
`uap deliver --team-solve`, which injects the loop machinery; standalone
|
|
38
|
+
runs fail closed). See
|
|
39
|
+
[examples/workflows/team-solve-deliver/README.md](../../examples/workflows/team-solve-deliver/README.md).
|
|
40
|
+
|
|
41
|
+
## Where workflows live
|
|
42
|
+
|
|
43
|
+
Search order — first hit wins, so a project can override a bundled workflow
|
|
44
|
+
without touching the install:
|
|
45
|
+
|
|
46
|
+
1. `<project>/.uap/workflows/<name>/` — project-owned (committed)
|
|
47
|
+
2. `~/.uap/workflows/<name>/` — operator-wide
|
|
48
|
+
3. `<package>/examples/workflows/<name>/` — bundled (team-solve ships here)
|
|
49
|
+
|
|
50
|
+
## The file format
|
|
51
|
+
|
|
52
|
+
```yaml
|
|
53
|
+
name: my-workflow
|
|
54
|
+
description: One line, shown in `show`.
|
|
55
|
+
defaultModel: qwen38-27b # optional pin ladder default (see below)
|
|
56
|
+
parallel: 2 # max concurrent independent steps (default 1)
|
|
57
|
+
maxRepairsPerStep: 1 # bounded re-execution of a failed step
|
|
58
|
+
steps:
|
|
59
|
+
- id: plan # lowercase slug (repair chains use <id>.r0-…)
|
|
60
|
+
kind: agent # agent | command | gate | loop
|
|
61
|
+
title: Planner # optional label
|
|
62
|
+
prompt: steps/plan.md # role-prompt file, relative to this directory
|
|
63
|
+
reads: [task.md] # run-dir files inlined into the prompt
|
|
64
|
+
writes: [plan.md] # writes[0] receives the reply (the ledger)
|
|
65
|
+
requires: [] # DAG edges: ids that must SUCCEED first
|
|
66
|
+
- id: verify
|
|
67
|
+
kind: command
|
|
68
|
+
command: bash check.sh --strict # cwd = the run directory
|
|
69
|
+
writes: [verify.log]
|
|
70
|
+
requires: [plan]
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
### Step kinds
|
|
74
|
+
|
|
75
|
+
- **agent** — one fresh completion: the role prompt (or inline `goal:`),
|
|
76
|
+
plus direct-dependency summaries and the inlined `reads:` files, sent to
|
|
77
|
+
the pinned model. The reply is written verbatim to `writes[0]`. There is
|
|
78
|
+
no chat: the run directory is the only memory. Agent steps take exactly
|
|
79
|
+
one of `prompt:` / `goal:`.
|
|
80
|
+
- **command** — a shell line in the run directory. Exit 0 = success. With
|
|
81
|
+
`neverFails: true` the output is captured as evidence and the step never
|
|
82
|
+
fails the run (team-solve's `test` step); without it, a nonzero exit IS
|
|
83
|
+
the verdict (team-solve's `verify`). Output lands in `writes[0]` (or
|
|
84
|
+
`<id>.log`). `timeoutMs` bounds it (default 120000). The workflow's own
|
|
85
|
+
directory is exported as `$UAP_WORKFLOW_DIR` — reference harness scripts
|
|
86
|
+
there (`bash "$UAP_WORKFLOW_DIR/check.sh"`) instead of copying them into
|
|
87
|
+
the run.
|
|
88
|
+
- **gate** — pauses the run for an operator decision. Default `waitMs: 0`
|
|
89
|
+
returns immediately as BLOCKED (exit 2); `uap workflow approve <runId>
|
|
90
|
+
<step>` records the approval, then `resume` re-runs the gate and passes.
|
|
91
|
+
Set `waitMs` if a run should hold a terminal open for approval instead.
|
|
92
|
+
- **loop** — a convergence loop owned by the CALLER, not the engine: the
|
|
93
|
+
engine assembles the instruction (role prompt or `goal:`, plus
|
|
94
|
+
direct-dependency summaries and inlined `reads:`) and hands it to an
|
|
95
|
+
injected `runLoop` seam, which drives a tools-and-gates iteration to
|
|
96
|
+
completion and reports `{ success, summary, turns }`. The step outcome is
|
|
97
|
+
the seam's result — there is no `writes[0]` (a loop works the project
|
|
98
|
+
tree; its DeliveryResult is the record). Loop-only fields: `turns:` (a
|
|
99
|
+
turn bound passed through to the seam) and `verifyOnly:` (the loop runs
|
|
100
|
+
the gates with NO executor turns — a verdict, not a builder; this is what
|
|
101
|
+
`team-solve-deliver`'s final step is). A standalone `uap workflow run`
|
|
102
|
+
cannot execute loop steps — no provider is injected — and fails closed at
|
|
103
|
+
run start with that message; `uap deliver --team-solve` is the caller
|
|
104
|
+
that can.
|
|
105
|
+
|
|
106
|
+
### Trust boundary
|
|
107
|
+
|
|
108
|
+
Authoring a workflow is authoring shell + prompts for your own account:
|
|
109
|
+
`command:` lines run through a shell as the invoking user (the same trust
|
|
110
|
+
level as a `package.json` script or a CI step — this is the design, not a
|
|
111
|
+
bug), and `--input` task directories are copied into the run directory
|
|
112
|
+
where later steps may read them. Authored commands may EXECUTE files a
|
|
113
|
+
model produced — team-solve's harness runs `python3 solution.py`, model
|
|
114
|
+
code under your privileges, exactly like `uap deliver` gate scripts. Model
|
|
115
|
+
replies are data on the way IN (prompts and files); whether they execute
|
|
116
|
+
is a decision the workflow author made when writing the command step. For
|
|
117
|
+
untrusted tasks: insert a `gate` before the first command step, or wrap
|
|
118
|
+
the run — `uap sandbox [--net off] -- uap workflow run <name>`.
|
|
119
|
+
|
|
120
|
+
Two consequences of the trust model worth knowing:
|
|
121
|
+
|
|
122
|
+
- **Workflows are discovered from the current directory** (`.uap/workflows/`
|
|
123
|
+
first). Running `uap workflow run <name>` inside a cloned repository
|
|
124
|
+
executes that repo's command steps — the same act as running its npm
|
|
125
|
+
scripts. Read a repo's workflows before running them.
|
|
126
|
+
- **`--endpoint <url>` receives your prompts.** Prompt assembly inlines
|
|
127
|
+
`reads:` file contents; a third-party endpoint sees them. Local pins
|
|
128
|
+
keep that traffic on the loopback.
|
|
129
|
+
|
|
130
|
+
### Model pins
|
|
131
|
+
|
|
132
|
+
Steps name no model — the pin ladder answers:
|
|
133
|
+
|
|
134
|
+
```
|
|
135
|
+
step.model → workflow.defaultModel → uap workflow run --model → fleet default
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
The fleet default is `qwen38-27b` (the strata arm through the proxy,
|
|
139
|
+
`http://127.0.0.1:4000/v1`). `--endpoint <url>` overrides the endpoint for
|
|
140
|
+
every arm, so the same workflow runs on any OpenAI-compatible server.
|
|
141
|
+
|
|
142
|
+
### Authoring rules (enforced at load, exit 3 on violation)
|
|
143
|
+
|
|
144
|
+
- Step ids are lowercase slugs; `requires:` must reference known ids; the
|
|
145
|
+
graph must be a DAG (cycles are authoring bugs, surfaced not survived).
|
|
146
|
+
- Fields are kind-checked: `command` on an agent step, `reads` on a command
|
|
147
|
+
step, etc. are rejected.
|
|
148
|
+
- Two steps that write the same file with no `requires:` path between them
|
|
149
|
+
are rejected — parallel siblings must not be able to clobber each other
|
|
150
|
+
(the graph-safety FALSE INDEPENDENCE invariant, checked at authoring
|
|
151
|
+
time instead of merge time).
|
|
152
|
+
- `reads`/`writes` are run-directory relative; absolute paths and `..`
|
|
153
|
+
escapes are rejected.
|
|
154
|
+
- Steps may not write `run-state.json` or `.gates/` — the runner's own files.
|
|
155
|
+
|
|
156
|
+
## Runs, state, resume
|
|
157
|
+
|
|
158
|
+
Every run gets `.uap/workflow-runs/<runId>/`: the step output files (the
|
|
159
|
+
ledger), `run-state.json` (status + accepted steps + summaries), and gate
|
|
160
|
+
approvals. `--input <dir>` copies a task directory (e.g. `task.md` +
|
|
161
|
+
`tests/`) into the fresh run dir first.
|
|
162
|
+
|
|
163
|
+
State persists after EVERY accepted step. An interrupted run resumes with
|
|
164
|
+
`uap workflow resume <runId>` — accepted steps are seeded into the
|
|
165
|
+
blackboard and never re-executed; the model is only paid for what remains.
|
|
166
|
+
|
|
167
|
+
## What this is not
|
|
168
|
+
|
|
169
|
+
- Not a replacement for `uap deliver`: deliver's convergence loop is for
|
|
170
|
+
repo-editing tasks judged by real gates; a workflow's agent steps are
|
|
171
|
+
single completions producing files. Use a workflow when the SHAPE is the
|
|
172
|
+
asset — when the same role graph should run on many inputs.
|
|
173
|
+
- Not the Memdoor harness: the engine is UAP's orchestrator; the ported
|
|
174
|
+
team-solve keeps the method's structure and discipline, not its byte
|
|
175
|
+
parity. Its published numbers belong to the authors.
|
|
176
|
+
- The validation claim (does role-DAG structure beat the gate loop on the
|
|
177
|
+
same local model?) is option O7 in
|
|
178
|
+
[model-performance-proof-options.md](../plans/model-performance-proof-options.md)
|
|
179
|
+
and starts only on an explicit go, like every option there.
|
|
@@ -153,12 +153,51 @@ comparator (K3's published 88.3, harness caveat attached).
|
|
|
153
153
|
- **Discrimination status:** 89 tasks × 5 attempts is powered for the
|
|
154
154
|
published convention's effect sizes; report Δ with bootstrap CI + McNemar.
|
|
155
155
|
|
|
156
|
+
### O7 — Role-DAG structure vs the gate loop (free, GPU-local)
|
|
157
|
+
|
|
158
|
+
**Claims:** C1 (its strongest form) · **Cost:** GPU-hours only ·
|
|
159
|
+
**Prereqs:** `uap workflow` (shipped) + the bundled
|
|
160
|
+
[team-solve](../../examples/workflows/team-solve/README.md) +
|
|
161
|
+
[team-solve-deliver](../../examples/workflows/team-solve-deliver/README.md)
|
|
162
|
+
(auto-applied by `uap deliver` on high/critical complexity) + a
|
|
163
|
+
LiveCodeBench-Hard-style task set.
|
|
164
|
+
|
|
165
|
+
The measured prior is external and published: five Qwen3.8-27B roles on a
|
|
166
|
+
static relay-with-cross-correction graph matched the authors' own harness on
|
|
167
|
+
LiveCodeBench Hard (9.8/10 vs 9.6 on their first ten; Fable 5 single-call at
|
|
168
|
+
8.0) — structure, not model class, carried the result. UAP now has the
|
|
169
|
+
capability to test that prior against ITS OWN baseline: the gate loop. The
|
|
170
|
+
experiment now validates EXACTLY what ships — team-solve-deliver is the
|
|
171
|
+
auto-applied posture for high/critical missions, so arm C is the product's
|
|
172
|
+
default hardest-tier behavior, not a bespoke rig:
|
|
173
|
+
|
|
174
|
+
| arm | treatment |
|
|
175
|
+
|---|---|
|
|
176
|
+
| A | single-shot (one completion per problem) |
|
|
177
|
+
| B | `uap deliver --no-team-solve` gate loop (the pre-existing C1 baseline) |
|
|
178
|
+
| C | `uap deliver --team-solve` (the shipped auto posture: roles direct the convergence loop; solve/repair are gates-driven loops; verify is gates-only) |
|
|
179
|
+
|
|
180
|
+
- **Would prove:** whether role-DAG structure lifts a weak local model
|
|
181
|
+
BEYOND the gate loop alone — the C1 claim's strongest form, and the first
|
|
182
|
+
number that would justify the workflow layer (and its auto-engagement on
|
|
183
|
+
the hardest tier) on measured grounds rather than the GVS5H prior.
|
|
184
|
+
- **Would NOT prove:** parity with the authors' harness (their numbers are
|
|
185
|
+
theirs — different prompts, OpenRouter host, their grader; UAP's port
|
|
186
|
+
keeps the method, not the bytes), anything about frontier models, or any
|
|
187
|
+
C2/C3 claim.
|
|
188
|
+
- **Discrimination status:** pilot first (10 problems × 3 arms × 3 passes);
|
|
189
|
+
the paired format's bootstrap CI sizes the power-up. Contamination
|
|
190
|
+
caveats travel with the tasks exactly as they did for the authors.
|
|
191
|
+
|
|
156
192
|
## Suggested order
|
|
157
193
|
|
|
158
194
|
1. **O1** (free, hours of analysis, no risk) → publishes the descriptive dataset and
|
|
159
195
|
targets O2's strata.
|
|
160
196
|
2. **O2 hardest-tier replication** (free, targeted) → upgrades the ladder's
|
|
161
197
|
strongest row from underpowered to powered, or retires it honestly.
|
|
198
|
+
**O7** (also free, also C1) can ride the same task-set effort: pilot the
|
|
199
|
+
three arms once the task set exists, and only power up whichever arm
|
|
200
|
+
delta the pilot says is worth powering.
|
|
162
201
|
3. **O4 + O5** (free, small) → the C3 throughput numbers, including the
|
|
163
202
|
demote-if-null honesty.
|
|
164
203
|
4. **O3** (the gap) → design review first (the sequence design is its own
|
package/docs/reference/CLI.md
CHANGED
|
@@ -53,6 +53,7 @@ runs, keeping `--help` fast.
|
|
|
53
53
|
| [`proxy`](#proxy) | Reference-counted, session-scoped local inference proxy lifecycle |
|
|
54
54
|
| [`orchestrator`](#orchestrator) | Toggle the long multi-turn deliver orchestrator |
|
|
55
55
|
| [`handsfree`](#handsfree) (`hf`) | Hands-free persistence: loop any model to a 100% ledger |
|
|
56
|
+
| [`workflow`](#workflow) | Static YAML step DAGs (agent/command/gate/loop) over the orchestrator |
|
|
56
57
|
| [`challenge`](#challenge) | Open multi-agent challenge with a significance-gated leaderboard |
|
|
57
58
|
| [`sandbox`](#sandbox) | Run a command with a kernel-enforced writable-dir boundary |
|
|
58
59
|
| [`react`](#react) | Resolve dynamic experts/skills/patterns for a lifecycle event (JSON) |
|
|
@@ -762,6 +763,38 @@ See [Orchestrator & Hands-Free](../guides/ORCHESTRATOR.md).
|
|
|
762
763
|
|
|
763
764
|
---
|
|
764
765
|
|
|
766
|
+
## `workflow`
|
|
767
|
+
|
|
768
|
+
Declarative workflows: a static `workflow.yaml` step DAG (agent | command |
|
|
769
|
+
gate | loop) executed on the blackboard orchestrator — role-specialized steps
|
|
770
|
+
in fresh contexts talking through files, deterministic command verdicts,
|
|
771
|
+
approval gates, resume at the step boundary. `loop` steps run through a
|
|
772
|
+
caller-injected convergence loop (`uap deliver --team-solve` is the caller
|
|
773
|
+
that can; a standalone run of a loop-bearing workflow fails closed).
|
|
774
|
+
|
|
775
|
+
`uap workflow [list | show <name> | run <name> | status [runId] | resume <runId> | approve <runId> <step>]`
|
|
776
|
+
|
|
777
|
+
| Flag | Purpose |
|
|
778
|
+
|------|---------|
|
|
779
|
+
| `--input <dir>` | Task directory copied into the run (what the steps read) |
|
|
780
|
+
| `--model <preset>` | Model pin (ladder: step → workflow default → this → fleet default) |
|
|
781
|
+
| `--endpoint <url>` | Override the model endpoint (OpenAI-compatible `/v1`) |
|
|
782
|
+
| `--run-id <id>` | Name the run directory under `.uap/workflow-runs/` |
|
|
783
|
+
| `--mission <text>` | One-line objective assembled into every step context |
|
|
784
|
+
| `--dry-run` | Validate and print the execution plan; run nothing |
|
|
785
|
+
|
|
786
|
+
Exit codes: `0` success · `1` step failed · `2` blocked at a gate (resumable) ·
|
|
787
|
+
`3` invalid workflow.
|
|
788
|
+
|
|
789
|
+
```bash
|
|
790
|
+
uap workflow run team-solve --input examples/workflows/team-solve/sample
|
|
791
|
+
uap workflow show team-solve-deliver # the deliver-driven variant (loop steps)
|
|
792
|
+
```
|
|
793
|
+
|
|
794
|
+
See [Workflows](../guides/WORKFLOWS.md).
|
|
795
|
+
|
|
796
|
+
---
|
|
797
|
+
|
|
765
798
|
## `challenge`
|
|
766
799
|
|
|
767
800
|
Open multi-agent challenge: a shared goal, verified submissions, and a
|
|
@@ -104,6 +104,7 @@ Top level: `version`, `project`, `memory`, `worktree`, `costOptimization`,
|
|
|
104
104
|
|---|---|---|
|
|
105
105
|
| `deliver.orchestrate` | Blackboard orchestration for decomposed missions (`"on"`/`"off"`) | `"on"` (preflight seeds it) |
|
|
106
106
|
| `deliver.epics` | Epic controller outer loop (`"on"`/`"off"`) | `"on"` (preflight seeds it) |
|
|
107
|
+
| `deliver.teamSolve` | Team-solve posture — roles direct the convergence loop (`"on"`/`"off"`; unset = AUTO: high/critical complexity only) | unset (auto) |
|
|
107
108
|
| `deliver.parallelTasks` | Worktree-isolated parallel dispatch: independent READY orchestrator tasks run concurrently, each convergence loop + its gates in a detached git worktree seeded with the mission's current state; deltas merge back serially, and a merge conflict fails the task into the minimal-repair retry. Completion-triggered scheduling: a finished task's slot refills the moment it settles (no wave barrier — a fast sibling never idles behind a slow one), and a task whose dependencies all succeeded starts as soon as its slowest sibling-dependency settles. Settlement stays strictly sequential (one task at a time); which task settles next is timing-dependent. Clamped to 1–8. **Parallel is the default** — an unset key resolves to 4. Parallelism is conditional on worktree isolation: outside a git repo (or with an unborn HEAD) the run stays sequential and logs why. Set `1` (or `UAP_DELIVER_PARALLEL_TASKS=1`) to force sequential execution. | `4` (parallel, worktree-isolated) |
|
|
108
109
|
| `deliver.adversarialGate` | Adversarial red-team gate before a converged run is accepted (`false`/`0`/`"off"` disables; a positive integer retunes the round budget, cap 4). Overridden by `UAP_DELIVER_ADVERSARIAL_GATE`. | on (≤2 rounds) |
|
|
109
110
|
| `deliver.criteriaLint` | Plan-time lint of planner/split-emitted acceptance criteria; behavioral criteria gain an executable-evidence clause. Overridden by `UAP_DELIVER_CRITERIA_LINT`. | on |
|
|
@@ -189,6 +189,24 @@ you'd run by hand.
|
|
|
189
189
|
| Sandbox grants + egress boundary (`sandbox-grants.ts`, `sandbox.ts`) | Recorded authority for the bubblewrap sandbox: extra writable holes are per-session GRANTS (`uap sandbox grant <path> --session <id> --reason "..."`, TTL 8h default / 7d cap, revocable) from a consent ledger at the main checkout root, audit-logged per run — the legacy `UAP_SANDBOX_ALLOW` env still works but is named UNRECORDED in the audit. The out-of-checkout trust anchor (`sandbox-anchor.ts`) records the sanitized-ledger hash after every recorded mutation; a ledger changed outside a consent operation is detected on load — new grants refused, the session resolves ZERO grant-derived holes (fail-closed), and the grants listing stamps every row unverified. The audit JSONL is seq-stamped and bounded (past 12,000 lines it compacts to the 10,000 newest event lines plus the triggering event and one recorded `audit-compaction` marker — 10,002 total). `uap dashboard` (session view) surfaces the active grants per session with the anchor verdict and a revoke command per grant. `--net off` is the kernel egress boundary: own network namespace with nothing up (no TCP, no DNS, not even loopback) for untrusted code, evals, and data-only work. The proxy's optional `PROXY_SOCKET` (0600 AF_UNIX) is the permission-gated model choke point; `uap proxy connect` re-serves it on local TCP for AF_UNIX-less clients. |
|
|
190
190
|
| Integrity (`integrity.ts`) | Test-protection / integrity guard — stops the model from gutting the tests to "pass." |
|
|
191
191
|
|
|
192
|
+
## Workflows (`src/workflow/`) — Build (structured)
|
|
193
|
+
|
|
194
|
+
> **Stage: Build.** The declarative layer over the blackboard orchestrator: a
|
|
195
|
+
> workflow is DATA — a static `workflow.yaml` step DAG with role prompts,
|
|
196
|
+
> deterministic command steps, and approval gates — so the graph that five
|
|
197
|
+
> small models execute is authored once, reviewed, and re-run, instead of
|
|
198
|
+
> re-planned by a model every mission. The GVS5H/Memdoor team-solve shape:
|
|
199
|
+
> role-specialized steps cross-correct each other through files, and a
|
|
200
|
+
> command step's exit code — not a model's claim — is the verdict.
|
|
201
|
+
|
|
202
|
+
| Feature | Description |
|
|
203
|
+
|---------|-------------|
|
|
204
|
+
| Workflow loader (`loader.ts`) | Fail-closed front door: zod schema + graph checks (unique ids, known `requires:`, cycle rejection, kind/field confusion, parallel write collision, run-dir path escapes, reserved paths) before any model is paid. Search roots: project `.uap/workflows/` → `~/.uap/workflows/` → bundled `examples/workflows/`. |
|
|
205
|
+
| Workflow runner (`runner.ts`) | Four step kinds onto `orchestrate()` unchanged: `agent` (fresh single completion; `reads:` inlined into the prompt, the reply IS the file — the ledger discipline), `command` (exit-code verdict with `neverFails` evidence mode), `gate` (approval pause → `uap workflow approve`, exit 2, resumable), `loop` (a caller-injected convergence-loop seam — the engine assembles the instruction, the caller drives tools-and-gates iteration; `verifyOnly:` runs the gates with no executor turns). Run state persists after every accepted step; resume seeds `initialDone` so finished work is never redone. Exit contract 0/1/2/3 for CI/cron. |
|
|
206
|
+
| Workflow CLI (`cli/workflow.ts`) | `uap workflow list | show | run | status | resume | approve`. Steps name no model: the pin ladder (step → workflow `defaultModel` → `--model` → the local fleet default, `qwen38-27b` on the strata arm) answers, so one workflow file is portable across arms and endpoints. |
|
|
207
|
+
| Bundled team-solve (`examples/workflows/team-solve/`) | The flagship: plan → parallel ideate-a/ideate-b → solve → deterministic test → review → repair → verify, five roles, files only. The sample LIS task runs end-to-end on the local fleet; its provenance and caveats (the 9.8/10 LiveCodeBench Hard numbers are the authors', on OpenRouter) are documented in place. |
|
|
208
|
+
| Bundled team-solve-deliver (`examples/workflows/team-solve-deliver/`) | The combined engine: team roles directing the deliver convergence loop. plan → parallel ideations → solve (loop, the mission dispatch with the team's approach enriching the instruction) → fresh-context review → repair (loop, review-seeded) → verify (loop, gates-only verdict). `uap deliver --team-solve` injects its machinery as the seams and AUTO-engages it for high/critical-complexity missions (`--no-team-solve` / `UAP_DELIVER_TEAM_SOLVE=0` / `.uap.json` `deliver.teamSolve` control it; ADR 0014). |
|
|
209
|
+
|
|
192
210
|
## MCP Router (`src/mcp-router/`) — Cross-cutting (lean context)
|
|
193
211
|
|
|
194
212
|
> **Stage: Every station.** A stuffed context window slows and confuses the whole
|