foundry-testing-actor 0.1.0__py3-none-any.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- foundry_testing_actor/__init__.py +56 -0
- foundry_testing_actor/cards/actor-data.yaml +68 -0
- foundry_testing_actor/cards/actor-message.yaml +32 -0
- foundry_testing_actor/cards/actor-synchronous-messaging.yaml +59 -0
- foundry_testing_actor/cards/actor.yaml +19 -0
- foundry_testing_actor/cli.py +181 -0
- foundry_testing_actor/config.py +525 -0
- foundry_testing_actor/conformance.py +133 -0
- foundry_testing_actor/correlation.py +193 -0
- foundry_testing_actor/engine.py +889 -0
- foundry_testing_actor/grounding.py +206 -0
- foundry_testing_actor/handler.py +248 -0
- foundry_testing_actor/instance.py +129 -0
- foundry_testing_actor/runner/Dockerfile +32 -0
- foundry_testing_actor/schemas/agentic-context.schema.yaml +168 -0
- foundry_testing_actor/serve.py +129 -0
- foundry_testing_actor-0.1.0.dist-info/METADATA +258 -0
- foundry_testing_actor-0.1.0.dist-info/RECORD +20 -0
- foundry_testing_actor-0.1.0.dist-info/WHEEL +4 -0
- foundry_testing_actor-0.1.0.dist-info/entry_points.txt +2 -0
|
@@ -0,0 +1,206 @@
|
|
|
1
|
+
"""Grounding — putting the capability's own context in front of the session before turn one.
|
|
2
|
+
|
|
3
|
+
THE MECHANISM, AND WHY IT IS THIS ONE. `CLAUDE.md` at the working directory's root is loaded by
|
|
4
|
+
the `claude` CLI **before the first turn**, and its `@relative/path.md` imports are resolved
|
|
5
|
+
eagerly at the same moment. Verified live in this actor's exact invocation shape (`--print
|
|
6
|
+
--output-format stream-json --permission-mode acceptEdits`): a question only the fetched context
|
|
7
|
+
could answer came back in `num_turns: 1` with zero tool calls. Nothing was read, because nothing
|
|
8
|
+
needed to be — it was already in the window.
|
|
9
|
+
|
|
10
|
+
WHAT THIS REPLACES. The previous arrangement fetched the same envelopes into a tempdir **beside**
|
|
11
|
+
the clone and asked the session, in prose, to "read it before you start". That is not broken —
|
|
12
|
+
reads outside the cwd work and no permission denial was ever recorded — but it is *advisory*.
|
|
13
|
+
Whether the context entered the window at all was the model's choice, re-made every session, and
|
|
14
|
+
a session that skipped it looked exactly like one that had read it. Writing the envelopes inside
|
|
15
|
+
the clone and naming them from a generated `CLAUDE.md` turns grounding from a request into a
|
|
16
|
+
precondition.
|
|
17
|
+
|
|
18
|
+
WHY INSIDE THE CLONE, SPECIFICALLY. A `@`-import resolves relative to the file that contains it.
|
|
19
|
+
An envelope in a sibling tempdir can be *described* to a session but never imported by one, so
|
|
20
|
+
the sibling-tempdir arrangement could not have been fixed by writing a better prompt.
|
|
21
|
+
|
|
22
|
+
TWO TIERS, AND WHY THE CHOICE IS A MEASUREMENT. `load: eager` costs its full token weight on
|
|
23
|
+
every session, unconditionally, whether or not the task touches what it describes. `load:
|
|
24
|
+
on-demand` costs one line — its `answers:` and its path — and the session pays the rest only if
|
|
25
|
+
it opens the file. Neither is the right default in general; the right one for a given source is
|
|
26
|
+
whatever its measured envelope size says. One such read was already recorded as putting "40 kB of
|
|
27
|
+
JSON on screen", so this is not a hypothetical budget.
|
|
28
|
+
|
|
29
|
+
CONTAINMENT IS UNAFFECTED. The handler stages only `components[].tests`, so everything written
|
|
30
|
+
here — the `.foundry/` envelopes and the generated `CLAUDE.md` alike — is never staged, never
|
|
31
|
+
committed, and dies with the clone. No `.gitignore` entry is needed, and adding one would be a
|
|
32
|
+
second place to state a boundary that is already stated once.
|
|
33
|
+
|
|
34
|
+
BOTH DOORS GROUND IDENTICALLY, INTO THE TESTING CLONE. The session's working directory is always
|
|
35
|
+
this actor's own clone — `propose-acceptance` reads the implementation repo beside it, but the
|
|
36
|
+
standing context is the capability's, not either repo's, and a proposal needs exactly the context
|
|
37
|
+
the tests it proposes will later be written from.
|
|
38
|
+
|
|
39
|
+
MOVED FROM `foundry-implementation-actor` (ADR-FTA-0001), below the docstring changed only in the
|
|
40
|
+
marker's name and the word the boundary is stated with. The two are copies by decision, not a
|
|
41
|
+
shared kit: generic across capabilities, not across actor kinds.
|
|
42
|
+
"""
|
|
43
|
+
from __future__ import annotations
|
|
44
|
+
|
|
45
|
+
import json
|
|
46
|
+
import subprocess
|
|
47
|
+
from pathlib import Path
|
|
48
|
+
|
|
49
|
+
from papeete_actor_synchronous_messaging.engine import EngineError
|
|
50
|
+
|
|
51
|
+
from .config import CapabilityConfig, Grounding
|
|
52
|
+
|
|
53
|
+
CLAUDE_MD = "CLAUDE.md"
|
|
54
|
+
|
|
55
|
+
DEFAULT_FETCH_TIMEOUT_S = 120
|
|
56
|
+
|
|
57
|
+
# The marker that lets a generated block be told apart from prose a repo wrote for itself. It is
|
|
58
|
+
# not parsed — nothing here ever edits a previous block, because every clone is fresh — but a
|
|
59
|
+
# human reading a session's clone should be able to see instantly which half is machine-written.
|
|
60
|
+
_BEGIN = "<!-- foundry-testing-actor: generated grounding -->"
|
|
61
|
+
|
|
62
|
+
|
|
63
|
+
def ground(config: CapabilityConfig, clone_dir: Path, *,
|
|
64
|
+
timeout: int = DEFAULT_FETCH_TIMEOUT_S) -> list[Grounding]:
|
|
65
|
+
"""Fetch every `ground_in` source, write it into the clone, and render `CLAUDE.md`.
|
|
66
|
+
|
|
67
|
+
Returns the entries that were grounded, in declaration order. Raises `EngineError` if any
|
|
68
|
+
fetch fails — a session grounded in half its context is worse than one that never started,
|
|
69
|
+
because only the second is visible.
|
|
70
|
+
"""
|
|
71
|
+
for entry in config.ground_in:
|
|
72
|
+
envelope = fetch(config, entry, timeout=timeout)
|
|
73
|
+
write_envelope(config, entry, clone_dir, envelope)
|
|
74
|
+
render_claude_md(config, clone_dir)
|
|
75
|
+
return list(config.ground_in)
|
|
76
|
+
|
|
77
|
+
|
|
78
|
+
def fetch(config: CapabilityConfig, entry: Grounding, *,
|
|
79
|
+
timeout: int = DEFAULT_FETCH_TIMEOUT_S) -> str:
|
|
80
|
+
"""Run one entry's `fetch:` argv and return its stdout.
|
|
81
|
+
|
|
82
|
+
This package knows no knowledge tool by name. It knows "run this, ground the session in what
|
|
83
|
+
comes back" — which is why a fourth source is a YAML entry rather than a change here.
|
|
84
|
+
"""
|
|
85
|
+
argv = config.expand(entry.fetch)
|
|
86
|
+
try:
|
|
87
|
+
result = subprocess.run(argv, capture_output=True, text=True, timeout=timeout)
|
|
88
|
+
except FileNotFoundError as e:
|
|
89
|
+
raise EngineError(
|
|
90
|
+
f"grounding '{entry.name}': '{argv[0]}' is not on PATH. The sidecar names the tools "
|
|
91
|
+
f"this capability grounds itself in; installing them is the consuming image's job, "
|
|
92
|
+
f"not this package's."
|
|
93
|
+
) from e
|
|
94
|
+
except subprocess.TimeoutExpired as e:
|
|
95
|
+
raise EngineError(
|
|
96
|
+
f"grounding '{entry.name}': {' '.join(argv)} timed out after {timeout}s"
|
|
97
|
+
) from e
|
|
98
|
+
if result.returncode != 0:
|
|
99
|
+
raise EngineError(
|
|
100
|
+
f"grounding '{entry.name}': {' '.join(argv)} failed: {result.stderr.strip()}"
|
|
101
|
+
)
|
|
102
|
+
return result.stdout
|
|
103
|
+
|
|
104
|
+
|
|
105
|
+
def write_envelope(config: CapabilityConfig, entry: Grounding, clone_dir: Path,
|
|
106
|
+
envelope: str) -> Path:
|
|
107
|
+
"""Write one fetched envelope into the clone, as Markdown wrapping the raw payload.
|
|
108
|
+
|
|
109
|
+
Markdown rather than the bare `.json` the previous arrangement wrote, for one reason: a
|
|
110
|
+
`CLAUDE.md` `@`-import pulls in a file whole, so the file has to carry its own title and its
|
|
111
|
+
own "what this answers" line, or the session receives a wall of JSON with no idea which
|
|
112
|
+
question it settles. The payload itself is untouched inside the fence.
|
|
113
|
+
"""
|
|
114
|
+
destination = clone_dir / entry.into
|
|
115
|
+
destination.parent.mkdir(parents=True, exist_ok=True)
|
|
116
|
+
destination.write_text(
|
|
117
|
+
f"# {entry.name}\n\n"
|
|
118
|
+
f"{entry.answers}\n\n"
|
|
119
|
+
f"Fetched fresh for this session by `{' '.join(config.expand(entry.fetch))}`. Read it as "
|
|
120
|
+
f"given — it is this capability's own standing context, not something to re-derive.\n\n"
|
|
121
|
+
f"```json\n{_pretty(envelope)}\n```\n"
|
|
122
|
+
)
|
|
123
|
+
return destination
|
|
124
|
+
|
|
125
|
+
|
|
126
|
+
def render_claude_md(config: CapabilityConfig, clone_dir: Path) -> Path:
|
|
127
|
+
"""Write the clone's `CLAUDE.md`, or append to one the repo already commits.
|
|
128
|
+
|
|
129
|
+
APPEND, NEVER OVERWRITE. The capability repo has no `CLAUDE.md` of its own today, which is
|
|
130
|
+
exactly why this is safe to introduce now — there is nothing to clobber, so the clean-slate
|
|
131
|
+
case is the one being exercised. The day one is committed, it is the repo's own standing
|
|
132
|
+
guidance for anyone working in it, and silently replacing it with a generated block would
|
|
133
|
+
remove the very thing a session most needs to obey.
|
|
134
|
+
"""
|
|
135
|
+
body = _claude_md_body(config)
|
|
136
|
+
path = clone_dir / CLAUDE_MD
|
|
137
|
+
if path.exists():
|
|
138
|
+
existing = path.read_text().rstrip("\n")
|
|
139
|
+
path.write_text(f"{existing}\n\n---\n\n{body}")
|
|
140
|
+
else:
|
|
141
|
+
path.write_text(body)
|
|
142
|
+
return path
|
|
143
|
+
|
|
144
|
+
|
|
145
|
+
def _claude_md_body(config: CapabilityConfig) -> str:
|
|
146
|
+
eager = [g for g in config.ground_in if g.eager]
|
|
147
|
+
on_demand = [g for g in config.ground_in if not g.eager]
|
|
148
|
+
|
|
149
|
+
lines = [
|
|
150
|
+
_BEGIN,
|
|
151
|
+
"",
|
|
152
|
+
f"# {config.capability} — context for this session",
|
|
153
|
+
"",
|
|
154
|
+
"You are working inside a private clone made for one task. The context below was fetched "
|
|
155
|
+
"fresh for this session from this capability's own knowledge registry. It is authoritative: "
|
|
156
|
+
"read it as given rather than re-deriving what it already answers.",
|
|
157
|
+
"",
|
|
158
|
+
]
|
|
159
|
+
|
|
160
|
+
if eager:
|
|
161
|
+
lines += ["## Standing context", ""]
|
|
162
|
+
for entry in eager:
|
|
163
|
+
lines.append(f"{entry.answers}:")
|
|
164
|
+
lines.append("")
|
|
165
|
+
lines.append(f"@{entry.into}")
|
|
166
|
+
lines.append("")
|
|
167
|
+
|
|
168
|
+
if on_demand:
|
|
169
|
+
lines += [
|
|
170
|
+
"## Available on demand",
|
|
171
|
+
"",
|
|
172
|
+
"Not loaded. Open the file if the task needs what it answers.",
|
|
173
|
+
"",
|
|
174
|
+
]
|
|
175
|
+
for entry in on_demand:
|
|
176
|
+
lines.append(f"- **{entry.name}** — {entry.answers}. `{entry.into}`")
|
|
177
|
+
lines.append("")
|
|
178
|
+
|
|
179
|
+
lines += [
|
|
180
|
+
"## Boundaries",
|
|
181
|
+
"",
|
|
182
|
+
f"Write ONLY under {_join(config.writes_only_under)}. Nothing else in this clone is yours "
|
|
183
|
+
f"to change, and a staged path outside those roots is refused rather than committed.",
|
|
184
|
+
"",
|
|
185
|
+
"Do not `git add`, `git commit`, or `git push` — that is handled outside this session.",
|
|
186
|
+
"",
|
|
187
|
+
]
|
|
188
|
+
return "\n".join(lines) + "\n"
|
|
189
|
+
|
|
190
|
+
|
|
191
|
+
def _pretty(envelope: str) -> str:
|
|
192
|
+
"""Pretty-print a JSON envelope; pass anything else through untouched.
|
|
193
|
+
|
|
194
|
+
The fetch contract is "whatever the tool prints", not "JSON" — a source that emits Markdown or
|
|
195
|
+
plain text is equally groundable, and reformatting is a convenience for the JSON case, never a
|
|
196
|
+
requirement placed on the tool.
|
|
197
|
+
"""
|
|
198
|
+
text = envelope.strip()
|
|
199
|
+
try:
|
|
200
|
+
return json.dumps(json.loads(text), indent=2, ensure_ascii=False)
|
|
201
|
+
except (json.JSONDecodeError, ValueError):
|
|
202
|
+
return text
|
|
203
|
+
|
|
204
|
+
|
|
205
|
+
def _join(items) -> str:
|
|
206
|
+
return ", ".join(items)
|
|
@@ -0,0 +1,248 @@
|
|
|
1
|
+
"""`test_task` — the deterministic, auditable half of the `test-task` door.
|
|
2
|
+
|
|
3
|
+
`ClaudeCodeTesterEngine.judge()` embodies the creative decision: what black-box pytest to write or
|
|
4
|
+
extend, grounded in the caller's own payload, the agreed acceptance surface, and the capability's
|
|
5
|
+
own standing context. Everything here is mechanical and never trusted to the engine's own
|
|
6
|
+
judgement:
|
|
7
|
+
|
|
8
|
+
- **Containment.** `git add <tests root>` for each declared component — never `-A`, never a bare
|
|
9
|
+
`.` — then assert every `git diff --cached --name-only` path starts with one of those roots.
|
|
10
|
+
This check, not the session's own write-boundary instruction, is what actually enforces the
|
|
11
|
+
write boundary. A staged path outside it is refused rather than committed.
|
|
12
|
+
- **The correlation id.** `TASK-NNN` is threaded through the branch name (from the engine), the
|
|
13
|
+
commit message, and — via `correlation.py`, bound at the top of this door — every log record
|
|
14
|
+
this actor emits for the request, alongside the trace id its caller propagated.
|
|
15
|
+
- **Publishing.** For each component the commit actually touched, this actor builds a small
|
|
16
|
+
test-runner image in the cluster's shared buildkit and pushes it to the registry — the
|
|
17
|
+
component's FULL accumulated tests root as the build context (this task's increment plus every
|
|
18
|
+
prior task's), so the published image always runs the whole regression suite — named and
|
|
19
|
+
versioned by convention (`papeete_version.compute`). No Docker daemon is involved anywhere:
|
|
20
|
+
`buildctl` reaches `BUILDKIT_HOST`, which is why an actor running this can be an ordinary Pod.
|
|
21
|
+
- **Never opens a PR, never renders a verdict.** This door accepts, pushes, and publishes. Running
|
|
22
|
+
the image and judging the result belongs to the orchestrating actor alone.
|
|
23
|
+
|
|
24
|
+
THE BOUNDARY IS READ, NOT RESTATED. The hand-written actor this replaces kept `WRITES_ONLY_UNDER`
|
|
25
|
+
as a module constant here, in sync by hand with the sidecar's own declaration of the same fact. It
|
|
26
|
+
is now `config.writes_only_under`, derived from `components[].tests`. There is one source, and
|
|
27
|
+
this file is not it.
|
|
28
|
+
"""
|
|
29
|
+
from __future__ import annotations
|
|
30
|
+
|
|
31
|
+
import os
|
|
32
|
+
import shutil
|
|
33
|
+
import subprocess
|
|
34
|
+
from pathlib import Path
|
|
35
|
+
|
|
36
|
+
from papeete_version.version import compute as compute_version
|
|
37
|
+
from papeete_version.version import normalize_name
|
|
38
|
+
|
|
39
|
+
from . import correlation
|
|
40
|
+
from .config import CapabilityConfig, runner_path
|
|
41
|
+
|
|
42
|
+
|
|
43
|
+
class HandlerError(RuntimeError):
|
|
44
|
+
"""Raised for anything that stops this door short — including a containment violation.
|
|
45
|
+
|
|
46
|
+
`Actor.receive()` turns an uncaught exception from a handler into a `Refusal`
|
|
47
|
+
(`{self.name}'s own handler for '{offer.id}' failed: {e}`), which the HTTP binding replies as
|
|
48
|
+
400 — the same path an undeclared door or a schema violation already takes. A containment
|
|
49
|
+
breach is therefore refused, never silently swallowed or half-committed.
|
|
50
|
+
"""
|
|
51
|
+
|
|
52
|
+
|
|
53
|
+
def make_test_task(config: CapabilityConfig):
|
|
54
|
+
"""Bind one capability's config to the `test-task` handler.
|
|
55
|
+
|
|
56
|
+
A factory rather than a module-level function reading `actor.engines["claude-code"]`: the
|
|
57
|
+
engine's registered name comes from the sidecar, so looking the config up through a hardcoded
|
|
58
|
+
engine key would reintroduce exactly the literal this package exists to remove.
|
|
59
|
+
"""
|
|
60
|
+
|
|
61
|
+
def test_task(actor, payload: dict, from_: str, judged: dict | None = None) -> dict:
|
|
62
|
+
# Bound again here, not only in the engine's own `judge()`: a refusal path reaches this
|
|
63
|
+
# door with `judged=None`, having never entered the engine at all, and that refusal is
|
|
64
|
+
# exactly the record you want carrying a task id.
|
|
65
|
+
correlation.bind(correlation_id=correlation.correlation_id(),
|
|
66
|
+
task_id=payload.get("task_id"), door="test-task", caller=from_)
|
|
67
|
+
|
|
68
|
+
if judged is None or not judged.get("testable"):
|
|
69
|
+
reason = (judged or {}).get("reason", "not testable")
|
|
70
|
+
correlation.event("test-task-refused", because=reason)
|
|
71
|
+
return {"accepted": False, "because": reason}
|
|
72
|
+
|
|
73
|
+
task_id = payload["task_id"]
|
|
74
|
+
clone_dir = Path(judged["clone_dir"])
|
|
75
|
+
code_clone_dir = Path(judged["code_clone_dir"]) if judged.get("code_clone_dir") else None
|
|
76
|
+
branch = judged["branch"]
|
|
77
|
+
summary = judged.get("summary", "")
|
|
78
|
+
|
|
79
|
+
try:
|
|
80
|
+
with correlation.stage("containment-commit",
|
|
81
|
+
writes_only_under=list(config.writes_only_under)):
|
|
82
|
+
components = _containment_commit(config, clone_dir, task_id, summary)
|
|
83
|
+
correlation.event("components-touched", components=components)
|
|
84
|
+
with correlation.stage("push-branch", branch=branch, repo=config.source_repo):
|
|
85
|
+
_push(config, clone_dir, branch, _github_token(actor, config))
|
|
86
|
+
images = []
|
|
87
|
+
for component in components:
|
|
88
|
+
with correlation.stage("publish-test-image", component=component):
|
|
89
|
+
images.append(_publish_test_image(config, clone_dir, component, task_id))
|
|
90
|
+
correlation.event("test-image-published", component=component,
|
|
91
|
+
image=images[-1])
|
|
92
|
+
return {"accepted": True, "branch": branch, "images": images}
|
|
93
|
+
finally:
|
|
94
|
+
shutil.rmtree(clone_dir, ignore_errors=True)
|
|
95
|
+
if code_clone_dir is not None:
|
|
96
|
+
shutil.rmtree(code_clone_dir, ignore_errors=True)
|
|
97
|
+
|
|
98
|
+
return test_task
|
|
99
|
+
|
|
100
|
+
|
|
101
|
+
def _github_token(actor, config: CapabilityConfig) -> str:
|
|
102
|
+
engine = actor.engines.get(config.engine)
|
|
103
|
+
token = getattr(engine, "github_token", None) or os.environ.get("GITHUB_TOKEN")
|
|
104
|
+
if not token:
|
|
105
|
+
raise HandlerError("no GITHUB_TOKEN available (neither on the engine nor the environment)")
|
|
106
|
+
return token
|
|
107
|
+
|
|
108
|
+
|
|
109
|
+
def _redact(text: str, secret: str) -> str:
|
|
110
|
+
return text.replace(secret, "***")
|
|
111
|
+
|
|
112
|
+
|
|
113
|
+
def _run(clone_dir: Path, args: list[str], *, redact: str | None = None) -> str:
|
|
114
|
+
try:
|
|
115
|
+
result = subprocess.run(args, cwd=clone_dir, check=True, capture_output=True, text=True)
|
|
116
|
+
except subprocess.CalledProcessError as e:
|
|
117
|
+
stderr = _redact(e.stderr, redact) if redact else e.stderr
|
|
118
|
+
raise HandlerError(f"{' '.join(args[:2])} failed: {stderr}") from e
|
|
119
|
+
return result.stdout
|
|
120
|
+
|
|
121
|
+
|
|
122
|
+
def _containment_commit(config: CapabilityConfig, clone_dir: Path,
|
|
123
|
+
task_id: str, summary: str) -> list[str]:
|
|
124
|
+
boundary = config.writes_only_under
|
|
125
|
+
for path in boundary:
|
|
126
|
+
# `git add` errors hard (pathspec did not match any files) on a prefix that doesn't exist
|
|
127
|
+
# on disk AT ALL, not just one with zero changes — true the first time any task ever
|
|
128
|
+
# touches a given component's tests root, and verified live against exactly that.
|
|
129
|
+
if (clone_dir / path).exists():
|
|
130
|
+
_run(clone_dir, ["git", "add", path])
|
|
131
|
+
staged = [line for line in _run(clone_dir, ["git", "diff", "--cached", "--name-only"])
|
|
132
|
+
.splitlines() if line]
|
|
133
|
+
|
|
134
|
+
offending = [p for p in staged if not any(p.startswith(prefix) for prefix in boundary)]
|
|
135
|
+
if offending:
|
|
136
|
+
_run(clone_dir, ["git", "reset"])
|
|
137
|
+
raise HandlerError(
|
|
138
|
+
f"{task_id}: refusing to commit — staged path(s) outside "
|
|
139
|
+
f"{', '.join(boundary)}: {offending}"
|
|
140
|
+
)
|
|
141
|
+
if not staged:
|
|
142
|
+
raise HandlerError(
|
|
143
|
+
f"{task_id}: nothing staged under {', '.join(boundary)} — the session made no change "
|
|
144
|
+
f"there"
|
|
145
|
+
)
|
|
146
|
+
|
|
147
|
+
message = f"test({task_id}): tests written by {config.actor_name}\n\nTask: {task_id}"
|
|
148
|
+
if summary:
|
|
149
|
+
message += f"\n\n{summary}"
|
|
150
|
+
_run(clone_dir, [
|
|
151
|
+
"git", "-c", f"user.name={config.git_author_name}",
|
|
152
|
+
"-c", f"user.email={config.git_author_email}",
|
|
153
|
+
"commit", "-m", message,
|
|
154
|
+
])
|
|
155
|
+
|
|
156
|
+
# Which component a staged path belongs to is resolved by LONGEST declared tests root, not by
|
|
157
|
+
# taking the path's first segment. The first-segment shortcut is correct only while every
|
|
158
|
+
# root sits directly under a folder named after its component, and reports the wrong
|
|
159
|
+
# component — silently — the day one of them does not.
|
|
160
|
+
return config.components_for(staged)
|
|
161
|
+
|
|
162
|
+
|
|
163
|
+
def _push(config: CapabilityConfig, clone_dir: Path, branch: str, token: str) -> None:
|
|
164
|
+
# --force: `test/<task_id>` is rebuilt from the default branch on every attempt, so a retry's
|
|
165
|
+
# branch is a replacement for the last attempt's rather than a continuation of it.
|
|
166
|
+
push_url = f"https://x-access-token:{token}@github.com/{config.source_repo}.git"
|
|
167
|
+
_run(clone_dir, ["git", "push", "--force", push_url, f"HEAD:refs/heads/{branch}"],
|
|
168
|
+
redact=token)
|
|
169
|
+
|
|
170
|
+
|
|
171
|
+
def _image_registry() -> str:
|
|
172
|
+
registry = os.environ.get("IMAGE_REGISTRY")
|
|
173
|
+
if not registry:
|
|
174
|
+
raise HandlerError(
|
|
175
|
+
"no IMAGE_REGISTRY set — this actor publishes to a registry, and refuses to build an "
|
|
176
|
+
"image nothing else could ever pull"
|
|
177
|
+
)
|
|
178
|
+
return registry.rstrip("/")
|
|
179
|
+
|
|
180
|
+
|
|
181
|
+
def runner_dir(config: CapabilityConfig, clone_dir: Path, component: str) -> Path:
|
|
182
|
+
"""The directory `buildctl --local dockerfile=` is pointed at for one component.
|
|
183
|
+
|
|
184
|
+
A declared `runner:` is the testing repo's own, and lives in the clone. An undeclared one is
|
|
185
|
+
the runner this package ships (`runner_path()`), which lives beside this module on whatever
|
|
186
|
+
machine runs the actor — `buildctl` reads a `--local` directory client-side, so the two are
|
|
187
|
+
addressed identically and neither needs copying anywhere.
|
|
188
|
+
"""
|
|
189
|
+
declared = config.component(component)
|
|
190
|
+
if declared is None: # unreachable via `components_for`, cheap to hold
|
|
191
|
+
raise HandlerError(f"{component}: not a declared component of {config.capability}")
|
|
192
|
+
if declared.runner is None:
|
|
193
|
+
path = runner_path()
|
|
194
|
+
if not (path / "Dockerfile").is_file():
|
|
195
|
+
# A wheel that lost its runner. Said here rather than by `buildctl`, whose own error
|
|
196
|
+
# names a site-packages path an operator has no reason to connect to a release.
|
|
197
|
+
raise HandlerError(
|
|
198
|
+
f"{path}: the runner this package ships is missing — the build shipped without "
|
|
199
|
+
f"it. Report it against the release, or declare `runner:` for {component}."
|
|
200
|
+
)
|
|
201
|
+
return path
|
|
202
|
+
path = clone_dir / declared.runner
|
|
203
|
+
if not path.is_dir():
|
|
204
|
+
# Late failure here is expensive: the commit has already landed and the branch is already
|
|
205
|
+
# pushed. Saying which declared path is missing beats `buildctl`'s own error, which names
|
|
206
|
+
# a temp path the operator has no way to map back to the sidecar.
|
|
207
|
+
raise HandlerError(
|
|
208
|
+
f"{component}: declared runner directory '{declared.runner}' does not exist in the "
|
|
209
|
+
f"clone — the sidecar and the repo disagree"
|
|
210
|
+
)
|
|
211
|
+
return path
|
|
212
|
+
|
|
213
|
+
|
|
214
|
+
def _publish_test_image(config: CapabilityConfig, clone_dir: Path,
|
|
215
|
+
component: str, task_id: str) -> str:
|
|
216
|
+
"""Build this component's test-runner image in the cluster's shared buildkit and push it —
|
|
217
|
+
versioned by convention, never an explicitly-passed tag.
|
|
218
|
+
|
|
219
|
+
No Docker daemon and no docker socket: `buildctl` talks to the buildkitd Service named by
|
|
220
|
+
`BUILDKIT_HOST`. It is pushed because the orchestrating actor runs it as a Job in an ephemeral
|
|
221
|
+
namespace, which can only reach it through a registry.
|
|
222
|
+
|
|
223
|
+
THE CONTEXT IS THE WHOLE TESTS ROOT. The component's own persistent, accumulated suite — every
|
|
224
|
+
prior task's tests plus this one's — so the published image always runs the full regression
|
|
225
|
+
suite for that component, never one task's slice.
|
|
226
|
+
|
|
227
|
+
THE REF IS A CONTRACT WITH ACTORS THIS PACKAGE NEVER SEES. The orchestrating actor parses it
|
|
228
|
+
back apart, so it is `config.test_image_ref` output or nothing. Never invent a tag scheme here.
|
|
229
|
+
"""
|
|
230
|
+
declared = config.component(component)
|
|
231
|
+
dockerfile_dir = runner_dir(config, clone_dir, component)
|
|
232
|
+
context = declared.tests.rstrip("/")
|
|
233
|
+
|
|
234
|
+
version = compute_version(
|
|
235
|
+
folder=clone_dir / context,
|
|
236
|
+
name=config.test_image_name(component),
|
|
237
|
+
label="feature",
|
|
238
|
+
feature_name=normalize_name(task_id),
|
|
239
|
+
)
|
|
240
|
+
image = config.test_image_ref(_image_registry(), component, version)
|
|
241
|
+
_run(clone_dir, [
|
|
242
|
+
"buildctl", "build",
|
|
243
|
+
"--frontend", "dockerfile.v0",
|
|
244
|
+
"--local", f"context={context}",
|
|
245
|
+
"--local", f"dockerfile={dockerfile_dir}",
|
|
246
|
+
"--output", f"type=image,name={image},push=true",
|
|
247
|
+
])
|
|
248
|
+
return image
|
|
@@ -0,0 +1,129 @@
|
|
|
1
|
+
"""Rendering one use's cards from the definition this package ships.
|
|
2
|
+
|
|
3
|
+
WHAT A USE USED TO BE. A folder carrying its own copy of the actor's four cards, named for the
|
|
4
|
+
capability it serves, beside the sidecar that binds it to that capability. The copy was made by
|
|
5
|
+
hand, and `conformance.py` exists because a hand copy drifts: both folders pass `lint-card`
|
|
6
|
+
independently, neither gate has an opinion about the other, and the day the definition gains a
|
|
7
|
+
door every use in the world is silently wrong until someone edits four files in each of them.
|
|
8
|
+
|
|
9
|
+
WHAT A USE IS NOW. A folder carrying a sidecar. The cards are rendered into it — at `docker build`
|
|
10
|
+
time in the image this package ships, or by hand with `render-cards` anywhere else — from
|
|
11
|
+
`cards_path()`, which `cards_path()`'s own docstring always said was "what a spawned instance
|
|
12
|
+
would be rendered from". A gate that catches drift is strictly worse than a construction that
|
|
13
|
+
cannot drift. `conformance.check` stays for the uses that still carry a copy, and for a use
|
|
14
|
+
pinned to an older image; it just stops being the only thing between a stale card and a caller
|
|
15
|
+
refused at a door (ADR-FIA-0005, whose construction this package copies — ADR-FTA-0001).
|
|
16
|
+
|
|
17
|
+
WHAT IS RENDERED, AND WHAT IS COPIED. Three of the four cards name no capability and are copied
|
|
18
|
+
byte-for-byte under a banner — the data dictionary, the message catalog, and the doors are what
|
|
19
|
+
the actor IS, and a use that differed in any of them would be a different actor. Only
|
|
20
|
+
`actor.yaml` is rendered, because only identity differs per use: `name:` is the `actor_name` the
|
|
21
|
+
sidecar already derives, and the description says which capability this instance serves.
|
|
22
|
+
|
|
23
|
+
WHAT IS DELIBERATELY LOST. The hand-written prose. A use's `means:` used to name its real peers by
|
|
24
|
+
name; the rendered one says "an orchestrating actor", with no name, because the sidecar declares
|
|
25
|
+
no peers. `conformance.check` already refuses to compare prose for
|
|
26
|
+
exactly this reason, so nothing is checked that now fails — but a reader of `describe` gets less.
|
|
27
|
+
That is the trade, and it is written down in ADR-FIA-0005 rather than discovered.
|
|
28
|
+
"""
|
|
29
|
+
from __future__ import annotations
|
|
30
|
+
|
|
31
|
+
from pathlib import Path
|
|
32
|
+
|
|
33
|
+
import yaml
|
|
34
|
+
|
|
35
|
+
from .config import CapabilityConfig, cards_path, version
|
|
36
|
+
|
|
37
|
+
# The four, in the order `papeete_actor_synchronous_messaging.card` opens them. It opens exactly
|
|
38
|
+
# these and never globs, which is why a use missing one does not boot rather than booting smaller.
|
|
39
|
+
CARD_FILES = ("actor.yaml", "actor-data.yaml", "actor-message.yaml",
|
|
40
|
+
"actor-synchronous-messaging.yaml")
|
|
41
|
+
|
|
42
|
+
# Only identity differs per use. Everything else about the actor is the same actor.
|
|
43
|
+
_RENDERED = "actor.yaml"
|
|
44
|
+
|
|
45
|
+
|
|
46
|
+
def _banner(source: str = "") -> str:
|
|
47
|
+
"""The comment that stops someone editing a file which is overwritten on every build."""
|
|
48
|
+
body = (
|
|
49
|
+
f"RENDERED — do not edit. `foundry-testing-actor render-cards` wrote this from the "
|
|
50
|
+
f"actor's own definition, shipped in foundry-testing-actor=={version()}{source}. "
|
|
51
|
+
f"Change it by changing the definition and taking a new version of the package, not by "
|
|
52
|
+
f"editing here: the next `docker build` discards whatever is written in this file."
|
|
53
|
+
)
|
|
54
|
+
return _wrap(body, width=98, indent="# ")
|
|
55
|
+
|
|
56
|
+
|
|
57
|
+
def _render_manifest(config: CapabilityConfig, definition: Path) -> str:
|
|
58
|
+
"""`actor.yaml`, the only card that differs per use.
|
|
59
|
+
|
|
60
|
+
Emitted rather than substituted-into: the definition's own header says "THIS FOLDER IS THE
|
|
61
|
+
DEFINITION, NOT A USE", which would be false the moment it were copied into one. The
|
|
62
|
+
description is the definition's own, with the capability this instance serves appended — so
|
|
63
|
+
the sentence a reader gets still comes from the definition and cannot silently diverge from it.
|
|
64
|
+
"""
|
|
65
|
+
card = yaml.safe_load(definition.read_text())
|
|
66
|
+
described = " ".join(str(card["description"]).split())
|
|
67
|
+
return (
|
|
68
|
+
_banner(", named for the capability this use's sidecar declares")
|
|
69
|
+
+ f"manifest: {card['manifest']}\n"
|
|
70
|
+
+ f"name: {config.actor_name}\n"
|
|
71
|
+
+ "description: >-\n"
|
|
72
|
+
+ _wrap(f"{described} This instance serves {config.capability}, whose repository is "
|
|
73
|
+
f"{config.source_repo}.")
|
|
74
|
+
)
|
|
75
|
+
|
|
76
|
+
|
|
77
|
+
def _wrap(text: str, width: int = 96, indent: str = " ") -> str:
|
|
78
|
+
"""Fold one paragraph into a YAML block scalar body.
|
|
79
|
+
|
|
80
|
+
`textwrap` would do this, and would also collapse the sentence differently across Python
|
|
81
|
+
versions' defaults for `break_on_hyphens` — a capability id is full of dots and a repo name is
|
|
82
|
+
full of hyphens, and a card that reflows between two machines is a diff nobody asked for.
|
|
83
|
+
"""
|
|
84
|
+
lines: list[str] = []
|
|
85
|
+
current = ""
|
|
86
|
+
for word in text.split():
|
|
87
|
+
candidate = f"{current} {word}" if current else word
|
|
88
|
+
if current and len(indent) + len(candidate) > width:
|
|
89
|
+
lines.append(indent + current)
|
|
90
|
+
current = word
|
|
91
|
+
else:
|
|
92
|
+
current = candidate
|
|
93
|
+
if current:
|
|
94
|
+
lines.append(indent + current)
|
|
95
|
+
return "\n".join(lines) + "\n"
|
|
96
|
+
|
|
97
|
+
|
|
98
|
+
def render_cards(config: CapabilityConfig, dest: str | Path) -> list[Path]:
|
|
99
|
+
"""Write this actor's four cards into `dest`, named for the capability `config` serves.
|
|
100
|
+
|
|
101
|
+
Returns the paths written, in the order they were written. Overwrites: these files are build
|
|
102
|
+
output, and the banner on each one says so.
|
|
103
|
+
"""
|
|
104
|
+
definition = cards_path()
|
|
105
|
+
missing = [name for name in CARD_FILES if not (definition / name).exists()]
|
|
106
|
+
if missing:
|
|
107
|
+
# A wheel that lost its cards. `lint-card` in CI and in the release workflow exists to stop
|
|
108
|
+
# that reaching a registry; this turns the leftover case into a sentence rather than a
|
|
109
|
+
# traceback from inside a `docker build` someone is watching.
|
|
110
|
+
raise FileNotFoundError(
|
|
111
|
+
f"{definition}: this package's own cards are missing ({', '.join(missing)}), so there "
|
|
112
|
+
f"is nothing to render from. The build shipped without them — report it against the "
|
|
113
|
+
f"release."
|
|
114
|
+
)
|
|
115
|
+
|
|
116
|
+
out = Path(dest)
|
|
117
|
+
out.mkdir(parents=True, exist_ok=True)
|
|
118
|
+
written: list[Path] = []
|
|
119
|
+
for name in CARD_FILES:
|
|
120
|
+
target = out / name
|
|
121
|
+
if name == _RENDERED:
|
|
122
|
+
target.write_text(_render_manifest(config, definition / name))
|
|
123
|
+
else:
|
|
124
|
+
# Byte-for-byte under a banner. Not parsed and re-emitted: a round-trip through
|
|
125
|
+
# `yaml.safe_dump` would drop every comment in the definition, and those comments are
|
|
126
|
+
# where the reasoning for each door and each data item is written down.
|
|
127
|
+
target.write_text(_banner("") + definition.joinpath(name).read_text())
|
|
128
|
+
written.append(target)
|
|
129
|
+
return written
|
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
# The default test runner — the image this actor PUBLISHES for a component, never its own image.
|
|
2
|
+
#
|
|
3
|
+
# SHIPPED IN THE WHEEL, NOT IN EVERY TESTING REPO. The hand-written actor this package replaces kept
|
|
4
|
+
# this file in its own repository, where it named nothing about the capability: a Python base, a
|
|
5
|
+
# pytest, and a CMD. It is the contract between this actor and the orchestrating actor that runs
|
|
6
|
+
# what it publishes, so it lives with the actor. A component whose sidecar entry declares no
|
|
7
|
+
# `runner:` is built with this one; `buildctl --local dockerfile=` reads it straight out of the
|
|
8
|
+
# installed package (see handler.py's `runner_dir`).
|
|
9
|
+
#
|
|
10
|
+
# Static and task-agnostic on purpose: `_publish_test_image` builds this same Dockerfile for every
|
|
11
|
+
# task, pointing its build CONTEXT at that component's own persistent, accumulated tests root —
|
|
12
|
+
# buildctl build --local context=<component tests root> --local dockerfile=<this folder>
|
|
13
|
+
# — every prior task's tests plus this one's, so the published image always runs the full
|
|
14
|
+
# regression suite. Nothing here ever needs to know a task id or a capability.
|
|
15
|
+
#
|
|
16
|
+
# Run by the orchestrating actor's own Job: the generated tests read their target component's base
|
|
17
|
+
# URL from an environment variable named <COMPONENT>_URL (uppercase, e.g. BACKEND_URL), or whichever
|
|
18
|
+
# variable the agreed acceptance surface named — set by that Job to the in-namespace service
|
|
19
|
+
# address, never baked into this image.
|
|
20
|
+
#
|
|
21
|
+
# WHAT IT CARRIES, AND WHEN TO DECLARE YOUR OWN. pytest to run the suite; requests for HTTP
|
|
22
|
+
# handles; aio-pika for event handles on an AMQP bus. A capability whose tests need anything else
|
|
23
|
+
# declares `runner:` on that component and owns the file — keeping its CMD a pytest run, because
|
|
24
|
+
# that is what the orchestrating actor reads a verdict from.
|
|
25
|
+
FROM python:3.12-slim
|
|
26
|
+
|
|
27
|
+
RUN pip install --no-cache-dir "pytest==8.*" "requests==2.*" "aio-pika==9.*"
|
|
28
|
+
|
|
29
|
+
WORKDIR /tests
|
|
30
|
+
COPY . /tests
|
|
31
|
+
|
|
32
|
+
CMD ["pytest", "-v", "--tb=line"]
|