@hydraharness/harness-code-runtime-python 0.1.1-rc.10
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +27 -0
- package/lib/index.js +413 -0
- package/lib/invariant.js +24 -0
- package/lib/types/index.d.ts +12 -0
- package/lib/types/invariant.d.ts +16 -0
- package/lib/types/protocol.d.ts +329 -0
- package/package.json +42 -0
- package/py/protocol.py +138 -0
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 DeepSeek
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
ADDED
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
# @hydraharness/harness-code-runtime-python
|
|
2
|
+
|
|
3
|
+
CPython-subprocess implementation of the [`@hydraharness/harness-code-runtime`](../code-runtime/README.md) seam. Companion to [`@hydraharness/harness-code-runtime-worker-thread`](../code-runtime-worker-thread/README.md); trades the Node worker thread for a fresh `python3` subprocess so model code is Python instead of TypeScript.
|
|
4
|
+
|
|
5
|
+
The package owns the wire protocol for that seam: the host-side frame codec and the Python-side mirror of the same message vocabulary.
|
|
6
|
+
|
|
7
|
+
## Wire protocol
|
|
8
|
+
|
|
9
|
+
The host and the CPython subprocess exchange a versionless, JSON-lines protocol on the child's fd 3 — one JSON object per line, leaving stdout/stderr free for the program's own output. `src/protocol.ts` is the host side; `py/protocol.py` mirrors its message shapes and the shared truncation-marker text on the Python side.
|
|
10
|
+
|
|
11
|
+
- **fd 3, not stdout** — Node pins the channel positionally with `stdio: ['pipe','pipe','pipe','pipe']`; the Python bootstrap reads the same `PROTOCOL_FD` constant. JSON-lines framing.
|
|
12
|
+
- **Host treats every inbound frame as hostile** — model code has full access to fd 3 and can post anything through it, so `validateChildFrame` shape-validates and REBUILDS each frame before the host reads it: forged extra fields never ride along, a non-number call id can never be echoed into a reply, and junk drops to `undefined` rather than throwing in the host's message handler. The Python side trusts host replies (the host is not model-controlled).
|
|
13
|
+
- **Lossless-JSON crossing** — completion values and binding arguments cross as exact JSON. `encodeJsonPlain` serializes a `JSON.parse`-produced value without recursion, so a deep value below the byte budget crosses intact instead of dying on `JSON.stringify`'s stack limit; `checkDoneValue` meters a forged completion value's byte length AND number losslessness in one traversal that rejects an over-budget payload before the incremental work it would add (the enqueued children; strings and keys are metered by a non-allocating escaped-size scan, so the escaped copy is never materialized) — the frame's own width is already parsed and capped upstream by the host's fd-3 receive buffer, not re-bounded here; `hasUnsafeIntegerToken` reads the raw frame text to catch an integer token that `JSON.parse` would silently round; `hasNonLosslessNumber` rejects a non-finite or negative-zero number in unbounded `call.args`. Beyond-safe-range integral doubles serialize through `BigInt` digits so the exact integer crosses, not the rounded `String()` form.
|
|
14
|
+
- **Shared truncation marker** — `logTruncationMarker(maxBytes)` produces byte-identical text on both sides, so a truncated log run reads the same however the cap was hit. The `log` frame's `truncated` flag distinguishes the child ledger's own marker from program output.
|
|
15
|
+
|
|
16
|
+
## Model Experience
|
|
17
|
+
|
|
18
|
+
Indirectly, through Code Mode in [`@hydraharness/harness-tools`](../../core/tools/README.md), which renders this backend's exact completion value when it fits (or an explicit `invalid-output` / `output-limit` failure), plus the exact `[@hydraharness/harness-code-runtime-python] log capture truncated at <maxLogBytes> bytes` log marker, into a retained `run_code` result.
|
|
19
|
+
|
|
20
|
+
#### KV Cache effect
|
|
21
|
+
|
|
22
|
+
No direct invalidation; the named consumer owns any request-prefix changes.
|
|
23
|
+
|
|
24
|
+
## Known Limitations and Deferred Work
|
|
25
|
+
|
|
26
|
+
- **The cross-language guard covers the runtime-executed surfaces and the frame field shapes** — `tests/protocol-mirror.e2e.ts` spawns a real `python3` and asserts, against `src/protocol.ts`, both `PROTOCOL_FD` / the log truncation marker text AND each `TypedDict`'s required/optional wire field set in `py/protocol.py`. What it does not compare is the field *types* (e.g. that `cpuSeconds` is an `int` on both sides): comparing type declarations across TypeScript and Python has no mechanical equivalent here, so a type-level drift is still caught by review plus the backend's real-subprocess suite rather than this package's tests.
|
|
27
|
+
- **`src/index.ts` exports the protocol vocabulary only** — the package carries no subprocess execution path and no Python-side JSON codec, so nothing here spawns `python3` outside the mirror test.
|
package/lib/index.js
ADDED
|
@@ -0,0 +1,413 @@
|
|
|
1
|
+
Object.fromEntries(Object.entries({
|
|
2
|
+
BootMessage: {
|
|
3
|
+
type: "required",
|
|
4
|
+
cpuSeconds: "required",
|
|
5
|
+
addressSpaceBytes: "required",
|
|
6
|
+
maxLogBytes: "required",
|
|
7
|
+
maxValueBytes: "required",
|
|
8
|
+
namespaces: "required"
|
|
9
|
+
},
|
|
10
|
+
Namespace: {
|
|
11
|
+
global: "required",
|
|
12
|
+
names: "required",
|
|
13
|
+
errorClass: "optional"
|
|
14
|
+
},
|
|
15
|
+
RunMessage: {
|
|
16
|
+
type: "required",
|
|
17
|
+
program: "required"
|
|
18
|
+
},
|
|
19
|
+
BootAckMessage: { type: "required" },
|
|
20
|
+
CallMessage: {
|
|
21
|
+
type: "required",
|
|
22
|
+
id: "required",
|
|
23
|
+
global: "required",
|
|
24
|
+
name: "required",
|
|
25
|
+
args: "required"
|
|
26
|
+
},
|
|
27
|
+
LogMessage: {
|
|
28
|
+
type: "required",
|
|
29
|
+
text: "required",
|
|
30
|
+
truncated: "optional"
|
|
31
|
+
},
|
|
32
|
+
DoneErrorField: {
|
|
33
|
+
kind: "required",
|
|
34
|
+
message: "required"
|
|
35
|
+
},
|
|
36
|
+
DoneMessage: {
|
|
37
|
+
type: "required",
|
|
38
|
+
value: "optional",
|
|
39
|
+
error: "optional"
|
|
40
|
+
},
|
|
41
|
+
ErrorClass: {
|
|
42
|
+
name: "required",
|
|
43
|
+
memberNameProperty: "required"
|
|
44
|
+
},
|
|
45
|
+
ReplyOk: {
|
|
46
|
+
type: "required",
|
|
47
|
+
id: "required",
|
|
48
|
+
ok: "required",
|
|
49
|
+
value: "required"
|
|
50
|
+
},
|
|
51
|
+
ReplyErr: {
|
|
52
|
+
type: "required",
|
|
53
|
+
id: "required",
|
|
54
|
+
ok: "required",
|
|
55
|
+
message: "required"
|
|
56
|
+
}
|
|
57
|
+
}).map(([frame, roles]) => {
|
|
58
|
+
return [frame, {
|
|
59
|
+
required: Object.keys(roles).filter((key) => roles[key] === "required").sort(),
|
|
60
|
+
optional: Object.keys(roles).filter((key) => roles[key] === "optional").sort()
|
|
61
|
+
}];
|
|
62
|
+
}));
|
|
63
|
+
/**
|
|
64
|
+
* The in-band marker text announcing that log capture stopped at the byte
|
|
65
|
+
* budget. Shared wire vocabulary: the Python-side LogBuffer emits it when ITS
|
|
66
|
+
* ledger exhausts, and the host emits identical text when its own ledger drops
|
|
67
|
+
* a frame first (forged fd-3 traffic, stray stdout bytes) — a truncated run
|
|
68
|
+
* reads the same however the cap was hit.
|
|
69
|
+
* @param maxBytes - the configured `maxLogBytes` the marker names.
|
|
70
|
+
* @returns the marker line.
|
|
71
|
+
*/
|
|
72
|
+
function logTruncationMarker(maxBytes) {
|
|
73
|
+
return `[hydra-code-runtime-python] log capture truncated at ${maxBytes} bytes`;
|
|
74
|
+
}
|
|
75
|
+
/**
|
|
76
|
+
* Serialize one JSON-parse-produced value without recursion. `JSON.stringify`
|
|
77
|
+
* recurses per nesting level and throws `RangeError` a few thousand levels
|
|
78
|
+
* deep, but the seam's `CodeJsonValue` has no depth limit — an honest deep
|
|
79
|
+
* completion or binding resolution below the byte budget must cross intact
|
|
80
|
+
* (the worker backend's wire is equally stack-safe). Callers must pass a value
|
|
81
|
+
* produced by `JSON.parse` (or equally JSON-plain): only `null`, finite
|
|
82
|
+
* numbers, booleans, strings, dense arrays, and plain objects — this encoder
|
|
83
|
+
* validates nothing. Output matches compact `JSON.stringify` byte for byte
|
|
84
|
+
* EXCEPT on an integral double beyond the safe range, where {@link scalarJson}
|
|
85
|
+
* emits the exact integer's BigInt digits rather than `JSON.stringify`'s rounded
|
|
86
|
+
* spelling (`1152921504606846976`, not `...847000`) so the seam's lossless-JSON
|
|
87
|
+
* promise holds across the wire.
|
|
88
|
+
* @param value - a JSON-plain value (e.g. straight from `JSON.parse`).
|
|
89
|
+
* @returns the compact JSON encoding.
|
|
90
|
+
*/
|
|
91
|
+
function encodeJsonPlain(value) {
|
|
92
|
+
const chunks = [];
|
|
93
|
+
const tasks = [{ value }];
|
|
94
|
+
for (let task = tasks.pop(); task !== void 0; task = tasks.pop()) {
|
|
95
|
+
if ("text" in task) {
|
|
96
|
+
chunks.push(task.text);
|
|
97
|
+
continue;
|
|
98
|
+
}
|
|
99
|
+
const current = task.value;
|
|
100
|
+
if (typeof current === "string") chunks.push(JSON.stringify(current));
|
|
101
|
+
else if (Array.isArray(current)) {
|
|
102
|
+
chunks.push("[");
|
|
103
|
+
tasks.push({ text: "]" });
|
|
104
|
+
for (let index = current.length - 1; index >= 0; index--) {
|
|
105
|
+
if (index < current.length - 1) tasks.push({ text: "," });
|
|
106
|
+
tasks.push({ value: current[index] });
|
|
107
|
+
}
|
|
108
|
+
} else if (typeof current === "object" && current !== null) {
|
|
109
|
+
const record = current;
|
|
110
|
+
chunks.push("{");
|
|
111
|
+
tasks.push({ text: "}" });
|
|
112
|
+
const keys = Object.keys(record);
|
|
113
|
+
for (let index = keys.length - 1; index >= 0; index--) {
|
|
114
|
+
const key = keys[index];
|
|
115
|
+
if (index < keys.length - 1) tasks.push({ text: "," });
|
|
116
|
+
tasks.push({ value: record[key] });
|
|
117
|
+
tasks.push({ text: `${JSON.stringify(key)}:` });
|
|
118
|
+
}
|
|
119
|
+
} else chunks.push(scalarJson(current));
|
|
120
|
+
}
|
|
121
|
+
return chunks.join("");
|
|
122
|
+
}
|
|
123
|
+
/**
|
|
124
|
+
* One scalar (null, boolean, finite number) as JSON text. A beyond-safe-range
|
|
125
|
+
* integral double needs BigInt digits: `String(2 ** 60)` emits the ROUNDED
|
|
126
|
+
* `...847000` form, and echoing that to the child would silently change the
|
|
127
|
+
* integer the seam promised to carry losslessly — `BigInt(2 ** 60)` prints the
|
|
128
|
+
* exact `...846976` the double actually holds.
|
|
129
|
+
* @param current - a JSON-plain scalar (JSON.parse emits nothing else).
|
|
130
|
+
* @returns its JSON encoding.
|
|
131
|
+
*/
|
|
132
|
+
function scalarJson(current) {
|
|
133
|
+
if (typeof current === "number" && Number.isInteger(current) && !Number.isSafeInteger(current)) return BigInt(current).toString();
|
|
134
|
+
return String(current);
|
|
135
|
+
}
|
|
136
|
+
/**
|
|
137
|
+
* Exact UTF-8 byte length of one string's compact JSON form (quotes + escapes),
|
|
138
|
+
* computed by a single non-allocating scan that stops the instant the running
|
|
139
|
+
* total exceeds `maxBytes`. Used instead of `Buffer.byteLength(JSON.stringify(s))`
|
|
140
|
+
* so a control-heavy forged string — whose escaped copy expands up to ~6x — is
|
|
141
|
+
* rejected BEFORE that copy is materialized: `JSON.stringify` would allocate the
|
|
142
|
+
* full escaped form first, the very hundreds-of-MB spike the metered traversal
|
|
143
|
+
* exists to avoid. Mirrors `JSON.stringify`'s escaping byte-for-byte: `"` and
|
|
144
|
+
* `\` and the five short C0 escapes cost 2, other C0 controls `\uXXXX` cost 6, a
|
|
145
|
+
* valid surrogate pair is one astral code point emitted as raw 4-byte UTF-8, a
|
|
146
|
+
* LONE surrogate becomes `\uXXXX` at 6, and any other code point costs its raw
|
|
147
|
+
* UTF-8 width.
|
|
148
|
+
* @param text - the string to meter.
|
|
149
|
+
* @param maxBytes - largest serialized size the caller can still admit.
|
|
150
|
+
* @returns the exact serialized byte length, or `undefined` once it exceeds `maxBytes`.
|
|
151
|
+
*/
|
|
152
|
+
function jsonStringBytesUpTo(text, maxBytes) {
|
|
153
|
+
let bytes = 2;
|
|
154
|
+
if (bytes > maxBytes) return void 0;
|
|
155
|
+
for (let index = 0; index < text.length; index++) {
|
|
156
|
+
const code = text.charCodeAt(index);
|
|
157
|
+
if (code === 34 || code === 92 || code === 8 || code === 9 || code === 10 || code === 12 || code === 13) bytes += 2;
|
|
158
|
+
else if (code < 32) bytes += 6;
|
|
159
|
+
else if (code < 128) bytes += 1;
|
|
160
|
+
else if (code < 2048) bytes += 2;
|
|
161
|
+
else if (code >= 55296 && code <= 56319 && index + 1 < text.length) {
|
|
162
|
+
const next = text.charCodeAt(index + 1);
|
|
163
|
+
if (next >= 56320 && next <= 57343) {
|
|
164
|
+
bytes += 4;
|
|
165
|
+
index++;
|
|
166
|
+
} else bytes += 6;
|
|
167
|
+
} else if (code >= 55296 && code <= 57343) bytes += 6;
|
|
168
|
+
else bytes += 3;
|
|
169
|
+
if (bytes > maxBytes) return void 0;
|
|
170
|
+
}
|
|
171
|
+
return bytes;
|
|
172
|
+
}
|
|
173
|
+
/**
|
|
174
|
+
* Meter a `JSON.parse`-produced done value's compact-JSON byte length AND its
|
|
175
|
+
* number losslessness in one traversal, stopping the instant `maxBytes` is
|
|
176
|
+
* crossed. This bounds the INCREMENTAL allocation the check itself would add on
|
|
177
|
+
* top of the already-parsed value — the enqueued children; strings and keys are
|
|
178
|
+
* metered by {@link jsonStringBytesUpTo} without allocating an escaped copy —
|
|
179
|
+
* not the parse that produced `value`.
|
|
180
|
+
* That upstream width is bounded separately, by the host-side cap on inbound
|
|
181
|
+
* fd-3 frame size before `JSON.parse` runs (owned by the runtime that reads the
|
|
182
|
+
* channel), so `value` cannot be arbitrarily large when it reaches here. The
|
|
183
|
+
* budget is the `maxValueBytes` the boot frame carries — a required wire field
|
|
184
|
+
* with no default at this layer. The traversal rejects over-budget BEFORE
|
|
185
|
+
* materializing a string's escaped form or enqueuing an array's/object's
|
|
186
|
+
* children, so a forgery within that frame cap cannot force those secondary
|
|
187
|
+
* allocations. Object key COUNTING is
|
|
188
|
+
* unavoidably O(keys) — JS has no lazy own-key iterator, and the parse already
|
|
189
|
+
* built the key set — but the check still refuses the per-entry work before the
|
|
190
|
+
* enqueue loop. A non-lossless number (non-finite, negative zero) is caught only
|
|
191
|
+
* when the value fits the budget — an over-budget value is rejected regardless,
|
|
192
|
+
* so the distinction is moot. Same JSON-plain precondition and traversal shape
|
|
193
|
+
* as {@link encodeJsonPlain}; a number's byte length is measured through
|
|
194
|
+
* {@link scalarJson} (matching the encoder, so a beyond-safe-range integer
|
|
195
|
+
* meters its exact BigInt digits, not `JSON.stringify`'s rounded spelling) and
|
|
196
|
+
* a string's/key's through {@link jsonStringBytesUpTo} (the exact escaped size,
|
|
197
|
+
* scanned without allocating the escaped copy).
|
|
198
|
+
* @param value - a JSON-plain value (e.g. straight from `JSON.parse`).
|
|
199
|
+
* @param maxBytes - the completion-value budget in bytes.
|
|
200
|
+
* @returns `{ ok: true, bytes }` with the exact serialized size, or
|
|
201
|
+
* `{ ok: false, reason }` — `over-budget` once the size exceeds `maxBytes`,
|
|
202
|
+
* `non-lossless` on a non-finite or negative-zero number.
|
|
203
|
+
*/
|
|
204
|
+
function checkDoneValue(value, maxBytes) {
|
|
205
|
+
let bytes = 0;
|
|
206
|
+
let nonLossless = false;
|
|
207
|
+
const stack = [value];
|
|
208
|
+
while (stack.length > 0) {
|
|
209
|
+
const current = stack.pop();
|
|
210
|
+
if (typeof current === "number") {
|
|
211
|
+
if (!Number.isFinite(current) || Object.is(current, -0)) nonLossless = true;
|
|
212
|
+
bytes += Buffer.byteLength(scalarJson(current), "utf8");
|
|
213
|
+
} else if (typeof current === "string") {
|
|
214
|
+
const stringBytes = jsonStringBytesUpTo(current, maxBytes - bytes);
|
|
215
|
+
if (stringBytes === void 0) return {
|
|
216
|
+
ok: false,
|
|
217
|
+
reason: "over-budget"
|
|
218
|
+
};
|
|
219
|
+
bytes += stringBytes;
|
|
220
|
+
} else if (Array.isArray(current)) {
|
|
221
|
+
bytes += 2 + (current.length > 1 ? current.length - 1 : 0);
|
|
222
|
+
if (bytes + current.length > maxBytes) return {
|
|
223
|
+
ok: false,
|
|
224
|
+
reason: "over-budget"
|
|
225
|
+
};
|
|
226
|
+
for (const item of current) stack.push(item);
|
|
227
|
+
} else if (typeof current === "object" && current !== null) {
|
|
228
|
+
const record = current;
|
|
229
|
+
let count = 0;
|
|
230
|
+
for (const key in record) if (Object.hasOwn(record, key)) count += 1;
|
|
231
|
+
bytes += 2 + (count > 1 ? count - 1 : 0);
|
|
232
|
+
if (bytes + count * 4 > maxBytes) return {
|
|
233
|
+
ok: false,
|
|
234
|
+
reason: "over-budget"
|
|
235
|
+
};
|
|
236
|
+
for (const key in record) {
|
|
237
|
+
if (!Object.hasOwn(record, key)) continue;
|
|
238
|
+
const keyBytes = jsonStringBytesUpTo(key, maxBytes - bytes);
|
|
239
|
+
if (keyBytes === void 0) return {
|
|
240
|
+
ok: false,
|
|
241
|
+
reason: "over-budget"
|
|
242
|
+
};
|
|
243
|
+
bytes += keyBytes + 1;
|
|
244
|
+
stack.push(record[key]);
|
|
245
|
+
}
|
|
246
|
+
} else bytes += Buffer.byteLength(scalarJson(current), "utf8");
|
|
247
|
+
if (bytes > maxBytes) return {
|
|
248
|
+
ok: false,
|
|
249
|
+
reason: "over-budget"
|
|
250
|
+
};
|
|
251
|
+
}
|
|
252
|
+
if (nonLossless) return {
|
|
253
|
+
ok: false,
|
|
254
|
+
reason: "non-lossless"
|
|
255
|
+
};
|
|
256
|
+
return {
|
|
257
|
+
ok: true,
|
|
258
|
+
bytes
|
|
259
|
+
};
|
|
260
|
+
}
|
|
261
|
+
/**
|
|
262
|
+
* Whether a raw JSON line contains an integer token that would lose precision
|
|
263
|
+
* as a JavaScript number. `JSON.parse` silently rounds such a token
|
|
264
|
+
* (`9007199254740993` becomes `...992`) BEFORE any validation can see it, so
|
|
265
|
+
* the check must read the source text; a beyond-safe-range token whose double
|
|
266
|
+
* parse round-trips exactly (`2**53`, `2**60`) is lossless and passes. The scan walks the line skipping string literals (a digit run
|
|
267
|
+
* inside a string is data, not a number token) and tests every number token
|
|
268
|
+
* in plain integer form — no fraction or exponent, which parse as doubles by
|
|
269
|
+
* intent. A reviver cannot do this job: the reviver walk recurses per nesting
|
|
270
|
+
* level and would reintroduce the depth limit `encodeJsonPlain` removes.
|
|
271
|
+
* @param line - the raw UTF-8 text of one JSON-lines frame.
|
|
272
|
+
* @returns true when an unsafe integer token is present outside strings.
|
|
273
|
+
*/
|
|
274
|
+
function hasUnsafeIntegerToken(line) {
|
|
275
|
+
for (let index = 0; index < line.length; index++) {
|
|
276
|
+
const char = line[index];
|
|
277
|
+
if (char === "\"") {
|
|
278
|
+
for (index++; index < line.length; index++) if (line[index] === "\\") index++;
|
|
279
|
+
else if (line[index] === "\"") break;
|
|
280
|
+
continue;
|
|
281
|
+
}
|
|
282
|
+
if (char === "-" || char !== void 0 && char >= "0" && char <= "9") {
|
|
283
|
+
let end = index + 1;
|
|
284
|
+
while (end < line.length) {
|
|
285
|
+
const c = line[end];
|
|
286
|
+
if (c >= "0" && c <= "9" || c === "." || c === "e" || c === "E" || c === "+" || c === "-") end++;
|
|
287
|
+
else break;
|
|
288
|
+
}
|
|
289
|
+
const token = line.slice(index, end);
|
|
290
|
+
if (/^-?\d+$/.test(token)) {
|
|
291
|
+
const parsed = Number(token);
|
|
292
|
+
if (!Number.isFinite(parsed)) return true;
|
|
293
|
+
if (!Number.isSafeInteger(parsed) && BigInt(token) !== BigInt(parsed)) return true;
|
|
294
|
+
}
|
|
295
|
+
index = end - 1;
|
|
296
|
+
}
|
|
297
|
+
}
|
|
298
|
+
return false;
|
|
299
|
+
}
|
|
300
|
+
/**
|
|
301
|
+
* Lazily yield one plain object's own enumerable property values. A generator
|
|
302
|
+
* (not `Object.values`/`Object.entries`) because {@link hasNonLosslessNumber}
|
|
303
|
+
* walks breadth it cannot bound: those helpers copy the whole VALUE (or
|
|
304
|
+
* key/value pair) list into a fresh array up front, so a wide object would cost
|
|
305
|
+
* that second full-breadth allocation before a single value is examined. The
|
|
306
|
+
* `for...in` here does not make the walk sublinear — V8 still materializes the
|
|
307
|
+
* key-name enumeration when the loop starts — but it avoids the extra value
|
|
308
|
+
* array, yielding each value straight off the already-parsed object.
|
|
309
|
+
* @param record - a JSON-parse-produced object.
|
|
310
|
+
* @yields each own enumerable property value, in key order.
|
|
311
|
+
*/
|
|
312
|
+
function* ownValues(record) {
|
|
313
|
+
for (const key in record) if (Object.hasOwn(record, key)) yield record[key];
|
|
314
|
+
}
|
|
315
|
+
/**
|
|
316
|
+
* Whether a JSON.parse-produced value contains a number outside lossless
|
|
317
|
+
* JSON: non-finite (`1e400` parses to `Infinity`) or negative zero (`-0.0`
|
|
318
|
+
* parses to JS `-0`, whose sign bit a re-serialization drops). The honest
|
|
319
|
+
* child's validator rejects these before sending, so a frame carrying one is
|
|
320
|
+
* forged.
|
|
321
|
+
*
|
|
322
|
+
* Runs on `call.args`, which — unlike a completion value — has NO seam byte
|
|
323
|
+
* cap, so there is no budget to reject a wide payload against the way
|
|
324
|
+
* {@link checkDoneValue} does. The traversal therefore holds ONE cursor per
|
|
325
|
+
* NESTING LEVEL (an array or {@link ownValues} iterator) instead of one entry
|
|
326
|
+
* per member: a forged flat `args` at the top of the host's inbound frame-size
|
|
327
|
+
* cap would
|
|
328
|
+
* otherwise push tens of millions of stack entries — and `Object.values` would
|
|
329
|
+
* copy each object's full breadth — allocating hundreds of megabytes beyond
|
|
330
|
+
* what `JSON.parse` already holds. Iterative either way, so a deep frame
|
|
331
|
+
* cannot overflow the host stack.
|
|
332
|
+
* @param value - a JSON-parse-produced value from an fd-3 frame.
|
|
333
|
+
* @returns true when any contained number is non-finite or negative zero.
|
|
334
|
+
*/
|
|
335
|
+
function hasNonLosslessNumber(value) {
|
|
336
|
+
const cursors = [[value].values()];
|
|
337
|
+
while (cursors.length > 0) {
|
|
338
|
+
const step = cursors.at(-1).next();
|
|
339
|
+
if (step.done === true) {
|
|
340
|
+
cursors.pop();
|
|
341
|
+
continue;
|
|
342
|
+
}
|
|
343
|
+
const current = step.value;
|
|
344
|
+
if (typeof current === "number") {
|
|
345
|
+
if (!Number.isFinite(current) || Object.is(current, -0)) return true;
|
|
346
|
+
} else if (Array.isArray(current)) cursors.push(current.values());
|
|
347
|
+
else if (typeof current === "object" && current !== null) cursors.push(ownValues(current));
|
|
348
|
+
}
|
|
349
|
+
return false;
|
|
350
|
+
}
|
|
351
|
+
/**
|
|
352
|
+
* Runtime shape gate for inbound fd-3 traffic. Model code has full access to
|
|
353
|
+
* fd 3 and can post anything — `null`, primitives, poisoned fields — so the
|
|
354
|
+
* compile-time union means nothing here: every field is validated and REBUILT
|
|
355
|
+
* before the host reads it (forged extras never ride along; a non-number id
|
|
356
|
+
* can never be echoed into a reply). Junk returns `undefined` and is dropped
|
|
357
|
+
* so a throw in the host's `message` handler cannot crash the host process.
|
|
358
|
+
* @param raw - one JSON-parsed frame from fd 3.
|
|
359
|
+
* @returns the rebuilt frame, or `undefined` to drop it silently.
|
|
360
|
+
*/
|
|
361
|
+
function validateChildFrame(raw) {
|
|
362
|
+
if (typeof raw !== "object" || raw === null) return void 0;
|
|
363
|
+
const m = raw;
|
|
364
|
+
switch (m.type) {
|
|
365
|
+
case "boot-ack": return { type: "boot-ack" };
|
|
366
|
+
case "log":
|
|
367
|
+
if (typeof m.text !== "string") return void 0;
|
|
368
|
+
return {
|
|
369
|
+
type: "log",
|
|
370
|
+
text: m.text,
|
|
371
|
+
...m.truncated === true ? { truncated: true } : {}
|
|
372
|
+
};
|
|
373
|
+
case "call":
|
|
374
|
+
if (typeof m.id !== "number" || !Number.isFinite(m.id) || Object.is(m.id, -0) || typeof m.global !== "string" || typeof m.name !== "string") return void 0;
|
|
375
|
+
if (!Object.hasOwn(m, "args")) return void 0;
|
|
376
|
+
if (hasNonLosslessNumber(m.args)) return void 0;
|
|
377
|
+
return {
|
|
378
|
+
type: "call",
|
|
379
|
+
id: m.id,
|
|
380
|
+
global: m.global,
|
|
381
|
+
name: m.name,
|
|
382
|
+
args: m.args
|
|
383
|
+
};
|
|
384
|
+
case "done": {
|
|
385
|
+
const err = m.error;
|
|
386
|
+
if (err === void 0) return m.value === void 0 ? { type: "done" } : {
|
|
387
|
+
type: "done",
|
|
388
|
+
value: m.value
|
|
389
|
+
};
|
|
390
|
+
if (typeof err !== "object" || err === null) return void 0;
|
|
391
|
+
const { kind, message } = err;
|
|
392
|
+
if (typeof message !== "string") return void 0;
|
|
393
|
+
if (kind !== "exception" && kind !== "invalid-output" && kind !== "output-limit") return void 0;
|
|
394
|
+
return m.value === void 0 ? {
|
|
395
|
+
type: "done",
|
|
396
|
+
error: {
|
|
397
|
+
kind,
|
|
398
|
+
message
|
|
399
|
+
}
|
|
400
|
+
} : {
|
|
401
|
+
type: "done",
|
|
402
|
+
value: m.value,
|
|
403
|
+
error: {
|
|
404
|
+
kind,
|
|
405
|
+
message
|
|
406
|
+
}
|
|
407
|
+
};
|
|
408
|
+
}
|
|
409
|
+
default: return;
|
|
410
|
+
}
|
|
411
|
+
}
|
|
412
|
+
//#endregion
|
|
413
|
+
export { checkDoneValue, encodeJsonPlain, hasNonLosslessNumber, hasUnsafeIntegerToken, logTruncationMarker, validateChildFrame };
|
package/lib/invariant.js
ADDED
|
@@ -0,0 +1,24 @@
|
|
|
1
|
+
//#region lib/types/invariant.js
|
|
2
|
+
/**
|
|
3
|
+
* Package-owned invariant companion for `@hydraharness/harness-code-runtime-python`.
|
|
4
|
+
* @module @hydraharness/harness-code-runtime-python/invariant
|
|
5
|
+
*/
|
|
6
|
+
const PACKAGE_NAME = "@hydraharness/harness-code-runtime-python";
|
|
7
|
+
/** Cordis companion plugin name. */
|
|
8
|
+
const name = "code-runtime-python-invariant";
|
|
9
|
+
/** Service required before the companion can reserve package ownership. */
|
|
10
|
+
const inject = ["invariants"];
|
|
11
|
+
/**
|
|
12
|
+
* No runtime invariant: this package ships only the fd-3 wire-protocol codec and its Python mirror,
|
|
13
|
+
* exposing no runtime event sequence or mutable data relation; `protocol.spec.ts` and
|
|
14
|
+
* `protocol-mirror.e2e.ts` cover the protocol's behavior.
|
|
15
|
+
*/
|
|
16
|
+
const install = () => {};
|
|
17
|
+
/**
|
|
18
|
+
* Register this package's invariant companion.
|
|
19
|
+
* @param ctx - Cordis context carrying the invariant service.
|
|
20
|
+
* @returns the installed registration's disposer after setup succeeds.
|
|
21
|
+
*/
|
|
22
|
+
const apply = (ctx) => Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install));
|
|
23
|
+
//#endregion
|
|
24
|
+
export { apply, inject, name };
|
|
@@ -0,0 +1,12 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* CPython subprocess code runtime for the Hydra harness code-execution seam.
|
|
3
|
+
*
|
|
4
|
+
* The package owns the versionless fd-3 wire protocol between the Node host and
|
|
5
|
+
* the CPython subprocess. The protocol's host-side codec and hostile-frame
|
|
6
|
+
* validators are re-exported so every consumer of the wire shares one
|
|
7
|
+
* vocabulary.
|
|
8
|
+
* @module @hydraharness/harness-code-runtime-python
|
|
9
|
+
*/
|
|
10
|
+
export type { BootMessage, ChildToHost, ReplyMessage } from './protocol.ts';
|
|
11
|
+
export { checkDoneValue, encodeJsonPlain, hasNonLosslessNumber, hasUnsafeIntegerToken, logTruncationMarker, validateChildFrame, } from './protocol.ts';
|
|
12
|
+
//# sourceMappingURL=index.d.ts.map
|
|
@@ -0,0 +1,16 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Package-owned invariant companion for `@hydraharness/harness-code-runtime-python`.
|
|
3
|
+
* @module @hydraharness/harness-code-runtime-python/invariant
|
|
4
|
+
*/
|
|
5
|
+
import type { Context } from '@hydraharness/cordis';
|
|
6
|
+
/** Cordis companion plugin name. */
|
|
7
|
+
export declare const name = "code-runtime-python-invariant";
|
|
8
|
+
/** Service required before the companion can reserve package ownership. */
|
|
9
|
+
export declare const inject: string[];
|
|
10
|
+
/**
|
|
11
|
+
* Register this package's invariant companion.
|
|
12
|
+
* @param ctx - Cordis context carrying the invariant service.
|
|
13
|
+
* @returns the installed registration's disposer after setup succeeds.
|
|
14
|
+
*/
|
|
15
|
+
export declare const apply: (ctx: Context) => Promise<() => void>;
|
|
16
|
+
//# sourceMappingURL=invariant.d.ts.map
|
|
@@ -0,0 +1,329 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Versionless, JSON-lines wire protocol between the Node host and the CPython subprocess. Frames
|
|
3
|
+
* travel on the child's fd 3 (one JSON object per line), leaving stdout/stderr free for the
|
|
4
|
+
* program's own output. Host treats every inbound frame as hostile because model code can post
|
|
5
|
+
* anything through the same fd; the Python bootstrap trusts host replies.
|
|
6
|
+
* @module @hydraharness/harness-code-runtime-python/src/protocol
|
|
7
|
+
*/
|
|
8
|
+
/**
|
|
9
|
+
* The framed-JSON channel's file descriptor from the child's perspective. The
|
|
10
|
+
* host pins it positionally when it spawns the child (`stdio` index 3, i.e.
|
|
11
|
+
* `['pipe','pipe','pipe','pipe']`), and the Python bootstrap reads the same
|
|
12
|
+
* number from its own `protocol.py`. Exported as the single TS-side source of
|
|
13
|
+
* truth: the host wiring uses it, and the cross-language mirror test asserts the
|
|
14
|
+
* Python constant equals it, so a drift on either side breaks the boot channel
|
|
15
|
+
* loudly rather than silently.
|
|
16
|
+
*/
|
|
17
|
+
export declare const PROTOCOL_FD = 3;
|
|
18
|
+
/**
|
|
19
|
+
* One binding namespace declaration inside a {@link BootMessage}. `global` is
|
|
20
|
+
* the program-visible name the namespace is materialized under; `errorClass`,
|
|
21
|
+
* when present, asks the bootstrap to mint a program-visible exception class.
|
|
22
|
+
*/
|
|
23
|
+
interface Namespace {
|
|
24
|
+
global: string;
|
|
25
|
+
names: string[];
|
|
26
|
+
errorClass?: ErrorClass;
|
|
27
|
+
}
|
|
28
|
+
/**
|
|
29
|
+
* A namespace's program-visible exception class: rejected calls raise its
|
|
30
|
+
* instances carrying the failed member name on `memberNameProperty`.
|
|
31
|
+
*/
|
|
32
|
+
interface ErrorClass {
|
|
33
|
+
name: string;
|
|
34
|
+
memberNameProperty: string;
|
|
35
|
+
}
|
|
36
|
+
/**
|
|
37
|
+
* What the host sends immediately after spawn, as the first line on fd 3. The
|
|
38
|
+
* Python bootstrap reads this, applies resource limits, then waits for the
|
|
39
|
+
* subsequent run frame. Separated from the run so the run message stays
|
|
40
|
+
* pure model input.
|
|
41
|
+
*/
|
|
42
|
+
export interface BootMessage {
|
|
43
|
+
type: 'boot';
|
|
44
|
+
/** RLIMIT_CPU seconds; the Python bootstrap sets this on itself before executing model code. */
|
|
45
|
+
cpuSeconds: number;
|
|
46
|
+
/** RLIMIT_AS bytes; caps address space so a runaway allocation fails cleanly. */
|
|
47
|
+
addressSpaceBytes: number;
|
|
48
|
+
/** Shared byte budget for captured log text (Python-side ledger). */
|
|
49
|
+
maxLogBytes: number;
|
|
50
|
+
/** Byte cap for the rendered completion value. */
|
|
51
|
+
maxValueBytes: number;
|
|
52
|
+
/**
|
|
53
|
+
* The namespaces to materialize inside the program (globals + names;
|
|
54
|
+
* functions stay host-side). See {@link Namespace}.
|
|
55
|
+
*/
|
|
56
|
+
namespaces: Namespace[];
|
|
57
|
+
}
|
|
58
|
+
/** Python → host: acknowledges boot completed and resource limits are in place. */
|
|
59
|
+
interface BootAckMessage {
|
|
60
|
+
type: 'boot-ack';
|
|
61
|
+
}
|
|
62
|
+
/** Python → host: one bridged binding call (`await tools.name(args)` inside the program). */
|
|
63
|
+
interface CallMessage {
|
|
64
|
+
type: 'call';
|
|
65
|
+
/** Python-issued correlation id; the host answers each id at most once and ignores duplicates. */
|
|
66
|
+
id: number;
|
|
67
|
+
/** The namespace global the call targets. */
|
|
68
|
+
global: string;
|
|
69
|
+
/** The function name within the namespace. */
|
|
70
|
+
name: string;
|
|
71
|
+
/** The JSON-safe argument the model program passed. */
|
|
72
|
+
args: unknown;
|
|
73
|
+
}
|
|
74
|
+
/**
|
|
75
|
+
* Python → host: captured text, streamed eagerly so output survives a
|
|
76
|
+
* mid-run termination (RLIMIT_CPU, SIGTERM/SIGKILL, host wall-timeout).
|
|
77
|
+
*/
|
|
78
|
+
interface LogMessage {
|
|
79
|
+
type: 'log';
|
|
80
|
+
text: string;
|
|
81
|
+
/**
|
|
82
|
+
* Set when this frame IS the child ledger's truncation marker rather than
|
|
83
|
+
* program output. The two ledgers can exhaust at different points — one
|
|
84
|
+
* child entry larger than `maxLogBytes` sends only the marker while the host
|
|
85
|
+
* ledger is still nearly empty — so the host cannot infer the child's state
|
|
86
|
+
* from its own budget, and comparing the text against the marker string
|
|
87
|
+
* would also honour a program that printed that string itself. Carrying it
|
|
88
|
+
* as a field lets the host stop capturing at the same point the child did
|
|
89
|
+
* and keeps exactly one marker in `logs`.
|
|
90
|
+
*/
|
|
91
|
+
truncated?: boolean;
|
|
92
|
+
}
|
|
93
|
+
/** The failure carried on a {@link DoneMessage}: one of three kinds plus text. */
|
|
94
|
+
interface DoneErrorField {
|
|
95
|
+
kind: 'exception' | 'invalid-output' | 'output-limit';
|
|
96
|
+
message: string;
|
|
97
|
+
}
|
|
98
|
+
/**
|
|
99
|
+
* Python → host: the program settled. `error` carries a program exception
|
|
100
|
+
* (traceback text), an `invalid-output` (completion value was not lossless
|
|
101
|
+
* JSON), or an `output-limit` (serialized completion exceeded the configured
|
|
102
|
+
* cap); wall/CPU budgets, aborts, and substrate death are observed host-side.
|
|
103
|
+
* From the honest child `value` is present only on a clean completion that
|
|
104
|
+
* produced one, and crosses as exact lossless JSON — never substituted or
|
|
105
|
+
* truncated. A forged frame CAN carry both `value` and `error`;
|
|
106
|
+
* {@link validateChildFrame} preserves both rather than guessing which to drop,
|
|
107
|
+
* so a consumer MUST check `error` first and ignore `value` when it is set.
|
|
108
|
+
*/
|
|
109
|
+
interface DoneMessage {
|
|
110
|
+
type: 'done';
|
|
111
|
+
value?: unknown;
|
|
112
|
+
error?: DoneErrorField;
|
|
113
|
+
}
|
|
114
|
+
/**
|
|
115
|
+
* Every message the Python side sends. The member interfaces stay module-
|
|
116
|
+
* private: consumers match on the union's discriminant; the host sends the
|
|
117
|
+
* boot and run frames as inline literals.
|
|
118
|
+
*/
|
|
119
|
+
export type ChildToHost = BootAckMessage | CallMessage | LogMessage | DoneMessage;
|
|
120
|
+
/** Host → Python: successful answer to one {@link CallMessage}. */
|
|
121
|
+
interface ReplyOk {
|
|
122
|
+
type: 'reply';
|
|
123
|
+
id: number;
|
|
124
|
+
ok: true;
|
|
125
|
+
value: unknown;
|
|
126
|
+
}
|
|
127
|
+
/** Host → Python: failed answer to one {@link CallMessage}. */
|
|
128
|
+
interface ReplyErr {
|
|
129
|
+
type: 'reply';
|
|
130
|
+
id: number;
|
|
131
|
+
ok: false;
|
|
132
|
+
message: string;
|
|
133
|
+
}
|
|
134
|
+
/** Host → Python: the answer to one {@link CallMessage}. */
|
|
135
|
+
export type ReplyMessage = ReplyOk | ReplyErr;
|
|
136
|
+
/**
|
|
137
|
+
* Each frame's wire fields tagged by required/optional, keyed by field name so
|
|
138
|
+
* the mapping is exhaustive over the frame interface (see {@link FrameFieldRoles})
|
|
139
|
+
* across the whole {@link WireFrameShapes} roster. Bound to the interfaces by
|
|
140
|
+
* `satisfies` below; {@link WIRE_FRAME_FIELDS} projects it to sorted
|
|
141
|
+
* required/optional arrays for the cross-language mirror comparison. `global` is
|
|
142
|
+
* the JSON key {@link CallMessage} and {@link Namespace} send (a reserved word
|
|
143
|
+
* the Python side carries via a functional `TypedDict`).
|
|
144
|
+
*/
|
|
145
|
+
declare const WIRE_FRAME_FIELD_ROLES: {
|
|
146
|
+
readonly BootMessage: {
|
|
147
|
+
readonly type: "required";
|
|
148
|
+
readonly cpuSeconds: "required";
|
|
149
|
+
readonly addressSpaceBytes: "required";
|
|
150
|
+
readonly maxLogBytes: "required";
|
|
151
|
+
readonly maxValueBytes: "required";
|
|
152
|
+
readonly namespaces: "required";
|
|
153
|
+
};
|
|
154
|
+
readonly Namespace: {
|
|
155
|
+
readonly global: "required";
|
|
156
|
+
readonly names: "required";
|
|
157
|
+
readonly errorClass: "optional";
|
|
158
|
+
};
|
|
159
|
+
readonly RunMessage: {
|
|
160
|
+
readonly type: "required";
|
|
161
|
+
readonly program: "required";
|
|
162
|
+
};
|
|
163
|
+
readonly BootAckMessage: {
|
|
164
|
+
readonly type: "required";
|
|
165
|
+
};
|
|
166
|
+
readonly CallMessage: {
|
|
167
|
+
readonly type: "required";
|
|
168
|
+
readonly id: "required";
|
|
169
|
+
readonly global: "required";
|
|
170
|
+
readonly name: "required";
|
|
171
|
+
readonly args: "required";
|
|
172
|
+
};
|
|
173
|
+
readonly LogMessage: {
|
|
174
|
+
readonly type: "required";
|
|
175
|
+
readonly text: "required";
|
|
176
|
+
readonly truncated: "optional";
|
|
177
|
+
};
|
|
178
|
+
readonly DoneErrorField: {
|
|
179
|
+
readonly kind: "required";
|
|
180
|
+
readonly message: "required";
|
|
181
|
+
};
|
|
182
|
+
readonly DoneMessage: {
|
|
183
|
+
readonly type: "required";
|
|
184
|
+
readonly value: "optional";
|
|
185
|
+
readonly error: "optional";
|
|
186
|
+
};
|
|
187
|
+
readonly ErrorClass: {
|
|
188
|
+
readonly name: "required";
|
|
189
|
+
readonly memberNameProperty: "required";
|
|
190
|
+
};
|
|
191
|
+
readonly ReplyOk: {
|
|
192
|
+
readonly type: "required";
|
|
193
|
+
readonly id: "required";
|
|
194
|
+
readonly ok: "required";
|
|
195
|
+
readonly value: "required";
|
|
196
|
+
};
|
|
197
|
+
readonly ReplyErr: {
|
|
198
|
+
readonly type: "required";
|
|
199
|
+
readonly id: "required";
|
|
200
|
+
readonly ok: "required";
|
|
201
|
+
readonly message: "required";
|
|
202
|
+
};
|
|
203
|
+
};
|
|
204
|
+
/**
|
|
205
|
+
* The wire field names of each frame, split into sorted required and optional
|
|
206
|
+
* key arrays — the shape the cross-language mirror test compares against
|
|
207
|
+
* `py/protocol.py`'s `TypedDict` `__required_keys__`/`__optional_keys__`.
|
|
208
|
+
* Projected from {@link WIRE_FRAME_FIELD_ROLES}, so it inherits that mapping's
|
|
209
|
+
* exhaustive, optionality-checked binding to the frame interfaces: a TS-side
|
|
210
|
+
* field add, remove, rename, or optionality flip fails typecheck at the roles
|
|
211
|
+
* map, and a Python-side divergence fails the mirror test at runtime.
|
|
212
|
+
*/
|
|
213
|
+
export declare const WIRE_FRAME_FIELDS: Record<keyof typeof WIRE_FRAME_FIELD_ROLES, {
|
|
214
|
+
required: string[];
|
|
215
|
+
optional: string[];
|
|
216
|
+
}>;
|
|
217
|
+
/**
|
|
218
|
+
* The in-band marker text announcing that log capture stopped at the byte
|
|
219
|
+
* budget. Shared wire vocabulary: the Python-side LogBuffer emits it when ITS
|
|
220
|
+
* ledger exhausts, and the host emits identical text when its own ledger drops
|
|
221
|
+
* a frame first (forged fd-3 traffic, stray stdout bytes) — a truncated run
|
|
222
|
+
* reads the same however the cap was hit.
|
|
223
|
+
* @param maxBytes - the configured `maxLogBytes` the marker names.
|
|
224
|
+
* @returns the marker line.
|
|
225
|
+
*/
|
|
226
|
+
export declare function logTruncationMarker(maxBytes: number): string;
|
|
227
|
+
/**
|
|
228
|
+
* Serialize one JSON-parse-produced value without recursion. `JSON.stringify`
|
|
229
|
+
* recurses per nesting level and throws `RangeError` a few thousand levels
|
|
230
|
+
* deep, but the seam's `CodeJsonValue` has no depth limit — an honest deep
|
|
231
|
+
* completion or binding resolution below the byte budget must cross intact
|
|
232
|
+
* (the worker backend's wire is equally stack-safe). Callers must pass a value
|
|
233
|
+
* produced by `JSON.parse` (or equally JSON-plain): only `null`, finite
|
|
234
|
+
* numbers, booleans, strings, dense arrays, and plain objects — this encoder
|
|
235
|
+
* validates nothing. Output matches compact `JSON.stringify` byte for byte
|
|
236
|
+
* EXCEPT on an integral double beyond the safe range, where {@link scalarJson}
|
|
237
|
+
* emits the exact integer's BigInt digits rather than `JSON.stringify`'s rounded
|
|
238
|
+
* spelling (`1152921504606846976`, not `...847000`) so the seam's lossless-JSON
|
|
239
|
+
* promise holds across the wire.
|
|
240
|
+
* @param value - a JSON-plain value (e.g. straight from `JSON.parse`).
|
|
241
|
+
* @returns the compact JSON encoding.
|
|
242
|
+
*/
|
|
243
|
+
export declare function encodeJsonPlain(value: unknown): string;
|
|
244
|
+
/**
|
|
245
|
+
* Meter a `JSON.parse`-produced done value's compact-JSON byte length AND its
|
|
246
|
+
* number losslessness in one traversal, stopping the instant `maxBytes` is
|
|
247
|
+
* crossed. This bounds the INCREMENTAL allocation the check itself would add on
|
|
248
|
+
* top of the already-parsed value — the enqueued children; strings and keys are
|
|
249
|
+
* metered by {@link jsonStringBytesUpTo} without allocating an escaped copy —
|
|
250
|
+
* not the parse that produced `value`.
|
|
251
|
+
* That upstream width is bounded separately, by the host-side cap on inbound
|
|
252
|
+
* fd-3 frame size before `JSON.parse` runs (owned by the runtime that reads the
|
|
253
|
+
* channel), so `value` cannot be arbitrarily large when it reaches here. The
|
|
254
|
+
* budget is the `maxValueBytes` the boot frame carries — a required wire field
|
|
255
|
+
* with no default at this layer. The traversal rejects over-budget BEFORE
|
|
256
|
+
* materializing a string's escaped form or enqueuing an array's/object's
|
|
257
|
+
* children, so a forgery within that frame cap cannot force those secondary
|
|
258
|
+
* allocations. Object key COUNTING is
|
|
259
|
+
* unavoidably O(keys) — JS has no lazy own-key iterator, and the parse already
|
|
260
|
+
* built the key set — but the check still refuses the per-entry work before the
|
|
261
|
+
* enqueue loop. A non-lossless number (non-finite, negative zero) is caught only
|
|
262
|
+
* when the value fits the budget — an over-budget value is rejected regardless,
|
|
263
|
+
* so the distinction is moot. Same JSON-plain precondition and traversal shape
|
|
264
|
+
* as {@link encodeJsonPlain}; a number's byte length is measured through
|
|
265
|
+
* {@link scalarJson} (matching the encoder, so a beyond-safe-range integer
|
|
266
|
+
* meters its exact BigInt digits, not `JSON.stringify`'s rounded spelling) and
|
|
267
|
+
* a string's/key's through {@link jsonStringBytesUpTo} (the exact escaped size,
|
|
268
|
+
* scanned without allocating the escaped copy).
|
|
269
|
+
* @param value - a JSON-plain value (e.g. straight from `JSON.parse`).
|
|
270
|
+
* @param maxBytes - the completion-value budget in bytes.
|
|
271
|
+
* @returns `{ ok: true, bytes }` with the exact serialized size, or
|
|
272
|
+
* `{ ok: false, reason }` — `over-budget` once the size exceeds `maxBytes`,
|
|
273
|
+
* `non-lossless` on a non-finite or negative-zero number.
|
|
274
|
+
*/
|
|
275
|
+
export declare function checkDoneValue(value: unknown, maxBytes: number): {
|
|
276
|
+
ok: true;
|
|
277
|
+
bytes: number;
|
|
278
|
+
} | {
|
|
279
|
+
ok: false;
|
|
280
|
+
reason: 'over-budget' | 'non-lossless';
|
|
281
|
+
};
|
|
282
|
+
/**
|
|
283
|
+
* Whether a raw JSON line contains an integer token that would lose precision
|
|
284
|
+
* as a JavaScript number. `JSON.parse` silently rounds such a token
|
|
285
|
+
* (`9007199254740993` becomes `...992`) BEFORE any validation can see it, so
|
|
286
|
+
* the check must read the source text; a beyond-safe-range token whose double
|
|
287
|
+
* parse round-trips exactly (`2**53`, `2**60`) is lossless and passes. The scan walks the line skipping string literals (a digit run
|
|
288
|
+
* inside a string is data, not a number token) and tests every number token
|
|
289
|
+
* in plain integer form — no fraction or exponent, which parse as doubles by
|
|
290
|
+
* intent. A reviver cannot do this job: the reviver walk recurses per nesting
|
|
291
|
+
* level and would reintroduce the depth limit `encodeJsonPlain` removes.
|
|
292
|
+
* @param line - the raw UTF-8 text of one JSON-lines frame.
|
|
293
|
+
* @returns true when an unsafe integer token is present outside strings.
|
|
294
|
+
*/
|
|
295
|
+
export declare function hasUnsafeIntegerToken(line: string): boolean;
|
|
296
|
+
/**
|
|
297
|
+
* Whether a JSON.parse-produced value contains a number outside lossless
|
|
298
|
+
* JSON: non-finite (`1e400` parses to `Infinity`) or negative zero (`-0.0`
|
|
299
|
+
* parses to JS `-0`, whose sign bit a re-serialization drops). The honest
|
|
300
|
+
* child's validator rejects these before sending, so a frame carrying one is
|
|
301
|
+
* forged.
|
|
302
|
+
*
|
|
303
|
+
* Runs on `call.args`, which — unlike a completion value — has NO seam byte
|
|
304
|
+
* cap, so there is no budget to reject a wide payload against the way
|
|
305
|
+
* {@link checkDoneValue} does. The traversal therefore holds ONE cursor per
|
|
306
|
+
* NESTING LEVEL (an array or {@link ownValues} iterator) instead of one entry
|
|
307
|
+
* per member: a forged flat `args` at the top of the host's inbound frame-size
|
|
308
|
+
* cap would
|
|
309
|
+
* otherwise push tens of millions of stack entries — and `Object.values` would
|
|
310
|
+
* copy each object's full breadth — allocating hundreds of megabytes beyond
|
|
311
|
+
* what `JSON.parse` already holds. Iterative either way, so a deep frame
|
|
312
|
+
* cannot overflow the host stack.
|
|
313
|
+
* @param value - a JSON-parse-produced value from an fd-3 frame.
|
|
314
|
+
* @returns true when any contained number is non-finite or negative zero.
|
|
315
|
+
*/
|
|
316
|
+
export declare function hasNonLosslessNumber(value: unknown): boolean;
|
|
317
|
+
/**
|
|
318
|
+
* Runtime shape gate for inbound fd-3 traffic. Model code has full access to
|
|
319
|
+
* fd 3 and can post anything — `null`, primitives, poisoned fields — so the
|
|
320
|
+
* compile-time union means nothing here: every field is validated and REBUILT
|
|
321
|
+
* before the host reads it (forged extras never ride along; a non-number id
|
|
322
|
+
* can never be echoed into a reply). Junk returns `undefined` and is dropped
|
|
323
|
+
* so a throw in the host's `message` handler cannot crash the host process.
|
|
324
|
+
* @param raw - one JSON-parsed frame from fd 3.
|
|
325
|
+
* @returns the rebuilt frame, or `undefined` to drop it silently.
|
|
326
|
+
*/
|
|
327
|
+
export declare function validateChildFrame(raw: unknown): ChildToHost | undefined;
|
|
328
|
+
export {};
|
|
329
|
+
//# sourceMappingURL=protocol.d.ts.map
|
package/package.json
ADDED
|
@@ -0,0 +1,42 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "@hydraharness/harness-code-runtime-python",
|
|
3
|
+
"description": "CPython subprocess implementation of the Hydra harness code-execution seam",
|
|
4
|
+
"version": "0.1.1-rc.10",
|
|
5
|
+
"publishConfig": {
|
|
6
|
+
"access": "public"
|
|
7
|
+
},
|
|
8
|
+
"repository": {
|
|
9
|
+
"type": "git",
|
|
10
|
+
"url": "git+https://github.com/MaiHongPhong2010/Hydra-Harness.git",
|
|
11
|
+
"directory": "packages/code-runtime/code-runtime-python"
|
|
12
|
+
},
|
|
13
|
+
"type": "module",
|
|
14
|
+
"main": "lib/index.js",
|
|
15
|
+
"types": "lib/types/index.d.ts",
|
|
16
|
+
"exports": {
|
|
17
|
+
".": {
|
|
18
|
+
"types": "./lib/types/index.d.ts",
|
|
19
|
+
"default": "./lib/index.js"
|
|
20
|
+
},
|
|
21
|
+
"./invariant": {
|
|
22
|
+
"types": "./lib/types/invariant.d.ts",
|
|
23
|
+
"default": "./lib/invariant.js"
|
|
24
|
+
},
|
|
25
|
+
"./package.json": "./package.json"
|
|
26
|
+
},
|
|
27
|
+
"files": [
|
|
28
|
+
"lib/index.js",
|
|
29
|
+
"lib/invariant.js",
|
|
30
|
+
"py/**/*.py",
|
|
31
|
+
"lib/types/**/*.d.ts"
|
|
32
|
+
],
|
|
33
|
+
"license": "MIT",
|
|
34
|
+
"peerDependencies": {
|
|
35
|
+
"@hydraharness/harness-invariants": "^0.1.1-rc.10",
|
|
36
|
+
"@hydraharness/cordis": "^4.0.2"
|
|
37
|
+
},
|
|
38
|
+
"devDependencies": {
|
|
39
|
+
"@hydraharness/harness-invariants": "^0.1.1-rc.10",
|
|
40
|
+
"@hydraharness/cordis": "^4.0.2"
|
|
41
|
+
}
|
|
42
|
+
}
|
package/py/protocol.py
ADDED
|
@@ -0,0 +1,138 @@
|
|
|
1
|
+
"""Wire protocol vocabulary for the Python side of hydra-code-runtime-python.
|
|
2
|
+
|
|
3
|
+
Mirrors ``src/protocol.ts``. Frames travel on fd 3 as JSON-lines (one JSON
|
|
4
|
+
object per line). The host validates every inbound frame; this side trusts
|
|
5
|
+
host replies.
|
|
6
|
+
|
|
7
|
+
The wire uses the JSON key ``global`` (a Python keyword), so the frame
|
|
8
|
+
``TypedDict``s that carry it are declared with the functional syntax rather than
|
|
9
|
+
class bodies: a class attribute cannot be named ``global``, and a ``global_``
|
|
10
|
+
attribute would describe a key the wire never sends. Optional-field messages
|
|
11
|
+
pair a required base with a ``total=False`` subclass so a required field such as
|
|
12
|
+
``type`` cannot be dropped while ``value``/``error``/``truncated`` stay optional.
|
|
13
|
+
"""
|
|
14
|
+
|
|
15
|
+
from __future__ import annotations
|
|
16
|
+
|
|
17
|
+
from typing import Any, Literal, TypedDict, Union
|
|
18
|
+
|
|
19
|
+
# The protocol fd from the child's perspective. Node passes
|
|
20
|
+
# ``stdio: [pipe, pipe, pipe, pipe]`` so the fourth entry (fd 3) is the
|
|
21
|
+
# framed-JSON channel; stdout/stderr stay clear for the program's own output.
|
|
22
|
+
PROTOCOL_FD = 3
|
|
23
|
+
|
|
24
|
+
|
|
25
|
+
class ErrorClass(TypedDict):
|
|
26
|
+
"""A namespace's program-visible exception class: rejected calls raise its
|
|
27
|
+
instances carrying the failed member name on ``memberNameProperty``."""
|
|
28
|
+
|
|
29
|
+
name: str
|
|
30
|
+
memberNameProperty: str
|
|
31
|
+
|
|
32
|
+
|
|
33
|
+
# ``global`` is a Python keyword, so the required part is declared functionally
|
|
34
|
+
# to hold the real wire key; ``errorClass`` is optional per the TS `errorClass?`.
|
|
35
|
+
_NamespaceRequired = TypedDict("_NamespaceRequired", {"global": str, "names": "list[str]"})
|
|
36
|
+
|
|
37
|
+
|
|
38
|
+
class Namespace(_NamespaceRequired, total=False):
|
|
39
|
+
"""One binding namespace declaration: the ``global`` name, its function
|
|
40
|
+
``names``, and an optional program-visible ``errorClass`` for rejected calls."""
|
|
41
|
+
|
|
42
|
+
errorClass: ErrorClass
|
|
43
|
+
|
|
44
|
+
|
|
45
|
+
class BootMessage(TypedDict):
|
|
46
|
+
"""Host → child, first frame on fd 3. Carries every cap and the namespaces."""
|
|
47
|
+
|
|
48
|
+
type: Literal["boot"]
|
|
49
|
+
cpuSeconds: int
|
|
50
|
+
addressSpaceBytes: int
|
|
51
|
+
maxLogBytes: int
|
|
52
|
+
maxValueBytes: int
|
|
53
|
+
namespaces: "list[Namespace]"
|
|
54
|
+
|
|
55
|
+
|
|
56
|
+
class RunMessage(TypedDict):
|
|
57
|
+
"""Host → child, sent after ``boot-ack``. Carries only the program body."""
|
|
58
|
+
|
|
59
|
+
type: Literal["run"]
|
|
60
|
+
program: str
|
|
61
|
+
|
|
62
|
+
|
|
63
|
+
class BootAckMessage(TypedDict):
|
|
64
|
+
"""Child → host: resource limits applied, ready for the run message."""
|
|
65
|
+
|
|
66
|
+
type: Literal["boot-ack"]
|
|
67
|
+
|
|
68
|
+
|
|
69
|
+
# ``global`` wire key: whole message declared functionally, all fields required.
|
|
70
|
+
CallMessage = TypedDict(
|
|
71
|
+
"CallMessage",
|
|
72
|
+
{"type": Literal["call"], "id": int, "global": str, "name": str, "args": Any},
|
|
73
|
+
)
|
|
74
|
+
|
|
75
|
+
|
|
76
|
+
_LogMessageRequired = TypedDict("_LogMessageRequired", {"type": Literal["log"], "text": str})
|
|
77
|
+
|
|
78
|
+
|
|
79
|
+
class LogMessage(_LogMessageRequired, total=False):
|
|
80
|
+
"""Child → host: one captured text chunk, streamed eagerly.
|
|
81
|
+
|
|
82
|
+
``truncated`` is set only on the frame that IS the child ledger's truncation
|
|
83
|
+
marker (not program output), so the host stops capturing at the same point
|
|
84
|
+
the child did — mirrors the TS `truncated?`.
|
|
85
|
+
"""
|
|
86
|
+
|
|
87
|
+
truncated: bool
|
|
88
|
+
|
|
89
|
+
|
|
90
|
+
class DoneErrorField(TypedDict):
|
|
91
|
+
"""Child → host: the failure carried on a ``done`` frame. ``kind`` is one of
|
|
92
|
+
the three the host validates; ``message`` is the traceback or diagnostic."""
|
|
93
|
+
|
|
94
|
+
kind: Literal["exception", "invalid-output", "output-limit"]
|
|
95
|
+
message: str
|
|
96
|
+
|
|
97
|
+
|
|
98
|
+
_DoneMessageRequired = TypedDict("_DoneMessageRequired", {"type": Literal["done"]})
|
|
99
|
+
|
|
100
|
+
|
|
101
|
+
class DoneMessage(_DoneMessageRequired, total=False):
|
|
102
|
+
"""Child → host: the program settled. ``value`` and ``error`` are optional per the TS mirror."""
|
|
103
|
+
|
|
104
|
+
value: Any
|
|
105
|
+
error: DoneErrorField
|
|
106
|
+
|
|
107
|
+
|
|
108
|
+
ChildToHost = Union[BootAckMessage, CallMessage, LogMessage, DoneMessage]
|
|
109
|
+
|
|
110
|
+
|
|
111
|
+
class ReplyOk(TypedDict):
|
|
112
|
+
type: Literal["reply"]
|
|
113
|
+
id: int
|
|
114
|
+
ok: Literal[True]
|
|
115
|
+
value: Any
|
|
116
|
+
|
|
117
|
+
|
|
118
|
+
class ReplyErr(TypedDict):
|
|
119
|
+
type: Literal["reply"]
|
|
120
|
+
id: int
|
|
121
|
+
ok: Literal[False]
|
|
122
|
+
message: str
|
|
123
|
+
|
|
124
|
+
|
|
125
|
+
ReplyMessage = Union[ReplyOk, ReplyErr]
|
|
126
|
+
# The host sends ``boot`` and ``run`` before any ``reply``, so the child-facing
|
|
127
|
+
# inbound union covers all three, not replies alone.
|
|
128
|
+
HostToChild = Union[BootMessage, RunMessage, ReplyMessage]
|
|
129
|
+
|
|
130
|
+
|
|
131
|
+
def log_truncation_marker(max_bytes: int) -> str:
|
|
132
|
+
"""Return the in-band marker for a log ledger that exhausted its budget.
|
|
133
|
+
|
|
134
|
+
Byte-identical text on both sides of the wire so a truncated run reads the
|
|
135
|
+
same however the cap was hit.
|
|
136
|
+
"""
|
|
137
|
+
|
|
138
|
+
return f"[hydra-code-runtime-python] log capture truncated at {max_bytes} bytes"
|