@gfdelarue/testkit 0.3.5 → 0.3.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +7 -73
- package/dist/app/run/engine-execution.d.ts +2 -0
- package/dist/app/run/engine-execution.js +1 -1
- package/dist/engines/adapters/node-startup-cache.d.ts +8 -0
- package/dist/engines/adapters/node-startup-cache.js +1 -0
- package/dist/engines/api/execution.d.ts +2 -0
- package/dist/engines/managed-module-resolution.js +8 -1
- package/dist/engines/node-vitest/adapter/adapter.js +1 -1
- package/dist/engines/playwright/adapter/adapter.js +1 -1
- package/dist/environment/ready-environment.d.ts +3 -1
- package/dist/environment/ready-environment.js +1 -1
- package/dist/environment/run-environment.js +1 -1
- package/dist/environment/runtime-contexts.d.ts +1 -0
- package/dist/environment/runtime-contexts.js +1 -1
- package/dist/environment/runtime-manager.js +1 -1
- package/dist/process/execa-bundle.json +1 -1
- package/docs/cloud-control.md +3 -0
- package/docs/engine-architecture.md +20 -0
- package/docs/testkit-cloud-boundary.md +12 -24
- package/package.json +1 -1
- package/testkit-config-schema.md +2 -0
package/README.md
CHANGED
|
@@ -7,7 +7,9 @@
|
|
|
7
7
|
`@gfdelarue/testkit` is the orchestrator and managed distribution for product test
|
|
8
8
|
suites. It discovers `*.testkit.ts` files, infers suite ownership from the
|
|
9
9
|
filesystem, starts local services, provisions Docker-managed local Postgres
|
|
10
|
-
databases, schedules suites, and collects results and artifacts.
|
|
10
|
+
databases, schedules suites, and collects results and artifacts. Coding agents
|
|
11
|
+
use it to write tests and hunt bugs, and CI uses it to run those tests. It needs
|
|
12
|
+
no account or server.
|
|
11
13
|
|
|
12
14
|
Testkit exists to replace false confidence with inspectable execution evidence.
|
|
13
15
|
It cannot prove that an entire application works. It can make narrower claims
|
|
@@ -47,33 +49,6 @@ Testkit writes the immutable, verifiable run bundle under
|
|
|
47
49
|
`.testkit/results/runs/<run-id>/`. Upload `.testkit/results` with `if: always()`
|
|
48
50
|
when CI evidence should survive a failed job.
|
|
49
51
|
|
|
50
|
-
Publishing that evidence to Testkit Cloud is an additional explicit step. Set
|
|
51
|
-
`TESTKIT_CLOUD_PROJECT_ID` and the secret `TESTKIT_CLOUD_TOKEN`, then run:
|
|
52
|
-
|
|
53
|
-
```bash
|
|
54
|
-
npx testkit cloud publish
|
|
55
|
-
```
|
|
56
|
-
|
|
57
|
-
The command verifies the immutable bundle before upload. Ordinary runs never
|
|
58
|
-
contact Cloud, and the presence of `CI` does not authorize publication.
|
|
59
|
-
|
|
60
|
-
## Connect to Cloud
|
|
61
|
-
|
|
62
|
-
For interactive use, approve your CLI through the browser once:
|
|
63
|
-
|
|
64
|
-
```sh
|
|
65
|
-
npx testkit cloud login
|
|
66
|
-
npx testkit cloud project create
|
|
67
|
-
npx testkit cloud publish --project PROJECT_ID --run RUN_ID
|
|
68
|
-
```
|
|
69
|
-
|
|
70
|
-
Login works outside Git. Project creation infers the repository from Git origin
|
|
71
|
-
or accepts `--repository owner/repo`. It resumes after any required provider
|
|
72
|
-
consent. `cloud list`, `cloud show`, and `cloud apply --file` provide discovery,
|
|
73
|
-
settings and product control. See the [Cloud control boundary](docs/testkit-cloud-boundary.md#complete-cloud-control)
|
|
74
|
-
and [complete operation reference](https://github.com/gfdelarue/testkit-cloud/blob/master/docs/cloud-control.md).
|
|
75
|
-
Explicit `TESTKIT_CLOUD_TOKEN` still takes precedence over the saved account.
|
|
76
|
-
|
|
77
52
|
## Execution engines
|
|
78
53
|
|
|
79
54
|
Testkit has three explicit built-in engines rather than one implicit runtime:
|
|
@@ -219,40 +194,9 @@ Batch `run` output stays intentionally short: one line per completed file,
|
|
|
219
194
|
concise failure blocks, and a final summary. Service logs, captured runtime
|
|
220
195
|
output, and emitted artifacts are persisted under `.testkit/results/`.
|
|
221
196
|
|
|
222
|
-
Testkit
|
|
223
|
-
|
|
224
|
-
output
|
|
225
|
-
Publish that exact evidence with `testkit cloud publish --run <id>`. Omitting
|
|
226
|
-
`--run` selects the newest bundle for interactive use. Historical summaries
|
|
227
|
-
and trends belong to Testkit Cloud.
|
|
228
|
-
|
|
229
|
-
For CI, preserve the run exit status and publish the ID from that invocation:
|
|
230
|
-
|
|
231
|
-
```sh
|
|
232
|
-
run_exit=0
|
|
233
|
-
testkit run --json > testkit-run.json || run_exit=$?
|
|
234
|
-
run_id=$(node -e 'const r = JSON.parse(require("node:fs").readFileSync("testkit-run.json", "utf8")); if (typeof r.runId !== "string" || !r.runId) process.exit(1); process.stdout.write(r.runId)') || {
|
|
235
|
-
if [ "$run_exit" -ne 0 ]; then exit "$run_exit"; fi
|
|
236
|
-
exit 1
|
|
237
|
-
}
|
|
238
|
-
publish_exit=0
|
|
239
|
-
testkit cloud publish --run "$run_id" || publish_exit=$?
|
|
240
|
-
if [ "$publish_exit" -ne 0 ]; then
|
|
241
|
-
echo "Cloud publication failed with exit $publish_exit" >&2
|
|
242
|
-
fi
|
|
243
|
-
exit "$run_exit"
|
|
244
|
-
```
|
|
245
|
-
|
|
246
|
-
Set `TESTKIT_CLOUD_PROJECT_ID` and `TESTKIT_CLOUD_TOKEN` in the CI environment.
|
|
247
|
-
`TESTKIT_CLOUD_URL`, or the `--url` flag on `cloud publish`, `cloud apply`, and
|
|
248
|
-
`cloud snapshot`, changes the Testkit Cloud origin. The bearer token and the
|
|
249
|
-
run bundle are sent only to that origin, so protect the variable like the
|
|
250
|
-
token itself. Leave it unset to use `https://testkit.cloud`.
|
|
251
|
-
A failed run can still publish a valid sealed bundle. Publication reports its
|
|
252
|
-
own exit status and does not change the test outcome. This recipe reports a
|
|
253
|
-
publication failure and retains the run status as the job status. See the
|
|
254
|
-
[Cloud boundary](docs/testkit-cloud-boundary.md) and
|
|
255
|
-
[breaking-change migration](https://github.com/gfdelarue/testkit/blob/master/docs/release-notes.md).
|
|
197
|
+
Testkit seals an immutable local run bundle. `testkit run` prints
|
|
198
|
+
`Run ID: <id>` after sealing in text output and returns `runId` in `--json`
|
|
199
|
+
output, for passed and failed runs.
|
|
256
200
|
|
|
257
201
|
Test execution also maintains a scheduler cache at `.testkit/timings.json`.
|
|
258
202
|
Completed file-level task durations are used to rank future runs with a
|
|
@@ -653,16 +597,6 @@ Programmatic consumers can import `listRunBundles`, `loadRunBundleArtifact`,
|
|
|
653
597
|
`loadRunBundleManifest`, and `verifyRunBundle` from
|
|
654
598
|
`@gfdelarue/testkit/workspace`.
|
|
655
599
|
|
|
656
|
-
The run bundle is also the boundary for Testkit Cloud. The local runner never
|
|
657
|
-
reads regression state or mutates GitHub. See
|
|
658
|
-
[`docs/testkit-cloud-boundary.md`](docs/testkit-cloud-boundary.md) for ownership,
|
|
659
|
-
ingestion, and migration rules.
|
|
660
|
-
|
|
661
|
-
Use `testkit cloud snapshot` to read versioned Cloud regression state. Apply
|
|
662
|
-
reviewed mutations from a file with `testkit cloud apply --file commands.json`;
|
|
663
|
-
the file must contain a schema version, idempotency key, expected snapshot
|
|
664
|
-
version, and commands.
|
|
665
|
-
|
|
666
600
|
## Authoring
|
|
667
601
|
|
|
668
602
|
### Node setup beside a test
|
|
@@ -1039,7 +973,7 @@ identifier plus canonical metadata such as:
|
|
|
1039
973
|
- `dependsOn`
|
|
1040
974
|
- `displayName`
|
|
1041
975
|
|
|
1042
|
-
|
|
976
|
+
Discovery output has no cross-run counts, last outcomes, durations, or trends.
|
|
1043
977
|
See the [status and history migration](https://github.com/gfdelarue/testkit/blob/master/docs/release-notes.md#local-status-and-history-removal)
|
|
1044
978
|
for the removed discovery fields and consumer migration.
|
|
1045
979
|
|
|
@@ -21,6 +21,8 @@ export interface EngineExecutionLifecycle {
|
|
|
21
21
|
readonly signal: AbortSignal;
|
|
22
22
|
}
|
|
23
23
|
export interface EngineExecutionLease extends ExecutionEnvironmentLease {
|
|
24
|
+
readonly engineCacheDirectory?: string;
|
|
25
|
+
readonly engineWorkspaceDirectory?: string;
|
|
24
26
|
readonly leaseDir: string;
|
|
25
27
|
readonly leaseId: string;
|
|
26
28
|
}
|
|
@@ -1 +1 @@
|
|
|
1
|
-
import U from"node:fs";import F from"node:path";import{EngineTaskDeadlineExceededError as Me}from"../../engines/api/index.js";import"../../domain/identifiers.js";import"../../domain/test-outcomes.js";import{resolveEngineProfile as Ie}from"../../engines/catalog.js";import{BUILT_IN_ENGINE_ORCHESTRATION as be}from"./engine-orchestration.js";import{persistTaskArtifacts as Oe}from"../../evidence/index.js";import{createEvidenceLedger as se,persistTrackedEngineEvidence as ae,requireTaskArtifacts as Ne,requireTaskEvidenceLedger as _e}from"./engine-evidence-persistence.js";import{applyConsumerNodeSetupCleanupFailures as ce,cleanupPreparedNodeSetups as ue}from"./engine-execution-cleanup.js";import{buildBridgeError as Re,EngineExecutionBridgeError as Fe,executeManagedEngineRequest as Ae,infraBridgeDisposition as q,managerFailureBridgeDisposition as Be,productBridgeDisposition as z,reporterBridgeDisposition as G,resolveBridgeDisposition as de,runtimeNodeSetupBridgeDisposition as Ce}from"./engine-execution-failures.js";import{captureEngineReporterCallback as fe,DEFAULT_ENGINE_REPORTER_CALLBACK_TIMEOUT_MS as $e,invokeEngineReporterCallback as le}from"./engine-execution-reporting.js";import{createEngineExecutionObserver as xe}from"./engine-execution-observer.js";export{EngineExecutionBridgeError}from"./engine-execution-failures.js";import{waitForEndpointReadinessPreflight as Pe}from"./endpoint-readiness.js";import{collectCheckDetailsFromRuntimeArtifacts as Le,collectFailureDetailsFromRuntimeArtifacts as je}from"./failure-details.js";import{buildRuntimeFixtureManifest as Je}from"./runtime-fixtures.js";import{MAX_RUNTIME_NODE_SETUP_CLEANUP_TIMEOUT_MS as Ue,runRuntimeNodeSetup as qe}from"./setup/runtime-node-setup.js";export async function executeEngineTask(e,t,n,r,s=null){const i=t,u={signal:r.signal,timeoutMs:$e},m="batchTasks"in n?n.batchTasks:[n],w=m[0],S=t.services.get(w.targetName);if(!S)throw new Error(`Runtime instance is missing target config "${w.targetName}"`);const d=S.config;we(d.productDir,"product directory"),we(i.leaseDir,"runtime lease directory");const D=Ie(w.testType,w.engineId),M=be.resolve(D),O=Xe(d.testkit.execution.fileTimeoutSeconds),ke=Date.now(),V=pe(ke,O),B=Ge(t),N=Object.hasOwn(B,w.serviceName)?B[w.serviceName]:null;if(M.endpointReadinessOwner==="orchestrator-http"){if(N===null)throw new Error(`Service "${d.name}" requires local.baseUrl for ${D.id} suites`);const C=Date.now(),v=H(r.signal,V);let y;try{y=await Pe({baseUrl:N,serviceName:d.name,lifecycle:r,signal:v.signal,options:Ye(Qe(V))})}finally{v.dispose()}if(!y.ok){const I=r.isStopRequested()||r.signal.aborted,k=Date.now(),c=buildEndpointReadinessOutcomes(m,n,y,I,C,k),a=[];let g=null;for(const E of c){const l=a.length;await fe(a,"taskStarted",()=>s?.taskStarted?.(E.task,d),u),g===null&&a.length>l&&(g=String(E.task.id)),await fe(a,"taskFinished",()=>s?.taskFinished?.(E.task,E),u),g===null&&a.length>l&&(g=String(E.task.id))}if(a.length>0){const E=a.length===1?a[0]:new AggregateError(a,"Engine readiness reporting failed");throw new Fe(`${y.reason}; ${a.map(l=>l.message).join("; ")}`,E,de(a,[],!1,I?{classification:void 0,notifyRuntimeError:!1,runtimeImpact:"none",status:"cancelled"}:G("fail-graph")),new Map(m.map(l=>[String(l.id),[]])),new Map(c.map(l=>[String(l.task.id),l])),g)}return{outcomes:c}}}return e.withEngine(D.engineId,r.signal,async C=>{const v=He(i,d.productDir);U.mkdirSync(v.temporaryDirectory,{recursive:!0}),U.mkdirSync(v.artifactDirectory,{recursive:!0});const y=new Map(m.map(p=>[String(p.id),[]])),I=new Map(m.map(p=>[String(p.id),se()])),k=new Map,c=[];let a=z(),g=!1;const E=[],l=[],W=[];let X=null;const Q=new Set,$=async(p,o)=>{const f=String(p.id);Q.has(f)||(Q.add(f),await le("taskFinished",()=>s?.taskFinished?.(p,o),u))};try{for(const[p,o]of m.entries()){a=G();const f=String(o.id);X=f,await le("taskStarted",()=>s?.taskStarted?.(o,d),u),a=z();const Z=M.buildRuntimeEnvironment({baseUrl:N,database:S.database,targetConfig:d,task:o}),_=pe(Date.now(),O),ee=H(r.signal,_);let b;try{b=await qe({cleanupTimeoutMs:Math.min(O,Ue),deadlineMs:_,targetConfig:d,task:o,lease:i,runtimeEnv:Z,signal:ee.signal})}catch(h){throw a=Ce(h),h}finally{ee.dispose()}E.push({cleanup:b.cleanup,taskId:f}),a=q();for(const h of b.artifacts){const T=Oe(d.productDir,o,[h]);y.get(String(o.id))?.push(...T)}a=z();const R={baseUrl:N,lease:i,profile:D,runtimeEnvironment:Z,targetConfig:d,task:o},te=M.resolveConfiguration(R),ve={...te===void 0?{}:{configuration:te},environment:M.buildEnvironment(R),fixtures:nt(Je(d.productDir,o.file),`fixture manifest for ${o.file}`),setupData:he(b.setupData,`node setup data for ${o.file}`),supportModules:M.resolveSupportModules(R)},ye=ze(R);a=q();const De=c.length,ne=se(),Te=xe({artifactReferencesByTaskId:y,completedOutcomesByTaskId:k,errors:c,executableTask:n,invocationEvidenceLedger:ne,productDir:d.productDir,taskBatchIndex:p,taskEvidenceLedgers:I,tasks:[o],tasksById:new Map([[f,o]]),workspaceArtifactDirectory:v.artifactDirectory},ge),x=H(r.signal,_);try{const h={invocationId:Ve(r,i,o),observer:Te,profile:D,signal:x.signal,target:{endpoints:B,inputs:Ke()},tasks:[{deadlineMs:_,id:f,file:o.file,serviceName:o.serviceName,suiteName:o.suiteName,testType:o.testType,timeoutMs:O,inputs:ve,orchestration:ye}],workspace:v};let T;try{T=await Ae(C,h)}catch(oe){throw a=Be(r.signal,x.signal,oe),oe}const re=mapEngineOutcomes([o],T)[0],j=Ne(y,f),Se=_e(I,f);ae(re.outcome.evidence,{artifactRoot:v.artifactDirectory,errors:c,ledger:Se,mode:"reconcile",productDir:d.productDir,references:j,scope:"task",task:o});const J=ge(re.outcome,o,j,n,p);k.set(f,J),J.infraFailed&&J.blockedByRuntime&&(a=q("fail-graph"));const ie=et(T);tt(ie)&&ae(ie,{artifactRoot:v.artifactDirectory,errors:c,ledger:ne,mode:"reconcile",productDir:d.productDir,references:j,scope:"invocation",task:o})}finally{x.dispose()}c.length>De&&(g=!0),E.pop();const P=await ue([{cleanup:b.cleanup,taskId:f}]);if(P.length>0){l.push(...P),ce(k,P);const h=k.get(f);if(h!==void 0)try{await $(o,h)}catch(T){c.push(A(T))}break}if(c.length>0)break;const L=k.get(f);if(L===void 0){c.push(new Error(`Engine task ${f} completed without a mapped outcome`)),g=!0;break}W.push(L),a=G();try{await $(o,L)}catch(h){c.push(A(h)),g=!0;break}}}catch(p){const o=c.length;c.push(A(p)),(o>0||a.classification==="infra")&&(g=!0)}const Y=await ue(E);l.push(...Y),ce(k,Y);for(const p of m){const o=k.get(String(p.id));if(o!==void 0)try{await $(p,o)}catch(f){c.push(A(f))}}if(c.length>0||l.length>0)throw Re(c,l,de(c,l,g,a),y,k,X);return{outcomes:W}})}function ze(e){const t=e.targetConfig.testkit.mode;if(t!==void 0&&(typeof t!="string"||t.trim().length===0))throw new Error("Testkit execution mode must be a non-empty string");const n=e.targetConfig.runtimeId;return{lease:{directory:e.lease.leaseDir,id:String(e.lease.leaseId)},mode:t??"test",...n==null?{}:{runtimeId:String(n)}}}export function mapEngineOutcomes(e,t){const n=new Map(e.map(i=>[String(i.id),i])),r=new Map;for(const i of t.outcomes){const u=n.get(i.taskId);if(!u)throw new Error(`Engine manager returned an outcome for unknown task ${i.taskId}`);if(r.has(i.taskId))throw new Error(`Engine manager returned duplicate outcomes for task ${i.taskId}`);r.set(i.taskId,{task:u,outcome:i})}const s=e.map(i=>String(i.id)).filter(i=>!r.has(i));if(s.length>0)throw new Error(`Engine manager did not return outcomes for task(s): ${s.join(", ")}`);return e.map(i=>{const u=r.get(String(i.id));if(!u)throw new Error("Engine outcome mapping invariant failed");return u})}export function buildEndpointReadinessOutcomes(e,t,n,r,s,i){const u=Math.max(0,i-s);return e.map((m,w)=>({task:m,status:r?"cancelled":"not_run",error:null,reason:n.reason,infraFailed:!r,blockedByRuntime:!r,durationMs:w===0?u:0,startedAt:s,finishedAt:i,artifacts:[],failureDetails:r?[]:[...n.failureDetails],checkDetails:[],...Ee(t,m)}))}function Ge(e){return Object.freeze(Object.fromEntries([...e.services.values()].flatMap(({config:t,baseUrl:n})=>typeof n=="string"&&n.length>0?[[t.name,n]]:[])))}function He(e,t){const n=F.join(e.leaseDir,"engine-execution");return{rootDirectory:t,temporaryDirectory:F.join(n,"temporary"),artifactDirectory:F.join(n,"artifacts")}}function Ke(){return{environment:{},fixtures:{},supportModules:{}}}function Ve(e,t,n){return[e.runId,t.leaseId,We(n)].join(":")}function We(e){return"batchId"in e?e.batchId:String(e.id)}function Xe(e){const t=e*1e3;if(!Number.isSafeInteger(t)||t<=0)throw new Error("Engine task timeout must be a positive safe integer");return t}function pe(e,t){const n=e+t;if(!Number.isSafeInteger(n)||n<=e)throw new Error("Engine task deadline must be a positive safe integer");return n}function Qe(e){return Math.max(1,e-Date.now())}function Ye(e){const t=me(process.env.TESTKIT_HTTP_ENDPOINT_PREFLIGHT_TIMEOUT_MS,3e4),n=me(process.env.TESTKIT_HTTP_ENDPOINT_PREFLIGHT_ATTEMPT_TIMEOUT_MS,5e3);return{timeoutMs:Math.min(t,e),attemptTimeoutMs:Math.min(n,e)}}function me(e,t){const n=Number(e);return Number.isInteger(n)&&n>0?n:t}function H(e,t){const n=new AbortController;let r=null;const s=()=>{n.abort(e.reason)},i=()=>{const u=t-Date.now();if(u<=0){n.abort(new Me(t));return}r=setTimeout(i,Math.min(u,2147483647)),r.unref()};return e.aborted?s():(e.addEventListener("abort",s,{once:!0}),i()),{signal:n.signal,dispose(){r!==null&&clearTimeout(r),e.removeEventListener("abort",s)}}}function ge(e,t,n,r,s){const i=je(e.evidence.artifacts),u=i.some(D=>t.engineId==="node-vitest"&&D.kind==="vitest-test"),m=[...e.failure&&!u?[Ze(e.failure)]:[],...i],w=Le(e.evidence.artifacts),S=e.status==="not_run",d=S||e.failure?.classification==="infra";return{task:t,status:e.status,error:e.status==="failed"?e.failure?.message??"Engine task failed without a message":null,reason:e.status==="failed"||e.status==="passed"?null:e.failure?.message??`Engine task ${e.status}`,infraFailed:d,blockedByRuntime:S,durationMs:e.durationMs,startedAt:e.startedAtMs,finishedAt:e.finishedAtMs,artifacts:n,failureDetails:m,checkDetails:w,...Ee(r,t,s)}}function Ee(e,t,n){if(!("batchTasks"in e))return{};const r=n??e.batchTasks.findIndex(s=>s.id===t.id&&s.file===t.file);if(r<0)throw new Error(`Task ${String(t.id)} is missing from its engine batch`);return{runtimeBatch:{id:e.batchId,index:r,fileCount:e.batchTasks.length,files:e.batchTasks.map(s=>s.file)}}}function Ze(e){const t=e.code??"ENGINE_TASK_FAILED",n={code:t,details:e.details===void 0?null:rt(e.details)};return{kind:"engine-failure",key:t,title:t,...e.classification===void 0?{}:{classification:e.classification},count:1,message:e.message,actual:n,...e.stack===void 0?{}:{stack:e.stack}}}function et(e){return{artifacts:K(e.evidence.artifacts,e.outcomes.flatMap(t=>t.evidence.artifacts)),logs:K(e.evidence.logs,e.outcomes.flatMap(t=>t.evidence.logs)),metrics:K(e.evidence.metrics,e.outcomes.flatMap(t=>t.evidence.metrics)),...e.evidence.native===void 0?{}:{native:e.evidence.native}}}function K(e,t){const n=new Map;for(const r of t){const s=JSON.stringify(r);n.set(s,(n.get(s)??0)+1)}return e.filter(r=>{const s=JSON.stringify(r),i=n.get(s)??0;return i===0?!0:(n.set(s,i-1),!1)})}function tt(e){return e.artifacts.length>0||e.logs.length>0||e.metrics.length>0||e.native!==void 0}function nt(e,t){const n=he(e,t);if(n===null||Array.isArray(n)||typeof n!="object")throw new TypeError(`${t} must contain a JSON object`);return n}function he(e,t){try{const n=JSON.stringify(e);if(typeof n!="string")throw new TypeError(`${t} must be JSON-serializable`);return JSON.parse(n)}catch(n){throw new TypeError(`${t} must be JSON-serializable`,{cause:n})}}function rt(e){return JSON.parse(JSON.stringify(e))}function we(e,t){if(!F.isAbsolute(e))throw new Error(`Engine execution ${t} must be absolute`);if(!U.statSync(e).isDirectory())throw new Error(`Engine execution ${t} must be a directory`)}function A(e){return e instanceof Error?e:new Error(String(e))}
|
|
1
|
+
import U from"node:fs";import F from"node:path";import{EngineTaskDeadlineExceededError as Me}from"../../engines/api/index.js";import"../../domain/identifiers.js";import"../../domain/test-outcomes.js";import{resolveEngineProfile as Ie}from"../../engines/catalog.js";import{BUILT_IN_ENGINE_ORCHESTRATION as be}from"./engine-orchestration.js";import{persistTaskArtifacts as Oe}from"../../evidence/index.js";import{createEvidenceLedger as se,persistTrackedEngineEvidence as ae,requireTaskArtifacts as Ne,requireTaskEvidenceLedger as _e}from"./engine-evidence-persistence.js";import{applyConsumerNodeSetupCleanupFailures as ce,cleanupPreparedNodeSetups as ue}from"./engine-execution-cleanup.js";import{buildBridgeError as Re,EngineExecutionBridgeError as Fe,executeManagedEngineRequest as Ae,infraBridgeDisposition as q,managerFailureBridgeDisposition as Ce,productBridgeDisposition as z,reporterBridgeDisposition as G,resolveBridgeDisposition as de,runtimeNodeSetupBridgeDisposition as Be}from"./engine-execution-failures.js";import{captureEngineReporterCallback as fe,DEFAULT_ENGINE_REPORTER_CALLBACK_TIMEOUT_MS as $e,invokeEngineReporterCallback as le}from"./engine-execution-reporting.js";import{createEngineExecutionObserver as xe}from"./engine-execution-observer.js";export{EngineExecutionBridgeError}from"./engine-execution-failures.js";import{waitForEndpointReadinessPreflight as Pe}from"./endpoint-readiness.js";import{collectCheckDetailsFromRuntimeArtifacts as Le,collectFailureDetailsFromRuntimeArtifacts as je}from"./failure-details.js";import{buildRuntimeFixtureManifest as Je}from"./runtime-fixtures.js";import{MAX_RUNTIME_NODE_SETUP_CLEANUP_TIMEOUT_MS as Ue,runRuntimeNodeSetup as qe}from"./setup/runtime-node-setup.js";export async function executeEngineTask(e,t,n,r,s=null){const i=t,u={signal:r.signal,timeoutMs:$e},m="batchTasks"in n?n.batchTasks:[n],w=m[0],S=t.services.get(w.targetName);if(!S)throw new Error(`Runtime instance is missing target config "${w.targetName}"`);const d=S.config;we(d.productDir,"product directory"),we(i.leaseDir,"runtime lease directory");const v=Ie(w.testType,w.engineId),M=be.resolve(v),O=Xe(d.testkit.execution.fileTimeoutSeconds),ke=Date.now(),V=pe(ke,O),C=Ge(t),N=Object.hasOwn(C,w.serviceName)?C[w.serviceName]:null;if(M.endpointReadinessOwner==="orchestrator-http"){if(N===null)throw new Error(`Service "${d.name}" requires local.baseUrl for ${v.id} suites`);const B=Date.now(),y=H(r.signal,V);let D;try{D=await Pe({baseUrl:N,serviceName:d.name,lifecycle:r,signal:y.signal,options:Ye(Qe(V))})}finally{y.dispose()}if(!D.ok){const I=r.isStopRequested()||r.signal.aborted,k=Date.now(),c=buildEndpointReadinessOutcomes(m,n,D,I,B,k),a=[];let g=null;for(const E of c){const l=a.length;await fe(a,"taskStarted",()=>s?.taskStarted?.(E.task,d),u),g===null&&a.length>l&&(g=String(E.task.id)),await fe(a,"taskFinished",()=>s?.taskFinished?.(E.task,E),u),g===null&&a.length>l&&(g=String(E.task.id))}if(a.length>0){const E=a.length===1?a[0]:new AggregateError(a,"Engine readiness reporting failed");throw new Fe(`${D.reason}; ${a.map(l=>l.message).join("; ")}`,E,de(a,[],!1,I?{classification:void 0,notifyRuntimeError:!1,runtimeImpact:"none",status:"cancelled"}:G("fail-graph")),new Map(m.map(l=>[String(l.id),[]])),new Map(c.map(l=>[String(l.task.id),l])),g)}return{outcomes:c}}}return e.withEngine(v.engineId,r.signal,async B=>{const y=He(i,d.productDir);U.mkdirSync(y.temporaryDirectory,{recursive:!0}),U.mkdirSync(y.artifactDirectory,{recursive:!0});const D=new Map(m.map(p=>[String(p.id),[]])),I=new Map(m.map(p=>[String(p.id),se()])),k=new Map,c=[];let a=z(),g=!1;const E=[],l=[],W=[];let X=null;const Q=new Set,$=async(p,o)=>{const f=String(p.id);Q.has(f)||(Q.add(f),await le("taskFinished",()=>s?.taskFinished?.(p,o),u))};try{for(const[p,o]of m.entries()){a=G();const f=String(o.id);X=f,await le("taskStarted",()=>s?.taskStarted?.(o,d),u),a=z();const Z=M.buildRuntimeEnvironment({baseUrl:N,database:S.database,targetConfig:d,task:o}),_=pe(Date.now(),O),ee=H(r.signal,_);let b;try{b=await qe({cleanupTimeoutMs:Math.min(O,Ue),deadlineMs:_,targetConfig:d,task:o,lease:i,runtimeEnv:Z,signal:ee.signal})}catch(h){throw a=Be(h),h}finally{ee.dispose()}E.push({cleanup:b.cleanup,taskId:f}),a=q();for(const h of b.artifacts){const T=Oe(d.productDir,o,[h]);D.get(String(o.id))?.push(...T)}a=z();const R={baseUrl:N,lease:i,profile:v,runtimeEnvironment:Z,targetConfig:d,task:o},te=M.resolveConfiguration(R),ye={...te===void 0?{}:{configuration:te},environment:M.buildEnvironment(R),fixtures:nt(Je(d.productDir,o.file),`fixture manifest for ${o.file}`),setupData:he(b.setupData,`node setup data for ${o.file}`),supportModules:M.resolveSupportModules(R)},De=ze(R);a=q();const ve=c.length,ne=se(),Te=xe({artifactReferencesByTaskId:D,completedOutcomesByTaskId:k,errors:c,executableTask:n,invocationEvidenceLedger:ne,productDir:d.productDir,taskBatchIndex:p,taskEvidenceLedgers:I,tasks:[o],tasksById:new Map([[f,o]]),workspaceArtifactDirectory:y.artifactDirectory},ge),x=H(r.signal,_);try{const h={invocationId:Ve(r,i,o),observer:Te,profile:v,signal:x.signal,target:{endpoints:C,inputs:Ke()},tasks:[{deadlineMs:_,id:f,file:o.file,serviceName:o.serviceName,suiteName:o.suiteName,testType:o.testType,timeoutMs:O,inputs:ye,orchestration:De}],workspace:y};let T;try{T=await Ae(B,h)}catch(oe){throw a=Ce(r.signal,x.signal,oe),oe}const re=mapEngineOutcomes([o],T)[0],j=Ne(D,f),Se=_e(I,f);ae(re.outcome.evidence,{artifactRoot:y.artifactDirectory,errors:c,ledger:Se,mode:"reconcile",productDir:d.productDir,references:j,scope:"task",task:o});const J=ge(re.outcome,o,j,n,p);k.set(f,J),J.infraFailed&&J.blockedByRuntime&&(a=q("fail-graph"));const ie=et(T);tt(ie)&&ae(ie,{artifactRoot:y.artifactDirectory,errors:c,ledger:ne,mode:"reconcile",productDir:d.productDir,references:j,scope:"invocation",task:o})}finally{x.dispose()}c.length>ve&&(g=!0),E.pop();const P=await ue([{cleanup:b.cleanup,taskId:f}]);if(P.length>0){l.push(...P),ce(k,P);const h=k.get(f);if(h!==void 0)try{await $(o,h)}catch(T){c.push(A(T))}break}if(c.length>0)break;const L=k.get(f);if(L===void 0){c.push(new Error(`Engine task ${f} completed without a mapped outcome`)),g=!0;break}W.push(L),a=G();try{await $(o,L)}catch(h){c.push(A(h)),g=!0;break}}}catch(p){const o=c.length;c.push(A(p)),(o>0||a.classification==="infra")&&(g=!0)}const Y=await ue(E);l.push(...Y),ce(k,Y);for(const p of m){const o=k.get(String(p.id));if(o!==void 0)try{await $(p,o)}catch(f){c.push(A(f))}}if(c.length>0||l.length>0)throw Re(c,l,de(c,l,g,a),D,k,X);return{outcomes:W}})}function ze(e){const t=e.targetConfig.testkit.mode;if(t!==void 0&&(typeof t!="string"||t.trim().length===0))throw new Error("Testkit execution mode must be a non-empty string");const n=e.targetConfig.runtimeId;return{lease:{directory:e.lease.leaseDir,id:String(e.lease.leaseId)},mode:t??"test",...n==null?{}:{runtimeId:String(n)}}}export function mapEngineOutcomes(e,t){const n=new Map(e.map(i=>[String(i.id),i])),r=new Map;for(const i of t.outcomes){const u=n.get(i.taskId);if(!u)throw new Error(`Engine manager returned an outcome for unknown task ${i.taskId}`);if(r.has(i.taskId))throw new Error(`Engine manager returned duplicate outcomes for task ${i.taskId}`);r.set(i.taskId,{task:u,outcome:i})}const s=e.map(i=>String(i.id)).filter(i=>!r.has(i));if(s.length>0)throw new Error(`Engine manager did not return outcomes for task(s): ${s.join(", ")}`);return e.map(i=>{const u=r.get(String(i.id));if(!u)throw new Error("Engine outcome mapping invariant failed");return u})}export function buildEndpointReadinessOutcomes(e,t,n,r,s,i){const u=Math.max(0,i-s);return e.map((m,w)=>({task:m,status:r?"cancelled":"not_run",error:null,reason:n.reason,infraFailed:!r,blockedByRuntime:!r,durationMs:w===0?u:0,startedAt:s,finishedAt:i,artifacts:[],failureDetails:r?[]:[...n.failureDetails],checkDetails:[],...Ee(t,m)}))}function Ge(e){return Object.freeze(Object.fromEntries([...e.services.values()].flatMap(({config:t,baseUrl:n})=>typeof n=="string"&&n.length>0?[[t.name,n]]:[])))}function He(e,t){const n=e.engineWorkspaceDirectory??F.join(e.leaseDir,"engine-execution");return{rootDirectory:t,...e.engineCacheDirectory===void 0?{}:{cacheDirectory:e.engineCacheDirectory},temporaryDirectory:F.join(n,"temporary"),artifactDirectory:F.join(n,"artifacts")}}function Ke(){return{environment:{},fixtures:{},supportModules:{}}}function Ve(e,t,n){return[e.runId,t.leaseId,We(n)].join(":")}function We(e){return"batchId"in e?e.batchId:String(e.id)}function Xe(e){const t=e*1e3;if(!Number.isSafeInteger(t)||t<=0)throw new Error("Engine task timeout must be a positive safe integer");return t}function pe(e,t){const n=e+t;if(!Number.isSafeInteger(n)||n<=e)throw new Error("Engine task deadline must be a positive safe integer");return n}function Qe(e){return Math.max(1,e-Date.now())}function Ye(e){const t=me(process.env.TESTKIT_HTTP_ENDPOINT_PREFLIGHT_TIMEOUT_MS,3e4),n=me(process.env.TESTKIT_HTTP_ENDPOINT_PREFLIGHT_ATTEMPT_TIMEOUT_MS,5e3);return{timeoutMs:Math.min(t,e),attemptTimeoutMs:Math.min(n,e)}}function me(e,t){const n=Number(e);return Number.isInteger(n)&&n>0?n:t}function H(e,t){const n=new AbortController;let r=null;const s=()=>{n.abort(e.reason)},i=()=>{const u=t-Date.now();if(u<=0){n.abort(new Me(t));return}r=setTimeout(i,Math.min(u,2147483647)),r.unref()};return e.aborted?s():(e.addEventListener("abort",s,{once:!0}),i()),{signal:n.signal,dispose(){r!==null&&clearTimeout(r),e.removeEventListener("abort",s)}}}function ge(e,t,n,r,s){const i=je(e.evidence.artifacts),u=i.some(v=>t.engineId==="node-vitest"&&v.kind==="vitest-test"),m=[...e.failure&&!u?[Ze(e.failure)]:[],...i],w=Le(e.evidence.artifacts),S=e.status==="not_run",d=S||e.failure?.classification==="infra";return{task:t,status:e.status,error:e.status==="failed"?e.failure?.message??"Engine task failed without a message":null,reason:e.status==="failed"||e.status==="passed"?null:e.failure?.message??`Engine task ${e.status}`,infraFailed:d,blockedByRuntime:S,durationMs:e.durationMs,startedAt:e.startedAtMs,finishedAt:e.finishedAtMs,artifacts:n,failureDetails:m,checkDetails:w,...Ee(r,t,s)}}function Ee(e,t,n){if(!("batchTasks"in e))return{};const r=n??e.batchTasks.findIndex(s=>s.id===t.id&&s.file===t.file);if(r<0)throw new Error(`Task ${String(t.id)} is missing from its engine batch`);return{runtimeBatch:{id:e.batchId,index:r,fileCount:e.batchTasks.length,files:e.batchTasks.map(s=>s.file)}}}function Ze(e){const t=e.code??"ENGINE_TASK_FAILED",n={code:t,details:e.details===void 0?null:rt(e.details)};return{kind:"engine-failure",key:t,title:t,...e.classification===void 0?{}:{classification:e.classification},count:1,message:e.message,actual:n,...e.stack===void 0?{}:{stack:e.stack}}}function et(e){return{artifacts:K(e.evidence.artifacts,e.outcomes.flatMap(t=>t.evidence.artifacts)),logs:K(e.evidence.logs,e.outcomes.flatMap(t=>t.evidence.logs)),metrics:K(e.evidence.metrics,e.outcomes.flatMap(t=>t.evidence.metrics)),...e.evidence.native===void 0?{}:{native:e.evidence.native}}}function K(e,t){const n=new Map;for(const r of t){const s=JSON.stringify(r);n.set(s,(n.get(s)??0)+1)}return e.filter(r=>{const s=JSON.stringify(r),i=n.get(s)??0;return i===0?!0:(n.set(s,i-1),!1)})}function tt(e){return e.artifacts.length>0||e.logs.length>0||e.metrics.length>0||e.native!==void 0}function nt(e,t){const n=he(e,t);if(n===null||Array.isArray(n)||typeof n!="object")throw new TypeError(`${t} must contain a JSON object`);return n}function he(e,t){try{const n=JSON.stringify(e);if(typeof n!="string")throw new TypeError(`${t} must be JSON-serializable`);return JSON.parse(n)}catch(n){throw new TypeError(`${t} must be JSON-serializable`,{cause:n})}}function rt(e){return JSON.parse(JSON.stringify(e))}function we(e,t){if(!F.isAbsolute(e))throw new Error(`Engine execution ${t} must be absolute`);if(!U.statSync(e).isDirectory())throw new Error(`Engine execution ${t} must be a directory`)}function A(e){return e instanceof Error?e:new Error(String(e))}
|
|
@@ -0,0 +1,8 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Shares compiled code, never evaluated modules, between fresh engine processes.
|
|
3
|
+
* The caller owns the cache root and must retain it until its processes exit.
|
|
4
|
+
* Cache failures are optional optimization failures and must not prevent tests.
|
|
5
|
+
*/
|
|
6
|
+
export declare function getNodeStartupCacheEnvironment(cacheDirectory: string | undefined, consumerEnvironment: Readonly<Record<string, string>>, options?: Readonly<{
|
|
7
|
+
coverageEnabled?: boolean;
|
|
8
|
+
}>): Promise<Readonly<Record<string, string>>>;
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
import n from"node:fs/promises";import u from"node:path";import{makeTempDirAsync as m}from"../../process/files/temp-dir.js";const r=new Map,C=128,y=new Set(["NODE_COMPILE_CACHE","NODE_DISABLE_COMPILE_CACHE","NODE_V8_COVERAGE"]);export async function getNodeStartupCacheEnvironment(t,a,s={}){if(t===void 0||!u.isAbsolute(t)||s.coverageEnabled===!0||Object.keys(a).some(o=>y.has(o.toUpperCase())))return{};try{await n.mkdir(t,{mode:448,recursive:!0});const o=await n.lstat(t);if(!o.isDirectory()||o.isSymbolicLink())return{};const p=await n.realpath(t),c=`${p}\0${E(o)}`;let i=r.get(c);if(i!==void 0){const e=await i,d=await n.lstat(e.directory).catch(()=>null);if(d!==null&&f(d)&&E(d)===e.identity&&await n.realpath(e.directory)===e.directory)return{NODE_COMPILE_CACHE:u.toNamespacedPath(e.directory)};r.delete(c)}if(i=r.get(c),i===void 0&&(i=O(p),r.set(c,i),r.size>C)){const e=r.keys().next().value;e!==void 0&&r.delete(e)}const l=await i;return{NODE_COMPILE_CACHE:u.toNamespacedPath(l.directory)}}catch{return{}}}async function O(t){const a=await m("node-compile-",{parentDir:t}),s=await n.lstat(a);if(!f(s))throw new Error("Node startup cache directory is not private");return{directory:a,identity:E(s)}}function E(t){return`${t.dev.toString()}:${t.ino.toString()}:${t.birthtimeMs.toString()}`}function f(t){return t.isDirectory()&&!t.isSymbolicLink()&&(process.platform==="win32"||(t.mode&63)===0&&t.uid===process.getuid?.())}
|
|
@@ -57,6 +57,8 @@ export interface EngineExecutionTarget {
|
|
|
57
57
|
}
|
|
58
58
|
export interface EngineExecutionWorkspace {
|
|
59
59
|
readonly artifactDirectory: string;
|
|
60
|
+
/** Optional caller-owned compilation cache, removed after all executions finish. */
|
|
61
|
+
readonly cacheDirectory?: string;
|
|
60
62
|
readonly rootDirectory: string;
|
|
61
63
|
readonly temporaryDirectory: string;
|
|
62
64
|
}
|
|
@@ -168,7 +168,14 @@ function observedRealpath(file) {
|
|
|
168
168
|
recordResolutionWork("resolver.realpath");
|
|
169
169
|
// Match policy inspection on Windows: native realpath expands 8.3 names and
|
|
170
170
|
// changes their spelling, mixing physical paths with the inspected roots.
|
|
171
|
-
|
|
171
|
+
if (process.platform === "win32") return fs.realpathSync(file);
|
|
172
|
+
try {
|
|
173
|
+
return fs.realpathSync.native(file);
|
|
174
|
+
} catch {
|
|
175
|
+
// Keep Node's portable implementation for hosts where native realpath
|
|
176
|
+
// is unavailable, including minimal Linux environments without /proc.
|
|
177
|
+
return fs.realpathSync(file);
|
|
178
|
+
}
|
|
172
179
|
}
|
|
173
180
|
|
|
174
181
|
function observedRead(file, encoding) {
|
|
@@ -1 +1 @@
|
|
|
1
|
-
import{makeTempDirAsync as
|
|
1
|
+
import{makeTempDirAsync as h}from"../../../process/files/temp-dir.js";import{classifyManagedProcessResult as H,terminalOutcome as E,processOutputEvidence as J,emptyEvidence as R,failureFromError as p,firstOutputLine as x,processExitFailure as Y,toError as M,executeEngineTasks as G,managedProcessFailure as Q,taskTimeoutFailure as X}from"../../adapters/execution.js";import{requireRootDirectory as Z,requireAbsoluteWorkspacePath as P,requireConsumerFile as L,safeSegment as A}from"../../adapters/workspace.js";import f from"node:fs/promises";import c from"node:path";import{pathToFileURL as q}from"node:url";import{isEngineTaskDeadlineExceededReason as tt,resolveEngineTaskInputs as et}from"../../api/index.js";import{createInvocationProcessEnvironment as it,runManagedProcess as rt}from"../../../process/index.js";import{preflightNodeVitestInstallation as nt}from"../../installations/node-vitest.js";import{collectNodeVitestEvidence as ot}from"./evidence.js";import{assertNodeVitestSpawnEnvironmentBudget as C,NODE_VITEST_RUNTIME_INPUT_FILE_ENV as at,NODE_VITEST_RUNTIME_INPUT_SHA256_ENV as st,selectNodeVitestForwardedEnvironment as ct,verifyOwnedNodeVitestRuntimeInput as ut,writeOwnedNodeVitestRuntimeInput as dt}from"../runtime/runtime-input.js";import{writeNodeVitestWorkspace as lt}from"./workspace.js";import{engineEvidenceIdentity as pt}from"../../adapters/evidence-identity.js";import{getNodeStartupCacheEnvironment as ft}from"../../adapters/node-startup-cache.js";const I={codePrefix:"NODE_VITEST",label:"Node/Vitest"};class b extends Error{constructor(e){const i=M(e);super(i.message,{cause:i}),this.name="NodeVitestConsumerPreparationError"}}export const nodeVitestExecutorDependencies={removeOwnedDirectory:t=>f.rm(t,{force:!0,recursive:!0}),writeRuntimeInput:dt,writeWorkspace:lt};export async function executeNodeVitestEngine(t,e,i,r=nodeVitestExecutorDependencies){const n=await mt(t,e,r,i);return G(t,n.tasks,I,(a,o)=>Et(t,n,a,o),yt(t,n.installation))}async function mt(t,e,i,r){const n=t.profile.id,a=e.packages.vitest.cli,o=await Z(t.workspace.rootDirectory);P(t.workspace.temporaryDirectory,"temporary directory"),P(t.workspace.artifactDirectory,"artifact directory"),await f.mkdir(t.workspace.temporaryDirectory,{recursive:!0}),await f.mkdir(t.workspace.artifactDirectory,{recursive:!0});const d=await f.realpath(t.workspace.temporaryDirectory),T=await f.realpath(t.workspace.artifactDirectory),l=Object.freeze({...e.authoringEntrypoints,cli:e.packages.vitest.cli,packages:e.packages}),m=await h(`node-vitest-${A(t.invocationId)}-`,{parentDir:d});r.defer(()=>i.removeOwnedDirectory(m));const U=await it(m),g=[];for(const[B,s]of t.tasks.entries()){const v=et(t,s),S=`${B.toString().padStart(3,"0")}-${A(s.id)}`,u=c.join(m,S),y=await h(`${S}-`,{parentDir:T}),w=c.join(u,"testkit-vitest-report.json"),$=c.join(y,"data","manifest.json"),z=c.join(u,"setup.mjs"),O=c.join(u,"input-bootstrap.mjs"),F=c.join(u,"resolution-bootstrap.cjs"),V=c.join(u,"vitest.config.mjs");try{await f.mkdir(u,{mode:448,recursive:!1});const D=await N(()=>L(o,s.file,"Node/Vitest task file")),k=await N(()=>gt(o,v)),W=await N(()=>vt(t,n,s,v)),_=await i.writeRuntimeInput(u,W);await i.writeWorkspace({report:{identity:{engineId:"node-vitest",invocationId:t.invocationId,taskId:s.id,sourceFile:s.file},reportFile:w,workspaceDirectory:u},configFile:k.testkitConfig,entrypoints:l,inputBootstrapFile:O,resolutionBootstrapFile:F,rootDirectory:o,runtimeInput:{expectedFile:_.file,expectedInvocationId:t.invocationId,expectedProfileId:n,expectedSha256:_.sha256,expectedSizeBytes:_.sizeBytes,expectedTaskId:s.id},setupFile:z,sourceFile:D,timeoutMs:Math.max(1,s.deadlineMs-Date.now()),tsconfigFile:k.tsconfig,vitestConfigFile:V}),g.push({artifactDirectory:y,inputs:v,task:s,preparationFailure:null,prepared:{dataManifestFile:$,inputBootstrapFile:O,reportFile:w,resolutionBootstrapFile:F,runtimeInput:_,vitestConfigFile:V}})}catch(D){g.push({artifactDirectory:y,inputs:v,task:s,preparationFailure:Dt(D),prepared:null})}}return Object.freeze({cliFile:a,installation:e,invocationEnvironment:U.environment,profileId:n,rootDirectory:o,tasks:Object.freeze(g)})}async function Et(t,e,i,r){try{await ut(i.prepared.runtimeInput)}catch(l){return E(i.task.id,"failed",r,R(),{...p(l,"NODE_VITEST_INPUT_INVALID"),classification:"infra"})}const n=await Tt(t,e,i),a=i.task.deadlineMs-Date.now();if(a<=0)return E(i.task.id,"failed",r,R(),X(i.task.timeoutMs,I));const o=await rt({args:[e.cliFile,"run","--config",i.prepared.vitestConfigFile,"--configLoader","runner"],cwd:e.rootDirectory,displayLabel:"Node/Vitest",env:n,executable:process.execPath,signal:t.signal,timeoutMs:a});let d;try{d=await ot({identity:{engineId:"node-vitest",invocationId:t.invocationId,taskId:i.task.id,sourceFile:i.task.file},artifactDirectory:i.artifactDirectory,dataManifestFile:i.prepared.dataManifestFile,profileId:e.profileId,reportFile:i.prepared.reportFile,rootDirectory:e.rootDirectory,stderr:o.stderr,stdout:o.stdout})}catch(l){const m=J(o);return E(i.task.id,"failed",r,m,nodeVitestEvidenceCollectionFailure(o,l))}if(d.reportSummary===null&&o.exitCode!==0&&!t.signal.aborted&&!o.timedOut&&!o.outputLimitExceeded&&o.terminationError===null&&o.error===null)try{await nt(e.installation,{signal:t.signal})}catch(l){if(!t.signal.aborted)return E(i.task.id,"failed",r,d.evidence,{...p(l,"NODE_VITEST_INSTALLATION_BROKEN"),classification:"infra"})}const T=classifyNodeVitestProcessResult(o,d,i.task.timeoutMs,tt(t.signal.reason));return E(i.task.id,T.status,r,d.evidence,T.failure)}export function classifyNodeVitestProcessResult(t,e,i,r){const n=H(t,i,r,I);return n!==null?n:e.reportSummary?.numTotalTests===0?{failure:{code:"NODE_VITEST_NO_TESTS",message:"Vitest did not discover any tests for the task file"},status:"failed"}:It(e.reportSummary)?{failure:null,status:"skipped"}:t.exitCode!==0?Y(t,I,e.reportFailureMessage??x(t.stderr)??x(t.stdout),"Vitest"):e.reportError!==null?{failure:{...p(e.reportError,"NODE_VITEST_REPORT_INVALID"),classification:"infra"},status:"failed"}:e.reportFailureMessage!==null?{failure:{code:"NODE_VITEST_REPORT_FAILED",message:e.reportFailureMessage},status:"failed"}:e.dataEvidenceError!==null?{failure:p(e.dataEvidenceError,"NODE_VITEST_DATA_EVIDENCE_INVALID"),status:"failed"}:e.reportSummary===null||!e.reportSummary.success||e.reportSummary.numFailedTests>0||e.reportSummary.numPassedTests===0?{failure:{code:"NODE_VITEST_REPORT_FAILED",message:"Vitest report did not contain a completed passing test"},status:"failed"}:{failure:null,status:"passed"}}export function nodeVitestEvidenceCollectionFailure(t,e){const i=M(e);return Q(t,I,i)??{...p(i,"NODE_VITEST_EVIDENCE_FAILED"),classification:"infra"}}function It(t){return t!==null&&t.success&&t.numTotalTests>0&&t.numPassedTests===0&&t.numFailedTests===0&&(t.numPendingTests>0||t.numTodoTests>0)}async function Tt(t,e,i){const r=i.task,n=j(t,r.serviceName),a={...e.invocationEnvironment};Object.assign(a,{...n===null?{}:{BASE_URL:n},FORCE_COLOR:"0",NODE_OPTIONS:_t(i.prepared.resolutionBootstrapFile,i.prepared.inputBootstrapFile),NO_COLOR:"1",TESTKIT_ACTIVE:"1",[at]:i.prepared.runtimeInput.file,[st]:i.prepared.runtimeInput.sha256,TESTKIT_INTERNAL_FILE_DEADLINE_MS:String(r.deadlineMs),TESTKIT_INTERNAL_FILE_TIMEOUT_SECONDS:String(Math.max(1,Math.ceil(r.timeoutMs/1e3))),TESTKIT_INVOCATION_ID:t.invocationId,TESTKIT_LEASE_DIR:r.orchestration.lease.directory,TESTKIT_LEASE_ID:r.orchestration.lease.id,TESTKIT_MODE:r.orchestration.mode,...r.orchestration.runtimeId===void 0?{}:{TESTKIT_RUNTIME_ID:r.orchestration.runtimeId},TESTKIT_TEST_FILE:r.file,TESTKIT_TEST_ID:r.id,...e.profileId==="node-vitest-data"?{TESTKIT_DATA_ARTIFACT_DIR:c.dirname(i.prepared.dataManifestFile)}:{}}),C(a),Object.assign(a,await ft(t.workspace.cacheDirectory,i.inputs.environment));try{C(a)}catch{delete a.NODE_COMPILE_CACHE}return a}function vt(t,e,i,r){const n=j(t,i.serviceName),a=ct(r.environment);return{context:{baseUrl:n,engineId:"node-vitest",invocationId:t.invocationId,profileId:e,serviceName:i.serviceName,task:{file:i.file,id:i.id,suiteName:i.suiteName,testType:i.testType}},engineId:"node-vitest",environment:a,fixtures:r.fixtures,invocationId:t.invocationId,profileId:e,...r.setupData===void 0?{}:{setupData:r.setupData},taskId:i.id,version:1}}function _t(t,e){return[`--require=${JSON.stringify(t)}`,`--import=${JSON.stringify(q(e).href)}`].join(" ")}function j(t,e){return Object.hasOwn(t.target.endpoints,e)?t.target.endpoints[e]:null}async function gt(t,e){return{testkitConfig:await K(t,e.supportModules,"testkitConfig"),tsconfig:await K(t,e.supportModules,"tsconfig")}}async function K(t,e,i){if(!Object.hasOwn(e,i))return null;const r=e[i];if(!c.isAbsolute(r))throw new Error(`supportModules.${i} must be an absolute path`);return L(t,r,`supportModules.${i}`)}function yt(t,e){return{...pt({engineId:"node-vitest",installationKind:"root-direct-packages",profile:t.profile}),directPackages:{esbuild:e.packages.esbuild.version,typescript:e.packages.typescript.version,vite:e.packages.vite.version,vitest:e.packages.vitest.version}}}function Dt(t){return{...p(t,"NODE_VITEST_PREPARE_FAILED"),classification:t instanceof b?"product":"infra"}}async function N(t){try{return await t()}catch(e){throw new b(e)}}
|
|
@@ -1 +1 @@
|
|
|
1
|
-
import{makeTempDirAsync as
|
|
1
|
+
import{makeTempDirAsync as D}from"../../../process/files/temp-dir.js";import{classifyManagedProcessResult as F,terminalOutcome as T,processOutputEvidence as N,failureFromError as P,firstOutputLine as R,processExitFailure as C,toError as h,executeEngineTasks as M,managedProcessFailure as x,emptyEvidence as b,taskTimeoutFailure as W}from"../../adapters/execution.js";import{requireRootDirectory as B,requireAbsoluteWorkspacePath as O,requireConsumerFile as Y,safeSegment as S}from"../../adapters/workspace.js";import u from"node:fs/promises";import H from"node:path";import{setTimeout as U}from"node:timers/promises";import{pathToFileURL as $}from"node:url";import{isEngineTaskDeadlineExceededReason as K,resolveEngineTaskInputs as G}from"../../api/index.js";import{createNativeInvocationProcessEnvironment as V,runManagedProcess as j}from"../../../process/index.js";import{PlaywrightConsumerConfigurationError as J,playwrightExactFileFilter as z,writePlaywrightTaskWorkspace as X}from"./config.js";import{collectPlaywrightEvidence as Q}from"./evidence.js";import{engineEvidenceIdentity as Z}from"../../adapters/evidence-identity.js";import{getNodeStartupCacheEnvironment as q}from"../../adapters/node-startup-cache.js";const y={codePrefix:"PLAYWRIGHT",label:"Playwright"};export const PLAYWRIGHT_SPAWN_ENVIRONMENT_MAX_BYTES=24*1024;const k=64;class L extends Error{constructor(t){const r=h(t);super(r.message,{cause:r}),this.name="PlaywrightConsumerPreparationError"}}export const playwrightAdapterDependencies={createInvocationEnvironment:V,removeOwnedDirectory:et,writeTaskWorkspace:X};const tt=new Set(["EBUSY","ENOTEMPTY","EPERM"]),rt=5;async function et(e){for(let t=0;;t+=1)try{await u.rm(e,{force:!0,recursive:!0});return}catch(r){const i=r instanceof Error&&"code"in r?r.code:void 0;if(t===rt||typeof i!="string"||!tt.has(i))throw r;await U((t+1)*100)}}export async function executePlaywrightEngine(e,t,r,i=playwrightAdapterDependencies){const o=await it(e,t,i,r);return M(e,o.tasks,y,(a,s)=>ot(e,o,a,s),ct(e,o.installation))}async function it(e,t,r,i){const o=await B(e.workspace.rootDirectory);O(e.workspace.temporaryDirectory,"temporary directory"),O(e.workspace.artifactDirectory,"artifact directory"),await u.mkdir(e.workspace.temporaryDirectory,{recursive:!0}),await u.mkdir(e.workspace.artifactDirectory,{recursive:!0});const a=await u.realpath(e.workspace.temporaryDirectory),s=await u.realpath(e.workspace.artifactDirectory),n=await D(`playwright-${S(e.invocationId)}-`,{parentDir:a});i.defer(()=>r.removeOwnedDirectory(n));const l=await r.createInvocationEnvironment(n);i.defer(()=>r.removeOwnedDirectory(l.temporaryDirectory));const p=[];for(const[g,c]of e.tasks.entries()){const I=G(e,c),v=`${g.toString().padStart(3,"0")}-${S(c.id)}`,_=H.join(n,v),f=await D(`${v}-`,{parentDir:s});try{const m=st(e,c.serviceName),E=await lt(()=>Y(o,c.file,"Playwright task file")),w=await r.writeTaskWorkspace({artifactDirectory:f,baseUrl:m,browserFile:t.browserExecutable,endpoints:e.target.endpoints,inputs:I,invocationId:e.invocationId,managedModuleEntrypoints:{authoring:t.authoringEntrypoints.authoring,cli:t.family.cli,playwright:t.family.playwright.entrypoint,playwrightCore:t.family.playwrightCore.entrypoint,playwrightTest:t.family.playwrightTest.entrypoint,root:t.authoringEntrypoints.root,rootPackageJson:t.authoringEntrypoints.rootPackageJson},moduleFile:t.family.playwrightTest.entrypoint,orchestration:c.orchestration,rootDirectory:o,serviceName:c.serviceName,sourceFile:E,taskDirectory:_,taskId:c.id});i.defer(async()=>{try{await r.removeOwnedDirectory(w.attachmentDirectory)}catch(d){if(!(d instanceof Error&&"code"in d&&d.code==="ENOTDIR"))throw d}}),p.push({artifactDirectory:f,inputs:I,task:c,preparationFailure:null,prepared:{baseUrl:m,sourceFile:E,workspace:w}})}catch(m){const E=(await pt([_,f],r)).map(w=>w.error);p.push({artifactDirectory:f,inputs:I,task:c,preparationFailure:playwrightPreparationFailure(m,E),prepared:null})}}return Object.freeze({installation:t,invocationEnvironment:l.environment,rootDirectory:o,tasks:Object.freeze(p)})}async function ot(e,t,r,i){const a={...await q(e.workspace.cacheDirectory,r.inputs.environment),...nt(e,t,r)};try{A(a)}catch{delete a.NODE_COMPILE_CACHE}const s=r.task.deadlineMs-Date.now();if(s<=0)return T(r.task.id,"failed",i,b(),W(r.task.timeoutMs,y));const n=await j({args:[t.installation.family.cli,"test",z(r.prepared.sourceFile),"--config",r.prepared.workspace.configFile],cwd:t.rootDirectory,displayLabel:"Playwright",env:a,executable:process.execPath,signal:e.signal,timeoutMs:s});let l;try{l=await Q({identity:{engineId:"playwright",invocationId:e.invocationId,taskId:r.task.id,sourceFile:r.prepared.sourceFile},artifactDirectory:r.artifactDirectory,outputDirectory:r.prepared.workspace.outputDirectory,reportFile:r.prepared.workspace.reportFile,rootDirectory:t.rootDirectory,sourceFile:r.prepared.sourceFile,stderr:n.stderr,stdout:n.stdout})}catch(g){return T(r.task.id,"failed",i,N(n),playwrightEvidenceCollectionFailure(n,g))}const p=classifyPlaywrightProcessResult(n,l,r.task.timeoutMs,K(e.signal.reason));return T(r.task.id,p.status,i,l.evidence,p.failure)}export function classifyPlaywrightProcessResult(e,t,r,i){const o=F(e,r,i,y);if(o!==null)return o;if(t.outputLimit!==null){const n=t.outputLimit;return{failure:{code:"PLAYWRIGHT_OUTPUT_ARTIFACT_LIMIT",details:{...n},message:`Playwright output exceeded its ${n.limit.toString()} ${n.kind==="bytes"?"byte aggregate":"entry"} artifact limit`},status:"failed"}}const a=t.attachmentFailures.find(n=>n.code==="PLAYWRIGHT_ATTACHMENT_LIMIT");if(a!==void 0)return{failure:a,status:"failed"};if(t.report.failure!==null)return t.report;const s=t.attachmentFailures.at(0);return s!==void 0?{failure:s,status:"failed"}:e.exitCode!==0?C(e,y,R(e.stderr)??R(e.stdout)):t.reportError!==null?{failure:{...P(t.reportError,"PLAYWRIGHT_REPORT_INVALID"),classification:"infra"},status:"failed"}:t.report}export function playwrightEvidenceCollectionFailure(e,t){const r=h(t);return x(e,y,r)??{...P(r,"PLAYWRIGHT_EVIDENCE_FAILED"),classification:"infra"}}function nt(e,t,r){const i={...t.invocationEnvironment},o=r.task,a=r.prepared.workspace.inputBootstrapFile,s=r.prepared.workspace.resolutionBootstrapFile;return Object.assign(i,{BASE_URL:r.prepared.baseUrl,FORCE_COLOR:"0",NO_COLOR:"1",PLAYWRIGHT_BROWSERS_PATH:t.installation.registry.canonicalValue,PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD:"1",TESTKIT_ACTIVE:"1",TESTKIT_INTERNAL_FILE_DEADLINE_MS:String(o.deadlineMs),TESTKIT_INTERNAL_FILE_TIMEOUT_SECONDS:String(Math.max(1,Math.ceil(o.timeoutMs/1e3))),TESTKIT_INVOCATION_ID:e.invocationId,TESTKIT_LEASE_DIR:o.orchestration.lease.directory,TESTKIT_LEASE_ID:o.orchestration.lease.id,TESTKIT_MODE:o.orchestration.mode,...o.orchestration.runtimeId===void 0?{}:{TESTKIT_RUNTIME_ID:o.orchestration.runtimeId},TESTKIT_TEST_FILE:o.file,TESTKIT_TEST_ID:o.id}),i.NODE_OPTIONS=at(s,a),A(i),i}function at(e,t){return["--experimental-strip-types",`--require=${JSON.stringify(e)}`,`--import=${JSON.stringify($(t).href)}`].join(" ")}export function playwrightSpawnEnvironmentBytes(e){return Object.entries(e).reduce((t,[r,i])=>t+(process.platform==="win32"?(r.length+i.length+2)*2:Buffer.byteLength(r,"utf8")+Buffer.byteLength(i,"utf8")+2),0)}function A(e){if(Object.keys(e).length>k)throw new Error(`Playwright physical spawn environment exceeds its ${k.toString()} entry limit`);if(playwrightSpawnEnvironmentBytes(e)>PLAYWRIGHT_SPAWN_ENVIRONMENT_MAX_BYTES)throw new Error(`Playwright physical spawn environment exceeds its ${PLAYWRIGHT_SPAWN_ENVIRONMENT_MAX_BYTES.toString()} byte limit`)}function st(e,t){const r=e.target.endpoints[t];if(typeof r!="string"||r.trim().length===0)throw new Error(`Playwright task service ${t} does not have a published endpoint`);let i;try{i=new URL(r)}catch(o){throw new Error(`Playwright task service ${t} endpoint is not a valid URL`,{cause:o})}if(i.protocol!=="http:"&&i.protocol!=="https:")throw new Error(`Playwright task service ${t} endpoint must use http or https`);return i.href}function ct(e,t){return{...Z({engineId:"playwright",installationKind:"root-playwright-family",profile:e.profile}),browserName:"chromium",packages:{playwright:t.family.playwright.version,playwrightCore:t.family.playwrightCore.version,playwrightTest:t.family.playwrightTest.version},registryMode:t.registry.mode}}export function playwrightPreparationFailure(e,t){if(t.length===0)return{...P(e,"PLAYWRIGHT_PREPARE_FAILED"),classification:e instanceof L||e instanceof J?"product":"infra"};const r=h(e);return{classification:"infra",code:"PLAYWRIGHT_PREPARE_CLEANUP_FAILED",details:{cleanupFailures:t.map(i=>i.message),preparationFailure:r.message},message:`Playwright task preparation failed and its partial workspace could not be removed: ${t.map(i=>i.message).join("; ")}`,...r.stack===void 0?{}:{stack:r.stack}}}async function lt(e){try{return await e()}catch(t){throw new L(t)}}async function pt(e,t){const r=[];for(const i of e)try{await t.removeOwnedDirectory(i)}catch(o){r.push({directory:i,error:h(o)})}return r}
|
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
import { type DatabaseInfo } from "../database/state.js";
|
|
2
|
-
import type
|
|
2
|
+
import { type RuntimeInstanceContext } from "./runtime-contexts.js";
|
|
3
3
|
import type { ReadyServiceConfig } from "./runtime-service.js";
|
|
4
4
|
export interface ReadyService {
|
|
5
5
|
readonly baseUrl: string | null;
|
|
@@ -7,6 +7,8 @@ export interface ReadyService {
|
|
|
7
7
|
readonly database: Readonly<DatabaseInfo> | null;
|
|
8
8
|
}
|
|
9
9
|
export interface ReadyEnvironment {
|
|
10
|
+
readonly engineCacheDirectory?: string;
|
|
11
|
+
readonly engineWorkspaceDirectory?: string;
|
|
10
12
|
readonly graphKey: string;
|
|
11
13
|
readonly leaseDir: string;
|
|
12
14
|
readonly leaseId: string;
|
|
@@ -1 +1 @@
|
|
|
1
|
-
import{readDatabaseInfo as
|
|
1
|
+
import i from"node:path";import{readDatabaseInfo as m}from"../database/state.js";import{runtimeEngineDirectory as s}from"./runtime-contexts.js";export function createReadyEnvironment(t,a){const r=new Map(t.runtimeConfigs.map(e=>[e.name,m(e.stateDir)]));return{...a,engineCacheDirectory:i.join(s(t),"cache"),engineWorkspaceDirectory:i.join(s(t),"leases",a.leaseId),graphKey:t.graphKey,runtimeId:t.runtimeId,targetNames:[...t.targetNames],services:new Map(t.runtimeConfigs.map(e=>{const n={...e.testkit.templateContext};return delete n.databaseBindingsReader,[e.name,{baseUrl:e.testkit.local?.baseUrl||null,database:r.get(e.name)??null,config:{...e,testkit:{...e.testkit,templateContext:{...n,databaseBindingsByService:r}}}}]}))}}
|
|
@@ -1 +1 @@
|
|
|
1
|
-
import{cleanupOwnedRunLifecycleResourcesWithinLifecycleFence as
|
|
1
|
+
import{cleanupOwnedRunLifecycleResourcesWithinLifecycleFence as R,cleanupRunsWithinLifecycleFence as v,listRunManifests as F,startRunLifecycle as q}from"./lifecycle.js";import{createRuntimeManager as I}from"./runtime-manager.js";import{resolveCommittedRunFinalization as S}from"../evidence/finalization.js";import{withProductLifecycleFence as b}from"../process/lifecycle-fence.js";export function createLocalRunEnvironment(s){return{async start(d){const n=await q(s,d),c=n.startHeartbeat(2e3,()=>({kind:"controller",status:n.isStopRequested()?"interrupting":"running"}));let i=null;const p=new Map,l=()=>{if(i===null)throw new Error("Run runtimes have not been provisioned");return i};let m=null;return{runId:n.runId,signal:n.signal,isStopRequested:()=>n.isStopRequested(),getInterruptReason:()=>n.getInterruptReason(),requestStop:e=>{n.requestStop(e)},beginFinalization:c,dispose(){c(),n.removeSignalHandlers(),n.dispose()},get committedFinalization(){return m},provision(e,t){if(i!==null)throw new Error("Run runtimes are already provisioned");i=I({productDir:s,graphs:e,lifecycle:n,runtimeOptions:t})},async acquire(e,t){const r=await l().acquire(e,t);p.set(r.leaseId,r);const{engineCacheDirectory:o,engineWorkspaceDirectory:u,leaseId:a,leaseDir:h,runtimeId:g,graphKey:f,targetNames:w,services:y}=r;return{engineCacheDirectory:o,engineWorkspaceDirectory:u,leaseId:a,leaseDir:h,runtimeId:g,graphKey:f,targetNames:w,services:y}},canAcquire:e=>l().canAcquire(e),async release(e,t){const r=p.get(e);r!==void 0&&(await l().release(r,t),p.delete(e))},async cleanupAll(){await i?.cleanupAll()},getStats:()=>i?.getStats()??[],getGraph(e){const t=i?.getGraph(e);return t?{key:t.key,targetNames:[...t.targetNames]}:null},getGraphFailure:e=>i?.getGraphFailure(e)??null,markGraphFailed:(e,t)=>{l().markGraphFailed(e,t)},async finish(e,t,r){return c(),b(s,async o=>{const u=F(s).find(a=>a.runId===n.runId);if(u!==void 0&&u.services.length>0)throw new Error(r==="normal"?"Cannot commit a run while application service ownership remains published":"Cannot publish terminal failure while application service ownership remains published");m??=S(e,n.commitFinalization(e==="failed"?"failed":"finished"));try{await v(o,{protectedLiveRunIds:new Set([n.runId])}),await R(o,n)}catch(a){if(r==="normal")throw a}return t(m)})}}}}}
|
|
@@ -26,6 +26,7 @@ export interface RuntimeInstanceContext {
|
|
|
26
26
|
startupPromise: Promise<void> | null;
|
|
27
27
|
targetNames: string[];
|
|
28
28
|
}
|
|
29
|
+
export declare function runtimeEngineDirectory(context: Pick<RuntimeInstanceContext, "graphDir" | "runtimeId">): string;
|
|
29
30
|
export declare function createRuntimeInstanceContext(runtimeId: string, graph: RuntimeGraph<RuntimeContextSourceConfig>, productDir: string): RuntimeInstanceContext;
|
|
30
31
|
export declare function ensureRuntimeInstanceReady(context: RuntimeInstanceContext, startApplicationServices: boolean, lifecycle: LocalServiceLifecycle, options?: Readonly<RuntimeContextOptions>): Promise<RuntimeInstanceContext>;
|
|
31
32
|
export declare function cleanupRuntimeInstanceContext(context: RuntimeInstanceContext | null | undefined, lifecycle: LocalServiceLifecycle): Promise<void>;
|
|
@@ -1 +1 @@
|
|
|
1
|
-
import
|
|
1
|
+
import o from"fs";import n from"path";import{prepareRuntimeServices as m}from"./runtime-preparation.js";import{writeGraphMetadata as p}from"./state.js";import{startLocalServices as u,stopLocalServices as f}from"./services.js";import{resolveRuntimeInstanceConfigs as c}from"./template.js";export function runtimeEngineDirectory(r){return n.join(r.graphDir,"node_modules","testkit-engine",r.runtimeId)}export function createRuntimeInstanceContext(r,e,i){const a=n.join(i,".testkit","_graphs",e.dirName),t=n.join(a,"runtimes",r);o.mkdirSync(t,{recursive:!0}),p(a,e);const s=c(e.runtimeConfigs,r,t,{graphDirName:e.dirName,portNamespaceIndex:e.portNamespaceIndex,portNamespaceStride:e.portNamespaceStride});return{graphKey:e.key,targetNames:[...e.targetNames],graphDir:a,runtimeDir:t,runtimeId:r,runtimeConfigs:s,prepared:!1,preparationPromise:null,started:!1,startupPromise:null,startedServices:[]}}export async function ensureRuntimeInstanceReady(r,e,i,a={}){return r.prepared||(r.preparationPromise||(r.preparationPromise=(async()=>{await prepareDatabases(r.runtimeConfigs,{...a,signal:i.signal}),await m(r.runtimeConfigs,{...a,signal:i.signal}),r.prepared=!0})().finally(()=>{r.preparationPromise=null})),await r.preparationPromise),e&&!r.started&&(r.startupPromise||(r.startupPromise=(async()=>{r.startedServices=await u(r.runtimeConfigs,i,a),r.started=!0})().finally(()=>{r.startupPromise=null})),await r.startupPromise),r}async function d(r,e){r?.started&&(await f(r.startedServices,e),r.started=!1,r.startedServices=[])}export async function cleanupRuntimeInstanceContext(r,e){if(!r)return;await d(r,e);const i=runtimeEngineDirectory(r);o.rmSync(n.join(i,"cache"),{recursive:!0,force:!0});for(const a of[n.join(i,"leases"),i])try{o.rmdirSync(a)}catch(t){const s=t.code;if(s!=="ENOENT"&&s!=="ENOTEMPTY"&&s!=="EEXIST")throw t}}export async function prepareDatabases(r,e={}){if(!r.some(a=>a.testkit.database))return;const{provisionDatabase:i}=await import("../database/index.js");for(const a of r)await i(a,{progress:{reporter:e.reporter,setupRegistry:e.setupRegistry},signal:e.signal,skipSchemaSourceVerify:e.skipSchemaSourceVerify})}
|
|
@@ -1 +1 @@
|
|
|
1
|
-
import
|
|
1
|
+
import y from"fs";import x from"path";import{buildRuntimeIds as A}from"../domain/runtime-identity.js";import{formatError as R}from"../kernel/error-serialization.js";import{cleanupRuntimeInstanceContext as D,createRuntimeInstanceContext as K,ensureRuntimeInstanceReady as T}from"./runtime-contexts.js";import{timeOperation as $}from"../evidence/operation-timings.js";import{createReadyEnvironment as E}from"./ready-environment.js";export function createRuntimeManager({productDir:t,graphs:a,lifecycle:n,runtimeOptions:r={}}){const i=new Map(a.map(e=>[e.key,e])),c=new Map,s=new Map,u=W(a),f=new Map;let S=1;return{canAcquire(e){if(f.has(e.graphKey))return!1;const o=w(c,i,e);return j(s,v(e))?P(o,e,u)!==null:!1},async acquire(e,o={}){const l=w(c,i,e);if(n.isStopRequested())throw n.signal.reason||new Error("testkit run interrupted");const{leaseId:p,slot:g}=await G(r,o,"testkit.runtime.slot_claim",{summary:"claim runtime slot",scope:{graphKey:e.graphKey,taskId:e.id,serviceName:e.serviceName||null,targetName:e.targetName||null},metadata:{decision:"claim",lockCount:v(e).length.toString(),resourceCost:d(e).toString()}},()=>{const m=`lease-${e.id.toString()}-${S.toString()}`;if(S+=1,!B(s,v(e),m))throw new Error(`Task ${e.id.toString()} was claimed before its locks were available`);const h=q(l,e,u);if(!h)throw C(s,v(e),m),new Error(`Task ${e.id.toString()} was claimed before runtime capacity was available`);return{leaseId:m,slot:h}});try{const m=await F(g,t,e,n,r,o);n.signal.throwIfAborted();const h=x.join(m.runtimeDir,"leases",p);return y.mkdirSync(h,{recursive:!0}),{...E(m,{leaseId:p,leaseDir:h}),lockNames:v(e),resourceCost:d(e),slot:g,context:m}}catch(m){throw I(g,d(e)),C(s,v(e),p),U({leaseDir:x.join(g.context?.runtimeDir||t,"leases",p)}),g.context||N(g,u),await k(g,n,u),n.isStopRequested()||M(f,e.graphKey,R(m)),m}},async release(e,o={}){e?.slot&&(o.invalidate&&(e.slot.draining=!0),e.releaseState||(e.releaseState={locks:!1,runtimeSlot:!1,leaseDir:!1}),e.releaseState.locks||(C(s,e.lockNames,e.leaseId),e.releaseState.locks=!0),e.releaseState.runtimeSlot||(I(e.slot,e.resourceCost),e.releaseState.runtimeSlot=!0),e.releaseState.leaseDir||(U(e),e.releaseState.leaseDir=!0),e.released=!0,e.slot.draining&&e.slot.activeLeaseCount===0&&await k(e.slot,n,u))},async cleanupAll(){const e=[];for(const o of c.values())for(const l of o.slots){l.draining=!0;try{await k(l,n,u)}catch(p){e.push({slot:l,error:p})}}if(e.length>0)throw new AggregateError(e.map(o=>o.error),`Failed to clean up ${e.length.toString()} runtime ${e.length===1?"slot":"slots"}: ${e.map(o=>`${o.slot.runtimeId}: ${R(o.error)}`).join("; ")}`)},getStats(){return[...c.values()].map(e=>({graphKey:e.slots[0]?.graph.key||null,targetNames:[...e.slots.at(0)?.graph.targetNames??[]],maxConcurrentTasks:Number.isFinite(e.slots[0]?.graph.maxConcurrentTasks)?e.slots[0]?.graph.maxConcurrentTasks??null:null,runtimeCount:e.slots.length,runtimes:e.slots.map(o=>({runtimeId:o.runtimeId,peakLeaseCount:o.peakLeaseCount,peakResourceUnits:o.peakResourceUnits,resourceReservations:o.resourceReservations.map(l=>({...l}))})),resourceBudgets:[...u.values()].filter(o=>e.slots.some(l=>l.graph.resourceReservations.some(p=>p.key===o.key))).map(o=>({key:o.key,capacity:o.capacity,reserved:o.reserved,available:o.available,active:o.active,peakActive:o.peakActive}))})).sort((e,o)=>(e.graphKey||"").localeCompare(o.graphKey||""))},getGraphFailure(e){return f.get(e)??null},getGraph(e){return i.get(e)||null},markGraphFailed(e,o){M(f,e,o)}}}function w(t,a,n){const r=t.get(n.graphKey);if(r)return r;const i=a.get(n.graphKey);if(!i)throw new Error(`Unknown graph "${n.graphKey}"`);const s={slots:A(i.instanceCount).map(u=>({graph:i,runtimeId:u,context:null,activeLeaseCount:0,activeResourceUnits:0,resourceReservations:[],peakLeaseCount:0,peakResourceUnits:0,draining:!1}))};return t.set(n.graphKey,s),s}function q(t,a,n){const r=d(a),i=t.slots.filter(s=>!s.draining);if(i.length===0)return null;const c=[...i].filter(s=>L(s,r)&&b(s,n)).sort((s,u)=>s.activeResourceUnits-u.activeResourceUnits||s.activeLeaseCount-u.activeLeaseCount||s.runtimeId.localeCompare(u.runtimeId)).at(0);return c?(H(c,n),c.activeLeaseCount+=1,c.activeResourceUnits+=r,c.peakLeaseCount=Math.max(c.peakLeaseCount,c.activeLeaseCount),c.peakResourceUnits=Math.max(c.peakResourceUnits,c.activeResourceUnits),c):null}function I(t,a){t.activeLeaseCount=Math.max(0,t.activeLeaseCount-1),t.activeResourceUnits=Math.max(0,t.activeResourceUnits-a)}async function F(t,a,n,r,i,c={}){t.context||(t.context=K(t.runtimeId,t.graph,a),r.trackGraphContext(t.context));const s=i.operationTimingRegistry||null,u=s?.start({name:"testkit.runtime.ensure",kind:"runtime",summary:"ensure runtime instance ready",parentId:c.operationTimingParentId||null,scope:{graphKey:t.graph.key,runtimeId:t.runtimeId,serviceName:n.serviceName,targetName:n.targetName}})??null;try{await T(t.context,n.startApplicationServices,r,i),s?.finish(u,{status:"passed"})}catch(f){throw s?.finish(u,{status:"failed",error:R(f)}),f}return t.context}function G(t,a,n,r,i){return $(t.operationTimingRegistry||null,{name:n,kind:"runtime",summary:r.summary||null,parentId:a.operationTimingParentId||null,scope:r.scope||null,metadata:r.metadata||null},i)}async function k(t,a,n){!t.context||t.activeLeaseCount>0||(await D(t.context,a),N(t,n),t.context=null,t.draining=!1)}function B(t,a,n){const r=[...new Set(a)].sort();for(const i of r)if(t.has(i))return!1;for(const i of r)t.set(i,n);return!0}function j(t,a){return[...new Set(a)].every(n=>!t.has(n))}function P(t,a,n){const r=d(a);return t.slots.find(i=>!i.draining&&L(i,r)&&b(i,n))||null}function L(t,a){return t.activeResourceUnits+a<=t.graph.maxConcurrentTasks}function W(t){const a=new Map;for(const n of t)for(const r of n.resourceReservations){const i=a.get(r.key)||{key:r.key,capacity:r.capacity||0,reserved:r.reserved||0,available:r.available||Math.max(0,(r.capacity||0)-(r.reserved||0)),active:0,peakActive:0};i.capacity=Math.max(i.capacity,r.capacity||0),i.reserved=Math.max(i.reserved,r.reserved||0),i.available=Math.max(0,i.capacity-i.reserved),a.set(r.key,i)}return a}function b(t,a){return t.resourceReservations.length>0?!0:t.graph.resourceReservations.every(n=>{const r=a.get(n.key);return r?r.active+n.perRuntime<=r.available:!0})}function H(t,a){if(t.resourceReservations.length>0)return;const n=[];for(const r of t.graph.resourceReservations){const i=a.get(r.key);i&&(i.active+=r.perRuntime,i.peakActive=Math.max(i.peakActive,i.active),n.push({key:r.key,units:r.perRuntime,services:r.services}))}t.resourceReservations=n}function N(t,a){for(const n of t.resourceReservations){const r=a.get(n.key);r&&(r.active=Math.max(0,r.active-n.units))}t.resourceReservations=[]}function C(t,a,n){for(const r of[...new Set(a)].sort())t.get(r)===n&&t.delete(r)}function M(t,a,n){!a||t.has(a)||t.set(a,n)}function v(t){return t.locks||[]}function d(t){return t.resourceCost??1}function U(t){t?.leaseDir&&(t.engineWorkspaceDirectory&&y.rmSync(t.engineWorkspaceDirectory,{recursive:!0,force:!0}),y.rmSync(t.leaseDir,{recursive:!0,force:!0}))}
|
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
"entrypoint": "dist/process/process.js",
|
|
4
4
|
"sourceSha256": "a54f01864b603cee568b2945390dff35d578e07f6d26b5add06014169c3d17b0",
|
|
5
5
|
"bundleSha256": "62f76cf9fd019f6c552bfde2787f058917caf279670a62e7a6bfa12be34cb4b1",
|
|
6
|
-
"packageLockSha256": "
|
|
6
|
+
"packageLockSha256": "199fa09ca74cbda901572a6d406108c5d987a057f48b55bae294e7a1d4bded4f",
|
|
7
7
|
"esbuildVersion": "0.28.1",
|
|
8
8
|
"options": {
|
|
9
9
|
"format": "esm",
|
package/docs/cloud-control.md
CHANGED
|
@@ -1,5 +1,8 @@
|
|
|
1
1
|
# Cloud command reference
|
|
2
2
|
|
|
3
|
+
The `testkit cloud` commands are deprecated. See the
|
|
4
|
+
[Cloud boundary](testkit-cloud-boundary.md) for the direction.
|
|
5
|
+
|
|
3
6
|
`testkit cloud apply --help` prints the account-command envelope, every supported
|
|
4
7
|
operation, its scope IDs, and its input fields. This reference ships with Testkit.
|
|
5
8
|
|
|
@@ -235,6 +235,26 @@ order. Every cleanup handler is attempted, cleanup has its own bounded deadline,
|
|
|
235
235
|
and any cleanup failure fails the invocation. Adapters do not carry a second
|
|
236
236
|
preparation or execution cleanup path.
|
|
237
237
|
|
|
238
|
+
The workspace can also provide a caller-owned `cacheDirectory`. Node/Vitest and
|
|
239
|
+
Playwright use a private child directory for Node's compilation cache so fresh
|
|
240
|
+
CLI and worker processes can reuse compiled code across files. Evaluated modules,
|
|
241
|
+
consumer globals, runtime inputs, and native runner state remain process-local.
|
|
242
|
+
The local run environment shares the cache across leases of one application
|
|
243
|
+
runtime and removes it after that runtime's executions and services stop.
|
|
244
|
+
Adapters leave invocation temporary directories empty after cleanup. Callers
|
|
245
|
+
that omit `cacheDirectory` retain execution without a shared compilation cache.
|
|
246
|
+
Explicit consumer Node cache controls or V8 coverage disable automatic caching,
|
|
247
|
+
and an unavailable cache falls back to normal execution.
|
|
248
|
+
On Windows, automatic cache allocation and Node startup use the extended path
|
|
249
|
+
form so deeply nested product directories can retain compilation caching.
|
|
250
|
+
|
|
251
|
+
The local run environment places engine workspaces and compilation caches below
|
|
252
|
+
the graph's `node_modules/testkit-engine` directory. Playwright skips that tree
|
|
253
|
+
before collecting files, so concurrent discovery cannot traverse another task's
|
|
254
|
+
temporary outputs as they change. Consumer test directories, ignore rules,
|
|
255
|
+
dependency projects, and snapshot paths keep their original configuration.
|
|
256
|
+
Releasing a lease removes its engine workspace; runtime cleanup removes its cache.
|
|
257
|
+
|
|
238
258
|
Adapters prepare the task batch before executing it. Shared `executeEngineTasks`
|
|
239
259
|
starts tasks in order and defers evidence and terminal notifications until all
|
|
240
260
|
tasks settle. It gives an expired task deadline precedence over cancellation,
|
|
@@ -1,8 +1,10 @@
|
|
|
1
1
|
# Testkit Cloud boundary
|
|
2
2
|
|
|
3
|
-
The
|
|
4
|
-
|
|
5
|
-
|
|
3
|
+
The `testkit cloud` commands are deprecated. Testkit works without Cloud, and
|
|
4
|
+
bugs and evidence stay in your repository and in GitHub or Gitea. Testkit Cloud
|
|
5
|
+
is becoming an optional analytics dashboard that reads facts from GitHub or
|
|
6
|
+
Gitea instead of receiving run bundles. This document describes the commands
|
|
7
|
+
and the publication contract that still ship.
|
|
6
8
|
|
|
7
9
|
```text
|
|
8
10
|
discover -> prepare -> execute -> seal and verify run bundle -> clean up
|
|
@@ -13,13 +15,11 @@ discover -> prepare -> execute -> seal and verify run bundle -> clean up
|
|
|
13
15
|
Testkit Cloud ingest
|
|
14
16
|
```
|
|
15
17
|
|
|
16
|
-
##
|
|
18
|
+
## Cloud commands
|
|
17
19
|
|
|
18
|
-
The CLI is a full control client for Cloud. New or changed product operations
|
|
19
|
-
ship their API, installed command or file input, and acceptance coverage together.
|
|
20
20
|
The local [Cloud command reference](cloud-control.md) lists supported operations
|
|
21
21
|
and file fields. `testkit cloud apply --help` includes the account-command format.
|
|
22
|
-
Cloud
|
|
22
|
+
Cloud checks identity, permissions, and domain rules on every command.
|
|
23
23
|
|
|
24
24
|
```sh
|
|
25
25
|
testkit cloud login
|
|
@@ -176,13 +176,13 @@ The token is an environment secret. It is not stored in `testkit.config.ts` or
|
|
|
176
176
|
accepted as a command flag. `CI` is execution context only; it does not enable
|
|
177
177
|
publication or supply authority.
|
|
178
178
|
|
|
179
|
-
## Cloud
|
|
179
|
+
## Cloud ingestion
|
|
180
180
|
|
|
181
|
-
Testkit Cloud
|
|
181
|
+
Testkit Cloud ingests a verified bundle and derives regression identities,
|
|
182
182
|
baselines, comparisons, recommendations, GitHub checks, and issue lifecycle
|
|
183
183
|
actions. Cloud state is separate from the product repository and may be shared
|
|
184
|
-
across repositories. Cloud
|
|
185
|
-
|
|
184
|
+
across repositories. Cloud keeps the source bundle identity for every derived
|
|
185
|
+
decision.
|
|
186
186
|
|
|
187
187
|
Cloud ingestion uses an explicit versioned contract. It rejects an invalid
|
|
188
188
|
bundle, preserves the underlying run outcome, and reports GitHub projection
|
|
@@ -197,21 +197,9 @@ The old `./regression-workflow` package export, runner integration,
|
|
|
197
197
|
Git retains their implementation history, but that does not migrate a
|
|
198
198
|
consumer's live state.
|
|
199
199
|
|
|
200
|
-
Before a Testkit package containing this removal is published:
|
|
201
|
-
|
|
202
|
-
1. Inventory known uses of the removed export and state file.
|
|
203
|
-
2. Assign each consumer an owner and decide whether to discard or import its
|
|
204
|
-
stored state.
|
|
205
|
-
3. Rehearse required consumers against the exact packed candidate.
|
|
206
|
-
4. Block publication when a required consumer cannot move atomically.
|
|
207
|
-
|
|
208
|
-
The removed implementation is useful source material for Testkit Cloud, but it
|
|
209
|
-
must not be copied back into the runner. Cloud should consume verified bundles
|
|
210
|
-
through its own adapter and own its persistence and GitHub credentials.
|
|
211
|
-
|
|
212
200
|
## Report an externally executed campaign
|
|
213
201
|
|
|
214
|
-
Cloud
|
|
202
|
+
Cloud keeps campaign state, finding verification, and counts. A user or agent
|
|
215
203
|
starts and runs the investigation independently. These commands never start
|
|
216
204
|
execution or infer a finding from a failed assertion.
|
|
217
205
|
|
package/package.json
CHANGED
package/testkit-config-schema.md
CHANGED
|
@@ -758,6 +758,8 @@ cloud?: {
|
|
|
758
758
|
};
|
|
759
759
|
```
|
|
760
760
|
|
|
761
|
+
This setting is deprecated with the `testkit cloud` commands.
|
|
762
|
+
|
|
761
763
|
`cloud.verification.minimumConfirmingRuns` is an integer from 1 through 64,
|
|
762
764
|
defaulting to 2. It controls whether Testkit submits a new `finding.verify`
|
|
763
765
|
campaign report. It counts distinct full run IDs from available cited accepted
|