@lazyingart/agintiflow 0.20.292 → 0.20.296
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/docs/supervision-campaign-ledger.md +79 -0
- package/package.json +1 -1
- package/scripts/smoke-dynamic-step-budget.js +1 -1
- package/scripts/smoke-integration-storage-authority.js +1 -0
- package/scripts/smoke-progressive-tool-selection.js +204 -0
- package/scripts/smoke-scs-evidence-visibility.js +12 -0
- package/scripts/smoke-truthful-completion.js +169 -0
- package/src/agent-runner.js +496 -15
- package/src/integration-storage-authority.js +17 -0
- package/src/progressive-tool-selection.js +27 -10
- package/src/scs-evidence.js +155 -4
|
@@ -470,3 +470,82 @@ block, passed the exact hidden contract and all 13 visible tests, and left a
|
|
|
470
470
|
clean target repository at `ef3c099`. Focused regressions and the complete npm
|
|
471
471
|
suite pass. These runtime and skill fixes are released in AgInTiFlow
|
|
472
472
|
`0.20.292`.
|
|
473
|
+
|
|
474
|
+
### Retained storage live authority poisoning
|
|
475
|
+
|
|
476
|
+
`retained-storage-live-poison-047` exercised the native integration storage
|
|
477
|
+
authority after independent audit addenda, using a dictated but realistic
|
|
478
|
+
operator prompt. The defect was an AgInTiFlow core integration-storage
|
|
479
|
+
authority gap: live retained handles and named bindings were mostly checked,
|
|
480
|
+
but the public limitation record omitted the generic `resolveBeneath:false`
|
|
481
|
+
fact, and protected file success paths could return after cleanup awaits
|
|
482
|
+
without one final permanent-poison check.
|
|
483
|
+
|
|
484
|
+
The storage authority now reports `procfsRequired:true`,
|
|
485
|
+
`resolveBeneath:false`, and `noXdev:false` truthfully across retained
|
|
486
|
+
directory/file/lock limitation records. Live retained owner, mode, fstat, and
|
|
487
|
+
named-binding divergence remains `INTEGRATION_STORAGE_POISONED`, not an
|
|
488
|
+
availability failure, and protected read/write/sync paths check permanent
|
|
489
|
+
poison again immediately before returning success.
|
|
490
|
+
|
|
491
|
+
External verification used the module-mocked retained-storage smoke for live
|
|
492
|
+
fstat, owner, mode, named-binding, concurrent poison, close/admission, cleanup,
|
|
493
|
+
and residual-FD adversaries. Retained durable-common, retained file-lock,
|
|
494
|
+
production-mount, integration authorities, runtime authority,
|
|
495
|
+
session-persistence, syntax, and whitespace checks passed on the working tree.
|
|
496
|
+
|
|
497
|
+
### Authoritative read-only routine first
|
|
498
|
+
|
|
499
|
+
`authoritative-readonly-routine-001` reproduced the LabCanvas status failure
|
|
500
|
+
from sessions `web-agent-labcanvas-cb6fe7cd-3464-42f7-99e4-b24c376e0115` and
|
|
501
|
+
`web-agent-labcanvas-6d0b04bc-79e8-4fd8-87b4-4d8e651dcd72`. The first session
|
|
502
|
+
had no selected routine and spent 25 model requests and 23 tools on broad
|
|
503
|
+
workspace/private-store exploration. The second session received a disclosed
|
|
504
|
+
canonical read-only routine, but AgInTiFlow still kept
|
|
505
|
+
`requiredProjectCommands` empty, then continued through 32 model requests and
|
|
506
|
+
32 tools after the status snapshot.
|
|
507
|
+
|
|
508
|
+
The defect was an AgInTiFlow core execution-contract/SCS handoff gap, with a
|
|
509
|
+
separate upstream routine-disclosure improvement already handled by
|
|
510
|
+
AgenticApp. AgInTiFlow now promotes an authoritative read-only routine's first
|
|
511
|
+
safe command into the exact required command batch, records an observed
|
|
512
|
+
nonzero status snapshot as read-only evidence instead of a failed verifier,
|
|
513
|
+
retains forbidden raw/private evidence scopes, and closes the following turn to
|
|
514
|
+
`finish` once normal SCS evidence is sufficient.
|
|
515
|
+
|
|
516
|
+
The focused regression uses a weak status prompt and a project-neutral
|
|
517
|
+
`sample-status` routine. It proves the first model turn sees only the exact
|
|
518
|
+
status command plus `finish`, a JSON status command exiting 1 still satisfies
|
|
519
|
+
the read-only evidence contract, the second turn is finish-only, and no
|
|
520
|
+
private/raw exploratory command is dispatched. `npm run smoke:progressive-tools`,
|
|
521
|
+
`npm run smoke:scs-evidence`, `npm run smoke:truthful-completion`, and
|
|
522
|
+
`npm run check` pass on the working tree.
|
|
523
|
+
|
|
524
|
+
Fresh compact retest `web-agent-labcanvas-2ebe5aef-83c8-4134-abcd-bec630a4ecb9`
|
|
525
|
+
reduced the flow to 6 model requests and 5 tool starts and used the canonical
|
|
526
|
+
compact status command, but both valid `finish` calls were rejected because the
|
|
527
|
+
human-facing status said `Still retrying: echomind_daily_pdf` and named a
|
|
528
|
+
`next attempt`. That was still an AgInTiFlow core completion-semantic defect:
|
|
529
|
+
the validator treated an observed external retry state as the agent promising
|
|
530
|
+
unfinished work. The completion predicate now rejects pending work only when it
|
|
531
|
+
is tied to the current task/report/validation/change or an agent-promised
|
|
532
|
+
future action. Read-only status answers may truthfully report external
|
|
533
|
+
pending/retrying/next-attempt state after sufficient evidence exists.
|
|
534
|
+
|
|
535
|
+
Installed retest `web-agent-labcanvas-5e6eb6b6-fb8d-419d-a549-aa4098252064`
|
|
536
|
+
then proved the next core gap. AgInTiFlow ran the same canonical compact
|
|
537
|
+
health command exactly once and avoided raw/private exploration, but the
|
|
538
|
+
finish-only verified-completion turns kept returning empty content and no tool
|
|
539
|
+
calls with `finishReason=length` at the inherited 768-token output cap. After
|
|
540
|
+
the bounded empty-response repair was exhausted, the runtime emitted the
|
|
541
|
+
generic verified fallback `Evidence: command` instead of summarizing the
|
|
542
|
+
authoritative JSON status snapshot.
|
|
543
|
+
|
|
544
|
+
That defect is still AgInTiFlow core completion-loop behavior, not AgenticApp
|
|
545
|
+
or a WeChat-specific routine. Verified-completion turns now have their own
|
|
546
|
+
final-answer output floor and explicit finish-only instruction. If a provider
|
|
547
|
+
still returns empty final responses after verified evidence exists, the
|
|
548
|
+
fallback derives a concise public answer from the successful bounded command
|
|
549
|
+
output. JSON status snapshots summarize visible delivered/retrying schedule
|
|
550
|
+
states, ingress reachability, queue counts, and top-level health while keeping
|
|
551
|
+
the original SCS checks for genuinely unfinished agent work.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@lazyingart/agintiflow",
|
|
3
|
-
"version": "0.20.
|
|
3
|
+
"version": "0.20.296",
|
|
4
4
|
"type": "module",
|
|
5
5
|
"description": "AgInTiFlow is a project-aware agent workspace for hybrid wet-dry R&D, hardware-aware intelligence, software automation, and industrial workflows.",
|
|
6
6
|
"license": "Apache-2.0",
|
|
@@ -9251,7 +9251,7 @@ try {
|
|
|
9251
9251
|
const verifiedCompletionText = JSON.stringify(verifiedCompletionRequest?.messages || []);
|
|
9252
9252
|
assert(
|
|
9253
9253
|
verifiedCompletionRequest?.mode === "verified-completion" &&
|
|
9254
|
-
verifiedCompletionRequest?.maxOutputTokens ===
|
|
9254
|
+
verifiedCompletionRequest?.maxOutputTokens === 2048,
|
|
9255
9255
|
"fresh passing evidence did not narrow the final response turn"
|
|
9256
9256
|
);
|
|
9257
9257
|
assert(
|
|
@@ -538,6 +538,7 @@ async function main() {
|
|
|
538
538
|
assert.equal(authority.attestation.limitations.preEnablePrimitive, true);
|
|
539
539
|
assert.equal(authority.attestation.limitations.procfsRequired, true);
|
|
540
540
|
assert.equal(authority.attestation.limitations.openat2ResolveBeneath, false);
|
|
541
|
+
assert.equal(authority.attestation.limitations.resolveBeneath, false);
|
|
541
542
|
assert.equal(authority.attestation.limitations.noXdev, false);
|
|
542
543
|
|
|
543
544
|
const identity = await authority.identity();
|
|
@@ -5501,6 +5501,55 @@ assertStrict.equal(
|
|
|
5501
5501
|
undefined,
|
|
5502
5502
|
"an explicit task artifact root still activated generic repository grounding"
|
|
5503
5503
|
);
|
|
5504
|
+
const scopedPathWriteDescriptor = {
|
|
5505
|
+
type: "function",
|
|
5506
|
+
function: {
|
|
5507
|
+
name: "write_file",
|
|
5508
|
+
description: "Write one workspace file.",
|
|
5509
|
+
parameters: {
|
|
5510
|
+
type: "object",
|
|
5511
|
+
properties: {
|
|
5512
|
+
path: { type: "string" },
|
|
5513
|
+
content: { type: "string" },
|
|
5514
|
+
},
|
|
5515
|
+
required: ["path", "content"],
|
|
5516
|
+
additionalProperties: false,
|
|
5517
|
+
},
|
|
5518
|
+
},
|
|
5519
|
+
};
|
|
5520
|
+
const scopedPathSurface = [
|
|
5521
|
+
...allTools.filter((item) => item.function.name !== "write_file"),
|
|
5522
|
+
scopedPathWriteDescriptor,
|
|
5523
|
+
];
|
|
5524
|
+
const scopedPathTools = selectProgressiveTools(scopedPathSurface, {
|
|
5525
|
+
config: {
|
|
5526
|
+
provider: "deepseek",
|
|
5527
|
+
progressiveTools: true,
|
|
5528
|
+
scopedArtifactTask: true,
|
|
5529
|
+
scopedArtifactRoot: scopedRuntime.scopedArtifactRoot,
|
|
5530
|
+
commandCwd: "/workspace",
|
|
5531
|
+
},
|
|
5532
|
+
goal: scopedRuntimePrompt,
|
|
5533
|
+
profile: "auto",
|
|
5534
|
+
messages: [{ role: "user", content: scopedRuntimePrompt }],
|
|
5535
|
+
});
|
|
5536
|
+
const scopedWritePathPattern = scopedPathTools.find(
|
|
5537
|
+
(item) => item.function.name === "write_file"
|
|
5538
|
+
)?.function?.parameters?.properties?.path?.pattern;
|
|
5539
|
+
assert(scopedWritePathPattern, "scoped artifact write tool omitted its path contract");
|
|
5540
|
+
const scopedWritePathRegex = new RegExp(scopedWritePathPattern);
|
|
5541
|
+
assert(
|
|
5542
|
+
scopedWritePathRegex.test("output/wechat_worker/task/report.md"),
|
|
5543
|
+
"scoped artifact write contract rejected its workspace-relative path"
|
|
5544
|
+
);
|
|
5545
|
+
assert(
|
|
5546
|
+
scopedWritePathRegex.test("/workspace/output/wechat_worker/task/report.md"),
|
|
5547
|
+
"scoped artifact write contract rejected the host-supplied exact absolute path"
|
|
5548
|
+
);
|
|
5549
|
+
assert(
|
|
5550
|
+
!scopedWritePathRegex.test("/workspace/output/wechat_worker/other/report.md"),
|
|
5551
|
+
"scoped artifact write contract accepted a sibling task path"
|
|
5552
|
+
);
|
|
5504
5553
|
|
|
5505
5554
|
const longScopedTaskRoot = path.join(repoRoot, "output/wechat_worker/long-scoped-task");
|
|
5506
5555
|
const longScopedScopeLine = `AGINTI_EVIDENCE_SCOPE_JSON: ${JSON.stringify({
|
|
@@ -7288,6 +7337,160 @@ async function runToolContractCase({
|
|
|
7288
7337
|
}
|
|
7289
7338
|
}
|
|
7290
7339
|
|
|
7340
|
+
function weakAuthoritativeRoutineGoal(command) {
|
|
7341
|
+
return [
|
|
7342
|
+
"AGINTI_EVIDENCE_SCOPE_JSON: {\"mode\":\"task\",\"request\":\"tell me phone and schedule status, read-only, dont send or change anything\"}",
|
|
7343
|
+
"Matched established routines",
|
|
7344
|
+
`- \`sample-status\` ready=true; commands=[${JSON.stringify(command)}, "node scripts/secondary-status.js --json"]; outputs=["status"]; guidance=For a read-only phone, message-intake, queue, or schedule question, run the canonical \`${command}\` command first. Treat the current status snapshot as authoritative. Do not inspect raw private stores, and do not send or mutate anything.`,
|
|
7345
|
+
"Operating contract",
|
|
7346
|
+
"- Forbidden evidence: raw/private ledgers are out of scope for this read-only answer.",
|
|
7347
|
+
].join("\n");
|
|
7348
|
+
}
|
|
7349
|
+
|
|
7350
|
+
const authoritativeRoutineCommand = "node scripts/status.js status --json";
|
|
7351
|
+
const authoritativeRoutineGoal = weakAuthoritativeRoutineGoal(
|
|
7352
|
+
authoritativeRoutineCommand
|
|
7353
|
+
);
|
|
7354
|
+
const authoritativeRoutineContract = completionTaskContract(
|
|
7355
|
+
{
|
|
7356
|
+
goal: authoritativeRoutineGoal,
|
|
7357
|
+
taskProfile: "auto",
|
|
7358
|
+
},
|
|
7359
|
+
{
|
|
7360
|
+
goal: authoritativeRoutineGoal,
|
|
7361
|
+
meta: {
|
|
7362
|
+
taskProfile: "auto",
|
|
7363
|
+
goalContract: {
|
|
7364
|
+
revision: 1,
|
|
7365
|
+
currentRequest: authoritativeRoutineGoal,
|
|
7366
|
+
},
|
|
7367
|
+
},
|
|
7368
|
+
}
|
|
7369
|
+
);
|
|
7370
|
+
assertStrict.deepEqual(
|
|
7371
|
+
authoritativeRoutineContract.requiredProjectCommands,
|
|
7372
|
+
[authoritativeRoutineCommand],
|
|
7373
|
+
"authoritative read-only routine was not promoted to an exact required command"
|
|
7374
|
+
);
|
|
7375
|
+
assertStrict.deepEqual(
|
|
7376
|
+
authoritativeRoutineContract.authoritativeRoutine?.forbiddenEvidenceScopes,
|
|
7377
|
+
["private", "raw"],
|
|
7378
|
+
"authoritative read-only routine did not retain forbidden evidence scopes"
|
|
7379
|
+
);
|
|
7380
|
+
|
|
7381
|
+
const hostedRequiredRoutineTools = selectProgressiveTools(allTools, {
|
|
7382
|
+
config: {
|
|
7383
|
+
provider: "deepseek",
|
|
7384
|
+
toolSurfacePolicy: "full",
|
|
7385
|
+
requiredProjectCommandPending: true,
|
|
7386
|
+
requiredProjectCommand: authoritativeRoutineCommand,
|
|
7387
|
+
},
|
|
7388
|
+
goal: authoritativeRoutineGoal,
|
|
7389
|
+
profile: "auto",
|
|
7390
|
+
});
|
|
7391
|
+
sameNames(
|
|
7392
|
+
hostedRequiredRoutineTools,
|
|
7393
|
+
["run_command", "finish"],
|
|
7394
|
+
"hosted/full policy did not constrain an authoritative routine first command"
|
|
7395
|
+
);
|
|
7396
|
+
assertStrict.deepEqual(
|
|
7397
|
+
enumFor(hostedRequiredRoutineTools, "run_command", "command"),
|
|
7398
|
+
[authoritativeRoutineCommand],
|
|
7399
|
+
"authoritative routine command was not exposed as an exact enum"
|
|
7400
|
+
);
|
|
7401
|
+
|
|
7402
|
+
let authoritativeRoutineStep = 0;
|
|
7403
|
+
const authoritativeRoutineRun = await runToolContractCase({
|
|
7404
|
+
id: "authoritative-readonly-routine",
|
|
7405
|
+
provider: "deepseek",
|
|
7406
|
+
profile: "auto",
|
|
7407
|
+
goal: authoritativeRoutineGoal,
|
|
7408
|
+
allowShellTool: true,
|
|
7409
|
+
maxSteps: 4,
|
|
7410
|
+
toolCalls: [
|
|
7411
|
+
contractCall("unused-authoritative", "finish", { result: "unused" }),
|
|
7412
|
+
],
|
|
7413
|
+
expectSuccess: true,
|
|
7414
|
+
setupWorkspace: async (workspace) => {
|
|
7415
|
+
await fs.mkdir(path.join(workspace, "scripts"), { recursive: true });
|
|
7416
|
+
await fs.writeFile(
|
|
7417
|
+
path.join(workspace, "scripts", "status.js"),
|
|
7418
|
+
[
|
|
7419
|
+
"console.log(JSON.stringify({ phone: 'login-required', schedule: 'current', authoritative: true }));",
|
|
7420
|
+
"process.exit(1);",
|
|
7421
|
+
"",
|
|
7422
|
+
].join("\n"),
|
|
7423
|
+
"utf8"
|
|
7424
|
+
);
|
|
7425
|
+
},
|
|
7426
|
+
responseFactory: ({ payload }) => {
|
|
7427
|
+
authoritativeRoutineStep += 1;
|
|
7428
|
+
const offered = Array.isArray(payload.tools) ? names(payload.tools) : [];
|
|
7429
|
+
if (authoritativeRoutineStep === 1) {
|
|
7430
|
+
assertStrict.deepEqual(
|
|
7431
|
+
offered,
|
|
7432
|
+
["run_command", "finish"],
|
|
7433
|
+
"first authoritative routine turn exposed exploratory tools"
|
|
7434
|
+
);
|
|
7435
|
+
assertStrict.deepEqual(
|
|
7436
|
+
enumFor(payload.tools, "run_command", "command"),
|
|
7437
|
+
[authoritativeRoutineCommand],
|
|
7438
|
+
"first authoritative routine turn did not constrain the exact command"
|
|
7439
|
+
);
|
|
7440
|
+
return assistantWithToolCalls([
|
|
7441
|
+
contractCall("run-authoritative-status", "run_command", {
|
|
7442
|
+
command: authoritativeRoutineCommand,
|
|
7443
|
+
}),
|
|
7444
|
+
]);
|
|
7445
|
+
}
|
|
7446
|
+
assertStrict.deepEqual(
|
|
7447
|
+
offered,
|
|
7448
|
+
["finish"],
|
|
7449
|
+
"authoritative routine observation did not shrink the next turn to finish-only"
|
|
7450
|
+
);
|
|
7451
|
+
return assistantWithToolCalls([
|
|
7452
|
+
contractCall("finish-authoritative-status", "finish", {
|
|
7453
|
+
result: "The authoritative status snapshot says phone login is required and the schedule snapshot is current.",
|
|
7454
|
+
}),
|
|
7455
|
+
]);
|
|
7456
|
+
},
|
|
7457
|
+
});
|
|
7458
|
+
assertStrict.equal(
|
|
7459
|
+
authoritativeRoutineRun.requests.length,
|
|
7460
|
+
2,
|
|
7461
|
+
"authoritative routine flow used more than two model requests"
|
|
7462
|
+
);
|
|
7463
|
+
assertStrict.equal(
|
|
7464
|
+
authoritativeRoutineRun.events.filter(
|
|
7465
|
+
(event) =>
|
|
7466
|
+
event.type === "tool.started" &&
|
|
7467
|
+
event.data?.toolName !== "finish"
|
|
7468
|
+
).length,
|
|
7469
|
+
1,
|
|
7470
|
+
"authoritative routine flow dispatched more than the canonical command"
|
|
7471
|
+
);
|
|
7472
|
+
assert(
|
|
7473
|
+
authoritativeRoutineRun.events.some(
|
|
7474
|
+
(event) =>
|
|
7475
|
+
event.type === "tool.completed" &&
|
|
7476
|
+
event.data?.toolName === "run_command" &&
|
|
7477
|
+
event.data?.ok === false &&
|
|
7478
|
+
event.data?.authoritativeReadOnlyRoutine === true &&
|
|
7479
|
+
event.data?.authoritativeRoutineObserved === true
|
|
7480
|
+
),
|
|
7481
|
+
"nonzero authoritative routine snapshot was not retained as observed evidence"
|
|
7482
|
+
);
|
|
7483
|
+
assert(
|
|
7484
|
+
!authoritativeRoutineRun.events.some(
|
|
7485
|
+
(event) =>
|
|
7486
|
+
event.type === "tool.started" &&
|
|
7487
|
+
/(?:^|[ /])(?:\\.private|private|raw|sqlite|jsonl)(?:$|[ /.-])/i.test(
|
|
7488
|
+
String(event.data?.args?.command || "")
|
|
7489
|
+
)
|
|
7490
|
+
),
|
|
7491
|
+
"authoritative routine flow still dispatched private/raw exploratory commands"
|
|
7492
|
+
);
|
|
7493
|
+
|
|
7291
7494
|
const hiddenDryRun = await runToolContractCase({
|
|
7292
7495
|
id: "native-hidden-dry-run",
|
|
7293
7496
|
goal: "Create hidden-dry-run.txt containing unsafe if this tool dispatches.",
|
|
@@ -7647,6 +7850,7 @@ console.log(
|
|
|
7647
7850
|
"finish-contract",
|
|
7648
7851
|
"per-turn-contract-preserved",
|
|
7649
7852
|
"schema-required-type-enum-extra",
|
|
7853
|
+
"authoritative-readonly-routine-first",
|
|
7650
7854
|
"native-hidden-dry-run-zero-dispatch",
|
|
7651
7855
|
"text-fallback-unoffered-zero-dispatch",
|
|
7652
7856
|
"strict-single-call-batch",
|
|
@@ -637,11 +637,23 @@ assert.equal(
|
|
|
637
637
|
"an incidental safety-clause 'cannot' was misclassified as a current external blocker"
|
|
638
638
|
);
|
|
639
639
|
assert.equal(finishResultClaimsIncompleteWork("Completed and verified the requested report."), false);
|
|
640
|
+
assert.equal(
|
|
641
|
+
finishResultClaimsIncompleteWork(
|
|
642
|
+
"Read-only status: Still retrying: external_pdf (quality_retry_pending; next attempt at 10:14). Nothing was sent or changed."
|
|
643
|
+
),
|
|
644
|
+
false,
|
|
645
|
+
"an external retry status was misclassified as unfinished agent work"
|
|
646
|
+
);
|
|
640
647
|
assert.equal(
|
|
641
648
|
finishResultClaimsIncompleteWork("The task is paused. A corrected implementation will be written next."),
|
|
642
649
|
true,
|
|
643
650
|
"future work was accepted as a completed result"
|
|
644
651
|
);
|
|
652
|
+
assert.equal(
|
|
653
|
+
finishResultClaimsIncompleteWork("The requested report is pending and I will complete it next."),
|
|
654
|
+
true,
|
|
655
|
+
"agent-owned pending report work was accepted as complete"
|
|
656
|
+
);
|
|
645
657
|
|
|
646
658
|
const outputFilenameContract = deriveScsTaskContract({
|
|
647
659
|
goal: "Save the complete cited report as `CLAIM_LEVEL_CITATION_RESEARCH.md` and include `negative evidence` in the report.",
|
|
@@ -66,6 +66,34 @@ assert.equal(
|
|
|
66
66
|
false,
|
|
67
67
|
"a no-further-action completion statement was mistaken for future work"
|
|
68
68
|
);
|
|
69
|
+
assert.equal(
|
|
70
|
+
finishResultClaimsIncompleteWork(
|
|
71
|
+
"Read-only check done, nothing sent or changed. Still retrying: nightly_pdf (quality_retry_pending; next attempt at 10:14). Queues are otherwise healthy."
|
|
72
|
+
),
|
|
73
|
+
false,
|
|
74
|
+
"external retry status in a read-only answer was mistaken for unfinished agent work"
|
|
75
|
+
);
|
|
76
|
+
assert.equal(
|
|
77
|
+
finishResultClaimsIncompleteWork(
|
|
78
|
+
"Schedule status: memo delivered, export retry pending, next attempt tomorrow. This is only a status report."
|
|
79
|
+
),
|
|
80
|
+
false,
|
|
81
|
+
"external pending status in a read-only answer was mistaken for unfinished agent work"
|
|
82
|
+
);
|
|
83
|
+
assert.equal(
|
|
84
|
+
finishResultClaimsIncompleteWork(
|
|
85
|
+
"The current report is pending and I will finish it next."
|
|
86
|
+
),
|
|
87
|
+
true,
|
|
88
|
+
"agent-owned pending report work was accepted as a completed result"
|
|
89
|
+
);
|
|
90
|
+
assert.equal(
|
|
91
|
+
finishResultClaimsIncompleteWork(
|
|
92
|
+
"Pending validation remains before this task is complete."
|
|
93
|
+
),
|
|
94
|
+
true,
|
|
95
|
+
"agent-owned pending validation was accepted as a completed result"
|
|
96
|
+
);
|
|
69
97
|
|
|
70
98
|
const staleCompletionRepair =
|
|
71
99
|
"The proposed completion was rejected because the requested action is not supported by concrete runtime evidence. Reason: Missing required git action(s): commit.";
|
|
@@ -173,7 +201,9 @@ async function runCase({
|
|
|
173
201
|
responses,
|
|
174
202
|
allowShellTool = false,
|
|
175
203
|
allowFileTools = false,
|
|
204
|
+
allowDestructive = false,
|
|
176
205
|
executionTier = "",
|
|
206
|
+
maxOutputTokens = undefined,
|
|
177
207
|
resume = false,
|
|
178
208
|
setup = null,
|
|
179
209
|
scsActive = false,
|
|
@@ -215,6 +245,8 @@ async function runCase({
|
|
|
215
245
|
commandCwd: workspace,
|
|
216
246
|
sandboxMode: "host",
|
|
217
247
|
packageInstallPolicy: "block",
|
|
248
|
+
...(maxOutputTokens ? { maxOutputTokens } : {}),
|
|
249
|
+
allowDestructive,
|
|
218
250
|
allowShellTool,
|
|
219
251
|
allowFileTools,
|
|
220
252
|
allowWrapperTools: false,
|
|
@@ -235,6 +267,8 @@ async function runCase({
|
|
|
235
267
|
useDockerSandbox: false,
|
|
236
268
|
sandboxMode: "host",
|
|
237
269
|
packageInstallPolicy: "block",
|
|
270
|
+
...(maxOutputTokens ? { maxOutputTokens } : {}),
|
|
271
|
+
allowDestructive,
|
|
238
272
|
allowShellTool,
|
|
239
273
|
allowFileTools,
|
|
240
274
|
allowWrapperTools: false,
|
|
@@ -587,6 +621,141 @@ try {
|
|
|
587
621
|
assert.match(futureWorkFinish.result.result, /verified the working directory/i);
|
|
588
622
|
assert(futureWorkFinish.events.some((event) => event.type === "completion.evidence_rejected"));
|
|
589
623
|
|
|
624
|
+
const readOnlyExternalRetryStatus = await runCase({
|
|
625
|
+
id: "read-only-external-retry-status",
|
|
626
|
+
goal: "Execute the shell command pwd and report the status without changing anything.",
|
|
627
|
+
taskProfile: "shell",
|
|
628
|
+
allowShellTool: true,
|
|
629
|
+
responses: [
|
|
630
|
+
assistant("", [toolCall("external-status-proof", "run_command", { command: "pwd" })]),
|
|
631
|
+
assistant("", [
|
|
632
|
+
toolCall("finish-external-status", "finish", {
|
|
633
|
+
result:
|
|
634
|
+
"Read-only check done; nothing was sent or changed. Still retrying: nightly_pdf (quality_retry_pending; next attempt at 10:14). The requested command evidence is present.",
|
|
635
|
+
}),
|
|
636
|
+
]),
|
|
637
|
+
],
|
|
638
|
+
});
|
|
639
|
+
assert.equal(readOnlyExternalRetryStatus.calls.length, 2);
|
|
640
|
+
assert.equal(readOnlyExternalRetryStatus.result.stopped, undefined);
|
|
641
|
+
assert.match(readOnlyExternalRetryStatus.result.result, /Still retrying: nightly_pdf/);
|
|
642
|
+
assert(
|
|
643
|
+
readOnlyExternalRetryStatus.events.some(
|
|
644
|
+
(event) =>
|
|
645
|
+
event.type === "completion.candidate_assessed" &&
|
|
646
|
+
event.data?.claimsIncompleteWork === false
|
|
647
|
+
),
|
|
648
|
+
"read-only external retry status was still marked claimsIncompleteWork=true"
|
|
649
|
+
);
|
|
650
|
+
assert(
|
|
651
|
+
!readOnlyExternalRetryStatus.events.some(
|
|
652
|
+
(event) => event.type === "completion.evidence_rejected"
|
|
653
|
+
),
|
|
654
|
+
"read-only external retry status still triggered a completion repair"
|
|
655
|
+
);
|
|
656
|
+
|
|
657
|
+
const compactHealthCommand = "PYTHONPATH=src python -m agenticapp wechat health --compact --json";
|
|
658
|
+
const compactHealthGoal = [
|
|
659
|
+
"User request:",
|
|
660
|
+
"i was checking phone and schedulr tell me which daily things actually got delivered today and which still retrying, also can msg from me and msg from other people both reach agent. dont send chat or change anything, just inspect and answer short with evidence",
|
|
661
|
+
"",
|
|
662
|
+
"Matched established routines",
|
|
663
|
+
`- \`wechat-chatops\` ready=true; commands=[${JSON.stringify(compactHealthCommand)}, "python agentic_tools/wechat_gui_agent/scripts/wechat_android_ingress.py --status"]; outputs=["messages", "files", "task records"]; guidance=For a read-only phone, message-intake, queue, or schedule question, run the canonical compact health command first; it already includes both Android lanes. Use the raw Android status commands only when compact health marks a lane unknown or stale. Treat that current snapshot as authoritative and stop once it answers the request. Do not inspect raw chat text or private message ledgers or artifact directories, and do not send or mutate anything, unless the current request explicitly needs it.`,
|
|
664
|
+
].join("\n");
|
|
665
|
+
const compactHealthFallback = await runCase({
|
|
666
|
+
id: "read-only-compact-health-empty-finish-fallback",
|
|
667
|
+
goal: compactHealthGoal,
|
|
668
|
+
taskProfile: "chatops",
|
|
669
|
+
allowShellTool: true,
|
|
670
|
+
allowDestructive: true,
|
|
671
|
+
maxOutputTokens: 768,
|
|
672
|
+
setup: async (workspace) => {
|
|
673
|
+
await fs.mkdir(path.join(workspace, "src", "agenticapp"), { recursive: true });
|
|
674
|
+
await fs.writeFile(path.join(workspace, "src", "agenticapp", "__init__.py"), "", "utf8");
|
|
675
|
+
await fs.writeFile(
|
|
676
|
+
path.join(workspace, "src", "agenticapp", "__main__.py"),
|
|
677
|
+
[
|
|
678
|
+
"import json",
|
|
679
|
+
"payload = {",
|
|
680
|
+
" 'ok': True,",
|
|
681
|
+
" 'operational': True,",
|
|
682
|
+
" 'degraded': True,",
|
|
683
|
+
" 'issues': ['wechat_login_required'],",
|
|
684
|
+
" 'phone_ingress': {",
|
|
685
|
+
" 'other_people': {'ok': True, 'fresh': True, 'reaches_agent': True, 'routes': 6},",
|
|
686
|
+
" 'self_authored': {'ok': True, 'fresh': True, 'reaches_agent': True, 'routes': 6, 'seeded_routes': 6},",
|
|
687
|
+
" },",
|
|
688
|
+
" 'queues': {",
|
|
689
|
+
" 'wechat': {'ok': True, 'pending': 0, 'active': 0, 'recent_failure_count': 0, 'stale_count': 0},",
|
|
690
|
+
" 'wecom': {'ok': True, 'pending': 0, 'active': 0, 'recent_failure_count': 0, 'stale_count': 0},",
|
|
691
|
+
" },",
|
|
692
|
+
" 'schedules': {",
|
|
693
|
+
" 'career_daily': {'delivered': True, 'retry_pending': False, 'running': True, 'status': 'delivered'},",
|
|
694
|
+
" 'echomind_daily_pdf': {'retry_pending': True, 'status': 'quality_retry_pending', 'next_attempt_at': '2026-08-31T02:47:35+00:00'},",
|
|
695
|
+
" 'memo_daily': {'delivered': True, 'required': True, 'retry_pending': False, 'status': 'delivered'},",
|
|
696
|
+
" }",
|
|
697
|
+
"}",
|
|
698
|
+
"print(json.dumps(payload, indent=2, sort_keys=True))",
|
|
699
|
+
"",
|
|
700
|
+
].join("\n"),
|
|
701
|
+
"utf8"
|
|
702
|
+
);
|
|
703
|
+
},
|
|
704
|
+
responses: [
|
|
705
|
+
assistant("", [toolCall("run-compact-health", "run_command", { command: compactHealthCommand })]),
|
|
706
|
+
reasoningOnly("The compact health JSON already answers the read-only status question, so I should produce a short final answer.", "length"),
|
|
707
|
+
reasoningOnly("I need to summarize delivered schedules, retrying schedules, phone ingress, and queues from the retained JSON.", "length"),
|
|
708
|
+
],
|
|
709
|
+
});
|
|
710
|
+
assert.equal(
|
|
711
|
+
compactHealthFallback.calls.length,
|
|
712
|
+
3,
|
|
713
|
+
"empty finish-only verified completion burned extra model turns before fallback"
|
|
714
|
+
);
|
|
715
|
+
assert.equal(
|
|
716
|
+
compactHealthFallback.calls[1]?.max_tokens,
|
|
717
|
+
2048,
|
|
718
|
+
"verified-completion turn did not raise the installed 768-token cap"
|
|
719
|
+
);
|
|
720
|
+
assert.equal(
|
|
721
|
+
compactHealthFallback.calls[2]?.max_tokens,
|
|
722
|
+
2048,
|
|
723
|
+
"verified-completion retry did not retain the raised output cap"
|
|
724
|
+
);
|
|
725
|
+
assert.match(compactHealthFallback.result.result, /Delivered: .*career_daily.*memo_daily/i);
|
|
726
|
+
assert.match(compactHealthFallback.result.result, /Still retrying: .*echomind_daily_pdf.*next attempt 2026-08-31T02:47:35\+00:00/i);
|
|
727
|
+
assert.match(compactHealthFallback.result.result, /phone_ingress reaches the agent for: .*other_people.*self_authored/i);
|
|
728
|
+
assert.match(compactHealthFallback.result.result, /Queues: .*wechat pending 0 active 0.*wecom pending 0 active 0/i);
|
|
729
|
+
assert.match(compactHealthFallback.result.result, /Verified from runtime evidence/i);
|
|
730
|
+
assert.doesNotMatch(compactHealthFallback.result.result, /^Completed the requested work and verified it from runtime evidence\. Evidence: command/i);
|
|
731
|
+
assert.equal(
|
|
732
|
+
compactHealthFallback.events.filter(
|
|
733
|
+
(event) => event.type === "tool.started" && event.data?.toolName !== "finish"
|
|
734
|
+
).length,
|
|
735
|
+
1,
|
|
736
|
+
"compact health fallback dispatched more than the authoritative command"
|
|
737
|
+
);
|
|
738
|
+
assert(
|
|
739
|
+
!compactHealthFallback.events.some(
|
|
740
|
+
(event) =>
|
|
741
|
+
event.type === "tool.started" &&
|
|
742
|
+
/(?:^|[ /])(?:\\.private|private|raw|sqlite|jsonl|artifact)(?:$|[ /.-])/i.test(
|
|
743
|
+
String(event.data?.args?.command || "")
|
|
744
|
+
)
|
|
745
|
+
),
|
|
746
|
+
"compact health fallback explored forbidden private/raw evidence"
|
|
747
|
+
);
|
|
748
|
+
assert.equal(
|
|
749
|
+
compactHealthFallback.events.filter((event) => event.type === "completion.verified_fallback").length,
|
|
750
|
+
1,
|
|
751
|
+
"compact health empty response did not persist exactly one verified fallback"
|
|
752
|
+
);
|
|
753
|
+
assert.equal(
|
|
754
|
+
compactHealthFallback.events.filter((event) => event.type === "completion.empty_response_repair_requested").length,
|
|
755
|
+
0,
|
|
756
|
+
"finish-only reasoning repair still spent a separate empty-response repair turn"
|
|
757
|
+
);
|
|
758
|
+
|
|
590
759
|
const wordCompletionWithoutArtifact = await runCase({
|
|
591
760
|
id: "word-completion-without-artifact",
|
|
592
761
|
goal: "Create an editable, phone-friendly project handoff from this folder.",
|