agentlinkops 0.6.4 → 0.6.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "agentlinkops",
3
- "version": "0.6.4",
3
+ "version": "0.6.6",
4
4
  "description": "Connect an agent to AgentLinkOps and run backlink work with it: the connect skill (views, the three discovery verbs, evidence rules), local ledger and campaign skills, a portable CRM, the source-cited content pipeline, user-managed disavow lists, HTML reports and monitoring-event sync.",
5
5
  "author": {
6
6
  "name": "AgentLinkOps"
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "agentlinkops",
3
- "version": "0.6.4",
3
+ "version": "0.6.6",
4
4
  "description": "Connect an agent to AgentLinkOps and run backlink work with it: the connect skill (views, the three discovery verbs, evidence rules), local ledger and campaign skills, a portable CRM, the source-cited content pipeline, user-managed disavow lists, HTML reports and monitoring-event sync.",
5
5
  "author": {
6
6
  "name": "AgentLinkOps"
package/CHANGELOG.md CHANGED
@@ -3,6 +3,19 @@
3
3
  All notable changes to the `agentlinkops` package are recorded here. The version follows the
4
4
  plugin manifests in `.claude-plugin/` and `.codex-plugin/`.
5
5
 
6
+ ## 0.6.6 (2026-09-20)
7
+
8
+ - Retain supplied browser usage receipts in a bounded local journal after each attempt and cleanup. Missing meter values remain unknown; failed writes stop the run.
9
+ - Close browser adapters after failed runs and admission errors. Cleanup uncertainty stays explicit in retained receipts.
10
+ - Check citation files against their recorded observation hashes before freezing reports or uploading evidence. Changed or unreferenced files cannot silently add samples.
11
+ - Include retained inspection-attempt status and rolling website quota in the command response schema.
12
+
13
+ ## 0.6.5 (2026-09-20)
14
+
15
+ - Browser measurement requires a configured proxy. Authorized direct diagnostics need an explicit setting; missing proxy configuration never selects direct traffic.
16
+ - Bound proxy acquisition, callback and cleanup waits. Failed cleanup quarantines the lease and blocks reuse; errors exclude provider details that could contain credentials.
17
+ - Keep supplied usage measurements separate from estimated costs. The standard URL proxy has no meter, and engine callback receipts are not persisted by the CLI runner.
18
+
6
19
  ## 0.6.4 (2026-09-20)
7
20
 
8
21
  - Import and export immutable local IndexNow submission receipts, with hashes and replay checks. These commands send no submissions.
package/README.md CHANGED
@@ -149,17 +149,34 @@ its sample size, classified against the previous epoch as declined, grown or
149
149
  not_distinguishable, or `insufficient_data` when n is too thin to interpret.
150
150
 
151
151
  ```sh
152
- export PERPLEXITY_API_KEY=... # live engine; omit it and the mock engine runs at $0
152
+ agentlinkops citation panel --domain example.com --brand Example --out panel.json
153
153
  agentlinkops citation run panel.json --max-usd 1.00
154
154
  ```
155
155
 
156
- The panel is JSON: `targets` (domain or url, brand, aliases), `prompts`, `engines`
157
- (`mock` or `perplexity`), optional `samples` and `maxUsd`. Every run writes immutable
158
- content-addressed answer snapshots plus append-only observation and epoch rows under
159
- `.agentlinkops/citations/`. The USD cap is checked before each call, so it can be approached
160
- but never exceeded, and a budget abort leaves a partial-epoch receipt and exit code 2. A cell
156
+ New panels use the free mock engine. A live engine requires its own configuration and credentials;
157
+ missing credentials never select mock results. The JSON panel contains `targets`, `prompts`,
158
+ `engines`, optional `samples` and `maxUsd`. Answer snapshots live in
159
+ `.agentlinkops/citations/evidence/`; observation and epoch ledgers live at the `.agentlinkops/` root.
160
+ The runner reserves estimated cost before each call and retry. A reported supplier
161
+ overrun stops later calls; estimates do not enforce a provider invoice cap. A budget
162
+ abort leaves a partial-epoch receipt and exit code 2. A cell
161
163
  that declined with complete evidence exits 1, like an expected link observed absent.
162
164
 
165
+ Browser measurement requires `AGENTLINKOPS_PROXY_URL` and refuses direct fallback.
166
+ For an explicitly authorized direct diagnostic, set `AGENTLINKOPS_BROWSER_EGRESS=direct-diagnostic`
167
+ and leave the proxy unset. The standard URL proxy provides no actual usage meter;
168
+ its cost remains an estimate. Browser login and live account acceptance are separate.
169
+
170
+ Browser epochs retain supplied usage and cleanup receipts in
171
+ `.agentlinkops/citations/egress/<epochId>.jsonl`. Unknown meter values stay null. A missing
172
+ final record means the journal is incomplete; it does not establish complete costs or cleanup.
173
+ Journals stop at 10,000 records or 8 MiB, and a failed write stops further samples.
174
+
175
+ Frozen inventories and evidence uploads check files against their retained observation hashes.
176
+ Historical hashes verify the recorded answer fields and screenshot filename; they do not prove
177
+ when an answer was captured or authenticate old screenshot pixels. Hosted uploads retain a
178
+ separate hash of the exact uploaded file bytes.
179
+
163
180
  ## Environment variables
164
181
 
165
182
  | Variable | Purpose |
@@ -168,6 +185,8 @@ that declined with complete evidence exits 1, like an expected link observed abs
168
185
  | `AGENTLINKOPS_API_URL` | Hosted origin when it is not in `config.json`. |
169
186
  | `AGENTLINKOPS_WEBHOOK_SECRET` | Verifies signed deliveries in `agentlinkops receive`. |
170
187
  | `AGENTLINKOPS_GSC_TOKEN`, `AGENTLINKOPS_GA4_TOKEN` | First-party context commands under `agentlinkops context`. |
188
+ | `AGENTLINKOPS_PROXY_URL` | Proxy URL for browser measurement; credentials are never printed. |
189
+ | `AGENTLINKOPS_BROWSER_EGRESS` | Defaults to `proxy-required`; `direct-diagnostic` explicitly permits an authorized diagnostic without a proxy. |
171
190
  | `PERPLEXITY_API_KEY` | Live engine credential for `agentlinkops citation`. Presence is reported by `doctor`; the value is never printed. |
172
191
 
173
192
  No value is ever printed. The `LINKTRAIL_*` spellings, the `linktrail` command and a `.linktrail/`
package/cli/catalog.js CHANGED
@@ -14687,6 +14687,111 @@ export const CATALOG_COMMANDS = [
14687
14687
  "type": "null"
14688
14688
  }
14689
14689
  ]
14690
+ },
14691
+ "attemptSummary": {
14692
+ "type": "object",
14693
+ "properties": {
14694
+ "observedAt": {
14695
+ "type": "string"
14696
+ },
14697
+ "latestAttempt": {
14698
+ "anyOf": [
14699
+ {
14700
+ "type": "object",
14701
+ "properties": {
14702
+ "id": {
14703
+ "type": "string",
14704
+ "pattern": "^[a-f0-9]{64}$"
14705
+ },
14706
+ "admittedAt": {
14707
+ "type": "string"
14708
+ },
14709
+ "status": {
14710
+ "type": "string",
14711
+ "enum": [
14712
+ "in_progress",
14713
+ "unsettled",
14714
+ "completed"
14715
+ ]
14716
+ },
14717
+ "receiptId": {
14718
+ "anyOf": [
14719
+ {
14720
+ "type": "string",
14721
+ "pattern": "^[a-f0-9]{64}$"
14722
+ },
14723
+ {
14724
+ "type": "null"
14725
+ }
14726
+ ]
14727
+ },
14728
+ "leaseExpiresAt": {
14729
+ "type": [
14730
+ "string",
14731
+ "null"
14732
+ ]
14733
+ }
14734
+ },
14735
+ "required": [
14736
+ "id",
14737
+ "admittedAt",
14738
+ "status",
14739
+ "receiptId",
14740
+ "leaseExpiresAt"
14741
+ ],
14742
+ "additionalProperties": false
14743
+ },
14744
+ {
14745
+ "type": "null"
14746
+ }
14747
+ ]
14748
+ },
14749
+ "unsettledCount": {
14750
+ "type": "integer",
14751
+ "minimum": 0,
14752
+ "maximum": 9007199254740991
14753
+ }
14754
+ },
14755
+ "required": [
14756
+ "observedAt",
14757
+ "latestAttempt",
14758
+ "unsettledCount"
14759
+ ],
14760
+ "additionalProperties": false
14761
+ },
14762
+ "websiteQuota": {
14763
+ "type": "object",
14764
+ "properties": {
14765
+ "windowStart": {
14766
+ "type": "string"
14767
+ },
14768
+ "observedAt": {
14769
+ "type": "string"
14770
+ },
14771
+ "admitted": {
14772
+ "type": "integer",
14773
+ "minimum": 0,
14774
+ "maximum": 9007199254740991
14775
+ },
14776
+ "limit": {
14777
+ "type": "integer",
14778
+ "exclusiveMinimum": 0,
14779
+ "maximum": 9007199254740991
14780
+ },
14781
+ "remaining": {
14782
+ "type": "integer",
14783
+ "minimum": 0,
14784
+ "maximum": 9007199254740991
14785
+ }
14786
+ },
14787
+ "required": [
14788
+ "windowStart",
14789
+ "observedAt",
14790
+ "admitted",
14791
+ "limit",
14792
+ "remaining"
14793
+ ],
14794
+ "additionalProperties": false
14690
14795
  }
14691
14796
  },
14692
14797
  "required": [
package/cli/citation.js CHANGED
@@ -68,7 +68,9 @@ Panel (JSON): targets (domain or url, brand, aliases), prompts, engines
68
68
  Live engines need their configured credentials or a usable browser session; the mock
69
69
  engine runs everything at zero cost. Every run states its estimated spend, and the
70
70
  cap reserves estimated cost before each call. A supplier charge above that estimate
71
- is recorded as an overrun and stops subsequent calls.
71
+ is recorded as an overrun and stops subsequent calls. Browser measurement requires
72
+ AGENTLINKOPS_PROXY_URL. Authorized direct diagnostics need
73
+ AGENTLINKOPS_BROWSER_EGRESS=direct-diagnostic with no proxy configured.
72
74
 
73
75
  Output language: citation RATE with a Wilson interval and n, classified against the
74
76
  previous epoch as declined, grown or not_distinguishable — never "removed", never
package/cli/client.js CHANGED
@@ -2,7 +2,7 @@ import {restResponseContract} from '../shared/response-contract.js';
2
2
  // A small HTTP client for the cloud. Scoped keys only; no human session ever reaches here.
3
3
  // DP-0036-T13: the CLI names itself so the usage counters can tell it from other REST callers.
4
4
  // The version follows the plugin manifests; bump it with them.
5
- export const CLI_VERSION = '0.6.4';
5
+ export const CLI_VERSION = '0.6.6';
6
6
  export const CLI_USER_AGENT = `agentlinkops-cli/${CLI_VERSION}`;
7
7
  export class CloudError extends Error {
8
8
  constructor(code, status, details = null) {
package/cli/doctor.js CHANGED
@@ -271,7 +271,7 @@ export async function doctorMain(argv = [], { cwd = process.cwd(), out = console
271
271
  execFile('camoufox', ['path'], { timeout: 8000 }, (error) => resolve(!error));
272
272
  });
273
273
  if (camoufoxOk) {
274
- checks.push({ status: 'ok', name: 'browser', detail: 'playwright-core + camoufox present; browser engines available' });
274
+ checks.push({ status: 'ok', name: 'browser', detail: 'playwright-core + camoufox present; browser dependencies present; proxy and account access remain unverified' });
275
275
  } else {
276
276
  checks.push({ status: 'skip', name: 'browser', detail: 'playwright-core ok but camoufox missing', fix: 'python3 -m pip install cloverlabs-camoufox[geoip] && python3 -m camoufox fetch' });
277
277
  }
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "agentlinkops",
3
- "version": "0.6.4",
3
+ "version": "0.6.6",
4
4
  "description": "A backlink and AI citation ledger that lives in your repository: local link and citation checks with dated evidence, supplier-export imports and optional sync to the hosted AgentLinkOps service.",
5
5
  "author": "AgentLinkOps (https://agentlinkops.com)",
6
6
  "license": "Apache-2.0",
@@ -14,6 +14,8 @@ Use `--competitors competitors.json` to add domains to check beside your own. Th
14
14
 
15
15
  Choose a live engine explicitly after checking access and budget. For an existing browser setup, use `--engine chatgpt:web-own-browser` or `--engine grok:web-own-browser`. `--engine google-aio` requires your configured supplier access. This workflow does not create accounts or buy services.
16
16
 
17
+ Starting with CLI 0.6.5, browser measurement requires `AGENTLINKOPS_PROXY_URL`. Missing, failed or quarantined proxy admission never selects direct traffic. An explicitly authorized direct diagnostic requires `AGENTLINKOPS_BROWSER_EGRESS=direct-diagnostic` with no proxy configured. The standard URL proxy has no usage meter; estimated costs are not a provider invoice cap.
18
+
17
19
  Set `--locale en-GB --country GB` for a British locale draft. The mock engine supports locale fixtures. The Google AIO supplier path supports US and GB location requests; other engine paths refuse non-default locale requests in this version. Legacy default browser observations do not prove US proxy egress. Reports disclose the requested context and its basis; they do not treat it as verified user geography.
18
20
 
19
21
  After reviewing the draft, run the [local CLI workflow](cli.md) and inspect the retained evidence:
@@ -10,5 +10,7 @@ export const indexSettingsResponse=z.object({projectId:id,enabled:z.boolean(),da
10
10
  export const indexObservation=z.looseObject({id:digest,observationId:id,checkedAt:time,recordedAt:time,expiresAt:time,evidenceSha256:digest,eventId:digest.nullable(),ignored:z.string().nullable(),version:z.literal(1),url:z.url(),lane:z.enum(['gsc','licensed_site']),backend:z.string(),source_key:z.string(),tier:z.enum(['indexed','not_indexed','likely_indexed','not_found_in_site_query','unknown']),confidence:z.enum(['google_index_snapshot','search_result_sample','unknown']),reason:z.string(),live_page_test:z.literal(false),submission_requested:z.literal(false)});
11
11
  export const indexReceiptResponse=z.object({observation:indexObservation,rawResponse:z.unknown(),evidenceAvailable:z.boolean()}).passthrough();
12
12
  export const indexInspectResponse=indexReceiptResponse.extend({replayed:z.boolean(),providerError:z.enum(['PROVIDER_PERMISSION_LOST','PROVIDER_RATE_LIMITED','PROVIDER_UNAVAILABLE','PROVIDER_RESPONSE_INVALID']).nullable().optional()});
13
- export const indexHistoryResponse=z.object({observations:z.array(indexObservation),nextCursor:digest.nullable()}).passthrough();
13
+ export const indexAttemptSummary=z.object({observedAt:time,latestAttempt:z.object({id:digest,admittedAt:time,status:z.enum(['in_progress','unsettled','completed']),receiptId:digest.nullable(),leaseExpiresAt:time.nullable()}).nullable(),unsettledCount:z.number().int().nonnegative()});
14
+ export const indexWebsiteQuota=z.object({windowStart:time,observedAt:time,admitted:z.number().int().nonnegative(),limit:z.number().int().positive(),remaining:z.number().int().nonnegative()});
15
+ export const indexHistoryResponse=z.object({observations:z.array(indexObservation),nextCursor:digest.nullable(),attemptSummary:indexAttemptSummary.optional(),websiteQuota:indexWebsiteQuota.optional()}).passthrough();
14
16
  export const indexEventsResponse=z.object({events:z.array(z.looseObject({id:digest,receiptId:digest,type:z.enum(['index.deindexed','index.reindexed']),createdAt:time,url:z.url(),source_key:z.string(),before:z.enum(['indexed','not_indexed']),after:z.enum(['indexed','not_indexed']),observation_ids:z.array(id)}))}).passthrough();
@@ -9,7 +9,7 @@ records. This page is the complete agent reference: the same text prints from
9
9
  full once per session, then use `describe` for any command's exact schema.
10
10
 
11
11
  <!-- generated:skill-version begin (npm run generate:tools) -->
12
- Pack version 0.6.3. 125 commands in 13 toolsets.
12
+ Pack version 0.6.5. 125 commands in 13 toolsets.
13
13
  <!-- generated:skill-version end -->
14
14
 
15
15
  ## Install and connect (once per machine)
@@ -14,6 +14,7 @@ import { homedir } from 'node:os';
14
14
  import { join } from 'node:path';
15
15
 
16
16
  import { EngineError } from '../adapter.js';
17
+ import {createProxyAdmission,proxyFailureReason} from './proxy-admission.js';
17
18
 
18
19
  const MODULE_LOADERS = {
19
20
  'playwright-core': () => import('playwright-core'),
@@ -56,20 +57,6 @@ const ENGINE_NOMINAL_COST = Object.freeze({
56
57
  });
57
58
  const DEFAULT_NOMINAL_COST = 0.03;
58
59
 
59
- function parseProxyUrl(raw) {
60
- if (!raw) return null;
61
- try {
62
- const url = new URL(raw);
63
- return {
64
- server: `${url.protocol}//${url.hostname}:${url.port}`,
65
- username: decodeURIComponent(url.username),
66
- password: decodeURIComponent(url.password),
67
- };
68
- } catch {
69
- throw new EngineError(`AGENTLINKOPS_PROXY_URL is not a valid proxy URL (length ${raw.length}, value never printed)`);
70
- }
71
- }
72
-
73
60
  export function createBrowserEngine({ engineName, env = process.env, onEvent = () => {}, runtime = {} } = {}) {
74
61
  if (!BROWSER_PROVIDERS.includes(engineName)) {
75
62
  throw new EngineError(`unknown browser engine: ${engineName} (known: ${BROWSER_PROVIDERS.join(', ')})`);
@@ -78,13 +65,21 @@ export function createBrowserEngine({ engineName, env = process.env, onEvent = (
78
65
  const sessionDir = runtime.sessionDir ?? join(homedir(), '.agentlinkops', 'citations', 'sessions');
79
66
  const load = runtime.loadModule ?? lazy;
80
67
  const platform = runtime.platform ?? process.platform;
68
+ const event=async text=>{
69
+ let timer;
70
+ try{await Promise.race([Promise.resolve().then(()=>onEvent(text)),new Promise((_,reject)=>{timer=setTimeout(()=>reject(Error()),runtime.supplierTimeoutMs??5000);})]);}
71
+ catch{throw new EngineError('Browser event reporting failed.');}
72
+ finally{clearTimeout(timer);}
73
+ };
81
74
 
75
+ const admission=createProxyAdmission({engine:engineName,env,supplier:runtime.proxySupplier,now:runtime.now,onReceipt:runtime.onEgressReceipt,timeoutMs:runtime.supplierTimeoutMs});
76
+ let running=false,contextCleanupFailed=false;
82
77
  let browserHandle = null; // { browser, cleanup } for the whole epoch
83
78
  let displayHandle = null;
84
79
  let providerConfig = null;
85
80
  async function closeQuietly(action) {
86
81
  let timer;
87
- try { await Promise.race([Promise.resolve().then(action).catch(() => null), new Promise(resolve => { timer = setTimeout(resolve, runtime.cleanupTimeoutMs ?? 5000); })]); }
82
+ try { return await Promise.race([Promise.resolve().then(action).then(() => true, () => false), new Promise(resolve => { timer = setTimeout(() => resolve(false), runtime.cleanupTimeoutMs ?? 5000); })]); }
88
83
  finally { clearTimeout(timer); }
89
84
  }
90
85
 
@@ -109,11 +104,11 @@ export function createBrowserEngine({ engineName, env = process.env, onEvent = (
109
104
  // engines that block headless render headfully inside a virtual display.
110
105
  if (platform === 'linux' && !display.detectDisplay()) {
111
106
  displayHandle = await display.ensureDisplay({ allowExistingDisplay: false });
112
- onEvent(`display: ${displayHandle.display}`);
107
+ await event(`display: ${displayHandle.display}`);
113
108
  }
114
109
 
115
- const proxy = parseProxyUrl(env.AGENTLINKOPS_PROXY_URL);
116
- if (proxy) onEvent(`proxy: ${proxy.server}`);
110
+ const proxy = admission.proxy;
111
+ await event(proxy?'egress: proxy-required':'egress: direct-diagnostic');
117
112
 
118
113
  const options = await resolveCamoufoxLaunchOptions({
119
114
  provider: engineName,
@@ -127,14 +122,17 @@ export function createBrowserEngine({ engineName, env = process.env, onEvent = (
127
122
  });
128
123
  const launchOpts = { ...options, executablePath: options.executablePath };
129
124
  if (proxy) launchOpts.proxy = proxy;
125
+ admission.assertReady();
130
126
  const browser = await fw.launch(launchOpts);
131
127
 
132
128
  browserHandle = {
133
129
  browser,
134
130
  cleanup: async () => {
135
- await closeQuietly(() => browser.close());
136
- await displayHandle?.cleanup?.().catch(() => null);
131
+ const browserClosed=await closeQuietly(() => browser.close());
132
+ const displayClosed=await closeQuietly(() => displayHandle?.cleanup?.());
137
133
  displayHandle = null;
134
+ if(browserClosed)contextCleanupFailed=false;
135
+ return browserClosed&&displayClosed;
138
136
  },
139
137
  };
140
138
  return browserHandle;
@@ -193,13 +191,14 @@ export function createBrowserEngine({ engineName, env = process.env, onEvent = (
193
191
  return null;
194
192
  }
195
193
 
196
- return {
194
+ const engine = {
197
195
  identity,
198
196
  estimateCostUsd: () => ENGINE_NOMINAL_COST[engineName] ?? DEFAULT_NOMINAL_COST,
199
197
 
200
198
  async run({ prompt }) {
201
199
  const config = await ensureProviderConfig();
202
200
  const { browser } = await ensureBrowser();
201
+ admission.assertReady();
203
202
  const storageState = await loadStorageState();
204
203
  const context = await browser.newContext({
205
204
  storageState,
@@ -318,9 +317,9 @@ export function createBrowserEngine({ engineName, env = process.env, onEvent = (
318
317
  : `${engineName}: answer extracted empty`;
319
318
  throw new EngineError(reason, { retriable: true }); }
320
319
  let extractionFailed = config.citationExtraction === 'unsupported';
321
- const sources = await config.extractSources(page).catch(() => {
320
+ const sources = await config.extractSources(page).catch(async () => {
322
321
  extractionFailed = true;
323
- onEvent('sources extraction failed; observation retained as unknown');
322
+ await event('sources extraction failed; observation retained as unknown');
324
323
  return [];
325
324
  });
326
325
  const screenshotPng = await page.screenshot({ fullPage: false }).catch(() => null);
@@ -347,16 +346,64 @@ export function createBrowserEngine({ engineName, env = process.env, onEvent = (
347
346
  error.unknown = true;
348
347
  throw error;
349
348
  } finally {
350
- await closeQuietly(() => context.close());
349
+ if(!await closeQuietly(() => context.close()))contextCleanupFailed=true;
351
350
  }
352
351
  },
353
352
 
354
353
  // Called by the runner at epoch end.
355
354
  async close() {
356
- await browserHandle?.cleanup?.();
357
- await displayHandle?.cleanup?.().catch(() => null);
358
- displayHandle = null;
359
- browserHandle = null;
355
+ let confirmed=true;
356
+ try{if(browserHandle)confirmed=await browserHandle.cleanup();}
357
+ finally{
358
+ const displayClosed=await closeQuietly(() => displayHandle?.cleanup?.());
359
+ confirmed=confirmed&&displayClosed;displayHandle=null;browserHandle=null;
360
+ }
361
+ return confirmed;
360
362
  },
361
363
  };
364
+ const sample=engine.run.bind(engine),closeBrowser=engine.close.bind(engine);
365
+ const policyCodes=new Set(['PROXY_REQUIRED','PROXY_POLICY_INVALID','PROXY_POLICY_CONFLICT','PROXY_CONFIGURATION_INVALID','PROXY_ACQUIRE_FAILED','PROXY_SUPPLIER_INVALID','PROXY_LEASE_INVALID','PROXY_LEASE_EXPIRED','PROXY_QUARANTINE_FAILED','PROXY_RELEASE_FAILED','PROXY_RECEIPT_FAILED','PROXY_CUSTODY_UNAVAILABLE','BROWSER_CLEANUP_UNCONFIRMED','BROWSER_RUN_IN_PROGRESS']);
366
+ const safeError=cause=>{
367
+ const code=policyCodes.has(cause?.code)?cause.code:'BROWSER_RUN_FAILED';
368
+ const error=Object.assign(new EngineError(code==='BROWSER_RUN_FAILED'?'Browser sample failed.':'Browser admission or cleanup failed.',{retriable:code==='BROWSER_RUN_FAILED'&&cause?.retriable===true}),{code});
369
+ if(code==='BROWSER_RUN_FAILED'||cause?.unknown===true)error.unknown=true;
370
+ return error;
371
+ };
372
+ async function retire(phase,reason=null){
373
+ const hadResources=Boolean(browserHandle||displayHandle||admission.proxy);let error,confirmed=false;
374
+ try{
375
+ try{confirmed=await closeBrowser();}catch{confirmed=false;}
376
+ if(!confirmed)reason='cleanup_unconfirmed';
377
+ try{if(hadResources)await admission.record({outcome:confirmed?'closed':'cleanup_unconfirmed',phase,reason});}catch(cause){error=cause;}
378
+ try{if(reason)await admission.quarantine(reason);}catch(cause){error??=cause;}
379
+ if(!confirmed)error??=Object.assign(new EngineError('Browser cleanup could not be confirmed.'),{code:'BROWSER_CLEANUP_UNCONFIRMED'});
380
+ }finally{try{await admission.release();}catch(cause){error??=cause;}}
381
+ if(error)throw safeError(error);
382
+ }
383
+ engine.run=async input=>{
384
+ if(running)throw Object.assign(new EngineError('A browser sample is already running.'),{code:'BROWSER_RUN_IN_PROGRESS'});
385
+ running=true;let result,error,reason=null,admitted=false;
386
+ try{
387
+ if(admission.expired)await retire('lease_expiry');
388
+ await admission.begin();admitted=true;
389
+ result=await sample(input);
390
+ if(contextCleanupFailed)throw Object.assign(new EngineError('Browser cleanup could not be confirmed.'),{code:'BROWSER_CLEANUP_UNCONFIRMED'});
391
+ }catch(cause){reason=proxyFailureReason(cause);error=safeError(cause);}
392
+ try{
393
+ if(admitted){
394
+ try{const receipt=await admission.record({outcome:error?'failed':result?.unknown?'unknown':'observed',reason});if(result)result.egressReceipt=receipt;}
395
+ catch(cause){error=safeError(cause);}
396
+ if(error){try{await retire('failure_cleanup',reason);}catch(cause){error=safeError(cause);}}
397
+ }
398
+ }finally{running=false;}
399
+ if(error)throw error;
400
+ return result;
401
+ };
402
+ engine.close=async()=>{
403
+ if(running)throw Object.assign(new EngineError('A browser sample is already running.'),{code:'BROWSER_RUN_IN_PROGRESS'});
404
+ running=true;
405
+ try{await retire('cleanup');}finally{running=false;}
406
+ };
407
+ engine.getEgressReceipts=admission.receipts;
408
+ return engine;
362
409
  }
@@ -0,0 +1,120 @@
1
+ // Local proxy custody. Supplier callbacks perform no work until browser admission.
2
+ // Meter values are supplier-observed cumulative counters, never nominal estimates.
3
+ import {createHash,randomUUID} from 'node:crypto';
4
+ import {EngineError} from '../adapter.js';
5
+ const digest=value=>createHash('sha256').update(value).digest('hex');
6
+ const quarantines=new Map();
7
+ const safeInteger=value=>Number.isSafeInteger(value)&&value>=0;
8
+ const failure=(code,message)=>Object.assign(new EngineError(message),{code});
9
+ export function parseProxyUrl(raw){
10
+ try{
11
+ if(typeof raw!=='string'||raw.length>8192)throw Error();
12
+ const url=new URL(raw);
13
+ if(!['http:','https:','socks5:'].includes(url.protocol)||!url.hostname||url.search||url.hash||(url.pathname&&url.pathname!=='/')||(url.protocol==='socks5:'&&!url.port))throw Error();
14
+ const username=decodeURIComponent(url.username),password=decodeURIComponent(url.password);
15
+ if(/[\r\n\0]/.test(username+password))throw Error();
16
+ return {server:`${url.protocol}//${url.hostname}${url.port?':'+url.port:''}`,...(username?{username}:{}),...(password?{password}:{})};
17
+ }catch{throw failure('PROXY_CONFIGURATION_INVALID','The browser proxy configuration is invalid.');}
18
+ }
19
+ export function createUrlProxySupplier({url,now=()=>Date.now()}={}){
20
+ const key=typeof url==='string'?digest(url):null;
21
+ return {
22
+ async acquire({engine}){
23
+ if(!url)throw failure('PROXY_REQUIRED','Configure a proxy before browser measurement, or explicitly select direct-diagnostic egress.');
24
+ parseProxyUrl(url);
25
+ if((quarantines.get(engine+key)??0)>now())throw failure('PROXY_QUARANTINED','The browser proxy is quarantined.');
26
+ return {id:randomUUID(),proxyUrl:url,expiresAt:new Date(now()+10*60*1000).toISOString()};
27
+ },
28
+ async quarantine({engine,reason}){quarantines.set(engine+key,reason==='cleanup_unconfirmed'?Infinity:now()+10*60*1000);},
29
+ async release(){},
30
+ };
31
+ }
32
+ // Only explicit structured signals identify a block. Empty answers/selector errors
33
+ // do not establish a proxy problem, and arbitrary exception text is never retained.
34
+ export function proxyFailureReason(error){
35
+ for(const value of [error,error?.cause]){
36
+ if(value?.status===403||value?.failureType==='bot_detection')return 'blocked';
37
+ if(value?.status===429||value?.failureType==='rate_limited')return 'rate_limited';
38
+ if(['ERR_PROXY_CONNECTION_FAILED','ECONNREFUSED','ECONNRESET','ETIMEDOUT'].includes(value?.code)||value?.failureType==='connection_error'||/\bERR_PROXY_CONNECTION_FAILED\b/.test(String(value?.message??'')))return 'connection_error';
39
+ }
40
+ return null;
41
+ }
42
+ export function createProxyAdmission({engine,env={},supplier,now=()=>Date.now(),onReceipt=()=>{},timeoutMs=5000}={}){
43
+ const mode=env.AGENTLINKOPS_BROWSER_EGRESS??'proxy-required';
44
+ if(!['proxy-required','direct-diagnostic'].includes(mode))throw failure('PROXY_POLICY_INVALID','Choose proxy-required or direct-diagnostic browser egress.');
45
+ if(mode==='direct-diagnostic'&&(supplier||env.AGENTLINKOPS_PROXY_URL))throw failure('PROXY_POLICY_CONFLICT','Direct diagnostics cannot also configure a proxy.');
46
+ const injected=Boolean(supplier);
47
+ supplier??=createUrlProxySupplier({url:env.AGENTLINKOPS_PROXY_URL,now});
48
+ let lease=null,baseline=null,poisoned=false;const receipts=[];
49
+ async function bounded(action,code){
50
+ let timer;
51
+ try{return await Promise.race([Promise.resolve().then(action),new Promise((_,reject)=>{timer=setTimeout(()=>reject(Error()),timeoutMs);})]);}
52
+ catch{throw failure(code,'Browser proxy custody failed. No direct fallback was attempted.');}
53
+ finally{clearTimeout(timer);}
54
+ }
55
+ async function meter(){
56
+ if(!lease||typeof supplier.readMeter!=='function')return null;
57
+ try{
58
+ const value=await bounded(()=>supplier.readMeter({engine,leaseId:lease.id}),'PROXY_METER_UNAVAILABLE');
59
+ if(!value||!safeInteger(value.bytes)||!(value.costMicrousd===null||value.costMicrousd===undefined||safeInteger(value.costMicrousd)))return null;
60
+ return {bytes:value.bytes,costMicrousd:value.costMicrousd??null};
61
+ }catch{return null;}
62
+ }
63
+ async function release(){
64
+ if(!lease)return;
65
+ const previous=lease;lease=null;baseline=null;
66
+ try{await bounded(()=>supplier.release({engine,leaseId:previous.id,disposition:poisoned?'quarantined':'clean'}),'PROXY_RELEASE_FAILED');}
67
+ catch(error){poisoned=true;throw error;}
68
+ }
69
+ return {
70
+ get proxy(){return lease?.proxy;},
71
+ get expired(){return lease&&Date.parse(lease.expiresAt)<=now();},
72
+ assertReady(){
73
+ if(poisoned)throw failure('PROXY_CUSTODY_UNAVAILABLE','Browser proxy custody needs operator review.');
74
+ if(mode==='proxy-required'&&(!lease||Date.parse(lease.expiresAt)<=now()))throw failure('PROXY_LEASE_EXPIRED','The browser proxy lease has expired.');
75
+ },
76
+ async begin(){
77
+ if(poisoned)throw failure('PROXY_CUSTODY_UNAVAILABLE','Browser proxy custody needs operator review.');
78
+ if(mode==='direct-diagnostic')return;
79
+ if(!injected&&!env.AGENTLINKOPS_PROXY_URL)throw failure('PROXY_REQUIRED','Configure a proxy before browser measurement, or explicitly select direct-diagnostic egress.');
80
+ if(lease){if(Date.parse(lease.expiresAt)<=now())throw failure('PROXY_LEASE_EXPIRED','The browser proxy lease has expired.');return;}
81
+ if(!['acquire','release','quarantine'].every(name=>typeof supplier?.[name]==='function'))throw failure('PROXY_SUPPLIER_INVALID','The browser proxy supplier is incomplete.');
82
+ let acquired;
83
+ const acquiring=Promise.resolve().then(()=>supplier.acquire({engine}));
84
+ try{acquired=await bounded(()=>acquiring,'PROXY_ACQUIRE_FAILED');}
85
+ catch(error){
86
+ poisoned=true;
87
+ // A timed-out acquisition can resolve later; retire that lease without launching.
88
+ acquiring.then(value=>{if(typeof value?.id==='string')return bounded(()=>supplier.release({engine,leaseId:value.id}),'PROXY_RELEASE_FAILED');}).catch(()=>{});
89
+ throw error;
90
+ }
91
+ try{
92
+ if(!acquired||typeof acquired.id!=='string'||!/^[A-Za-z0-9_-]{1,128}$/.test(acquired.id)||typeof acquired.expiresAt!=='string'||!Number.isFinite(Date.parse(acquired.expiresAt))||Date.parse(acquired.expiresAt)<=now())throw Error();
93
+ lease={id:acquired.id,expiresAt:acquired.expiresAt,proxy:parseProxyUrl(acquired.proxyUrl)};
94
+ }catch{
95
+ poisoned=true;
96
+ if(acquired&&typeof acquired.id==='string')await bounded(()=>supplier.release({engine,leaseId:acquired.id}),'PROXY_RELEASE_FAILED').catch(()=>{});
97
+ throw failure('PROXY_LEASE_INVALID','The browser proxy lease is invalid.');
98
+ }
99
+ baseline=await meter();
100
+ if(Date.parse(lease.expiresAt)<=now()){poisoned=true;await release();throw failure('PROXY_LEASE_EXPIRED','The browser proxy lease has expired.');}
101
+ },
102
+ async record({outcome,reason=null,phase='sample'}){
103
+ const current=await meter(),valid=baseline&&current&&current.bytes>=baseline.bytes;
104
+ const bytes=valid?current.bytes-baseline.bytes:null;
105
+ const costMicrousd=valid&&baseline.costMicrousd!==null&&current.costMicrousd!==null&&current.costMicrousd>=baseline.costMicrousd?current.costMicrousd-baseline.costMicrousd:null;
106
+ const row=Object.freeze({version:1,sequence:receipts.length+1,engine,egress:mode,phase,outcome,reason,leaseHash:lease?digest(lease.id):null,observedAt:new Date(now()).toISOString(),meterState:outcome==='cleanup_unconfirmed'?'incomplete':mode==='direct-diagnostic'?'not_applicable':bytes===null?'unavailable':costMicrousd===null?'bytes_only':'measured',bytes,costMicrousd});
107
+ baseline=current;receipts.push(row);
108
+ try{await bounded(()=>onReceipt({...row}),'PROXY_RECEIPT_FAILED');}catch(error){poisoned=true;throw error;}
109
+ return {...row};
110
+ },
111
+ async quarantine(reason){
112
+ // Stop this engine instance even if an external supplier cannot quarantine.
113
+ poisoned=true;
114
+ if(!lease)return;
115
+ await bounded(()=>supplier.quarantine({engine,leaseId:lease.id,reason}),'PROXY_QUARANTINE_FAILED');
116
+ },
117
+ release,
118
+ receipts:()=>receipts.map(row=>({...row})),
119
+ };
120
+ }
@@ -0,0 +1,80 @@
1
+ // Supplemental operational evidence; never part of citation-envelope hashes.
2
+ import { open, mkdir } from 'node:fs/promises';
3
+ import { join } from 'node:path';
4
+ import { EngineError } from './adapter.js';
5
+ export const EGRESS_LIMITS = Object.freeze({ rows: 10000, bytes: 8 * 1024 * 1024 });
6
+ const fail = code => Object.assign(new EngineError('Egress evidence could not be persisted; the epoch stopped.'), { code });
7
+ const codes = new Set(['PROXY_REQUIRED','PROXY_POLICY_INVALID','PROXY_POLICY_CONFLICT','PROXY_CONFIGURATION_INVALID','PROXY_ACQUIRE_FAILED','PROXY_SUPPLIER_INVALID','PROXY_LEASE_INVALID','PROXY_LEASE_EXPIRED','PROXY_QUARANTINED','PROXY_QUARANTINE_FAILED','PROXY_RELEASE_FAILED','PROXY_RECEIPT_FAILED','PROXY_CUSTODY_UNAVAILABLE','BROWSER_CLEANUP_UNCONFIRMED','BROWSER_RUN_IN_PROGRESS','BROWSER_RUN_FAILED']);
8
+ const safeCode = error => error ? codes.has(error.code) ? error.code : 'ENGINE_ERROR' : null;
9
+ const enumeration = (value, values) => { if (!values.includes(value)) throw fail('EGRESS_RECEIPT_INVALID'); return value; };
10
+ const count = value => { if (!Number.isSafeInteger(value) || value < 0) throw fail('EGRESS_RECEIPT_INVALID'); return value; };
11
+ function sanitize(row) {
12
+ if (!row || row.version !== 1 || count(row.sequence) < 1) throw fail('EGRESS_RECEIPT_INVALID');
13
+ const observedAt = typeof row.observedAt === 'string' && row.observedAt.length <= 35 ? new Date(row.observedAt) : null;
14
+ if (!observedAt || !Number.isFinite(observedAt.getTime())) throw fail('EGRESS_RECEIPT_INVALID');
15
+ if (row.leaseHash !== null && !/^[a-f0-9]{64}$/.test(row.leaseHash ?? '')) throw fail('EGRESS_RECEIPT_INVALID');
16
+ const meterState = enumeration(row.meterState, ['measured','bytes_only','unavailable','not_applicable','incomplete']);
17
+ const bytes = row.bytes === null ? null : count(row.bytes);
18
+ const costMicrousd = row.costMicrousd === null ? null : count(row.costMicrousd);
19
+ if ((meterState === 'measured' && (bytes === null || costMicrousd === null)) || (meterState === 'bytes_only' && (bytes === null || costMicrousd !== null)) || (['unavailable','not_applicable'].includes(meterState) && (bytes !== null || costMicrousd !== null))) throw fail('EGRESS_RECEIPT_INVALID');
20
+ return { version: 1, sequence: row.sequence,
21
+ egress: enumeration(row.egress, ['proxy-required','direct-diagnostic']),
22
+ phase: enumeration(row.phase, ['sample','cleanup','failure_cleanup','lease_expiry']),
23
+ outcome: enumeration(row.outcome, ['observed','unknown','failed','closed','cleanup_unconfirmed']),
24
+ reason: enumeration(row.reason, [null,'blocked','rate_limited','connection_error','cleanup_unconfirmed']),
25
+ leaseHash: row.leaseHash, observedAt: observedAt.toISOString(), meterState, bytes, costMicrousd };
26
+ }
27
+
28
+ export async function createEgressJournal({ dir, epochId, engines, identityHashes = new Map(), limits = EGRESS_LIMITS }) {
29
+ if (typeof epochId !== 'string' || !/^[A-Za-z0-9_-]{1,100}$/.test(epochId)) throw fail('EGRESS_JOURNAL_UNAVAILABLE');
30
+ const tracked = new Map(engines.map((engine, index) => [engine, { index, retained: [] }]));
31
+ if (!tracked.size) return null;
32
+ if (!Number.isSafeInteger(limits.rows) || limits.rows < 1 || !Number.isSafeInteger(limits.bytes) || limits.bytes < 1) throw fail('EGRESS_LIMIT_INVALID');
33
+ // A reusable adapter may retain earlier epochs in memory. Establish the cursor
34
+ // before this epoch's first request; never count or rewrite those older deltas.
35
+ for (const [engine, state] of tracked) {
36
+ try {
37
+ const prior = engine.getEgressReceipts?.() ?? [];
38
+ if (!Array.isArray(prior) || prior.length > limits.rows) throw fail('EGRESS_RECEIPT_LIMIT');
39
+ state.retained = prior.map((row, index) => {
40
+ const value = sanitize(row);
41
+ if (value.sequence !== index + 1) throw fail('EGRESS_RECEIPT_INVALID');
42
+ return JSON.stringify(value);
43
+ });
44
+ } catch { throw fail('EGRESS_RECEIPT_UNAVAILABLE'); }
45
+ }
46
+ let file;
47
+ try { await mkdir(join(dir, 'citations/egress'), { recursive: true }); file = await open(join(dir, 'citations/egress', `${epochId}.jsonl`), 'wx', 0o600); }
48
+ catch { throw fail('EGRESS_JOURNAL_UNAVAILABLE'); }
49
+ let rows = 0, bytes = 0;
50
+ async function append(row) {
51
+ const text = JSON.stringify(row) + '\n', size = Buffer.byteLength(text);
52
+ if (rows + 1 > limits.rows || bytes + size > limits.bytes) throw fail('EGRESS_RECEIPT_LIMIT');
53
+ try { await file.writeFile(text); await file.sync(); }
54
+ catch { throw fail('EGRESS_WRITE_FAILED'); }
55
+ rows++; bytes += size;
56
+ }
57
+ try { await append({ type: 'start', version: 1, epochId, engines: [...tracked].map(([engine, state]) => ({ engineIndex: state.index, firstSequence: state.retained.length + 1, identityHash: /^[a-f0-9]{64}$/.test(identityHashes.get(engine) ?? '') ? identityHashes.get(engine) : null })), completeness: 'open', costBasis: 'supplier_meter_only', totals: null }); }
58
+ catch (error) { await file.close().catch(() => {}); throw error; }
59
+ return {
60
+ async capture(engine, { phase, error = null, cellHash = null, runIndex = null, attempt = null } = {}) {
61
+ const state = tracked.get(engine); if (!state) return;
62
+ if (cellHash !== null && !/^[a-f0-9]{64}$/.test(cellHash)) throw fail('EGRESS_RECEIPT_INVALID');
63
+ if (runIndex !== null) count(runIndex);
64
+ if (attempt !== null) count(attempt);
65
+ let receipts;
66
+ try { receipts = engine.getEgressReceipts?.() ?? []; }
67
+ catch { throw fail('EGRESS_RECEIPT_UNAVAILABLE'); }
68
+ if (!Array.isArray(receipts) || receipts.length > limits.rows || receipts.length < state.retained.length) throw fail('EGRESS_RECEIPT_LIMIT');
69
+ for (let i = 0; i < receipts.length; i++) {
70
+ const receipt = sanitize(receipts[i]), serialized = JSON.stringify(receipt);
71
+ if (receipt.sequence !== i + 1 || (i < state.retained.length && state.retained[i] !== serialized)) throw fail('EGRESS_RECEIPT_INVALID');
72
+ if (i < state.retained.length) continue;
73
+ await append({ type: 'receipt', engineIndex: state.index, receipt }); state.retained.push(serialized);
74
+ }
75
+ await append({ type: 'status', engineIndex: state.index, phase: enumeration(phase, ['attempt','cleanup']), outcome: error ? 'error' : 'returned', code: safeCode(error), cellHash, runIndex, attempt, receiptsAvailable: typeof engine.getEgressReceipts === 'function' });
76
+ },
77
+ async finish({ failed, cleanupUncertain, evidenceComplete }) { await append({ type: 'finish', outcome: failed ? 'failed' : 'returned', cleanupUncertain, evidenceComplete, totals: null }); },
78
+ async close() { try { await file.close(); } catch { throw fail('EGRESS_WRITE_FAILED'); } },
79
+ };
80
+ }
@@ -0,0 +1,51 @@
1
+ import { createHash } from 'node:crypto';
2
+ const hash = value => createHash('sha256').update(value).digest('hex');
3
+
4
+ function suppliedFieldsPreserved(raw, parsed) {
5
+ if (raw === null || typeof raw !== 'object') return Object.is(raw, parsed);
6
+ if (Array.isArray(raw)) return Array.isArray(parsed) && raw.length === parsed.length && raw.every((value, index) => suppliedFieldsPreserved(value, parsed[index]));
7
+ return parsed !== null && typeof parsed === 'object' && !Array.isArray(parsed) &&
8
+ Object.keys(raw).every(key => Object.hasOwn(parsed, key) && suppliedFieldsPreserved(raw[key], parsed[key]));
9
+ }
10
+
11
+ // Historical files were written by JSON.stringify. Retain whitespace and key
12
+ // order compatibility, but reject duplicate keys and alternate encodings that
13
+ // JSON.parse would discard before semantic reconstruction. Quoted whitespace
14
+ // remains part of its token. This is not a replacement JSON grammar parser.
15
+ function canonicalEncoding(text, raw) {
16
+ return (text.match(/"(?:\\[\s\S]|[^"\\])*"|[^\s]/g) ?? []).join('') === JSON.stringify(raw);
17
+ }
18
+
19
+ // The v1 runner hashes its construction-order object BEFORE schema parsing and
20
+ // before adding screenshot_file. Do not rename or rewrite those historical files.
21
+ export function verifyLocalEvidenceDigest({ envelope, text, digest }) {
22
+ if (!envelope || typeof text !== 'string' || !/^[a-f0-9]{64}$/.test(digest ?? '')) return false;
23
+ try {
24
+ const raw = JSON.parse(text);
25
+ if (!suppliedFieldsPreserved(raw, envelope) || !canonicalEncoding(text, raw)) return false;
26
+ } catch { return false; }
27
+ if (envelope.screenshot_file && envelope.screenshot_file !== `${digest}.screenshot.png`) return false;
28
+ if (hash(text) === digest) return true;
29
+ const original = {
30
+ schema_version: envelope.schema_version, epoch_id: envelope.epoch_id,
31
+ cell_id: envelope.cell_id, run_index: envelope.run_index, prompt: envelope.prompt,
32
+ engine_identity: envelope.engine_identity,
33
+ ...(envelope.locale_context ? { locale_context: envelope.locale_context } : {}),
34
+ provider_model_version: envelope.provider_model_version,
35
+ answer: envelope.answer, citations: envelope.citations, fan_out: envelope.fan_out,
36
+ usage: envelope.usage, cost_estimate_usd: envelope.cost_estimate_usd, at: envelope.at,
37
+ ...(envelope.provenance ? { provenance: envelope.provenance } : {}),
38
+ ...(envelope.browser_context ? { browser_context: envelope.browser_context } : {}),
39
+ ...(envelope.failure ? { failure: envelope.failure } : {}),
40
+ };
41
+ // Token counts default to zero at schema parse. Try only those lossless zero
42
+ // defaults, not arbitrary omitted evidence or alternative citation contents.
43
+ for (const omitInput of [false, true]) for (const omitOutput of [false, true]) {
44
+ if ((omitInput && envelope.usage.input_tokens !== 0) || (omitOutput && envelope.usage.output_tokens !== 0)) continue;
45
+ const usage = { ...envelope.usage };
46
+ if (omitInput) delete usage.input_tokens;
47
+ if (omitOutput) delete usage.output_tokens;
48
+ if (hash(JSON.stringify({ ...original, usage })) === digest) return true;
49
+ }
50
+ return false;
51
+ }
@@ -1,20 +1,45 @@
1
1
  import { readFile, readdir, lstat, realpath } from 'node:fs/promises';
2
2
  import { join, resolve } from 'node:path';
3
3
  import { createHash } from 'node:crypto';
4
- import { citationPanelSchema, evidenceEnvelopeSchema, targetKey } from './contract.js';
4
+ import { citationPanelSchema, evidenceEnvelopeSchema, observationRowSchema, targetKey } from './contract.js';
5
5
  import { panelEpochs } from './report.js';
6
+ import { verifyLocalEvidenceDigest } from './evidence-integrity.js';
6
7
  import { analyzeRetainedAnswer } from './match.js';
7
8
  import { parseOpportunityRecord, opportunityRecordId, ELIGIBILITY_GATES } from '../../shared/opportunity-record.js';
8
9
  const hash = value => createHash('sha256').update(value).digest('hex');
9
10
 
11
+ async function observationReferences(root, epochId, cells) {
12
+ const path = join(root, 'citations-observations.jsonl');
13
+ let text;
14
+ try {
15
+ if (!(await lstat(path)).isFile() || !(await realpath(path)).startsWith(root + '/')) throw new Error('Unsafe observation ledger');
16
+ text = await readFile(path, 'utf8');
17
+ } catch (error) { if (error.code === 'ENOENT') return new Map(); throw error; }
18
+ const references = new Map();
19
+ for (const line of text.split('\n').filter(line => line.trim())) {
20
+ let raw;
21
+ try { raw = JSON.parse(line); } catch { throw new Error('Invalid observation ledger JSON'); }
22
+ if (raw.epoch_id !== epochId || !cells.some(cell => cell.cell_id === raw.cell_id)) continue;
23
+ const parsed = observationRowSchema.safeParse(raw);
24
+ if (!parsed.success || !/^[a-f0-9]{64}$/.test(parsed.data.evidence_sha256)) throw new Error('Invalid retained observation reference');
25
+ const row = parsed.data, identity = `${row.cell_id}|${row.run_index}`;
26
+ const previous = references.get(identity);
27
+ if (previous && JSON.stringify(previous) !== JSON.stringify(row)) throw new Error('Conflicting retained observation identity');
28
+ references.set(identity, row);
29
+ }
30
+ return references;
31
+ }
32
+
10
33
  export async function freezeCitationInventory({ dir, panel: input, epochId }) {
11
34
  const panel = citationPanelSchema.parse(input);
12
35
  const epochs = await panelEpochs({ dir, panel });
13
36
  const cells = epochId ? epochs.find(rows => rows[0]?.epoch_id === epochId) : epochs[0];
14
37
  if (!cells) throw new Error('No matching panel epoch to freeze');
38
+ if (cells.some(cell => !Number.isSafeInteger(cell.n) || cell.n < 0 || !Number.isSafeInteger(cell.unknowns) || cell.unknowns < 0 || !Number.isSafeInteger(cell.n + cell.unknowns))) throw new Error('Invalid retained epoch sample counts');
15
39
  epochId = cells[0].epoch_id;
16
40
  if (!/^[A-Za-z0-9_.-]+$/.test(epochId) || ['.', '..'].includes(epochId)) throw new Error('Invalid epoch reference');
17
41
  const root = await realpath(resolve(dir)); const evidenceDir = join(root, 'citations/evidence', epochId);
42
+ const references = await observationReferences(root, epochId, cells);
18
43
  let files = [];
19
44
  try {
20
45
  if (!(await lstat(evidenceDir)).isDirectory() || !(await realpath(evidenceDir)).startsWith(root + '/')) throw new Error('Unsafe evidence directory');
@@ -28,21 +53,26 @@ export async function freezeCitationInventory({ dir, panel: input, epochId }) {
28
53
  let envelope, text;
29
54
  try { text = await readFile(path, 'utf8'); envelope = evidenceEnvelopeSchema.parse(JSON.parse(text)); }
30
55
  catch { invalid++; continue; }
31
- if (envelope.epoch_id !== epochId || !cells.some(cell => cell.cell_id === envelope.cell_id)) continue;
56
+ const cell = cells.find(cell => cell.cell_id === envelope.cell_id);
57
+ if (envelope.epoch_id !== epochId || !cell) { invalid++; continue; }
32
58
  const identity = `${envelope.cell_id}|${envelope.run_index}`;
59
+ const reference = references.get(identity), digest = file.slice(0, -5);
60
+ if (!reference || reference.evidence_sha256 !== digest || envelope.run_index >= cell.n + cell.unknowns ||
61
+ !verifyLocalEvidenceDigest({ envelope, text, digest })) { invalid++; continue; }
33
62
  if (identities.has(identity)) throw new Error('Duplicate observation identity in retained evidence'); identities.add(identity);
34
63
  const [, promptId, key] = envelope.cell_id.split('|');
35
64
  const target = panel.targets.find(target => targetKey(target) === key);
36
65
  if (!target) continue;
37
66
  const prompt = panel.prompts.find(prompt => { const id = prompt.id ?? hash(prompt.text).slice(0, 10); return id === promptId || promptId?.startsWith(`${id}@`); });
38
- const analysis = envelope.engine_identity.startsWith('google-aio:') && !envelope.provenance ? { mentioned: null, cited: null, outcome: 'unknown' } : analyzeRetainedAnswer(target, envelope);
67
+ if (!prompt || prompt.text !== envelope.prompt || envelope.engine_identity !== envelope.cell_id.split('|')[0]) { invalid++; continue; }
68
+ const analysis = reference.outcome === 'unknown' || (envelope.engine_identity.startsWith('google-aio:') && !envelope.provenance) ? { mentioned: null, cited: null, outcome: 'unknown' } : analyzeRetainedAnswer(target, envelope);
39
69
  observations.push({ cell_id: envelope.cell_id, run_index: envelope.run_index, engine_identity: envelope.engine_identity,
40
70
  prompt_id: promptId, prompt: envelope.prompt, headline_eligible: !prompt?.branded, prompt_provenance: prompt?.provenance ?? null,
41
71
  intent_cluster: prompt?.intent_cluster ?? null, target: key, observed_at: envelope.at,
42
72
  provider_model_version: envelope.provider_model_version, locale_context: envelope.locale_context ?? null,
43
73
  mentioned: analysis.mentioned, cited: analysis.cited, outcome: analysis.outcome,
44
74
  citations: analysis.outcome === 'unknown' ? [] : envelope.citations,
45
- evidence_sha256: hash(text), fixture: envelope.engine_identity.startsWith('mock:') });
75
+ evidence_sha256: hash(text), local_evidence_sha256: digest, fixture: envelope.engine_identity.startsWith('mock:') });
46
76
  }
47
77
  const expected = cells.reduce((count, cell) => count + (cell.n ?? 0) + (cell.unknowns ?? 0), 0);
48
78
  const payload = { schema_version: 1, kind: 'frozen_citation_inventory', epoch_id: epochId, frozen_at: new Date().toISOString(),
@@ -14,6 +14,7 @@ import { citationPanelSchema, engineIdentity, cellId, targetKey, panelLocaleCont
14
14
  import { classifyOutcome, analyzeRetainedAnswer } from './match.js';
15
15
  import { wilsonInterval, smoothedRate, classifyEpoch, tallyOutcomes, confidenceSequence, compareEpochs, changepointPosterior } from './stats.js';
16
16
  import { EngineError } from './adapter.js';
17
+ import { createEgressJournal } from './egress-receipts.js';
17
18
 
18
19
  const sha256 = (text) => createHash('sha256').update(text).digest('hex');
19
20
  const nowIso = () => new Date().toISOString();
@@ -50,157 +51,178 @@ async function readPriorEpochs(dir) {
50
51
  * Retriable failures receive one retry; exhausted attempts remain unknown.
51
52
  */
52
53
  export async function runEpoch(panelInput, options = {}) {
53
- const panel = citationPanelSchema.parse(panelInput);
54
- const dir = options.dir ?? '.agentlinkops';
55
- const now = options.now ?? new Date();
56
- const samples = panel.samples ?? CITATION_LIMITS.defaultSamples;
57
- const maxUsd = panel.maxUsd ?? CITATION_LIMITS.defaultMaxUsd;
58
- const delayMs = options.delayMs ?? 0; // mock/tests: 0; live callers pass pacing
54
+ // Supplied adapters transfer epoch cleanup responsibility to this runner.
55
+ const uniqueEngines = [...new Set(options.engines?.values() ?? [])];
56
+ let journal, primaryError;
57
+ try {
58
+ const panel = citationPanelSchema.parse(panelInput);
59
+ const dir = options.dir ?? '.agentlinkops';
60
+ const now = options.now ?? new Date();
61
+ const samples = panel.samples ?? CITATION_LIMITS.defaultSamples;
62
+ const maxUsd = panel.maxUsd ?? CITATION_LIMITS.defaultMaxUsd;
63
+ const delayMs = options.delayMs ?? 0; // mock/tests: 0; live callers pass pacing
59
64
 
60
- const epochId = `${now.toISOString().slice(0, 10)}-${sha256(JSON.stringify({ panel, at: now.toISOString() })).slice(0, 10)}`;
61
- const evidenceRoot = join(dir, CITATION_FILES.evidenceDir, epochId);
62
- await mkdir(evidenceRoot, { recursive: true });
65
+ const epochId = `${now.toISOString().slice(0, 10)}-${sha256(JSON.stringify({ panel, at: now.toISOString() })).slice(0, 10)}`;
66
+ const evidenceRoot = join(dir, CITATION_FILES.evidenceDir, epochId);
67
+ await mkdir(evidenceRoot, { recursive: true });
63
68
 
64
- const cells = [];
65
- for (const engineSpec of panel.engines) {
66
- const identity = engineIdentity(engineSpec);
67
- const engine = options.engines?.get(identity);
68
- const context = panelLocaleContext(panel, engineSpec);
69
- if ((context.locale !== 'en-US' || context.country !== 'US') && !['mock', 'google-aio'].includes(engineSpec.engine)) throw new EngineError('this engine has only an unverified en-US/US context; use mock or a configured supplier for locale variants');
70
- if (!engine) throw new EngineError(`no engine instance provided for ${identity}`);
71
- for (const prompt of panel.prompts) {
72
- for (const target of panel.targets) {
73
- cells.push({ identity, prompt, target, engine, context, id: cellId(identity, localizedPromptId(prompt.id ?? sha256(prompt.text).slice(0, 10), context), targetKey(target)) });
69
+ const cells = [];
70
+ for (const engineSpec of panel.engines) {
71
+ const identity = engineIdentity(engineSpec);
72
+ const engine = options.engines?.get(identity);
73
+ const context = panelLocaleContext(panel, engineSpec);
74
+ if ((context.locale !== 'en-US' || context.country !== 'US') && !['mock', 'google-aio'].includes(engineSpec.engine)) throw new EngineError('this engine has only an unverified en-US/US context; use mock or a configured supplier for locale variants');
75
+ if (!engine) throw new EngineError(`no engine instance provided for ${identity}`);
76
+ for (const prompt of panel.prompts) {
77
+ for (const target of panel.targets) {
78
+ cells.push({ identity, prompt, target, engine, context, id: cellId(identity, localizedPromptId(prompt.id ?? sha256(prompt.text).slice(0, 10), context), targetKey(target)) });
79
+ }
74
80
  }
75
81
  }
76
- }
77
82
 
78
- let spent = 0;
79
- let aborted = null;
80
- const observationRows = [];
81
- const outcomesByCell = new Map(cells.map((c) => [c.id, []]));
83
+ let spent = 0;
84
+ let aborted = null;
85
+ const observationRows = [];
86
+ const outcomesByCell = new Map(cells.map((c) => [c.id, []]));
82
87
 
83
- outer:
84
- for (const cell of cells) {
85
- for (let runIndex = 0; runIndex < samples; runIndex += 1) {
86
- const estimate = cell.engine.estimateCostUsd?.() ?? 0;
87
- if (!Number.isFinite(estimate) || estimate < 0) throw new EngineError('engine cost estimate must be a finite nonnegative number');
88
- let run = null, lastCause = null, attemptCost = 0;
89
- for (let attempt = 0; attempt < 2 && !run; attempt += 1) {
90
- // Reserve each attempt, including retries, before calling a supplier.
91
- if (spent + estimate > maxUsd) {
92
- aborted = { reason: 'budget', spent, nextEstimate: estimate, maxUsd };
93
- break;
88
+ const browserEngines = uniqueEngines.filter(engine => typeof engine.getEgressReceipts === 'function' || cells.some(cell => cell.engine === engine && cell.identity.includes(':web-own-browser')));
89
+ journal = await createEgressJournal({ dir, epochId, engines: browserEngines, identityHashes: new Map(cells.map(cell => [cell.engine, sha256(cell.identity)])) });
90
+ outer:
91
+ for (const cell of cells) {
92
+ for (let runIndex = 0; runIndex < samples; runIndex += 1) {
93
+ const estimate = cell.engine.estimateCostUsd?.() ?? 0;
94
+ if (!Number.isFinite(estimate) || estimate < 0) throw new EngineError('engine cost estimate must be a finite nonnegative number');
95
+ let run = null, lastCause = null, attemptCost = 0;
96
+ for (let attempt = 0; attempt < 2 && !run; attempt += 1) {
97
+ // Reserve each attempt, including retries, before calling a supplier.
98
+ if (spent + estimate > maxUsd) {
99
+ aborted = { reason: 'budget', spent, nextEstimate: estimate, maxUsd };
100
+ break;
101
+ }
102
+ spent += estimate;
103
+ attemptCost += estimate;
104
+ let attemptError;
105
+ try {
106
+ run = await cell.engine.run({ prompt: cell.prompt.text, runIndex, localeContext: cell.context, maxCostUsd: estimate || maxUsd - spent });
107
+ const actual = run.costEstimateUsd ?? estimate;
108
+ if (!Number.isFinite(actual) || actual < 0) throw new EngineError('engine returned an invalid cost');
109
+ spent += actual - estimate;
110
+ attemptCost += actual - estimate;
111
+ if (spent > maxUsd) aborted = { reason: 'supplier_cost_exceeded', spent, maxUsd };
112
+ } catch (cause) {
113
+ attemptError = cause;
114
+ run = null;
115
+ lastCause = cause;
116
+ // Failed or timed-out requests may already be billed. Keep the
117
+ // reservation unless the supplier reports a greater charge.
118
+ const charged = Number.isFinite(cause?.costEstimateUsd) ? Math.max(estimate, cause.costEstimateUsd) : estimate;
119
+ spent += charged - estimate;
120
+ attemptCost += charged - estimate;
121
+ if (spent > maxUsd) aborted = { reason: 'supplier_cost_exceeded', spent, maxUsd };
122
+ if (!(cause instanceof EngineError) || (!cause.retriable && !cause.unknown)) throw cause;
123
+ if (!cause.retriable || aborted) break;
124
+ if (options.err && attempt === 0) options.err(`retrying observation (${cell.id}): ${cause.message}`);
125
+ } finally {
126
+ await journal?.capture(cell.engine, { phase: 'attempt', error: attemptError, cellHash: sha256(cell.id), runIndex, attempt });
127
+ }
94
128
  }
95
- spent += estimate;
96
- attemptCost += estimate;
97
- try {
98
- run = await cell.engine.run({ prompt: cell.prompt.text, runIndex, localeContext: cell.context, maxCostUsd: estimate || maxUsd - spent });
99
- const actual = run.costEstimateUsd ?? estimate;
100
- if (!Number.isFinite(actual) || actual < 0) throw new EngineError('engine returned an invalid cost');
101
- spent += actual - estimate;
102
- attemptCost += actual - estimate;
103
- if (spent > maxUsd) aborted = { reason: 'supplier_cost_exceeded', spent, maxUsd };
104
- } catch (cause) {
105
- run = null;
106
- lastCause = cause;
107
- // Failed or timed-out requests may already be billed. Keep the
108
- // reservation unless the supplier reports a greater charge.
109
- const charged = Number.isFinite(cause?.costEstimateUsd) ? Math.max(estimate, cause.costEstimateUsd) : estimate;
110
- spent += charged - estimate;
111
- attemptCost += charged - estimate;
112
- if (spent > maxUsd) aborted = { reason: 'supplier_cost_exceeded', spent, maxUsd };
113
- if (!(cause instanceof EngineError) || (!cause.retriable && !cause.unknown)) throw cause;
114
- if (!cause.retriable || aborted) break;
115
- if (options.err && attempt === 0) options.err(`retrying observation (${cell.id}): ${cause.message}`);
129
+ if (!run) {
130
+ if (!lastCause && aborted) break outer;
131
+ if (options.err) options.err(`unknown observation (${cell.id}): ${lastCause?.message ?? 'unspecified'}`);
132
+ run = { engine: cell.engine.identity?.engine ?? 'unknown', model: cell.engine.identity?.model ?? 'unknown', answer: '', citations: [], unknown: true,
133
+ provenance: lastCause?.provenance, failure: { code: lastCause?.code ?? 'ENGINE_UNAVAILABLE', message: lastCause?.message ?? 'Engine unavailable' } };
116
134
  }
117
- }
118
- if (!run) {
119
- if (!lastCause && aborted) break outer;
120
- if (options.err) options.err(`unknown observation (${cell.id}): ${lastCause?.message ?? 'unspecified'}`);
121
- run = { engine: cell.engine.identity?.engine ?? 'unknown', model: cell.engine.identity?.model ?? 'unknown', answer: '', citations: [], unknown: true,
122
- provenance: lastCause?.provenance, failure: { code: lastCause?.code ?? 'ENGINE_UNAVAILABLE', message: lastCause?.message ?? 'Engine unavailable' } };
123
- }
124
- run.costEstimateUsd = attemptCost;
135
+ run.costEstimateUsd = attemptCost;
125
136
 
126
- const envelope = {
127
- schema_version: CITATIONS_VERSION,
128
- epoch_id: epochId, cell_id: cell.id, run_index: runIndex, prompt: cell.prompt.text,
129
- engine_identity: cell.identity,
130
- locale_context: cell.context,
131
- provider_model_version: run.providerModelVersion ?? `${run.engine}:${run.model}`,
132
- answer: run.answer, citations: run.citations, fan_out: run.fanOut ?? [],
133
- usage: run.usage ?? { input_tokens: 0, output_tokens: 0 },
134
- cost_estimate_usd: run.costEstimateUsd ?? 0,
135
- at: nowIso(),
136
- ...(run.provenance ? { provenance: run.provenance } : {}),
137
- ...(run.browserContext ? { browser_context: run.browserContext } : {}),
138
- ...(run.failure ? { failure: run.failure } : {}),
139
- };
140
- const digest = sha256(JSON.stringify(envelope));
141
- // The screenshot is a sibling of the envelope: same content-addressed
142
- // directory, named by the envelope digest it belongs to. An observation
143
- // of a rendered surface carries the pixels of that surface.
144
- if (run.screenshotPng) {
145
- const shotName = `${digest}.screenshot.png`;
146
- await writeFile(join(evidenceRoot, shotName), run.screenshotPng);
147
- envelope.screenshot_file = shotName;
148
- }
149
- await writeFile(join(evidenceRoot, `${digest}.json`), JSON.stringify(evidenceEnvelopeSchema.parse(envelope)), 'utf8');
137
+ const envelope = {
138
+ schema_version: CITATIONS_VERSION,
139
+ epoch_id: epochId, cell_id: cell.id, run_index: runIndex, prompt: cell.prompt.text,
140
+ engine_identity: cell.identity,
141
+ locale_context: cell.context,
142
+ provider_model_version: run.providerModelVersion ?? `${run.engine}:${run.model}`,
143
+ answer: run.answer, citations: run.citations, fan_out: run.fanOut ?? [],
144
+ usage: run.usage ?? { input_tokens: 0, output_tokens: 0 },
145
+ cost_estimate_usd: run.costEstimateUsd ?? 0,
146
+ at: nowIso(),
147
+ ...(run.provenance ? { provenance: run.provenance } : {}),
148
+ ...(run.browserContext ? { browser_context: run.browserContext } : {}),
149
+ ...(run.failure ? { failure: run.failure } : {}),
150
+ };
151
+ const digest = sha256(JSON.stringify(envelope));
152
+ // The screenshot is a sibling of the envelope: same content-addressed
153
+ // directory, named by the envelope digest it belongs to. An observation
154
+ // of a rendered surface carries the pixels of that surface.
155
+ if (run.screenshotPng) {
156
+ const shotName = `${digest}.screenshot.png`;
157
+ await writeFile(join(evidenceRoot, shotName), run.screenshotPng);
158
+ envelope.screenshot_file = shotName;
159
+ }
160
+ await writeFile(join(evidenceRoot, `${digest}.json`), JSON.stringify(evidenceEnvelopeSchema.parse(envelope)), 'utf8');
150
161
 
151
- const tiered = run.unknown ? { outcome: 'unknown', citedUrls: [], verify: 'unknown' } : classifyOutcome(cell.target, run, options.verifyFetch ?? null);
152
- const analysis = analyzeRetainedAnswer(cell.target, run);
153
- outcomesByCell.get(cell.id).push(tiered.outcome);
154
- observationRows.push(observationRowSchema.parse({
155
- schema_version: CITATIONS_VERSION, epoch_id: epochId, cell_id: cell.id, run_index: runIndex,
156
- outcome: tiered.outcome, brand_mentioned: analysis.mentioned, target_cited: analysis.cited, cited_urls: tiered.citedUrls, verify: tiered.verify,
157
- evidence_sha256: digest, cost_estimate_usd: run.costEstimateUsd ?? 0, at: nowIso(),
158
- }));
159
- // Duck engines need extra pacing: their anonymous rate limit (~3-4 per exit)
160
- // benefits from a cooldown between observations even with exit rotation.
161
- if (aborted) break outer;
162
- const cellDelay = cell.identity.startsWith('duck') ? Math.max(delayMs, 2000) : delayMs;
163
- if (cellDelay > 0) await new Promise((r) => setTimeout(r, cellDelay));
162
+ const tiered = run.unknown ? { outcome: 'unknown', citedUrls: [], verify: 'unknown' } : classifyOutcome(cell.target, run, options.verifyFetch ?? null);
163
+ const analysis = analyzeRetainedAnswer(cell.target, run);
164
+ outcomesByCell.get(cell.id).push(tiered.outcome);
165
+ observationRows.push(observationRowSchema.parse({
166
+ schema_version: CITATIONS_VERSION, epoch_id: epochId, cell_id: cell.id, run_index: runIndex,
167
+ outcome: tiered.outcome, brand_mentioned: analysis.mentioned, target_cited: analysis.cited, cited_urls: tiered.citedUrls, verify: tiered.verify,
168
+ evidence_sha256: digest, cost_estimate_usd: run.costEstimateUsd ?? 0, at: nowIso(),
169
+ }));
170
+ // Duck engines need extra pacing: their anonymous rate limit (~3-4 per exit)
171
+ // benefits from a cooldown between observations even with exit rotation.
172
+ if (aborted) break outer;
173
+ const cellDelay = cell.identity.startsWith('duck') ? Math.max(delayMs, 2000) : delayMs;
174
+ if (cellDelay > 0) await new Promise((r) => setTimeout(r, cellDelay));
175
+ }
164
176
  }
165
- }
166
177
 
167
- const prior = await readPriorEpochs(dir);
168
- const epochRows = [];
169
- for (const cell of cells) {
170
- const outcomes = outcomesByCell.get(cell.id);
171
- const { k, n, unknowns, mentioned } = tallyOutcomes(outcomes);
172
- const ci = wilsonInterval(k, n);
173
- const baselineRow = prior.filter((r) => r.cell_id === cell.id && r.n >= CITATION_LIMITS.minSamplesForInterpretation).at(-1);
174
- const baseline = baselineRow ? { k: baselineRow.k, n: baselineRow.n } : null;
175
- const priorCount = prior.filter(row => row.cell_id === cell.id).length;
176
- const pairAlpha = priorCount > 0 ? 0.05 / (priorCount * (priorCount + 1)) : 0.05;
177
- const { run_length_posterior, ...changepoint } = changepointPosterior(outcomes);
178
- const statistics = { scope: 'fixed-epoch-supplemental',
179
- confidence_sequence: confidenceSequence(outcomes, { alpha: 0.05, comparisons: cells.length }).at(-1) ?? null,
180
- epoch_comparison: baseline ? compareEpochs({ k, n }, baseline, { alpha: pairAlpha, comparisons: cells.length }) : null,
181
- changepoint,
182
- };
183
- const row = epochRowSchema.parse({
184
- schema_version: CITATIONS_VERSION, epoch_id: epochId, cell_id: cell.id,
185
- k, n, unknowns, mentioned,
186
- rate: smoothedRate(k, n), ci_low: ci?.[0] ?? null, ci_high: ci?.[1] ?? null,
187
- classification: classifyEpoch({ k, n }, baseline, CITATION_LIMITS), statistics,
188
- baseline: baselineRow ? { epoch_id: baselineRow.epoch_id, rate: baselineRow.rate, ci_low: baselineRow.ci_low, ci_high: baselineRow.ci_high } : undefined,
189
- spent_estimate_usd: Number(spent.toFixed(6)), at: nowIso(),
190
- });
191
- epochRows.push(row);
192
- // Incremental flush: every completed cell's row is on disk immediately, so
193
- // a killed epoch loses nothing — the lesson of the lost 3x3 tallies.
194
- await writeFile(join(dir, CITATION_FILES.epochs), JSON.stringify(row) + '\n', { flag: 'a' });
195
- }
178
+ const prior = await readPriorEpochs(dir);
179
+ const epochRows = [];
180
+ for (const cell of cells) {
181
+ const outcomes = outcomesByCell.get(cell.id);
182
+ const { k, n, unknowns, mentioned } = tallyOutcomes(outcomes);
183
+ const ci = wilsonInterval(k, n);
184
+ const baselineRow = prior.filter((r) => r.cell_id === cell.id && r.n >= CITATION_LIMITS.minSamplesForInterpretation).at(-1);
185
+ const baseline = baselineRow ? { k: baselineRow.k, n: baselineRow.n } : null;
186
+ const priorCount = prior.filter(row => row.cell_id === cell.id).length;
187
+ const pairAlpha = priorCount > 0 ? 0.05 / (priorCount * (priorCount + 1)) : 0.05;
188
+ const { run_length_posterior, ...changepoint } = changepointPosterior(outcomes);
189
+ const statistics = { scope: 'fixed-epoch-supplemental',
190
+ confidence_sequence: confidenceSequence(outcomes, { alpha: 0.05, comparisons: cells.length }).at(-1) ?? null,
191
+ epoch_comparison: baseline ? compareEpochs({ k, n }, baseline, { alpha: pairAlpha, comparisons: cells.length }) : null,
192
+ changepoint,
193
+ };
194
+ const row = epochRowSchema.parse({
195
+ schema_version: CITATIONS_VERSION, epoch_id: epochId, cell_id: cell.id,
196
+ k, n, unknowns, mentioned,
197
+ rate: smoothedRate(k, n), ci_low: ci?.[0] ?? null, ci_high: ci?.[1] ?? null,
198
+ classification: classifyEpoch({ k, n }, baseline, CITATION_LIMITS), statistics,
199
+ baseline: baselineRow ? { epoch_id: baselineRow.epoch_id, rate: baselineRow.rate, ci_low: baselineRow.ci_low, ci_high: baselineRow.ci_high } : undefined,
200
+ spent_estimate_usd: Number(spent.toFixed(6)), at: nowIso(),
201
+ });
202
+ epochRows.push(row);
203
+ // Incremental flush: every completed cell's row is on disk immediately, so
204
+ // a killed epoch loses nothing — the lesson of the lost 3x3 tallies.
205
+ await writeFile(join(dir, CITATION_FILES.epochs), JSON.stringify(row) + '\n', { flag: 'a' });
206
+ }
196
207
 
197
- await writeFile(join(dir, CITATION_FILES.observations), observationRows.map((r) => JSON.stringify(r)).join('\n') + (observationRows.length ? '\n' : ''), { flag: 'a' });
208
+ await writeFile(join(dir, CITATION_FILES.observations), observationRows.map((r) => JSON.stringify(r)).join('\n') + (observationRows.length ? '\n' : ''), { flag: 'a' });
198
209
 
199
- // Browser engines hold a browser and maybe an Xvfb display for the whole
200
- // epoch; release both once the tallies are written.
201
- for (const engine of new Set(cells.map((c) => c.engine))) {
202
- try { await engine.close?.(); } catch { /* cleanup is best-effort */ }
210
+ return { epochId, cells: epochRows, spentEstimateUsd: Number(spent.toFixed(6)), aborted, samples, maxUsd };
211
+ } catch (error) {
212
+ primaryError = error;
213
+ throw error;
214
+ } finally {
215
+ let persistenceError, cleanupError;
216
+ for (const engine of uniqueEngines) {
217
+ let error;
218
+ try { await engine.close?.(); } catch (cause) { error = cause; cleanupError ??= cause; }
219
+ try { await journal?.capture(engine, { phase: 'cleanup', error }); }
220
+ catch (cause) { persistenceError ??= cause; }
221
+ }
222
+ try { await journal?.finish({ failed: Boolean(primaryError || persistenceError || cleanupError), cleanupUncertain: Boolean(cleanupError), evidenceComplete: !persistenceError && !String(primaryError?.code ?? '').startsWith('EGRESS_') }); }
223
+ catch (cause) { persistenceError ??= cause; }
224
+ try { await journal?.close(); } catch (cause) { persistenceError ??= cause; }
225
+ if (persistenceError) throw persistenceError;
226
+ if (!primaryError && cleanupError && journal) throw Object.assign(new EngineError('Browser cleanup could not be confirmed; inspect the egress journal.'), { code: 'BROWSER_CLEANUP_UNCONFIRMED' });
203
227
  }
204
-
205
- return { epochId, cells: epochRows, spentEstimateUsd: Number(spent.toFixed(6)), aborted, samples, maxUsd };
206
228
  }
@@ -8,6 +8,7 @@
8
8
  // truth; the hosted view is a projection.
9
9
  import { readFile } from 'node:fs/promises';
10
10
  import { join, resolve } from 'node:path';
11
+ import { verifyLocalEvidenceDigest } from './evidence-integrity.js';
11
12
 
12
13
  export async function syncEpochs({ dir, api, workspaceId, projectId }) {
13
14
  if (!api || !workspaceId || !projectId) {
@@ -83,7 +84,7 @@ export async function syncEvidence({ dir, api, projectId, cliVersion }) {
83
84
  try { text = await readFile(join(root, 'citations-observations.jsonl'), 'utf8'); }
84
85
  catch (error) { if (error.code !== 'ENOENT') throw error; return { synced: 0, total: 0, skipped: 0, errors: 0, reasons: {} }; }
85
86
  const rows = text.split('\n').filter(line => line.trim()).map(line => JSON.parse(line));
86
- const records = []; const reasons = {}; const seen = new Set();
87
+ const records = []; const reasons = {}; const seen = new Map();
87
88
  const skip = reason => { reasons[reason] = (reasons[reason] ?? 0) + 1; };
88
89
  const hash = bytes => createHash('sha256').update(bytes).digest('hex');
89
90
  async function safeFile(path) {
@@ -95,13 +96,19 @@ export async function syncEvidence({ dir, api, projectId, cliVersion }) {
95
96
  for (const row of rows) {
96
97
  if (!/^[A-Za-z0-9_.-]+$/.test(row.epoch_id ?? '') || ['.', '..'].includes(row.epoch_id) || !/^[a-f0-9]{64}$/.test(row.evidence_sha256 ?? '')) throw new Error('Malformed local evidence reference');
97
98
  const path = join(root, 'citations/evidence', row.epoch_id, `${row.evidence_sha256}.json`);
98
- if (seen.has(path)) continue; seen.add(path);
99
+ const identity = JSON.stringify([row.epoch_id, row.cell_id, row.run_index]);
100
+ if (seen.has(path)) {
101
+ if (seen.get(path) !== identity) throw new Error('Conflicting local evidence reference');
102
+ continue;
103
+ }
104
+ seen.set(path, identity);
99
105
  let file;
100
106
  try { file = await safeFile(path); } catch (error) { if (error.code === 'ENOENT') { skip('missing_envelope'); continue; } throw error; }
101
107
  if ((await lstat(file)).size > limits.envelopeBytes) throw new Error('Evidence envelope exceeds the upload limit');
102
108
  const envelopeJson = await readFile(file, 'utf8');
103
109
  const envelope = evidenceEnvelopeSchema.parse(JSON.parse(envelopeJson));
104
110
  if (envelope.epoch_id !== row.epoch_id || envelope.cell_id !== row.cell_id || envelope.run_index !== row.run_index) throw new Error('Evidence identity does not match its observation');
111
+ if (!verifyLocalEvidenceDigest({ envelope, text: envelopeJson, digest: row.evidence_sha256 })) throw new Error('Evidence digest does not match its retained observation');
105
112
  if (envelope.engine_identity.startsWith('google-aio:') && !envelope.provenance) { skip('legacy_aio_without_provenance'); continue; }
106
113
  const record = { envelopeJson, sha256: hash(envelopeJson), cliVersion, localEvidenceSha256: row.evidence_sha256 };
107
114
  if (envelope.screenshot_file) {