@memnexus-ai/mx-agent-cli 0.1.226 → 0.1.227

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -5,6 +5,7 @@ paths:
5
5
  - "mcp-server/agent-tests/**"
6
6
  - "core-api/scripts/*eval*"
7
7
  - "core-api/scripts/*benchmark*"
8
+ - "core-api/scripts/longmemeval-*"
8
9
  ---
9
10
 
10
11
  # Eval Harness Rules
@@ -49,3 +50,23 @@ Eval work is decision infrastructure. Treat harness code, fixtures, scoring, ana
49
50
  - SPIKE findings (any error-class rate >5% in one arm-round-family) are informational WARN, NOT auto-halt: every valid mini-tier run trips this via budget-exhaustion no_patch (baseline arms show 9-22%), so a spike is investigate-but-don't-halt and does not change the exit code. A tripwire that fired on every valid run would be ignored. Advisory WARNs also cover a patch_empty count exceeding the no_patch marker (runner write-path / marker drift) and unparsed result dirs excluded from comparison.
50
51
  - Exit-code contract: 0 = clean or spike/advisory-WARN only; 1 = differential (halt); 2 = usage/environment error (missing root, bad args, OR an existing root with 0 per_task.jsonl files — absence of data is never a clean verdict). Torn/partial trailing lines on a live file are skipped, counted, and reported ("N unparsable lines skipped") — informational only; a crash or read race must never masquerade as a halt-worthy alert.
51
52
  - The launch monitor must include the health check on its poll cadence, not just process-liveness.
53
+
54
+ ## Store Identity Assertion (PO-endorsed, issue #4427, 2026-07-23)
55
+
56
+ No measurement harness queries a store it has not proven is the intended store. Before any scored or data-producing call against an account:
57
+
58
+ - Fetch a known sentinel from that account (a memory ID from the user map's `memoryIds` list, or a pre-seeded marker) and hard-fail if the response is empty, returns a different ID, or returns content that does not match the expected hash.
59
+ - The check must use the same auth path (headers, credentials, port-forward) as the scored call. A check that authenticates differently than the run proves nothing.
60
+ - Hard-fail means abort the entire run for that account, not skip-and-continue. Wrong-universe data is indistinguishable from correct data once recorded (incident: F14 rank-tier replay, 2026-07-23 — stale embedded API keys silently queried wrong accounts for 24 qids, producing a retracted 72.7% figure; two-team two-hour harness diff; coordination msgs 1784811171332-0 → 1784817487358-0).
61
+ - The sentinel check is not optional for dry runs, replays, or cross-team handoffs. If the harness touches a store, it proves identity first.
62
+
63
+ ## Capture Fingerprints (issue #4427, 2026-07-23)
64
+
65
+ Any capture that produces a shareable number (accuracy, recall, delta, confusion matrix) must embed provenance metadata alongside the result:
66
+
67
+ - **Auth principal**: which credential or user identity was used.
68
+ - **Account ID**: the account or user ID queried.
69
+ - **Store size**: memory count or claim count at query time.
70
+ - **Content hash**: hash of the first result page (first N search results) for at least one representative query.
71
+
72
+ The fingerprint goes in the result artifact (JSONL metadata row, summary header, or sidecar file). The first line in any cross-team data disagreement becomes "same data, yes/no?" — the fingerprint answers that without a re-run.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@memnexus-ai/mx-agent-cli",
3
- "version": "0.1.226",
3
+ "version": "0.1.227",
4
4
  "description": "CLI for creating and managing AI agent teams",
5
5
  "type": "module",
6
6
  "bin": {