@tangle-network/agent-eval 0.140.1 → 0.141.0

package/docs/wire-protocol.md

@@ -96,14 +96,71 @@ GET /v1/version
 ```json
 {
   "package": "@tangle-network/agent-eval",
-  "version": "0.20.10",
+  "version": "0.140.1",
   "wireVersion": "1.0.0",
-  "apiSurface": ["judge", "listRubrics", "version"]
+  "apiSurface": ["judge", "listRubrics", "version", "feedback.ingest", "traces.ingest"]
 }
 ```
 
 `version` matches the package version. `wireVersion` bumps independently: only on breaking request/response schema changes. Package versions can differ across releases as long as `wireVersion` matches.
 
+### `traces.ingest`: batch-ingest production trace events
+
+```http
+POST /v1/traces/ingest
+{
+  "events": [
+    {
+      "eventId": "evt_01h...",
+      "runId": "run_01h...",
+      "spanId": "span_01h...",
+      "kind": "log",
+      "timestamp": 1732556400000,
+      "payload": { "message": "..." }
+    }
+  ]
+}
+```
+
+```json
+{
+  "accepted": 1,
+  "rejected": 0,
+  "errors": []
+}
+```
+
+Also accepts `content-type: application/x-ndjson` — one `TraceEvent` object per line instead of a wrapping `{"events": [...]}` body. Max 10,000 events per call; chunk bigger streams.
+
+Best-effort: each event is appended independently, so one bad event doesn't poison the batch — rejects land in `errors[]`, keyed by `eventId`. The underlying store is append-only: retries of the same payload produce duplicate events, so consumers should de-duplicate by `eventId` downstream (production traces frequently land via at-least-once buses — Kafka, SQS — where dedup is unavoidable). Requires a `traceStore` passed to `createApp`; without one this returns `503 service_unavailable`.
+
+`kind` is one of `log`, `error`, `budget_decrement`, `budget_breach`, `state_mutation`, `policy_violation`, `redaction_applied`, `custom`, matching the `TraceSchema` v1 `EventKind` enum. `payload` is free-form — the runtime owns its shape.
+
+### `feedback.ingest`: ingest a review/feedback trajectory
+
+```http
+POST /v1/feedback
+{
+  "id": "traj_01h...",
+  "task": { "intent": "Summarize the incident report." },
+  "labels": [
+    { "source": "user", "kind": "approve", "value": true, "createdAt": "2026-08-01T00:00:00Z" }
+  ],
+  "createdAt": "2026-08-01T00:00:00Z"
+}
+```
+
+```json
+{
+  "id": "traj_01h...",
+  "persisted": true
+}
+```
+
+Idempotent on `id`: re-posting the same id replaces the prior record. Requires a `feedbackStore` passed to `createApp`; without one this returns `503 service_unavailable`.
+
+`task.intent` and `createdAt` are required; `attempts` and `labels` default to `[]`; `outcome`, `split` (`train` / `dev` / `test` / `holdout`), and `tags` are optional. See `FeedbackTrajectorySchema` in `src/wire/schemas.ts` for the full shape, including per-attempt artifacts, proposed actions, and per-label severity.
+
 ### `GET /healthz`: liveness
 
 For probing whether a server is up. Returns `{ "status": "ok", "uptimeSec": <number> }`.
@@ -132,6 +189,7 @@ Every error response uses the same shape:
 | 404 | `rubric_not_found` | Unknown `rubricName`. |
 | 500 | `judge_error` | LLM returned malformed output. |
 | 500 | `internal_error` | Unexpected server error. |
+| 503 | `service_unavailable` | `traces.ingest` or `feedback.ingest` called without the matching store passed to `createApp`. |
 
 stdio RPC uses the same shape inside an envelope: `{"error": {...}}` instead of `{"result": {...}}`. Exit code is non-zero on error.
 
@@ -154,6 +212,16 @@ curl http://localhost:5005/v1/rubrics | jq
 curl -X POST http://localhost:5005/v1/judge \
   -H 'content-type: application/json' \
   -d '{"rubricName":"anti-slop","content":"We just shipped …"}'
+
+# traces ingest
+curl -X POST http://localhost:5005/v1/traces/ingest \
+  -H 'content-type: application/json' \
+  -d '{"events":[{"eventId":"evt_1","runId":"run_1","kind":"log","timestamp":1732556400000,"payload":{}}]}'
+
+# feedback ingest
+curl -X POST http://localhost:5005/v1/feedback \
+  -H 'content-type: application/json' \
+  -d '{"id":"traj_1","task":{"intent":"Summarize the incident report."},"createdAt":"2026-08-01T00:00:00Z"}'
 ```
 
 ## Using stdio RPC

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@tangle-network/agent-eval",
-  "version": "0.140.1",
+  "version": "0.141.0",
   "description": "Evaluate and improve AI agents from runs, traces, judges, and feedback. Compare candidates, cluster failures, measure lift, and gate releases.",
   "homepage": "https://github.com/tangle-network/agent-eval#readme",
   "repository": {
@@ -168,8 +168,9 @@
   "dependencies": {
     "@asteasolutions/zod-to-openapi": "^9.1.0",
     "@hono/node-server": "^2.0.12",
-    "@tangle-network/agent-core": "0.4.29",
-    "@tangle-network/agent-interface": "0.40.0",
+    "@tangle-network/agent-core": "0.4.30",
+    "@tangle-network/agent-interface": "0.41.0",
+    "@tangle-network/agent-trace-contract": "^1.0.2",
     "hono": "^4.12.32",
     "linear-sum-assignment": "1.0.9",
     "re2js": "2.8.6",
@@ -190,6 +191,7 @@
     "oxc-parser": "^0.141.0",
     "publint": "^0.3.22",
     "tsdown": "^0.22.14",
+    "tsx": "^4.23.1",
     "typescript": "^7.0.2",
     "vitest": "^4.1.10",
     "yaml": "2.9.0"