@iann29/rastro 0.1.0-alpha.3 → 0.1.0-alpha.5

package/docs/benchmarks/2026-08-21-formal-certification.md

@@ -0,0 +1,333 @@
+# Formal ingest certification, 2026-08-21
+
+This report preserves four isolated ten-minute benchmark runs against the
+non-production Synapse canary. It is engineering evidence for the tested commit
+and deployment, not a production SLA.
+
+## Environment
+
+| Item | Value |
+| --- | --- |
+| Git commit | `e6ae92658c86c1931e697fa2a38c80fd902f7afa` |
+| Package version | `0.1.0-alpha.3` |
+| Deployment | DEV `fast-finch-7454` |
+| Benchmark site | `kx75pnr9wap522n0zmh4nnb4jh8cv110` |
+| Driver | One local Node.js process |
+| Reconciliation | Exact run-scoped session IDs, indexed pages of 100 |
+| Health before/after | `ok: true` |
+| GeoIP provider | Disabled |
+
+All runs used `--duration 600 --certify`. Runs were sequential so the profiles
+did not compete with each other. A harness certification means the run lasted
+the required duration, had clean request accounting, and reconciled every
+expected stored event. Meeting the separate 100 million stored-events/day target
+requires at least 1,157.41 stored events/second.
+
+## Results
+
+| Profile | Concurrency | Batch | Accepted | Stored | Stored/s | Certified stored/day | p95 | Exact reconciliation | 100M/day target |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | --- | --- |
+| Lean | 100 | 50 | 952,600 | 952,600 | 1,582 | 136,654,617 | 4,543 ms | 952,600 / 952,600 | Pass |
+| Realistic | 75 | 50 | 885,700 | 842,080 | 1,398 | 120,808,947 | 4,247 ms | 842,080 / 842,080 | Pass |
+| Heavy | 100 | 20 | 353,980 | 353,980 | 588 | 50,769,480 | 5,787 ms | 353,980 / 353,980 | **Below target** |
+| Feature | 100 | 15 | 232,170 | 232,170 | 384 | 33,218,356 | 6,226 ms | 232,170 / 232,170 | **Below target** |
+
+The realistic run accepted 43,620 heartbeats, which intentionally update live
+presence without increasing persistent event counts. Its accepted-versus-stored
+difference is therefore exact. The heavy request measured 62,054 bytes and
+sustained 1,828,443 request bytes/second below the 64 KiB HTTP body limit.
+
+## Heavy recertification after byte-accounting optimization
+
+Commit `4460ad34314a6e15c24a0c78aab6e6ce85db5c60` reused each event's
+exact UTF-8 byte measurement for both batch validation and rate limits, removing
+one full-payload serialization from the ingestion transaction. A new isolated
+ten-minute run on 2026-08-22 used the same deployment, site, concurrency, batch
+size, and heavy payload.
+
+| Result | Original | Recertified | Change |
+| --- | ---: | ---: | ---: |
+| Stored events/s | 588 | 887 | +50.9% |
+| Certified stored/day | 50,769,480 | 76,623,702 | +50.9% |
+| Request bytes/s | 1,828,443 | 2,759,875 | +50.9% |
+| p95 latency | 5,787 ms | 3,670 ms | -36.6% |
+
+The recertification remained below the separate 100 million events/day target,
+but raised the measured heavy-payload ceiling without changing payload shape or
+weakening validation, rate limits, or reconciliation.
+
+```json
+{
+  "label": "heavy-2026-08-22T04:03:22.517Z",
+  "runId": "f3943151-bc71-4806-b609-69faf8bdfb0c",
+  "profile": "heavy",
+  "targetDurationSeconds": 600,
+  "elapsedMs": 601726,
+  "terminationReason": "duration",
+  "durationCompleted": true,
+  "certification": { "requested": true, "passed": true, "minimumDurationSeconds": 600 },
+  "concurrency": 100,
+  "batchSize": 20,
+  "sessionPool": 2048,
+  "maximumRequests": null,
+  "requests": { "started": 26682, "completed": 26682, "failures": 0 },
+  "events": {
+    "sent": 533640,
+    "heartbeats": 0,
+    "accepted": 533640,
+    "stored": 533640,
+    "duplicates": 0,
+    "rejected": 0,
+    "failed": 0
+  },
+  "errors": {},
+  "throughput": {
+    "eventsPerSecond": 887,
+    "storedEventsPerSecond": 887,
+    "estimatedEventsPerDay": 76623702,
+    "estimatedStoredEventsPerDay": 76623702,
+    "certifiedEventsPerDay": 76623702,
+    "certifiedStoredEventsPerDay": 76623702,
+    "requestBytesPerSecond": 2759875
+  },
+  "latencyMs": { "p50": 2035, "p95": 3670, "p99": 4755, "max": 8139 },
+  "reconciliation": {
+    "scope": "run-session-ids",
+    "reliable": true,
+    "expectedSessions": 2048,
+    "requestedSessions": 2048,
+    "foundSessions": 2048,
+    "acceptedEvents": 533640,
+    "acceptedHeartbeats": 0,
+    "expectedStoredEvents": 533640,
+    "storedEvents": 533640,
+    "matches": true
+  }
+}
+```
+
+## Run artifacts
+
+### Lean
+
+```json
+{
+  "label": "lean-2026-08-21T13:31:27.434Z",
+  "runId": "41788de1-964e-4315-8b00-f295e202f2e0",
+  "profile": "lean",
+  "targetDurationSeconds": 600,
+  "elapsedMs": 602282,
+  "terminationReason": "duration",
+  "durationCompleted": true,
+  "certification": { "requested": true, "passed": true, "minimumDurationSeconds": 600 },
+  "concurrency": 100,
+  "batchSize": 50,
+  "sessionPool": "unique-per-request",
+  "maximumRequests": null,
+  "requests": { "started": 19052, "completed": 19052, "failures": 0 },
+  "events": {
+    "sent": 952600,
+    "heartbeats": 0,
+    "accepted": 952600,
+    "stored": 952600,
+    "duplicates": 0,
+    "rejected": 0,
+    "failed": 0
+  },
+  "errors": {},
+  "throughput": {
+    "eventsPerSecond": 1582,
+    "storedEventsPerSecond": 1582,
+    "estimatedEventsPerDay": 136654617,
+    "estimatedStoredEventsPerDay": 136654617,
+    "certifiedEventsPerDay": 136654617,
+    "certifiedStoredEventsPerDay": 136654617,
+    "requestBytesPerSecond": 438830
+  },
+  "latencyMs": { "p50": 2979, "p95": 4543, "p99": 5217, "max": 5628 },
+  "reconciliation": {
+    "scope": "run-session-ids",
+    "reliable": true,
+    "expectedSessions": 19052,
+    "requestedSessions": 19052,
+    "foundSessions": 19052,
+    "acceptedEvents": 952600,
+    "acceptedHeartbeats": 0,
+    "expectedStoredEvents": 952600,
+    "storedEvents": 952600,
+    "matches": true
+  }
+}
+```
+
+### Realistic
+
+```json
+{
+  "label": "realistic-2026-08-21T13:42:07.118Z",
+  "runId": "8a883d9a-537a-4ce5-b300-be77995ba5da",
+  "profile": "realistic",
+  "targetDurationSeconds": 600,
+  "elapsedMs": 602238,
+  "terminationReason": "duration",
+  "durationCompleted": true,
+  "certification": { "requested": true, "passed": true, "minimumDurationSeconds": 600 },
+  "concurrency": 75,
+  "batchSize": 50,
+  "sessionPool": 2048,
+  "maximumRequests": null,
+  "requests": { "started": 17714, "completed": 17714, "failures": 0 },
+  "events": {
+    "sent": 885700,
+    "heartbeats": 43620,
+    "accepted": 885700,
+    "stored": 842080,
+    "duplicates": 0,
+    "rejected": 0,
+    "failed": 0
+  },
+  "errors": {},
+  "throughput": {
+    "eventsPerSecond": 1471,
+    "storedEventsPerSecond": 1398,
+    "estimatedEventsPerDay": 127066887,
+    "estimatedStoredEventsPerDay": 120808947,
+    "certifiedEventsPerDay": 127066887,
+    "certifiedStoredEventsPerDay": 120808947,
+    "requestBytesPerSecond": 445267
+  },
+  "latencyMs": { "p50": 2257, "p95": 4247, "p99": 5122, "max": 6133 },
+  "reconciliation": {
+    "scope": "run-session-ids",
+    "reliable": true,
+    "expectedSessions": 2048,
+    "requestedSessions": 2048,
+    "foundSessions": 2048,
+    "acceptedEvents": 885700,
+    "acceptedHeartbeats": 43620,
+    "expectedStoredEvents": 842080,
+    "storedEvents": 842080,
+    "matches": true
+  }
+}
+```
+
+### Heavy
+
+```json
+{
+  "label": "heavy-2026-08-21T13:52:43.382Z",
+  "runId": "8cd7221e-2e33-4e94-9f9d-13bb4a84c60e",
+  "profile": "heavy",
+  "targetDurationSeconds": 600,
+  "elapsedMs": 602407,
+  "terminationReason": "duration",
+  "durationCompleted": true,
+  "certification": { "requested": true, "passed": true, "minimumDurationSeconds": 600 },
+  "concurrency": 100,
+  "batchSize": 20,
+  "sessionPool": 2048,
+  "maximumRequests": null,
+  "requests": { "started": 17699, "completed": 17699, "failures": 0 },
+  "events": {
+    "sent": 353980,
+    "heartbeats": 0,
+    "accepted": 353980,
+    "stored": 353980,
+    "duplicates": 0,
+    "rejected": 0,
+    "failed": 0
+  },
+  "errors": {},
+  "throughput": {
+    "eventsPerSecond": 588,
+    "storedEventsPerSecond": 588,
+    "estimatedEventsPerDay": 50769480,
+    "estimatedStoredEventsPerDay": 50769480,
+    "certifiedEventsPerDay": 50769480,
+    "certifiedStoredEventsPerDay": 50769480,
+    "requestBytesPerSecond": 1828443
+  },
+  "latencyMs": { "p50": 2968, "p95": 5787, "p99": 6942, "max": 7839 },
+  "reconciliation": {
+    "scope": "run-session-ids",
+    "reliable": true,
+    "expectedSessions": 2048,
+    "requestedSessions": 2048,
+    "foundSessions": 2048,
+    "acceptedEvents": 353980,
+    "acceptedHeartbeats": 0,
+    "expectedStoredEvents": 353980,
+    "storedEvents": 353980,
+    "matches": true
+  }
+}
+```
+
+### Feature
+
+```json
+{
+  "label": "feature-2026-08-21T14:03:20.654Z",
+  "runId": "8e60bc5f-6fff-4791-b02c-0e7340767b55",
+  "profile": "feature",
+  "targetDurationSeconds": 600,
+  "elapsedMs": 603868,
+  "terminationReason": "duration",
+  "durationCompleted": true,
+  "certification": { "requested": true, "passed": true, "minimumDurationSeconds": 600 },
+  "concurrency": 100,
+  "batchSize": 15,
+  "sessionPool": 2048,
+  "maximumRequests": null,
+  "requests": { "started": 15478, "completed": 15478, "failures": 0 },
+  "events": {
+    "sent": 232170,
+    "heartbeats": 0,
+    "accepted": 232170,
+    "stored": 232170,
+    "duplicates": 0,
+    "rejected": 0,
+    "failed": 0
+  },
+  "errors": {},
+  "throughput": {
+    "eventsPerSecond": 384,
+    "storedEventsPerSecond": 384,
+    "estimatedEventsPerDay": 33218356,
+    "estimatedStoredEventsPerDay": 33218356,
+    "certifiedEventsPerDay": 33218356,
+    "certifiedStoredEventsPerDay": 33218356,
+    "requestBytesPerSecond": 119846
+  },
+  "latencyMs": { "p50": 3523, "p95": 6226, "p99": 7722, "max": 8524 },
+  "reconciliation": {
+    "scope": "run-session-ids",
+    "reliable": true,
+    "expectedSessions": 2048,
+    "requestedSessions": 2048,
+    "foundSessions": 2048,
+    "acceptedEvents": 232170,
+    "acceptedHeartbeats": 0,
+    "expectedStoredEvents": 232170,
+    "storedEvents": 232170,
+    "matches": true
+  }
+}
+```
+
+## Conclusions
+
+- Lean and realistic traffic exceeded 100 million stored events/day for a full
+  ten minutes with exact reconciliation and no failures.
+- Heavy and feature traffic completed cleanly but did not meet the 100 million
+  stored-events/day target at the tested payloads and concurrency. The latest
+  heavy ceiling is 76.62 million events/day after recertification.
+- The updated heavy result demonstrates a different ceiling: 2.76 MB/s of
+  near-limit requests. It must not be generalized to minimal or realistic
+  payloads.
+- Trusted financial conversions were not exercised. They require a separate
+  server-side driver because browser traffic cannot cross the financial trust
+  boundary.
+- GeoIP provider behavior was not benchmarked because provider enrichment was
+  disabled on this deployment.