src-auth-perms-sync 0.3.1__tar.gz → 0.4.0__tar.gz

{src_auth_perms_sync-0.3.1 → src_auth_perms_sync-0.4.0}/.gitignore

@@ -12,6 +12,7 @@ __pycache__
 *.py[cod]
 *.py[oc]
 *.swp
+*.tsv
 *.yaml
 build/
 dist/
@@ -24,3 +25,5 @@ wheels/
 !.env.example
 !.markdownlint-cli2.yaml
 !maps-example.yaml
+!tests/tests.yaml
+!tests/e2e/fixtures/**/maps.yaml

{src_auth_perms_sync-0.3.1 → src_auth_perms_sync-0.4.0}/AGENTS.md

@@ -1,5 +1,10 @@
 # AGENTS.md
 
+## Reference materials
+
+- GraphQL schema and database migrations (changes to SQL schema) are available in
+  <https://github.com/sourcegraph/artifacts>
+
 ## Linting
 
 ```bash
@@ -26,21 +31,40 @@ uv run src-auth-perms-sync --help
 
 ## Testing
 
-- First run a dry-run (default behaviour, without `--apply` flag) against a Sourcegraph instance
+All testing runs through one entrypoint: `tests/run.py`. Output goes to the
+console and to a per-run log file under `logs/`. Each level runs only its
+own checks.
 
 ```bash
-uv run src-auth-perms-sync [--get]
-uv run src-auth-perms-sync --set maps.yaml --full
-uv run src-auth-perms-sync --restore backups/<source>/<run>/before.json
+# Fast, no network (also what the pre-commit hook runs):
+# lint, format, pyright, unit + fixture tests, CLI rejection matrix,
+# randomized permission invariants
+uv run tests/run.py
+
+# End-to-end runs against the .env test instance with independent GraphQL
+# read-back verification, and a wheel install smoke test
+uv run tests/run.py --live
+
+# Run a subset: comma-delimited test names, substring match
+uv run tests/run.py --live full-overwrite-unions
+uv run tests/run.py --live wheel,baseline
+
+# Repeated timed runs with Jaeger trace retention, RSS sampling,
+# optional kubectl load monitoring, and baseline comparison
+uv run tests/run.py --performance --repeat 3
+uv run tests/run.py --performance --baseline-command "uvx src-auth-perms-sync@latest" \
+  --fail-on-memory-regression-percent 10
+
+# Regenerate fixture goldens after editing tests/e2e/fixtures/ cases
+uv run tests/run.py --update-golden
 ```
 
-- Read the output, and evaluate the expected changes
-- If the expected changes look correct
-  - Run with the `--apply` flag against the test instance
-  - Read and evaluate the output for expected changes
-  - Run with the `--restore` flag against the test instance
-  - Always inspect the before / after snapshots in
-    `src-auth-perms-sync-runs/<endpoint>/backups/` afterward to confirm the diff matches what you expected
+- Fixture cases live in `tests/e2e/fixtures/<case>/` — see the README there
+  for the format. Add cases there to cover new mapping behaviors.
+- For manual verification against a real instance, dry-run first (no
+  `--apply`), read the planned changes, then `--apply` on a scratch instance
+  and inspect the before/after snapshots under
+  `src-auth-perms-sync-runs/<endpoint>/runs/`.
 
 ## Release process
 

{src_auth_perms_sync-0.3.1 → src_auth_perms_sync-0.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: src-auth-perms-sync
-Version: 0.3.1
+Version: 0.4.0
 Summary: Set Sourcegraph permissions from authentication provider data
 Project-URL: Homepage, https://github.com/sourcegraph/src-auth-perms-sync
 Project-URL: Issues, https://github.com/sourcegraph/src-auth-perms-sync/issues

{src_auth_perms_sync-0.3.1 → src_auth_perms_sync-0.4.0}/dev/TODO.md

@@ -1,5 +1,22 @@
 # TODO
 
+## Medium priority: extend SAML-group live coverage to org sync
+
+tests/setup.py now fabricates SAML accounts with synthetic groups
+(`perms-sync-test-eng` / `perms-sync-test-sales`, see tests/setup.yaml).
+saml-group-live covers permission mapping; add a seeded
+`sync-saml-orgs --apply` live case that maps those groups to a throwaway
+org and asserts membership is added AND removed (today's
+sync-saml-orgs-apply only covers the single real Okta user, add-only).
+
+## Decide: pendingBindIDs / usersWithPendingPermissions
+
+The CLI cannot create pending permissions (it validates users exist), but
+snapshots record `pending_bindIDs`, and setup.py / the live hygiene check
+report (never delete) any that appear. Decide whether "grant before first
+login" is a customer need; if not, consider dropping the snapshot field.
+See the thread discussion 2026-06-11.
+
 ## High priority: Remote trigger on demand
 
 - Sourcegraph webhook for new user coming in v7.4.0
@@ -13,14 +30,6 @@
 - How do we avoid stampedes (e.g., bulk repo sync triggering thousands
   of re-runs)?
 
-## High priority: End to End test cases
-
-- Create test cases. Each test case should contain:
-  - Before state
-  - maps.yaml file
-  - Expected after state
-- Script to run the script, and verify the after state matches the expected after state
-
 ## High priority: Verify perms are updated when a user's SAML groups change
 
 - If a user gets added to a new SAML group, which hits a mapping, ensure they
@@ -29,14 +38,24 @@
 ## High priority: Reduce worst-case full-permission sync load
 
 - Use the stress-run evidence in
-  [memory-efficiency.md](./memory-efficiency.md)
+  [engineering-requests.md](./engineering-requests.md)
   to request Sourcegraph bulk explicit-permission read and write APIs.
+  New evidence 2026-06-10: the whole-instance apply (1,150 repo
+  overwrites x 10,002 bindIDs each at parallelism 16) crashed the test
+  instance's Postgres ("connection refused", "unexpected EOF"); the
+  client circuit breaker opened and the harness restored cleanly. That
+  stress cycle is now opt-in: `uv run tests/run.py --live "full cycle"`.
 - Add an explicit destructive/performance-test mode to the e2e runner so giant
   stress runs can skip or defer full restore cleanup when the goal is finding
   the server-side breaking point.
 - Revisit full snapshot capture once Sourcegraph exposes a bulk read path;
   replace aliased `User.permissionsInfo.repositories(source: API)` calls before
   raising concurrency further.
+- `get --repos <name>` still scans every user's explicit grants to find one
+  repo's holders (~400 s at 10k users). A repo-centric read
+  (`repository.permissionsInfo.users` + site-admin disambiguation, as the
+  test harness already does) would make it seconds — see the repo-centric
+  section below.
 
 ## Low priority: Repo-centric path, when users > repos, or for cross-checking
 

src_auth_perms_sync-0.3.1/dev/memory-efficiency.md → src_auth_perms_sync-0.4.0/dev/engineering-requests.md

@@ -1,173 +1,18 @@
-# Memory efficiency testing
-
-Use this when full snapshot capture or full-set apply is slow. The goal is to
-correlate `src-auth-perms-sync` structured logs with Sourcegraph Jaeger spans
-and pod/Postgres load, then use the evidence to ask Sourcegraph engineering for
-bulk explicit-permissions APIs.
-
-## Capture a focused trace
-
-Run a small command with `--fetch-sg-traces`. This sends
-`X-Sourcegraph-Request-Trace` and a W3C `traceparent` header on each GraphQL
-request. Sourcegraph may also return `x-trace`, `x-trace-span`, and
-`x-trace-url` response headers.
-
-```bash
-uv run src-auth-perms-sync get \
-  --fetch-sg-traces \
-  --sample-interval 0 \
-  --parallelism 2 \
-  --explicit-permissions-batch-size 25
-```
-
-Find the slow GraphQL HTTP requests in the run log:
-
-```bash
-LOG=src-auth-perms-sync-runs/<endpoint>/runs/<run>/log.json
-
-jq -r '
-  select(.event == "http_request" and .phase == "end") |
-  select(.url | endswith("/.api/graphql")) |
-  [
-    .duration_ms,
-    (.response_headers["x-trace"] // ""),
-    (.response_headers["x-trace-url"] // ""),
-    (.request_headers.traceparent // "")
-  ] | @tsv
-' "$LOG" | sort -nr | head -20
-```
-
-Prefer the `x-trace` value when present. If Sourcegraph did not return one,
-extract the trace ID from `traceparent`:
-
-```bash
-TRACEPARENT=00-<trace-id>-<span-id>-01
-TRACE_ID="$(printf '%s' "$TRACEPARENT" | cut -d- -f2)"
-```
-
-Fetch the trace JSON from Jaeger:
-
-```bash
-curl -sS \
-  -H "Authorization: token $SRC_ACCESS_TOKEN" \
-  "$SRC_ENDPOINT/-/debug/jaeger/api/traces/$TRACE_ID" \
-  > /tmp/sourcegraph-trace.json
-```
-
-Jaeger ingestion can lag. If the API returns `trace not found`, wait briefly
-and retry. For long runs, fetch traces as soon as the relevant command
-finishes; older trace IDs can disappear before a full matrix ends.
-
-Summarize the hottest spans:
-
-```bash
-uv run python - <<'PY'
-import collections
-import json
-
-trace = json.load(open("/tmp/sourcegraph-trace.json"))["data"][0]
-durations_by_operation = collections.defaultdict(list)
-for span in trace["spans"]:
-    durations_by_operation[span["operationName"]].append(span["duration"] / 1000)
-
-for operation, durations in sorted(
-    durations_by_operation.items(),
-    key=lambda item: sum(item[1]),
-    reverse=True,
-)[:15]:
-    print(
-        f"{operation}: count={len(durations)} "
-        f"sum_ms={sum(durations):.1f} "
-        f"max_ms={max(durations):.1f}"
-    )
-PY
-```
-
-Do not commit tokens, customer URLs, raw trace JSON, benchmark CSVs, or monitor
-artifacts. Keep them in `/tmp` unless a human asks to preserve them.
-
-## Trace the end-to-end matrix
-
-Prefer the end-to-end runner as the single orchestrator. With
-`--fetch-sg-traces`, it passes Sourcegraph debug trace collection to every child
-CLI command, tails child JSON logs, and fetches Jaeger traces in the background
-while each child command is still running.
-
-```bash
-uv run python dev/test-end-to-end.py \
-  --fetch-sg-traces \
-  --sample-interval 0 \
-  --external-sample-interval 0 \
-  --results-json /tmp/src-auth-perms-sync-end-to-end-trace.json \
-  --results-csv /tmp/src-auth-perms-sync-end-to-end-trace.csv
-```
-
-Useful trace options:
-
-- `--jaeger-trace-limit N`: fetch only the `N` slowest GraphQL traces per case.
-- `--jaeger-trace-limit 0`: send trace headers but skip Jaeger fetching.
-- `--jaeger-trace-parallelism N`: tune concurrent Jaeger fetches.
-- `--jaeger-trace-jsonl PATH`: stream compact trace summaries as JSON Lines.
-- `--jaeger-trace-dir PATH`: store complete raw Jaeger payloads.
-
-Raw trace files include:
-
-- `trace_request`: CLI-side HTTP and `graphql_query` correlation metadata,
-  including query name, page number, page size, cursor presence, query byte
-  count, variable names, response fields, status, and timing.
-- `jaeger_summary`: compact hot-operation and GraphQL-operation summary.
-- `jaeger_trace`: the complete Jaeger trace JSON returned by Sourcegraph.
+# Engineering requests
 
-All runner flags are Config-backed. You can set them in the shell or `.env`
-with `SRC_AUTH_PERMS_SYNC_E2E_*` names, plus `SRC_ENDPOINT`,
-`SRC_ACCESS_TOKEN`, and `SRC_AUTH_PERMS_SYNC_TEST_USER`.
+Use this when opening Sourcegraph Engineering issues from memory-efficiency
+evidence. Capture steps stay in [memory-efficiency.md](./memory-efficiency.md);
+this file keeps the request-ready problem statement, evidence, proposed API
+shape, and copy/paste issue text.
 
-For each tested batch size and parallelism, record:
+## Requested Sourcegraph changes
 
-- CLI `capture_explicit_grants` duration from the structured log
-- slowest GraphQL `http_request` duration and its trace metadata
-- Jaeger counts and summed duration for `GraphQL Request`, `repos.Get`,
-  `sql.conn.query`, and `database.PermsStore.LoadUserPermissions`
-- run-end `http_retry_count`, `http_request_attempt_count`, and timeout/error
-  counts
-
-## Monitor Sourcegraph load during e2e runs
-
-The runner can start the Sourcegraph pod/Postgres monitor and write monitor
-artifact paths into the result JSON:
-
-```bash
-uv run python dev/test-end-to-end.py \
-  --fetch-sg-traces \
-  --monitor-sourcegraph-load \
-  --sample-interval 0 \
-  --external-sample-interval 0 \
-  --results-json /tmp/src-auth-perms-sync-end-to-end-trace.json \
-  --results-csv /tmp/src-auth-perms-sync-end-to-end-trace.csv
-```
-
-By default, monitor output is written beside `--results-json` or
-`--results-csv` as `*-sourcegraph-load`, and the monitor's stdout/stderr goes
-to `*-sourcegraph-load.log`. Override the location with
-`--monitor-output-dir PATH`. Tune Kubernetes targets and sample intervals with
-the `--monitor-*` flags if the test namespace or pod names differ.
-
-The lower-level helper remains available for focused profiling outside a full
-e2e run:
-
-```bash
-dev/memory-efficiency-monitor-sourcegraph.sh \
-  --namespace m \
-  --output-dir /tmp/src-auth-perms-sync-sourcegraph-load-$(date -u +%Y%m%d-%H%M%S)
-```
-
-Stop the helper with Ctrl-C, or add `--duration-seconds N`. It samples
-Kubernetes CPU/memory, frontend and Postgres processes, cgroup CPU/memory
-pressure, Postgres active queries/waits/locks, `pg_stat_statements` when
-enabled, and frontend logs. On startup it runs `CREATE EXTENSION IF NOT EXISTS
-pg_stat_statements` and `pg_stat_statements_reset()` through `kubectl exec`
-against `pod/pgsql-0`, so statement summaries start clean for the monitored
-run.
+1. Add a bulk GraphQL read path for explicit API repository permissions.
+2. Add a cheaper presence/filter path for users without explicit API repo
+   permissions.
+3. Add Jaeger spans / metrics around the new store methods and around current
+   `ListUserPermissions` / `CountUserPermissions` paths.
+4. Follow up with a bulk overwrite API for large full-set applies.
 
 ## Current trace findings
 
@@ -226,11 +71,46 @@ slower under the large explicit-perms state. This reinforces that the CLI needs
 better Sourcegraph bulk read and write APIs for very large explicit permission
 sets.
 
-## Sourcegraph engineering request
-
-`src-auth-perms-sync` needs to snapshot explicit API permissions for many
-users. Today it calls `User.permissionsInfo.repositories(source: API)` with
-GraphQL aliases. This is correct, but expensive at scale.
+## Concurrent-operator evidence (2026-06-10)
+
+Four `src-auth-perms-sync` processes ran full explicit-permissions captures
+concurrently against the 10k-user / 50k-repo test instance (each at
+`--parallelism 8`, `--explicit-permissions-batch-size 25`), while a fifth ran a
+small `set` command. Instance: single `pgsql-0` on an 8-core node.
+
+Observed during the concurrent captures:
+
+- `pgsql-0` CPU (`kubectl top`): 7,636–7,683 millicores of 8,000 (saturated).
+- `frontend` / `gitserver` CPU: 124–138m / 2–3m (idle bystanders).
+- `pg_stat_activity`: 29 active statements, all
+  `permsStore.ListUserPermissions`, **zero wait events** — pure CPU, no lock
+  contention.
+- `pg_stat_statements`: `permsStore.ListUserPermissions` at 24,026 calls,
+  27,635.6s total, 1,150ms mean.
+- Per-client capture throughput: 23 users/sec solo → 2–4 users/sec at 4-way
+  concurrency.
+- Aggregate throughput: 8–16 users/sec at 4-way — **below the 23 users/sec a
+  single client achieves alone** (negative scaling).
+- ALB (CloudWatch): no 5xx, no rejected connections — the edge and frontend
+  are not the bottleneck.
+- Collateral failure: the fifth client's queries exceeded the 60s read timeout
+  under this load; 5 retry attempts exhausted; its run failed with exit 1.
+
+Implications for the engineering request:
+
+- A single per-user `permissionsInfo.repositories(source: API)` read costs
+  roughly 0.3–0.4s of Postgres CPU at this state size (1,150ms mean execution
+  under contention), so one operator at modest parallelism can saturate the
+  database by itself, and two concurrent operators degrade each other below
+  single-operator throughput.
+- Timeout/retry behavior amplifies the problem: once statements exceed the
+  client read timeout, retries re-run the same expensive queries, adding load
+  exactly when the database is saturated.
+- A bulk read API (one query returning explicit grants for many users or for
+  whole repos) would replace ~10,000 × ~1s statements per capture with a
+  single scan, and would also make concurrent operators viable.
+
+## Sourcegraph codepath findings
 
 [Deep Search findings](https://sourcegraph.sourcegraph.com/deepsearch/52a24164-1eb3-4db1-a92d-e320ef1c7557)
 from `github.com/sourcegraph/sourcegraph`:
@@ -254,8 +134,14 @@ from `github.com/sourcegraph/sourcegraph`:
   permissions, but the read path uses per-user connection queries and repo
   resolver fanout.
 
-Request a bulk read API for explicit permissions. GraphQL semantics make this
-a query, not a mutation:
+## Proposed bulk read API
+
+`src-auth-perms-sync` needs to snapshot explicit API permissions for many
+users. Today it calls `User.permissionsInfo.repositories(source: API)` with
+GraphQL aliases. This is correct, but expensive at scale.
+
+Request a bulk read API for explicit permissions. GraphQL semantics make this a
+query, not a mutation:
 
 ```graphql
 type ExplicitRepositoryPermission {
@@ -298,6 +184,8 @@ Important requirements:
 Expected benefit: replace hundreds or thousands of per-repo resolver SQL spans
 per request with one indexed `user_repo_permissions` join per user batch.
 
+## Proposed presence/filter API
+
 The `get --users-without-explicit-perms` path also needs a cheaper presence
 check. Today it has to ask
 `User.permissionsInfo.repositories(source: API, first: 1)` for every candidate
@@ -326,15 +214,17 @@ extend type Query {
 }
 ```
 
-Expected benefit: `src-auth-perms-sync get --users-without-explicit-perms`
-can either check explicit-permission presence for candidate users in one indexed
+Expected benefit: `src-auth-perms-sync get --users-without-explicit-perms` can
+either check explicit-permission presence for candidate users in one indexed
 batch query, or ask Sourcegraph for the filtered user set directly instead of
 probing every user through the expensive permissions connection resolver.
 
-The stress profile also needs attention on the write path. A purpose-built
-bulk overwrite API that accepts many repo/user edges at once, streams or stages
-the input server-side, and avoids repeated per-repo permission reconciliation
-would make worst-case full syncs much safer.
+## Bulk overwrite follow-up
+
+The stress profile also needs attention on the write path. A purpose-built bulk
+overwrite API that accepts many repo/user edges at once, streams or stages the
+input server-side, and avoids repeated per-repo permission reconciliation would
+make worst-case full syncs much safer.
 
 ## Copy/paste request
 

{src_auth_perms_sync-0.3.1 → src_auth_perms_sync-0.4.0}/dev/hooks/pre-commit

@@ -19,10 +19,6 @@ run git diff --cached --check
 run git diff --cached --stat
 run git diff --stat
 
-run uv run ruff check .
-run uv run ruff format . --check
-run uv run pyright
-run uv run python -m unittest discover -s tests
-run uv run src-auth-perms-sync --help
+run uv run tests/run.py --local
 
 printf '\nPre-commit quality checks passed.\n'

src_auth_perms_sync-0.4.0/dev/memory-efficiency.md

@@ -0,0 +1,177 @@
+# Memory efficiency testing
+
+Use this when full snapshot capture or full-set apply is slow. The goal is to
+correlate `src-auth-perms-sync` structured logs with Sourcegraph Jaeger spans
+and pod/Postgres load. Request-ready evidence and copy/paste text for
+Sourcegraph Engineering live in
+[engineering-requests.md](./engineering-requests.md).
+
+## Capture a focused trace
+
+Run a small command with `--fetch-sg-traces`. This sends
+`X-Sourcegraph-Request-Trace` and a W3C `traceparent` header on each GraphQL
+request. Sourcegraph may also return `x-trace`, `x-trace-span`, and
+`x-trace-url` response headers.
+
+```bash
+uv run src-auth-perms-sync get \
+  --fetch-sg-traces \
+  --sample-interval 0 \
+  --parallelism 2 \
+  --explicit-permissions-batch-size 25
+```
+
+Find the slow GraphQL HTTP requests in the run log:
+
+```bash
+LOG=src-auth-perms-sync-runs/<endpoint>/runs/<run>/log.json
+
+jq -r '
+  select(.event == "http_request" and .phase == "end") |
+  select(.url | endswith("/.api/graphql")) |
+  [
+    .duration_ms,
+    (.response_headers["x-trace"] // ""),
+    (.response_headers["x-trace-url"] // ""),
+    (.request_headers.traceparent // "")
+  ] | @tsv
+' "$LOG" | sort -nr | head -20
+```
+
+Prefer the `x-trace` value when present. If Sourcegraph did not return one,
+extract the trace ID from `traceparent`:
+
+```bash
+TRACEPARENT=00-<trace-id>-<span-id>-01
+TRACE_ID="$(printf '%s' "$TRACEPARENT" | cut -d- -f2)"
+```
+
+Fetch the trace JSON from Jaeger:
+
+```bash
+curl -sS \
+  -H "Authorization: token $SRC_ACCESS_TOKEN" \
+  "$SRC_ENDPOINT/-/debug/jaeger/api/traces/$TRACE_ID" \
+  > /tmp/sourcegraph-trace.json
+```
+
+Jaeger ingestion can lag. If the API returns `trace not found`, wait briefly
+and retry. For long runs, fetch traces as soon as the relevant command
+finishes; older trace IDs can disappear before a full matrix ends.
+
+Summarize the hottest spans:
+
+```bash
+uv run python - <<'PY'
+import collections
+import json
+
+trace = json.load(open("/tmp/sourcegraph-trace.json"))["data"][0]
+durations_by_operation = collections.defaultdict(list)
+for span in trace["spans"]:
+    durations_by_operation[span["operationName"]].append(span["duration"] / 1000)
+
+for operation, durations in sorted(
+    durations_by_operation.items(),
+    key=lambda item: sum(item[1]),
+    reverse=True,
+)[:15]:
+    print(
+        f"{operation}: count={len(durations)} "
+        f"sum_ms={sum(durations):.1f} "
+        f"max_ms={max(durations):.1f}"
+    )
+PY
+```
+
+Do not commit tokens, customer URLs, raw trace JSON, benchmark CSVs, or monitor
+artifacts. Keep them in `/tmp` unless a human asks to preserve them.
+
+## Trace the end-to-end matrix
+
+Prefer the end-to-end runner as the single orchestrator. With
+`--fetch-sg-traces`, it passes Sourcegraph debug trace collection to every child
+CLI command, tails child JSON logs, and fetches Jaeger traces in the background
+while each child command is still running.
+
+```bash
+uv run python dev/test-end-to-end.py \
+  --fetch-sg-traces \
+  --sample-interval 0 \
+  --external-sample-interval 0 \
+  --results-json /tmp/src-auth-perms-sync-end-to-end-trace.json \
+  --results-csv /tmp/src-auth-perms-sync-end-to-end-trace.csv
+```
+
+Useful trace options:
+
+- `--jaeger-trace-limit N`: fetch only the `N` slowest GraphQL traces per case.
+- `--jaeger-trace-limit 0`: send trace headers but skip Jaeger fetching.
+- `--jaeger-trace-parallelism N`: tune concurrent Jaeger fetches.
+- `--jaeger-trace-jsonl PATH`: stream compact trace summaries as JSON Lines.
+- `--jaeger-trace-dir PATH`: store complete raw Jaeger payloads.
+
+Raw trace files include:
+
+- `trace_request`: CLI-side HTTP and `graphql_query` correlation metadata,
+  including query name, page number, page size, cursor presence, query byte
+  count, variable names, response fields, status, and timing.
+- `jaeger_summary`: compact hot-operation and GraphQL-operation summary.
+- `jaeger_trace`: the complete Jaeger trace JSON returned by Sourcegraph.
+
+All runner flags are Config-backed. You can set them in the shell or `.env`
+with `SRC_AUTH_PERMS_SYNC_E2E_*` names, plus `SRC_ENDPOINT`,
+`SRC_ACCESS_TOKEN`, and `SRC_AUTH_PERMS_SYNC_TEST_USER`.
+
+For each tested batch size and parallelism, record:
+
+- CLI `capture_explicit_grants` duration from the structured log
+- slowest GraphQL `http_request` duration and its trace metadata
+- Jaeger counts and summed duration for `GraphQL Request`, `repos.Get`,
+  `sql.conn.query`, and `database.PermsStore.LoadUserPermissions`
+- run-end `http_retry_count`, `http_request_attempt_count`, and timeout/error
+  counts
+
+## Monitor Sourcegraph load during e2e runs
+
+The runner can start the Sourcegraph pod/Postgres monitor and write monitor
+artifact paths into the result JSON:
+
+```bash
+uv run python dev/test-end-to-end.py \
+  --fetch-sg-traces \
+  --monitor-sourcegraph-load \
+  --sample-interval 0 \
+  --external-sample-interval 0 \
+  --results-json /tmp/src-auth-perms-sync-end-to-end-trace.json \
+  --results-csv /tmp/src-auth-perms-sync-end-to-end-trace.csv
+```
+
+By default, monitor output is written beside `--results-json` or
+`--results-csv` as `*-sourcegraph-load`, and the monitor's stdout/stderr goes
+to `*-sourcegraph-load.log`. Override the location with
+`--monitor-output-dir PATH`. Tune Kubernetes targets and sample intervals with
+the `--monitor-*` flags if the test namespace or pod names differ.
+
+The lower-level helper remains available for focused profiling outside a full
+e2e run:
+
+```bash
+dev/memory-efficiency-monitor-sourcegraph.sh \
+  --namespace m \
+  --output-dir /tmp/src-auth-perms-sync-sourcegraph-load-$(date -u +%Y%m%d-%H%M%S)
+```
+
+Stop the helper with Ctrl-C, or add `--duration-seconds N`. It samples
+Kubernetes CPU/memory, frontend and Postgres processes, cgroup CPU/memory
+pressure, Postgres active queries/waits/locks, `pg_stat_statements` when
+enabled, and frontend logs. On startup it runs `CREATE EXTENSION IF NOT EXISTS
+pg_stat_statements` and `pg_stat_statements_reset()` through `kubectl exec`
+against `pod/pgsql-0`, so statement summaries start clean for the monitored
+run.
+
+## Engineering requests
+
+Request-ready trace findings, stress evidence, Sourcegraph codepath notes,
+proposed GraphQL APIs, and copy/paste issue text now live in
+[engineering-requests.md](./engineering-requests.md).