src-auth-perms-sync 0.3.0__tar.gz → 0.4.0__tar.gz

{src_auth_perms_sync-0.3.0 → src_auth_perms_sync-0.4.0}/.gitignore

@@ -12,6 +12,7 @@ __pycache__
 *.py[cod]
 *.py[oc]
 *.swp
+*.tsv
 *.yaml
 build/
 dist/
@@ -24,3 +25,5 @@ wheels/
 !.env.example
 !.markdownlint-cli2.yaml
 !maps-example.yaml
+!tests/tests.yaml
+!tests/e2e/fixtures/**/maps.yaml

{src_auth_perms_sync-0.3.0 → src_auth_perms_sync-0.4.0}/AGENTS.md

@@ -1,5 +1,10 @@
 # AGENTS.md
 
+## Reference materials
+
+- GraphQL schema and database migrations (changes to SQL schema) are available in
+  <https://github.com/sourcegraph/artifacts>
+
 ## Linting
 
 ```bash
@@ -26,21 +31,40 @@ uv run src-auth-perms-sync --help
 
 ## Testing
 
-- First run a dry-run (default behaviour, without `--apply` flag) against a Sourcegraph instance
+All testing runs through one entrypoint: `tests/run.py`. Output goes to the
+console and to a per-run log file under `logs/`. Each level runs only its
+own checks.
 
 ```bash
-uv run src-auth-perms-sync [--get]
-uv run src-auth-perms-sync --set maps.yaml --full
-uv run src-auth-perms-sync --restore backups/<source>/<run>/before.json
+# Fast, no network (also what the pre-commit hook runs):
+# lint, format, pyright, unit + fixture tests, CLI rejection matrix,
+# randomized permission invariants
+uv run tests/run.py
+
+# End-to-end runs against the .env test instance with independent GraphQL
+# read-back verification, and a wheel install smoke test
+uv run tests/run.py --live
+
+# Run a subset: comma-delimited test names, substring match
+uv run tests/run.py --live full-overwrite-unions
+uv run tests/run.py --live wheel,baseline
+
+# Repeated timed runs with Jaeger trace retention, RSS sampling,
+# optional kubectl load monitoring, and baseline comparison
+uv run tests/run.py --performance --repeat 3
+uv run tests/run.py --performance --baseline-command "uvx src-auth-perms-sync@latest" \
+  --fail-on-memory-regression-percent 10
+
+# Regenerate fixture goldens after editing tests/e2e/fixtures/ cases
+uv run tests/run.py --update-golden
 ```
 
-- Read the output, and evaluate the expected changes
-- If the expected changes look correct
-  - Run with the `--apply` flag against the test instance
-  - Read and evaluate the output for expected changes
-  - Run with the `--restore` flag against the test instance
-  - Always inspect the before / after snapshots in
-    `src-auth-perms-sync-runs/<endpoint>/backups/` afterward to confirm the diff matches what you expected
+- Fixture cases live in `tests/e2e/fixtures/<case>/` — see the README there
+  for the format. Add cases there to cover new mapping behaviors.
+- For manual verification against a real instance, dry-run first (no
+  `--apply`), read the planned changes, then `--apply` on a scratch instance
+  and inspect the before/after snapshots under
+  `src-auth-perms-sync-runs/<endpoint>/runs/`.
 
 ## Release process
 

{src_auth_perms_sync-0.3.0 → src_auth_perms_sync-0.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: src-auth-perms-sync
-Version: 0.3.0
+Version: 0.4.0
 Summary: Set Sourcegraph permissions from authentication provider data
 Project-URL: Homepage, https://github.com/sourcegraph/src-auth-perms-sync
 Project-URL: Issues, https://github.com/sourcegraph/src-auth-perms-sync/issues

{src_auth_perms_sync-0.3.0 → src_auth_perms_sync-0.4.0}/dev/TODO.md

@@ -1,20 +1,21 @@
 # TODO
 
-## High priority: Sync modes
+## Medium priority: extend SAML-group live coverage to org sync
 
-### Fast
+tests/setup.py now fabricates SAML accounts with synthetic groups
+(`perms-sync-test-eng` / `perms-sync-test-sales`, see tests/setup.yaml).
+saml-group-live covers permission mapping; add a seeded
+`sync-saml-orgs --apply` live case that maps those groups to a throwaway
+org and asserts membership is added AND removed (today's
+sync-saml-orgs-apply only covers the single real Okta user, add-only).
 
-- Additive modes, to add new users’ perms quickly,
-  without the extraneous load on the database of a full sync
-- Take a list of usernames and/or email addresses as input,
-  query users on the instance for these,
-  then trigger a perms sync for found users
-- Query the instance for all new users, which do not yet have explicit perms
-- Query the instance for all new repos, which do not yet have explicit perms
+## Decide: pendingBindIDs / usersWithPendingPermissions
 
-### Full: Overwrite all perms
-
-- Separate full sync mode with an arg
+The CLI cannot create pending permissions (it validates users exist), but
+snapshots record `pending_bindIDs`, and setup.py / the live hygiene check
+report (never delete) any that appear. Decide whether "grant before first
+login" is a customer need; if not, consider dropping the snapshot field.
+See the thread discussion 2026-06-11.
 
 ## High priority: Remote trigger on demand
 
@@ -29,14 +30,6 @@
 - How do we avoid stampedes (e.g., bulk repo sync triggering thousands
   of re-runs)?
 
-## High priority: End to End test cases
-
-- Create test cases. Each test case should contain:
-  - Before state
-  - maps.yaml file
-  - Expected after state
-- Script to run the script, and verify the after state matches the expected after state
-
 ## High priority: Verify perms are updated when a user's SAML groups change
 
 - If a user gets added to a new SAML group, which hits a mapping, ensure they
@@ -45,14 +38,24 @@
 ## High priority: Reduce worst-case full-permission sync load
 
 - Use the stress-run evidence in
-  [memory-efficiency.md](./memory-efficiency.md)
+  [engineering-requests.md](./engineering-requests.md)
   to request Sourcegraph bulk explicit-permission read and write APIs.
+  New evidence 2026-06-10: the whole-instance apply (1,150 repo
+  overwrites x 10,002 bindIDs each at parallelism 16) crashed the test
+  instance's Postgres ("connection refused", "unexpected EOF"); the
+  client circuit breaker opened and the harness restored cleanly. That
+  stress cycle is now opt-in: `uv run tests/run.py --live "full cycle"`.
 - Add an explicit destructive/performance-test mode to the e2e runner so giant
   stress runs can skip or defer full restore cleanup when the goal is finding
   the server-side breaking point.
 - Revisit full snapshot capture once Sourcegraph exposes a bulk read path;
   replace aliased `User.permissionsInfo.repositories(source: API)` calls before
   raising concurrency further.
+- `get --repos <name>` still scans every user's explicit grants to find one
+  repo's holders (~400 s at 10k users). A repo-centric read
+  (`repository.permissionsInfo.users` + site-admin disambiguation, as the
+  test harness already does) would make it seconds — see the repo-centric
+  section below.
 
 ## Low priority: Repo-centric path, when users > repos, or for cross-checking
 

src_auth_perms_sync-0.3.0/dev/memory-efficiency.md → src_auth_perms_sync-0.4.0/dev/engineering-requests.md

@@ -1,173 +1,18 @@
-# Memory efficiency testing
-
-Use this when full snapshot capture or full-set apply is slow. The goal is to
-correlate `src-auth-perms-sync` structured logs with Sourcegraph Jaeger spans
-and pod/Postgres load, then use the evidence to ask Sourcegraph engineering for
-bulk explicit-permissions APIs.
-
-## Capture a focused trace
-
-Run a small command with `--fetch-sg-traces`. This sends
-`X-Sourcegraph-Request-Trace` and a W3C `traceparent` header on each GraphQL
-request. Sourcegraph may also return `x-trace`, `x-trace-span`, and
-`x-trace-url` response headers.
-
-```bash
-uv run src-auth-perms-sync get \
-  --fetch-sg-traces \
-  --sample-interval 0 \
-  --parallelism 2 \
-  --explicit-permissions-batch-size 25
-```
-
-Find the slow GraphQL HTTP requests in the run log:
-
-```bash
-LOG=src-auth-perms-sync-runs/<endpoint>/runs/<run>/log.json
-
-jq -r '
-  select(.event == "http_request" and .phase == "end") |
-  select(.url | endswith("/.api/graphql")) |
-  [
-    .duration_ms,
-    (.response_headers["x-trace"] // ""),
-    (.response_headers["x-trace-url"] // ""),
-    (.request_headers.traceparent // "")
-  ] | @tsv
-' "$LOG" | sort -nr | head -20
-```
-
-Prefer the `x-trace` value when present. If Sourcegraph did not return one,
-extract the trace ID from `traceparent`:
-
-```bash
-TRACEPARENT=00-<trace-id>-<span-id>-01
-TRACE_ID="$(printf '%s' "$TRACEPARENT" | cut -d- -f2)"
-```
-
-Fetch the trace JSON from Jaeger:
-
-```bash
-curl -sS \
-  -H "Authorization: token $SRC_ACCESS_TOKEN" \
-  "$SRC_ENDPOINT/-/debug/jaeger/api/traces/$TRACE_ID" \
-  > /tmp/sourcegraph-trace.json
-```
-
-Jaeger ingestion can lag. If the API returns `trace not found`, wait briefly
-and retry. For long runs, fetch traces as soon as the relevant command
-finishes; older trace IDs can disappear before a full matrix ends.
-
-Summarize the hottest spans:
-
-```bash
-uv run python - <<'PY'
-import collections
-import json
-
-trace = json.load(open("/tmp/sourcegraph-trace.json"))["data"][0]
-durations_by_operation = collections.defaultdict(list)
-for span in trace["spans"]:
-    durations_by_operation[span["operationName"]].append(span["duration"] / 1000)
-
-for operation, durations in sorted(
-    durations_by_operation.items(),
-    key=lambda item: sum(item[1]),
-    reverse=True,
-)[:15]:
-    print(
-        f"{operation}: count={len(durations)} "
-        f"sum_ms={sum(durations):.1f} "
-        f"max_ms={max(durations):.1f}"
-    )
-PY
-```
-
-Do not commit tokens, customer URLs, raw trace JSON, benchmark CSVs, or monitor
-artifacts. Keep them in `/tmp` unless a human asks to preserve them.
-
-## Trace the end-to-end matrix
-
-Prefer the end-to-end runner as the single orchestrator. With
-`--fetch-sg-traces`, it passes Sourcegraph debug trace collection to every child
-CLI command, tails child JSON logs, and fetches Jaeger traces in the background
-while each child command is still running.
-
-```bash
-uv run python dev/test-end-to-end.py \
-  --fetch-sg-traces \
-  --sample-interval 0 \
-  --external-sample-interval 0 \
-  --results-json /tmp/src-auth-perms-sync-end-to-end-trace.json \
-  --results-csv /tmp/src-auth-perms-sync-end-to-end-trace.csv
-```
-
-Useful trace options:
-
-- `--jaeger-trace-limit N`: fetch only the `N` slowest GraphQL traces per case.
-- `--jaeger-trace-limit 0`: send trace headers but skip Jaeger fetching.
-- `--jaeger-trace-parallelism N`: tune concurrent Jaeger fetches.
-- `--jaeger-trace-jsonl PATH`: stream compact trace summaries as JSON Lines.
-- `--jaeger-trace-dir PATH`: store complete raw Jaeger payloads.
-
-Raw trace files include:
-
-- `trace_request`: CLI-side HTTP and `graphql_query` correlation metadata,
-  including query name, page number, page size, cursor presence, query byte
-  count, variable names, response fields, status, and timing.
-- `jaeger_summary`: compact hot-operation and GraphQL-operation summary.
-- `jaeger_trace`: the complete Jaeger trace JSON returned by Sourcegraph.
-
-All runner flags are Config-backed. You can set them in the shell or `.env`
-with `SRC_AUTH_PERMS_SYNC_E2E_*` names, plus `SRC_ENDPOINT`,
-`SRC_ACCESS_TOKEN`, and `SRC_AUTH_PERMS_SYNC_TEST_USER`.
-
-For each tested batch size and parallelism, record:
-
-- CLI `capture_explicit_grants` duration from the structured log
-- slowest GraphQL `http_request` duration and its trace metadata
-- Jaeger counts and summed duration for `GraphQL Request`, `repos.Get`,
-  `sql.conn.query`, and `database.PermsStore.LoadUserPermissions`
-- run-end `http_retry_count`, `http_request_attempt_count`, and timeout/error
-  counts
-
-## Monitor Sourcegraph load during e2e runs
-
-The runner can start the Sourcegraph pod/Postgres monitor and write monitor
-artifact paths into the result JSON:
-
-```bash
-uv run python dev/test-end-to-end.py \
-  --fetch-sg-traces \
-  --monitor-sourcegraph-load \
-  --sample-interval 0 \
-  --external-sample-interval 0 \
-  --results-json /tmp/src-auth-perms-sync-end-to-end-trace.json \
-  --results-csv /tmp/src-auth-perms-sync-end-to-end-trace.csv
-```
+# Engineering requests
 
-By default, monitor output is written beside `--results-json` or
-`--results-csv` as `*-sourcegraph-load`, and the monitor's stdout/stderr goes
-to `*-sourcegraph-load.log`. Override the location with
-`--monitor-output-dir PATH`. Tune Kubernetes targets and sample intervals with
-the `--monitor-*` flags if the test namespace or pod names differ.
+Use this when opening Sourcegraph Engineering issues from memory-efficiency
+evidence. Capture steps stay in [memory-efficiency.md](./memory-efficiency.md);
+this file keeps the request-ready problem statement, evidence, proposed API
+shape, and copy/paste issue text.
 
-The lower-level helper remains available for focused profiling outside a full
-e2e run:
+## Requested Sourcegraph changes
 
-```bash
-dev/memory-efficiency-monitor-sourcegraph.sh \
-  --namespace m \
-  --output-dir /tmp/src-auth-perms-sync-sourcegraph-load-$(date -u +%Y%m%d-%H%M%S)
-```
-
-Stop the helper with Ctrl-C, or add `--duration-seconds N`. It samples
-Kubernetes CPU/memory, frontend and Postgres processes, cgroup CPU/memory
-pressure, Postgres active queries/waits/locks, `pg_stat_statements` when
-enabled, and frontend logs. On startup it runs `CREATE EXTENSION IF NOT EXISTS
-pg_stat_statements` and `pg_stat_statements_reset()` through `kubectl exec`
-against `pod/pgsql-0`, so statement summaries start clean for the monitored
-run.
+1. Add a bulk GraphQL read path for explicit API repository permissions.
+2. Add a cheaper presence/filter path for users without explicit API repo
+   permissions.
+3. Add Jaeger spans / metrics around the new store methods and around current
+   `ListUserPermissions` / `CountUserPermissions` paths.
+4. Follow up with a bulk overwrite API for large full-set applies.
 
 ## Current trace findings
 
@@ -226,11 +71,46 @@ slower under the large explicit-perms state. This reinforces that the CLI needs
 better Sourcegraph bulk read and write APIs for very large explicit permission
 sets.
 
-## Sourcegraph engineering request
-
-`src-auth-perms-sync` needs to snapshot explicit API permissions for many
-users. Today it calls `User.permissionsInfo.repositories(source: API)` with
-GraphQL aliases. This is correct, but expensive at scale.
+## Concurrent-operator evidence (2026-06-10)
+
+Four `src-auth-perms-sync` processes ran full explicit-permissions captures
+concurrently against the 10k-user / 50k-repo test instance (each at
+`--parallelism 8`, `--explicit-permissions-batch-size 25`), while a fifth ran a
+small `set` command. Instance: single `pgsql-0` on an 8-core node.
+
+Observed during the concurrent captures:
+
+- `pgsql-0` CPU (`kubectl top`): 7,636–7,683 millicores of 8,000 (saturated).
+- `frontend` / `gitserver` CPU: 124–138m / 2–3m (idle bystanders).
+- `pg_stat_activity`: 29 active statements, all
+  `permsStore.ListUserPermissions`, **zero wait events** — pure CPU, no lock
+  contention.
+- `pg_stat_statements`: `permsStore.ListUserPermissions` at 24,026 calls,
+  27,635.6s total, 1,150ms mean.
+- Per-client capture throughput: 23 users/sec solo → 2–4 users/sec at 4-way
+  concurrency.
+- Aggregate throughput: 8–16 users/sec at 4-way — **below the 23 users/sec a
+  single client achieves alone** (negative scaling).
+- ALB (CloudWatch): no 5xx, no rejected connections — the edge and frontend
+  are not the bottleneck.
+- Collateral failure: the fifth client's queries exceeded the 60s read timeout
+  under this load; 5 retry attempts exhausted; its run failed with exit 1.
+
+Implications for the engineering request:
+
+- A single per-user `permissionsInfo.repositories(source: API)` read costs
+  roughly 0.3–0.4s of Postgres CPU at this state size (1,150ms mean execution
+  under contention), so one operator at modest parallelism can saturate the
+  database by itself, and two concurrent operators degrade each other below
+  single-operator throughput.
+- Timeout/retry behavior amplifies the problem: once statements exceed the
+  client read timeout, retries re-run the same expensive queries, adding load
+  exactly when the database is saturated.
+- A bulk read API (one query returning explicit grants for many users or for
+  whole repos) would replace ~10,000 × ~1s statements per capture with a
+  single scan, and would also make concurrent operators viable.
+
+## Sourcegraph codepath findings
 
 [Deep Search findings](https://sourcegraph.sourcegraph.com/deepsearch/52a24164-1eb3-4db1-a92d-e320ef1c7557)
 from `github.com/sourcegraph/sourcegraph`:
@@ -254,8 +134,14 @@ from `github.com/sourcegraph/sourcegraph`:
   permissions, but the read path uses per-user connection queries and repo
   resolver fanout.
 
-Request a bulk read API for explicit permissions. GraphQL semantics make this
-a query, not a mutation:
+## Proposed bulk read API
+
+`src-auth-perms-sync` needs to snapshot explicit API permissions for many
+users. Today it calls `User.permissionsInfo.repositories(source: API)` with
+GraphQL aliases. This is correct, but expensive at scale.
+
+Request a bulk read API for explicit permissions. GraphQL semantics make this a
+query, not a mutation:
 
 ```graphql
 type ExplicitRepositoryPermission {
@@ -298,10 +184,47 @@ Important requirements:
 Expected benefit: replace hundreds or thousands of per-repo resolver SQL spans
 per request with one indexed `user_repo_permissions` join per user batch.
 
-The stress profile also needs attention on the write path. A purpose-built
-bulk overwrite API that accepts many repo/user edges at once, streams or stages
-the input server-side, and avoids repeated per-repo permission reconciliation
-would make worst-case full syncs much safer.
+## Proposed presence/filter API
+
+The `get --users-without-explicit-perms` path also needs a cheaper presence
+check. Today it has to ask
+`User.permissionsInfo.repositories(source: API, first: 1)` for every candidate
+user, in aliased batches. Recent test runs show the client can parallelize
+those batches, but the Sourcegraph frontend / load balancer can still return
+502/503s under that resolver load. Add one or both direct APIs:
+
+```graphql
+type ExplicitRepositoryPermissionPresence {
+  userID: ID!
+  hasExplicitRepositoryPermissions: Boolean!
+}
+
+extend type Query {
+  explicitRepositoryPermissionPresenceForUsers(
+    userIDs: [ID!]!
+    source: PermissionSource = API
+  ): [ExplicitRepositoryPermissionPresence!]!
+
+  usersWithoutExplicitRepositoryPermissions(
+    createdAt: DateTimeFilter
+    source: PermissionSource = API
+    first: Int
+    after: String
+  ): UserConnection!
+}
+```
+
+Expected benefit: `src-auth-perms-sync get --users-without-explicit-perms` can
+either check explicit-permission presence for candidate users in one indexed
+batch query, or ask Sourcegraph for the filtered user set directly instead of
+probing every user through the expensive permissions connection resolver.
+
+## Bulk overwrite follow-up
+
+The stress profile also needs attention on the write path. A purpose-built bulk
+overwrite API that accepts many repo/user edges at once, streams or stages the
+input server-side, and avoids repeated per-repo permission reconciliation would
+make worst-case full syncs much safer.
 
 ## Copy/paste request
 
@@ -324,7 +247,10 @@ Request: add a bulk explicit-permissions read API that accepts many user IDs and
 returns compact permission edges (`userID`, `repositoryID`, `repositoryName`,
 `updatedAt`) for `source: API`, without resolving full `Repository` GraphQL
 objects. A single indexed query over `user_repo_permissions` joined to `repo`
-should be enough for each user batch.
+should be enough for each user batch. Also add a cheaper presence/filter path
+for `get --users-without-explicit-perms`: either `userID -> has explicit API
+repo permissions` for many users, or a direct query for users without explicit
+API repo permissions, optionally filtered by `createdAt`.
 
 Acceptance criteria:
 
@@ -336,6 +262,9 @@ Acceptance criteria:
   latency visible.
 - `src-auth-perms-sync` can replace its aliased
   `User.permissionsInfo.repositories(source: API)` calls with this API.
+- `src-auth-perms-sync get --users-without-explicit-perms` can stop probing
+  every candidate user through `User.permissionsInfo.repositories(source: API,
+  first: 1)`.
 - Follow-up: evaluate a bulk overwrite API for large full-set applies. The
   stress run planned roughly 10 million grants and observed
   `permsStore.upsertUserRepoPermissions-range1` averaging about 2.5s per call.

{src_auth_perms_sync-0.3.0 → src_auth_perms_sync-0.4.0}/dev/hooks/pre-commit

@@ -19,10 +19,6 @@ run git diff --cached --check
 run git diff --cached --stat
 run git diff --stat
 
-run uv run ruff check .
-run uv run ruff format . --check
-run uv run pyright
-run uv run python -m unittest discover -s tests
-run uv run src-auth-perms-sync --help
+run uv run tests/run.py --local
 
 printf '\nPre-commit quality checks passed.\n'

{src_auth_perms_sync-0.3.0/dev → src_auth_perms_sync-0.4.0/dev/memory-analysis}/memory-efficiency-generate.py

@@ -551,7 +551,7 @@ def list_external_services(client: src.SourcegraphClient) -> list[ExternalServic
             )
         )
     if not services:
-        raise SystemExit("No external services found on the Sourcegraph instance")
+        raise SystemExit("No code hosts found on the Sourcegraph instance")
     return services