@ssheleg/sheleg-dev 0.7.0 → 0.9.0

package/plugins/sheleg-dev/skills/error-tracking/SKILL.md

@@ -0,0 +1,259 @@
+---
+name: error-tracking
+description: >-
+  Use when wiring error tracking into a product — adding Sentry to a service, deciding what a
+  DSN is and where it may live, keeping secrets out of events before they reach a third party,
+  tying releases to commits so a stack trace names the change that caused it, or judging whether
+  a green health check means anything. Covers the Sentry SDK options that matter for a backend,
+  the two different tools both called sentry, the auth-token taxonomy and which kind can create
+  a project, and the MCP server with the config channel it belongs in. Triggers - "add Sentry",
+  "set up error tracking", "SENTRY_DSN", "sentry-cli", "sentry mcp", "scrub secrets from
+  errors", "release tracking", "suspect commits", "why is Sentry empty", "подключить Sentry",
+  "трекинг ошибок", "секреты в Sentry", "релизы в Sentry", "наблюдаемость". NOT for judging an
+  agent's trajectory (agent-evals), triaging tickets in a tracker (triage-issue), or choosing an
+  uptime vendor.
+license: MIT
+metadata:
+  version: 0.9.0
+---
+
+# error-tracking — Sentry wired so it does not leak, and so a stack trace names a commit
+
+Adding Sentry is four lines of SDK setup, which is why it is usually done badly.
+The four failures below were all observed, on this machine, in one session, by an
+agent working without this skill.
+
+| Observed failure | What it cost |
+|---|---|
+| Called "get a DSN" a manual human step needing an account | The account existed and the CLI was already authenticated. Error tracking sat switched off for no reason |
+| Wrote a `before_send` scrubber without knowing `EventScrubber` exists and runs by default | Could not say which layer covered what, so could not say whether the code was redundant or load-bearing |
+| Did not know the MCP is OAuth | Would have declared it in the machine's shared gateway, where an OAuth flow cannot complete |
+| Would have set a DSN and stopped | Every issue lands on a release with no commits; suspect-commit attribution never works |
+
+## The one that actually leaks
+
+**Sentry's built-in scrubber matches KEY NAMES. It does not look inside string
+values.** A credential embedded in a URL — inside a message, an exception value,
+a breadcrumb — passes through untouched.
+
+Measured against `sentry-sdk` 2.19.2 on 2026-08-24:
+
+```python
+from sentry_sdk.scrubber import EventScrubber
+event = {"message": "could not connect to postgresql://u:SUPERSECRET@host/db"}
+EventScrubber().scrub_event(event)
+assert "SUPERSECRET" in str(event)   # passes — it was NOT scrubbed
+```
+
+`EventScrubber` is on by default, its denylist holds 32 keys (`password`,
+`token`, `api_key`, `auth`, `secret`, `authorization`, …), and **`dsn` and
+`database_url` are not among them.**
+
+So a service that already writes a database URL to its log — a common enough
+mistake — starts forwarding that same password to a third party the moment
+Sentry is added, with a wider blast radius than the log had.
+
+**Two layers, always:**
+
+```python
+from sentry_sdk.scrubber import DEFAULT_DENYLIST, EventScrubber
+
+sentry_sdk.init(
+    dsn=..., environment=..., release=...,
+    event_scrubber=EventScrubber(                 # layer 1 — keys
+        denylist=DEFAULT_DENYLIST + ["dsn", "database_url", "auth_key"],
+        recursive=True,
+    ),
+    before_send=scrub_values,                     # layer 2 — values
+    send_default_pii=False,
+)
+```
+
+Layer 2 is a regex over URL credentials. Write a test that asserts **layer 1
+alone still leaks** — if that test ever goes red, Sentry started scrubbing
+values and layer 2 may be redundant, which is a thing to learn from a red test
+rather than never. Both layers, the regex and that test: `references/scrubbing.md`.
+
+## A DSN is not an auth token
+
+They are different objects and conflating them produces both paranoia and
+carelessness in the wrong places.
+
+| | DSN | Auth token |
+|---|---|---|
+| Direction | write-only ingest | full API |
+| Lives in | the built artifact, client-side included | CI secrets, a developer's keychain |
+| If it leaks | someone can send you junk events; rate-limit and rotate | someone can read your issues and change your org |
+| Needed to | report an error | create a project, read issues, cut a release |
+
+A DSN in a repository is untidy. An auth token in a repository is an incident.
+**Neither the CLI nor the MCP removes the need for a DSN** — they operate Sentry,
+they do not instrument your app. What they remove is the manual step of getting
+one.
+
+## Setting it up without a human in the loop
+
+Before calling any of this a manual step, check — the answer is often that it is
+already done:
+
+```bash
+sentry auth status          # already authenticated? (credentials in ~/.sentry)
+sentry org list --json      # which orgs this token can see
+sentry project list --json  # what already exists
+```
+
+To fetch a DSN for an existing project, which is what the app needs:
+
+```bash
+sentry project view <org>/<project> --json | python3 -c "import json,sys; print(json.load(sys.stdin)[0]['dsn'])"
+```
+
+Full procedure — install, auth, project creation, wiring the DSN into a host's
+config, and the MCP declaration: read `references/setup.md` when you are adding
+Sentry to a project for the first time.
+
+### The 403 that looks like an auth problem and is not
+
+`sentry project create` can fail with:
+
+```
+Your organization has disabled this feature for members.
+```
+
+even when **you are the owner of that organization**. Measured 2026-08-24. The
+cause is an org policy, `allowMemberProjectCreation: false`, evaluated against
+the token's scopes rather than the human's role — and the OAuth device-flow
+token that `sentry auth` stores carries `project:admin` and `team:write` but
+**no `org:write` or `org:admin`**. The team-scoped endpoint
+(`POST /teams/{org}/{team}/projects/`) fails identically; it is the same policy.
+
+Check before assuming a bug:
+
+```bash
+sentry org view <org> --json | grep allowMemberProjectCreation
+sentry api "/" --json                     # prints the token's actual scopes
+```
+
+Three ways out, in the order worth trying: create the project once in the web
+UI; flip the org setting; or issue an **Internal Integration token**, which is
+the only token type Sentry documents as granting the org-level API access that
+programmatic project creation needs. Organization Tokens are the CI
+recommendation but cannot do org-level operations, and personal tokens are
+explicitly discouraged for CI. Table with the trade-offs: `references/setup.md`.
+
+## Two different tools are called sentry
+
+Confusing them wastes a debugging round, because commands from one silently do
+not exist in the other.
+
+| | `sentry` | `sentry-cli` |
+|---|---|---|
+| From | `cli.sentry.dev`, `brew install getsentry/tools/sentry` | `github.com/getsentry/sentry-cli` |
+| Shape | agent-oriented; `issue explain`, `issue plan`, `api` like `gh api` | build-oriented; source maps, debug files |
+| Auth | `sentry auth` — OAuth device flow, stored in `~/.sentry/` | `SENTRY_AUTH_TOKEN` |
+| Both do | `release create / set-commits / finalize / deploy`, `project`, `org` | |
+
+Either can run the release commands. Pick one per repository and say which in
+the README, or two people will wire two halves.
+
+**Check for a shadow before trusting a version.** Both install into different
+prefixes; `~/.local/bin` typically precedes `/opt/homebrew/bin`, so an older
+copy wins silently:
+
+```bash
+which -a sentry | while read p; do printf '%-34s %s\n' "$p" "$("$p" --version 2>&1 | head -1)"; done
+```
+
+Observed here: 0.38.0 shadowing 0.43.0. Credentials live in `~/.sentry/`, not
+beside the binary, so removing the stale copy does not log you out.
+
+## The MCP is OAuth, which decides where it is declared
+
+`https://mcp.sentry.dev/mcp` answers an unauthenticated request with:
+
+```
+401  www-authenticate: Bearer realm="OAuth", …, resource_metadata="https://mcp.sentry.dev/.well-known/oauth-protected-resource/mcp"
+```
+
+That `www-authenticate` header carrying `resource_metadata` **is** the test for
+OAuth. An OAuth upstream declares itself as the resource at its own address, so
+a client pointed at a local gateway must reject the mismatch and the flow never
+starts. **OAuth MCP servers therefore belong in the agent's own config, never
+behind a shared MCP gateway.** Run the curl before choosing a channel; a static
+token upstream goes to the gateway, an OAuth one does not.
+
+```bash
+claude mcp add --transport http sentry https://mcp.sentry.dev/mcp/<org>/<project>
+```
+
+The MCP is for reading and triaging from inside a session. It is an accelerator:
+everything it does, the CLI also does, and a machine without it loses no
+capability.
+
+## Releases, or the stack trace names nothing
+
+A release in Sentry is only useful if its version string is **identical** to
+what the SDK reports as `release`. When they drift, issues attach to a release
+with no commits and suspect-commit attribution silently does nothing — which
+reads as "Sentry is not very good" rather than as a wiring bug.
+
+Pick one source for the version and use it on both sides. On Heroku that is the
+slug commit:
+
+```python
+RELEASE = os.getenv("HEROKU_SLUG_COMMIT") or os.getenv("RELEASE")
+```
+
+```bash
+VERSION="$(git rev-parse HEAD)"          # the same value
+sentry release create   "<org>/$VERSION" --project <project> --finalize
+sentry release set-commits "<org>/$VERSION" --auto
+sentry release deploy   "<org>/$VERSION" production
+```
+
+**Release bookkeeping is non-fatal.** A deploy that reached production must not
+be reported as failed because a release call did not land. Guard each command so
+it logs and continues; the deploy-script shape is in `references/releases.md`,
+which also covers what `--auto` needs from your repository integration.
+
+## A process that is up is not a service that works
+
+Observed 2026-08-24: a bot token was rotated, and for ten minutes the platform
+reported the dyno `up` with a clean log while the API returned `401` to every
+call and no user was served. Nothing in the system could tell the difference.
+
+**Liveness must assert the thing the service exists to do**, not that a process
+exists. For anything holding a long-lived authenticated connection, check the
+credential periodically and exit non-zero when it fails — a crash loop is
+visible, a silent zombie is not. Sentry helps only if something raises; a
+connection that stopped working without raising produces no event at all. Cron
+monitors (`sentry monitor`) cover the shape where a job stops running.
+
+## Degradation
+
+- **Not Claude Code** (Cursor, Codex, skills CLI, the API container): no MCP, no
+  `/command`. Everything here is CLI and SDK, both of which work anywhere with a
+  shell — except the API container, which has no network and no package install,
+  so treat this skill as Claude Code and Cursor only for the setup half.
+- **`sentry` CLI absent**: say so once, then use the REST API directly with a
+  token (`curl https://sentry.io/api/0/…`), or the web UI for the one-off. Do
+  not loop on the missing binary.
+- **MCP absent**: no capability is lost — read issues with `sentry issue list`.
+- **Not authenticated, or a scope is missing**: interactive auth is a human
+  step. State the exact command once, name what is blocked, and continue with
+  everything that does not need it.
+
+## Verify, do not assume
+
+```bash
+sentry auth status                                   # authenticated, and as whom
+sentry api "/" --json                                # the token's real scopes
+sentry project view <org>/<project> --json           # the DSN, and that it exists
+sentry release list <org>/<project> --limit 3        # releases arriving at all
+python3 -c "import sentry_sdk; print(sentry_sdk.VERSION)"
+```
+
+Two claims worth re-measuring rather than trusting from this page, because both
+are version-dependent: the default denylist contents, and whether the SDK has
+started scrubbing values. `references/scrubbing.md` carries the command for
+each.

package/plugins/sheleg-dev/skills/error-tracking/references/releases.md

@@ -0,0 +1,159 @@
+# Releases, deploys, and health that means something
+
+Read when wiring Sentry into a deploy script, when issues show no suspect
+commits, or when a service reports healthy while serving nobody.
+
+## Contents
+
+- [The version contract](#the-version-contract)
+- [The four release commands](#the-four-release-commands)
+- [In a deploy script, non-fatally](#in-a-deploy-script-non-fatally)
+- [What set-commits --auto needs](#what-set-commits---auto-needs)
+- [Health that means something](#health-that-means-something)
+- [Cron monitors](#cron-monitors)
+
+## The version contract
+
+**The version string you create in Sentry and the one the SDK reports as
+`release` must be byte-identical.** When they differ, every issue attaches to a
+release that has no commits, suspect-commit attribution silently does nothing,
+and "which change broke this" stays unanswerable. Nothing errors. It reads as
+Sentry being unhelpful.
+
+Pick one source and use it on both sides.
+
+| Platform | The value | Read in the app as |
+|---|---|---|
+| Heroku | slug commit | `os.getenv("HEROKU_SLUG_COMMIT")` |
+| Vercel | commit SHA | `os.getenv("VERCEL_GIT_COMMIT_SHA")` |
+| GitHub Actions | `github.sha` | `os.getenv("GITHUB_SHA")` |
+| Anywhere | `git rev-parse HEAD` | inject it at build time |
+
+```python
+RELEASE = os.getenv("HEROKU_SLUG_COMMIT") or os.getenv("RELEASE")
+sentry_sdk.init(dsn=..., release=RELEASE, environment=...)
+```
+
+```bash
+VERSION="$(git rev-parse HEAD)"    # the same string, on the other side
+```
+
+`sentry release propose-version` will invent one, which is convenient and
+exactly the thing that drifts. Prefer an explicit value both sides can derive.
+
+## The four release commands
+
+```bash
+sentry release create      "<org>/$VERSION" --project <project> --finalize
+sentry release set-commits "<org>/$VERSION" --auto
+sentry release deploy      "<org>/$VERSION" <environment>
+sentry release list        "<org>/<project>" --limit 5
+```
+
+- `--finalize` marks it released now. Omit it when you create the release
+  before the code is live and call `sentry release finalize` afterwards.
+- `set-commits --auto` derives the commit range from the previous release.
+  `--local` uses local git history instead when the repository is not linked.
+- `deploy` records environment and timing, which is what "released 4 minutes
+  before the first error" in the UI is built from.
+
+## In a deploy script, non-fatally
+
+Release bookkeeping records what happened; it does not make it happen. A deploy
+that reached production must never be reported as failed because a bookkeeping
+call did not land.
+
+```bash
+notify_sentry() {
+  local version org_project
+  version="$(git rev-parse HEAD)"
+  org_project="${SENTRY_ORG:?}/${SENTRY_PROJECT:?}"
+
+  command -v sentry >/dev/null || { log "sentry CLI absent — skipping"; return 0; }
+  sentry auth status >/dev/null 2>&1 || { log "sentry not authenticated — skipping"; return 0; }
+
+  sentry release create "${SENTRY_ORG}/${version}" \
+      --project "${SENTRY_PROJECT}" --finalize >/dev/null 2>&1 \
+    || { log "could not create the release — continuing"; return 0; }
+  sentry release set-commits "${SENTRY_ORG}/${version}" --auto >/dev/null 2>&1 \
+    || log "could not attach commits (repository linked in Sentry?) — continuing"
+  sentry release deploy "${SENTRY_ORG}/${version}" production >/dev/null 2>&1 \
+    || log "could not mark the deploy — continuing"
+  log "Sentry: release ${version:0:8} recorded for ${org_project}"
+}
+notify_sentry
+```
+
+Under `set -euo pipefail` every branch must return 0 explicitly, or the guard
+you wrote becomes the thing that fails the deploy. Test it with the project
+deliberately absent:
+
+```bash
+bash -c 'set -euo pipefail; . ./deploy.sh --source-only; notify_sentry; echo "exit $?"'
+```
+
+Order matters: record the release **after** the code is live, or the first
+errors from the new build arrive before the release exists and attach to the
+previous one.
+
+## What set-commits --auto needs
+
+`--auto` asks Sentry to resolve commits itself, which requires the repository to
+be linked through a source-control integration (GitHub, GitLab, Bitbucket) in
+the org's settings. Without it the call fails and the release still exists —
+useful, minus attribution.
+
+```bash
+sentry repo list <org>              # what is linked
+sentry release set-commits "<org>/$VERSION" --local   # fallback: local git history
+```
+
+`--local` needs the deploy to run from a real checkout with history, not a
+shallow clone. CI defaults to `fetch-depth: 1`; set it to `0` when using
+`--local`.
+
+## Health that means something
+
+Observed 2026-08-24: a credential was rotated, and for ten minutes the platform
+reported the process `up` with a clean log while the upstream API returned `401`
+to every call. No user was served. Nothing in the system could tell the
+difference between that and working.
+
+**Sentry sees exceptions. It does not see a connection that quietly stopped
+working**, because nothing raises. Two different failures, two different checks:
+
+| Failure | Caught by |
+|---|---|
+| Code raises | Sentry, automatically |
+| Credential revoked, connection idle | a periodic check that asserts authorization |
+| Process died | the platform's own supervisor |
+| Job stopped running | a cron monitor |
+
+For a worker holding a long-lived authenticated connection:
+
+```python
+async def assert_still_authorized(client):
+    """Exit non-zero rather than idle in a state that serves nobody.
+    A crash loop is visible; a silent zombie is not."""
+    try:
+        await client.get_me()
+    except AuthError:
+        logger.error("Credential no longer valid — exiting so the platform restarts us")
+        sys.exit(1)
+```
+
+The rule generalises: **liveness asserts the thing the service exists to do.**
+Anything weaker reports a process, and a process is not a service.
+
+## Cron monitors
+
+For work that is supposed to happen on a schedule, absence is the failure and
+absence raises nothing.
+
+```bash
+sentry monitor list
+```
+
+Wrap the job so Sentry is told it started and finished; a missed check-in then
+becomes an alert. This is the only shape here that catches "it stopped running"
+rather than "it ran and threw".

package/plugins/sheleg-dev/skills/error-tracking/references/scrubbing.md

@@ -0,0 +1,170 @@
+# Keeping secrets out of Sentry events
+
+Read before switching Sentry on in any service that has ever handled a
+credential — which is every service with a database.
+
+## Contents
+
+- [The measurement](#the-measurement)
+- [What each layer covers](#what-each-layer-covers)
+- [Layer 1 — keys, by the SDK](#layer-1--keys-by-the-sdk)
+- [Layer 2 — values, by you](#layer-2--values-by-you)
+- [The test that guards the boundary](#the-test-that-guards-the-boundary)
+- [Re-measuring after an SDK upgrade](#re-measuring-after-an-sdk-upgrade)
+
+## The measurement
+
+`sentry-sdk` 2.19.2, run on 2026-08-24:
+
+```python
+from sentry_sdk.scrubber import EventScrubber
+
+LEAK = "postgresql://u4fc:SUPERSECRETPW@host.rds.amazonaws.com:5432/db"
+event = {
+    "message": f"could not connect to {LEAK}",
+    "exception": {"values": [{"value": f"FATAL: auth failed for {LEAK}"}]},
+    "extra": {"password": "caught", "database_url": LEAK},
+}
+EventScrubber().scrub_event(event)
+
+# extra.password      -> AnnotatedValue           (scrubbed: key is on the denylist)
+# extra.database_url  -> untouched                (key is NOT on the denylist)
+# message             -> untouched                (values are never inspected)
+# exception value     -> untouched
+assert "SUPERSECRETPW" in str(event)
+```
+
+Two facts follow, and neither is a criticism of Sentry — they are the boundary
+of what that layer claims to do:
+
+1. The default denylist holds **32 keys**, and **`dsn` and `database_url` are
+   not among them**. Present: `password`, `passwd`, `secret`, `api_key`,
+   `apikey`, `auth`, `credentials`, `token`, `session`, `authorization`, and
+   similar.
+2. The scrubber matches **key names**. A credential that lives inside a string
+   value — a connection URL in a log line that became a message, an exception
+   from a driver that echoes the DSN — is not seen at all.
+
+Fact 2 is the one that bites. A service that prints a database URL anywhere near
+an error is common; adding Sentry to it forwards that password to a third party
+on every exception, with a wider reach than the log had.
+
+## What each layer covers
+
+| | Matches on | Catches | Misses |
+|---|---|---|---|
+| `EventScrubber` (SDK, default on) | key names, against a denylist | `extra.password`, `request.headers.authorization` | anything inside a string value; keys not on the list |
+| `before_send` (yours) | the shape of the value | `postgres://u:pw@host` wherever it appears | keys whose value is not URL-shaped |
+
+They are complementary, not alternatives. Run both.
+
+## Layer 1 — keys, by the SDK
+
+```python
+from sentry_sdk.scrubber import DEFAULT_DENYLIST, EventScrubber
+
+EXTRA_DENYLIST = ["dsn", "database_url", "bot_token", "api_hash", "auth_key", "session"]
+
+sentry_sdk.init(
+    ...,
+    event_scrubber=EventScrubber(
+        denylist=DEFAULT_DENYLIST + EXTRA_DENYLIST,
+        recursive=True,     # off by default; nested payloads otherwise escape
+    ),
+)
+```
+
+`recursive=False` is the default. Any structure deeper than one level — a
+config dict inside `extra`, a nested exception context — is not walked. Turn it
+on unless you have measured the cost on a payload you actually send.
+
+Add to `EXTRA_DENYLIST` every key your own code uses for a secret. The default
+list knows Sentry's world, not your variable names.
+
+## Layer 2 — values, by you
+
+Match the shape, not a list of known names — a newly added secret is then
+covered by default instead of leaking until someone updates a list.
+
+```python
+import re
+
+_URL_CREDENTIALS = re.compile(r"(?P<scheme>[a-zA-Z][a-zA-Z0-9+.-]*://)(?P<user>[^:/@\s]+):(?P<pw>[^@/\s]+)@")
+_REDACTED = "<redacted>"
+
+def scrub_text(value):
+    if not isinstance(value, str):
+        return value
+    return _URL_CREDENTIALS.sub(rf"\g<scheme>\g<user>:{_REDACTED}@", value)
+
+def scrub_values(event, _hint=None):
+    def walk(node):
+        if isinstance(node, dict):
+            return {k: walk(v) for k, v in node.items()}
+        if isinstance(node, (list, tuple)):
+            return type(node)(walk(v) for v in node)
+        return scrub_text(node)
+    try:
+        return walk(event)
+    except Exception:
+        # Returning None DROPS the event. A scrubber bug must not silently
+        # switch error tracking off.
+        return {"message": "event dropped by scrubber", "level": "error"}
+```
+
+Three properties that matter:
+
+- **Keep the user and host.** A fully redacted URL is useless for debugging.
+  Redact the password only.
+- **Scheme-agnostic.** `amqps://`, `redis://`, `mongodb+srv://` leak the same way.
+- **Never return `None`.** In `before_send` that discards the event. A bug in
+  the scrubber then reads as "Sentry is quiet", which is indistinguishable from
+  "nothing is broken".
+
+## The test that guards the boundary
+
+The important assertion is the one that says **the SDK layer alone still leaks**.
+It documents why layer 2 exists, and it goes red the day that stops being true —
+which is exactly when you want to be told.
+
+```python
+def test_the_sdk_scrubber_alone_would_leak_it():
+    """If this fails, the SDK started scrubbing values and layer 2 may be
+    redundant. Check before deleting it."""
+    from sentry_sdk.scrubber import EventScrubber
+    event = {"message": f"could not connect to {LEAK}"}
+    EventScrubber().scrub_event(event)
+    assert "SUPERSECRETPW" in str(event)
+
+def test_our_layer_catches_what_the_sdk_misses():
+    assert "SUPERSECRETPW" not in str(scrub_values({"message": f"…{LEAK}"}))
+
+def test_extra_denylist_names_what_the_default_omits():
+    from sentry_sdk.scrubber import DEFAULT_DENYLIST
+    for key in ("dsn", "database_url"):
+        assert key not in DEFAULT_DENYLIST, f"{key} is covered now; drop it from EXTRA_DENYLIST"
+        assert key in EXTRA_DENYLIST
+```
+
+The third one keeps the extra list honest: when Sentry adds a key to its
+default, the test tells you to stop carrying it yourself.
+
+**Never put a real credential in a fixture.** Use an obviously fake value of the
+same shape. A test that asserts a password is scrubbed, while carrying that
+password in the repository, has moved the leak rather than closed it.
+
+## Re-measuring after an SDK upgrade
+
+Both facts on this page are version-dependent. After any `sentry-sdk` bump:
+
+```bash
+python3 -c "
+from sentry_sdk.scrubber import DEFAULT_DENYLIST as D
+print(len(D), 'keys'); print([k for k in ('dsn','database_url') if k in D], 'now covered by default')
+"
+python3 -m pytest -k scrubber        # the boundary tests above
+```
+
+If `dsn` or `database_url` appear in the default list, drop them from
+`EXTRA_DENYLIST`. If the value test goes red, layer 2's job changed — read what
+the SDK now does before removing it.