stackvitals 0.1.0__py3-none-any.whl

stackvitals/web/app/help/connectors.json

@@ -0,0 +1 @@
+{"connectors":"## How each integration was tested\n\n**Live** means run against the real service; **recorded** means tested against saved answers in the shape the\nservice's documentation gives, with no account of ours; **experimental** means built but not yet checked either way.\n\n| Integration | Tested | How |\n|---|---|---|\n| GitHub (Actions, pull requests, issues, deployments, checks, dependency graph) | Live | A sandbox repo with the built-in token, and public repos' history |\n| Jira Cloud | Live for reading, recorded for write-back | A public site read without a key, and a keyed test site (2026-10-28) |\n| Linear, PagerDuty | Live sign-in and read, recorded for write-back and full collection | Test accounts of ours (2026-10-28) |\n| incident.io, GitHub Copilot usage | Recorded | incident.io's free plan has no API; Copilot usage needs an organisation with seats |\n| Slack and Discord alerts | Live | A test message posted through each service's webhook (2026-10-28) |\n| Microsoft Teams alerts; the instant-alerts Worker | Recorded | Payloads checked against Teams' format |\n| Discourse | Live | Five public forums (cpython, rust, pytorch, julia, transformers) |\n| Slack and Discord community channels | Live sign-in and channel read | A test workspace and server of ours (2026-10-28) |\n| GitHub Discussions | Recorded | |\n| GitLab CI | Live | gitlab-org/cli's pipelines |\n| GitLab CI job (data branch, Pages, merge request note) | Token check live, job recorded | A test project's token checked for api and write_repository (2026-10-28); the job itself against a stand-in API |\n| Test reports from Jenkins, CircleCI, Buildkite | Recorded | The upload command with sample reports |\n| SonarQube and SonarCloud | Live for public projects, recorded with a token | A public SonarCloud project's quality gate and measures read without a key (2026-10-29) |\n| Bitbucket Cloud and Azure DevOps (pull requests, Bitbucket deployments) | Live | Public repos read without a key: atlassian-connect-express and PowerShell (2026-10-29) |\n| GitLab merge requests and deployments | Live | gitlab-org/cli, 153 merge requests |\n| Deployments and incidents from any tool (`stackvitals record`, repository_dispatch) | Recorded | The command and the dispatch payload against a stand-in API |\n| GitHub Enterprise Server | Recorded | The API and GraphQL URLs pointed at a stand-in server |\n| AI spend | Live for OpenAI, recorded for Anthropic, Claude Code and Cursor | An OpenAI admin key read 30 days of costs (2026-10-29) |\n| Backstage `catalog-info.yaml` | Live | 198 real files from public repos |\n| DevLake tables, Postgres, OpenTelemetry exports | Recorded | DevLake's own SQL run on our export of 42 generated repos; OTel spans written from a real repo's runs |\n| OSV, deps.dev, CISA KEV, EPSS, OpenSSF Scorecard | Live | The crawler's daily runs |\n| Model triage (Gemini) | Live | Public failures |\n| OSV-Scanner (every lockfile) | Live | 100 crawled repos in ten languages, and lockfiles in eight ecosystems |\n| GuardDog (opt-in code check) | Experimental | Runs only on Linux runners; checked against recorded output |\n| Test connection: Discourse, Gemini triage | Live | Checked against the real services on 2026-10-24 (a public forum; a free Gemini key) |\n| Test connection: Jira, Linear, PagerDuty, Slack, Discord, chat webhooks, GitLab | Live | `stackvitals connectors check` against test accounts of ours on 2026-10-28; every one passed |\n| Test connection: incident.io, Copilot, S3 bucket | Recorded | Each service's success and refusal answers, saved; no keyed accounts of ours |\n\n## Test a connection\n\nThe app's Test connection button starts a manual run with the request `{\"version\": 1, \"check\": [\"jira\", \"slack\"]}` (or\n`\"check\": \"all\"` for every connector set up); a request issue with that block, or a `/stackvitals check jira slack`\ncomment from someone with write access, does the same. A Test connection run from the app gets its own concurrency\ngroup, so a busy data queue never drops it; its write to the data branch retries on top of any run that pushed first.\nThat run does nothing else: it makes one cheap call per\nconnector, writes the result under the connector's entry in `docs/connectors.json` as `\"check\": {\"ok\", \"as\", \"detail\",\n\"checked_at\", \"fix\"}`, and puts one line per connector in the job summary (or the answer on the issue). What each check\ncalls: Jira, your account and a one-issue search with `scope_jql`; Linear, the key's user and teams; PagerDuty, each\nconfigured service and a one-incident list; incident.io, a one-incident list; Slack, `auth.test` and a one-message read\nof each channel; Discord, the bot's user and a one-message read of each channel; Discourse, the forum's `latest.json`\n(no key); GitLab, the project and the token's own scopes (it needs `api` and `write_repository`; at\n`CI_API_V4_URL` for a self-hosted GitLab); the bucket, a list of the prefix, then a tiny `.stackvitals-check` object written and\ndeleted; model triage, the provider's model list; Copilot, the org's usage report for one day. The chat webhooks\n(`webhook` and each team webhook) get one message titled \"StackVitals connection test\", only when a check is asked\nfor, and never retried, so a slow chat doesn't get it twice. If the bucket kept the test object (the key can't\ndelete), the result says where it was left so you can delete it. A failure says what went wrong (a missing secret by name, a wrong or expired key, a missing permission or scope, a\nwrong site, team, service or channel, or a service that is down) and the next step. Results never hold a key, and on a\npublic repo they name no account, bot, project or bucket (only the forum and the model provider are named), and a\nfailure says \"one configured channel\" (or team, service, bucket) instead of its id. To test your keys locally, set the same variables the Action\nuses (`STACKVITALS_JIRA_TOKEN`, `STACKVITALS_SLACK_TOKEN`, ...; a `.env` file works, and the repository secret names\nsuch as `JIRA_API_TOKEN` or `SLACK_BOT_TOKEN` are read too) and run\n`stackvitals connectors check jira slack` (or `--all`, `--json`).\n\n## Read a service again\n\nWhen a service's numbers look off (a key was wrong for a week, an outage at the service), read just that service\nagain from a date instead of starting over: the **Read again** button on the app's Data health page opens a request\nissue (change the date in it first if you like), or comment `/stackvitals resync jira since 2026-09-01` (several\nservices at once, or no date for the whole 180 days), or send `{\"version\": 1, \"resync\": [{\"source\": \"jira\", \"since\":\n\"2026-09-01\"}]}` as the workflow's request. Someone with write access is needed, as for settings. The same run reads\nthe service again from that date (Jira, Linear, PagerDuty, incident.io, Copilot; Slack, Discord and Discourse read\ntheir whole window). The two GitHub sources work the same way: `ci` lists each day's runs again from the date and\nfetches only the runs it doesn't have (about one request a day), and `flow` reads every pull request updated since\nthe date again; what was kept stays, and records read again win by id, so nothing is counted twice. The run\nanswers with what it will read and the Activity page logs who asked.\n\n## GitHub (built in)\n\n| Module | What it reads | Permission in the workflow |\n|---|---|---|\n| CI Health | workflow runs, their attempts and jobs; test reports (JUnit, CTRF, Jest/Vitest, Go, TRX) in run artifacts | `actions: read` (`write` to re-run flaky jobs) |\n| Team Flow | pull requests with their reviews, review requests, comments, labels, checks; releases and deployments | `pull-requests: read`, `checks: read`, `statuses: read`, `deployments: read` |\n| Delivery | what Team Flow and CI Health read, plus production deployments (GitHub's Deployments API), deploy workflows and releases | as above |\n| Community | issues and pull requests with their first answers; optionally traffic through your own GitHub App | `issues: read` |\n| Portal | the org's repos, CODEOWNERS, teams, branch rules (an org-owned read-only App for private repos) | an App token |\n| Dependency Steward | the lockfiles changed in a pull request; package evidence from the daily crawl | `contents: read` |\n\nAll of it goes into the repo's own `stackvitals-data` branch. Without a permission, that part is skipped and the job\nsummary names the permission to add.\n\n## Issue trackers: Jira Cloud and Linear (one optional key each)\n\nEach is read by your own workflow run with one key from your repo's secrets, on the schedule and manual runs, and kept in\n`connectors/<name>/` on the data branch. The first pull reads 180 days of issues; later pulls read only what changed (at\nmost 20 pages a run, so a big backlog fills in over a few runs, each run carrying on where the last one stopped). Nothing\ngoes through a server of ours, and the keys are only used to read unless you turn on write-back (below).\n\n- A public repo's data branch is as public as its pages, so for one that isn't known to be private, issue titles, issue\n  links and sprint names are never stored; sprints are named by their end date instead.\n- Changing a connector's settings (or making the repo public) makes it read everything again from 180 days back, so\n  narrower settings drop what they now leave out and wider ones fill in the past.\n- On a private repo, Jira and PagerDuty also keep each answer as the service sent it (`connectors/<name>/raw/`), so\n  when a StackVitals update reads records differently, they are rebuilt from those answers with no new calls, and the\n  job summary says so. A public repo keeps no answers (they hold titles) and reads again instead. Answers follow\n  the records' 400-day rule and count toward the branch's size cap.\n- Every connector asks its service the same way: a dropped connection or a server error is tried again, a rate limit\n  asking to wait up to a minute is waited out, and a longer one stops that run, which the next run carries on.\n- Flaky-test tickets: `flaky_tickets = true` with `ticket_project = \"WEB\"` under `[connectors.jira]` (or\n  `ticket_team = \"ENG\"` under `[connectors.linear]`) opens one ticket per test that turns flaky, with its flake count,\n  runs and usual failure, at most 5 a run; once the test counts as fixed the ticket gets a comment and moves to\n  `done_transition` / `done_state` (or Done). What was filed is kept on the data branch, so a test is never filed\n  twice. Jira tickets are Bugs, or Tasks in projects without that type. Checked live on 2026-10-29: Jira SCRUM-9 and\n  Linear ENG-5, each opened and closed.\n- Jira custom fields: `team_field` and `severity_field` under `[connectors.jira]` name a field by its name (`Team`) or\n  id (`customfield_10001`). Each issue keeps its value as text: done issues are counted per tracker team in the Issues\n  view, and an incident shows its severity. A public repo never keeps the team.\n- On a private repo, Jira and Linear issues keep who they are assigned to (an account id and, when the service\n  shows it, an email). With `[identity] aliases` naming those people's GitHub logins, Delivery counts issues done\n  per team, and Slack and Discord count a reply from an aliased team member as the team's, so nobody is listed twice.\n  Without an alias, an assignee whose email name is a GitHub login of this repo (`bo@acme.com` for `bo`) counts as\n  that login; one whose email or name only reads like a login (`kaushik.yadav@` or \"Kaushik Yadav\" for\n  `kaushik-yadav`) is not counted, but the Issues view offers the alias line to paste. A public repo keeps no assignee.\n- Only connectors whose key is in the workflow count: remove a key, and its issues and incidents stop counting (what\n  was pulled stays on the branch and counts again if the key comes back).\n- Like the rest of the history, connector day files are kept 400 days (Copilot reports two years) and count toward\n  the branch's size cap.\n\n| | Jira Cloud | Linear |\n|---|---|---|\n| Key | an API token for the account to read as, plus its email: secrets `JIRA_API_TOKEN` and `JIRA_EMAIL` | a personal API key with read access: secret `LINEAR_API_KEY` |\n| Workflow lines (new installs have them) | `jira-email: ${{ secrets.JIRA_EMAIL }}`, `jira-token: ${{ secrets.JIRA_API_TOKEN }}` | `linear-token: ${{ secrets.LINEAR_API_KEY }}` |\n| Settings | `[connectors.jira]` `url` (your `https://<name>.atlassian.net` site; no other host is accepted, so the key can't be sent elsewhere), `scope_jql`, `incident_jql`, `key_expires` | `[connectors.linear]` `teams`, `incident_labels`, `key_expires` |\n| Read | issues (type, status, created, resolved, points, labels), status and sprint changes, sprints | issues (state, created, completed or canceled, estimate, labels), state and cycle changes, cycles |\n| API | REST v3 enhanced search (`/rest/api/3/search/jql`), resuming from the newest update read | GraphQL, one fixed span per pass, resuming from its cursor |\n\nWhat they unlock: issue lead time (created to done) and cycle time (first in-progress status to done) by type; sprints\nand cycles with scope, done by the end, carried over and added after start; incidents (Jira issues found by\n`incident_jql`, Linear issues with an incident label) in DORA's change failure rate and recovery, linked to the last\ndeployment before them; pull requests linked to issues by a key in the title or branch (`WEB-12`, `feature/web-12-x`);\nand DevLake's `issues`, `issue_changelogs`, `sprints`, `sprint_issues` and `pull_request_issues` tables. The app's\nConnections page shows each connector's light, its last pull, any error in plain words, and a warning 14 days before\n`key_expires`; a refused key or an answer that isn't JSON never loses what was pulled before or stops the other\nmodules. A Jira issue in a done status without a resolution date counts as done from its last move into that status.\n\n**Write-back (opt-in, off by default):** with `write_back = true` under `[connectors.jira]` or `[connectors.linear]`,\neach run comments on the issue a newly merged pull request names (\"Pull request o/r#12 merged on 2026-10-21\" with its\nlink), and moves it through `done_transition` (Jira) or to `done_state` (Linear) when one is set. Only merges after\nwrite-back was last turned on count, so switching it on (or back on) never floods old issues; reverts never count. Each\npull request and issue is written at most once, at most 10 a run, and only after the run has saved its history, so a\nretried run never writes twice. The key then needs permission to comment and move issues; the job summary says what\nwas written, a refused comment stops that run's writing (those aren't retried), and a refused move is said while the\ncomments go on. Reading never stops. Tested against recorded API answers, not a live Jira or Linear.\n\n**Filing a flaky test in Linear:** with `[connectors.linear] teams` set, each flaky-test issue StackVitals opens has a\n\"File it in Linear\" link that opens a new Linear issue prefilled with the test and its evidence (no write key needed).\nJira's prefilled create link needs a numeric project id, so Jira teams link the GitHub issue with Jira's own GitHub\nintegration instead.\n\n## Community channels: Discourse, Slack and Discord\n\nQuestions asked outside GitHub count too. Each channel keeps, per question, only when it was asked, when anyone first\nreplied and when the team did, and (on Discourse) whether it was solved: no text, usernames or user ids. The Community\npage shows the team's median first answer with questions still waiting counted, beside issues and Discussions.\n\n| | Discourse | Slack | Discord |\n|---|---|---|---|\n| Key | none: its public JSON | a bot token with `channels:history`, invited to each channel: secret `SLACK_BOT_TOKEN`, input `slack-token` | a bot with View Channel and Read Message History (message content isn't needed): secret `DISCORD_BOT_TOKEN`, input `discord-token` |\n| Settings | `[connectors.discourse]` `url` (https), `team` (usernames), `groups` (group names) | `[connectors.slack]` `channels` (ids like C0123), `team` (user ids) | `[connectors.discord]` `channels`, `team` (ids) |\n| A question | a topic opened by someone outside the team | a top-level message; replies are its thread | a message that isn't a reply; replies point at it |\n| The team | forum staff, plus `team` and `groups` | the `team` user ids | the `team` user ids |\n\nName the team: on five public forums, staff first-answered from 1% to 13% of last month's topics while someone answered\nfrom 23% to 83% ([bench/accuracy/community](../bench/accuracy/community/README.md)), so staff alone undercount the\npeople who answer for a project.\nA run reads at most 120 forum topics and 150 Slack threads; the next run carries on. Slack and Discord readers\nsigned in and read a channel live; reading whole threads is tested on recorded answers.\n\n## Incidents: PagerDuty and incident.io\n\nWith either key, DORA's change failure rate and time to restore use that tool's incidents instead of incident labels,\nreverts and hotfixes (which would count the same outage twice), and every page names where incidents came from. Each\nincident is linked to the last production deployment before it began, as DevLake does. Until `services` (PagerDuty) or\n`incident_types` (incident.io) is set, every incident in the account counts for this repo, and the Connections page\nsays so.\n\n| | PagerDuty | incident.io |\n|---|---|---|\n| Key | a read-only REST API key: secret `PAGERDUTY_API_KEY`, input `pagerduty-token` | an API key that can view incidents: secret `INCIDENT_IO_API_KEY`, input `incident-io-token` |\n| Settings | `[connectors.pagerduty]` `services` (empty: all), `urgencies` (default `[\"high\"]`), `key_expires` | `[connectors.incident_io]` `incident_types` (names; empty: all), `key_expires` |\n| Read | List incidents (v2) by when they began, the last two weeks again on each pull for late resolutions | List incidents (v2) by last update; declined, canceled, merged, test and tutorial incidents never count; resolved time from \"Resolved at\" |\n\n## Code quality: SonarQube or SonarCloud\n\nSet `project` under `[connectors.sonar]` (the project key, like `myorg_myrepo`) and, for your own SonarQube, `url`.\nOnce a day the Action reads the project's quality gate and its measures (bugs, vulnerabilities, code smells, security\nhotspots, coverage, duplication, size and the three ratings) and keeps each day's reading; Delivery's Code quality tab\nshows the gate, each measure with how it moved over the month, and coverage by day, beside the delivery numbers.\nSonarCloud's public projects need no key; a private project or a SonarQube server needs a token in the `SONAR_TOKEN`\nsecret passed as `sonar-token`.\n\n## AI spend\n\nWhat the organisation pays for AI tools, set beside the agents' and AI-marked pull requests merged in the same days\n(Delivery, AI-assisted work). Any one key is enough; each is read by your own workflow run and goes only to its\nown provider:\n\n| Provider | Key (repository secret) | Workflow line | What is read |\n|---|---|---|---|\n| Anthropic | an Admin API key (`sk-ant-admin...`): `ANTHROPIC_ADMIN_KEY` | `anthropic-admin-key: ${{ secrets.ANTHROPIC_ADMIN_KEY }}` | the cost report, per day (`/v1/organizations/cost_report`), and Claude Code's daily analytics (`/v1/organizations/usage_report/claude_code`: sessions, lines, commits, PRs, edits accepted) |\n| OpenAI | an admin key: `OPENAI_ADMIN_KEY` | `openai-admin-key: ${{ secrets.OPENAI_ADMIN_KEY }}` | the costs, per day (`/v1/organization/costs`) |\n| Cursor | a team admin API key: `CURSOR_ADMIN_KEY` | `cursor-admin-key: ${{ secrets.CURSOR_ADMIN_KEY }}` | the billing cycle's spend (`/teams/spend`) and daily agent requests and accepted lines (`/teams/daily-usage-data`) |\n\nThe first pull reads 90 days; later pulls read from the last one, two days back, since cost reports fill in late.\nAmounts are kept in dollars to the cent, day by day, for two years, in `connectors/ai_spend/`. The page shows the last\n30 days per provider (Cursor's current billing cycle, which can't be split by day), the total, and the total over\nthe AI-involved pull requests merged in those days. That last number is an association: the keys cover all of an\norganisation's API use, not only coding, and an unmarked pull request can still have had AI help.\nWith the Anthropic key, Claude Code's analytics and Cursor's daily use sit side by side in one table (lines added,\npull requests opened, edits accepted, sessions or agent requests). Claude Code's own cost estimate is already in\nAnthropic's cost report, so it is never added to the spend twice. Its analytics cover Claude Code signed in to your\nConsole organisation, not Claude subscription plans.\n\n## GitHub Copilot usage reports\n\nGitHub's report API (it replaced the old metrics API in April 2026) hands out a daily report per organization and per\nrepository, kept by GitHub for a limited time. StackVitals copies each day into `connectors/copilot/`, so the history\nlasts as long as your data branch: the org's active users, suggestions accepted, and pull requests by Copilot, org-wide\nand for this repo. The Delivery page's AI-assisted tab shows the last 28 days beside what happened after AI-assisted\npull requests merged. The reports need \"Organization Copilot metrics: read\", which the workflow's own token can't have:\nan org owner registers a private GitHub App with that permission (no webhook), installs it on the org, and the workflow\nmakes a token with `actions/create-github-app-token` and passes it as `copilot-token`. `[connectors.copilot] org` picks\nanother org than the repo's owner. A first pull reads the last 60 days; later pulls ask for each new day two days after it\nends, and ask again on later runs until GitHub has it.\n\n## Chat: Slack, Microsoft Teams, Discord\n\nThe webhook in `webhook` gets the newly flaky tests, delivery alerts and the weekly digest, each in the chat's own\nshape: Slack Block Kit with a header and link buttons (and Slack's own markdown), a Teams Adaptive Card with open-link\nactions (for Teams Workflows, which replaced the old Office 365 connectors in 2026), a Discord embed, or plain JSON with\nthe numbers for anything else. With `team-webhooks` (a secret with one `@org/team = https://...` line per CODEOWNERS\nteam), a team's newly flaky tests go to its own chat and the rest to `webhook`. Buttons only open links (the app, the\nalerts issue, the workflow runs), so nothing needs a server to answer clicks.\n\n## Other CI systems\n\n`stackvitals ci upload` sends test reports (JUnit, CTRF, Jest or Vitest JSON, `go test -json`, TRX) from any CI (GitLab\nCI, Jenkins, CircleCI, Buildkite, ...) into the\nsame branch as one run, so flaky tests, failure groups and monitors work there too. Failure fingerprints strip each of\nthose systems' checkout folders, so the same failure groups together wherever it ran.\n\n## Deployments and incidents from elsewhere\n\n- Deployments: GitHub deployments to a production environment are read first (Vercel and many hosting services create\n  them), then deploy workflows, then releases. If your deploys happen in another repo or a tool GitHub can't see,\n  set `source` or `deploy_workflow` under `[delivery]` in `.github/stackvitals.toml`.\n- Incidents: issues with an incident label (configurable), revert pull requests and hotfix pull requests.\n- **From any tool (Argo CD, Jenkins, Spinnaker, a script, Opsgenie or Rootly webhooks):** send each deployment or\n  incident to GitHub as a `repository_dispatch` event. The StackVitals workflow keeps it on the data branch\n  (`delivery/recorded.ndjson`) and the next scheduled run counts it like GitHub's own: a production deployment joins\n  the deployments, and recorded incidents are used alone, as a pager tool's are. Sending the same `id` again updates\n  the record (an incident's `resolved_at`, a deployment's status). The token needs write access to the repo's\n  contents, GitHub's rule for dispatch events; no server of ours is involved.\n\n  ```bash\n  stackvitals record deployment owner/repo --id argo-1287 --sha \"$GIT_SHA\" --environment production --tool argo-cd\n  stackvitals record incident owner/repo --id INC-42 --at 2026-10-29T09:12:00Z --resolved-at 2026-10-29T10:03:00Z\n  # or without StackVitals installed:\n  curl -X POST -H \"Authorization: Bearer $TOKEN\" https://api.github.com/repos/owner/repo/dispatches     -d '{\"event_type\":\"stackvitals-deployment\",\"client_payload\":{\"id\":\"argo-1287\",\"sha\":\"'\"$GIT_SHA\"'\",\"finished_at\":\"2026-10-29T09:00:00Z\"}}'\n  ```\n\n  Fields: deployments take `id`, `finished_at`, and optionally `sha`, `environment` (default production), `status`\n  (success or failure), `started_at` and `tool`; incidents take `id`, `created_at`, and optionally `resolved_at` and\n  `tool`. A refused event says what to fix in the run's summary. Workflows set up before this need the\n  `repository_dispatch` trigger added (the current template has it).\n\n  Ready recipes (each sends the deployment when it has finished; `$TOKEN` is a token with write access to the repo's\n  contents, kept in the tool's secret store):\n\n  - **Argo CD** (its notifications engine, in `argocd-notifications-cm`; subscribe the app to `on-deployed`):\n\n    ```yaml\n    service.webhook.stackvitals: |\n      url: https://api.github.com/repos/owner/repo/dispatches\n      headers:\n      - name: Authorization\n        value: Bearer $stackvitals-token\n    template.stackvitals-deployed: |\n      webhook:\n        stackvitals:\n          method: POST\n          body: |\n            {\"event_type\":\"stackvitals-deployment\",\"client_payload\":{\"id\":\"{{.app.metadata.name}}-{{.app.status.operationState.syncResult.revision}}\",\"sha\":\"{{.app.status.operationState.syncResult.revision}}\",\"finished_at\":\"{{.app.status.operationState.finishedAt}}\",\"environment\":\"production\",\"tool\":\"argo-cd\"}}\n    ```\n\n  - **Jenkins** (a `post { success { ... } }` step): `sh 'stackvitals record deployment owner/repo --id \"jenkins-$BUILD_TAG\" --sha \"$GIT_COMMIT\" --tool jenkins'`\n  - **GitLab CI** (a job after the deploy job): `stackvitals record deployment owner/repo --id \"gitlab-$CI_PIPELINE_ID\" --sha \"$CI_COMMIT_SHA\" --environment \"$CI_ENVIRONMENT_NAME\" --tool gitlab`\n  - **Spinnaker, Octopus, a script:** the `curl` line above, with the pipeline's id and commit.\n\n## Bitbucket and Azure DevOps\n\n**Delivery, read-only:** `stackvitals delivery workspace/repo --bitbucket` and `stackvitals delivery org/project/repo\n--azure` read the window's merged pull requests with their activity (the first answer from someone else, approvals) and\nrun the same Delivery build as for GitHub: review phases, and DORA from Bitbucket Pipelines' deployments to production.\nAzure DevOps keeps deployments in release pipelines, outside its Git API: send them with `stackvitals record`, or count\nmerges (`source = \"merges\"`). Public projects need no token; set `BITBUCKET_TOKEN` (an access token) or\n`AZURE_DEVOPS_TOKEN` (a PAT with Code read) for private ones. Bots (Renovate, Dependabot) are left out of people's\nnumbers as on GitHub. Checked live on atlassian/atlassian-connect-express (Bitbucket) and powershell/PowerShell (Azure\nDevOps).\n\n## GitLab\n\n**CI, read-only (built):** `stackvitals ci gitlab group/project` reads a GitLab project's pipelines and jobs (retries\nincluded) and reports them like GitHub Actions: flaky jobs, slow jobs, re-run share. A retried job becomes a re-run\nattempt, merge request pipelines count as pull requests, and a job allowed to fail never fails its pipeline. Public\nprojects need no token; set `GITLAB_TOKEN` for private ones, and `--base` for a self-managed GitLab. Checked on\ngitlab-org/cli (78 pipelines in 7 days read with 80 requests).\n\n**Delivery, read-only (built):** `stackvitals delivery group/project --gitlab` reads the window's merged merge\nrequests with their notes (the first answer from someone else, approvals, draft and ready) and the project's\ndeployments, and runs the same Delivery build as for GitHub: review phases, DORA from successful production\ndeployments, deploy attempts with failed ones. gitlab.com shows notes only with a token, even on public projects;\nwithout `GITLAB_TOKEN` the phases end at the merge. Checked on gitlab-org/cli: 153 merge requests in 30 days.\n\n**The rest (later):** GitLab issues with an incident label as incidents, and the same inside the GitLab CI template.\n\n**CI inside GitLab (built, preview):** include `templates/gitlab/stackvitals.yml` in `.gitlab-ci.yml` and add a masked\nCI/CD variable `GITLAB_TOKEN` (a project access token with `api` and `write_repository`). Its job runs in the last stage\n(`.post`) of merge request, default-branch and scheduled pipelines, and of branch pipelines with an open merge request\n(so the note works with merge request pipelines or plain branch pipelines): it reads the finished pipelines, keeps them\nin the project's `stackvitals-data` branch, writes CI Health into `public/` for GitLab Pages, and when the merge\nrequest's pipeline failed it leaves one note (edited on later pipelines, and to \"CI is passing now\" once green) saying\nwhich failures are known flakes or also fail on the default branch. If the push to `stackvitals-data` is refused (a\nprotected branch, a push rule, or a token without `write_repository`), the job fails and says so. GitLab has no \"after\nthe pipeline\" trigger, so the job reads its own pipeline while it runs; there is no verdict check to block a merge. It\nis tested against a stand-in GitLab API, not a live project yet. The include works once the `v0` tag is made at launch;\nuntil then use `main` in its URL and set the `STACKVITALS_REF` variable to `main`.\n\nThe merge request, deployment and incident records above come next, in the same job.\n","triage":"When StackVitals' history and the log's signs can't decide whether a pull request's failure came from the change, a model can read the failing step's error lines and the changed file names and give its view. The view is shown beside the verdict in the pull request comment, with its track record; it never changes the verdict or the StackVitals verdict check. At most 3 failures a run are sent, across all pull requests, and a failure the model has already judged is not sent again (its view is kept in the StackVitals comment).\n\nEach provider needs two things: its section in `.github/stackvitals.toml`, and its key as a repository secret passed to the Action as `triage-key`. A private repo's logs are sent only with `allow_private = true`.\n\n## Gemini\n\nFree tier with a Google AI Studio key, no card.\n\n```toml\n[ci.triage]\nprovider = \"gemini\"\n# model = \"gemini-3.5-flash-lite\"   # the default\n```\n\n```yaml\n        with:\n          triage-key: ${{ secrets.GEMINI_API_KEY }}\n```\n\n## OpenAI\n\nPaid; any OpenAI-compatible endpoint by setting url.\n\n```toml\n[ci.triage]\nprovider = \"openai\"\n# model = \"gpt-6-luna\"   # the default\n```\n\n```yaml\n        with:\n          triage-key: ${{ secrets.OPENAI_API_KEY }}\n```\n\n## Anthropic\n\nPaid.\n\n```toml\n[ci.triage]\nprovider = \"anthropic\"\n# model = \"claude-haiku-5-5\"   # the default\n```\n\n```yaml\n        with:\n          triage-key: ${{ secrets.ANTHROPIC_API_KEY }}\n```\n\n## Mistral\n\nFree experiment tier.\n\n```toml\n[ci.triage]\nprovider = \"mistral\"\n# model = \"mistral-small-latest\"   # the default\n```\n\n```yaml\n        with:\n          triage-key: ${{ secrets.MISTRAL_API_KEY }}\n```\n\n## Groq\n\nFree tier, open models.\n\n```toml\n[ci.triage]\nprovider = \"groq\"\n# model = \"openai/gpt-oss-20b\"   # the default\n```\n\n```yaml\n        with:\n          triage-key: ${{ secrets.GROQ_API_KEY }}\n```\n\n## Any other OpenAI-compatible endpoint\n\nName any provider above and point `url` at the other endpoint (https), with its model:\n\n```toml\n[ci.triage]\nprovider = \"openai\"\nurl = \"https://your-endpoint.example.com/v1\"\nmodel = \"your-model\"\n```\n"}

stackvitals/web/app/help/getting-started.json

@@ -0,0 +1 @@
+{"app":"Every run publishes one app over all the modules that ran: `https://OWNER.github.io/REPO/` once GitHub Pages serves the\n`stackvitals-data` branch. For a private repo, `stackvitals ui owner/repo` opens the same app on `127.0.0.1` from a copy\nof that branch. The app is `docs/index.html` plus `docs/app/` in the branch: Preact and TypeScript bundled ahead of\ntime, with its font, the docs for Help and the demo data inside. It loads no code from other sites (apart from the SQL\nexplorer, which fetches DuckDB once) and has no tracking. The bundled libraries and their licences are listed in\n`app/LICENSES.md`.\n\n## Opening it\n\n| Address | What opens |\n|---|---|\n| `index.html` | The repo whose data sits beside the page (the usual Pages site) |\n| `index.html?demo=1` | The demo: three example repos with full data, nothing to connect |\n| `index.html?repo=owner/name` | A repo's `stackvitals-data` branch, read from GitHub |\n| `index.html?pages=https://owner.github.io/name/` | Another repo's Pages site |\n| `index.html?welcome` | The Welcome screen |\n\nLinks from the earlier app still work: `#/home` opens Overview and `#/integrations` opens Connections.\n\n## Layout\n\nThe sidebar groups the pages: **Health** (Overview, Insights, CI Health, Pull Requests, Delivery, Community,\nDependencies), **Organisation** (Investment, Portal, Explore), **Team** (Activity, Settings) and **Install** (Setup,\nConnections, Agents, Help, About). A badge shows what waits on a page, for example flaky tests or newcomers waiting. A\npage for a module that isn't set up is dimmed. On a phone the sidebar becomes a tab bar (Overview, Insights, CI, PRs)\nwith the rest under **More**.\n\nThe top bar holds the repo switcher: the repos you follow, **Add a repository** (by name or Pages address), your GitHub\nrepos, the demo, reloading the data, and appearance (light, dark or auto). Click the logo to pick one of ten marks; the\nchoice changes the sidebar, tab icon and splash and is remembered in this browser.\n\n## First run\n\nWhere no repo's data sits beside the page (the project site, or a fresh install), the app opens on **Welcome** with three\nchoices:\n\n1. **Connect GitHub.** Lists your repos: ones that run StackVitals open their dashboard, the others go to Setup.\n2. **Try the demo.** Three example repos with full data.\n3. **Open a public repo.** Type `owner/repo`; the app reads its published data from its Pages site or data branch, or\n   offers setup when it has none.\n\nA returning visitor with a remembered connection goes straight to the last repo.\n\n## Connecting GitHub\n\n- **With a token.** The app opens GitHub's token page with the name, a 90-day expiry and the permissions filled in; you\n  pick the repos and paste the token back. It is checked at once, and the app lists what it can and can't do in plain\n  words. The token goes only to `api.github.com`. It stays in this tab by default; switch on **Remember on this device**\n  to keep it across visits. **Sign out** forgets it.\n- **Sign in with GitHub.** Shown only when the site was built with a sign-in relay (`worker/relay`). The app uses\n  GitHub's sign-in with PKCE; the relay only swaps the one-time code (and later a refresh) for a token and keeps\n  nothing. A signed-in token lasts 8 hours and is renewed shortly before.\n\n## Setup in a few clicks\n\nOnce GitHub is connected, setup takes a few clicks; connecting takes one token, made from a prefilled GitHub page.\n**Setup** picks a repo, checks what it has and suggests a plan (modules ticked from what it finds) that you can change.\n**Set it up** then commits the workflow (a pull request when the branch is protected), starts the first run and shows it\nlive, waits for the data branch and turns on Pages for public repos. Each step is saved, so a reload picks up where it\nstopped. Without a token you get the file ready to add on GitHub, the YAML to copy, or the CLI command, and the app\nchecks for the first run at a pace that stays inside GitHub's limit for visitors.\n\n## Pages\n\n| Page | What it shows | What you can do there |\n|---|---|---|\n| Overview | Installation health, the state of each area, the trend you pick, what needs a decision, what to fix first and decisions ending soon | Open any item |\n| Insights | Every finding ranked by the hours at stake, with its sample, evidence, age and decision; flaky tests traced to the pull requests they held red | Snooze or acknowledge; query it in Explore |\n| CI Health | The first-try green share with the pushes that missed it, flaky and failing tests and jobs by cost, failure groups (each with a prompt to copy into an AI agent), slowest tests and jobs, new and skipped tests, coverage; a page per item with its flake rate, the runs that show it and the decision in force | Quarantine (tests), mute, acknowledge, reopen |\n| Pull Requests | Who holds the next move on each open pull request, stuck ones, working-agreement exceptions | Filter to your handle; leave a pull request out of the numbers |\n| Delivery | DORA tiles with tiers (each opens into the deployments, pull requests and incidents it was counted from), deployment rework rate, the phases of a change and their trend, AI-assisted work, issues and sprints with Jira or Linear, the merge queue | Open the pull requests behind each phase and each DORA number |\n| Community | Newcomers waiting for an answer, bus factor (no names), first answers for newcomers against regulars, stars and traffic | |\n| Dependencies | Vulnerabilities and malware in every lockfile the scanner reads, runtimes out of support, update pull requests by age, the supply chain from the SBOM | Accept a risk or a licence until a date |\n| Investment | Where merged work went each month (feature, fix, maintenance, security, docs) and how each pull request was sorted | |\n| Portal | (portal repos) every repo with its level, owners and links; failing rules; the catalog; campaigns; golden paths | Open a repo's own app |\n| Explore | The tables behind every page, one row per pull request, job, test or deployment | Download any table; open the SQL explorer (`explore.html`, DuckDB in your browser) |\n| Activity | Every decision and settings change: who, when, why, until when | Undo in one click |\n| Data health | Each source (every step of the run, every connected service): current, filling in, failing since when and why, when it last worked, what it read; how far back each history goes against how long it is kept, and what retention or the size cap last removed | |\n| Settings | Every setting in groups, with its default, the value in force and what a change would do | Change settings and send them |\n| Setup | Setup in a few clicks (above) | |\n| Connections | Everything StackVitals reads or sends (GitHub, Jira, Linear, chat webhooks, other CI systems, DevLake), grouped as connected, needs attention or available | Save a secret, change its settings, test it |\n| Agents | Connecting a coding agent to the MCP server, one config per client; the tools by module, the skills | Copy the config or the install command |\n| Help | The StackVitals docs, with search; links between docs stay in the app | |\n| About | The path from GitHub to this page, where the data lives, how numbers are counted, how accurate each rule is, what each permission is for, how to remove it all | |\n\n## Where a number comes from\n\nThe headline numbers open into what they were counted from: deployment frequency into the deployments, lead time into each deployed pull request with its time (longest first), change fail rate into the deployments and the revert, hotfix or incident issue tied to each, recovery into each failed deployment and its time to resolve, every review phase into its pull requests, and the first-try green share into the pushes that missed it, each linked to GitHub. The same lists are in `delivery.json` and `data.json` (`evidence`, `not_first_try`), and the MCP tools return them, so an agent can answer \"why is lead time 26 hours?\" with the pull requests. Long lists keep the longest or newest first and say the total. From a terminal: `stackvitals explain owner/repo lead-time` (or `deployments`, `change-failure`, `recovery`, `first-try`, or a review phase like `pickup`) prints the number, what it means and each item with its GitHub link; `--json` for all of them.\n\n## Data health\n\nEvery run writes `health.json` beside the other files: for each step of the run and each connected service, when it\nlast worked, whether it is failing and since when (with what it said), whether it has finished reading its history\n(**Filling in** while a big history is read a few hundred items a run, within GitHub's and the service's limits), and\nwhat it read. It also lists how far back each kind of history goes against how long it is kept, and what retention or\nthe size cap last removed, so a gap in a chart has a reason. A failing source keeps its last good data and the next\nrun tries again; a source with no run for two days may mean the schedule stopped (GitHub pauses schedules in repos\nwith no activity for 60 days). Times and counts refresh with the next run that changes something, so a quiet run\nnever commits just to move a clock.\n\n## Connections\n\nEach connection opens a sheet with what it unlocks, the steps, its secrets and its settings. A secret you type is sealed\nin your browser with the repo's own public key and only the sealed value goes to GitHub, the same way GitHub's own\nsettings page saves it; the app can list secret names but never read a value back. **Test connection** starts the\nworkflow's check and reads the result back from `connectors.json`. For Linear the app can also test the key straight\nfrom the page; the key then goes only to `api.linear.app`.\n\n## Making a decision\n\n\"Decide\" on a test, job, finding or pull request asks what to do, for how long and why. Quarantine and mute last 30\ndays, snooze 14, an accepted risk 30, unless you pick another length or a date. Then pick how it reaches the repo:\n\n1. **Open the request on GitHub** (the default). A prefilled issue opens; press Submit. The workflow checks you have\n   triage access or more, records the decision, answers and closes the issue. No token is involved.\n2. **Apply now** with your connected GitHub account or a pasted token with **Actions: read and write**: the app starts the\n   workflow with the request and shows the run.\n3. **Copy as a comment.** The command to paste in any issue or pull request comment, for example\n   `/stackvitals mute test:<id> for 7d because <reason>`.\n\nDecisions land in `decisions/log.ndjson` on the data branch and show in Activity. Issues from people without access\nstart no run.\n\n## Changing settings\n\nEdits collect in a save bar. **Review** lists each change (old to new); for alert thresholds it replays the last 90 days\nand says how often the new value would have fired. With GitHub connected, one click commits `.github/stackvitals.toml`\n(a pull request when the branch is protected) or runs the workflow with the request; workflow inputs are edited in the\nworkflow file the same way. Without a connection, open a prefilled issue or copy the TOML. Settings need write access.\n\n## On this device\n\nThe app keeps your repo list, theme, logo and GitHub handle in this browser only. A token stays in this tab unless you\nask to remember it on this device.\n\n## For agents\n\n`stackvitals propose owner/repo --set KEY=VALUE --decide \"/stackvitals ...\"` (MCP: `propose_change`) prints the same\nprefilled request link, checked with the workflow's own parsers. A person opens it and presses Submit.\n","faq":"## Is it really free? What's the catch?\n\nIt is licensed under Apache 2.0 and runs in your own GitHub Actions minutes, which are free for public repos and included in every\nplan for private ones. There is no paid tier, no account and no StackVitals-hosted backend.\nOptional integrations (a tracker, chat, a model for triage, storage outside git) use services and keys you choose, under\ntheir own terms ([governance](../GOVERNANCE.md)).\n\n## Does my code or data leave GitHub?\n\nHistory is files on a branch of your repo, and your code is never sent anywhere. By default the dependency check\nsends your dependencies' package names to deps.dev, OpenSSF Scorecard and the registries' download counts; set\n`[deps] outside_lookups = false` to keep them in GitHub. Explore and the printable pages load their libraries from\npublic CDNs. Anything else (your Jira, your chat) needs a key you add. Each host and what it gets is listed on\n[privacy](privacy.md).\n\n## How many Actions minutes does it use?\n\nA run takes about a minute on a small repo. It starts after your CI workflows and every 3 hours, so a busy repo runs\nit often; on private repos, check that against your plan's minutes. For the busiest public repos (2,000 CI runs a\nday), CI Health alone is estimated at under 40% of the built-in token's API limit ([scaling](scaling.md)).\n\n## How is this different from asking an AI agent to look at my CI?\n\nAn agent with GitHub access sees what GitHub still has: 90 days of logs and whatever it can read in one chat.\nStackVitals keeps the history after GitHub deletes it, has already counted how often each test flaked, and compares\nyour repo with similar public ones from a daily crawl. On a single fresh failure a good agent does as well alone; we\nmeasured that and say so ([agents card](../bench/accuracy/agents/README.md)). The two work together: the MCP server\ngives agents StackVitals' history.\n\n## How is it different from LinearB, Swarmia, Trunk or DevLake?\n\nThey are good tools. StackVitals differs in three ways: it is free and has no hosted backend; the rules behind its verdicts\nand headline numbers are checked on public data with the losses published ([accuracy](accuracy.md)); and it compares like with like (the same repo,\nmonth and size) before saying something changed. Where we measured their default rules on the same data, the\nresults are on the cards, including where theirs tie ours.\n\n## Can it rate developers?\n\nNo. People are named only to route work (an owner, who acts next on a pull request). The [DORA guide](dora-guide.md)\nexplains why team numbers must never be used for individual reviews.\n\n## Why does it say \"not enough data\" so often?\n\nBecause a number from three pull requests misleads. Each figure shows its sample size, and below a floor it says so\ninstead of guessing. Small repos borrow strength from similar repos where that has been checked to help.\n\n## A verdict was wrong. What do I do?\n\nSay so with `/stackvitals dispute test:<id> because <why>` (or `confirm` when it was right). It is recorded with who\nand why, and CI Health shows how often your team confirmed each kind of verdict. If the run is public, an issue with\nits link helps the next accuracy check.\n\n## Does it work with GitLab, Jenkins or Buildkite?\n\nGitLab CI: a job you include in `.gitlab-ci.yml` keeps the pipeline history, publishes CI Health on GitLab Pages and\nnotes failed merge requests (preview; [connectors](connectors.md#gitlab)). Test reports from Jenkins, CircleCI and\nBuildkite can be uploaded with `stackvitals ci upload`. Everything else reads GitHub ([connectors](connectors.md), [limitations](limitations.md)).\n\n## Does it work on GitHub Enterprise Server?\n\nThe Action reads the API and GraphQL addresses GitHub Actions gives every job (`GITHUB_API_URL`,\n`GITHUB_GRAPHQL_URL`), so on GitHub Enterprise Server it talks to your server, not github.com. Two limits: links in\nthe app point to github.com pages, and the daily crawl's peer comparisons cover public github.com repos only. The\ncommand line takes the same two variables when you run it against your server.\n\n## Is there a login, with roles for who sees what?\n\nYes, and it is GitHub's own. The app signs in with GitHub and reads each repo's data branch with your access, so a\nperson sees exactly the repos GitHub already lets them read, and an org's roles (members, outside collaborators, teams)\napply unchanged; Portal's org view shows only the repos the viewer can open. Nothing is copied to a server of ours to\nguard, so there is no second user list to keep in step. What you don't get is a separate admin screen for roles: you\nmanage them where you already do, in GitHub.\n\n## Our repos are private. Can we still see the dashboard?\n\nYes, three ways. Run `stackvitals ui OWNER/REPO` to open it on your machine. Set `pages: artifact`: each run that\nchanged the app uploads it as a download kept 7 days; unzip it and run `python -m http.server 8000 --bind 127.0.0.1`\nin the folder (opening `index.html` directly doesn't work). Or use Pages if your plan keeps Pages private\n([enterprise](enterprise.md)).\n\n## What happens if a run fails halfway?\n\nNothing is half-written: each run commits once, and the next run catches up. The tested failure modes are listed in\n[limitations](limitations.md#failure-modes).\n","install":"StackVitals is a GitHub Action, `uses: stackvitals/stackvitals@v0`, run by one workflow file in your repo. The file below is exactly what `stackvitals ci init owner/repo` prints (that command also fills in your CI workflows' names) and what the app's Setup page commits for you. Save it as `.github/workflows/stackvitals.yml`; committing it starts the first run. No settings file is needed: a missing `.github/stackvitals.toml` means the defaults ([settings.md](settings.md)).\n\n## Modules\n\nPick modules with the `modules` input, comma separated. To run one alone, list only it, for example `modules: ci`. Delivery and Insights have no id of their own: they come with `flow`.\n\n| Id | What you get | Runs on | Needs |\n|---|---|---|---|\n| `ci` | CI Health: flaky and slowing jobs and tests, the PR verdict comment and check, test reports, owners | after each CI run finishes, and every 3 hours | nothing; alone is fine |\n| `flow` | Team Flow, and with it the Delivery page (cycle time, DORA, AI-assisted work, investment) and Insights (fix first, alerts, the weekly digest) | the schedule and manual runs | nothing; Delivery's CI parts (failed deployments from CI, merge-queue flakes) and Insights' red-CI findings also need `ci` |\n| `deps` | Dependency Steward: known vulnerabilities and malware in every lockfile, the check on PRs that change a lockfile, runtimes, and the weekly supply chain page | after each CI run (with `ci`) and every 3 hours for vulnerabilities; pull requests that change a lockfile; weekly for the supply chain | nothing; alone is fine |\n| `community` | Community: newcomers, first answers, who carries the work, stars and traffic | the schedule and manual runs | nothing; it reads the PRs itself when `flow` isn't on |\n| `portal` | Portal: the org's catalog, owners and scorecards | the schedule, from the org's portal repo with its own workflow (below) | a portal repo; an org App for private repos |\n\nThe default is `ci,flow,deps,community`. `stackvitals ci init --modules ci,deps` (or the app's Setup picker) writes a file with only those modules' triggers and permissions, each lowered to read where nothing of theirs writes; turning on a writing switch later (like `deps-comments`, which needs `pull-requests: write`) means raising that permission too.\n\n## The workflow file\n\nChange `workflows: [\"CI\"]` to the `name:` lines of your CI workflows (the names, not the file names): `workflow_run` matches names. The permissions are what every default switch needs; each comment says what to lower to turn a part off. Team Flow's automation (review requests, size labels) and PR comments need `pull-requests: write`; the weekly digest, commands and flaky-test issues need `issues: write`. A settings change applied from the app opens a pull request, which needs the repo setting \"Allow GitHub Actions to create and approve pull requests\" (Settings, Actions, General).\n\n```yaml\nname: StackVitals\non:\n  workflow_run:\n    workflows: [\"CI\"]   # your CI workflows; each finished run is read right away\n    types: [completed]\n  schedule:\n    - cron: \"17 */3 * * *\"   # Team Flow every 3 hours, and a CI catch-up\n  repository_dispatch:\n    types: [stackvitals-deployment, stackvitals-incident]   # sent by `stackvitals record` from any deploy tool\n  workflow_dispatch:\n    inputs:\n      request:\n        description: \"A change sent from the StackVitals app (leave empty for a normal run)\"\n        required: false\n        default: \"\"\n  push:                      # your settings file applies at once; adding this file starts the first run\n    paths: [\".github/stackvitals.toml\", \".github/workflows/stackvitals.yml\"]\n  issues:\n    types: [opened, closed, reopened]  # requests from the app; flaky-test issues closed as not planned, or reopened\n  issue_comment:\n    types: [created]         # /stackvitals commands\n  pull_request:              # the dependency check, only when a lockfile changes (never runs the PR's code)\n    paths: [\"**/*.lock\", \"**/*lock.json\", \"**/*lock.yaml\", \"**/npm-shrinkwrap.json\", \"**/requirements*.txt\", \"**/go.mod\",\n            \"**/*.lockfile\", \"**/pom.xml\", \"**/pylock*.toml\", \"**/Package.resolved\", \"**/gems.locked\",\n            \"**/verification-metadata.xml\", \"**/*.freeze\"]\npermissions:\n  contents: write        # saves history to the stackvitals-data branch; settings changes asked for from the app\n  actions: read          # reads your CI runs, jobs and test reports (write: [automation] rerun_flaky re-runs known flakes)\n  checks: read           # Team Flow: sees which PRs have failing checks (write: [automation] verdict_check)\n  statuses: read\n  deployments: read      # Team Flow: delivery from deployments\n  issues: write          # the weekly digest, answers to /stackvitals commands, flaky-test and alert issues\n  pull-requests: write   # reads PRs for Team Flow; one comment on failed PR runs (set to read to turn comments off)\njobs:\n  vitals:\n    # Only what concerns StackVitals starts a runner: settings pushed to the default branch, requests and commands from\n    # people with access to the repo, and flaky-test issues being closed or reopened.\n    if: >-\n      (github.event_name != 'push' || github.ref_name == github.event.repository.default_branch) &&\n      (github.event_name != 'issues' ||\n        (github.event.action == 'opened' && contains(github.event.issue.body, '```stackvitals') &&\n         contains(fromJSON('[\"OWNER\", \"MEMBER\", \"COLLABORATOR\"]'), github.event.issue.author_association)) ||\n        (github.event.action != 'opened' && contains(github.event.issue.labels.*.name, 'flaky-test'))) &&\n      (github.event_name != 'issue_comment' ||\n        (contains(github.event.comment.body, '/stackvitals') &&\n         contains(fromJSON('[\"OWNER\", \"MEMBER\", \"COLLABORATOR\"]'), github.event.comment.author_association))) &&\n      (github.event_name != 'workflow_run' || !github.event.repository.private ||\n        github.event.workflow_run.conclusion != 'success' || vars.STACKVITALS_EVERY_CI_RUN == 'true')\n    # Private repos pay for Actions minutes: a CI run that passed starts no job (the 3-hourly run reads it), unless the\n    # repo variable STACKVITALS_EVERY_CI_RUN is \"true\" (the verdict check needs it). On the job, so events it skips never\n    # wait in (or cancel) the queue; PR checks and Test connection runs never wait for (or get dropped behind) data runs.\n    concurrency:\n      group: stackvitals-${{ github.event.pull_request.number || (contains(github.event.inputs.request, '\"check\"') && format('check-{0}', github.run_id)) || 'data' }}\n      cancel-in-progress: false\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    steps:\n      - uses: stackvitals/stackvitals@v0\n        env:\n          STACKVITALS_EVERY_CI_RUN: ${{ vars.STACKVITALS_EVERY_CI_RUN }}\n        with:\n          modules: ci,flow,deps,community\n          jira-email: ${{ secrets.JIRA_EMAIL }}        # optional: Jira Cloud (see Integrations); unused while unset\n          jira-token: ${{ secrets.JIRA_API_TOKEN }}\n          linear-token: ${{ secrets.LINEAR_API_KEY }}  # optional: Linear\n          pagerduty-token: ${{ secrets.PAGERDUTY_API_KEY }}       # optional: PagerDuty or incident.io incidents\n          incident-io-token: ${{ secrets.INCIDENT_IO_API_KEY }}\n          triage-key: ${{ secrets.GEMINI_API_KEY }}     # optional: a model's view on undecided failures ([ci.triage])\n          # traffic-token: ${{ steps.app.outputs.token }}   # optional: keep traffic past GitHub's 14 days (see docs/modules.md)\n```\n\n## The portal workflow\n\nPortal runs in the org's own portal repo (private is best) with this file, which `stackvitals portal init` prints; `stackvitals portal check ORG` says what your token can read first.\n\n```yaml\nname: StackVitals Portal\non:\n  schedule:\n    - cron: \"41 5 * * *\"   # once a day\n  workflow_dispatch:\n    inputs:\n      request:\n        description: \"A request from the StackVitals app, like a golden path (leave empty for a normal run)\"\n        required: false\n        default: \"\"\n  issues:\n    types: [opened]          # golden path requests from the app, run at once\n  issue_comment:\n    types: [created]         # \"/stackvitals approve\" on a golden path that needs an approval\npermissions:\n  contents: write          # keeps the catalog, its history and the pages in this repo's stackvitals-data branch\n  issues: write            # one tracking issue per campaign, and answers to requests, in this repo only\nconcurrency:\n  group: stackvitals-portal\n  cancel-in-progress: false\njobs:\n  portal:\n    # Request issues start a run only when someone with access to this repo opened one with a StackVitals block.\n    if: >-\n      (github.event_name != 'issues' ||\n       (contains(github.event.issue.body, '```stackvitals') &&\n        contains(fromJSON('[\"OWNER\", \"MEMBER\", \"COLLABORATOR\"]'), github.event.issue.author_association))) &&\n      (github.event_name != 'issue_comment' ||\n       (startsWith(github.event.comment.body, '/stackvitals approve') &&\n        contains(fromJSON('[\"OWNER\", \"MEMBER\", \"COLLABORATOR\"]'), github.event.comment.author_association)))\n    runs-on: ubuntu-latest\n    timeout-minutes: 30\n    steps:\n      # Private repos: create an org-owned read-only GitHub App (see docs/modules.md), store its client ID and key, then switch on.\n      # - uses: actions/create-github-app-token@64ab729a6294000f5aa4a4bc0b73d1f32bc45510 # v3\n      #   id: app\n      #   with:\n      #     client-id: ${{ vars.PORTAL_CLIENT_ID }}\n      #     private-key: ${{ secrets.PORTAL_PRIVATE_KEY }}\n      #     owner: ${{ github.repository_owner }}\n      - uses: stackvitals/stackvitals@v0\n        with:\n          modules: portal\n          # portal-token: ${{ steps.app.outputs.token }}\n```\n\n## Every input\n\nFrom [action.yml](https://github.com/stackvitals/stackvitals/blob/main/action.yml). Inputs use hyphens (`rerun-flaky`); the same switches in `.github/stackvitals.toml` use underscores (`rerun_flaky`), and the file wins when both are set.\n\n| Input | Default | What it does |\n|---|---|---|\n| `modules` | `ci,flow,deps,community` | Modules to run, comma separated. Available now - ci (CI health), flow (Team Flow, on schedule and manual runs), deps (dependency check on pull requests that change a lockfile), community (newcomers, first answers, stars and traffic, on schedule and manual runs), portal (the org catalog, run from the org's private portal repo) |\n| `token` | `${{ github.token }}` | Token that can read Actions and write the stackvitals-data branch |\n| `portal-token` |  | Optional token from an org-owned read-only GitHub App, so Portal reads the org's private repos (without it, public repos only) |\n| `portal-org` |  | The org Portal catalogs (default - the owner of the repo it runs in) |\n| `traffic-token` |  | Optional token from your own GitHub App with Administration read, so Community keeps traffic past GitHub's 14 days (the built-in token can't read traffic) |\n| `days` | `90` | Days of history to fill in on the first runs (GitHub keeps at most 90) |\n| `budget` | `800` | Most API requests per run (the built-in token allows 1,000 an hour) |\n| `tests` | `true` | Read test reports (JUnit XML or CTRF JSON) from your run artifacts to find single flaky tests |\n| `test-artifacts` |  | Regex for the artifact names that hold test reports (empty = names with junit, test-results, ctrf...) |\n| `pr-comments` | `true` | One comment on failed PR runs saying which failures are known flakes (needs pull-requests write) |\n| `rerun-flaky` | `false` | Re-run failed jobs once when every failure is a known flake (needs actions write) |\n| `quarantine` | `false` | Keep a list of flaky tests for the stackvitals ci quarantine command |\n| `preflight-comments` | `false` | Team Flow - one note on new outside PRs that miss what merged PRs here usually include (needs pull-requests write) |\n| `deps-comments` | `false` | Dependency check - one comment on same-repo PRs listing changed packages and their verdicts (needs pull-requests write) |\n| `issues` | `false` | Keep one issue per flaky test or job, closed once fixed (needs issues write) |\n| `mention-owners` | `false` | @-mention a flaky test's code owners in the issues StackVitals opens (off: owners are named without a notification) |\n| `webhook` |  | Slack, Discord, Microsoft Teams (Workflows) or any JSON webhook URL (pass a secret) to hear when something newly turns flaky and when a delivery number crosses its threshold |\n| `team-webhooks` |  | One \"@org/team = https://...\" line per team (pass a secret); a team's newly flaky tests go to its own chat, the rest to webhook |\n| `jira-email` |  | The Jira Cloud account's email (pass a secret); with jira-token and [connectors.jira] url, issues, sprints and incidents are read |\n| `jira-token` |  | A Jira Cloud API token for that account (pass a secret; it expires within a year, so set key_expires to be warned) |\n| `linear-token` |  | A Linear personal API key (pass a secret); issues and cycles of the teams under [connectors.linear] are read |\n| `pagerduty-token` |  | A PagerDuty REST API key, read-only (pass a secret); its incidents then count in DORA instead of labels, reverts and hotfixes |\n| `incident-io-token` |  | An incident.io API key with read access (pass a secret); its incidents then count in DORA instead of labels, reverts and hotfixes |\n| `sonar-token` |  | A SonarQube or SonarCloud token (pass a secret), only for a private project or your own server; public SonarCloud projects need none |\n| `copilot-token` |  | A token that can read the org's Copilot usage reports (an org App with \"Organization Copilot metrics\" read, or a fine-grained token); kept day by day |\n| `anthropic-admin-key` |  | An Anthropic Admin API key (sk-ant-admin..., pass a secret); the organisation's daily Claude API cost is set beside AI-involved pull requests |\n| `openai-admin-key` |  | An OpenAI admin key (pass a secret); the organisation's daily API cost is set beside AI-involved pull requests |\n| `cursor-admin-key` |  | A Cursor admin API key (pass a secret); the team's billing-cycle spend and daily agent use are set beside AI-involved pull requests |\n| `slack-token` |  | A Slack bot token (channels:history) that reads the support channels under [connectors.slack]; only times and flags are kept |\n| `discord-token` |  | A Discord bot token (View Channel, Read Message History) that reads the support channels under [connectors.discord]; only times and flags are kept |\n| `imports-json` |  | Path to the list of packages your code imports (`stackvitals deps imports --out imports.json` in an earlier step, after checkout) for Python, JavaScript and Rust repos; known vulnerabilities then say whether your code imports the package (an import, not a call path) |\n| `govulncheck-json` |  | Path to govulncheck's JSON output (`govulncheck -format json ./... > govulncheck.json` in an earlier step) for Go repos; the supply page then says which advisories the code really calls |\n| `triage-key` |  | A model API key (pass a secret, e.g. a free Gemini key from Google AI Studio) for [ci.triage]; up to 3 undecided failures per run get a model's view beside the verdict |\n| `label-stuck` | `false` | Label pull requests whose asked-for review has not started after 3 days (\"stackvitals: waiting for review\"), and take the label off once one has (needs pull-requests or issues write) |\n| `retention-days` | `400` | Days of pull request, issue and Portal history kept in the data branch |\n| `ci-retention-days` | `180` | Days of CI run and test history kept in the data branch |\n| `size-cap-mb` | `250` | Most history kept in the data branch, in MB; the oldest days go first |\n| `squash-history` | `false` | Every 1,000 commits, rewrite the data branch to one commit (files unchanged; the branch is force-pushed) |\n| `pages` | `branch` | \"branch\" (default): GitHub Pages serves the stackvitals-data branch. \"deploy\": this job publishes the monitor itself (Pages source \"GitHub Actions\"; the job needs pages and id-token write and the github-pages environment), for busy repos past 10 Pages builds an hour. \"artifact\": no Pages; each run that changed the app uploads it as a workflow artifact (kept 7 days, or less if your org sets a shorter limit) to download and open locally, for private repos; anyone who can read the repo can download it |\n| `storage` | `git` | Where history is kept: \"git\" (default, the stackvitals-data branch) or an S3-compatible bucket as s3://bucket/prefix (AWS S3, Cloudflare R2, MinIO: the service must support replace-only-if-unchanged writes, If-Match), for very busy repos and big orgs. Switching starts a fresh history (the branch is not copied). With a bucket, publish the app with pages \"deploy\" or \"artifact\"; the tools that read the stackvitals-data branch (stackvitals ui and sql, Explore, the MCP server, the Portal org view, quarantine in your CI) do not read a bucket yet |\n| `s3-endpoint` |  | The bucket service's endpoint, like https://s3.us-east-1.amazonaws.com or https://<account>.r2.cloudflarestorage.com |\n| `s3-region` | `auto` | The bucket's region (\"auto\" for R2) |\n| `s3-key-id` |  | An access key id that can read, write, delete and list that prefix (pass a secret) |\n| `s3-secret` |  | Its secret (pass a secret) |\n","quickstart":"A GitHub repo is all you need; CI on GitHub Actions makes the CI part useful. Nothing has to be installed: the StackVitals app's Setup page checks your repo and, with a GitHub token, commits the workflow, starts the first run and turns on Pages (public repos only) in a few clicks once GitHub is connected; connecting takes one token, made from a prefilled GitHub page (without a token it opens GitHub with the file ready). From a terminal, [uv](https://docs.astral.sh/uv/) or pipx prints the same file. A public repo gets its monitor on GitHub Pages; a private repo works the same way, and you open its monitor on your machine or as a download instead (step 3), because Pages from a private repo is public on most plans.\n\n## 1. Add the workflow\n\nPrint a workflow that already names your CI workflows, and save it in your repo:\n\n```\nuvx stackvitals ci init OWNER/REPO > .github/workflows/stackvitals.yml\n```\n\nIt runs CI health, Team Flow, the dependency check and Community, and starts after the CI workflows it names (the ones that look like tests and builds; edit the list as you like). To install only some modules, print the file for them, for example `uvx stackvitals ci init OWNER/REPO --modules ci,deps`: it asks only for the permissions and triggers those modules use (the app's Setup page has the same choice). Every permission in the file has a comment saying what it is for. The whole file, what each module id needs and every input: [install.md](install.md).\n\n## 2. Commit it\n\nCommit the file to your default branch. That starts the first run by itself (it also runs after every CI run and every 3 hours). The first run reads up to 90 days of CI history and a year of pull requests; on a small repo it finished 55 seconds after the commit, and on a busy repo it may say \"still filling in older history\", and the next runs continue where it stopped. The supply chain part needs GitHub's dependency graph (on for public repos; for a private repo turn it on under Settings, Code security); until then the job says so once a week.\n\n## 3. Turn on the dashboard\n\nOpen **Settings → Pages**. Under \"Build and deployment\", choose **Deploy from a branch**, branch `stackvitals-data`, folder `/docs`, and save. After a minute the dashboard is at:\n\n```\nhttps://OWNER.github.io/REPO/\n```\n\nThat address opens the StackVitals app: Overview first, then a page per module, Settings (change any setting and save it with your GitHub token, or send it as an issue the workflow applies), Activity (every decision and change), Connections and Help. Add `?demo=1` to the address for three example repos; [app.md](app.md) walks through it. The module pages (`ci.html`, `flow.html`, ...) stay as printable views and work without JavaScript.\n\n**A private repo:** skip Pages and run `pip install stackvitals` then `stackvitals ui OWNER/REPO` (it uses your `gh` login or `GITHUB_TOKEN` for one git command and keeps nothing). The same app opens on `127.0.0.1` in a few seconds.\n\n**No install at all:** `stackvitals ui OWNER/REPO --collect` reads a repo's history on your machine (CI runs and pull requests, 90 days by default) into a local data branch and opens the app, with no Action, server, database or Docker. It only reads from GitHub: nothing is posted, labelled or committed there. On pallets/click the first read took 11 minutes.\n\n## What next\n\n- **Single flaky tests:** upload your test results (JUnit XML, CTRF, Jest or Vitest JSON, `go test -json`, .NET TRX) with `actions/upload-artifact` under a name containing `test-results`, `junit` or `ctrf`.\n- **Opt-in automation:** turn switches on under `[automation]` in `.github/stackvitals.toml` (`rerun_flaky`, `issues`, `assign_owners`, `mention_owners`, `quarantine`, `preflight_comments`, `deps_comments`, `label_stuck`), or with the Action input of the same name written with hyphens (`rerun-flaky`; the file wins when both are set); [docs/settings.md](settings.md) lists every setting and the permission each needs. The `webhook` input takes a Slack, Discord, Teams or JSON URL.\n- **Traffic history:** pass a token from your own GitHub App as `traffic-token:` (see [Community](modules.md#community)).\n- **Your org:** Portal runs from a separate private repo (`stackvitals portal init`).\n- **Agents:** `stackvitals serve` starts the MCP server; `stackvitals contribute brief OWNER/REPO` prints how changes get merged in a repo.\n- **Busy repos:** GitHub Pages builds a branch about 10 times an hour at most. Past that, publish from the job instead: set `pages: deploy` on the Action, choose **GitHub Actions** as the Pages source, and give the job `pages: write`, `id-token: write` and `environment: github-pages`.\n\n## If something goes wrong\n\n- **\"Resource not accessible by integration\":** a permission is missing from the workflow file; the job summary names it.\n- **No dashboard after enabling Pages:** check that the `stackvitals-data` branch exists (it is created by the first run) and that Pages points at its `/docs` folder.\n- **Private repos:** GitHub Pages from a private repo is public on most plans, so set `pages: artifact` (each run that changed the app uploads it as a download kept 7 days; unzip it and follow its OPEN-ME.txt), or use `stackvitals ui OWNER/REPO` on your machine instead.\n","guides-maintainer":"You want red CI to stop eating contributors' time, newcomers answered, and fewer surprise dependency breaks. The Action\nis free on public repos and needs only the built-in token.\n\n## Set it up\n\n[Quickstart](../quickstart.md): one workflow file, then GitHub Pages from the `stackvitals-data` branch. Your\ndashboard is public like your repo. Your code stays in GitHub; the dependency check sends package names to deps.dev\nand the registries unless you set `[deps] outside_lookups = false` ([privacy](../privacy.md)).\n\n## CI that contributors can trust\n\n- When CI fails on a pull request, one comment says which failures are known flakes, which already fail on the\n  default branch and which are likely the change's own ([modules](../modules.md#ci-health)). The verdict check still\n  blocks until CI is green, so a flake is re-run, never waved through.\n- Upload test reports (JUnit XML, CTRF, Jest, Vitest, `go test -json`, TRX) to see single flaky tests, not just jobs.\n- Opt in under `[automation]` to re-run known flakes once, open one issue per flaky test, or quarantine one.\n- Contributors and agents can ask `stackvitals contribute brief owner/repo` for how changes get merged here: what\n  merged pull requests usually include, who reviews each folder, and which tests are flaky.\n\n## Newcomers\n\nCommunity shows how fast issues get a first answer from the project (maintainers, regular responders and triagers;\ndrive-by comments don't count), where newcomers drop out, and who carries the work\n([modules](../modules.md#community)). Issues still waiting stay in the median, so a slow week can't hide.\n\n## Dependencies\n\nOn each pull request that changes a lockfile, the Dependency Steward checks what was added or upgraded: advisories on\nthe installed version, look-alike names, licence changes, and how the same upgrade went in other public repos\n([modules](../modules.md#dependency-steward)).\n\n## Commands\n\nAnyone with triage access can decide from a comment: `/stackvitals acknowledge`, `mute`, `snooze`, `quarantine`, `accept` and more, with `for 14d` and\n`because <reason>`. Every decision is recorded with who and why, and temporary ones end on their own ([all commands](../modules.md#decide-on-github-hear-back-every-week)).\n","guides-manager":"You want to know where your team's time goes and whether changes help, without rating people. This is the path.\n\n## First week\n\n1. **Set it up** on each of your team's repos ([quickstart](../quickstart.md)), or on one repo first. It reads up to\n   90 days of CI and a year of pull requests on the first run, so you start with history, not an empty page.\n2. **Open Insights** in the app: one list of what to fix first across CI, reviews and delivery, ranked by hours at\n   stake ([modules](../modules.md#insights-what-to-fix-first)). Start at the top.\n3. **Read Delivery** for cycle time in LinearB's phases and DORA's five metrics. Before you share a DORA number,\n   read [the DORA guide](../dora-guide.md): what each number can't tell, how it gets gamed, and why it is never for\n   individual reviews.\n\n## Every week\n\n- The **weekly digest** (Monday, one comment on a pinned issue; add your chat under `[digest] channels`) says what\n  changed. It reports a change only when it stands out from how much your repo's weeks usually swing, so false alarms\n  are rare, but a quiet digest means nothing stood out, not that nothing moved: on small repos even large slowdowns\n  can go unflagged ([modules](../modules.md#decide-on-github-hear-back-every-week)).\n- **Working agreements** you set under `[agreements]` (first review within a day, PRs under 400 lines) are previewed\n  week by week before you turn them on, then counted in the digest.\n- The digest asks three **pulse** questions by reaction; StackVitals shows only the counts (GitHub itself shows who\n  reacted to anyone who can see the issue).\n\n## Questions you'll be asked\n\n- **\"Did AI tools help?\"** Delivery compares AI-assisted pull requests with others from the same repo, month and size.\n  Raw comparisons mislead: on public repos a 9-point gap all but disappears once compared like with like\n  ([AI card](../../bench/accuracy/ai/README.md)). The result is an association, never a cause.\n- **\"How do we compare?\"** Benchmarks are against similar public repos, with the sample size beside each.\n- **\"Can I see this per person?\"** No. StackVitals names people only to route work (an owner, who acts next).\n\n## Across several teams\n\nAsk your platform team for the Portal's org view: every repo's figures added up, with the repos leading each\nproblem ([platform guide](platform.md)).\n","guides-platform":"You run GitHub for an org and want one view of every repo, owners that are real, standards people can see coming,\nand a security review you can sign off.\n\n## Roll it out\n\n1. **Portal first.** Create a private repo (for example `portal`) and save the workflow from\n   `stackvitals portal init` in it (`stackvitals portal check my-org` first says what your token can read). With the built-in token it reads public repos; for private ones, an org\n   owner creates a read-only org App (`stackvitals portal app-link my-org` opens GitHub's form filled in)\n   ([modules](../modules.md#portal)).\n2. **The suite per repo.** Each team adds the workflow ([quickstart](../quickstart.md)); `uvx stackvitals ci init\n   OWNER/REPO` writes it with that repo's CI workflows named. Private repos set `pages: artifact` (each run that\n   changed the app uploads it as a download kept 7 days) or open the app with `stackvitals ui OWNER/REPO`.\n3. **Settings as code.** `.github/stackvitals.toml` per repo; every setting, its default and the permission it needs\n   are in [settings.md](../settings.md).\n\n## What the Portal gives you\n\n- **Owners, checked:** CODEOWNERS as GitHub reads it, teams with members, people still in the org.\n- **Scorecards** (bronze, silver, gold) with evidence, your own rules in `portal.toml`, and `from = <date>` so a new\n  rule shows what it would say before it counts.\n- **Campaigns** with a date, a tracking issue and reminders; **required checks** for a level; **golden paths** that\n  open a pull request with your standard files; a **docs index**; a **Backstage** export.\n- **The org view:** totals and medians of what every repo published, with the five worst repos on each.\n\n## Security and compliance\n\n- [Threat model](../threat-model.md): each threat, what stops it, and the test that holds it.\n- [Enterprise controls](../enterprise.md): SSO, permissions, audit log and data residency all come from GitHub.\n- StackVitals' own workflows pin every action to a commit, and the release workflow signs releases with Sigstore and\n  adds an SBOM; pin the Action itself to a release's commit SHA ([SECURITY.md](../../SECURITY.md)).\n- [What leaves your repos](../privacy.md): every outside host the code can reach, each optional.\n\n## Scale\n\nFor busy repos (2,000 CI runs a day), CI Health alone is estimated at under 40% of the built-in token's hourly limit;\ntheir CI history keeps about four months under the default 250 MB cap ([scaling](../scaling.md)). Portal reads about 180 to 260 repos per run with the\nbuilt-in token and continues where it stopped.\n"}

stackvitals/web/app/help/index.json

@@ -0,0 +1 @@
+{"groups":["Getting started","Modules","Connectors","Agents","Privacy and security","Accuracy","Reference"],"entries":[{"doc":"quickstart","title":"Quick start: your monitor in about five minutes","summary":"A GitHub repo is all you need; CI on GitHub Actions makes the CI part useful. Nothing has to be installed: the StackVitals app's Setup page checks your repo and, with a GitHub token, commits the workflow, starts the…","group":0},{"doc":"install","title":"Install","summary":"StackVitals is a GitHub Action, uses: stackvitals/stackvitals@v0, run by one workflow file in your repo. The file below is exactly what stackvitals ci init owner/repo prints (that command also fills in your CI workflows'…","group":0},{"doc":"app","title":"The StackVitals app","summary":"Every run publishes one app over all the modules that ran: https://OWNER.github.io/REPO/ once GitHub Pages serves the stackvitals-data branch. For a private repo, stackvitals ui owner/repo opens the same app on 127.0.0.1…","group":0},{"doc":"faq","title":"Questions people ask","summary":"It is licensed under Apache 2.0 and runs in your own GitHub Actions minutes, which are free for public repos and included in every plan for private ones. There is no paid tier, no account and no StackVitals-hosted…","group":0},{"doc":"guides-maintainer","title":"StackVitals for an open-source maintainer","summary":"You want red CI to stop eating contributors' time, newcomers answered, and fewer surprise dependency breaks. The Action is free on public repos and needs only the built-in token.","group":0},{"doc":"guides-manager","title":"StackVitals for an engineering manager","summary":"You want to know where your team's time goes and whether changes help, without rating people. This is the path.","group":0},{"doc":"guides-platform","title":"StackVitals for a platform team","summary":"You run GitHub for an org and want one view of every repo, owners that are real, standards people can see coming, and a security review you can sign off.","group":0},{"doc":"modules","title":"Modules","summary":"What each StackVitals module does, how to try it on any public repo from your machine, and how to set it up in your own repo. The README has the overview and the quickstart the five-minute setup.","group":1},{"doc":"dora-guide","title":"Using delivery numbers without fooling yourself","summary":"DORA's numbers describe a team's delivery system, not a person's work. They are useful for spotting where changes wait and whether shipping got safer; they become useless the moment people work to move them. This page…","group":1},{"doc":"chaoss","title":"StackVitals and CHAOSS","summary":"CHAOSS defines open source community health metrics. Each row below is a CHAOSS metric StackVitals answers, the code that computes it, and where ours differs and why. The list is built from…","group":1},{"doc":"coverage","title":"Coverage and peer groups","summary":"When StackVitals compares your repo with \"similar public repos\", the peers come from the MergeScout crawler, which reads public GitHub repos daily on GitHub Actions. This page says what it reads, what it misses, and how…","group":1},{"doc":"scaling","title":"How StackVitals grows with you","summary":"StackVitals keeps each repo's history in that repo's own stackvitals-data branch. That branch is always the source of truth: everything below is a view of it or a copy of it, and you can delete any copy and rebuild it…","group":1},{"doc":"connectors","title":"What StackVitals reads, and from where","summary":"Live means run against the real service; recorded means tested against saved answers in the shape the service's documentation gives, with no account of ours; experimental means built but not yet checked either way.","group":2},{"doc":"triage","title":"Model triage: setup","summary":"When StackVitals' history and the log's signs can't decide whether a pull request's failure came from the change, a model can read the failing step's error lines and the changed file names and give its view. The view is…","group":2},{"doc":"privacy","title":"What leaves your repo","summary":"StackVitals runs in your own workflow and keeps its history in your repo. There is no StackVitals server: nothing is sent to us. This page lists everything a run (or the app) sends anywhere, built from the code's own…","group":4},{"doc":"threat-model","title":"Threat model","summary":"StackVitals runs inside your own GitHub Actions with your repo's tokens, reads text that strangers write (pull request titles, test names, logs, issues, comments) and writes back to GitHub (comments, labels, issues, the…","group":4},{"doc":"enterprise","title":"Enterprise controls","summary":"StackVitals has no server and no account: it runs in your GitHub Actions with your tokens and keeps everything in your repos. So the controls an enterprise asks a vendor for are GitHub's own, applied to StackVitals like…","group":4},{"doc":"accuracy","title":"How accurate StackVitals is","summary":"The rules behind StackVitals' verdicts and headline numbers were each checked against public data and, where a judgement was needed, against labelled examples. Each card in bench/accuracy/ holds the method, the sample…","group":5},{"doc":"methods","title":"Methods: the exact settings behind every number","summary":"Every headline number in StackVitals carries its sample size and a 95% range. This page lists the method and the settings each one uses, so anyone can check them against the code (src/stackvitals/core/stats.py…","group":5},{"doc":"rules","title":"Rule reference","summary":"Every rule a StackVitals verdict or number rests on, from bench/accuracy/registry.json: what it does, the data it was tuned on and checked on, the bar it had to meet (set before the check where a date says so), its…","group":5},{"doc":"limitations","title":"Known limitations","summary":"What each part of StackVitals can't see or do, where its measured accuracy is weak, and what it needs from your repo. Numbers come from the accuracy cards (accuracy.md).","group":5},{"doc":"settings","title":"StackVitals settings","summary":"Put these in .github/stackvitals.toml on your default branch. A merged change applies on the next run (the workflow runs when the file is pushed). A missing file means the defaults; a value StackVitals can't use keeps…","group":6},{"doc":"architecture","title":"How StackVitals works","summary":"StackVitals has no hosted backend. Everything happens in three places you already have: a GitHub Actions job in your repo, a branch of your repo, and the reader's browser.","group":6},{"doc":"roadmap","title":"Roadmap","summary":"What comes next, in rough order. Dates are when the data allows a result, not promises. Suggestions are welcome as issues; the ones backed by a real repo's problem move up.","group":6},{"doc":"modules","h":"ci-health","title":"CI Health","summary":"Flaky and failing tests and jobs, time to green and the cost of re-runs, from every CI run kept in your repo.","group":1},{"doc":"modules","h":"insights-what-to-fix-first","title":"Insights: what to fix first","summary":"When Team Flow runs, each job also writes docs/insights.html and docs/insights.json, and the job summary starts with the top three. Each finding is one sentence with its number, how many pull requests it rests on, links…","group":1},{"doc":"modules","h":"decide-on-github-hear-back-every-week","title":"Decide on GitHub, hear back every week","summary":"Commands. Anyone with triage or write access can type a command in a comment on any issue or pull request. StackVitals answers on the issue and logs who decided what, when and why in the stackvitals-data branch…","group":1},{"doc":"modules","h":"insights-red-ci-on-pull-requests","title":"Insights: red CI on pull requests","summary":"When CI Health and Team Flow both run, each job also writes the red-CI part of docs/insights.json and a section of the job summary: how many of the people's pull requests merged in the last 90 days sat ready for review…","group":1},{"doc":"modules","h":"delivery-cycle-time-and-dora","title":"Delivery: cycle time and DORA","summary":"When Team Flow runs, each scheduled job also writes docs/delivery.html and docs/delivery.json. For any public repo from your machine: stackvitals delivery owner/repo (--json for the numbers). MCP: delivery_health…","group":1},{"doc":"modules","h":"team-flow","title":"Team Flow","summary":"It answers \"what is blocking this repo's pull requests, and who has to act next?\"","group":1},{"doc":"modules","h":"dependency-steward","title":"Dependency Steward","summary":"Known vulnerabilities in every lockfile, every ecosystem: after CI runs and on the schedule, the Action reads the default branch's lockfiles through the API (npm, yarn, pnpm and bun; pip requirements, Poetry, uv, Pipenv…","group":1},{"doc":"modules","h":"community","title":"Community","summary":"It answers \"is our project growing, and where do contributors drop out?\"","group":1},{"doc":"modules","h":"portal","title":"Portal","summary":"Light by design: no server, one daily run, pages in a private repo. It covers the catalog, checked ownership, scorecards against similar public repos, campaigns, scorecards as required checks, golden paths, a docs index…","group":1},{"doc":"modules","h":"contribute-mergescout","title":"Contribute (MergeScout)","summary":"mergescout (or stackvitals contribute) runs the MergeScout MCP server and CLI: issues an outsider can realistically get merged, with merge rates, review times and live availability checks.","group":1},{"doc":"app","h":"for-agents","title":"For agents","summary":"Agents can propose a settings change or a decision as a prefilled link; a person opens it and submits.","group":3},{"doc":"limitations","h":"the-mcp-server","title":"The MCP server","summary":"What the MCP server adds for a coding agent and where it doesn't help: it needs a token, never changes a repo and answers from kept history.","group":3},{"doc":"accuracy","h":"against-a-general-ai-agent","title":"Against a general AI agent","summary":"The same questions, answered by an agent with live GitHub access, with and without StackVitals. On single failures the agent is as accurate either way; what StackVitals adds is answers with nobody asking (the PR comment…","group":3},{"doc":"faq","h":"how-is-this-different-from-asking-an-ai-agent-to-look-at-my-ci","title":"How is this different from asking an AI agent to look at my CI?","summary":"An agent with GitHub access sees what GitHub still has: 90 days of logs and whatever it can read in one chat. StackVitals keeps the history after GitHub deletes it, has already counted how often each test flaked, and…","group":3}],"docs":[{"id":"accuracy","file":"accuracy.md","title":"How accurate StackVitals is","group":5,"chunk":"accuracy","headings":[[2,"Weekly changes and ranges","weekly-changes-and-ranges"],[2,"AI-assisted pull requests, like with like","ai-assisted-pull-requests-like-with-like"],[2,"Flaky CI","flaky-ci"],[2,"Deployments and failures","deployments-and-failures"],[2,"First responses","first-responses"],[2,"Against a general AI agent","against-a-general-ai-agent"],[2,"Rule changes","rule-changes"]]},{"id":"app","file":"app.md","title":"The StackVitals app","group":0,"chunk":"getting-started","headings":[[2,"Opening it","opening-it"],[2,"Layout","layout"],[2,"First run","first-run"],[2,"Connecting GitHub","connecting-github"],[2,"Setup in a few clicks","setup-in-a-few-clicks"],[2,"Pages","pages"],[2,"Where a number comes from","where-a-number-comes-from"],[2,"Data health","data-health"],[2,"Connections","connections"],[2,"Making a decision","making-a-decision"],[2,"Changing settings","changing-settings"],[2,"On this device","on-this-device"],[2,"For agents","for-agents"]]},{"id":"architecture","file":"architecture.md","title":"How StackVitals works","group":6,"chunk":"reference","headings":[[2,"One job, one commit","one-job-one-commit"],[2,"Storage","storage"],[2,"The app","the-app"],[2,"Rules and their evidence","rules-and-their-evidence"],[2,"Cross-repo evidence","cross-repo-evidence"],[2,"Code map","code-map"]]},{"id":"chaoss","file":"chaoss.md","title":"StackVitals and CHAOSS","group":1,"chunk":"modules","headings":[]},{"id":"connectors","file":"connectors.md","title":"What StackVitals reads, and from where","group":2,"chunk":"connectors","headings":[[2,"How each integration was tested","how-each-integration-was-tested"],[2,"Test a connection","test-a-connection"],[2,"Read a service again","read-a-service-again"],[2,"GitHub (built in)","github-built-in"],[2,"Issue trackers: Jira Cloud and Linear (one optional key each)","issue-trackers-jira-cloud-and-linear-one-optional-key-each"],[2,"Community channels: Discourse, Slack and Discord","community-channels-discourse-slack-and-discord"],[2,"Incidents: PagerDuty and incident.io","incidents-pagerduty-and-incidentio"],[2,"Code quality: SonarQube or SonarCloud","code-quality-sonarqube-or-sonarcloud"],[2,"AI spend","ai-spend"],[2,"GitHub Copilot usage reports","github-copilot-usage-reports"],[2,"Chat: Slack, Microsoft Teams, Discord","chat-slack-microsoft-teams-discord"],[2,"Other CI systems","other-ci-systems"],[2,"Deployments and incidents from elsewhere","deployments-and-incidents-from-elsewhere"],[2,"Bitbucket and Azure DevOps","bitbucket-and-azure-devops"],[2,"GitLab","gitlab"]]},{"id":"coverage","file":"coverage.md","title":"Coverage and peer groups","group":1,"chunk":"modules","headings":[[2,"What the crawl reads","what-the-crawl-reads"],[2,"What it misses","what-it-misses"],[2,"Peer groups, benchmark by benchmark","peer-groups-benchmark-by-benchmark"],[2,"How fresh the numbers are","how-fresh-the-numbers-are"],[2,"How to read a comparison","how-to-read-a-comparison"]]},{"id":"dora-guide","file":"dora-guide.md","title":"Using delivery numbers without fooling yourself","group":1,"chunk":"modules","headings":[[2,"DORA's five metrics","doras-five-metrics"],[2,"Never for individual reviews","never-for-individual-reviews"],[2,"Each number: what it can't tell, and how it gets gamed","each-number-what-it-cant-tell-and-how-it-gets-gamed"],[2,"How StackVitals helps you read them","how-stackvitals-helps-you-read-them"],[2,"Further reading","further-reading"]]},{"id":"enterprise","file":"enterprise.md","title":"Enterprise controls","group":4,"chunk":"privacy-and-security","headings":[[2,"Rate limits on Enterprise Cloud","rate-limits-on-enterprise-cloud"]]},{"id":"faq","file":"faq.md","title":"Questions people ask","group":0,"chunk":"getting-started","headings":[[2,"Is it really free? What's the catch?","is-it-really-free-whats-the-catch"],[2,"Does my code or data leave GitHub?","does-my-code-or-data-leave-github"],[2,"How many Actions minutes does it use?","how-many-actions-minutes-does-it-use"],[2,"How is this different from asking an AI agent to look at my CI?","how-is-this-different-from-asking-an-ai-agent-to-look-at-my-ci"],[2,"How is it different from LinearB, Swarmia, Trunk or DevLake?","how-is-it-different-from-linearb-swarmia-trunk-or-devlake"],[2,"Can it rate developers?","can-it-rate-developers"],[2,"Why does it say \"not enough data\" so often?","why-does-it-say-not-enough-data-so-often"],[2,"A verdict was wrong. What do I do?","a-verdict-was-wrong-what-do-i-do"],[2,"Does it work with GitLab, Jenkins or Buildkite?","does-it-work-with-gitlab-jenkins-or-buildkite"],[2,"Does it work on GitHub Enterprise Server?","does-it-work-on-github-enterprise-server"],[2,"Is there a login, with roles for who sees what?","is-there-a-login-with-roles-for-who-sees-what"],[2,"Our repos are private. Can we still see the dashboard?","our-repos-are-private-can-we-still-see-the-dashboard"],[2,"What happens if a run fails halfway?","what-happens-if-a-run-fails-halfway"]]},{"id":"install","file":"install.md","title":"Install","group":0,"chunk":"getting-started","headings":[[2,"Modules","modules"],[2,"The workflow file","the-workflow-file"],[2,"The portal workflow","the-portal-workflow"],[2,"Every input","every-input"]]},{"id":"limitations","file":"limitations.md","title":"Known limitations","group":5,"chunk":"accuracy","headings":[[2,"Every module","every-module"],[2,"CI health","ci-health"],[2,"Team Flow","team-flow"],[2,"Delivery","delivery"],[2,"Dependencies","dependencies"],[2,"Community","community"],[2,"Portal","portal"],[2,"Connectors","connectors"],[2,"The app","the-app"],[2,"The MCP server","the-mcp-server"],[2,"Failure modes","failure-modes"]]},{"id":"methods","file":"methods.md","title":"Methods: the exact settings behind every number","group":5,"chunk":"accuracy","headings":[[2,"Shares: Wilson ranges","shares-wilson-ranges"],[2,"Medians and percentiles: order-statistic ranges","medians-and-percentiles-order-statistic-ranges"],[2,"Waiting times: Kaplan-Meier","waiting-times-kaplan-meier"],[2,"Weekly changes: a test per number, then false-alarm control","weekly-changes-a-test-per-number-then-false-alarm-control"],[2,"Small repos: shrinkage toward similar repos","small-repos-shrinkage-toward-similar-repos"],[2,"AI-assisted work: like with like","ai-assisted-work-like-with-like"],[2,"Comparing StackVitals with other rules","comparing-stackvitals-with-other-rules"],[2,"Labels","labels"]]},{"id":"modules","file":"modules.md","title":"Modules","group":1,"chunk":"modules","headings":[[2,"CI Health","ci-health"],[2,"Insights: what to fix first","insights-what-to-fix-first"],[2,"Decide on GitHub, hear back every week","decide-on-github-hear-back-every-week"],[2,"Insights: red CI on pull requests","insights-red-ci-on-pull-requests"],[2,"Delivery: cycle time and DORA","delivery-cycle-time-and-dora"],[2,"Team Flow","team-flow"],[2,"Dependency Steward","dependency-steward"],[2,"Community","community"],[2,"Portal","portal"],[2,"Contribute (MergeScout)","contribute-mergescout"]]},{"id":"privacy","file":"privacy.md","title":"What leaves your repo","group":4,"chunk":"privacy-and-security","headings":[[2,"Nothing leaves GitHub","nothing-leaves-github"],[2,"What each token can reach","what-each-token-can-reach"]]},{"id":"quickstart","file":"quickstart.md","title":"Quick start: your monitor in about five minutes","group":0,"chunk":"getting-started","headings":[[2,"1. Add the workflow","1-add-the-workflow"],[2,"2. Commit it","2-commit-it"],[2,"3. Turn on the dashboard","3-turn-on-the-dashboard"],[2,"What next","what-next"],[2,"If something goes wrong","if-something-goes-wrong"]]},{"id":"roadmap","file":"roadmap.md","title":"Roadmap","group":6,"chunk":"reference","headings":[[2,"Checks already running, read on a date","checks-already-running-read-on-a-date"],[2,"Accuracy","accuracy"],[2,"Coverage","coverage"],[2,"Not planned","not-planned"]]},{"id":"rules","file":"rules.md","title":"Rule reference","group":5,"chunk":"accuracy","headings":[[2,"flaky.event (version 3)","flakyevent-version-3"],[2,"flaky.known_flaky (version 4)","flakyknown_flaky-version-4"],[2,"flaky.outside_failure (version 1)","flakyoutside_failure-version-1"],[2,"flaky.already_failing (version 1)","flakyalready_failing-version-1"],[2,"flaky.maybe_flaky (version 1)","flakymaybe_flaky-version-1"],[2,"flaky.pr_details (version 1)","flakypr_details-version-1"],[2,"dora.deploy_source (version 2)","doradeploy_source-version-2"],[2,"dora.fix_chain (version 2)","dorafix_chain-version-2"],[2,"community.regular_responders (version 2)","communityregular_responders-version-2"],[2,"community.first_response_km (version 2)","communityfirst_response_km-version-2"],[2,"uncertainty.digest (version 2)","uncertaintydigest-version-2"],[2,"uncertainty.shrinkage (version 1)","uncertaintyshrinkage-version-1"],[2,"ai.matched (version 1)","aimatched-version-1"],[2,"dora.incident_link (version 1)","doraincident_link-version-1"],[2,"ai.adoption (version 1)","aiadoption-version-1"],[2,"ai.detection (version 2)","aidetection-version-2"]]},{"id":"scaling","file":"scaling.md","title":"How StackVitals grows with you","group":1,"chunk":"modules","headings":[[2,"The ladder","the-ladder"],[3,"When would the files be too big?","when-would-the-files-be-too-big"],[3,"Busy repos","busy-repos"],[2,"Actions minutes (private repos)","actions-minutes-private-repos"],[2,"Free Postgres services (checked 2026-10-21; they change, so check before you rely on them)","free-postgres-services-checked-2026-10-21-they-change-so-check-before-you-rely-on-them"],[2,"What stays the same at every step","what-stays-the-same-at-every-step"]]},{"id":"settings","file":"settings.md","title":"StackVitals settings","group":6,"chunk":"reference","headings":[[2,"[ci.monitors.pass_on_retry]","cimonitorspass_on_retry"],[2,"[ci.monitors.failure_rate]","cimonitorsfailure_rate"],[2,"[ci.monitors.broken]","cimonitorsbroken"],[2,"[ci.monitors.slow]","cimonitorsslow"],[2,"[ci.monitors.timeout_inflation]","cimonitorstimeout_inflation"],[2,"[ci.mute]","cimute"],[2,"[ci.quarantine]","ciquarantine"],[2,"[ci.coverage]","cicoverage"],[2,"[ci.triage]","citriage"],[2,"[delivery]","delivery"],[2,"[investment]","investment"],[2,"[rules]","rules"],[2,"[calendar]","calendar"],[2,"[services]","services"],[2,"[identity]","identity"],[2,"[alerts]","alerts"],[2,"[automation]","automation"],[2,"[agreements]","agreements"],[2,"[digest]","digest"],[2,"[deps.policy]","depspolicy"],[2,"[deps]","deps"],[2,"[deps.licenses]","depslicenses"],[2,"[benchmarks]","benchmarks"],[2,"[community]","community"],[2,"[connectors.jira]","connectorsjira"],[2,"[connectors.linear]","connectorslinear"],[2,"[connectors.discourse]","connectorsdiscourse"],[2,"[connectors.slack]","connectorsslack"],[2,"[connectors.discord]","connectorsdiscord"],[2,"[connectors.pagerduty]","connectorspagerduty"],[2,"[connectors.incident_io]","connectorsincident_io"],[2,"[connectors.copilot]","connectorscopilot"],[2,"[connectors.sonar]","connectorssonar"],[2,"[connectors.ai_spend]","connectorsai_spend"]]},{"id":"threat-model","file":"threat-model.md","title":"Threat model","group":4,"chunk":"privacy-and-security","headings":[[2,"Untrusted input","untrusted-input"],[2,"Fork pull requests and workflow_run","fork-pull-requests-and-workflow_run"],[2,"Who can change things","who-can-change-things"],[2,"Keys and tokens","keys-and-tokens"],[2,"The app in the browser","the-app-in-the-browser"],[2,"Private data on public pages","private-data-on-public-pages"],[2,"Losing or corrupting history","losing-or-corrupting-history"],[2,"Not covered","not-covered"]]},{"id":"triage","file":"triage.md","title":"Model triage: setup","group":2,"chunk":"connectors","headings":[[2,"Gemini","gemini"],[2,"OpenAI","openai"],[2,"Anthropic","anthropic"],[2,"Mistral","mistral"],[2,"Groq","groq"],[2,"Any other OpenAI-compatible endpoint","any-other-openai-compatible-endpoint"]]},{"id":"guides-maintainer","file":"guides/maintainer.md","title":"StackVitals for an open-source maintainer","group":0,"chunk":"getting-started","headings":[[2,"Set it up","set-it-up"],[2,"CI that contributors can trust","ci-that-contributors-can-trust"],[2,"Newcomers","newcomers"],[2,"Dependencies","dependencies"],[2,"Commands","commands"]]},{"id":"guides-manager","file":"guides/manager.md","title":"StackVitals for an engineering manager","group":0,"chunk":"getting-started","headings":[[2,"First week","first-week"],[2,"Every week","every-week"],[2,"Questions you'll be asked","questions-youll-be-asked"],[2,"Across several teams","across-several-teams"]]},{"id":"guides-platform","file":"guides/platform.md","title":"StackVitals for a platform team","group":0,"chunk":"getting-started","headings":[[2,"Roll it out","roll-it-out"],[2,"What the Portal gives you","what-the-portal-gives-you"],[2,"Security and compliance","security-and-compliance"],[2,"Scale","scale"]]}]}

stackvitals/web/app/help/modules.json

@@ -0,0 +1 @@
+{"chaoss":"[CHAOSS](https://chaoss.community) defines open source community health metrics. Each row below is a CHAOSS metric StackVitals answers, the code that computes it, and where ours differs and why. The list is built from `stackvitals/community/chaoss.py`, and a test checks every function it names exists.\n\n| CHAOSS metric | Computed by | How StackVitals differs |\n|---|---|---|\n| [Time to First Response](https://chaoss.community/kb/metric-time-to-first-response/) | `community.analyze.first_answer` | Only answers from the project count (maintainers, people who triage, regular responders, or a close), not the first comment from anyone; issues still waiting stay in the median (Kaplan-Meier) instead of dropping out. On labelled public issues, 46% of the first comments other tools count are drive-bys (bench/accuracy/community). |\n| [Issue Response Time](https://chaoss.community/kb/metric-issue-response-time/) | `community.analyze.first_answer` | The same rule as the first response above, for issues. |\n| [Newcomer Experience](https://chaoss.community/kb/metric-newcomer-experience/) | `community.analyze.answers_by_tenure` | Measured, not surveyed: how fast first-time contributors are answered against people who contributed before. |\n| [New Contributors](https://chaoss.community/kb/metric-new-contributors/) | `community.analyze.funnel` | Bots, AI agents and accounts opening 20 or more items in a day are left out; the funnel also says how many came back within 90 days (CHAOSS has no single retention metric; Inactive Contributors is the nearest). |\n| [Contributor Absence Factor](https://chaoss.community/kb/metric-contributor-absence-factor/) | `community.analyze.dependency` | The same definition (the fewest people behind half of the merged work, formerly Bus Factor) over 180 days, with bots and AI agents left out and similar public repos' middle half shown instead of a rank. |\n| [Elephant Factor](https://chaoss.community/kb/metric-elephant-factor/) | `community.analyze.elephant` | Organisations from the company on people's public profiles; people without one are counted as unknown and shown, never guessed. The factor is over the work with a known organisation, and none is claimed when most is unknown. |\n| [Organizational Diversity](https://chaoss.community/kb/metric-organizational-diversity/) | `community.analyze.elephant` | The share of merged work by organisation, unknown shown. |\n| [Project Popularity](https://chaoss.community/kb/metric-project-popularity/) | `community.analyze.star_trend` | Weekly stars kept over time and placed among similar repos, so a launch spike reads as one. |\n| [Clones](https://chaoss.community/kb/metric-clones/) | `community.analyze.traffic_summary` | Kept past GitHub's 14 days when a traffic token is set. |\n| [Change Request Acceptance Ratio](https://chaoss.community/kb/metric-change-request-acceptance-ratio/) | `flow.analyze.analyze` | Outsiders' pull requests only (insiders merge their own), with its sample size and range. |\n| [Change Request Review Duration](https://chaoss.community/kb/metric-change-request-review-duration/) | `flow.analyze.stages` | Split into waiting for a first review and the review itself, counted from when the PR was ready, not opened. |\n| [Change Requests Duration](https://chaoss.community/kb/metric-change-requests-duration/) | `core.delivery.phases` | LinearB's four phases (coding, pickup, review, deploy), with review split into its time to approve and time to merge, from first commit to deployment. |\n| [Release Frequency](https://chaoss.community/kb/metric-release-frequency/) | `core.delivery.dora` | Counted as DORA's deployment frequency from the source the repo really ships with (deployments, deploy workflows or releases), with the source shown. |\n| [Time to Close](https://chaoss.community/kb/metric-time-to-close/) | `connectors.tracker.lead_times` | From Jira or Linear when connected: created to done, by issue type. |\n| [Labor Investment](https://chaoss.community/kb/metric-labor-investment/) | `core.investment.mix` | Where merged work went each month (feature, fix, maintenance, security, docs), with how each PR was sorted. |\n| [Test Coverage](https://chaoss.community/kb/metric-test-coverage/) | `ci.analyze._coverage` | From the coverage reports CI already uploads, with the change on each pull request. |\n| [Upstream Code Dependencies](https://chaoss.community/kb/metric-upstream-code-dependencies/) | `deps.supply.sbom` | From GitHub's dependency graph, with upgrade evidence from public repos for each package. |\n| [SPDX Document](https://chaoss.community/kb/metric-spdx-document/) | `deps.supply.parse_spdx` | Read from the SPDX SBOM GitHub generates; StackVitals doesn't write one. |\n| [Code of Conduct for a Project](https://chaoss.community/kb/metric-code-of-conduct-for-a-project/) | `portal.inventory.repo_files` | Present or missing, per repo in the Portal catalog. |\n| [Bot Activity](https://chaoss.community/kb/metric-bot-activity/) | `core.actors.kind_of` | Bots and AI agents are found in every module and counted apart from people, never mixed into people's numbers. |\n\n20 of CHAOSS's 89 metrics. Most of the rest are about events, governance documents and surveys, which no repo's history can measure; the community page shows a repo's stated policies (governance, funding, response times) beside its numbers instead.\n","coverage":"When StackVitals compares your repo with \"similar public repos\", the peers come from the MergeScout crawler, which reads public GitHub repos daily on GitHub Actions. This page says what it reads, what it misses, and how each peer group is formed.\n\n## What the crawl reads\n\n- **Repos:** public repos with 100+ stars pushed in the last 90 days, without forks or archived repos; about 125,000. The list is rebuilt from scratch each month.\n- **Order:** source repos of tracked packages first, then repos with the most beginner-labelled issues per API cost.\n- **Per repo:** settings and contributing rules; up to 300 pull requests opened in the last 365 days, newest first, with their first 5 reviews; up to 200 open issues active in the last 90 days, with their last 5 comments.\n- **Refreshes:** the crawl runs hourly until that hour's API points run out. Repos with beginner-labelled issues are due again after 1 day, others after 7. A refresh fetches only changes; a full crawl comes back every 30 days.\n- **CI, stars and shipping samples:** a separate daily job reads a 28th of the repos with 100+ stars pushed in the last 30 days (about 90,000), so each is read about once every 4 weeks: the newest 50 push runs, the weekly star history, and 90 days of deployments and releases.\n\nThe first full crawl (fetched 2026-10-16 to 18) held 1,653,419 pull requests from 23,100 repos.\n\n## What it misses\n\n- Private and enterprise repos; repos under 100 stars, forks, archived repos, and repos with no recent push.\n- GitLab, Bitbucket and other hosts; CI other than GitHub Actions; CI runs on pull requests.\n- Older history of busy repos: only the newest 300 pull requests of the year are read, so their window is shorter. In the first full crawl, 2,430 of 23,100 repos reached 300.\n- A pull request's CI result, read only for dependency-bot pull requests: empty for 98% of pull requests in the first full crawl. Only the upgrade ledger uses it.\n- Draft history (pickup counts from opening), who closed an issue, reviews after the fifth, and deployments made outside GitHub.\n- Repos the queue has not reached yet. Until it has, the crawled set leans toward repos with beginner-labelled issues.\n\n## Peer groups, benchmark by benchmark\n\nMost groups use the repo's main language (the 25 most common; the rest are \"Other\") and its stars band (100-1k, 1k-10k, 10k+). If that group is missing or too small, StackVitals tries the language at any size, then any language at that size, then all repos. If none qualifies, no comparison is shown. People's numbers leave out bots and AI agents.\n\n| Benchmark | Used on | How peers are grouped | Minimum group size | Refresh | Notes |\n|---|---|---|---|---|---|\n| CI push time and first-try green (`ci-benchmarks.json`) | CI Health; Portal silver rule | Language, stars band | 20 repos | Daily sample, each repo every 4 weeks | Under 30 pushes, your share is also pulled toward the group. |\n| Review flow: first review, opened to merged, outside PRs merged, PR size (`flow-benchmarks.json`) | Team Flow; contributor brief; Portal first-review rules | Language, stars band; repos with 10+ people's PRs in 90 days | 50 repos per metric | Daily | Portal silver: better than half; gold: better than three quarters. |\n| Delivery P75: pickup, review, opened to merged (`delivery-benchmarks.json`) | Delivery page | As flow; 5+ values per metric | 50 repos per metric | Daily | Shares inside LinearB's 2026 bands. |\n| Coding agents' PRs next to people's (`flow-benchmarks.json`) | Delivery page; contributor brief | As flow; the gap matches repo, month and size | 20 repos | Daily, 180 days | Merge shares need 5+ decided agent and outside PRs per repo. |\n| Community: first answer, newcomers, top person's share, bus factor (`community-benchmarks.json`) | Community page and score | Language, stars band; 10+ outside PRs, 10+ newcomers or 20+ merged PRs | 50 repos per fact | Daily | Score needs two facts. Newcomer table pools all repos. |\n| Star growth, 12 weeks (`stars-benchmarks.json`) | Community page | Language, stars band | 20 repos | Daily sample, each repo every 4 weeks | Only repos with 100+ stars are compared. |\n| Upgrade outcomes (`deps-ledger.json`) | Dependency Steward, `deps check`, MCP `supply_chain` | Ecosystem, package, step (major, minor, patch) | 3 PRs to publish; rates from 30 PRs in 10 repos | Daily, 180 days | Dependabot and Renovate PRs. |\n| Package map, advisories, owners, trends (`deps-map`, `deps-osv`, `deps-owners`, `deps-trends`) | As above | Per package, no peer group | None | Daily; map reused a week, advisories until changed | 1,000 most-used npm and PyPI packages, plus upgraded ones. |\n| Shipping frequency (`deploys-benchmarks.json`) | Not shown yet | Language, stars band | 20 repos with a source | Daily sample | Built, not read by StackVitals. |\n\nSizes in the files built 2026-10-20 to 22: CI 6,623 repos, review flow and delivery 35,925, community 22,957, stars 5,916, coding agents 611. The smallest published groups held 20 repos (CI, stars) and 53 (flow, delivery, community). The upgrade ledger built 2026-10-21 held 125,992 merged upgrades in 6,420 cells.\n\n## How fresh the numbers are\n\n- Flow, delivery, community and dependency files are rebuilt daily from the crawl. A file that fails to build keeps yesterday's version.\n- CI, stars and shipping files are rebuilt daily from 35 days of samples, so a repo's sample can be 5 weeks old.\n- A repo's crawled data is 1 to 7 days old when the hourly crawl keeps up, older when it falls behind.\n- StackVitals keeps a downloaded file for a day. Every comparison shows when its file was built.\n- Windows: CI 30 days; review flow, delivery and first answers 90 days; who carries the work, coding agents and upgrades 180 days.\n\n## How to read a comparison\n\n- Read the group name and repo count first. \"Python repos with 1k-10k stars\" is closer than \"public repos we track\"; a wide group means the narrow one was too small.\n- Places are rough: read from the group's 10th to 90th percentiles, kept between 10% and 90%, rounded down to 5.\n- Your repo is measured the crawl's way: same window, people only. Flow and delivery need 10+ people's PRs from your repo in 90 days.\n- Delivery ranges are 95% ranges for the typical value; Community's low and high are the group's 25th and 75th percentiles.\n- Peers are public repos with 100+ stars. A private team can differ for good reasons.\n- To use your own file, set `STACKVITALS_BENCH_URL` or the matching variable for that benchmark.\n","dora-guide":"DORA's numbers describe a team's delivery system, not a person's work. They are useful for spotting where changes\nwait and whether shipping got safer; they become useless the moment people work to move them. This page says what each\none can't tell you and how each is gamed, so a team can read them honestly.\n\n## DORA's five metrics\n\nDORA groups its [five software delivery metrics](https://dora.dev/guides/dora-metrics/) in two: throughput (deployment\nfrequency, change lead time, failed deployment recovery time) and instability (change fail rate, deployment rework\nrate). StackVitals counts the first four with Apache DevLake's definitions (an incident blames the last production\ndeployment before it), and shows beside the change fail rate a second one where a revert blames the deployment that\nshipped what it undid; rework rate is the share of deployments that shipped an unplanned fix. The tiers (elite,\nhigh, medium, low) come from DevLake's 2023 table, which predates rework rate, so rework rate has no tier.\nA pull request counts toward the production deployment that shipped it: the one whose commits (everything between it\nand the deployment before it, read once and kept) hold its merge commit, as DevLake does; only when no range holds it\ndoes the first deployment after the merge count. So a deploy of an older commit or a rollback doesn't take the pull\nrequests merged just before it.\n\n## Never for individual reviews\n\nEvery number here is a property of how a team, its process and its tooling work together. A person's cycle time\ndepends on what they were given, who could review it and what broke that week. StackVitals never ranks people on its\npages, and its numbers shouldn't be used in performance reviews: once they are, people optimise the number instead of\nthe delivery, and the number stops meaning anything (Goodhart's law).\n\n## Each number: what it can't tell, and how it gets gamed\n\n| Number | What it can't tell you | How teams game it | What to look at beside it |\n|---|---|---|---|\n| Deployment frequency | Whether a deployment shipped anything users wanted; a repo that releases a library weekly isn't worse than one deploying a website hourly | Splitting one release into many, deploying empty or trivial changes, counting staging as production | Lead time and the deploy dry run (`stackvitals delivery owner/repo --deploys`): what counts as a deployment and why |\n| Change lead time | Whether the change was the right one; time spent before the first commit (design, waiting for a decision) | Merging small cosmetic PRs to pull the median down, opening PRs late, skipping review | The four phases (coding, pickup, review, deploy; review split into approve and merge), the PR size mix, and merges without review |\n| Change fail rate | Failures nobody linked to a deployment; failures caught before release; which deployment really caused it (in 15 regression reports that named the release that broke, the last release before the report was it 6 times) | Not labelling incidents, fixing forward without a revert or hotfix label, hiding incidents in a private tracker | Rework rate (deployments that shipped a revert or hotfix), and where incidents come from (labels, reverts, a pager tool) |\n| Failed deployment recovery time | How bad the outage was or how many users it hit | Closing the incident issue before the fix ships, opening a new issue instead of reopening | The incident list with the deployment each is linked to, and whether a revert's fix chain agrees |\n| Deployment rework rate | Planned refactoring (it counts only unplanned fixes) | Fixes without a revert or hotfix label | Change fail rate and the incident list |\n\n## How StackVitals helps you read them\n\n- **Ranges, not just numbers:** each figure comes with how sure it is (a range for the median, the number of PRs or\n  deployments it rests on), and says \"not enough data\" below 10.\n- **Where the numbers come from:** the deploy dry run shows which workflows, releases and environments count as\n  deployments and why the rest don't; the incident list names each incident's source.\n- **Tiers that could be either:** when the range crosses a DORA tier boundary, the page says which tiers the team could\n  be in, not just one.\n- **Measured against labels:** how often the deployment and incident rules are right is on the accuracy page\n  (`docs/accuracy.md`), including where they fail.\n\n## Further reading\n\n- DORA's own guidance on the metrics and their misuse: <https://dora.dev/guides/dora-metrics-four-keys/>\n- LinearB's definitions of the six cycle-time phases: <https://linearb.helpdocs.io/article/1s38stsugw-glossary-metrics>\n","modules":"What each StackVitals module does, how to try it on any public repo from your machine, and how to set it up in your own repo. The [README](../README.md) has the overview and the [quickstart](quickstart.md) the five-minute setup.\n\n## CI Health\n\n**Try it on any public repo from your machine** (needs a GitHub token in `GITHUB_TOKEN`, or `gh auth login`):\n\n```\nstackvitals ci report owner/repo --days 14 --html ./ci-report\n```\n\n**Install it in a repo:** the StackVitals app's Setup page checks your repo and, with a GitHub token, commits the workflow, starts the first run and turns on Pages for public repos in a few clicks once GitHub is connected (connecting takes one token, made from a prefilled GitHub page; about five minutes; [docs/quickstart.md](quickstart.md); the whole file, every module id and every input: [docs/install.md](install.md)), or print it and save it as `.github/workflows/stackvitals.yml`:\n\n```\nstackvitals ci init owner/repo\n```\n\nCommitting it starts the first run. After that it runs after each of your CI runs finishes, every 3 hours, and whenever `.github/stackvitals.toml` changes on your default branch. Each run:\n\n1. reads new CI runs, including every re-run attempt and its jobs and steps;\n2. stores them in the `stackvitals-data` branch of your repo, one compressed file per day;\n3. rebuilds the dashboard in that branch's `docs/` folder.\n\nTo view the dashboard on a public repo, set GitHub Pages to deploy from branch `stackvitals-data`, folder `/docs`.\n\n**The app:** `https://OWNER.github.io/REPO/` opens one app over every module that ran. Overview shows installation health, each area's state, the trend you pick, what needs a decision, what to fix first and decisions ending soon. Each page (Insights, CI Health, Pull Requests, Delivery, Community, Dependencies, Investment, Portal, Explore) has its charts and tables; Settings, Activity, Setup, Connections, Agents and Help sit beside them, with a tab bar on phones. With a GitHub token (kept in the tab unless you ask to remember it) the app sets up more repos in a few clicks, saves settings and decisions, and tests connections. The app itself runs only its own files (no CDN); Explore and the printable pages load DuckDB and Chart.js from public CDNs ([docs/privacy.md](privacy.md)). For a private repo, `stackvitals ui owner/repo` opens the same app on your machine; `?demo=1` shows three example repos. The older pages (`ci.html`, `flow.html`, ...) stay as printable views, each with an \"Open in the app\" link. How to use it: [docs/app.md](app.md).\n\n**Your data branch:** every module writes into one checkout and the job makes one commit. The branch keeps 400 days of pull requests, issues and Portal history and 180 days of CI runs and test results (`retention-days`, `ci-retention-days`), and at most 250 MB of history in all (`size-cap-mb`; the oldest days go first; the pages never count). `squash-history: true` rewrites the branch to a single commit every 1,000 commits: the files stay byte for byte, only the git history behind them goes, and the branch is force-pushed, so leave it off if anything else commits there. Every published file carries a `schema` number. The `summary/` folder holds one row per pull request, job and test as JSON lines, small enough for a browser or any SQL engine to read without a server; `stackvitals sql --repo owner/name \"select role, count(*) from prs group by role\"` queries them with DuckDB (`pip install 'stackvitals[sql]'`; tables `prs`, `ci_jobs`, `tests` and DevLake's `devlake_*`).\n\n**Explore:** `docs/explore.html` runs SQL over the `summary/` files in your browser, with DuckDB (WebAssembly, loaded from jsDelivr at a pinned version; its worker and WebAssembly are checked against integrity hashes). Saved queries include Apache DevLake's own dashboard SQL (its DORA panels, PR cycle time, and panels from its GitHub, Engineering Overview and Contributor Experience dashboards), ported to DuckDB and run on DevLake-shaped tables of your repo (`summary/devlake_*.jsonl`), plus our own (merged PRs per week, insiders and outside contributors, flakiest jobs, tests that fail most). On public repos the tables are published beside the pages (`docs/summary/`), so it reads them from your own site.\n\n**On your machine, for private repos too:** `stackvitals ui owner/repo` keeps a copy of the data branch and opens every page, Explore included, on `127.0.0.1` (the token is used for that one git command and never stored).\n\n**Exports:** `stackvitals export owner/repo --format devlake` (DevLake's tables and its DORA SQL as `dora.sql`), `--format ctrf` (every test with its history over the window, as a CTRF report) `--format postgres` (DevLake's tables as CSV and `postgres.sql` to load them into Neon, Supabase or any Postgres) or `--format otel` (CI runs, jobs and steps as OpenTelemetry traces, OTLP JSON for Grafana Tempo, Honeycomb, Datadog or any collector; `--tests` adds failed and flaky tests as events). Several repos at once: `stackvitals sql --repo org/a --repo org/b \"select repo, ...\"`. How it grows from one repo to an org, with measured sizes and the free Postgres tiers: [docs/scaling.md](scaling.md); what it reads and from where, and the GitLab plan: [docs/connectors.md](connectors.md).\n\nThe CI health, Team Flow, Delivery, Insights, Community, Portal and Explore pages share one header. Long tables can be searched and filtered. Weekly and monthly trends are drawn as charts (Chart.js, loaded from cdnjs at a pinned version with an integrity hash); each chart is drawn from the numbers table under it, which is all you see with JavaScript off. Every flaky or failing test and job gets its own page in `docs/history/`: its runs over the kept history, newest first, with flakes marked and each run linked.\n\n**Private repos:** Pages from a private repo is public on most plans, so it would expose test names. Use `stackvitals ui OWNER/REPO`, `pages: artifact` (a download kept 7 days) or the job summary instead.\n\n**Badge:** each run writes `docs/badge.json`, the share of pushes whose checks all passed on the first try. On a public repo, add this to your README (replace OWNER/REPO):\n\n```\n![CI first try](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/OWNER/REPO/stackvitals-data/docs/badge.json)\n```\n\n**Compared with similar repos:** reports put two numbers next to the typical public repo of the same language and size: the time from a push until all its checks finish, and the share of pushes green on the first try. Each comparison names how many repos it covers. The numbers come from a daily sample of about 90,000 active public repos with 100+ stars (each read once every 4 weeks). A group is shown only when it has at least 20 repos. Under 30 pushes, the first-try share also comes pulled toward the group's pattern, with its range: on 6,623 crawled repos, predicting held-out pushes from 3 to 20 of them, the pulled share cut squared error by 29% to 3%, and a 95% range for the share of later pushes held 95% of them (the page's range is for the share itself, so it is narrower). Set `STACKVITALS_BENCH_URL` to a benchmarks file or URL to choose the source.\n\n**Peers that aren't public open source (opt-in):** with `[benchmarks] share = true`, each CI run's summary offers a link that files this repo's rounded figures (pushes to the nearest 10, time to green to the nearest 30 seconds, first-try, failing and re-run shares to the nearest point; language and star tier; no names, no code, the repo's name only with `name = true`) as an issue on StackVitals' public repo, through a labelled issue form, with a random id kept in the data branch so next month's figures replace this month's. Nothing is sent unless a person opens the link and files it; the issue shows the account that files it, and closing it withdraws the figures. The daily job turns the latest month of each shared repo into peer groups, published only with 10+ repos of a kind and counting one figure set per GitHub account, and an id only for the account that first filed it (filed figures can't be verified); with sharing on, the CI comparison uses those groups first and says so.\n\n**Single flaky tests:** save your test results with `actions/upload-artifact`, under a name containing `test-results`, `junit` or `ctrf` (or `jest`, `vitest`, `go-test`, `trx`). StackVitals reads JUnit XML from pytest, Maven, Gradle and others, CTRF JSON (Playwright and others), Jest and Vitest JSON, `go test -json` output and .NET TRX files from each run before GitHub deletes the files. A test that a retry plugin re-ran and passed within the same job (Gradle test-retry, Surefire, pytest-rerunfailures, Playwright retries, jest.retryTimes, gotestsum's re-runs) counts as flaky. Each parameter of a parametrized test is tracked on its own (pytest's `[...]`, Go subtests, .NET data rows), and when a run's order was random (Surefire's random order, `go test -shuffle`) the seed is kept, so a failure can be replayed.\n\n**Why a test is flaky, and since when:** each test's state comes from monitors you can tune, and the page says which ones hold it there and since when (kept in `ci/status.json`, with every change logged in `ci/transitions/`):\n\n| Monitor | Switches on | Switches off |\n|---|---|---|\n| Pass on retry | failed and passed on the same code twice in the window | after 14 quiet days |\n| Failure rate | 15% or more of the last 50 default-branch runs failed (at least 20 runs) | below 5% |\n| Keeps failing | the last 3 default-branch runs failed, or 80% of the last 5 | below 50% |\n\nLabels: **slow** (median over 5 seconds) and **timeout inflation** (failures take at least twice as long as the 95th percentile of passes). Change any threshold, mute a test until a date, or pin the quarantine list in `.github/stackvitals.toml` on your default branch (read on every run, so a merged change applies at once; a bad value keeps its default and is named in the job summary). Every setting is listed in [docs/settings.md](settings.md):\n\n```toml\n[ci.monitors.pass_on_retry]\nquiet_days = 7          # Trunk's default\n\n[ci.mute]\n\"tests/test_network.py::*\" = 2026-12-31   # measured, but no alerts or issues until then\n\n[ci.quarantine]\nalways = [\"tests/e2e/test_upload.py::test_resume\"]\nnever = [\"tests/test_billing.py::test_charge\"]\n```\n\n**Failure groups:** every failure gets a fingerprint in Sentry's order: the root cause's exception type (the last \"Caused by\", not the wrapper) and the innermost frames of your own code, leaving out the language, the test framework and installed packages; the message counts only when there is no stack. Failures with one fingerprint are one group across tests and runs, so the page shows \"the same failure in 9 tests since October 1\" with a link to the latest run, and a failed PR's comment says so. Checked on 53 hand-labelled real failures from netty (Java), pandas-datareader and Context-Engine (Python) and rayfin (TypeScript): no two different causes merged, 98% of same-cause pairs grouped. Works for Java and Kotlin, Python (pytest and plain tracebacks), JavaScript and TypeScript (Jest, Vitest), Go, Rust and C#.\n\n**Test reports from other CI systems:** run `stackvitals ci upload --sha \"$SHA\" --branch \"$BRANCH\" --run-id \"$BUILD_ID\" --provider jenkins test-results/*.xml` at the end of a Jenkins, GitLab, CircleCI or Buildkite job, with `STACKVITALS_TOKEN` set to a token that can write the repo and `--repo owner/name`. The reports join the same history; pass `--attempt 2` when your CI re-runs a job, and a test that failed then passed on the same commit counts as flaky.\n\n**On pull requests:** when a PR's CI fails, StackVitals posts one comment and edits it on later runs. The comment says which failures are likely flakes (the job flaked 3 or more times lately, one of them in the step that failed now, and that step's log shows a flake's sign, an outside service, the runner or a wait failing with no failing test, and names no file the pull request changed: check the error, then re-run it; the row shows that evidence), which are maybe yours (a job that flaked before, but whose failing step names a file the pull request changed), which failed because an outside service or the runner did (the failing step's log says so: re-run it), which jobs flaked before but too rarely, elsewhere or with no flake's sign to rule out the change, which already fail on the default branch with the same error (one more log read: the same step alone isn't enough, since on labelled data 4 of 20 such failures were the change's own), which never flaked but failed the way flaky checks do (a wait running out, a benchmark gate, a live service: \"maybe flaky, re-run it once to tell\"), which are checks on the PR itself (a changelog label, a description checkbox, a title prefix, in a workflow that runs again when the PR is edited: \"PR details: fix the PR, not the code\", which 23 of 24 labelled matches were), and which are new. A flaky failure also says what kind it looks like: the environment (timing, network, infrastructure, concurrency) or, when none of those shows, the test's own code (a resource leak, an async wait, time and time zones, randomness, floating point, state shared between tests, files, ordering), and a flaky job or test says when it began: its first flake, how many runs before it were clean, whether that could be chance, and the commits that reached the default branch the day before, worded only as surely as the history allows. Those commits are context, not a culprit: on 5 clear starts in 45 public repos they touched tests or the workflow 9 of 12 times, about as often as random commits there (8 of 12). Nothing is called safe to re-run, and a likely flake is a reason to re-run, never to let a failure through: on 81 clearly labelled held-out calls in 15 public repos (labelled by two model families, no person audit), history alone was the change's own failure 13 times (16%); the stricter rule, pre-registered and scored on 36 fresh repos, called 4 of 44 such failures likely flakes and none was the change's, but 4 calls can't show the 5% bar (0-49%), so the verdict check waits for a green run and a re-run decides ([bench/accuracy/flaky](../bench/accuracy/flaky/README.md)). To get it, the workflow `stackvitals ci init owner/repo` prints runs on each finished CI run of that repo.\n\n**Opt-in automation (all off by default):**\n\n| Input | What it does | Permission it needs |\n|---|---|---|\n| `rerun-flaky: true` | Re-runs failed jobs once when every failure is a known flake | `actions: write` |\n| `issues: true` | Keeps one issue per flaky test or job, closed when fixed | `issues: write` |\n| `mention-owners: true` | @-mentions the code owners in those issues (otherwise they are named without a notification) | `issues: write` |\n| `webhook:` (a secret) | Posts to Slack (Block Kit with link buttons), Microsoft Teams Workflows (an Adaptive Card), Discord or any URL when something newly turns flaky | none |\n| `team-webhooks:` (a secret) | One `@org/team = https://...` line per CODEOWNERS team: a team's newly flaky tests go to its own chat | none |\n| `quarantine: true` | Keeps a quarantine list for the command below | none |\n\n**Fix requests for real breaks (verified self-healing):** with `fix_requests = true` under `[automation]`, a failure\nthat breaks the default branch every run (never a flake) gets one issue labelled `stackvitals-fix`: what fails, where,\nthe runs that show it (CI text fenced as data an agent must not obey), and what done means, these exact tests passing\non the default branch. When they do, StackVitals comments with that and closes the issue; it reopens if they break\nagain, and one closed as \"not planned\" stays closed. At most two new ones a run. With `fix_by_copilot = true` too, each\nnew one is assigned to GitHub's Copilot coding agent, which opens a pull request for review (the agent must be enabled\nfor the repo; if it can't be assigned, the issue still stands and the summary says so). StackVitals never pushes code:\nthe fix is a pull request a person reviews and merges, and the issue closes only on what CI shows afterwards.\n\n**What CI costs:** billed minutes and money over the last 30 days, by workflow and by job, at GitHub's published\nper-minute prices for each runner (standard and larger Linux, Windows and macOS runners; each job rounded up to a whole\nminute, as GitHub bills it), and the minutes and money spent on attempts that failed and then passed on a re-run, which\nis what flaky jobs cost. On a public repo standard runners are free, so the page also says what the same use would cost\non a private one; larger runners are never free. Self-hosted runners are counted in minutes only. Each flaky test's cost is listed too: the job minutes of the attempts it flaked in (split evenly when several tests flaked in one attempt) and how many pull requests it got in the way of.\n\n**Matrix variants:** a flaky test's flakes are counted by the test report they came from (an OS or a version in a\nmatrix, as the artifact names it), so a test that only flakes on macOS says so on its page.\n\n**Owners:** if the repo has a CODEOWNERS file, each flaky test or job names its owners on the dashboard, on its history page, in its issue and in the webhook alert. They are looked up with GitHub's own rules (the last matching line wins, and owners GitHub ignores, such as people without write access, are left out) for the test's file, else for the workflow file. A test's file comes from its id (`tests/test_api.py::test_x`), or from its class name matched against the repo's files (`tests.test_api`, `com.acme.ApiTest`); when two files fit, the workflow's owners are used instead.\n\n**PR checks that read the PR itself:** a job that reads a PR's title, description or labels, in a workflow that also runs when those change (a PR template check, say), can pass and fail on the same code. Same-code differences in such a job are not counted as flakes; a failed job that passes on a re-run still is, and test reports are always judged. Each workflow file run by pull requests is read once a week.\n\n**Catch a flaky test before it merges:** run the tests a pull request adds or changes several times; the step fails when one of them both passes and fails:\n\n```\nstackvitals ci burn-in --base origin/main --times 5 --reports \"out/*.xml\" -- pytest {files} --junitxml out/results.xml\n```\n\n`{files}` becomes the added or changed test files (pytest, Go, Jest/Vitest, JUnit-style Java, Kotlin and C# names); with none, nothing runs. After `git fetch origin stackvitals-data` it also knows which tests the default branch has never run, and says how many of the repeated tests are new; `--only-new` fails the step only when a new test both passes and fails, since known flaky ones are quarantine's job. It also shows a fix on the pull request itself: a test the data branch calls flaky that passes every repeat is reported with the chance it would have anyway at its old failure rate (a test that failed 40% of runs passes 10 in a row 0.6% of the time), and the fix counts as shown under 5%; otherwise it asks for more repeats. A test failing every time is listed, and your normal CI run fails for it.\n\n**Quarantine command:** wrap your test command like this:\n\n```\nstackvitals ci quarantine --reports \"out/*.xml\" -- pytest --junitxml out/results.xml\n```\n\nIt passes only when every failure is a quarantined test. If the quarantine list can't be read, nothing is quarantined.\n\n**Or inside the test runner:** `stackvitals ci quarantine --print jest` (or `vitest`, `go`, `pytest`, `playwright`, `maven` for Surefire, `dotnet`) prints, one per\nline, the arguments that keep quarantined tests from failing a run, so the CI command stays as it is:\n\n```\nmapfile -t skip < <(stackvitals ci quarantine --print jest) && npx jest \"${skip[@]}\"\npytest $(stackvitals ci quarantine --print pytest)\n```\n\nJest and Vitest get a name pattern that skips each quarantined test by its name (inside `describe` blocks too; another\ntest ending in the same name is skipped as well), `go test` a `-skip` pattern (top-level Test, Fuzz and Example names),\nand for pytest the list is written to `.stackvitals-quarantine.json` and a plugin loaded that still runs each quarantined\ntest (matched by its file, class and name) but marks it an expected failure, so its result stays in the report. Only\ntests on the list are touched; a test failing every time (broken, not flaky) is never put on the list by StackVitals. Quarantine can end: `max_days` under `[ci.quarantine]` takes a flaky test out after that many days, so it fails builds again until someone fixes it, and the CI page lists it as overdue; tests someone quarantined by hand (or listed under `always`) stay.\n\n**Run only the tests a change can affect:** `stackvitals ci impact --base origin/main --tests 'tests/**/*.py'`\nprints the test files to run (or `all`, or nothing for a change no test touches, like docs). A test file is picked when\nit changed, when its name matches a changed file (`cart.py` and `test_cart.py`), or when it failed before on pull\nrequests that changed the same folder; a change to build, lock or settings files (`pyproject.toml`, `package.json`,\n`conftest.py`, `.github/`...) runs everything. No coverage agent is needed. Each run of the Action learns the folders\nand failures from the repo's own pull requests (`ci/impact.json` on the data branch, which the command reads after\n`git fetch origin stackvitals-data`) and replays the rule on them, each judged with only what was known before it:\nthe CI page shows how many of their real failures it would have run and how much it would have skipped. Treat it as\na fast first pass until that number is high on your repo. On public repos it can't be measured yet: of 329,177 test results read from five repos that keep test reports, one failure was on a pull request run ([bench/accuracy/impact](../bench/accuracy/impact/README.md)).\n\n**Retry only what failed, never what is broken:** `stackvitals ci retry --reports 'out/*.xml' -- pytest --junitxml\nout/r.xml --then pytest --lf --junitxml out/r.xml` runs the suite and, when it fails, reads which tests failed. If one of\nthem is a test the data branch calls broken (failing every time on the default branch), the job fails at once and\nsays so, since a retry would only hide it; otherwise your runner's own \"only failures\" command (`pytest --lf`,\n`jest --onlyFailures`, `rspec --only-failures`) runs up to `--times` times, and the job passes when a retry does. The\nretried tests' results land in the same reports, so their flakes are counted.\n\n**Check a report before CI reads it:** `stackvitals ci validate 'build/test-results/**/*.xml'` reads each file as the\nAction does and names what would quietly break things: no tests in it, test cases with no name, none naming a file or\nclass (owners, shards and test impact need one), every duration 0, a suite count that doesn't match its cases, or a\ntimestamp from another week (an old report left in the folder). It exits non-zero on errors, so it can run as a CI step.\n\n**Shards that finish together:** split your test files into N groups by their recorded times, read from the newest green run on the default branch that kept test reports (or from report files on your machine with `--reports`):\n\n```\nstackvitals ci shards owner/repo --n 4                       # all four shards and their expected time\npytest $(stackvitals ci shards owner/repo --n 4 --index 2 --py-paths)   # one shard, for a CI matrix job\n```\n\nFiles with no recorded time yet (`--files \"tests/**/*.py\"`) count as a typical file. An empty shard prints nothing and says so, since most test runners run everything when given no files.\n\n**Coverage, from the reports CI already uploads:** Cobertura (`coverage.xml` from coverage.py, Istanbul or Go\nconverters), LCOV (`lcov.info`) or JaCoCo XML in an artifact whose name says coverage. Each run's total is kept, so the\ndefault branch's line coverage has a daily trend; on pull requests, StackVitals reads the PR's diff and says how many of\nthe added lines ran (\"patch coverage\") and which did not. With `min_patch` under `[ci.coverage]` (a share, 0 is off) the\nverdict check fails when fewer added lines ran than that. Line hits are read once and never stored; only the totals are.\n\n**New and skipped tests:** a test is named as new the first time a report shows it (after the first run, which only learns what exists), with how it has done since; the app lists tests added in the last 14 days, the ones already failing or flaky first. Tests skipped on the default branch are listed with since when (as far back as the stored reports go) and in how many matrix jobs, newly skipped ones marked. Slow tests and jobs carry their daily median time.\n\n**An investigation prompt per failure group:** the app's Failure groups tab has a \"Copy prompt\" button, and the MCP tool `failure_group` returns the same text: what failed, where, how often, the runs that show it, which of its tests are known flaky, and what to find out. Paste it into Claude Code, Codex or Copilot to start from the history instead of one log.\n\n**In your editor:** `stackvitals editor vscode` (or `cursor`, `vscodium`) installs a small extension that marks flaky and broken tests where they are declared (Python, Go, JS/TS, Java, Kotlin, C#, Rust), with how often each flaked, its usual failure and a link to the latest run that shows it. It reads `docs/data.json` from the `stackvitals-data` branch through your own git, so it needs no token and works for private repos; \"StackVitals: Refresh flaky tests\" fetches the newest.\n\n**For AI agents:** `stackvitals serve` is one MCP server, a memory for coding agents: what this repo's history knows that a live search can't (is this failure a known flake, already failing on main, how changes get merged here, who should review, how an upgrade went elsewhere). By default it offers 11 tools (`ci_failure`, `is_known_flake`, `ci_health`, `repo_brief`, `preflight`, `who_should_review`, `pr_queue`, `check_package`, `scan_lockfile`, `insights`, `delivery_health`), which keeps an agent's context small (in an informal check, on 11 everyday questions, a Haiku agent picked the right tool from the 11 and from all 35 alike); `stackvitals serve --tools all` (or `STACKVITALS_MCP_TOOLS=all`) adds the rest; the skills below and the MCP registry entry use all of them. All tools: the MergeScout tools plus `ci_health`, `is_known_flake`, `ci_failure` (now with each failure's group), `ci_slowest` (with daily medians, new and skipped tests), `test_status` (state, monitors, since when, mutes, group) and `failure_group` (with an investigation prompt). The skill in `skills/stackvitals-ci/` tells Claude Code or Codex to check history before \"fixing\" a failing test; `skills/` has one for every module (flow, delivery, deps, community, portal, contribute), each naming only tools the server has (a test checks). Every answer carries `as_of` (when its numbers are from) and `sources` (where they came from). StackVitals is the evidence an agent reads, not a replacement for one: on a single failure inside GitHub's log retention, an agent reading the log does as well on its own; the edge is kept history and comparisons with similar repos, which an agent can't rebuild in one chat ([bench/accuracy/agents](../bench/accuracy/agents/)).\n\n**What counts as flaky:** a job that failed and then passed with no code change. Either a re-run of the same run passed, or another push or PR run of the identical code, on the same branch or tag, passed. Each flaky item links to the runs that prove it.\n\n**What is ignored:**\n\n- outages, where most jobs fail together;\n- \"gatekeeper\" jobs that only fail because another job failed or was cancelled;\n- scheduled and triggered runs;\n- GitHub-managed workflows such as Dependabot.\n\n**States:**\n\n| State | Meaning |\n|---|---|\n| flaky | 2 or more flakes in 30 days |\n| one flake seen | 1 flake in 30 days |\n| fixed | at least 14 quiet days and 10 or more clean runs after being flaky |\n| keeps failing | the last 3 runs on the default branch failed |\n\n**Security:**\n\n- GitHub is read with the built-in `GITHUB_TOKEN` only; the optional Jira and Linear keys live in your repo's secrets, are sent only to your own Jira Cloud site or Linear, and are only used to read unless you turn on write-back ([docs/connectors.md](connectors.md)). Everything a run sends anywhere: [docs/privacy.md](privacy.md). The template asks for `contents: write` (the `stackvitals-data` branch), `pull-requests: write` (the PR comment, review requests and size labels), `issues: write` (the weekly digest, commands and flaky-test issues), and read access to `actions`, `checks`, `statuses` and `deployments` ([docs/install.md](install.md)). Opt-in features need one more: `actions: write` to re-run known flakes, `issues: write` for flaky-test and alert issues.\n- Job names can come from fork PRs, so they are escaped everywhere they are shown.\n- Nothing is sent anywhere outside your repo.\n\n## Insights: what to fix first\n\nWhen Team Flow runs, each job also writes `docs/insights.html` and `docs/insights.json`, and the job summary starts with the top three. Each finding is one sentence with its number, how many pull requests it rests on, links to the evidence and what to do, and they are ranked by the hours of waiting (or outage) at stake:\n\n- a CI job or test that failed and then passed on the same code, by the hours it kept pull requests red (the runs linked);\n  when its flaking clearly began on a day, the finding names the pull requests merged just before (\"right after #N\n  merged: look there first\") and links them, a place to look and never a blamed change;\n- review requests not started after 3 days, and older ones (over 30 days) to review or close;\n- how long merged pull requests sat with failing checks, and dependency updates that did;\n- an incident (a revert, a hotfix or an `incident` issue) and the deployment before it;\n- outside contributors waiting much longer than insiders for a first review.\n\nThe page ends with an estimated opportunity: the hours at stake across the findings, each hour counted once (a flaky job's hours sit inside the red-CI hours, so they are shown as part of them, not added). It is an estimate of what fixing them could save, not a measured saving.\n\nEach finding keeps how big it was when first seen and the days it ranked first (findings someone acknowledged or snoozed never count as the top one). Under the list, a track record says how many findings ranked first in the last 90 days, how many later shrank to half their hours or less, and how many cleared (not seen for a week): whether the top finding gets acted on, read over months. The page is built from the public numbers only, so it never names a reviewer. For any public repo from your machine: `stackvitals insights owner/repo` (MCP: `insights`).\n\n**Does ranking flaky jobs by PR-hours beat counting failures? Not shown, so not claimed.** We ranked jobs from 60 days of history and measured how much of the next 30 days' flaky waiting each ranking's top three covered, against rankings by failure count, pass-on-retry count and PRs affected, on pip, pytest, guidellm, aiohttp, dask/distributed and astronomer-cosmos. Waits that only a re-run cleared were rare on pull requests (0 to 22 a month per repo) and moved between matrix jobs from month to month, so no ranking predicted the next month well and none beat the others. One difference held: the count-based rankings put label and PR-template gates in their top three (2 of 18 picks), ours never did, because checks that read a pull request's own labels or title don't count as flaky CI here; the test also found a label gate read through a script that the filter had missed (fixed).\n\n**Alerts.** Thresholds on the last 28 days, set under `[alerts]` in `.github/stackvitals.toml`; the defaults are LinearB's 2026 \"good\" band (P75 pickup 4 h, review 14 h, cycle time to merge 72 h) and a 15% change failure rate, and `0` turns one off. A heads-up comes at 80% of a threshold and a miss once the number's likely range (95%) is past it; a number past the threshold whose range still reaches under it is a heads-up that says so. Each is sent once (and once more when it recovers); with a new threshold the job summary says how many times it would have fired over the last 90 days. They go to the `webhook` input (Slack, Discord, Microsoft Teams Workflows, or any URL as JSON) and, with `channels = [\"webhook\", \"issue\"]`, to one \"StackVitals: delivery alerts\" issue kept up to date.\n\n**Fixing, only when you turn it on:** re-run flaky jobs (`rerun-flaky`), quarantine (`quarantine`), one issue per flaky test (`issues`), name owners (`mention-owners`), and a `stackvitals: waiting for review` label on pull requests whose asked-for review has not started after 3 days, taken off once one has (`label-stuck`). With `verdict_check = true` under `[automation]` (and `checks: write`), every pull request gets a \"StackVitals verdict\" check that passes when its newest code is green (known flakes are re-run, with `rerun-flaky` automatically, never let through); it stays in progress while other workflows on that commit still run, and a cancelled or timed-out run never passes. Require it in your branch rules instead of a flaky job; merge-queue commits get it too (list every CI workflow under `workflow_run` so each finished run updates it).\n\n**How the verdict decides, exactly:**\n- Every workflow on the pull request's newest commit has finished and passed: **pass**. That includes a job that failed\n  and then passed when re-run on the same commit, the way GitHub's own required checks treat a re-run.\n- Anything failed, was cancelled or timed out on the newest commit: **fail**, whatever the comment called it. A known\n  flake is never let through; it is re-run (by hand, or with `rerun-flaky`), and the check passes only if the re-run does.\n- Other workflows on that commit still running: **in progress**.\n- With patch coverage under `[ci.coverage] min_patch`: **fail**.\n- **Strict mode** (`verdict_strict = true` under `[automation]`): when the comment called a failure on this commit\n  likely from the change (or flaky only before, in another step) and a re-run of the same commit then passed, the check\n  stays **failed** until someone with triage access comments `/stackvitals accept rerun` on the pull request. A green\n  re-run doesn't prove the change clean: on public CI history, between 3 of 45 and 15 of 52 known-flaky calls that a\n  re-run turned green were the change's own failure, depending on the labeller\n  ([bench/accuracy/replay](../bench/accuracy/replay/README.md)).\n- On a private repo, set the repository variable `STACKVITALS_EVERY_CI_RUN` to `true` so passing CI runs start a job and\n  the check can pass at once ([scaling.md](scaling.md#actions-minutes-private-repos)).\n\n**Alerts within seconds:** the Action runs on a schedule, so for in-the-moment messages (checks failed on a pull request, a review requested, a pull request approved) deploy the small Cloudflare Worker in [`worker/`](../worker/README.md) to your own free account; it checks GitHub's signature on every request.\n\n## Decide on GitHub, hear back every week\n\n**Commands.** Anyone with triage or write access can type a command in a comment on any issue or pull request. StackVitals answers on the issue and logs who decided what, when and why in the `stackvitals-data` branch (`decisions/log.ndjson`); every page shows it.\n\n| Command | What it does |\n|---|---|\n| `/stackvitals quarantine` (on a flaky-test issue, or with `test:<id>`) | puts the test on the quarantine list for 30 days |\n| `/stackvitals unquarantine test:<id>` | keeps a flaky test off the list, so it still blocks |\n| `/stackvitals mute job:\"ci.yml / lint\" for 7d` | still measured, but no alert and no new issue |\n| `/stackvitals acknowledge` | the team lives with it: no alert and no issue until someone reopens it |\n| `/stackvitals snooze finding:review_wait for 14d` | takes a finding off \"what to fix first\" until then |\n| `/stackvitals exclude` (on a pull request, or `#123`) | leaves it out of every number; the `stackvitals: exclude` label does the same |\n| `/stackvitals confirm test:<id>` or `dispute ... because <why>` | says StackVitals' verdict on it was right or wrong; it changes nothing else, and CI health shows how often your team confirmed each kind of verdict, with a range |\n| `/stackvitals accept rerun` (on a pull request) | in the verdict's strict mode, lets a green re-run pass after a failure that looked like the change's own |\n\nAdd `for 14d`, `for 2 weeks` or `until 2026-12-31`, and `because <reason>`, to any command; temporary decisions end on their own and the digest lists those ending within a week. Closing a flaky-test issue as *not planned* counts as acknowledged, and the issue stays closed; reopening it undoes that. An issue closed as done stays closed unless the test flakes again after the close. A command whose run GitHub cancelled while it waited (a burst of commands) is picked up by the next run.\n\n**Requests from the app.** The app's buttons open a prefilled issue with a `stackvitals` block. The workflow checks that you have the access, applies it, changes `.github/stackvitals.toml` when asked (or opens a pull request when the branch is protected), answers and closes the issue. Issues from people without access start no run. With \"Apply now with my token\" the app starts the workflow itself instead (a fine-grained token with Actions read and write on that one repo, kept in the page's memory). AI agents get the same link without changing anything: `stackvitals propose owner/repo --set alerts.pickup_p75_hours=6 --decide \"/stackvitals mute test:<id> for 7d because <reason>\"` (MCP: `propose_change`) prints it for a person to submit.\n\n**Weekly digest.** Every Monday after 06:00 UTC, one comment on the pinned \"StackVitals weekly digest\" issue (subscribe to it for e-mail) and, with `channels = [\"issue\", \"webhook\"]`, your webhook: what changed this week in CI and delivery, what to fix first, newcomers waiting, and decisions ending soon. Nobody is mentioned. Each number's latest week is checked against the four before it and judged against how much that repo's weeks usually swing; only changes that survive a false-alarm control (Benjamini-Hochberg at 10%) are reported. On 26 public repos with shuffled weeks, 8% of digests flagged anything, against 95% for a digest that reports every difference; a steady week says how big a change it could have shown. The checks, the replay and its results: [bench/accuracy/uncertainty](../bench/accuracy/uncertainty/). Set the day, hour and channels under `[digest]`.\n\n## Insights: red CI on pull requests\n\nWhen CI Health and Team Flow both run, each job also writes the red-CI part of `docs/insights.json` and a section of the job summary: how many of the people's pull requests merged in the last 90 days sat ready for review with a failing check on their newest commit, for how long in all and per PR (with ranges), which waits ended only with a re-run that passed on the same code (flaky checks) and the jobs behind them, the longest waits with a link to the run that started each one, and open PRs that are red right now. The clock starts at the first failing job, as GitHub's own PR view turns red, and stops at a new commit, a passing re-run or the merge. Time as a draft, checks that read the PR's own title or labels, and `pull_request_target` runs (they run the base branch's workflow) never count. For any public repo from your machine: `stackvitals insights owner/repo` (MCP: `insights`).\n\nChecked on 37 pull requests of pip, pytest and guidellm against GitHub's Checks API, a different source from the runs it reads: 36 matched exactly; the last differed only because GitHub no longer lists a commit that was force-pushed away. Across those repos 14-18% of merged PRs went red, from 8 hours in all on pip (most of it flaky re-runs) to 222 hours on guidellm.\n\n## Delivery: cycle time and DORA\n\nWhen Team Flow runs, each scheduled job also writes `docs/delivery.html` and `docs/delivery.json`. For any public repo from your machine: `stackvitals delivery owner/repo` (`--json` for the numbers). MCP: `delivery_health`, `cycle_time`, `stuck_prs`, `reviewer_load`.\n\n**Check what counts before trusting it:** `stackvitals delivery owner/repo --deploys` (and the \"What counts as a deployment\" box on the app's Delivery page) shows which source counts as deployments and why the others were passed over, each workflow counted or skipped with the reason, and each environment counted as production or shown as staging or publishing. These numbers describe a team's system, never a person: [docs/dora-guide.md](dora-guide.md) says what each one can't tell and how each gets gamed.\n\n- **Cycle time in LinearB's four phases** (as its [metrics glossary](https://linearb.helpdocs.io/article/1s38stsugw-glossary-metrics) defines them): coding (first commit to opened), pickup (ready for review to the first comment or review by someone else), review (that to merge) and deploy, with review also split into LinearB's time to approve and time to merge, with P50, P75, a range for the median and the number of PRs. Drafts count from when they were marked ready; bots and coding agents are left out; PRs over 60 days, and above the 98th percentile once there are 30+, are left out as LinearB does. Under 10 PRs a phase says \"not enough data\".\n- **DORA's five metrics.** The first four (deployment frequency, change lead time, change fail rate, failed deployment recovery time) use DevLake's definitions and table shapes, counted as its DORA dashboard counts them (an incident blames the last production deployment before it). Beside the change fail rate the page also shows the rate with each revert blamed on the deployment that shipped what it undid, because on 89 checked reverts the last deployment didn't contain the reverted change 71 times: deployment days per week and month (weeks with none count as zero), median lead time of the PRs each deployment shipped, the share of deployments followed by an incident, and the time from a failed deployment to the fix. Tiers come from DevLake's 2023 table, or 2021 if you choose it. Each key shows its likely range (95%: Wilson for rates, order statistics for medians), and when that range crosses a tier line, the page says which tiers it could be. DORA's fifth metric, **deployment rework rate** (an instability metric DORA added in 2024), is the share of deployments that shipped a revert, a hotfix or a pull request whose title names an incident issue; it has no tier until DORA publishes one, so the page shows the number and how many deployments it covers.\n- **What DORA's numbers can't tell:**\n  - They describe a team's delivery system, never a person, and StackVitals never ranks people by them.\n  - Used as targets, they get gamed: tiny deploys that ship nothing, incidents filed late or not at all, hotfixes\n    logged as features.\n  - A tier depends on what counts as a deployment: on 4,618 public repos it changed between \"every merge\" and \"every\n    release\" for 62% ([bench/accuracy/dora](../bench/accuracy/dora/)). So every tier shows its definition and source,\n    and its range when that crosses a tier line.\n- **Deployments, with no setup, and how sure it is:** GitHub deployments to a production environment (sure), else a deploy workflow that runs on a good share of the default branch's commits (a guess), else published versions (alpha, beta and rc tags left out; several packages released from one commit count once), else release workflows run on a version tag. Workflows that only prepare, preview, check, sign, notify or open release PRs are skipped: on 51 workflows of 15 repos it was not tuned on, 85% of the ones it kept really ship (47% when matching the name alone). No setting-free rule is right everywhere: on 8 public repos it picked the right source for 5; the others deploy from another repo or a hosting service. So the page names the source, says when it is a guess, lists what it counted, and one line in the settings fixes it.\n- **Merge queue health, which GitHub doesn't report:** how long pull requests wait from entering the merge queue to merging (median and 9 in 10), why they were taken out (merge conflicts, failed checks), and which removals followed a merge-group run that failed only on jobs CI Health knows to be flaky.\n- **Where the work went:** each merged pull request by people or coding agents in the last 6 months, sorted into feature, fix, maintenance, security or docs work, month by month, with how each was decided: your own label rules first (`[investment.labels]`, like `maintenance = [\"area: platform\"]`), then a security label, the linked Jira or Linear issue's type, common labels, its changelog fragment (`news/123.bugfix.rst`, `changelog/`, `CHANGES/`), the title's prefix (`fix:`, `docs:`), the files it changed (only docs, tests or CI), and the title's first verb. Release PRs count as maintenance, and update bots' PRs are counted apart. On pip, pytest and aiohttp 3 to 7% stayed unsorted; repos whose titles name a component instead of a kind of work keep more unsorted until a label rule is set.\n- **Jira Cloud and Linear, with one optional key each:** issue lead and cycle time by type, sprints and cycles (scope, done, carried over, added after start), tracker incidents in DORA, and pull requests linked to issues by key. The key stays in your repo's secrets and is only used to read; setup and details in [docs/connectors.md](connectors.md), and the app's Integrations section shows each one's status.\n- **PagerDuty or incident.io, with one optional key:** their incidents then count in DORA's change failure rate and time to restore instead of labels, reverts and hotfixes, and every page names the source ([docs/connectors.md](connectors.md)).\n- **GitHub Copilot usage, kept day by day:** with an org App that can read the Copilot usage reports, the org's active users, suggestions accepted and pull requests by Copilot are copied into the data branch each day, so the history outlasts GitHub's own, and shown beside what happened after AI-assisted pull requests merged.\n- **Incidents without a paid tracker:** issues labelled `incident` (or `outage`, `sev1`, `p0`...), merged revert PRs and merged `hotfix` PRs, each tied to the last deployment before it.\n- **Release and GitFlow branches:** a pull request merged into `develop` counts once a deployment's commits hold its\n  merge commit (develop merged into main later), with no setting. `production_branches` (globs, like `release/*` or\n  `production`) also counts deploy workflow runs on those branches and pull requests merged into them.\n- **Any dates:** pick any range in the last year and the four DORA numbers are counted for it in the page, from\n  compact rows of each deployment (finished, followed by an incident, recovery) and each shipped pull request (lead\n  time), with DevLake's rules: deployment days in the median week (weeks with none count), the median lead time, the\n  share of deployments followed by an incident and the median recovery.\n- **Week by week:** lead time, change fail rate, recovery and deployments for each of the last 13 weeks, each week\n  counted with the same DevLake rules as the window, in the chart picker at the top of the page.\n- **Deployments:** the window's last 30 deployments, each with the pull requests it shipped, how many people wrote\n  them, their median lead time and the incidents blamed on it, linked to the commit on GitHub.\n- **Deploy attempts:** every deploy in the window, failed and cancelled ones too: how many succeeded and how long a\n  successful one took at the median (Deployments API: from creation to success; deploy workflows: run start to end).\n  DORA still counts successful deployments only, as DevLake does; this line shows what didn't make it.\n- **Your own incident rules** (`[delivery]` in `.github/stackvitals.toml`): `fix_patterns` marks fix PRs by their\n  title or branch, with one group naming the PR they fix (`\"^fix.*#([0-9]+)\"`, `\"^hotfix/([0-9]+)-\"`), and the\n  incident blames the deployment that shipped that PR; `incident_sources` picks exactly which sources count (`issues`,\n  `reverts`, `hotfixes`, `fixes`, `pager`, `tracker`, `recorded`), so a team with a pager can still count its reverts;\n  `ignore_reverts_of` leaves out reverts of PRs that touched only some paths (`docs/**`) or came from some authors\n  (`dependabot[bot]`). Teams that ship every merge can set `source = \"merges\"`; it is never picked on its own, since\n  on our checked repos every merge matched real deployments 8% of the time.\n- **Stuck PRs** by LinearB's published cut-offs (review asked 3+ days ago and not started, open 30+ days, 10+ comments), plus failing checks nobody acted on for a day; PR size bands, merged without review, reviews under 20 minutes, review depth and merge frequency.\n- **By team:** cycle time per team in your settings and per owning team in CODEOWNERS (single people are never listed). Teams in your settings also get all four DORA metrics, from the deployments that shipped their members' pull requests (DevLake's DORA by team).\n- **Code that lasted (AI-assisted work tab):** of the lines a squash-merged pull request added, the share git blame\n  still gives to it in the default branch 30 and 90 days after it merged, for people's and agents' pull requests.\n  Each pull request is measured once at each age (about 12 a run, through GitHub's API), pull requests merged with a\n  merge commit are left out (their lines belong to other commits), and so are ones touching more than 8 files.\n- **Releases:** for each of the last six releases (pre-releases left out), the pull requests merged since the release\n  before it, the share that were fixes, which of them a later revert undid, and the incidents in the two weeks after it\n  went out (\"so far\" until two weeks have passed). DevLake's release quality dashboard counts the first two; the rest\n  is what happened to the release afterwards.\n- **Bugs:** the bug backlog week by week, as DevLake's weekly bug retro shows it: bugs opened and closed each week, how\n  many were still open at the end of it, and the oldest open ones. Bugs are GitHub issues with a bug label (`bug`,\n  `type: bug`, `kind/bug` and the like), plus Jira and Linear bugs when connected; cancelled ones don't count.\n- **Change risk:** which open pull requests look risky by this repo's own history. A merged pull request counts as\n  having caused trouble when a revert undid it, it shipped in a small deployment that failed, or a fix (by its title or\n  a label) changed the same files within 14 days (the lightweight form of SZZ; files most pull requests touch, like a\n  lockfile or the changelog, link nothing). Each open pull request gets a chance of needing a fix, from its folders'\n  record, its size, whether its author has changed those folders before, whether tests changed and whether a coding\n  agent wrote it, with the reasons. The score is learnt on the older 70% of merged pull requests and checked on the\n  newest 30% first, and shown only when it ranks trouble there clearly better than chance (AUC 0.65 or more);\n  otherwise the tab says so. On 13 public repos it reached AUC 0.72 to 0.80 on six (pytest, pip, aiohttp,\n  astronomer-cosmos, guidellm, vercel-labs/run), 0.65 on nodejs.org and about chance on three, which is why each\n  repo is checked on its own.\n- **Working calendar:** with `[calendar] weekends = [\"saturday\", \"sunday\"]` and `holidays = [\"2026-12-25\", ...]`, those\n  days (whole UTC days) are left out of every review phase and cycle time, so a pull request opened on Friday evening\n  isn't charged the weekend. DORA's numbers stay in calendar time so they still match DevLake and the DORA report, and\n  a working-time lead time is shown beside them.\n- **By service (monorepos):** name the services inside the repo under `[services.<name>]` with `paths` (CODEOWNERS-style\n  patterns) and, optionally, `deploy` (a pattern matched against each deployment's environment or workflow name). Each\n  service gets its review phases, and with `deploy` its deployments, lead time and change failure rate, counted the\n  same way as the repo's. A pull request touching two services counts in both; the repo's own numbers don't change.\n- **Compared with similar public repos:** your P75 pickup, review and opened-to-merged, measured the same way as for thousands of public repos of the same language and size in the daily crawl, with the typical repo's value, a 95% range for it, your place, and the share of those repos inside LinearB's good band. The page says who is in the comparison and what it can't see (private teams; pickup counted from opening, since the crawl doesn't keep draft history).\n- **AI-assisted work, compared like with like, totals only:** coding agents' PRs (an agent's account, or a branch an agent opened such as `codex/`) and people's AI-marked PRs (an AI co-author trailer or a coding tool's label such as `claude-code-assisted`) are kept apart. A person's AI-marked PRs are compared with the same person's other PRs of a similar size; agents' PRs with people's PRs of the same month and size; each on reverts, CI failures, failed deployments, merges and merges by someone other than the author, with a 95% range, the share of PRs that found a match, and the gap it could have detected when it finds none. Small repos read \"too few to compare\", with the same like-for-like gap across similar public repos beside it. On 1.46 million crawled PRs, agents' PRs trail people's by 9 points in merges by someone else, and the gap is gone within the same repo, month and size (+0.5, -0.0 to +1.1); by agent, Copilot's and Cursor's agent PRs are merged at all about 14 points less often like with like (agents' PRs and people's AI-marked PRs are separate groups and never pooled) ([bench/accuracy/ai](../bench/accuracy/ai/)). An association, never a cause (matching removes repo, month, size and author differences, not the choice of what work goes to an agent), and never a person.\n\n**Working agreements:** a team's own rules under `[agreements]` (first review within N hours, merged within N days, PRs under N lines, PRs link an issue, reviewed before merge), all off until set. The Delivery page lists the pull requests of the last 14 days that broke one (open ones already past a time limit too), the digest counts them, and the app previews how often a rule would have been broken week by week.\n\n**Settings** in `.github/stackvitals.toml` (all optional; the full list is in [docs/settings.md](settings.md)):\n\n```toml\n[delivery]\ndeploy_workflow = \"(?i)(deploy|release|publish|push-image)\"  # which workflow names ship\nproduction = [\"production\", \"prod\", \"live\"]                    # deployment environments that count\nincident_labels = [\"incident\", \"outage\"]\nhotfix_labels = [\"hotfix\"]\nsource = \"auto\"          # or deployments, workflows, releases\ndora_report = \"2023\"     # or 2021\n\n[services.api]                      # a monorepo's services (optional)\npaths = [\"services/api/\", \"libs/shared/**\"]\ndeploy = \"api-prod\"                 # its deployments' environment or workflow name\n\n[identity]\nbots = [\"release-captain\"]          # accounts to treat as bots\npeople = [\"build-bot-jane\"]         # accounts to treat as people\n\n[identity.aliases]                  # an old login, or one person's id in another tool, names the GitHub login\n\"old-login\" = \"new-login\"\n\"jira:5b10a2844c20165700ede21g\" = \"amy\"   # a Jira account id (Profile, or the assignee in an issue's JSON)\n\"linear:2f6b1c3e-...\" = \"amy\"             # a Linear user id\n\"slack:U0123ABC\" = \"bob\"                  # a Slack member id (Profile, More, Copy member ID)\n\"discord:81384788765712384\" = \"bob\"       # a Discord user id (Developer Mode, Copy User ID)\n\"bob@acme.com\" = \"bob\"                    # an email Jira or Linear shows for the assignee\n\n[identity.teams]\nweb = [\"amy\", \"bob\"]\n\n[alerts]\npickup_p75_hours = 4     # P75 over the last 28 days; 0 turns an alert off\nreview_p75_hours = 14\ncycle_p75_hours = 72\nchange_failure_rate = 0.15\nchannels = [\"webhook\", \"issue\"]\n```\n\n**One project across several repos:** `stackvitals project web org/frontend org/api org/worker` pools the repos'\nDevLake tables into one DevLake project and runs DevLake's own DORA SQL on it: deployment days across all of them,\nthe median lead time of every pull request any of them shipped, and the share of all their deployments followed by an\nincident, next to each repo's own (`--json` for the numbers; needs `pip install 'stackvitals[sql]'`). That is what a\nDevLake project of those repos shows, with no DevLake, MySQL or Grafana running.\n\n**Open in DevLake's shape:** `stackvitals export owner/repo --format devlake` writes DevLake's domain tables (`pull_requests`, `project_pr_metrics`, `cicd_deployment_commits`, `cicd_pipelines`, `cicd_tasks`, `cicd_pipeline_commits`, `incidents`, `issues` and the project links; every run also writes them to the data branch's `summary/` as `devlake_<table>.jsonl`, with the last 90 days of CI runs) as JSON lines, plus `dora.sql`: DevLake's own DORA dashboard SQL ported to DuckDB. Run `duckdb < dora.sql` in that folder (or install `stackvitals[sql]` and the command prints it) to see DevLake's DORA panels and its PR cycle time panel computed on your data, with no DevLake, MySQL or Grafana running. Our tests run that SQL on 42 generated repos and require the same answers as our page.\n\n## Team Flow\n\n**Try it on any public repo:**\n\n```\nstackvitals flow report owner/repo\n```\n\nIt answers \"what is blocking this repo's pull requests, and who has to act next?\":\n\n- **Who acts next, per open PR:** the reviewers asked, the author (changes requested or failing checks), or the maintainers (approved but not merged, or nobody asked to review), with the event that put it there and since when.\n- **Overdue** means waiting longer than usual for this repo (its own 75th percentile for that step, judged separately for insiders and outside contributors) and over a day. PRs waiting over 30 days are counted as stale backlog. Drafts and PRs stacked on another open PR are never blamed on reviewers.\n- **Review stages,** with GitHub's own boundaries: ready for review to first review, first to final review, final review to merge. Bots, the author and Copilot are left out.\n- **Insiders and outside contributors** side by side: merge rate and how long the first review takes.\n- **PR rules as code:** `[[rules]]` in `.github/stackvitals.toml`, each with a `name`, a `when` and a `then`:\n\n  ```toml\n  [[rules]]\n  name = \"Agent changes to payments\"\n  when = { author = \"agent\", paths = [\"payments/\"] }      # author: agent, person or bot\n  then = { label = \"needs-second-review\", reviewers = [\"amy\", \"@acme/payments\"], comment = \"An agent changed payments code.\" }\n\n  [[rules]]\n  name = \"Big risky change\"\n  when = { lines_over = 400, risk = \"high\" }              # risk: this repo's change risk (where its backtest works)\n  then = { label = \"risky\" }\n  dry_run = true                                         # only reports what it matches\n  ```\n\n  `when` can also ask for `labels` (any of) and `first_time = true` (an author with no merged pull request yet). Each\n  action happens once per pull request: a label someone removes stays removed, a comment is never repeated. The Pull\n  Requests page lists each rule with the open pull requests it matches and what it matched among the last 200 merged\n  ones, so a new rule can be checked (with `dry_run`) before it acts. It runs on the workflow's own token (needs\n  `pull-requests: write`), with no app to install.\n- **Suggested reviewers** for PRs nobody was asked to review: maintainers who wrote and reviewed merged PRs in the same folders, who reviewed this author's earlier PRs, plus CODEOWNERS, eased off by how many reviews each already has open. Only people who can review there are suggested. On 1,186 PRs in 26 public repos a real reviewer was in the top 3 82% of the time, against 79% for the three busiest reviewers ([bench/accuracy/team](../bench/accuracy/team/README.md)).\n- **Team automation, off until you turn it on** (each part under `[automation]` in `.github/stackvitals.toml`, or from the app's Settings page): the top suggested reviewer is requested once on ready PRs nobody was asked to review (`review_requests`), every open PR gets a `size/XS` to `size/XL` label (`size_labels`), and docs-only PRs get \"low risk: docs only\" (`docs_label`; never an approval). The job summary says what it did. With working agreements set and a webhook, each open PR breaking one is announced once in chat. The weekly digest issue asks three pulse questions (flow, friction, satisfaction) answered by reaction, and the next digest shows the counts beside the numbers, never who answered (`[digest] pulse`). Each week's counts are kept for a year (`docs/pulse.json`), and the Pull Requests page shows the trend with each question beside the number it explains (flow with pickup time, friction with first-try green, satisfaction with deployments); a week with fewer than 5 answers to a question shows no share.\n- **Knowledge map:** folders known mostly by one person, main authors who have gone quiet, and the busiest folders (lock files left out).\n- **Delivery:** releases per week, how long a merge waits for a release, and how many releases were followed by a revert or hotfix within 7 days (a proxy for change failure, labelled as such).\n\n**In the Action,** add `flow` to `modules:` (the template from `stackvitals ci init` has `ci,flow,deps,community`). It runs on the schedule (every 3 hours) and manual runs, keeps PR history in the `stackvitals-data` branch and writes `docs/flow.html`. One GitHub request reads about 25 pull requests with their review history, so a year of history loads in one run.\n\n**People:** the public page shows who a PR is waiting on and open requests per reviewer, never speed rankings. Each reviewer's own answer time appears only in the local `flow report` view.\n\n- **Pre-flight:** what merged PRs touching the same folders usually include (tests, a changelog entry, docs) and how big they are, plus what maintainers most often ask for in review there. `stackvitals flow preflight owner/repo --paths src/db/pool.py --lines 120` lists what a planned change is missing. Review comments are sorted into plain categories as they are read; their text is never stored.\n- **Pre-flight note on PRs (off by default):** with `preflight-comments: true`, a new PR from an outside contributor that misses a local habit gets one short note. Never twice, never on PRs that were already open.\n\n- **Compared with similar repos:** time to the first maintainer review, opened to merged, how many outside PRs get merged and typical PR size, next to public repos of the same language and size from a daily crawl of active repos (groups of 50+ repos, with the number of repos shown).\n- **Triage of outside PRs (private, for maintainers):** `stackvitals flow triage owner/repo` gives each open outside PR's chance of being merged within 30 days and the main reasons (the author's past PRs here: none, some but none merged, or merged ones; size; a linked issue; how this repo usually treats outside PRs). The model is trained on real outcomes of outside PRs across public repos and checked on repos it never saw; the file says how well. It is computed on demand with your own token and is never stored, shown on a page or posted on a PR. It scores the PR's chances, not the person.\n\n**For AI agents:** `pr_queue`, `flow_health`, `who_should_review`, `preflight` and `triage_pr` in `stackvitals serve`.\n\n## Dependency Steward\n\n**Known vulnerabilities in every lockfile, every ecosystem:** after CI runs and on the schedule, the Action reads the\ndefault branch's lockfiles through the API (npm, yarn, pnpm and bun; pip requirements, Poetry, uv, Pipenv and PDM; Go;\nCargo; Bundler; Composer; NuGet and .NET project files; Gradle and Maven; SPDX and CycloneDX SBOMs; and more) and scans them with Google's OSV-Scanner, pinned to a\nversion and checked against its published SHA-256 (no Docker, no checkout of your code). Each finding shows the advisory,\nits severity, the version that fixes it, the lockfile and the day it was first seen; fixed ones move to \"fixed lately\". An advisory the repo ignores in `osv-scanner.toml` at its root (`[[IgnoredVulns]]` with an `id`, an `ignoreUntil` date and a `reason`, the file OSV-Scanner itself reads) stays listed, marked ignored with its reason, and isn't named as new; once the date passes it counts again. Each fix comes with a plan, as OSV-Scanner's guided remediation gives one: the smallest version that fixes it, whether that is a patch, minor or major jump (a major one may break things; 0.x minors count as major), and the command that makes it in that ecosystem (npm install, pip or uv, go get, cargo update --precise, composer require, dotnet add package...). A package only development uses is marked \"dev only\" where the lockfile says so (npm's and pnpm's `dev: true`, Pipfile.lock's develop section), so it can wait behind what ships; other lockfiles don't record it, so nothing is guessed. The GitHub Actions your workflows use are checked too, since no lockfile lists them: each action used by tag (`tj-actions/changed-files@v45`) is matched against OSV's GitHub Actions advisories (their version ranges are read and compared here, as OSV's API doesn't match versions for actions; a major-only tag like `v4` counts only when no 4.x release is fixed); actions pinned to a full commit can't be matched to a version. The open findings are also written as `docs/vulns.sarif` for GitHub's Security tab: the Dependencies page in the app makes `.github/workflows/stackvitals-code-scanning.yml`, which uploads it (and `docs/security.sarif`) with `github/codeql-action/upload-sarif@v4` after each StackVitals run, and adds it in one click (GitHub's editor, or a pull request with a token; it needs `security-events: write`).\nLockfiles are rescanned when they change and every 6 hours for new advisories. On 100 crawled repos in ten languages it\nread 24,003 packages; the deep evidence below covered a third of them ([bench/accuracy/deps](../bench/accuracy/deps/README.md)).\nWith `[deps] outside_lookups = false` it downloads OSV's databases instead, so no package name leaves the runner.\n\n**Malware between scans:** on each scheduled or post-CI run, GitHub's newest malware advisories are matched against\nevery package the last scan read, so a compromised release listed there is flagged without waiting for the next full\nrescan or for OSV. A match stays open while the package is still in a lockfile. The day each bad version was found in the\nrepo is kept. How much earlier this is than OSV hasn't been measured.\n\n**On pull requests:** the check lists the known vulnerabilities a pull request adds or removes in any ecosystem\n(\"lodash 4.17.15: GHSA-35jh-r3h4-6jhm, high, fixed in 4.17.21\"), and, for npm and PyPI packages it adds or upgrades,\nsigns that can come before any advisory (checked on known attacks in [bench/accuracy/deps](../bench/accuracy/deps/README.md#release-signs-on-known-attacks-incidentspy)): a new install script, new dependencies, or build provenance the previous version had\nand this one lacks make it \"look first\"; a release under 48 hours old is noted. With `[deps] guarddog = true`, DataDog's\nGuardDog also reads each new version's code (Linux runners; experimental).\n\n**Does your code import it:** for Python, JavaScript/TypeScript and Rust, run `stackvitals deps imports --out imports.json` in a step after checkout and pass `imports-json: imports.json`: each known vulnerability then says whether your own code imports the package (tests and vendored code left out). An import is a hint for which advisories to read first, not proof the vulnerable code runs, and a package with no direct import may still run through another package. Go gets call paths from govulncheck (`govulncheck-json`), shown on each Go advisory in the vulnerability list (\"your code calls it\", imported, or only required, with the call path on hover); for Rust the check reads `use` lines and paths like `tokio::spawn`, since its call analysis needs a full build.\n\n**Runtimes:** the Node.js, Python, PHP, Ruby and Go versions the repo runs on (version files, Dockerfiles, workflow setup\nsteps) are checked against their support dates: a line out of support gets no security fixes, and a pinned patch older\nthan the newest misses the fixes since. Runtime flaws (a flaw in the runtime itself, like a PHP security release) are not package advisories, so lockfile\nscans never see them. Container base images count too, with no Docker: the operating system under each Dockerfile image (debian:bookworm, ubuntu:22.04, alpine:3.18, or the codename at the end of a language image like python:3.11-slim-bullseye) is checked the same way, since an end-of-life base gets no security fixes for its packages.\n\n**Update hygiene, from the pull requests already kept:** open Dependabot and Renovate updates by age, security updates waiting more than 48 hours, how fast updates merge, and a warning when the bot has opened nothing for 30 days (Dependabot pauses itself after 90 days without interaction), in `docs/deps.json` and the digest. `/stackvitals accept package:<name> until 2026-12-31 because <reason>` takes a security update off the overdue list until then.\n\n**This repo's own security, once a week:** OpenSSF Scorecard's checks, run by your workflow on the repo itself, so\nthey work on private repos too. Each probe uses Scorecard's name where the two overlap, says why it matters and what to\ndo, and points at the file and job:\n\n- workflows: a `pull_request_target` or `workflow_run` workflow checking out the pull request's code (one behind a\n  label, actor or author-association guard is shown as guarded, not as a hole), outsider text pasted into a `run:`\n  script, no read-only `permissions:` at the top, actions used by tag instead of a full commit, and `curl | sh`;\n- branch protection in Scorecard's five tiers, read from rulesets, classic protection, and without an admin token from\n  the branch's public summary (classic protection on, which blocks force pushes and deletion, and the checks it\n  requires); what still can't be read is \"can't read\", never a pass, and protection that got weaker since last week is\n  called out;\n- SECURITY.md (in the repo or the owner's `.github` repo) and whether private vulnerability reporting is on; signed or\n  attested assets on the last five releases; an SBOM; a publishing workflow; code scanning (CodeQL, Semgrep,\n  SonarCloud); committed binaries (by Scorecard's list of extensions, and executable files with no extension read for an ELF, Mach-O or Windows header, as Scorecard does); fuzzing (OSS-Fuzz, ClusterFuzzLite, fuzz folders); the OpenSSF Best Practices badge\n  (looked up for public repos only).\n\nOne 0-10 score weighs the probes by risk (critical 10, high 7.5, medium 5, low 2.5) among those that could be read, and\neach week's score is kept. `docs/security.sarif` holds the workflow findings for GitHub's Security tab: the same\n`stackvitals-code-scanning.yml` workflow from the Dependencies page uploads it with `github/codeql-action/upload-sarif@v4`\n(it needs `security-events: write`). Side by side with Scorecard's published results on 16 repos drawn from the crawl, the two agreed on 137 of 139\nchecks both could read ([bench/accuracy/scorecard](../bench/accuracy/scorecard/README.md)); classic branch protection\nneeds an admin token, so it often says \"can't read\" where Scorecard counts it as met.\n\n**Supply chain, once a week:** the repo's SBOM from GitHub's dependency graph (through the asynchronous report that\nreplaces the synchronous endpoint closing on 13 November 2026), each package's licence (from deps.dev when GitHub's SBOM\nleaves it blank), a licence rule (`deny` under `[deps.licenses]`, like `[\"GPL-3.0*\", \"AGPL-*\"]`; a package that offers an\nallowed choice is fine), licence changes between quarterly snapshots, and the OpenSSF Scorecard of each project behind the\npackages, with drops of a point or more and scores under 4. The actions your workflows use are listed too, with their own\nrepos' Scorecards (the licence rule skips them: they run in CI and don't ship).\nThe same rule runs on pull requests: each package version a PR brings in has its licence looked up on deps.dev, and a denied one is named in the check (an annotation, the job summary and the PR note when comments are on) before it merges.\n`/stackvitals accept package:<name> for 30d because <reason>` accepts a denied licence. The workflow needs\n`contents: read` (it has it) and the dependency graph turned on (Settings, Code security; public repos have it already).\nMCP: `supply_chain`.\n\n**Security depth:** each advisory's CVE is checked against CISA's known-exploited list and FIRST's EPSS (the chance of\nexploitation in the next 30 days), daily and free: a package with an exploited or likely-exploited open advisory is\nflagged, whatever its severity label (on the crawl, about half of the exploited or likely-exploited advisories are rated\nbelow high). In a pull request, the version being installed is checked against each advisory's affected range (\"2.30.0\nis affected by GHSA-..., fixed in 2.31.0\"), and a flagged package pulled in by another names the way it came (\"via\nexpress → body-parser → qs\", from npm and uv lockfiles). Your own policy goes in `[deps.policy]`:\n`max_severity` flags a changed package with an open advisory rated above it, `min_scorecard` lists the projects under\nit; Scorecard stays one input and never decides a verdict ([bench/accuracy/deps](../bench/accuracy/deps/README.md)).\n\n**Before a CVE exists:** a package a pull request adds whose name is one edit from a popular one (\"exprses\", \"reqests\")\nand that almost nobody downloads (under 1,000 a week) is flagged as a likely look-alike; on OpenSSF's malicious-packages\nlist every sampled malicious look-alike had at most 75 downloads a week, and every legitimate neighbour (extend and\nxtend) hundreds of thousands. **Is the vulnerable code even called?** For Go, add govulncheck to the workflow\n(`go install golang.org/x/vuln/cmd/govulncheck@latest && govulncheck -format json ./... > govulncheck.json`) and pass\n`govulncheck-json: govulncheck.json` to the Action: the supply page then says which advisories your code calls, with\nthe call path (on three public Go repos, about half the third-party advisories were never called).\n\nShould you add, keep or upgrade a package? Evidence over time, for npm and PyPI: the 1,000 most-used packages of each (by repos depending on them, from ecosyste.ms) and every package that crawled repos upgrade.\n\n```\nstackvitals deps check requests\nstackvitals deps scan package-lock.json      # also requirements*.txt, uv.lock, poetry.lock, Pipfile.lock\n```\n\n- **Upgrades in other repos:** how Dependabot and Renovate upgrades of the package went across public repos: merged, closed, failed CI next to the failure rate those same repos usually see, needed extra commits, reverted within 14 days. Rates are shown from 30 PRs in 10 repos; below that it says how much evidence there is.\n- **Its maintainers over time:** whether outside PRs still get merged and how fast maintainers answer, quarter by quarter, next to repos of similar size.\n- **Who publishes it:** a daily watch of npm publishers and maintainers and PyPI owners; a release published without the trusted CI publishing used before, or a new maintainer, is flagged with its date.\n- **Advisories and malware:** OSV history with duplicate ids merged, open advisories, bursts of new ones, and any malware record ever, with the versions it names.\n- **Its source repo:** proven by a signed build or publish, or claimed by the package and checked against that repo's own manifest.\n\nThe verdict is use, monitor or investigate, with the evidence lines behind it, or none when evidence is thin. It weighs what happened; it never calls a package safe.\n\n**On pull requests:** with the `deps` module, a PR that changes a lockfile gets each added or changed package's verdict in the job summary, with an annotation for anything to investigate or monitor. Files are read through the API at the base and head commits; the PR's code is never checked out or run. A PR comment is opt-in (`deps-comments: true`) and only on PRs from the same repo.\n\n**For AI agents:** `check_package` (ask before adding a dependency) and `scan_lockfile` in `stackvitals serve`.\n\n## Community\n\n**Try it on any public repo:**\n\n```\nstackvitals community report owner/repo\n```\n\nIt answers \"is our project growing, and where do contributors drop out?\":\n\n- **Newcomers waiting:** open PRs and issues that are someone's first contribution and that nobody has answered yet, oldest first. These are the replies that matter most.\n- **The funnel, last 12 months:** first-time contributors, how many came back within 90 days, regulars (active in 3 of the last 4 quarters) and active maintainers.\n- **First answers:** how often and how fast outside PRs and issues get a first answer from a maintainer: a review or reply, or a merge or close by someone other than the author. Maintainers are owners, members, collaborators and anyone who merges PRs; on issues, a reply from a regular responder counts too (someone who answered 10 or more other people's issues or first-reviewed 2 or more of their PRs in the last year, often a triager GitHub shows as a plain contributor). Bots and accounts opening 20 or more PRs or issues in a day are left out. Issues still waiting stay in the median (Kaplan-Meier), so a slow repo can't look fast by answering only the easy ones. Newcomers (a first contribution here) and regulars are measured apart, so you can see whether first-timers wait longer.\n- **Who carries the work:** for the last 180 days of merged pull requests by people, how many people merged work, the most active person's share and the bus factor (CHAOSS: the fewest people behind half of it), next to the typical value for similar public repos from the crawl. No names, and no ranking.\n- **Newcomers who came back, by first answer (public repos):** from our crawl, the share of first-time contributors who came back with new work (a day or more later, within 90 days), by how long their first PR waited for an answer. Across 259,586 newcomers in 48,745 public repos (October 2026): answered within two weeks, 19-24% came back; after more than two weeks, 13%; never answered, 8%. Merged first PRs brought 22% back, closed ones 15%, ones left open 9%: silence loses the most. This is a correlation, not a promise.\n- **Score and badges:** the score (0-100) is, on average, the share of similar public repos (same language and size) this repo does better than, on three facts shown next to it: first-answer speed, newcomers answered within two weeks, newcomers who come back. A fact with too little data is left out, never counted as zero. Two shields.io badges are written to the data branch: `docs/community-badge.json` (the facts) and `docs/community-score.json` (the score).\n- **In the project's own words (optional):** `capacity` and `norms` under `[community]` in `.github/stackvitals.toml` say who maintains the project and how much time they have, and how it works with people (like \"two volunteers, evenings and weekends\"). They're shown in the Community page's \"What the project states\" box and in the summary's \"Who carries the work\", so the numbers are read against what the project can give.\n- **Stars:** GitHub's weekly star history (no names) and growth next to similar repos.\n- **Traffic (optional):** GitHub deletes views, clones and referrers after 14 days, and the built-in token can't read them. To keep them, create your own GitHub App with the repository permission \"Administration: read\", install it on the repo, store its ID and private key as secrets, and pass a token from `actions/create-github-app-token` as `traffic-token:`. Without it, everything else runs.\n\n**People:** the public page lists waiting PRs and issues by number, never people. Nobody is ranked and nobody is listed as having left.\n\n**In the Action,** add `community` to `modules:` (the template has it) and `issues: read` to the permissions. It runs on the schedule and manual runs and reuses Team Flow's PR history.\n\n**For AI agents:** `community_health` and `newcomers_waiting` in `stackvitals serve`.\n\n## Portal\n\nLight by design: no server, one daily run, pages in a private repo. It covers the catalog, checked ownership,\nscorecards against similar public repos, campaigns, scorecards as required checks, golden paths, a docs index and a\nBackstage export. Ten repos' files are read in one GraphQL query, six repos are checked at a time and unchanged answers are read\nconditionally: a run costs about 4 requests a repo instead of 10 (under 3 on later runs, when little changed), so the\nbuilt-in token covers about 180 to 260 repos a run (70 before), and a 10,000-repo catalog reaches the app as about a\n1 MB compressed download (8.4 MB unpacked), estimated from a real org's entries ([bench/accuracy/portal](../bench/accuracy/portal/README.md)).\n\n**Try it on any org you can see:**\n\n```\nstackvitals portal check my-org     # what your token can read\nstackvitals portal report my-org\n```\n\nIt answers \"what do we own, who owns it, and is it in good shape?\":\n\n- **Every repo** with its language, last push, branch rules, custom properties (tier, lifecycle) and, where the suite runs, its CI health, review flow and community score. Archived repos and forks are listed but not judged.\n- **Owners, checked:** the CODEOWNERS file GitHub would use, GitHub's own error check for it, teams that actually have members, and named users who are still in the org. Each repo is valid, weak (with the reasons) or has no file; without a file, recent committers are offered as suggestions, never as owners.\n- **Scorecards:** each repo reaches bronze, silver or gold when every rule of that level and the levels below passes. Each result carries its evidence. Data the token can't read is \"not available\", never a fail. The default rules:\n  - bronze: a CODEOWNERS file, reviews required on the default branch, a README and a licence;\n  - silver: owners that check out, a contributing guide, CI green on the first try and first reviews at least as good as half of similar public repos;\n  - gold: first reviews faster than three quarters of similar repos, and OpenSSF Scorecard's Security-Policy and Token-Permissions checks (public repos).\n\n  Where CI Health doesn't run, Portal measures the newest 50 push runs itself, the same way the benchmark does. To change the rules, put a `portal.toml` in the portal repo (`[[rule]]` with `id`, `check`, `level` and `peer` or `min`; `[[exemption]]` with `repo`, `rule`, `until` and `reason`). Your own checks: `check = \"file\"` with a `path` (the file must exist), plus `contains` or `lacks` with a pattern (a runbook with an on-call section, no `:latest` image tags); only yes or no is kept, never the file. `check = \"catalog\"` asks for a `catalog-info.yaml` with the `fields` you list, and `check = \"catalog_owner\"` for its owner to match CODEOWNERS. Any rule can take `from = 2027-01-01`: until then it shows what it would say (\"not counted yet\") without changing a level, so teams see it coming.\n- **Typed catalog:** each repo's `catalog-info.yaml`, Backstage's format (which Port and Cortex also read), becomes a Catalog tab: components, APIs, resources and systems with their type, lifecycle, owner, system and what they depend on, whether CODEOWNERS agrees with the owner, and any problems in the file. Read with safe YAML only, one request per repo. On 12 public repos' files it read every one with no problems; the owner check passed for grafana's and caught real mismatches in three others.\n- **Org view:** one tab adds up what every repo published (flaky and broken jobs, open and stale pull requests,\n  deployments, findings, newcomers waiting) and takes medians of the rest (first-try share, pickup, review and cycle\n  hours, change failure rate, community score), with the five worst repos on each; a repo that never ran a module is\n  left out of that module's figures, never counted as zero.\n- **History:** a line per repo per day, so the page says since when (\"weak since 2026-10-17\", \"bronze since ...\").\n- **Required checks (opt-in):** `[checks] level = \"silver\"` in `portal.toml`, with the org App allowed to write checks,\n  puts a \"StackVitals portal\" check on each repo's default branch and open pull requests: it passes at that level and\n  lists the rules that keep a repo below it. Branch protection can require it, but it is posted once a day: a pull\n  request's new commits get it on the next portal run, so requiring it can hold a PR for up to a day. At most 300 checks\n  a run; each run starts where the last one stopped.\n- **Golden paths (opt-in):** `[[golden_path]]` entries in `portal.toml` (`id`, `title`, `kind = \"repo\"` with a\n  `template` repo, or `kind = \"files\"` with a `folder` of the portal repo). From the app's Golden paths tab, someone with\n  write access to the portal repo creates a repo from the template, or, with write access to the target repo too,\n  opens a pull request adding the folder's files (a standard CI workflow, a CODEOWNERS) to it; the org App carries it\n  out on the portal workflow's next run (at once for a request issue) and each run is recorded. With `approvals = 1` on a\n  `[[golden_path]]` in `portal.toml` (0 or 1), a request waits until someone else with write access to the portal repo\n  comments `/stackvitals approve <code>` on the request issue; the waiting note gives the code, which changes if the\n  request is edited, so what was approved is what runs. The requester needs write access too, and can't approve their\n  own request; the approver is recorded beside them. A portal repo set up before this needs its workflow printed again\n  (`stackvitals portal init`) for the `issue_comment` trigger. A \"run now\" from the app can't wait for approval: send\n  those paths as a request.\n- **Docs:** a Docs tab lists every repo's `docs/` pages in one search, each opening on GitHub; each repo also says how\n  it runs (Kubernetes, a container, Terraform, serverless) from its root folder.\n- **Backstage:** `portal/entities.yaml` on the data branch holds a Component for every repo without its own\n  `catalog-info.yaml`, with its owner from CODEOWNERS, its StackVitals level as an annotation and a link to the portal;\n  register its URL as one location in Backstage.\n- **Campaigns:** a goal with a date, in `portal.toml` (`[[campaign]]` with `id`, `title`, `rules`, optional `repos` like `acme/api-*` and `language`, and `until`). The page shows how many targeted repos pass, which still fail and the failing count over time; one tracking issue per campaign in the portal repo is kept up to date as a checklist (ticked again on every run) and closed when every repo passes; a reminder comment names the repos still failing 7 days and 1 day before the date.\n\n**In your org:** create a private repo (for example `portal`) and save the workflow from `stackvitals portal init` in it. It runs daily and keeps the catalog, its history and a Markdown page in that repo's `stackvitals-data` branch, which only people who can read the repo can open.\n\n- **Public mode (no setup):** with the built-in token, Portal reads the org's public repos.\n- **Private mode:** to include private repos, an org owner creates an org-owned GitHub App with read-only permissions and no webhook, installs it on the org's repos, stores its client ID and private key, and switches on the token step in the workflow. This is not a personal token. `stackvitals portal app-link my-org` prints a link that opens GitHub's form with the permissions filled in (Actions, Contents, Metadata, Members and Custom properties, all read only).\n- **Required checks and golden paths** need an App that may also write: `stackvitals portal app-link my-org --writes`\n  fills in checks, contents, pull requests and workflows write, and administration write (to create repos). Without\n  them Portal stays read-only.\n- **Privacy:** if the portal repo is public, private repos are left out of every page and file it writes.\n\n**For AI agents:** `org_catalog`, `repo_scorecard` and `owners_health` in `stackvitals serve` read the org's published catalog (or build a quick one live).\n\n## Contribute (MergeScout)\n\n`mergescout` (or `stackvitals contribute`) runs the MergeScout MCP server and CLI: issues an outsider can realistically get merged, with merge rates, review times and live availability checks.\n\n**How changes get merged in a repo** (`stackvitals contribute brief owner/repo`, or the MCP tool `repo_brief`): a short brief, measured from the repo's last 12 months of pull requests, of the things its files don't say:\n\n- per folder: what merged PRs usually include (tests, a changelog entry, docs), their usual size, what maintainers most often ask for, who reviews there and how fast, and the CODEOWNERS;\n- CI jobs and tests known to be flaky (re-run before chasing a failure);\n- how outside contributors' PRs fare here (maintainers are recognised by what they do, not only by their badge), by size, against similar public repos, with a best estimate for a new PR and how far off that kind of estimate was on other repos.\n\nEvery number carries the count it comes from; thin history is said, not guessed. `--write DIR` also writes it as a marked section of `AGENTS.md` (read by many coding agents; the rest of the file is kept, and the run stops rather than guess if the markers were edited) and as GitHub Copilot path files (`.github/instructions/*.instructions.md`). Names that contributors control (test and folder names) are written as plain text, never as instructions. In a small test on 41 recently merged PRs in six repos (the same model, no tools or live search, planning each PR with and without the brief), the brief helped modestly: the actual first reviewer was in the model's top two 29 times against 23, and sizes were guessed a little better. On famous repos the model already knew, it added little.\n","scaling":"StackVitals keeps each repo's history in that repo's own `stackvitals-data` branch. That branch is always the source of\ntruth: everything below is a view of it or a copy of it, and you can delete any copy and rebuild it from the branch.\n\n## The ladder\n\n1. **One repo, nothing to set up.** The Action writes the pages, the published JSON files and `summary/*.jsonl` (one row\n   per pull request, CI job, test, and DevLake's tables) into the branch. Read them on Pages, in the job summary, with\n   `stackvitals ui owner/repo` on your machine (private repos too), or with `stackvitals sql --repo owner/repo \"...\"`.\n2. **SQL in the browser.** `docs/explore.html` runs DuckDB (WebAssembly) over the summaries, reading only the tables a\n   query names. Measured on 2026-10-21 with a synthetic busy repo (20,648 pull requests, 23 MB of summaries): the first\n   query took 2.8 to 3.2 s including starting DuckDB, DevLake's lead-time panel 0.6 s, later panels 0.2 s.\n3. **Several repos, one query.** `stackvitals sql --repo org/web --repo org/api --repo org/worker \"select repo,\n   median(cycle_s) / 3600 from prs where merged_at is not null group by repo\"`: each summary table becomes one view over\n   all of them with a `repo` column. Portal covers the org's catalog and owners.\n4. **A database, when a team wants one.** `stackvitals export owner/repo --format postgres` writes DevLake's tables as CSV\n   and `postgres.sql`; `psql \"$DATABASE_URL\" -f postgres.sql` in that folder creates and loads them. Any Postgres works.\n\n### When would the files be too big?\n\nThe summaries take about 1.2 KB per pull request (measured on pytest, aiohttp, pip and guidellm). With 400 days kept, a\nrepo merging 30 pull requests a day (12,000 in 400 days) has about 15 MB of summaries, the largest file about 7 MB. Explore and `stackvitals\nsql` read them comfortably. A Parquet copy would be added only when one file passes 25 MB (about 40,000 pull requests\nin 400 days) or the first query in Explore passes 3 s (about 20,000), a small fraction of a percent of repos. Until\nthen the Action needs no extra libraries.\n\n### Busy repos\n\nEstimated for public repos with 1,900 to 2,500 CI runs a day (pytorch, vscode, home-assistant), from the 3.5 requests a\nrun StackVitals' collector made on a full day of astral-sh/uv: CI Health alone needs 28% to 37% of the built-in token's\n1,000 requests an hour, so it keeps up without a personal token. Repos with many more jobs a run need more. At about\n1 KB stored a run (measured on click and home-assistant), their CI history grows by about 2 MB a day, so the 250 MB cap keeps about four months of it (the oldest days go first); raise `size-cap-mb` or\nlower `ci-retention-days` to choose ([bench/accuracy/platform](../bench/accuracy/platform/README.md)).\n\n## Actions minutes (private repos)\n\nPublic repos run on GitHub's free minutes. On a private repo every StackVitals job is billed, at least one minute each,\nat GitHub's per-minute price for the runner. A small repo's job takes about a minute; a busy repo's, one to three.\n\nWhat starts a job, by default:\n- the schedule, 8 runs a day (Team Flow, Community, the CI catch-up);\n- a CI run that **failed** (it gets its pull request comment at once); a CI run that passed starts nothing on a private\n  repo, and the next scheduled run reads it;\n- settings changes, request issues, `/stackvitals` commands, and pull requests that change a lockfile.\n\nJobs share one queue (`concurrency`), so a burst of finished CI runs collapses into one running job and one waiting (pull request checks and Test connection runs have their own, so they are never dropped behind it).\n\n| CI runs a day | Failed (about 15%) | StackVitals minutes a day, roughly |\n|---|---|---|\n| 20 | 3 | 10-20 |\n| 200 | 30 | 40-90 |\n| 2,000 | 300 | 300-600 (the queue caps it: one job at a time) |\n\nThese are estimates from run counts, not measured bills. The verdict check needs every CI run, passing ones included,\nto start a job: set the repository variable `STACKVITALS_EVERY_CI_RUN` to `true` (Settings, Secrets and variables,\nActions, Variables). That roughly multiplies the second row's minutes by six. Turning modules off (`modules:`) shortens\neach job.\n\n## Free Postgres services (checked 2026-10-21; they change, so check before you rely on them)\n\n| Service | Free tier | Watch out for |\n|---|---|---|\n| [Neon](https://neon.com/pricing) | 1 GB per project, 20 GB in all, 100 projects, 100 compute hours a project, no card | sleeps after 5 minutes idle (wakes on the next query) |\n| [Prisma Postgres](https://www.prisma.io/pricing) | 2 GB, 50 databases, 200k operations a month, no card | counts operations |\n| [Supabase](https://supabase.com/pricing) | 500 MB per project, 2 active projects | pauses after a week without use |\n| [Aiven](https://aiven.io/pricing) | 1 GB storage, 1 CPU, 1 GB RAM | no choice of region; idle services may be shut down |\n| [Nile](https://www.thenile.dev/pricing) | 1 GB, 50 million query tokens | metered by query tokens |\n| Your own server | as big as you make it | you run it |\n\nAt about 1.2 KB a pull request, 1 GB holds roughly 800,000 pull requests: years of a large org.\n\n## What stays the same at every step\n\n- The Action only ever writes to the repo's own branch; nothing is sent to any service unless you load it there yourself.\n- Every published number carries its sample size and the time it is from.\n- Deleting a copy (a database, a local folder) loses nothing: `stackvitals ui` and the export rebuild it from the branch.\n"}