sentry-test 2.1.0__tar.gz → 2.2.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {sentry_test-2.1.0 → sentry_test-2.2.0}/LICENSE +21 -21
- sentry_test-2.2.0/PKG-INFO +315 -0
- sentry_test-2.2.0/README.md +283 -0
- {sentry_test-2.1.0 → sentry_test-2.2.0}/pyproject.toml +1 -1
- {sentry_test-2.1.0 → sentry_test-2.2.0}/setup.cfg +4 -4
- sentry_test-2.2.0/src/sentry_test.egg-info/PKG-INFO +315 -0
- {sentry_test-2.1.0 → sentry_test-2.2.0}/src/sentry_test.egg-info/SOURCES.txt +16 -0
- {sentry_test-2.1.0 → sentry_test-2.2.0}/src/sentrytest/__init__.py +1 -1
- {sentry_test-2.1.0 → sentry_test-2.2.0}/src/sentrytest/adapters/case_specs.py +244 -244
- {sentry_test-2.1.0 → sentry_test-2.2.0}/src/sentrytest/adapters/local_tools.py +606 -604
- {sentry_test-2.1.0 → sentry_test-2.2.0}/src/sentrytest/adapters/terminal.py +334 -334
- {sentry_test-2.1.0 → sentry_test-2.2.0}/src/sentrytest/adapters/toml_config.py +9 -9
- {sentry_test-2.1.0 → sentry_test-2.2.0}/src/sentrytest/application/analyze.py +349 -353
- {sentry_test-2.1.0 → sentry_test-2.2.0}/src/sentrytest/application/archive.py +408 -432
- {sentry_test-2.1.0 → sentry_test-2.2.0}/src/sentrytest/application/cases.py +128 -128
- {sentry_test-2.1.0 → sentry_test-2.2.0}/src/sentrytest/application/context.py +156 -156
- {sentry_test-2.1.0 → sentry_test-2.2.0}/src/sentrytest/application/coverage_context.py +26 -26
- {sentry_test-2.1.0 → sentry_test-2.2.0}/src/sentrytest/application/dimensions.py +140 -140
- {sentry_test-2.1.0 → sentry_test-2.2.0}/src/sentrytest/application/error_paths.py +154 -154
- {sentry_test-2.1.0 → sentry_test-2.2.0}/src/sentrytest/application/frontend_evidence.py +98 -98
- {sentry_test-2.1.0 → sentry_test-2.2.0}/src/sentrytest/application/impact.py +78 -78
- sentry_test-2.2.0/src/sentrytest/application/promo.py +253 -0
- {sentry_test-2.1.0 → sentry_test-2.2.0}/src/sentrytest/application/reporting.py +408 -412
- {sentry_test-2.1.0 → sentry_test-2.2.0}/src/sentrytest/application/reuse.py +193 -193
- {sentry_test-2.1.0 → sentry_test-2.2.0}/src/sentrytest/application/traceability.py +218 -218
- sentry_test-2.2.0/src/sentrytest/application/training.py +363 -0
- {sentry_test-2.1.0 → sentry_test-2.2.0}/src/sentrytest/application/watch.py +126 -126
- {sentry_test-2.1.0 → sentry_test-2.2.0}/src/sentrytest/cli.py +912 -863
- {sentry_test-2.1.0 → sentry_test-2.2.0}/src/sentrytest/domain/catalog.py +85 -85
- {sentry_test-2.1.0 → sentry_test-2.2.0}/src/sentrytest/domain/models.py +102 -102
- {sentry_test-2.1.0 → sentry_test-2.2.0}/src/sentrytest/domain/rules.py +66 -66
- {sentry_test-2.1.0 → sentry_test-2.2.0}/src/sentrytest/init_project.py +157 -118
- {sentry_test-2.1.0 → sentry_test-2.2.0}/src/sentrytest/ports/inputs.py +36 -36
- {sentry_test-2.1.0 → sentry_test-2.2.0}/src/sentrytest/skills.py +422 -343
- {sentry_test-2.1.0 → sentry_test-2.2.0}/tests/test_analyze.py +365 -365
- {sentry_test-2.1.0 → sentry_test-2.2.0}/tests/test_archive.py +250 -252
- {sentry_test-2.1.0 → sentry_test-2.2.0}/tests/test_arquivo_visual_por_rota.py +107 -0
- {sentry_test-2.1.0 → sentry_test-2.2.0}/tests/test_autoanalise_workflow.py +53 -53
- {sentry_test-2.1.0 → sentry_test-2.2.0}/tests/test_cache.py +215 -215
- {sentry_test-2.1.0 → sentry_test-2.2.0}/tests/test_case_specs.py +184 -184
- {sentry_test-2.1.0 → sentry_test-2.2.0}/tests/test_cases_application.py +162 -162
- {sentry_test-2.1.0 → sentry_test-2.2.0}/tests/test_changed_coverage.py +231 -231
- {sentry_test-2.1.0 → sentry_test-2.2.0}/tests/test_cli.py +501 -501
- {sentry_test-2.1.0 → sentry_test-2.2.0}/tests/test_context.py +183 -183
- {sentry_test-2.1.0 → sentry_test-2.2.0}/tests/test_contextual_reporting.py +83 -83
- {sentry_test-2.1.0 → sentry_test-2.2.0}/tests/test_custo.py +126 -126
- {sentry_test-2.1.0 → sentry_test-2.2.0}/tests/test_dimensions.py +155 -155
- sentry_test-2.2.0/tests/test_docs_aba_de_videos.py +93 -0
- sentry_test-2.2.0/tests/test_docs_agente_configuravel.py +92 -0
- sentry_test-2.2.0/tests/test_docs_arvore_sentry.py +52 -0
- sentry_test-2.2.0/tests/test_docs_bem_vindo_videos.py +56 -0
- sentry_test-2.2.0/tests/test_docs_card_comandos.py +47 -0
- {sentry_test-2.1.0 → sentry_test-2.2.0}/tests/test_docs_consistency.py +91 -91
- sentry_test-2.2.0/tests/test_docs_refazer_video.py +68 -0
- sentry_test-2.2.0/tests/test_docs_ressalva_ia.py +51 -0
- sentry_test-2.2.0/tests/test_docs_sem_repeticao_dos_videos.py +81 -0
- sentry_test-2.2.0/tests/test_docs_skill_pasta.py +96 -0
- sentry_test-2.2.0/tests/test_docs_tokens_videos.py +63 -0
- {sentry_test-2.1.0 → sentry_test-2.2.0}/tests/test_domain_models.py +18 -18
- {sentry_test-2.1.0 → sentry_test-2.2.0}/tests/test_error_paths.py +127 -127
- {sentry_test-2.1.0 → sentry_test-2.2.0}/tests/test_execution_evidence.py +477 -477
- {sentry_test-2.1.0 → sentry_test-2.2.0}/tests/test_frontend_evidence.py +92 -92
- {sentry_test-2.1.0 → sentry_test-2.2.0}/tests/test_git_context.py +385 -385
- {sentry_test-2.1.0 → sentry_test-2.2.0}/tests/test_history_cli.py +252 -252
- {sentry_test-2.1.0 → sentry_test-2.2.0}/tests/test_impact.py +155 -155
- sentry_test-2.2.0/tests/test_init_brag_output.py +63 -0
- {sentry_test-2.1.0 → sentry_test-2.2.0}/tests/test_init_project.py +108 -108
- sentry_test-2.2.0/tests/test_init_skill_pasta.py +150 -0
- {sentry_test-2.1.0 → sentry_test-2.2.0}/tests/test_input_adapters.py +6 -6
- {sentry_test-2.1.0 → sentry_test-2.2.0}/tests/test_modo_instantaneo.py +152 -152
- sentry_test-2.2.0/tests/test_promo.py +404 -0
- {sentry_test-2.1.0 → sentry_test-2.2.0}/tests/test_reporting.py +72 -72
- {sentry_test-2.1.0 → sentry_test-2.2.0}/tests/test_review_cli.py +97 -97
- {sentry_test-2.1.0 → sentry_test-2.2.0}/tests/test_rules.py +88 -88
- {sentry_test-2.1.0 → sentry_test-2.2.0}/tests/test_sem_spec.py +121 -121
- {sentry_test-2.1.0 → sentry_test-2.2.0}/tests/test_status.py +235 -235
- {sentry_test-2.1.0 → sentry_test-2.2.0}/tests/test_terminal_color.py +295 -295
- {sentry_test-2.1.0 → sentry_test-2.2.0}/tests/test_terminal_ui.py +1 -1
- {sentry_test-2.1.0 → sentry_test-2.2.0}/tests/test_toml_config.py +43 -43
- {sentry_test-2.1.0 → sentry_test-2.2.0}/tests/test_traceability.py +241 -241
- sentry_test-2.2.0/tests/test_training.py +536 -0
- {sentry_test-2.1.0 → sentry_test-2.2.0}/tests/test_watch.py +131 -131
- sentry_test-2.1.0/PKG-INFO +0 -314
- sentry_test-2.1.0/README.md +0 -282
- sentry_test-2.1.0/src/sentry_test.egg-info/PKG-INFO +0 -314
- {sentry_test-2.1.0 → sentry_test-2.2.0}/src/sentry_test.egg-info/dependency_links.txt +0 -0
- {sentry_test-2.1.0 → sentry_test-2.2.0}/src/sentry_test.egg-info/entry_points.txt +0 -0
- {sentry_test-2.1.0 → sentry_test-2.2.0}/src/sentry_test.egg-info/top_level.txt +0 -0
- {sentry_test-2.1.0 → sentry_test-2.2.0}/src/sentrytest/adapters/__init__.py +0 -0
- {sentry_test-2.1.0 → sentry_test-2.2.0}/src/sentrytest/application/__init__.py +0 -0
- {sentry_test-2.1.0 → sentry_test-2.2.0}/src/sentrytest/application/formatting.py +0 -0
- {sentry_test-2.1.0 → sentry_test-2.2.0}/src/sentrytest/domain/__init__.py +0 -0
- {sentry_test-2.1.0 → sentry_test-2.2.0}/src/sentrytest/ports/__init__.py +0 -0
- {sentry_test-2.1.0 → sentry_test-2.2.0}/tests/test_formato_brasileiro.py +0 -0
- {sentry_test-2.1.0 → sentry_test-2.2.0}/tests/test_relatorio_nome_com_data.py +0 -0
|
@@ -1,21 +1,21 @@
|
|
|
1
|
-
MIT License
|
|
2
|
-
|
|
3
|
-
Copyright (c) 2026 Lucas Nicolau Ferreira
|
|
4
|
-
|
|
5
|
-
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
-
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
-
in the Software without restriction, including without limitation the rights
|
|
8
|
-
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
-
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
-
furnished to do so, subject to the following conditions:
|
|
11
|
-
|
|
12
|
-
The above copyright notice and this permission notice shall be included in all
|
|
13
|
-
copies or substantial portions of the Software.
|
|
14
|
-
|
|
15
|
-
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
-
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
-
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
-
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
-
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
-
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALING IN THE
|
|
21
|
-
SOFTWARE.
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Lucas Nicolau Ferreira
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALING IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1,315 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: sentry-test
|
|
3
|
+
Version: 2.2.0
|
|
4
|
+
Summary: Test quality intelligence for change-oriented reviews
|
|
5
|
+
Author: Lucas Nicolau Ferreira
|
|
6
|
+
License: MIT License
|
|
7
|
+
|
|
8
|
+
Copyright (c) 2026 Lucas Nicolau Ferreira
|
|
9
|
+
|
|
10
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
11
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
12
|
+
in the Software without restriction, including without limitation the rights
|
|
13
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
14
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
15
|
+
furnished to do so, subject to the following conditions:
|
|
16
|
+
|
|
17
|
+
The above copyright notice and this permission notice shall be included in all
|
|
18
|
+
copies or substantial portions of the Software.
|
|
19
|
+
|
|
20
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
21
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
22
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
23
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
24
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
25
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALING IN THE
|
|
26
|
+
SOFTWARE.
|
|
27
|
+
|
|
28
|
+
Requires-Python: >=3.11
|
|
29
|
+
Description-Content-Type: text/markdown
|
|
30
|
+
License-File: LICENSE
|
|
31
|
+
Dynamic: license-file
|
|
32
|
+
|
|
33
|
+
# Sentry
|
|
34
|
+
|
|
35
|
+
[Português](README.pt.md) | English
|
|
36
|
+
|
|
37
|
+
[](https://www.python.org/)
|
|
38
|
+
|
|
39
|
+
[](https://pypi.org/project/sentry-test/)
|
|
40
|
+
[](LICENSE)
|
|
41
|
+
|
|
42
|
+
Change-oriented test quality CLI. `sentry` creates the spec folder, validates the case matrix in markdown, runs the suite, reads the diff and the coverage, and issues an auditable verdict — and writes the workflow down for the AI agent that will implement the change.
|
|
43
|
+
|
|
44
|
+
Markdown is the source of intent: the CLI parses and validates `CASES.md` and `PROMPT.md` — it never writes them. The AI agent does the drafting; Sentry guarantees structure, traceability and a verdict.
|
|
45
|
+
|
|
46
|
+
> Not to be confused with [getsentry's Sentry](https://sentry.io) (error monitoring). This project is distributed as `sentry-test`.
|
|
47
|
+
|
|
48
|
+
## The division of responsibility
|
|
49
|
+
|
|
50
|
+
- **The AI agent declares intent** — writes `CASES.md`: requirement, layer, type, priority, input, expected result.
|
|
51
|
+
- **Sentry measures reality** — decides status, associated test, evidence and verdict.
|
|
52
|
+
|
|
53
|
+
The agent never writes status. Sentry never calls a model to reach a verdict. That's what makes the verdict auditable and reproducible. The optional video commands (`promo` and `training`) are the exception: they hand the work to the agent, which spends tokens, and they never touch the verdict.
|
|
54
|
+
|
|
55
|
+
## Installation
|
|
56
|
+
|
|
57
|
+
```bash
|
|
58
|
+
pip install sentry-test
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
Requires Python 3.11+. Check with `sentry --version` (prints the package version).
|
|
62
|
+
|
|
63
|
+
If `pytest` or `coverage` are missing, `init` warns you. To install them too:
|
|
64
|
+
|
|
65
|
+
```bash
|
|
66
|
+
sentry init --install
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
## Setup
|
|
70
|
+
|
|
71
|
+
```bash
|
|
72
|
+
cd your-project
|
|
73
|
+
sentry init
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
Creates `.sentry/` (specs, runs, reports and database), `sentry.toml`, the `.gitignore` entries, and writes the workflow in two places:
|
|
77
|
+
|
|
78
|
+
- **`sentry-cases` skill** (`SKILL.md`), written to the folder(s) declared in `[init] skills_dirs` in `sentry.toml` — `.claude/skills/` by default; `--skills-dir` picks another. An agent that reads skills from that folder loads it automatically.
|
|
79
|
+
- **`AGENT-SENTRY.md`** at the project root — the same workflow in plain markdown, without depending on any tool's convention.
|
|
80
|
+
|
|
81
|
+
For other agents (Cursor, Windsurf, Codex, opencode), point them to `AGENT-SENTRY.md` from the rules file each one already uses — for example, a line in `.cursor/rules` or `AGENTS.md`:
|
|
82
|
+
|
|
83
|
+
```markdown
|
|
84
|
+
To write or review test cases, follow AGENT-SENTRY.md.
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
`init` is idempotent: it doesn't erase history, doesn't overwrite existing configuration, doesn't duplicate structures.
|
|
88
|
+
|
|
89
|
+
## The workflow
|
|
90
|
+
|
|
91
|
+
1. `sentry new "customer registration"` — creates `.sentry/specs/customer-registration/` with `PROMPT.md` (request preserved) and a blank `CASES.md`
|
|
92
|
+
2. **`sentry-cases`** — your agent asks about anything ambiguous and fills in `CASES.md` following the template
|
|
93
|
+
3. `sentry check customer-registration` — is the structure valid? are the catalog's equivalence classes covered?
|
|
94
|
+
4. your agent links each case to the real test with the `# scenario: <exact case name>` marker
|
|
95
|
+
5. `sentry watch` — leave it running while you write: it re-evaluates on every save without executing tests, and escalates to the full run when the file you saved is a test
|
|
96
|
+
6. `sentry context --json` — when a gap shows up, your agent works over the gaps instead of re-reading the code
|
|
97
|
+
7. `sentry review` — `check` + `run` + report in one command, as a local pre-commit hook
|
|
98
|
+
8. `sentry status` — periodically, the whole application instead of just the diff: which files no test reaches at all
|
|
99
|
+
9. `sentry report` / `sentry history` — read back and compare between runs
|
|
100
|
+
|
|
101
|
+
Every step also works without an agent, via the commands below.
|
|
102
|
+
|
|
103
|
+
## Commands
|
|
104
|
+
|
|
105
|
+
Exit code `0` on success; see the exit code table further down.
|
|
106
|
+
|
|
107
|
+
| Command | What it does |
|
|
108
|
+
| ----------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
109
|
+
| `sentry init [--install] [--skills-dir DIR]` | Prepares the repository: `.sentry/`, `sentry.toml`, `.gitignore`, agent guide and skills. With `--install`, installs missing dependencies. `--skills-dir` (repeatable) picks the folder(s) for the `sentry-cases` skill and records them in `[init] skills_dirs`; it refuses an unsafe folder (absolute or with `..`) and refuses to override what `sentry.toml` already declares. |
|
|
110
|
+
| `sentry new <name> [--prompt "..."] [--json]` | Creates the spec folder with a slug derived from the name. `--json` emits the template, accepted vocabulary and required classes, for the agent to consume. |
|
|
111
|
+
| `sentry check [<slug>\|all]` | Validates `CASES.md`: structure, vocabulary and equivalence class coverage. `all` validates every spec together. |
|
|
112
|
+
| `sentry run [--spec <slug>\|all] [--run-tests]` | Runs the analysis and persists it. Without `--run-tests` it runs in instant mode: no suite is executed and coverage is the one from the last full run, stamped as such. With `--run-tests`, a full run whose inputs haven't changed comes back from the cache. With no `--spec` and no `CASES.md` at all, it measures what doesn't depend on a spec instead of demanding one. |
|
|
113
|
+
| `sentry review [--spec <slug>] [--base REF] [--no-tests]` | `check` + `run` + report in a single command, with tests on by default. Works in a repository with no `.sentry/` and no `sentry.toml`. |
|
|
114
|
+
| `sentry watch [--spec <slug>] [--interval SECONDS]` | Re-evaluates on save. Instant mode for the typing loop, escalating to the full run when the saved file is a test — that's where a change becomes evidence. Polls file mtime (default interval `0.4s`), so it adds no runtime dependency. |
|
|
115
|
+
| `sentry context --json` | Emits the last run's gaps as JSON, for the AI agent: uncovered line ranges, error paths no test executed, failing tests by name, scenarios without a test, missing equivalence classes, orphan markers and declared limitations. |
|
|
116
|
+
| `sentry status [--json]` | Measures the whole application, not just the diff: every source file is treated as changed, against every declared spec (`--spec all`). Always runs the full suite, never from the cache — it's the periodic, authoritative snapshot, not the fast loop. Reports which files have zero coverage at all. |
|
|
117
|
+
| `sentry report` | Shows the latest report (`.sentry/reports/latest-<date>.md`), flagging it when its commit is no longer HEAD. |
|
|
118
|
+
| `sentry history` | Lists runs and compares the last two: coverage, tests, new/resolved/persistent findings. |
|
|
119
|
+
| `sentry clear [--keep-last N] [--yes]` | Prunes old runs and reports. Without `--yes` it only shows what would be removed — deleting history is irreversible. Never touches `.sentry/specs/`. |
|
|
120
|
+
| `sentry archive <module> --version X.Y.Z [--specs A,B,C] [--image] [--video]` | Two modes, chosen by how the module is declared in `sentry.toml`. **By specs** (`[modules] <module> = [...]`): runs the module's e2e suite and, once approved, archives its media under `.sentry/storage/<module>-<version>/` with a self-contained `README.md`. Refuses a rejected verdict. The composition is declared once, with `--specs`, on version 1.0.0. **By route** (`[modules.<module>] rotas = [...]`): photographs each declared route with Playwright (desktop + mobile, optional navigation video), one folder per route — even for a module the Sentry has never tested. `login = true` plus `[archive.login]` logs in once before photographing, with credentials only from `SENTRY_LOGIN_USUARIO`/`SENTRY_LOGIN_SENHA`, never in `sentry.toml`. An optional `specs` list inside the module certifies the result the same way; without it, the `README.md` states it's a visual record, not a certification. A route that fails to load doesn't stop the others — the command exits with the warning code and the `README.md` names which route failed. Without `--image`/`--video`, everything produced is archived; with either, only that media type is kept, and `--video` also turns on Playwright's video recording for that run. |
|
|
121
|
+
| `sentry promo [--lang pt\|en]` | Generates the project's promo video with the brag skill, run by the agent declared in `[video] agente` in `sentry.toml` (Claude by default, run as `claude -p "/brag ..."`). Needs that agent and `ffmpeg`; if either is missing, or the agent fails or leaves no `.mp4`, it stops with a clear error and a non-zero exit code — never a silent fallback. Saves `.sentry/video/promo-<lang>.mp4` (moved from `brag-output/`, brag's working folder, together with the rest of its output: composition and share copy; `init` adds `brag-output/` to `.gitignore`). With the default agent, if only the brag skill is missing, the Sentry installs it (`claude plugin marketplace add latent-spaces/brag` and `claude plugin install brag@brag`) and carries on; with a declared agent the skill is that agent's business: the Sentry asks for it by name and installs nothing. `--lang en` runs brag again in English; Portuguese is the default. It runs the agent, so it spends tokens, only when you ask. To change the video you don't need to run the command again: ask the agent to change the script (the brag project in `.sentry/video/`) until you get the result you want. |
|
|
122
|
+
| `sentry training <module> [--lang pt\|en]` | Records a module's training video. Reads the declarative script `.sentry/training/<module>.json` (`base`, `titulo` and `passos`; each step has `titulo`, `fala` and an `acao` that is data, never code: `ir`, `digitar`, `clicar` or `apontar`), records it with Python Playwright (bundled in the Sentry, so it works in any project) timing every step, and has the brag skill (through the configured agent, as in `promo`) assemble the video with captions. It certifies nothing: it only records and issues no verdict — only the verdict certifies. Saves `.sentry/video/training-<module>-<lang>.mp4`. Stops with a clear error and a non-zero exit code if the module name is unsafe, the script is missing or invalid (`sentry check` validates every script too), the app in `base` doesn't answer, or the agent or `ffmpeg` is missing; no silent fallback. Whatever is missing of brag and Playwright is installed automatically: brag as in `promo`, and Playwright with the same Python that runs the Sentry (`python -m pip install playwright`, then `python -m playwright install chromium`, about 150 MB). If that installation fails, it stops and shows the manual commands (`pip install playwright && playwright install chromium`). `--lang en` runs brag again in English. It runs the agent, so it spends tokens, only when you ask. To change the video you don't need to run the command again: ask the agent to change the script (`.sentry/training/<module>.json`) until you get the result you want. |
|
|
123
|
+
|
|
124
|
+
## Two modes, by design
|
|
125
|
+
|
|
126
|
+
The mode decides what the report is allowed to claim about coverage.
|
|
127
|
+
|
|
128
|
+
- **Instant** — no test process is started. It reports structure, traceability, orphan markers, the diff, and the coverage of the last full run. Reused coverage is always stamped as coming from that run, with its timestamp, never as measured now. With no previous full run, coverage comes out unavailable and no number is asserted.
|
|
129
|
+
- **Full** — executes the suite. It's the mode for the pre-commit hook and for anything that needs a count of its own.
|
|
130
|
+
|
|
131
|
+
`sentry watch` picks between them on every save: instant for source files, full when the saved file is a test.
|
|
132
|
+
|
|
133
|
+
A full run reuses the previous one when nothing that matters has changed. The cache key is a content hash of three inputs and nothing else: the files the diff lists, the test files, and the specs. Change any of them and the suite runs again — either the code is different, or what's being asked of it is. A run that comes back from the cache says so in the report, with the id and the timestamp of the run the evidence came from: reusing without saying it would turn old evidence into a new claim.
|
|
134
|
+
|
|
135
|
+
## The gap payload
|
|
136
|
+
|
|
137
|
+
`sentry context --json` is the loop's other half: the agent stops reading the code and reads what has no evidence. Keys: `summary`, `uncovered_lines`, `uncovered_error_paths`, `failing_tests`, `scenarios_without_tests`, `missing_equivalence_classes`, `orphan_markers`, `limitations`.
|
|
138
|
+
|
|
139
|
+
Every key always exists, empty when there's nothing — an absent key would be ambiguous between "nothing to do" and "Sentry didn't look". Nothing there is recalculated: each gap was already decided by whatever held the data, and the payload only crops it, so it can never disagree with the report.
|
|
140
|
+
|
|
141
|
+
## Cost
|
|
142
|
+
|
|
143
|
+
Every report ends with a `## Custo` block: the tokens Sentry spent — **zero**, because no model is called to reach the verdict (the optional video commands hand their work to the agent and stay outside this block), which is what makes the same commit always produce the same verdict — and the size of what it summarized, in bytes: the diff it read and the report itself. The conversion to tokens is declared as an estimate at 4 characters per token, with no real tokenizer; it's an order of magnitude for sizing a context budget, not an exact count.
|
|
144
|
+
|
|
145
|
+
## Exit codes
|
|
146
|
+
|
|
147
|
+
Four distinguishable states, to separate "poorly tested code" from "my environment broke":
|
|
148
|
+
|
|
149
|
+
| Code | Meaning |
|
|
150
|
+
| ---- | ------------------------------------ |
|
|
151
|
+
| `0` | approved |
|
|
152
|
+
| `1` | approved with caveats |
|
|
153
|
+
| `2` | rejected |
|
|
154
|
+
| `3` | inconclusive or infrastructure error |
|
|
155
|
+
|
|
156
|
+
An infrastructure error never produces an approved verdict: a suite that failed to run is different from a suite that failed.
|
|
157
|
+
|
|
158
|
+
`sentry check` keeps its own semantics: `0` valid structure, `1` structural errors, `2` couldn't resolve the spec.
|
|
159
|
+
|
|
160
|
+
## Skills
|
|
161
|
+
|
|
162
|
+
Generated for each configured agent; `AGENT-SENTRY.md` covers the rest.
|
|
163
|
+
|
|
164
|
+
| Workflow | What the agent does |
|
|
165
|
+
| -------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
166
|
+
| `sentry-cases` | Takes the free-text request, creates the spec, **asks before writing** anything ambiguous that would change a case, fills in `CASES.md`, links each case to a test with `# scenario:` and runs `check` until it closes clean. Never writes status. |
|
|
167
|
+
|
|
168
|
+
## Deterministic rules
|
|
169
|
+
|
|
170
|
+
Ten rules, with severity configurable per project.
|
|
171
|
+
|
|
172
|
+
| Rule | Default severity | Triggers when |
|
|
173
|
+
| --------------------------------- | ---------------- | --------------------------------------------------------- |
|
|
174
|
+
| `test-failing` | critical | the suite has a failing test |
|
|
175
|
+
| `case-spec-invalid` | critical | `CASES.md` has a structural error |
|
|
176
|
+
| `changed-code-uncovered` | high | changed code coverage is zero |
|
|
177
|
+
| `scenario-without-test` | high | a declared case has no associated test |
|
|
178
|
+
| `error-path-without-test` | high | a `raise`/`throw` on a changed line that no test executed |
|
|
179
|
+
| `missing-equivalence-class` | high | a class required by the catalog that no case covers |
|
|
180
|
+
| `coverage-below-threshold` | high | changed code coverage below the declared threshold |
|
|
181
|
+
| `requirement-without-scenario` | medium | a requirement with no matching scenario |
|
|
182
|
+
| `coverage-missing` | medium | changed code coverage couldn't be calculated |
|
|
183
|
+
| `global-coverage-below-threshold` | medium | global coverage below the declared threshold |
|
|
184
|
+
|
|
185
|
+
Without a declared threshold, Sentry doesn't invent a minimum. The project decides what's "enough", and the report records the number applied.
|
|
186
|
+
|
|
187
|
+
## Equivalence class catalog
|
|
188
|
+
|
|
189
|
+
A fixed table of situations that need a test, **per field type**. It doesn't generate cases: it flags the ones the agent left undeclared.
|
|
190
|
+
|
|
191
|
+
Known types: `cpf`, `cnpj`, `email`, `senha` (password), `data` (date), `telefone` (phone), `cep` (postal code), `inteiro` (integer), `decimal`, `texto` (text), `rota` (route), `formulario` (form), `navegacao` (navigation), `responsivo` (responsive), `acessibilidade` (accessibility) — the last four are for `frontend`-layer cases verified by Playwright evidence, not line coverage.
|
|
192
|
+
|
|
193
|
+
A class that doesn't make sense for a field can be dismissed **with a justification**, instead of becoming an artificial case or an eternal complaint:
|
|
194
|
+
|
|
195
|
+
```markdown
|
|
196
|
+
## Classes não aplicáveis
|
|
197
|
+
|
|
198
|
+
- **exclude/tamanho-maximo-excedido**: it's a configuration parameter, not a form field
|
|
199
|
+
```
|
|
200
|
+
|
|
201
|
+
The dismissal removes the finding, but it stays recorded in the report — nothing disappears silently.
|
|
202
|
+
|
|
203
|
+
## Coverage dimensions
|
|
204
|
+
|
|
205
|
+
Each one reports `covered`, `partial`, `not covered` or `not applicable`, with evidence.
|
|
206
|
+
|
|
207
|
+
| Dimension | Where the evidence comes from |
|
|
208
|
+
| ------------------------------------------------ | ----------------------------------------------------------- |
|
|
209
|
+
| requirements and business rules | spec scenarios with an associated test |
|
|
210
|
+
| APIs, persistence, transactions and integrations | `contract`/`integration`-type cases and `integration` layer |
|
|
211
|
+
| exceptions, resilience and recovery | changed error paths executed by some test |
|
|
212
|
+
| security and authorization | `route`-type fields with all access classes covered |
|
|
213
|
+
| interface and user flow | `frontend`-layer cases with e2e execution evidence |
|
|
214
|
+
|
|
215
|
+
`not applicable` is distinct from `not covered`: a project with no routes isn't penalized on the security dimension.
|
|
216
|
+
|
|
217
|
+
## Configuration
|
|
218
|
+
|
|
219
|
+
`sentry.toml` at the root, versionable, with no secrets. Everything beyond what `init` already writes is optional.
|
|
220
|
+
|
|
221
|
+
```toml
|
|
222
|
+
[project]
|
|
223
|
+
name = "my-project"
|
|
224
|
+
|
|
225
|
+
[specs]
|
|
226
|
+
path = ".sentry/specs"
|
|
227
|
+
|
|
228
|
+
[test] # any runner that exports JUnit XML
|
|
229
|
+
command = "npx jest"
|
|
230
|
+
junit_xml = "reports/junit.xml" # required outside pytest: it's the only source of counts
|
|
231
|
+
|
|
232
|
+
[tests] # where to look for tests
|
|
233
|
+
paths = ["tests"] # default: tests, test, spec, __tests__
|
|
234
|
+
|
|
235
|
+
[coverage] # report generated by your own suite
|
|
236
|
+
path = "coverage/lcov.info"
|
|
237
|
+
format = "lcov" # optional: detected by content when omitted
|
|
238
|
+
|
|
239
|
+
[e2e] # optional: second suite for frontend-layer cases
|
|
240
|
+
command = "npx playwright test --reporter=junit"
|
|
241
|
+
junit_xml = "frontend/reports/junit.xml"
|
|
242
|
+
paths = ["frontend/e2e"]
|
|
243
|
+
|
|
244
|
+
[video] # optional: the agent that runs the brag (default: Claude)
|
|
245
|
+
agente = ["meu-agente", "--rodar", "{prompt}"] # `{prompt}` marks where the request goes
|
|
246
|
+
|
|
247
|
+
[init] # optional: where the sentry-cases skill is written (default: .claude/skills)
|
|
248
|
+
skills_dirs = [".claude/skills", ".agentes/skills"] # one or more folders; [] writes no skill
|
|
249
|
+
|
|
250
|
+
[analysis]
|
|
251
|
+
run_tests_by_default = false
|
|
252
|
+
timeout_seconds = 300
|
|
253
|
+
exclude = ["frontend/"] # directories out of the analysis scope
|
|
254
|
+
|
|
255
|
+
[policy.thresholds] # without this, no minimum is enforced
|
|
256
|
+
changed_coverage = 85
|
|
257
|
+
global_coverage = 90
|
|
258
|
+
|
|
259
|
+
[policy.severities] # overrides the severity of any rule
|
|
260
|
+
coverage-missing = "high"
|
|
261
|
+
|
|
262
|
+
[catalog.fields] # field types from your own domain
|
|
263
|
+
matricula = ["empty", "invalid-format", "valid"]
|
|
264
|
+
|
|
265
|
+
[dimensions] # axes that don't apply to the project
|
|
266
|
+
disabled = []
|
|
267
|
+
```
|
|
268
|
+
|
|
269
|
+
`.sentry/` holds specs, runs, reports and the database. It stays out of Git, with one deliberate exception: the current report, `.sentry/reports/latest-<date>.md`, is versioned, so the verdict shows up in the PR diff without the reviewer having to run Sentry. The name carries the date and time of the run, and there is only ever one: the previous one is replaced when a new run writes.
|
|
270
|
+
|
|
271
|
+
History is kept indefinitely, and grows with every run. To prune it:
|
|
272
|
+
|
|
273
|
+
```bash
|
|
274
|
+
sentry clear --keep-last 10 # shows what would be removed
|
|
275
|
+
sentry clear --keep-last 10 --yes # removes it
|
|
276
|
+
```
|
|
277
|
+
|
|
278
|
+
Specs are never removed: they're declared intent, not generated evidence.
|
|
279
|
+
|
|
280
|
+
## Supported stacks
|
|
281
|
+
|
|
282
|
+
**Derivation** — from the request to the case matrix — is language-agnostic: `CASES.md` is markdown and the catalog reasons about data type, not code.
|
|
283
|
+
|
|
284
|
+
**Verification** depends on the exchange format your suite exports, not on the tool:
|
|
285
|
+
|
|
286
|
+
| Capability | Support |
|
|
287
|
+
| ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
|
|
288
|
+
| Suite execution | any command that exports **JUnit XML** — pytest, Jest, Vitest, `go test` (gotestsum), Surefire, `dotnet test`, RSpec, PHPUnit |
|
|
289
|
+
| Coverage | **lcov** (nyc, c8, Jest, simplecov), **Cobertura XML** (JaCoCo, coverlet), **coverage.py** (JSON) — detected by content |
|
|
290
|
+
| Case↔test traceability | `.py`, `.js`/`.jsx`/`.ts`/`.tsx`, `.go`, `.java`/`.kt`, `.cs`, `.rb`, `.php`, `.rs` — and the `scenario:` marker works in any comment |
|
|
291
|
+
| Error paths | via AST in Python; via syntactic pattern (`throw`, `catch`, `panic`, `rescue`, `panic!`) in the rest |
|
|
292
|
+
| Impact analysis | 12 source-code extensions |
|
|
293
|
+
|
|
294
|
+
Error path detection outside Python is less precise than AST, and the report records that difference as a limitation — it never hides it.
|
|
295
|
+
|
|
296
|
+
In Python, **pytest** is the only runner instrumented automatically: Sentry recognizes it as `pytest`, `python -m pytest`, and the venv's executable, wraps it in `coverage run` and collects the count on its own. In Django, prefer `command = "python -m pytest"` with `pytest-django` over `manage.py test`. Any other command — including `python -m unittest` — runs **exactly as declared**, with no flag injected: to measure it, declare the `junit_xml` your suite generates, otherwise the verdict comes out `not executed` for lack of evidence.
|
|
297
|
+
|
|
298
|
+
The `frontend` layer is verified by a second suite, declared under `[e2e]` — Playwright's JUnit reporter is a native fit. Its cases never get line coverage: status comes from **execution evidence** (the scenario ran and passed, trace/screenshot attached), never from a coverage number, which the report states as a declared limitation.
|
|
299
|
+
|
|
300
|
+
## Local-first
|
|
301
|
+
|
|
302
|
+
No telemetry, no external calls, no code or diff ever sent anywhere. All history stays on the machine.
|
|
303
|
+
|
|
304
|
+
## Development
|
|
305
|
+
|
|
306
|
+
```bash
|
|
307
|
+
python -m pip install -e .
|
|
308
|
+
python -m pytest
|
|
309
|
+
```
|
|
310
|
+
|
|
311
|
+
Sentry analyzes itself: `sentry run --spec all --run-tests` at the root of the repository matches the cases declared in `.sentry/specs/` against the real test functions and reports the five dimensions.
|
|
312
|
+
|
|
313
|
+
## License
|
|
314
|
+
|
|
315
|
+
MIT
|