errgrade 0.1.0__tar.gz → 0.2.0__tar.gz

{errgrade-0.1.0 → errgrade-0.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: errgrade
-Version: 0.1.0
+Version: 0.2.0
 Summary: Grade the error messages your code raises: what broke, where, and what to do next.
 Author: mgkvibez
 License: MIT
@@ -108,6 +108,40 @@ The values come from the syntax tree, not the text: an f-string, `.format()` or
 counts as interpolation because a value genuinely flows in. `"{name}"` written as a plain
 string counts as a bug, because the reader sees the braces.
 
+## Grading the bodies you return
+
+`check` grades what you raise. `responses` grades what the caller receives, which is a
+different failure: a handler can raise a good message and still answer with `{"error":
+"failed"}`, or with the exception's text pasted in.
+
+```
+$ errgrade responses examples/api_errors.py
+errgrade: 3 finding(s) across 5 body(ies) in 1 file(s), worst high
+3 of 5 graded body(ies) fall below C
+
+[HIGH] examples/api_errors.py:15 (leaked-internals)
+  body: '<computed>'
+  why:     the body embeds the exception's own text, which discloses internals and changes shape between failures
+  missing: what, where, next
+  grade:   D (0/100)
+
+[HIGH] examples/api_errors.py:22 (leaked-internals)
+  body: 'failed'
+  why:     the body carries a traceback field, which is for the developer, not the caller
+  missing: what, where, next
+  grade:   D (0/100)
+
+[MEDIUM] examples/api_errors.py:26 (internal-path)
+  body: 'could not write /srv/app/exports/%s.csv'
+  why:     an absolute server path in a response tells an attacker the layout and is useless to a client
+  missing: where, next
+  grade:   D (40/100)
+```
+
+Those are the four shapes worth failing a build over: an exception's own text leaking out,
+an absolute server path disclosing the deployment layout, a message too vague to act on,
+and a body with no machine-readable code for a client to branch on.
+
 ## Rules
 
 | Severity | Rule | What it means |
@@ -121,6 +155,18 @@ string counts as a bug, because the reader sees the braces.
 One primary finding per message: a vague message is not also reported for context and
 next steps, because stacking three findings on one word is noise, not help.
 
+The response rules, used by `errgrade responses`:
+
+| Severity | Rule | What it means |
+|---|---|---|
+| high | `leaked-internals` | `str(exc)`, `repr(exc)`, `traceback.format_exc()`, `exc.args`, or a `traceback`/`stack`/`debug` field |
+| medium | `internal-path` | an absolute server path inside a response body |
+| medium | `vague-error-body` | the returned message is too vague to act on |
+| medium | `no-machine-code` | no code a client can branch on, so consumers must match your prose |
+
+A nested `{"error": {"code": ..., "message": ...}}` body is read as a whole, so the shape
+callers actually want is not flagged for the code hidden one level down.
+
 ## Suppressing a message
 
 Some messages are deliberately terse. Say so on the line:
@@ -147,7 +193,7 @@ Only that line is suppressed. There is no config file to fall out of sync.
 ## Tests
 
 ```bash
-python -m pytest -q      # 52 tests
+python -m pytest -q      # 68 tests
 bash scripts/verify.sh   # the release gate, 6 checks
 ```
 

{errgrade-0.1.0 → errgrade-0.2.0}/README.md

@@ -88,6 +88,40 @@ The values come from the syntax tree, not the text: an f-string, `.format()` or
 counts as interpolation because a value genuinely flows in. `"{name}"` written as a plain
 string counts as a bug, because the reader sees the braces.
 
+## Grading the bodies you return
+
+`check` grades what you raise. `responses` grades what the caller receives, which is a
+different failure: a handler can raise a good message and still answer with `{"error":
+"failed"}`, or with the exception's text pasted in.
+
+```
+$ errgrade responses examples/api_errors.py
+errgrade: 3 finding(s) across 5 body(ies) in 1 file(s), worst high
+3 of 5 graded body(ies) fall below C
+
+[HIGH] examples/api_errors.py:15 (leaked-internals)
+  body: '<computed>'
+  why:     the body embeds the exception's own text, which discloses internals and changes shape between failures
+  missing: what, where, next
+  grade:   D (0/100)
+
+[HIGH] examples/api_errors.py:22 (leaked-internals)
+  body: 'failed'
+  why:     the body carries a traceback field, which is for the developer, not the caller
+  missing: what, where, next
+  grade:   D (0/100)
+
+[MEDIUM] examples/api_errors.py:26 (internal-path)
+  body: 'could not write /srv/app/exports/%s.csv'
+  why:     an absolute server path in a response tells an attacker the layout and is useless to a client
+  missing: where, next
+  grade:   D (40/100)
+```
+
+Those are the four shapes worth failing a build over: an exception's own text leaking out,
+an absolute server path disclosing the deployment layout, a message too vague to act on,
+and a body with no machine-readable code for a client to branch on.
+
 ## Rules
 
 | Severity | Rule | What it means |
@@ -101,6 +135,18 @@ string counts as a bug, because the reader sees the braces.
 One primary finding per message: a vague message is not also reported for context and
 next steps, because stacking three findings on one word is noise, not help.
 
+The response rules, used by `errgrade responses`:
+
+| Severity | Rule | What it means |
+|---|---|---|
+| high | `leaked-internals` | `str(exc)`, `repr(exc)`, `traceback.format_exc()`, `exc.args`, or a `traceback`/`stack`/`debug` field |
+| medium | `internal-path` | an absolute server path inside a response body |
+| medium | `vague-error-body` | the returned message is too vague to act on |
+| medium | `no-machine-code` | no code a client can branch on, so consumers must match your prose |
+
+A nested `{"error": {"code": ..., "message": ...}}` body is read as a whole, so the shape
+callers actually want is not flagged for the code hidden one level down.
+
 ## Suppressing a message
 
 Some messages are deliberately terse. Say so on the line:
@@ -127,7 +173,7 @@ Only that line is suppressed. There is no config file to fall out of sync.
 ## Tests
 
 ```bash
-python -m pytest -q      # 52 tests
+python -m pytest -q      # 68 tests
 bash scripts/verify.sh   # the release gate, 6 checks
 ```
 

{errgrade-0.1.0 → errgrade-0.2.0}/errgrade/__init__.py

@@ -1,3 +1,3 @@
 """errgrade: grade the error messages your code raises."""
 
-__version__ = "0.1.0"
+__version__ = "0.2.0"

{errgrade-0.1.0 → errgrade-0.2.0}/errgrade/analyze.py

@@ -85,7 +85,8 @@ def _string_parts(node: ast.AST) -> Optional[Tuple[str, bool]]:
         return node.value, False
     if isinstance(node, ast.JoinedStr):
         text = "".join(
-            part.value if isinstance(part, ast.Constant) and isinstance(part.value, str) else "<value>"
+            part.value if isinstance(part, ast.Constant)
+            and isinstance(part.value, str) else "<value>"
             for part in node.values
         )
         has_value = any(isinstance(part, ast.FormattedValue) for part in node.values)
@@ -124,7 +125,7 @@ def _message_node(exc: ast.AST) -> Optional[ast.AST]:
     return None
 
 
-def _docstring_lines(tree: ast.AST) -> set:
+def _docstring_lines(tree: ast.AST) -> set:  # noqa: F811
     """Lines occupied by docstrings, so a message quoted in prose is not graded."""
     lines = set()
     for node in ast.walk(tree):
@@ -145,7 +146,6 @@ def analyze_source(source: str, path: str = "<src>") -> List[Finding]:
         return []
 
     ignored = {i + 1 for i, line in enumerate(source.splitlines()) if PRAGMA in line}
-    doc_lines = _docstring_lines(tree)
     findings: List[Finding] = []
 
     for node in ast.walk(tree):

{errgrade-0.1.0 → errgrade-0.2.0}/errgrade/cli.py

@@ -15,35 +15,38 @@ from typing import List, Optional, Sequence
 
 from . import __version__
 from .analyze import SEVERITY_ORDER, Report, analyze_files
+from .responses import analyze_files as analyze_response_files
 
 EXIT_OK = 0
 EXIT_FINDINGS = 1
 EXIT_USAGE = 2
 
 
-def _report(rep: Report, args: argparse.Namespace) -> str:
+def _report(rep: Report, args: argparse.Namespace, verb: str = "check") -> str:
+    noun = "body(ies)" if verb == "responses" else "message(s)"
     findings = rep.sorted_findings()
     if args.top:
         findings = findings[: args.top]
 
     if not findings:
         return (
-            "No issues found by errgrade v%s across %d message(s) in %d file(s).\n"
+            "No issues found by errgrade v%s across %d %s in %d file(s).\n"
             "That means no known weak pattern matched; it is not a guarantee."
-            % (__version__, rep.graded, rep.files)
+            % (__version__, rep.graded, noun, rep.files)
         )
 
     worst = SEVERITY_ORDER[findings[0].severity]
     worst_name = {0: "high", 1: "medium", 2: "low"}[worst]
     lines = [
-        "errgrade: %d finding(s) across %d message(s) in %d file(s), worst %s"
-        % (len(findings), rep.graded, rep.files, worst_name),
-        "%d of %d graded message(s) fall below C" % (rep.below_c, rep.graded),
+        "errgrade: %d finding(s) across %d %s in %d file(s), worst %s"
+        % (len(findings), rep.graded, noun, rep.files, worst_name),
+        "%d of %d graded %s fall below C" % (rep.below_c, rep.graded, noun),
         "",
     ]
     for f in findings:
         lines.append("[%s] %s:%d (%s)" % (f.severity.upper(), f.path, f.line, f.rule))
-        lines.append("  message: %r" % f.message)
+        label = "body" if verb == "responses" else "message"
+        lines.append("  %s: %r" % (label, f.message))
         lines.append("  why:     %s" % f.detail)
         if f.grade.missing:
             lines.append("  missing: %s" % ", ".join(f.grade.missing))
@@ -55,10 +58,11 @@ def _report(rep: Report, args: argparse.Namespace) -> str:
 def main(argv: Optional[Sequence[str]] = None) -> int:
     parser = argparse.ArgumentParser(
         prog="errgrade",
-        description="Grade the error messages your code raises: what broke, where, what to do next.",
+        description="Grade the messages your code raises, or the error bodies it returns.",
     )
     parser.add_argument("--version", action="version", version="errgrade %s" % __version__)
-    parser.add_argument("paths", nargs="*", help="files or directories (default: current directory)")
+    parser.add_argument("paths", nargs="*",
+                        help="files or directories (default: current directory)")
     parser.add_argument("--json", action="store_true", help="machine-readable output")
     parser.add_argument("--fail-on", choices=["high", "medium", "low", "none"], default="medium",
                         help="lowest severity that fails the run (default: medium)")
@@ -68,7 +72,9 @@ def main(argv: Optional[Sequence[str]] = None) -> int:
     args = parser.parse_args(argv)
 
     raw: List[str] = list(args.paths)
-    if raw and raw[0] == "check":       # optional verb, so `errgrade check .` works
+    verb = "check"
+    if raw and raw[0] in {"check", "responses", "bodies"}:
+        verb = "responses" if raw[0] in {"responses", "bodies"} else "check"
         raw = raw[1:]
     if not raw:
         raw = ["."]
@@ -79,7 +85,10 @@ def main(argv: Optional[Sequence[str]] = None) -> int:
         print("errgrade: no such path: %s" % ", ".join(missing), file=sys.stderr)
         return EXIT_USAGE
 
-    rep = analyze_files(targets, include_tests=args.include_tests)
+    if verb == "responses":
+        rep = analyze_response_files(targets, include_tests=args.include_tests)
+    else:
+        rep = analyze_files(targets, include_tests=args.include_tests)
 
     if args.json:
         payload = {
@@ -94,7 +103,7 @@ def main(argv: Optional[Sequence[str]] = None) -> int:
         }
         print(json.dumps(payload, indent=2))
     else:
-        print(_report(rep, args))
+        print(_report(rep, args, verb=verb))
 
     if args.fail_on == "none":
         return EXIT_OK

errgrade-0.2.0/errgrade/responses.py

@@ -0,0 +1,219 @@
+"""Grade the error bodies your API sends, not only the strings you raise.
+
+A handler can raise a perfect message and still answer with ``{"error": "failed"}`` or,
+worse, with the exception's text pasted in. This module finds the bodies that reach the
+caller: dict literals that are returned, handed to ``jsonify``, or passed as an exception's
+``detail``/``message`` keyword.
+
+One verdict per body, worst first. Four things are worth failing a build over: an internal
+detail leaking out, an absolute path disclosing the server's layout, a message too vague to
+act on, and a body with no machine-readable code at all.
+"""
+
+from __future__ import annotations
+
+import ast
+from pathlib import Path
+from typing import List, Optional, Sequence, Tuple
+
+from .analyze import (  # noqa: F401  (Report is re-exported for the CLI)
+    SKIP_DIRS,
+    Finding,
+    Report,
+    _looks_like_artifact,
+    _looks_like_test,
+)
+from .grade import grade_message
+
+#: Keys that make a dict an error body in the eyes of a caller.
+ERROR_KEYS = frozenset({"error", "detail", "message", "errors", "msg", "reason"})
+
+#: What a machine can branch on. A human string alone forces string matching.
+CODE_KEYS = frozenset({"code", "error_code", "errorcode", "type", "errno", "kind", "status"})
+
+#: Keys that exist only to help the developer who wrote the handler.
+LEAK_KEYS = frozenset({"traceback", "stack", "stacktrace", "debug", "exc", "exception"})
+
+#: Calls that take the exception as an argument.
+LEAK_WITH_ARG = frozenset({"str", "repr"})
+
+#: Calls that leak on their own, with no argument at all.
+LEAK_NO_ARG = frozenset({"format_exc", "print_exc"})
+
+PATH_MARKERS = ("/home/", "/Users/", "/srv/", "/var/www", "/app/", "C:\\\\", "\\\\Users\\\\")
+
+
+def _dict_keys(node: ast.Dict) -> set:
+    """The literal string keys of a dict, ignoring computed ones."""
+    out = set()
+    for key in node.keys:
+        if isinstance(key, ast.Constant) and isinstance(key.value, str):
+            out.add(key.value.strip().lower())
+    return out
+
+
+def _value_text(node: ast.AST) -> Optional[str]:
+    """The literal text of a value, or None when it is computed at runtime."""
+    if isinstance(node, ast.Constant) and isinstance(node.value, str):
+        return node.value
+    if isinstance(node, ast.BinOp) and isinstance(node.op, ast.Mod):
+        return _value_text(node.left)          # "…/exports/%s.csv" % name
+    if isinstance(node, ast.JoinedStr):
+        parts = []
+        for piece in node.values:
+            if isinstance(piece, ast.Constant) and isinstance(piece.value, str):
+                parts.append(piece.value)
+            else:
+                parts.append("{}")
+        return "".join(parts)
+    return None
+
+
+def _leaks_exception(node: ast.AST) -> bool:
+    """True when the body embeds an exception's own text: str(exc), repr(err), a traceback."""
+    for child in ast.walk(node):
+        if isinstance(child, ast.Call):
+            func = child.func
+            name = func.id if isinstance(func, ast.Name) else getattr(func, "attr", "")
+            if name in LEAK_NO_ARG:
+                return True
+            if name in LEAK_WITH_ARG and child.args:
+                return True
+        if isinstance(child, ast.Attribute) and child.attr == "args":
+            return True
+        if isinstance(child, ast.Attribute) and child.attr in {"__dict__", "__traceback__"}:
+            return True
+    return False
+
+
+def _has_path(text: str) -> bool:
+    return any(marker in text for marker in PATH_MARKERS)
+
+
+def _bodies(tree: ast.AST) -> List[Tuple[ast.Dict, str]]:
+    """Every dict that can reach a caller, with the reason it qualifies."""
+    found: List[Tuple[ast.Dict, str]] = []
+
+    for node in ast.walk(tree):
+        if isinstance(node, ast.Return) and isinstance(node.value, ast.Dict):
+            found.append((node.value, "returned"))
+        elif isinstance(node, ast.Call):
+            func = node.func
+            name = func.id if isinstance(func, ast.Name) else getattr(func, "attr", "")
+            if name == "jsonify":
+                for arg in node.args:
+                    if isinstance(arg, ast.Dict):
+                        found.append((arg, "jsonify"))
+            for kw in node.keywords:
+                if kw.arg in {"detail", "message", "msg", "reason"} \
+                        and isinstance(kw.value, ast.Dict):
+                    found.append((kw.value, "exception detail"))
+    return found
+
+
+def _verdict(node: ast.Dict, path: str, line: int) -> Optional[Finding]:
+    """The single most important thing wrong with this body, if anything."""
+    keys = _dict_keys(node)
+    if not keys & ERROR_KEYS:
+        return None
+
+    # {"error": {"code": ..., "message": ...}} is the shape callers want, so look inside:
+    # judging the outer dict alone would flag the best body in the file.
+    payload = node
+    for key, value in zip(node.keys, node.values):
+        if isinstance(key, ast.Constant) and isinstance(key.value, str) \
+                and key.value.strip().lower() in ERROR_KEYS \
+                and isinstance(value, ast.Dict):
+            payload = value
+            keys |= _dict_keys(value)
+            break
+
+    message_node, message_text = None, None
+    for key, value in zip(payload.keys, payload.values):
+        if isinstance(key, ast.Constant) and isinstance(key.value, str):
+            if key.value.strip().lower() in ERROR_KEYS:
+                message_node = value
+                message_text = _value_text(value)
+                break
+
+    if message_node is not None and _leaks_exception(message_node):
+        return Finding(path=path, line=line, rule="leaked-internals", severity="high",
+                       message=message_text or "<computed>",
+                       detail="the body embeds the exception's own text, which discloses "
+                              "internals and changes shape between failures",
+                       grade=grade_message(message_text or ""))
+
+    if keys & LEAK_KEYS:
+        return Finding(path=path, line=line, rule="leaked-internals", severity="high",
+                       message=message_text or "<computed>",
+                       detail="the body carries a %s field, which is for the developer, "
+                              "not the caller" % ", ".join(sorted(keys & LEAK_KEYS)),
+                       grade=grade_message(message_text or ""))
+
+    if message_text and _has_path(message_text):
+        return Finding(path=path, line=line, rule="internal-path", severity="medium",
+                       message=message_text,
+                       detail="an absolute server path in a response tells an attacker the "
+                              "layout and is useless to a client",
+                       grade=grade_message(message_text))
+
+    if message_text:
+        grade = grade_message(message_text)
+        if grade.letter == "D":
+            return Finding(path=path, line=line, rule="vague-error-body", severity="medium",
+                           message=message_text, detail="a caller cannot act on this: "
+                           + ", ".join(grade.missing) + " missing", grade=grade)
+
+    if not keys & CODE_KEYS:
+        return Finding(path=path, line=line, rule="no-machine-code", severity="medium",
+                       message=message_text or "<computed>",
+                       detail="no code a client can branch on, so every consumer has to "
+                              "match your prose and breaks when it is reworded",
+                       grade=grade_message(message_text or ""))
+    return None
+
+
+def analyze_responses(source: str, path: str = "<src>") -> List[Finding]:
+    """Every body in this source that a caller will actually receive."""
+    try:
+        tree = ast.parse(source)
+    except SyntaxError:
+        return []
+    out: List[Finding] = []
+    for node, _why in _bodies(tree):
+        finding = _verdict(node, path, getattr(node, "lineno", 0))
+        if finding is not None:
+            out.append(finding)
+    out.sort(key=lambda f: (0 if f.severity == "high" else 1, f.line))
+    return out
+
+
+def analyze_files(files: Sequence[Path], *, include_tests: bool = False) -> Report:
+    """Walk files and directories, skipping build artifacts and test code."""
+    report = Report()
+    sources: List[Path] = []
+    for entry in files:
+        entry = Path(entry)
+        if entry.is_file():
+            sources.append(entry)
+        elif entry.is_dir():
+            sources.extend(sorted(entry.rglob("*.py")))
+    for source_path in sources:
+        if _looks_like_artifact(source_path) or any(p in SKIP_DIRS for p in source_path.parts):
+            continue
+        if not include_tests and _looks_like_test(source_path):
+            continue
+        try:
+            text = source_path.read_text(encoding="utf-8", errors="replace")
+        except OSError:
+            continue
+        file_findings = analyze_responses(text, str(source_path))
+        report.files += 1
+        if text.strip():
+            try:
+                report.graded += len(_bodies(ast.parse(text)))   # 'graded' = bodies examined
+            except SyntaxError:
+                pass
+        report.below_c += sum(1 for f in file_findings if f.grade.letter in {"C", "D"})
+        report.findings.extend(file_findings)
+    return report

{errgrade-0.1.0 → errgrade-0.2.0}/errgrade.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: errgrade
-Version: 0.1.0
+Version: 0.2.0
 Summary: Grade the error messages your code raises: what broke, where, and what to do next.
 Author: mgkvibez
 License: MIT
@@ -108,6 +108,40 @@ The values come from the syntax tree, not the text: an f-string, `.format()` or
 counts as interpolation because a value genuinely flows in. `"{name}"` written as a plain
 string counts as a bug, because the reader sees the braces.
 
+## Grading the bodies you return
+
+`check` grades what you raise. `responses` grades what the caller receives, which is a
+different failure: a handler can raise a good message and still answer with `{"error":
+"failed"}`, or with the exception's text pasted in.
+
+```
+$ errgrade responses examples/api_errors.py
+errgrade: 3 finding(s) across 5 body(ies) in 1 file(s), worst high
+3 of 5 graded body(ies) fall below C
+
+[HIGH] examples/api_errors.py:15 (leaked-internals)
+  body: '<computed>'
+  why:     the body embeds the exception's own text, which discloses internals and changes shape between failures
+  missing: what, where, next
+  grade:   D (0/100)
+
+[HIGH] examples/api_errors.py:22 (leaked-internals)
+  body: 'failed'
+  why:     the body carries a traceback field, which is for the developer, not the caller
+  missing: what, where, next
+  grade:   D (0/100)
+
+[MEDIUM] examples/api_errors.py:26 (internal-path)
+  body: 'could not write /srv/app/exports/%s.csv'
+  why:     an absolute server path in a response tells an attacker the layout and is useless to a client
+  missing: where, next
+  grade:   D (40/100)
+```
+
+Those are the four shapes worth failing a build over: an exception's own text leaking out,
+an absolute server path disclosing the deployment layout, a message too vague to act on,
+and a body with no machine-readable code for a client to branch on.
+
 ## Rules
 
 | Severity | Rule | What it means |
@@ -121,6 +155,18 @@ string counts as a bug, because the reader sees the braces.
 One primary finding per message: a vague message is not also reported for context and
 next steps, because stacking three findings on one word is noise, not help.
 
+The response rules, used by `errgrade responses`:
+
+| Severity | Rule | What it means |
+|---|---|---|
+| high | `leaked-internals` | `str(exc)`, `repr(exc)`, `traceback.format_exc()`, `exc.args`, or a `traceback`/`stack`/`debug` field |
+| medium | `internal-path` | an absolute server path inside a response body |
+| medium | `vague-error-body` | the returned message is too vague to act on |
+| medium | `no-machine-code` | no code a client can branch on, so consumers must match your prose |
+
+A nested `{"error": {"code": ..., "message": ...}}` body is read as a whole, so the shape
+callers actually want is not flagged for the code hidden one level down.
+
 ## Suppressing a message
 
 Some messages are deliberately terse. Say so on the line:
@@ -147,7 +193,7 @@ Only that line is suppressed. There is no config file to fall out of sync.
 ## Tests
 
 ```bash
-python -m pytest -q      # 52 tests
+python -m pytest -q      # 68 tests
 bash scripts/verify.sh   # the release gate, 6 checks
 ```
 

{errgrade-0.1.0 → errgrade-0.2.0}/errgrade.egg-info/SOURCES.txt

@@ -5,6 +5,7 @@ errgrade/__init__.py
 errgrade/analyze.py
 errgrade/cli.py
 errgrade/grade.py
+errgrade/responses.py
 errgrade.egg-info/PKG-INFO
 errgrade.egg-info/SOURCES.txt
 errgrade.egg-info/dependency_links.txt
@@ -13,4 +14,5 @@ errgrade.egg-info/requires.txt
 errgrade.egg-info/top_level.txt
 tests/test_analyze.py
 tests/test_cli.py
-tests/test_grade.py
+tests/test_grade.py
+tests/test_responses.py

{errgrade-0.1.0 → errgrade-0.2.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "errgrade"
-version = "0.1.0"
+version = "0.2.0"
 description = "Grade the error messages your code raises: what broke, where, and what to do next."
 readme = "README.md"
 requires-python = ">=3.9"
@@ -29,3 +29,11 @@ errgrade = "errgrade.cli:main"
 
 [tool.setuptools]
 packages = ["errgrade"]
+
+[tool.ruff]
+target-version = "py39"
+line-length = 100
+# Explicit so lint means the same thing here as in CI: pyupgrade rules are absent because
+# they want `list[X]`/`X | None`, which mislead about the 3.9 floor.
+[tool.ruff.lint]
+select = ["E", "F", "W", "I"]

errgrade-0.2.0/tests/test_responses.py

@@ -0,0 +1,93 @@
+"""Tests for the response-body rules, including the two bugs this mode had on day one."""
+
+import pytest
+
+from errgrade.responses import analyze_files, analyze_responses
+
+
+def _rules(src):
+    return [f.rule for f in analyze_responses(src)]
+
+
+def test_embedding_the_exception_text_is_high():
+    src = ('def h():\n    try:\n        go()\n'
+           '    except Exception as exc:\n        return {"error": str(exc)}\n')
+    found = analyze_responses(src)
+    assert [f.rule for f in found] == ["leaked-internals"]
+    assert found[0].severity == "high"
+
+
+@pytest.mark.parametrize("expr", ["repr(exc)", "traceback.format_exc()", "exc.args[0]"])
+def test_other_ways_the_internals_leak(expr):
+    src = ('def h():\n    try:\n        go()\n'
+           '    except Exception as exc:\n        return {"error": %s}\n' % expr)
+    assert _rules(src) == ["leaked-internals"]
+
+
+def test_a_traceback_field_is_high():
+    src = 'def h():\n    return {"error": "failed", "traceback": "see logs"}\n'
+    found = analyze_responses(src)
+    assert found[0].rule == "leaked-internals"
+    assert "traceback" in found[0].detail
+
+
+@pytest.mark.parametrize("body", [
+    '{"detail": "could not write /srv/app/x.csv"}',
+    '{"detail": "could not write /srv/app/%s.csv" % name}',
+])
+def test_an_absolute_path_in_a_body_is_medium(body):
+    """The percent-formatted form was invisible until _value_text learned BinOp."""
+    src = "def h(name):\n    return %s\n" % body
+    found = analyze_responses(src)
+    assert [f.rule for f in found] == ["internal-path"]
+    assert found[0].severity == "medium"
+
+
+def test_a_body_a_caller_cannot_act_on_is_flagged():
+    src = 'def h():\n    return {"error": "failed"}\n'
+    assert _rules(src) == ["vague-error-body"]
+
+
+def test_a_nested_error_object_with_a_code_is_left_alone():
+    """This was flagged on day one: the code lived inside the nested object."""
+    src = ('def h():\n    return {"error": {"code": "card_declined",\n'
+           '            "message": "the issuer declined this card; ask for another"}}\n')
+    assert analyze_responses(src) == []
+
+
+def test_a_message_without_any_code_is_flagged():
+    src = 'def h():\n    return {"message": "the issuer declined this card; ask for another"}\n'
+    assert _rules(src) == ["no-machine-code"]
+
+
+def test_a_flat_body_with_a_code_key_is_clean():
+    src = 'def h():\n    return {"error": "card declined by issuer", "code": "card_declined"}\n'
+    assert analyze_responses(src) == []
+
+
+def test_jsonify_and_exception_detail_are_both_read():
+    src = ('def a():\n    return jsonify({"error": "failed"})\n\n'
+           'def b():\n    raise HTTPException(detail={"error": "failed"})\n')
+    assert _rules(src) == ["vague-error-body", "vague-error-body"]
+
+
+def test_a_dict_that_is_not_an_error_is_ignored():
+    src = 'def h():\n    return {"ok": True, "charge_id": "ch_1"}\n'
+    assert analyze_responses(src) == []
+
+
+def test_unparseable_source_is_skipped_not_crashed():
+    assert analyze_responses("def h(:\n") == []
+
+
+def test_one_verdict_per_body_however_many_rules_apply():
+    src = 'def h():\n    return {"error": str(exc), "traceback": "x"}\n'
+    assert len(analyze_responses(src)) == 1
+
+
+def test_test_files_are_skipped_unless_asked(tmp_path):
+    (tmp_path / "tests").mkdir()
+    (tmp_path / "tests" / "test_a.py").write_text('def h():\n    return {"error": "failed"}\n')
+    (tmp_path / "app.py").write_text('def h():\n    return {"error": "failed"}\n')
+    assert analyze_files([tmp_path]).files == 1
+    assert analyze_files([tmp_path], include_tests=True).files == 2