sarj-python-lint 0.35.0__tar.gz → 0.36.1__tar.gz

{sarj_python_lint-0.35.0 → sarj_python_lint-0.36.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: sarj-python-lint
-Version: 0.35.0
+Version: 0.36.1
 Summary: Custom Python lint rules — AST-based, pre-commit-friendly, hypermodern defaults
 Project-URL: Homepage, https://github.com/sarj-ai/standards/tree/main/packages/python
 Project-URL: Repository, https://github.com/sarj-ai/standards
@@ -137,12 +137,27 @@ SARJ050 flags none of them. The remedy deletes the section and keeps the
 summary, which was checked against the shipped strict config: ruff's D417 does
 not fire on a docstring with no parameter section.
 
-The sibling `Returns:` shape was measured and **rejected**: deleting a
-`Returns:` section makes DOC201 fire, so the only compliant remedy is deleting a
-docstring whose summary may be the valuable part. Two more were rejected on
-volume — property docstrings restating the property name and reST/epydoc type
-duplication (`:type x: int`, `:rtype:`) both measure **0** first-party findings
-outside generated code.
+Two more shapes were rejected on volume — property docstrings restating the
+property name and reST/epydoc type duplication (`:type x: int`, `:rtype:`) both
+measure **0** first-party findings outside generated code.
+
+### The `Returns:` half (0.36.0)
+
+```yaml
+    - id: sarj-docstring-returns-restate-signature  # SARJ087
+```
+
+`SARJ087` is the `Returns:` sibling of SARJ086, and it was **rejected once**:
+deleting a `Returns:` section used to make ruff's DOC201 fire, so the only
+compliant remedy was deleting the whole docstring. #164 then removed DOC201 from
+`ruff.strict.toml` as a rule that DEMANDS prose, and the premise expired — under
+the shipped config the section goes and the summary stays.
+
+756 findings over 33 OSS repos / 35,254 files; two seeded samples of 40 and 20
+read against source gave **~2%** false positives, the whole of which was one
+family (`Returns: A new X` — whether the value is a copy is the one thing
+`-> Self` cannot say) now guarded. Three findings on this repo's own source, all
+true, all deleted. Measurements: [docs/rules/SARJ087.md](../../docs/rules/SARJ087.md).
 
 ### House conventions moved out of consumer repos (0.21.0)
 

{sarj_python_lint-0.35.0 → sarj_python_lint-0.36.1}/README.md

@@ -119,12 +119,27 @@ SARJ050 flags none of them. The remedy deletes the section and keeps the
 summary, which was checked against the shipped strict config: ruff's D417 does
 not fire on a docstring with no parameter section.
 
-The sibling `Returns:` shape was measured and **rejected**: deleting a
-`Returns:` section makes DOC201 fire, so the only compliant remedy is deleting a
-docstring whose summary may be the valuable part. Two more were rejected on
-volume — property docstrings restating the property name and reST/epydoc type
-duplication (`:type x: int`, `:rtype:`) both measure **0** first-party findings
-outside generated code.
+Two more shapes were rejected on volume — property docstrings restating the
+property name and reST/epydoc type duplication (`:type x: int`, `:rtype:`) both
+measure **0** first-party findings outside generated code.
+
+### The `Returns:` half (0.36.0)
+
+```yaml
+    - id: sarj-docstring-returns-restate-signature  # SARJ087
+```
+
+`SARJ087` is the `Returns:` sibling of SARJ086, and it was **rejected once**:
+deleting a `Returns:` section used to make ruff's DOC201 fire, so the only
+compliant remedy was deleting the whole docstring. #164 then removed DOC201 from
+`ruff.strict.toml` as a rule that DEMANDS prose, and the premise expired — under
+the shipped config the section goes and the summary stays.
+
+756 findings over 33 OSS repos / 35,254 files; two seeded samples of 40 and 20
+read against source gave **~2%** false positives, the whole of which was one
+family (`Returns: A new X` — whether the value is a copy is the one thing
+`-> Self` cannot say) now guarded. Three findings on this repo's own source, all
+true, all deleted. Measurements: [docs/rules/SARJ087.md](../../docs/rules/SARJ087.md).
 
 ### House conventions moved out of consumer repos (0.21.0)
 

{sarj_python_lint-0.35.0 → sarj_python_lint-0.36.1}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "sarj-python-lint"
-version = "0.35.0"
+version = "0.36.1"
 description = "Custom Python lint rules — AST-based, pre-commit-friendly, hypermodern defaults"
 readme = "README.md"
 authors = [{ name = "sarj-ai" }]

{sarj_python_lint-0.35.0 → sarj_python_lint-0.36.1}/src/sarj_python_lint/rules/_registry.py

@@ -6,6 +6,9 @@ from sarj_python_lint.rules.conditional_assertion_in_test import ConditionalAsse
 from sarj_python_lint.rules.docstring_args_restate_signature import (
     DocstringArgsRestateSignature,
 )
+from sarj_python_lint.rules.docstring_returns_restate_signature import (
+    DocstringReturnsRestateSignature,
+)
 from sarj_python_lint.rules.duplicate_test_body import DuplicateTestBody
 from sarj_python_lint.rules.duplicated_override_docstring import (
     DuplicatedOverrideDocstring,
@@ -184,6 +187,7 @@ REGISTRY: dict[str, type[Rule]] = {
     DuplicatedOverrideDocstring.id: DuplicatedOverrideDocstring,
     RedundantClassDocstring.id: RedundantClassDocstring,
     DocstringArgsRestateSignature.id: DocstringArgsRestateSignature,
+    DocstringReturnsRestateSignature.id: DocstringReturnsRestateSignature,
 }
 
 __all__ = ["REGISTRY"]

sarj_python_lint-0.36.1/src/sarj_python_lint/rules/docstring_returns_restate_signature.py

@@ -0,0 +1,115 @@
+"""SARJ087 — A `Returns:` block that only re-spells the name and the return annotation.
+
+Examples: https://github.com/sarj-ai/standards/blob/main/packages/python/tests/rules/test_docstring_returns_restate_signature.py
+Evidence: https://github.com/sarj-ai/standards/blob/main/docs/rules/SARJ087.md
+"""
+
+from __future__ import annotations
+
+import ast
+import re
+from typing import TYPE_CHECKING, override
+
+from sarj_python_lint.rule_base import Diagnostic, Rule, parse_or_none
+from sarj_python_lint.rules._ast_index import children
+from sarj_python_lint.rules._comments import is_protected
+from sarj_python_lint.rules._docstrings import (
+    PROMPT_DECORATOR_MARKERS,
+    VALUE_MARKER_RE,
+    decorator_markers,
+    restates,
+    sections,
+    signature_stems,
+)
+from sarj_python_lint.rules._paths import is_generated
+
+
+if TYPE_CHECKING:
+    from pathlib import Path
+
+
+_RETURN_SECTIONS = ("Returns", "Return", "Yields", "Yield")
+
+# Identity semantics: whether the value handed back is a FRESH object or the
+# receiver itself is the one fact `-> Self` and `-> Foo` cannot carry, and the
+# words that state it (`new`, `copy`, `same`) are stopwords for the restatement
+# tokenizer, so the block reads as pure ceremony without this.
+_IDENTITY_RE = re.compile(
+    r"\b(?:new|copy|copies|copied|clone[ds]?|fresh|same|itself|self|shallow|deep|"
+    r"in[- ]place|unchanged|original)\b",
+    re.IGNORECASE,
+)
+
+
+def _return_block(docstring: str) -> str | None:
+    found = sections(docstring)
+    for name in _RETURN_SECTIONS:
+        if name in found:
+            return found[name]
+    return None
+
+
+class DocstringReturnsRestateSignature(Rule):
+    id: str = "docstring-returns-restate-signature"
+    code: str = "SARJ087"
+    has_evidence: bool = True
+    description: str = (
+        "`Returns:` block adds nothing the name and the return annotation do not "
+        "already say — delete the section and keep the summary."
+    )
+
+    @override
+    def check(self, path: Path, source: str) -> list[Diagnostic]:
+        if is_generated(path, source):
+            return []
+        tree = parse_or_none(path, source)
+        if tree is None:
+            return []
+        diags: list[Diagnostic] = []
+        self._walk(tree, None, path, diags)
+        return sorted(diags, key=lambda diag: diag.line)
+
+    def _walk(self, node: ast.AST, class_name: str | None, path: Path, diags: list[Diagnostic]) -> None:
+        for child in children(node):
+            if isinstance(child, ast.FunctionDef | ast.AsyncFunctionDef):
+                self._check_function(child, class_name, path, diags)
+                self._walk(child, class_name, path, diags)
+            elif isinstance(child, ast.ClassDef):
+                self._walk(child, child.name, path, diags)
+            else:
+                self._walk(child, class_name, path, diags)
+
+    def _check_function(
+        self,
+        node: ast.FunctionDef | ast.AsyncFunctionDef,
+        class_name: str | None,
+        path: Path,
+        diags: list[Diagnostic],
+    ) -> None:
+        docstring = ast.get_docstring(node, clean=True)
+        if not docstring:
+            return
+        block = _return_block(docstring)
+        if block is None:
+            return
+        if VALUE_MARKER_RE.search(block) or is_protected(block) or _IDENTITY_RE.search(block):
+            return
+        if decorator_markers(node) & PROMPT_DECORATOR_MARKERS:
+            return
+        stems = signature_stems(node, class_name)
+        # The whole-docstring case is SARJ050's; reporting it here too would
+        # make one deletion look like two findings.
+        if restates(docstring, stems):
+            return
+        if not restates(block, stems):
+            return
+        expr = node.body[0]
+        diags.append(
+            Diagnostic(
+                path=path,
+                line=expr.lineno,
+                col=expr.col_offset + 1,
+                code=self.code,
+                message=self.description,
+            )
+        )

{sarj_python_lint-0.35.0 → sarj_python_lint-0.36.1}/src/sarj_python_lint/rules/inefficient_string_concat_in_loop.py

@@ -19,6 +19,29 @@ if TYPE_CHECKING:
     from pathlib import Path
 
 
+def _src(node: ast.expr) -> str:
+    """`ast.unparse` for the shapes this rule compares, without `ast.unparse`'s cost.
+
+    The rule identifies an accumulator by the SOURCE TEXT of its target, so it unparses
+    once per Name/Attribute in every loop body. `ast.unparse` builds and runs a full
+    unparser on each call. For a bare `Name` the answer is the identifier, and for a
+    dotted chain over one it is the chain — byte-identical, far cheaper. Everything
+    else falls through to `ast.unparse`, so no comparison changes meaning.
+
+    This is why the rule stopped being the registry's performance outlier; see
+    `tests/test_perf.py`.
+
+    Returns:
+        The node's source text.
+
+    """
+    if isinstance(node, ast.Name):
+        return node.id
+    if isinstance(node, ast.Attribute) and isinstance(node.value, (ast.Name, ast.Attribute)):
+        return f"{_src(node.value)}.{node.attr}"
+    return ast.unparse(node)
+
+
 class InefficientStringConcatInLoop(Rule):
     id: str = "inefficient-string-concat-in-loop"
     code: str = "SARJ002"
@@ -99,7 +122,7 @@ class _ConcatVisitor(ast.NodeVisitor):
 
     def _is_probe_target(self, node: ast.AugAssign | ast.Assign) -> bool:
         """Report whether the concat target's intermediate values are consumed."""
-        target_src = ast.unparse(self._accumulation_target(node))
+        target_src = _src(self._accumulation_target(node))
         if any(target_src in names for names in self._while_probe_names):
             return True
         # Only the INNERMOST loop's reads consume the growth per iteration; a
@@ -109,7 +132,7 @@ class _ConcatVisitor(ast.NodeVisitor):
     def _is_loop_local_target(self, node: ast.AugAssign | ast.Assign) -> bool:
         """Report whether the concat target is freshly rebound earlier this iteration."""
         target = self._accumulation_target(node)
-        rebinds = self._loop_reassigns[-1].get(ast.unparse(target), ())
+        rebinds = self._loop_reassigns[-1].get(_src(target), ())
         return any(line < node.lineno for line in rebinds)
 
     def _accumulation_target(self, node: ast.AugAssign | ast.Assign) -> ast.expr:
@@ -149,7 +172,7 @@ class _ConcatVisitor(ast.NodeVisitor):
 
 def _test_names(test: ast.expr) -> set[str]:
     """Collect the source text of every Name/Attribute read in a while test."""
-    return {ast.unparse(n) for n in walk(test) if isinstance(n, (ast.Name, ast.Attribute))}
+    return {_src(n) for n in walk(test) if isinstance(n, (ast.Name, ast.Attribute))}
 
 
 def _loop_read_names(loop: ast.For | ast.AsyncFor | ast.While) -> frozenset[str]:
@@ -170,7 +193,7 @@ def _loop_read_names(loop: ast.For | ast.AsyncFor | ast.While) -> frozenset[str]
                 continue
         stack.extend(children(node))
         if isinstance(node, (ast.Name, ast.Attribute)) and isinstance(node.ctx, ast.Load):
-            reads.add(ast.unparse(node))
+            reads.add(_src(node))
     return frozenset(reads)
 
 
@@ -192,13 +215,13 @@ def _collect_reassignments(node: ast.AST, reassigns: dict[str, list[int]]) -> No
         for target in node.targets:
             for bound in _iter_binding_targets(target):
                 if not _is_accumulation_assign(bound, node.value):
-                    reassigns.setdefault(ast.unparse(bound), []).append(bound.lineno)
+                    reassigns.setdefault(_src(bound), []).append(bound.lineno)
     elif (
         isinstance(node, ast.AnnAssign)
         and node.value is not None
         and not _is_accumulation_assign(node.target, node.value)
     ):
-        reassigns.setdefault(ast.unparse(node.target), []).append(node.target.lineno)
+        reassigns.setdefault(_src(node.target), []).append(node.target.lineno)
     for child in children(node):
         _collect_reassignments(child, reassigns)
 
@@ -222,10 +245,10 @@ def _is_accumulation_assign(target: ast.expr, value: ast.expr) -> bool:
 
 def _other_add_operand(target: ast.expr, binop: ast.BinOp) -> ast.expr | None:
     """Return the non-target operand of `target + x` / `x + target`."""
-    target_src = ast.unparse(target)
-    if ast.unparse(binop.left) == target_src:
+    target_src = _src(target)
+    if _src(binop.left) == target_src:
         return binop.right
-    if ast.unparse(binop.right) == target_src:
+    if _src(binop.right) == target_src:
         return binop.left
     return None
 

{sarj_python_lint-0.35.0 → sarj_python_lint-0.36.1}/src/sarj_python_lint/rules/no_comment_cruft.py

@@ -56,6 +56,22 @@ _LICENSE_RE = re.compile(
     re.IGNORECASE,
 )
 
+# A licence header is a legally required block of text a contributor cannot
+# restructure, and the `# ---------` rules that box it in are part of it. The
+# rule already knows this -- `_flag_leading_preamble` returns early on
+# `_LICENSE_RE` -- but the BANNER branch did not, so "structure code with
+# functions, not ASCII rules" was reported against the one comment in the file
+# that is not about structure at all. 831 findings across 590 corpus files sat
+# on a punctuation rule in the first 8 lines of a file whose header carries a
+# copyright or SPDX line: `bokeh/docs/bokeh/docserver.py:1`,
+# `bokeh/release/checks.py:1`, `bokeh/release/__main__.py:1` are the shape --
+# `# ---…---` immediately above `# Copyright (c) Anaconda, Inc.`.
+#
+# Scoped to the file HEADER, not to every copyright mention: a banner beside a
+# licence reference in the middle of a file is an ordinary section rule.
+_LICENSE_HEADER_MAX_LINE = 8
+_LICENSE_HEADER_RADIUS = 4
+
 _BANNER_FULL_RE = re.compile(r"^[-=#*~_+.\s]{4,}$")
 # `[\u2500-\u257f]` is the Unicode box-drawing block. A `────────` rule is the
 # same section separator as `--------`, just prettier; 34 of them were sitting in
@@ -441,6 +457,24 @@ def _is_assign_or_call(snippet: str) -> bool:
     return isinstance(stmt, ast.Expr) and isinstance(stmt.value, ast.Call)
 
 
+def _license_header_lines(standalone: list[tuple[int, int, str]]) -> frozenset[int]:
+    """Collect the lines belonging to a file-header licence block.
+
+    Returns:
+        Every line within `_LICENSE_HEADER_RADIUS` of a licence comment that
+        sits in the first `_LICENSE_HEADER_MAX_LINE` lines of the file.
+
+    """
+    anchors = [
+        line for line, _, body in standalone if line <= _LICENSE_HEADER_MAX_LINE and _LICENSE_RE.search(body)
+    ]
+    return frozenset(
+        line
+        for anchor in anchors
+        for line in range(anchor - _LICENSE_HEADER_RADIUS, anchor + _LICENSE_HEADER_RADIUS + 1)
+    )
+
+
 class NoCommentCruft(Rule):
     id: str = "no-comment-cruft"
     code: str = "SARJ016"
@@ -468,6 +502,7 @@ class NoCommentCruft(Rule):
         referenced = _externally_referenced_lines(standalone)
         nested = nested_comment_lines(source)
         enumerated = [line for line, _, body in standalone if _ENUMERATION_RE.match(body)]
+        license_header = _license_header_lines(standalone)
         for line, col, body in standalone:
             if _is_directive(body) or _is_coding_cookie(body) or line in skip:
                 continue
@@ -478,6 +513,7 @@ class NoCommentCruft(Rule):
                 narration_protected=line in referenced,
                 isolated_enumeration=enumerated == [line],
                 nested=line in nested,
+                in_license_header=line in license_header,
             )
             if msg is not None:
                 diags[line] = Diagnostic(path=path, line=line, col=col + 1, code=self.code, message=msg)
@@ -492,6 +528,7 @@ class NoCommentCruft(Rule):
         narration_protected: bool,
         isolated_enumeration: bool,
         nested: bool,
+        in_license_header: bool,
     ) -> str | None:
         if _CODE_REGEN_CALL_RE.match(body):
             return None
@@ -500,7 +537,7 @@ class NoCommentCruft(Rule):
                 return "Untracked TODO/FIXME marker — add an issue ticket or context link."
             return None
         if _is_banner(body):
-            if _is_heading_underline(body, prev_body):
+            if _is_heading_underline(body, prev_body) or in_license_header:
                 return None
             return "Section-banner / region comment — structure code with functions, not ASCII rules."
         if _looks_like_code(body):

{sarj_python_lint-0.35.0 → sarj_python_lint-0.36.1}/src/sarj_python_lint/rules/no_implicit_attribute_access.py

@@ -28,6 +28,13 @@ _EXCLUDED_BASES = {
     "config",
     "kwargs",
     "env",
+    # A deliberately OPEN extension bag: the framework guarantees the mapping
+    # exists and guarantees nothing about its keys, because third-party code is
+    # what puts them there. `scrapy/core/downloader/handlers/http11.py:531`
+    # (`request.meta.get("download_maxsize", self._maxsize)`) is the shape --
+    # Scrapy documents `Request.meta` as the per-request extension dict, and no
+    # model can enumerate keys that downstream middlewares invent.
+    "meta",
     "os",
     "sys",
 }
@@ -114,6 +121,7 @@ class _FileFacts:
     """The whole-file context a single subscript cannot answer for itself."""
 
     annotation_nodes: frozenset[int]
+    decorator_nodes: frozenset[int]
     mutation_receivers: frozenset[int]
     schema_bound_names: frozenset[str]
     constant_tables: frozenset[str]
@@ -242,6 +250,8 @@ def _is_exempt(node: ast.Call | ast.Subscript, key: str, facts: _FileFacts) -> b
         return True
     if id(node) in facts.annotation_nodes:
         return True
+    if id(node) in facts.decorator_nodes:
+        return True
     if isinstance(node, ast.Subscript) and id(node) in facts.mutation_receivers:
         return True
     receiver = _receiver(node)
@@ -278,9 +288,10 @@ def _file_facts(tree: ast.Module) -> _FileFacts:
         The four indexes, each built from the memoized per-file node index.
 
     """
-    typed_dicts = _typed_dict_class_names(tree)
+    typed_dicts = _typed_dict_class_names(tree) | _declared_type_names(tree)
     return _FileFacts(
         annotation_nodes=_annotation_nodes(tree),
+        decorator_nodes=_decorator_nodes(tree),
         mutation_receivers=_mutation_receivers(tree),
         schema_bound_names=_schema_bound_names(tree, typed_dicts) if typed_dicts else frozenset(),
         constant_tables=_constant_tables(tree),
@@ -308,6 +319,27 @@ def _annotation_nodes(tree: ast.Module) -> frozenset[int]:
     return frozenset(id(inner) for root in roots for inner in walk(root))
 
 
+def _decorator_nodes(tree: ast.Module) -> frozenset[int]:
+    """Collect the identity of every node inside a decorator expression.
+
+    A decorator is never a mapping lookup, so `@router.get("")` is not one --
+    but `_looks_like_route_or_url` only recognises a value starting with `/` or
+    containing `://`, and the EMPTY-STRING route (the router-root registration
+    FastAPI projects write) slips through both. 23 findings across airflow,
+    litellm and prefect were exactly `@<router>.get("")` on the decorator line.
+    Position answers it exactly where the argument cannot, and costs no recall:
+    nothing in `decorator_list` is a payload field read.
+
+    Returns:
+        `id()` of each node in a decorator subtree.
+
+    """
+    roots: list[ast.expr] = []
+    for node in nodes(tree, ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef):
+        roots.extend(node.decorator_list)
+    return frozenset(id(inner) for root in roots for inner in walk(root))
+
+
 def _mutation_receivers(tree: ast.Module) -> frozenset[int]:
     """Collect subscripts that are the receiver of an in-place collection method.
 
@@ -403,6 +435,40 @@ def _parse_type_text(text: str) -> ast.expr | None:
         return None
 
 
+# Modules whose exports are containers and escape hatches rather than schemas.
+# `from typing import Any` followed by `payload: Any` declares nothing.
+_STRUCTURELESS_IMPORT_MODULES = frozenset(
+    {"builtins", "collections", "collections.abc", "typing", "typing_extensions"}
+)
+
+
+def _declared_type_names(tree: ast.Module) -> frozenset[str]:
+    """Collect names imported into this module that could name a declared shape.
+
+    A receiver annotated with a type this file IMPORTS has already been given a
+    schema by its author -- most often a TypedDict, which subscripts by design:
+    `AllMessageValues` in `litellm/…/prompt_templates/factory.py` is OpenAI's
+    message TypedDict, and `current_message["role"]` is the DECLARATIVE form,
+    not a substitute for one. `_typed_dict_class_names` sees only `class X
+    (TypedDict)` written in the same file, so every cross-module TypedDict --
+    which is nearly all of them -- was invisible.
+
+    This cannot resolve the import to check what it really is, so it is
+    deliberately generous: `typing`/`collections` exports are excluded because
+    `Any` and `dict` declare nothing, and everything else is taken at its word.
+
+    Returns:
+        The imported names usable as an annotation head.
+
+    """
+    declared: set[str] = set()
+    for node in nodes(tree, ast.Import, ast.ImportFrom):
+        if isinstance(node, ast.ImportFrom) and node.module in _STRUCTURELESS_IMPORT_MODULES:
+            continue
+        declared.update(alias.asname or alias.name.split(".")[0] for alias in node.names)
+    return frozenset(name for name in declared if name[:1].isupper())
+
+
 def _schema_bound_names(tree: ast.Module, typed_dicts: frozenset[str]) -> frozenset[str]:
     """Collect names whose declared type is one of this file's TypedDicts.