hackscan 0.3.0__tar.gz → 0.3.2__tar.gz

{hackscan-0.3.0 → hackscan-0.3.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: hackscan
-Version: 0.3.0
+Version: 0.3.2
 Summary: Python SAST orchestrator: own AST + taint engine, imports Semgrep/Bandit/CodeQL/Gitleaks, dedupes, emits SARIF.
 Project-URL: Source, https://github.com/kahledawad2019-netizen/hackscan
 Project-URL: Issues, https://github.com/kahledawad2019-netizen/hackscan/issues
@@ -202,7 +202,13 @@ HackScan's authors, so treat both tables as a sanity check rather than a ranking
 
 Sanitizers recognized: numeric/UUID conversions (all classes), `psycopg` `sql.Identifier`
 /`sql.Literal` (SQL). `shlex.quote` is POSIX-only, so quoted commands are kept as
-low-confidence candidates rather than suppressed.
+low-confidence candidates rather than suppressed. That verdict needs the command to start with
+a literal name from the same list of inert programs that command fixes use (plus `echo`, but
+not `printf`, whose shell builtin can evaluate a variable name; not `sh -c`, `ssh`, `xargs`,
+`tar`, interpreters, ...) and command text without shell syntax that starts or groups commands
+(`;`, `|`, `&`, `$(...)`, redirections, braces): a quoted value in the program position, or a
+program name assembled at runtime, gets none; nor does a call that may pick another shell
+(`executable=`, options passed with `**`).
 
 ### Custom rules
 
@@ -241,7 +247,7 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v4
-      - uses: kahledawad2019-netizen/hackscan@v0.3.0
+      - uses: kahledawad2019-netizen/hackscan@v0.3.2
         with:
           path: src
           fail-on: high
@@ -273,7 +279,7 @@ Python 3.12 on the runner. Without the action:
 ```yaml
 repos:
   - repo: https://github.com/kahledawad2019-netizen/hackscan
-    rev: v0.3.0
+    rev: v0.3.2
     hooks:
       - id: hackscan
 ```
@@ -298,15 +304,30 @@ apply as usual.
   confirmed or left as candidates: an assignment expression can rebind a name in the
   middle of an expression, which the statement-level analysis does not order. Sinks in
   the body of a generator expression are never suppressed either (it runs later), nor
-  are values produced by one. Module-level constants are trusted only when bound exactly
-  once and the module never writes its namespace dynamically (`globals()`, `vars()`,
-  `exec`, `eval`, `__dict__`, star imports); `global`/`nonlocal` names are never trusted.
+  are values produced by one, except by a generator that `str.join` consumes right away.
+  Module-level constants are trusted only when bound exactly once and the module never
+  writes its namespace dynamically (`globals()`, `vars()`, `exec`, `eval`, `__dict__`,
+  star imports); `global`/`nonlocal` names are never trusted.
 - Mutable objects (lists, dicts, sets) are tracked by variable name. Mutation through
   direct aliases, method calls (`d.get(k).append(x)`), bound methods
   (`add = parts.append`), conditional receivers and comprehension items is modeled;
   aliases created in less direct ways (stored in another object's attribute or in a
   container and mutated through it later, returned from a helper, reached through
   `getattr`) may be missed, so a constant suppression involving such code deserves review.
+- A sanitizer counts only through a name that cannot be something else in the same file:
+  one import or definition, no assignment to an attribute of that name, no `setattr` /
+  `delattr`. Replacing a sanitizer at runtime in other ways (from another file, or in the
+  same file through an alias of `setattr`, `__setattr__`, `__dict__` writes and the like)
+  is not modeled.
+- Sanitizer credit computed in one function is reused by its callers and callees, so a
+  transformation in another function that undoes it (slicing a shell-quoted argument,
+  for example) can leave such a finding a candidate instead of confirming it.
+- Numbers from `int()`, `float()`, `len()` and similar count as safe in SQL text, and SQL
+  token boundaries are not modeled: text whose structure a runtime number decides is not
+  detected, such as a `-` right before a possibly negative number (`"n -" + str(int(x))`
+  can form a `--` comment), a fragment repeated a runtime number of times (`"-" * int(x)`),
+  or a separator repeated between a runtime number of items. Bound query parameters avoid
+  all of these.
 - Python only. Imported tools may cover other languages; their findings are passed through.
 - Fix suggestions are deliberately conservative: many vulnerable lines get none. Command
   fixes (argument lists) are only offered for programs whose arguments are inert data

{hackscan-0.3.0 → hackscan-0.3.2}/PROJECT.md

@@ -452,3 +452,37 @@ hackscan/
   re-run so the HackScan rows show 0.3.0 (numbers unchanged). Next ideas (not started):
   weak-cipher detection (DES/RC4/ECB/unauthenticated modes; largest benchmark gap),
   taint through `self` attributes, further pass-2 aliasing hardening.
+- 2026-10-12 (v0.3.1): template fixes only, found by the security agent's held-out evaluation
+  (8 of 13 true findings had no template). A call copied verbatim from the original call
+  (`params.get("id")`, `re.sub(...)`) no longer counts as introduced code (multiset count, so
+  no duplication); SQL fixes now decompose `"...'{}'...".format(args)` (plain auto-numbered
+  fields, positional args used exactly once; never for command argv, where a non-str value would
+  crash). Detection is unchanged, so the benchmark tables still show the 0.3.0 measurement (Bandit,
+  Semgrep and CodeQL are not installed on the release machine to re-run every row).
+- 2026-10-13 (v0.3.2): soundness fixes found while reviewing an attempt to credit
+  datasette's `escape_sqlite` as an SQL identifier sanitizer. That attempt was dropped (Claude + Codex,
+  user-confirmed): the helper leaves words missing from its own reserved list bare (`returning`, `true`),
+  so it is no sound sanitizer. The review rounds instead found sanitizer-credit bugs that shipped in 0.3.1,
+  fixed here (tests/test_quoting_credit.py): shell-quoting credit is allow-listed (it survives only whole
+  insertion into plain text; containers, iteration, unpacking, indexing, methods, `!r`/padding, `%c`,
+  bytes and decoding drop it); text that may render quotes (bytes, containers, attributes, method results,
+  repr-like builtins, sliced quoting output, backslashes) is not quote-free; sanitizer names must be
+  stable (one binding, no attribute assignment, no `setattr`) in taint and LLM evidence; number credit
+  does not survive conversion to characters (methods other than case/strip/zfill, `%c`/`{:c}`, fill
+  specs, unknown templates, `format`/`bytes`/`bytearray`) or attribute access (`uuid.UUID(x).bytes`);
+  psycopg `sql.Identifier`/`sql.Literal` keep credit only while passed on as is (also not across calls);
+  `.format` field selectors and `*args`/`**kwargs` mappings; constant text indexed at runtime is no
+  constant; iteration items are no containers; `abs`/`round` pass their argument's credit through
+  (their dunders may return anything) instead of sanitizing; LLM sanitizer evidence is checked with the
+  taint engine itself (the credit must survive the sink's use of the value); the POSIX quoting verdict
+  needs the command to start with a literal inert program (the fix generator's list, now shared in
+  `taint/sanitizers.py`, plus `echo`, minus `printf`: bash's builtin evaluates a `-v` variable name)
+  and no shell syntax that starts or groups commands (user choice: allow-list, not a denylist of code
+  runners), and a sink that cannot pick another shell (`executable=`, Popen's positional executable,
+  `**` options). User decisions: in-file sanitizer replacement beyond the simple checks, credit
+  inherited across calls (missed confirmations only) and numbers/constants shaping SQL text
+  structure (sign next to `-`, runtime repetition) are documented README limitations (the last
+  pinned by strict-xfail tests). Codex review: 13 rounds, final "HS OK". The security agent's
+  held-out eval is identical to 0.3.1 on every finding, and the benchmark's HackScan numbers are
+  unchanged (`test_published_hackscan_numbers_are_current`), so the tables keep the 0.3.0
+  measurement (Bandit, Semgrep and CodeQL are not installed here to re-run every row).

{hackscan-0.3.0 → hackscan-0.3.2}/README.md

@@ -176,7 +176,13 @@ HackScan's authors, so treat both tables as a sanity check rather than a ranking
 
 Sanitizers recognized: numeric/UUID conversions (all classes), `psycopg` `sql.Identifier`
 /`sql.Literal` (SQL). `shlex.quote` is POSIX-only, so quoted commands are kept as
-low-confidence candidates rather than suppressed.
+low-confidence candidates rather than suppressed. That verdict needs the command to start with
+a literal name from the same list of inert programs that command fixes use (plus `echo`, but
+not `printf`, whose shell builtin can evaluate a variable name; not `sh -c`, `ssh`, `xargs`,
+`tar`, interpreters, ...) and command text without shell syntax that starts or groups commands
+(`;`, `|`, `&`, `$(...)`, redirections, braces): a quoted value in the program position, or a
+program name assembled at runtime, gets none; nor does a call that may pick another shell
+(`executable=`, options passed with `**`).
 
 ### Custom rules
 
@@ -215,7 +221,7 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v4
-      - uses: kahledawad2019-netizen/hackscan@v0.3.0
+      - uses: kahledawad2019-netizen/hackscan@v0.3.2
         with:
           path: src
           fail-on: high
@@ -247,7 +253,7 @@ Python 3.12 on the runner. Without the action:
 ```yaml
 repos:
   - repo: https://github.com/kahledawad2019-netizen/hackscan
-    rev: v0.3.0
+    rev: v0.3.2
     hooks:
       - id: hackscan
 ```
@@ -272,15 +278,30 @@ apply as usual.
   confirmed or left as candidates: an assignment expression can rebind a name in the
   middle of an expression, which the statement-level analysis does not order. Sinks in
   the body of a generator expression are never suppressed either (it runs later), nor
-  are values produced by one. Module-level constants are trusted only when bound exactly
-  once and the module never writes its namespace dynamically (`globals()`, `vars()`,
-  `exec`, `eval`, `__dict__`, star imports); `global`/`nonlocal` names are never trusted.
+  are values produced by one, except by a generator that `str.join` consumes right away.
+  Module-level constants are trusted only when bound exactly once and the module never
+  writes its namespace dynamically (`globals()`, `vars()`, `exec`, `eval`, `__dict__`,
+  star imports); `global`/`nonlocal` names are never trusted.
 - Mutable objects (lists, dicts, sets) are tracked by variable name. Mutation through
   direct aliases, method calls (`d.get(k).append(x)`), bound methods
   (`add = parts.append`), conditional receivers and comprehension items is modeled;
   aliases created in less direct ways (stored in another object's attribute or in a
   container and mutated through it later, returned from a helper, reached through
   `getattr`) may be missed, so a constant suppression involving such code deserves review.
+- A sanitizer counts only through a name that cannot be something else in the same file:
+  one import or definition, no assignment to an attribute of that name, no `setattr` /
+  `delattr`. Replacing a sanitizer at runtime in other ways (from another file, or in the
+  same file through an alias of `setattr`, `__setattr__`, `__dict__` writes and the like)
+  is not modeled.
+- Sanitizer credit computed in one function is reused by its callers and callees, so a
+  transformation in another function that undoes it (slicing a shell-quoted argument,
+  for example) can leave such a finding a candidate instead of confirming it.
+- Numbers from `int()`, `float()`, `len()` and similar count as safe in SQL text, and SQL
+  token boundaries are not modeled: text whose structure a runtime number decides is not
+  detected, such as a `-` right before a possibly negative number (`"n -" + str(int(x))`
+  can form a `--` comment), a fragment repeated a runtime number of times (`"-" * int(x)`),
+  or a separator repeated between a runtime number of items. Bound query parameters avoid
+  all of these.
 - Python only. Imported tools may cover other languages; their findings are passed through.
 - Fix suggestions are deliberately conservative: many vulnerable lines get none. Command
   fixes (argument lists) are only offered for programs whose arguments are inert data

{hackscan-0.3.0 → hackscan-0.3.2}/pyproject.toml

@@ -67,3 +67,4 @@ Issues = "https://github.com/kahledawad2019-netizen/hackscan/issues"
 "tests/test_remediate.py" = ["E501"]
 "tests/test_llm.py" = ["E501"]
 "tests/test_ui.py" = ["E501"]
+"tests/test_quoting_credit.py" = ["E501"]

{hackscan-0.3.0 → hackscan-0.3.2}/src/hackscan/__init__.py

@@ -1,3 +1,3 @@
 """HackScan: Python SAST orchestrator and verifier."""
 
-__version__ = "0.3.0"
+__version__ = "0.3.2"

{hackscan-0.3.0 → hackscan-0.3.2}/src/hackscan/analyzers/llm_pass.py

@@ -386,6 +386,7 @@ def verify_evidence(ctx: FileContext, finding: Finding, line: int, kind: str) ->
             _is_constant_value(stmt.value)
             if kind == "constant"
             else _is_class_sanitizer(stmt.value, ctx, finding.vuln_class)
+            and _credit_reaches_sink(ctx, func, stmt, call, finding.vuln_class)
         )
         covered = {target.id}
     elif kind == "guard":
@@ -404,6 +405,21 @@ def verify_evidence(ctx: FileContext, finding: Finding, line: int, kind: str) ->
     return ok and not _rebound_after(func, covered, stmt)
 
 
+def _credit_reaches_sink(
+    ctx: FileContext, func: ast.AST, stmt: ast.Assign, call: ast.Call, vuln_class: str
+) -> bool:
+    """Whether the cited sanitizer's credit survives the sink's use of the value, by the
+    taint engine's own rules (`"%c" % n` or an f-string of a psycopg object drop it)."""
+    from hackscan.analyzers.taint_pass import _FileEngine, _FunctionEngine, combine, source
+
+    engine = _FunctionEngine(_FileEngine(ctx, {}), func, frozenset())
+    name = stmt.targets[0].id  # type: ignore[attr-defined]
+    value = engine.eval(stmt.value, {name: source("untrusted input", stmt)})
+    args = [*call.args, *(k.value for k in call.keywords)]
+    result = combine(engine.eval(a, {name: value}) for a in args)
+    return vuln_class in result.safe_for
+
+
 def _sink_value_names(call: ast.Call) -> set[str] | None:
     """Every name the sink's arguments read, or None if they contain anything else that
     could carry data (calls, attributes, subscripts...). A suppression must vouch for
@@ -500,10 +516,13 @@ def _is_constant_value(expr: ast.AST) -> bool:
 
 
 def _is_class_sanitizer(expr: ast.AST, ctx: FileContext, vuln_class: str) -> bool:
+    from hackscan.analyzers.taint_pass import _FileEngine
     from hackscan.taint.sanitizers import QUOTING_SANITIZERS, SANITIZERS
 
     if not isinstance(expr, ast.Call):
         return False
+    if not _FileEngine(ctx, {}).stable_callee(expr):
+        return False  # `int = ...` / `uuid.UUID = ...` elsewhere: the call may be another
     return any(
         vuln_class in SANITIZERS.get(name, frozenset()) and name not in QUOTING_SANITIZERS
         for name in ctx.call_names(expr)
@@ -779,7 +798,7 @@ def _same_sql(original: ast.Call, new: ast.Call, ctx: FileContext) -> bool:
         return False
     if not isinstance(params, (ast.Tuple, ast.List)):
         return False
-    parts = _sql_parts(original.args[0])
+    parts = _sql_parts(original.args[0], sql=True)
     if parts is None:
         return False
     # The placeholder style must be the one the file's driver uses (`?` breaks psycopg);

{hackscan-0.3.0 → hackscan-0.3.2}/src/hackscan/analyzers/remediate.py

@@ -15,6 +15,7 @@ from __future__ import annotations
 import ast
 import difflib
 import re
+import string
 from collections.abc import Callable
 from dataclasses import replace
 
@@ -23,6 +24,7 @@ from hackscan.core.taxonomy import CMDI, CODEI, SQLI, WEAK_CRYPTO
 from hackscan.importers.common import SourceIndex
 from hackscan.plugins.base import FileContext, first_arg, keyword
 from hackscan.plugins.loader import builtin_plugins
+from hackscan.taint.sanitizers import INERT_PROGRAMS, program_name
 
 PRODUCER = "template"
 
@@ -157,7 +159,7 @@ def _sqli(call: ast.Call, ctx: FileContext):
     placeholder = _placeholder(ctx)
     if placeholder is None:
         return None
-    parts = _sql_parts(call.args[0])
+    parts = _sql_parts(call.args[0], sql=True)
     if parts is None or not any(isinstance(p, ast.AST) for p in parts):
         return None
     sql, params = _parameterize(parts, placeholder)
@@ -193,8 +195,13 @@ def _placeholder(ctx: FileContext) -> str | None:
     return "?" if qmark else "%s"
 
 
-def _sql_parts(expr: ast.AST) -> list[str | ast.AST] | None:
-    """Literal text and value expressions, in order, for f-strings, `+` and `%`."""
+def _sql_parts(expr: ast.AST, *, sql: bool = False) -> list[str | ast.AST] | None:
+    """Literal text and value expressions, in order, for f-strings, `+` and `%`.
+
+    `sql=True` also decomposes `"...".format(...)`. A bound SQL value skips the text conversion
+    (`__format__("")` for `{}`, like `__str__` for `%s` and f-strings) and keeps its type, as the
+    SQL fix description already says. Command argv must stay text, so `_argv` never uses it
+    (`"cmd {}".format(1)` would become a non-str argument and crash)."""
     if isinstance(expr, ast.JoinedStr):
         parts: list[str | ast.AST] = []
         for v in expr.values:
@@ -210,7 +217,7 @@ def _sql_parts(expr: ast.AST) -> list[str | ast.AST] | None:
                 return None
         return parts
     if isinstance(expr, ast.BinOp) and isinstance(expr.op, ast.Add):
-        left, right = _operand(expr.left), _operand(expr.right)
+        left, right = _operand(expr.left, sql=sql), _operand(expr.right, sql=sql)
         if left is None or right is None:
             return None
         return [*left, *right]
@@ -230,17 +237,49 @@ def _sql_parts(expr: ast.AST) -> list[str | ast.AST] | None:
             else:
                 out.append(values[i // 2])
         return out
+    if (
+        sql
+        and isinstance(expr, ast.Call)
+        and isinstance(expr.func, ast.Attribute)
+        and expr.func.attr == "format"
+        and isinstance(expr.func.value, ast.Constant)
+        and isinstance(expr.func.value.value, str)
+        and not expr.keywords
+        and not any(isinstance(a, ast.Starred) for a in expr.args)
+    ):
+        return _format_parts(expr.func.value.value, expr.args)
     if isinstance(expr, ast.Constant) and isinstance(expr.value, str):
         return [expr.value]
     return None
 
 
-def _operand(expr: ast.AST) -> list[str | ast.AST] | None:
+def _format_parts(template: str, values: list[ast.AST]) -> list[str | ast.AST] | None:
+    """`"...{}...".format(a, b)`: only auto-numbered `{}` fields without a conversion or
+    format spec, each argument used exactly once in order. Each field calls the value's
+    `__format__("")`, which usually equals `str()` (as `%s` does) but a custom type may differ."""
+    try:
+        fields = list(string.Formatter().parse(template))
+    except ValueError:
+        return None
+    out: list[str | ast.AST] = []
+    used = 0
+    for literal, name, spec, conversion in fields:
+        out.append(literal)
+        if name is None:
+            continue
+        if name != "" or spec or conversion is not None or used >= len(values):
+            return None
+        out.append(values[used])
+        used += 1
+    return out if used == len(values) else None
+
+
+def _operand(expr: ast.AST, *, sql: bool = False) -> list[str | ast.AST] | None:
     """An operand of `+`: string-building expressions decompose, anything else is a value."""
     if isinstance(expr, (ast.JoinedStr, ast.BinOp)) or (
         isinstance(expr, ast.Constant) and isinstance(expr.value, str)
     ):
-        return _sql_parts(expr)
+        return _sql_parts(expr, sql=sql)
     if isinstance(expr, ast.Constant):
         return None  # e.g. "id = " + 5: not a str concatenation
     return [expr]
@@ -436,23 +475,7 @@ def _argv(expr: ast.AST, ctx: FileContext) -> list[str] | None:
     return argv
 
 
-# Programs whose arguments are data: no option or operand runs a command, loads code or
-# writes an arbitrary file. An argument list only makes a dynamic value safe for these;
-# countless others execute operands or options (`sh -c`, `npx PKG`, `git -c alias=!cmd`,
-# `tar --to-command`, `sort --compress-program`, `find -exec`, `rg --pre`), so an
-# allowlist, not a denylist.
-INERT_PROGRAMS = {
-    "printf", "cat", "tac", "nl", "ls", "head", "tail", "wc", "cut",
-    "grep", "egrep", "fgrep", "file", "stat", "du", "df", "basename", "dirname",
-    "realpath", "readlink", "which", "whoami", "id", "ping", "host", "dig", "nslookup",
-}  # fmt: skip
-
-
-def program_name(literal: str) -> str:
-    """`/bin/grep` or `C:\\Tools\\grep.exe` -> `grep`. Only `.exe` is dropped: `.bat` and
-    `.cmd` files run through cmd.exe, which re-parses the arguments."""
-    name = literal.replace("\\", "/").rsplit("/", 1)[-1].lower()
-    return name[: -len(".exe")] if name.endswith(".exe") else name
+# INERT_PROGRAMS / program_name: see hackscan.taint.sanitizers (shared with taint_pass).
 
 
 def _argv_is_inert(words: list[str | None]) -> bool: