hackscan 0.2.0__tar.gz → 0.3.1__tar.gz

hackscan-0.3.1/.github/workflows/hackscan.yml

@@ -0,0 +1,22 @@
+name: HackScan
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+
+permissions:
+  contents: read
+  security-events: write
+
+jobs:
+  scan:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: ./
+        with:
+          path: src
+          fail-on: high
+          # Fork pull requests get a read-only token, so the Code Scanning upload is skipped there.
+          upload-sarif: ${{ github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository }}

{hackscan-0.2.0 → hackscan-0.3.1}/.gitignore

@@ -15,3 +15,6 @@ htmlcov/
 # Local tool caches/sandbox scratch (never packaged)
 .uv-cache/
 .test-tmp/
+
+# benchmark downloads
+benchmarks/.cache/

hackscan-0.3.1/.pre-commit-hooks.yaml

@@ -0,0 +1,9 @@
+- id: hackscan
+  name: HackScan
+  description: Scan the repository for Python security findings.
+  language: python
+  entry: hackscan scan
+  args: [".", "--fail-on", "high"]
+  pass_filenames: false
+  types: [python]
+  require_serial: true

{hackscan-0.2.0 → hackscan-0.3.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: hackscan
-Version: 0.2.0
+Version: 0.3.1
 Summary: Python SAST orchestrator: own AST + taint engine, imports Semgrep/Bandit/CodeQL/Gitleaks, dedupes, emits SARIF.
 Project-URL: Source, https://github.com/kahledawad2019-netizen/hackscan
 Project-URL: Issues, https://github.com/kahledawad2019-netizen/hackscan/issues
@@ -46,6 +46,7 @@ HIGH     confirmed  views.py:10:9  HS-SQLI-001  [100%]
 |---|---|
 | **Verdicts, not just matches** | Every injection candidate is **confirmed** (untrusted source reaches it, with a trace), **suppressed** (provably constant or sanitized on every path), or left as a **candidate** for review. |
 | **Sound suppression** | A finding is only dismissed if *every* path is safe. Branches, loops (`break`/`continue`), `try`/`except`/`finally`, aliasing and mutation of lists/dicts, closures and `:=` are modeled. A differential fuzzer executes thousands of random programs to check that no dismissed sink ever receives attacker input. |
+| **Across functions and files** | A sink fed by a function parameter is confirmed when a caller anywhere in the scanned code passes untrusted data (directly or through other functions), or when a called function returns it; the trace names the source file and line. |
 | **Framework-aware** | Sources: Flask `request.*`, Django/DRF/Starlette/FastAPI request objects (incl. `self.request`), route-handler parameters, `input()`, `sys.argv`. |
 | **One report from many tools** | Imports Semgrep, Bandit, CodeQL (SARIF) and Gitleaks; deduplicates by vulnerability class and location, keeping every tool as provenance. |
 | **Fixes you can apply** | Mechanical, validated fix suggestions: parameterized queries (driver-aware placeholders), argument lists instead of shell strings, `ast.literal_eval`, SHA-256. Each is an exact edit that re-parses; ambiguous cases get no fix rather than a wrong one. |
@@ -77,7 +78,7 @@ hackscan rules                                    # list rules
 | Option | Meaning |
 |---|---|
 | `--severity`, `--min-confidence` | Report filters. |
-| `--fail-on SEV` | Exit 1 if an open (candidate/confirmed) finding is at least `SEV`. Suppressed findings never count. |
+| `--fail-on SEV` | Exit 1 if an open (candidate/confirmed) finding is at least `SEV`. Suppressed findings never count. `never` turns off a threshold set in `.hackscan.yml`. |
 | `--ignore GLOB` | Skip paths (`tests/*`, `**/migrations/**`, `legacy`). `.venv`, `node_modules`, `build`, ... are always skipped. |
 | `--with TOOLS` / `--import FMT=FILE` | Run external tools / import reports (`sarif`, `semgrep`, `bandit`, `codeql`, `gitleaks`). A missing or failing tool is a warning unless `--strict-tools`. |
 | `-o FILE` | Write the report to `FILE`. Never overwrites anything except a previous HackScan report. |
@@ -152,6 +153,44 @@ Secrets are never echoed: findings of class `secret` (from any tool) have generi
 messages and redacted snippets, and values reported by Gitleaks are scrubbed from all
 output, including warnings.
 
+## Benchmark
+
+**Labeled suite.** Flask, Django, FastAPI and CLI code with 26 vulnerable sinks and 24
+look-alike safe traps (SQL, command and code injection, weak crypto), scored per function;
+each tool runs the configuration in its Version column (Bandit: all checks; Semgrep CE:
+the `p/python` ruleset; CodeQL: `python-security-extended`; HackScan: no options)
+([methodology and caveats](benchmarks/README.md), [full results](benchmarks/RESULTS.md)):
+
+| Tool | Version | TP | FP | FN | Precision | Recall | F1 |
+|---|---|---:|---:|---:|---:|---:|---:|
+| HackScan | 0.3.0 | 23 | 7 | 3 | 77% | 88% | 82% |
+| HackScan (confirmed only) | 0.3.0 | 18 | 2 | 8 | 90% | 69% | 78% |
+| Bandit | bandit[sarif]==1.9.4 | 24 | 19 | 2 | 56% | 92% | 70% |
+| Semgrep CE | semgrep==1.180.0 (p/python sha256:31c1dfa46e8d) | 16 | 6 | 10 | 73% | 62% | 67% |
+| CodeQL | CodeQL 2.27.2 (codeql/python-queries@1.8.12, security-extended) | 20 | 1 | 6 | 95% | 77% | 85% |
+
+**Known-vulnerable apps.** [vulpy](https://github.com/fportantier/vulpy),
+[PyGoat](https://github.com/adeyosemanputra/pygoat) and
+[DVPWA](https://github.com/anxolerd/dvpwa) at pinned commits; every report reviewed by
+hand. There is no complete ground truth, so recall is relative to the 21 distinct
+vulnerabilities any tool found ([details](benchmarks/APPS.md)):
+
+| Tool | Reports | TP | FP | Vulns found | Precision | Relative recall |
+|---|---:|---:|---:|---:|---:|---:|
+| HackScan | 18 | 15 | 3 | 15 | 83% | 71% |
+| HackScan (confirmed only) | 6 | 6 | 0 | 6 | 100% | 29% |
+| Bandit | 32 | 16 | 16 | 16 | 50% | 76% |
+| Semgrep CE | 12 | 11 | 1 | 11 | 92% | 52% |
+| CodeQL | 15 | 15 | 0 | 14 | 100% | 67% |
+
+CodeQL is the most precise; Bandit finds a little more but about half of its reports are
+noise. HackScan sits in between: higher recall than CodeQL and Semgrep, higher
+precision than Bandit. Its misses are weak ciphers (DES, RC4, ECB, unauthenticated
+modes), which its rule does not cover yet, and flows it does not follow (`self`
+attributes, ambiguous module names). Its false positives are sinks it cannot prove safe (allow-list
+guards, functions only called with constants). The suite is small and written by
+HackScan's authors, so treat both tables as a sanity check rather than a ranking.
+
 ## Rules
 
 | Rule | Detects | Severity |
@@ -187,19 +226,87 @@ class PickleLoads(RulePlugin):
 
 ## GitHub Actions
 
+The repository is a composite action (available from v0.3.0). It installs HackScan, writes
+SARIF, uploads it to GitHub Code Scanning and then fails the job if the `fail-on` threshold
+was hit, so findings appear in the Security tab even on a failing run.
+
+```yaml
+permissions:
+  contents: read
+  security-events: write   # needed for the Code Scanning upload
+  actions: read            # needed for the upload in private repositories
+
+jobs:
+  hackscan:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: kahledawad2019-netizen/hackscan@v0.3.1
+        with:
+          path: src
+          fail-on: high
+```
+
+| Input | Default | Meaning |
+|---|---|---|
+| `path` | `.` | File or directory to scan |
+| `version` | *(empty)* | HackScan version from PyPI; empty installs the action's own checkout |
+| `fail-on` | `high` | Fail at this severity or above; empty passes `--fail-on never` (HackScan 0.3.0+), so findings never fail the job but crashes and errors still do |
+| `args` | *(empty)* | Extra CLI arguments, word-split without globbing; pass only trusted values |
+| `sarif-file` | `hackscan.sarif` | Report path |
+| `upload-sarif` | `true` | Upload to Code Scanning (skip it for fork pull requests, whose token is read-only) |
+
+Outputs: `sarif-file` and `exit-code` (0, 1 = threshold hit, 2 = error). The action sets up
+Python 3.12 on the runner. Without the action:
+
 ```yaml
 - run: pipx install hackscan
 - run: hackscan scan . --format sarif --sarif-omit-suppressed -o hackscan.sarif --fail-on high
-- uses: github/codeql-action/upload-sarif@v3
+- uses: github/codeql-action/upload-sarif@v4
   if: always()
   with:
     sarif_file: hackscan.sarif
 ```
 
+## pre-commit
+
+```yaml
+repos:
+  - repo: https://github.com/kahledawad2019-netizen/hackscan
+    rev: v0.3.1
+    hooks:
+      - id: hackscan
+```
+
+The hook runs `hackscan scan . --fail-on high` once per commit that touches Python files. It
+scans the whole repository (the CLI takes a single path), so `.hackscan.yml` `ignore` globs
+apply as usual.
+
 ## Limitations
 
-- Taint analysis is intra-procedural: values arriving through function parameters are
-  candidates, not confirmations (inter-procedural analysis is on the roadmap).
+- Taint analysis follows calls across functions and files only to *confirm* findings: a
+  sink fed by a parameter is confirmed when some caller in the scanned code passes
+  untrusted data, or when a called function returns it. It never suppresses a finding
+  (callers outside the scan are unknown). Only undecorated module-level functions bound
+  once are linked (same module, `import`/`from ... import`, relative imports and
+  re-exports); methods, nested functions, decorated functions, modules with a star
+  import, `*args`/`**kwargs` call sites and module names that match more than one file
+  are not followed. Sources are not tracked through object attributes or globals. The
+  result of a call is assumed to carry its arguments, so a helper that sanitizes them
+  (`def clean(x): return int(x)`) does not prevent a confirmation.
+- Findings in a function (or module) that itself uses `:=` are never suppressed, only
+  confirmed or left as candidates: an assignment expression can rebind a name in the
+  middle of an expression, which the statement-level analysis does not order. Sinks in
+  the body of a generator expression are never suppressed either (it runs later), nor
+  are values produced by one. Module-level constants are trusted only when bound exactly
+  once and the module never writes its namespace dynamically (`globals()`, `vars()`,
+  `exec`, `eval`, `__dict__`, star imports); `global`/`nonlocal` names are never trusted.
+- Mutable objects (lists, dicts, sets) are tracked by variable name. Mutation through
+  direct aliases, method calls (`d.get(k).append(x)`), bound methods
+  (`add = parts.append`), conditional receivers and comprehension items is modeled;
+  aliases created in less direct ways (stored in another object's attribute or in a
+  container and mutated through it later, returned from a helper, reached through
+  `getattr`) may be missed, so a constant suppression involving such code deserves review.
 - Python only. Imported tools may cover other languages; their findings are passed through.
 - Fix suggestions are deliberately conservative: many vulnerable lines get none. Command
   fixes (argument lists) are only offered for programs whose arguments are inert data

{hackscan-0.2.0 → hackscan-0.3.1}/PROJECT.md

@@ -128,9 +128,9 @@ mature scanners, (3) taint-based confirmation, (4) optional local-LLM triage and
 | 13 | Ollama triage pass with graceful offline fallback | M4 |
 | 14 | Template + LLM remediation as validated edits; diff view | M4 |
 | 15 | Rich "Matrix" TUI (banner, spinners, tables); auto-off in CI / `--quiet` | M4 |
-| 16 | Inter-procedural & cross-file taint (own call graph from `ast`) | M5 |
-| 17 | Benchmark harness vs Bandit / Semgrep CE / CodeQL; published precision/recall | M5 |
-| 18 | GitHub Action (uploads SARIF to Code Scanning), pre-commit hook | M5 |
+| 16 | Inter-procedural & cross-file taint (own call graph from `ast`) | M5 ✅ |
+| 17 | Benchmark harness vs Bandit / Semgrep CE / CodeQL; published precision/recall | M5 ✅ |
+| 18 | GitHub Action (uploads SARIF to Code Scanning), pre-commit hook | M5 ✅ |
 | 19 | Auth bypass / IDOR heuristics (only if benchmark shows acceptable precision) | Later |
 | 20 | Docker image | Later, on demand |
 
@@ -142,7 +142,7 @@ mature scanners, (3) taint-based confirmation, (4) optional local-LLM triage and
 | M2 ✅ | Taint | Framework fixtures (Flask/Django/FastAPI); constant/sanitized flows suppressed; tests for taint boundaries | M1 |
 | M3 ✅ | CLI, SARIF, importers → **v0.1.0 on PyPI** (released 2026-10-08) | `pip install hackscan` works on clean venv; SARIF validates against official schema; `--fail-on` exit codes tested | M2 |
 | M4 ✅ | LLM + remediation + TUI → **v0.2.0** (2026-10-09; 14 Codex gate rounds, double-OK) | All LLM tests run against a mocked Ollama; offline degradation tested; prompt-injection fixture | M3 |
-| M5 | Inter-procedural taint, benchmark, GitHub Action → v0.3.0 | Benchmark report in README; Action used on the repo itself | M4 |
+| M5 ✅ | Inter-procedural taint, benchmark, GitHub Action → **v0.3.0** (2026-10-09) | Benchmark report in README; Action used on the repo itself | M4 |
 
 ## Test Strategy
 - Corpus: `tests/corpus/<rule>/{vulnerable,safe}/*.py`, each file annotated with expected findings
@@ -358,3 +358,104 @@ hackscan/
   redacting finding function names before IDs, output and triage. Double-quoted SQL
   identifiers and SQL text containing `$` also receive no fix. Numeric and concatenated
   secret literals are masked and collected.
+- 2026-10-09 (M5 step 1): Roles back to Claude implements, Codex reviews (double OK before
+  commit). Repo root is a composite GitHub Action: inputs reach `run:` scripts only via env,
+  extra `args` are word-split with globbing off, the scan step records the exit code so SARIF
+  is uploaded (only if written) before the job fails; the dogfood workflow skips upload on
+  fork PRs. pre-commit hook scans the whole repo (`pass_filenames: false`; CLI takes one path).
+  Review round 1 (Codex Not OK, fixed): an empty `fail-on` input now passes the new
+  `--fail-on never` (overrides `.hackscan.yml`) instead of rewriting exit 1 to 0, which hid
+  crashes; a plugin constructor exception is now a PluginError (exit 2, not 1); README adds
+  `actions: read` for private repositories.
+- 2026-10-09 (M5 step 2): Inter-procedural, cross-file taint is confirmation-only
+  (`analyzers/interproc.py`). Pass 2 records symbolic facts per file: parameter and
+  call-result atoms carried in `Taint.symbols` with per-atom sanitized classes, return
+  summaries of linkable functions, argument values at statically named call sites. The
+  pipeline links all files (same module, imports, relative imports, re-exports, src
+  layouts by unambiguous module-name suffix) and solves a least fixpoint; candidates
+  reached by an untrusted source become confirmed with a file:line trace. Never
+  suppresses: unknown callers may exist. Only undecorated, non-generator module-level
+  functions bound once are linked; methods, `*args` positions and star-import modules are
+  not. Cost: taint now runs on every file (~45 ms/file serial on site-packages, +50%);
+  `_closure_names` made single-pass.
+  Review round 1 (Codex OK, 3 non-blocking precision issues): `match` captures
+  (`MatchAs`/`MatchStar`/`MatchMapping.rest`) are now scope bindings; the module binding
+  census also counts walrus targets in keyword-only defaults, lambda defaults,
+  annotations and class bases. Call results carrying their arguments through a
+  sanitizing helper stay a documented limitation (same approximation as within a function).
+  Review round 2 (Codex Not OK, pre-existing pass-2 soundness bug, fixed): a `match`
+  mapping rest capture (`case {**rest}`) now takes the subject's taint, and captures of
+  a failed pattern or a false guard stay visible to later cases and after the `match`
+  (the no-match path joins them), so a reachable sink is no longer suppressed as constant.
+  Review round 3 (Codex Not OK, pre-existing pass-2 soundness bugs, fixed): a `:=` in a
+  part that may not run (later `and`/`or` operands, `x if c else y` branches,
+  comprehension bodies, also inside `match` guards) is applied on a copy and joined
+  instead of overwriting the current value (comprehension bodies iterate to a fixpoint);
+  a `match` capture is aliased with the mutable names in the subject.
+  Review round 4 (Codex Not OK, 6 blockers: 5 pre-existing, 1 regression from round 3's
+  comprehension rework, which was reverted): instead of ordering every `:=` effect, a scope
+  whose own code contains `:=` never suppresses its sinks (confirmation still works); a
+  module with `:=` gets no trusted module constants; sinks in a generator expression's
+  body (deferred execution) are never suppressed. Also fixed (pre-existing): a nested
+  function or lambda reading an enclosing local that shadows a module constant was
+  treated as constant; module constants now apply only if the name resolves to the module.
+  Review round 5 (Codex Not OK, 6 pre-existing blockers without `:=`, fixed with
+  conservative rules): generator expressions yield UNKNOWN-joined values; a comprehension
+  target shadows outer bindings and class-body names are hidden inside class
+  comprehensions; module constants require exactly one module-level binding and no
+  dynamic namespace writes (`globals`/`vars`/`locals`/`exec`/`eval`/`__import__`,
+  `__dict__`/`__globals__`/`modules`, star imports), and module-level sinks in such a
+  module are never suppressed; names a function declares `global`/`nonlocal` are joined
+  with UNKNOWN on every read; LLM guard evidence must cover the function's own locals.
+  Review round 6 (Codex Not OK, 3 blockers, fixed): namespace writers are also detected
+  as attributes (`builtins.exec`, `import_module(...).exec`), imported aliases
+  (`from builtins import exec as run`), `import builtins` and `__builtins__`; a class body
+  that may write its own namespace (`locals()[...]` etc. in that body) never suppresses;
+  a comprehension target bound by more than one `for` resolves to UNKNOWN.
+  Review round 7 (Codex Not OK, 4 blockers, fixed): namespace-writer aliases bound by
+  assignment are followed file-wide (`execute = exec` then used in a class body), and
+  importing a namespace attribute under any name counts (`_Namespace`, computed once per
+  file); eager comprehensions scan filters before the element and repeat their body to a
+  fixpoint (mutations of one item reach the next; widened to UNKNOWN on budget
+  exhaustion); comprehension targets are aliased with the mutable names in their
+  iterables. Serial scan of ~770 site-packages files: 22.8 s (v0.2.0) -> ~41 s.
+  Review round 8 (Codex Not OK, 3 blockers, fixed): `:=` creates writer aliases too; a
+  comprehension that may mutate one of its items (non-read-only method on a target,
+  target passed to a call other than a sink/sanitizer/pure builtin/string method, store
+  into a target) invalidates every mutable name in its iterables and resolves its targets
+  to UNKNOWN; subscript/attribute comprehension targets store into their base.
+  Review round 9 (Codex Not OK, 2 blockers, fixed): a comprehension target anywhere inside
+  a call argument counts as escaping (`change([cmd])`); mutations, stores and comprehension
+  targets find their container through method calls too (`_root_name`: `d.get(k).append(x)`
+  and `d.setdefault(k, [])[0] = x` mutate `d`), in all code, not only comprehensions.
+  Review round 10 (Codex Not OK, 2 blockers, fixed): a bound method taken off a mutable
+  object (`append = parts.append`, method attribute loaded but not called right away)
+  invalidates every mutable name in its receiver; a mutating method call updates every
+  mutable name in a compound receiver (`(a if c else b).append(x)`, `[cmd][0].append(x)`).
+  User decision (2026-10-09, after 10 rounds): fix round 10, document remaining indirect
+  mutable aliasing (aliases stored in attributes/containers, returned from helpers,
+  `getattr`) as a README limitation, confirm with Codex once, commit with both OKs; any
+  further pass-2 aliasing hardening is a separate task.
+- 2026-10-09 (M5 step 3): Benchmark (`benchmarks/`). User choices: Bandit + Semgrep +
+  CodeQL; a new labeled suite plus known-vulnerable apps. Suite: 26 `# vuln:` sinks and 24
+  `# safe:` traps in Flask/Django/FastAPI/CLI code, one labeled sink per function, scored
+  per function (tools report flows at sink, construction or source lines); all tools are
+  normalized through HackScan's SARIF importer. Taxonomy fix found on the way: Semgrep's
+  `tainted-sql-string` rule carries CWE-704 metadata and is now mapped to sqli. Apps:
+  vulpy, PyGoat, DVPWA at pinned commits (MIT), every reported key hand-reviewed in
+  `apps_review.json`; relative recall over the union of true positives. Tests keep
+  RESULTS.json, APPS.json and the README tables in sync with the engine. Results:
+  suite F1 HackScan 82%, CodeQL 85%, Bandit 70%, Semgrep 67%; apps precision/relative
+  recall HackScan 83%/71%, CodeQL 100%/67%, Bandit 50%/76%, Semgrep 92%/52%.
+- 2026-10-09 (v0.3.0 release): version 0.3.0; the Action uploads with
+  github/codeql-action/upload-sarif@v4 (v3 is deprecated in December 2026); benchmark
+  re-run so the HackScan rows show 0.3.0 (numbers unchanged). Next ideas (not started):
+  weak-cipher detection (DES/RC4/ECB/unauthenticated modes; largest benchmark gap),
+  taint through `self` attributes, further pass-2 aliasing hardening.
+- 2026-10-12 (v0.3.1): template fixes only, found by the security agent's held-out evaluation
+  (8 of 13 true findings had no template). A call copied verbatim from the original call
+  (`params.get("id")`, `re.sub(...)`) no longer counts as introduced code (multiset count, so
+  no duplication); SQL fixes now decompose `"...'{}'...".format(args)` (plain auto-numbered
+  fields, positional args used exactly once; never for command argv, where a non-str value would
+  crash). Detection is unchanged, so the benchmark tables still show the 0.3.0 measurement (Bandit,
+  Semgrep and CodeQL are not installed on the release machine to re-run every row).

{hackscan-0.2.0 → hackscan-0.3.1}/README.md

@@ -20,6 +20,7 @@ HIGH     confirmed  views.py:10:9  HS-SQLI-001  [100%]
 |---|---|
 | **Verdicts, not just matches** | Every injection candidate is **confirmed** (untrusted source reaches it, with a trace), **suppressed** (provably constant or sanitized on every path), or left as a **candidate** for review. |
 | **Sound suppression** | A finding is only dismissed if *every* path is safe. Branches, loops (`break`/`continue`), `try`/`except`/`finally`, aliasing and mutation of lists/dicts, closures and `:=` are modeled. A differential fuzzer executes thousands of random programs to check that no dismissed sink ever receives attacker input. |
+| **Across functions and files** | A sink fed by a function parameter is confirmed when a caller anywhere in the scanned code passes untrusted data (directly or through other functions), or when a called function returns it; the trace names the source file and line. |
 | **Framework-aware** | Sources: Flask `request.*`, Django/DRF/Starlette/FastAPI request objects (incl. `self.request`), route-handler parameters, `input()`, `sys.argv`. |
 | **One report from many tools** | Imports Semgrep, Bandit, CodeQL (SARIF) and Gitleaks; deduplicates by vulnerability class and location, keeping every tool as provenance. |
 | **Fixes you can apply** | Mechanical, validated fix suggestions: parameterized queries (driver-aware placeholders), argument lists instead of shell strings, `ast.literal_eval`, SHA-256. Each is an exact edit that re-parses; ambiguous cases get no fix rather than a wrong one. |
@@ -51,7 +52,7 @@ hackscan rules                                    # list rules
 | Option | Meaning |
 |---|---|
 | `--severity`, `--min-confidence` | Report filters. |
-| `--fail-on SEV` | Exit 1 if an open (candidate/confirmed) finding is at least `SEV`. Suppressed findings never count. |
+| `--fail-on SEV` | Exit 1 if an open (candidate/confirmed) finding is at least `SEV`. Suppressed findings never count. `never` turns off a threshold set in `.hackscan.yml`. |
 | `--ignore GLOB` | Skip paths (`tests/*`, `**/migrations/**`, `legacy`). `.venv`, `node_modules`, `build`, ... are always skipped. |
 | `--with TOOLS` / `--import FMT=FILE` | Run external tools / import reports (`sarif`, `semgrep`, `bandit`, `codeql`, `gitleaks`). A missing or failing tool is a warning unless `--strict-tools`. |
 | `-o FILE` | Write the report to `FILE`. Never overwrites anything except a previous HackScan report. |
@@ -126,6 +127,44 @@ Secrets are never echoed: findings of class `secret` (from any tool) have generi
 messages and redacted snippets, and values reported by Gitleaks are scrubbed from all
 output, including warnings.
 
+## Benchmark
+
+**Labeled suite.** Flask, Django, FastAPI and CLI code with 26 vulnerable sinks and 24
+look-alike safe traps (SQL, command and code injection, weak crypto), scored per function;
+each tool runs the configuration in its Version column (Bandit: all checks; Semgrep CE:
+the `p/python` ruleset; CodeQL: `python-security-extended`; HackScan: no options)
+([methodology and caveats](benchmarks/README.md), [full results](benchmarks/RESULTS.md)):
+
+| Tool | Version | TP | FP | FN | Precision | Recall | F1 |
+|---|---|---:|---:|---:|---:|---:|---:|
+| HackScan | 0.3.0 | 23 | 7 | 3 | 77% | 88% | 82% |
+| HackScan (confirmed only) | 0.3.0 | 18 | 2 | 8 | 90% | 69% | 78% |
+| Bandit | bandit[sarif]==1.9.4 | 24 | 19 | 2 | 56% | 92% | 70% |
+| Semgrep CE | semgrep==1.180.0 (p/python sha256:31c1dfa46e8d) | 16 | 6 | 10 | 73% | 62% | 67% |
+| CodeQL | CodeQL 2.27.2 (codeql/python-queries@1.8.12, security-extended) | 20 | 1 | 6 | 95% | 77% | 85% |
+
+**Known-vulnerable apps.** [vulpy](https://github.com/fportantier/vulpy),
+[PyGoat](https://github.com/adeyosemanputra/pygoat) and
+[DVPWA](https://github.com/anxolerd/dvpwa) at pinned commits; every report reviewed by
+hand. There is no complete ground truth, so recall is relative to the 21 distinct
+vulnerabilities any tool found ([details](benchmarks/APPS.md)):
+
+| Tool | Reports | TP | FP | Vulns found | Precision | Relative recall |
+|---|---:|---:|---:|---:|---:|---:|
+| HackScan | 18 | 15 | 3 | 15 | 83% | 71% |
+| HackScan (confirmed only) | 6 | 6 | 0 | 6 | 100% | 29% |
+| Bandit | 32 | 16 | 16 | 16 | 50% | 76% |
+| Semgrep CE | 12 | 11 | 1 | 11 | 92% | 52% |
+| CodeQL | 15 | 15 | 0 | 14 | 100% | 67% |
+
+CodeQL is the most precise; Bandit finds a little more but about half of its reports are
+noise. HackScan sits in between: higher recall than CodeQL and Semgrep, higher
+precision than Bandit. Its misses are weak ciphers (DES, RC4, ECB, unauthenticated
+modes), which its rule does not cover yet, and flows it does not follow (`self`
+attributes, ambiguous module names). Its false positives are sinks it cannot prove safe (allow-list
+guards, functions only called with constants). The suite is small and written by
+HackScan's authors, so treat both tables as a sanity check rather than a ranking.
+
 ## Rules
 
 | Rule | Detects | Severity |
@@ -161,19 +200,87 @@ class PickleLoads(RulePlugin):
 
 ## GitHub Actions
 
+The repository is a composite action (available from v0.3.0). It installs HackScan, writes
+SARIF, uploads it to GitHub Code Scanning and then fails the job if the `fail-on` threshold
+was hit, so findings appear in the Security tab even on a failing run.
+
+```yaml
+permissions:
+  contents: read
+  security-events: write   # needed for the Code Scanning upload
+  actions: read            # needed for the upload in private repositories
+
+jobs:
+  hackscan:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: kahledawad2019-netizen/hackscan@v0.3.1
+        with:
+          path: src
+          fail-on: high
+```
+
+| Input | Default | Meaning |
+|---|---|---|
+| `path` | `.` | File or directory to scan |
+| `version` | *(empty)* | HackScan version from PyPI; empty installs the action's own checkout |
+| `fail-on` | `high` | Fail at this severity or above; empty passes `--fail-on never` (HackScan 0.3.0+), so findings never fail the job but crashes and errors still do |
+| `args` | *(empty)* | Extra CLI arguments, word-split without globbing; pass only trusted values |
+| `sarif-file` | `hackscan.sarif` | Report path |
+| `upload-sarif` | `true` | Upload to Code Scanning (skip it for fork pull requests, whose token is read-only) |
+
+Outputs: `sarif-file` and `exit-code` (0, 1 = threshold hit, 2 = error). The action sets up
+Python 3.12 on the runner. Without the action:
+
 ```yaml
 - run: pipx install hackscan
 - run: hackscan scan . --format sarif --sarif-omit-suppressed -o hackscan.sarif --fail-on high
-- uses: github/codeql-action/upload-sarif@v3
+- uses: github/codeql-action/upload-sarif@v4
   if: always()
   with:
     sarif_file: hackscan.sarif
 ```
 
+## pre-commit
+
+```yaml
+repos:
+  - repo: https://github.com/kahledawad2019-netizen/hackscan
+    rev: v0.3.1
+    hooks:
+      - id: hackscan
+```
+
+The hook runs `hackscan scan . --fail-on high` once per commit that touches Python files. It
+scans the whole repository (the CLI takes a single path), so `.hackscan.yml` `ignore` globs
+apply as usual.
+
 ## Limitations
 
-- Taint analysis is intra-procedural: values arriving through function parameters are
-  candidates, not confirmations (inter-procedural analysis is on the roadmap).
+- Taint analysis follows calls across functions and files only to *confirm* findings: a
+  sink fed by a parameter is confirmed when some caller in the scanned code passes
+  untrusted data, or when a called function returns it. It never suppresses a finding
+  (callers outside the scan are unknown). Only undecorated module-level functions bound
+  once are linked (same module, `import`/`from ... import`, relative imports and
+  re-exports); methods, nested functions, decorated functions, modules with a star
+  import, `*args`/`**kwargs` call sites and module names that match more than one file
+  are not followed. Sources are not tracked through object attributes or globals. The
+  result of a call is assumed to carry its arguments, so a helper that sanitizes them
+  (`def clean(x): return int(x)`) does not prevent a confirmation.
+- Findings in a function (or module) that itself uses `:=` are never suppressed, only
+  confirmed or left as candidates: an assignment expression can rebind a name in the
+  middle of an expression, which the statement-level analysis does not order. Sinks in
+  the body of a generator expression are never suppressed either (it runs later), nor
+  are values produced by one. Module-level constants are trusted only when bound exactly
+  once and the module never writes its namespace dynamically (`globals()`, `vars()`,
+  `exec`, `eval`, `__dict__`, star imports); `global`/`nonlocal` names are never trusted.
+- Mutable objects (lists, dicts, sets) are tracked by variable name. Mutation through
+  direct aliases, method calls (`d.get(k).append(x)`), bound methods
+  (`add = parts.append`), conditional receivers and comprehension items is modeled;
+  aliases created in less direct ways (stored in another object's attribute or in a
+  container and mutated through it later, returned from a helper, reached through
+  `getattr`) may be missed, so a constant suppression involving such code deserves review.
 - Python only. Imported tools may cover other languages; their findings are passed through.
 - Fix suggestions are deliberately conservative: many vulnerable lines get none. Command
   fixes (argument lists) are only offered for programs whose arguments are inert data

hackscan-0.3.1/action.yml

@@ -0,0 +1,94 @@
+name: HackScan
+description: Scan Python code and upload SARIF to GitHub Code Scanning.
+
+inputs:
+  path:
+    description: File or directory to scan.
+    required: false
+    default: "."
+  version:
+    description: HackScan version to install; empty installs this action's checkout.
+    required: false
+    default: ""
+  fail-on:
+    description: Fail at this severity or above; empty never fails on findings (HackScan >= 0.3.0).
+    required: false
+    default: "high"
+  args:
+    description: Extra CLI arguments, split by the shell; must be trusted.
+    required: false
+    default: ""
+  sarif-file:
+    description: SARIF output path.
+    required: false
+    default: "hackscan.sarif"
+  upload-sarif:
+    description: Upload SARIF to GitHub Code Scanning when true.
+    required: false
+    default: "true"
+
+outputs:
+  sarif-file:
+    description: SARIF output path.
+    value: ${{ steps.scan.outputs.sarif-file }}
+  exit-code:
+    description: Scan exit code (0 OK, 1 threshold hit, 2 scan error).
+    value: ${{ steps.scan.outputs.exit-code }}
+
+runs:
+  using: composite
+  steps:
+    - uses: actions/setup-python@v5
+      with:
+        python-version: "3.12"
+    - name: Install HackScan
+      shell: bash
+      env:
+        HACKSCAN_VERSION: ${{ inputs.version }}
+        HACKSCAN_ACTION_PATH: ${{ github.action_path }}
+      run: |
+        if [[ -n "$HACKSCAN_VERSION" ]]; then
+          python -m pip install "hackscan==$HACKSCAN_VERSION"
+        else
+          python -m pip install "$HACKSCAN_ACTION_PATH"
+        fi
+    - name: Scan
+      id: scan
+      shell: bash
+      env:
+        HACKSCAN_PATH: ${{ inputs.path }}
+        HACKSCAN_FAIL_ON: ${{ inputs.fail-on }}
+        HACKSCAN_ARGS: ${{ inputs.args }}
+        HACKSCAN_SARIF_FILE: ${{ inputs.sarif-file }}
+      run: |
+        # An empty input disables even a threshold inherited from .hackscan.yml; the exit code
+        # is never rewritten, so a crash (Python exits 1) still fails the job.
+        threshold=(--fail-on never)
+        if [[ -n "$HACKSCAN_FAIL_ON" ]]; then
+          threshold=(--fail-on "$HACKSCAN_FAIL_ON")
+        fi
+        # Extra args intentionally undergo word splitting (globbing is off); supply only trusted values.
+        set -f
+        rm -f -- "$HACKSCAN_SARIF_FILE"
+        code=0
+        hackscan scan "$HACKSCAN_PATH" --format sarif --sarif-omit-suppressed \
+          -o "$HACKSCAN_SARIF_FILE" $HACKSCAN_ARGS "${threshold[@]}" || code=$?
+        written=false
+        if [[ -s "$HACKSCAN_SARIF_FILE" ]]; then
+          written=true
+        fi
+        printf 'sarif-file=%s\n' "$HACKSCAN_SARIF_FILE" >> "$GITHUB_OUTPUT"
+        printf 'sarif-written=%s\n' "$written" >> "$GITHUB_OUTPUT"
+        printf 'exit-code=%s\n' "$code" >> "$GITHUB_OUTPUT"
+    - name: Upload SARIF
+      if: ${{ always() && inputs.upload-sarif == 'true' && steps.scan.outputs.sarif-written == 'true' }}
+      uses: github/codeql-action/upload-sarif@v4
+      with:
+        sarif_file: ${{ inputs.sarif-file }}
+        category: hackscan
+    - name: Apply scan result
+      if: ${{ always() && steps.scan.outputs.exit-code != '' }}
+      shell: bash
+      env:
+        HACKSCAN_EXIT_CODE: ${{ steps.scan.outputs.exit-code }}
+      run: exit "$HACKSCAN_EXIT_CODE"