coretrace-python-analyzer 0.22.0__tar.gz → 0.24.0__tar.gz

{coretrace_python_analyzer-0.22.0 → coretrace_python_analyzer-0.24.0}/CHANGELOG.md

@@ -1,5 +1,145 @@
 # Changelog
 
+## 0.24.0 (2026-10-23)
+
+### Compatibility and migration
+
+- The cache format changes (19): caches built by an earlier version miss once.
+- A module in a directory without `__init__.py` inside a package is named after that
+  package, as Python names it (#264): `pkg/sub/app.py` is `pkg.sub.app`. Its relative
+  imports now resolve, and a file that was a `syntax-error` (`relative import beyond
+  top-level package`) is analysed, flask's `sansio` for one. The project symbols of its
+  functions change with its name (`python.pkg.sub.app.view`), which a model declared on
+  the former symbol must follow; baseline keys do not change, since they hold no module
+  name. A script beside it still imports it by its short name, which answers no import
+  from another directory: at the project root, `import os` is the library even beside a
+  `pkg/sub/os.py`, which a top-level module of a directory took before. Files that share
+  only that short name are no longer noted as `ambiguous-module`, nor is an import that
+  only such names from other directories match.
+- SARIF: each rule carries a title (`shortDescription`) and a description (`fullDescription`)
+  instead of repeating its id (#270), and each result carries its confidence as the
+  `confidence` property (#271). A flow reaching its sink inside a project function points at
+  that sink as a related location, which the result's message links to (#271).
+- Django templates are read as a browser reads them (#263, #272, #278): a template that
+  leaves a tag, a `<script>` or `<style>` block or an end tag open, or includes one that
+  ends inside such a construct, is no longer established as escaping, so `xss` may be
+  reported where it was not.
+- Django templates are also read as each page is put together (#273): a variable a
+  child's `{% block %}` prints inside its parent's `<script>`, an `{% include %}` that ends
+  the attribute value it stands in, or one of a name several files have outside HTML data,
+  is reported as `xss` where it was not, as is the variable `block` named anywhere but in
+  a bare `{{ block.super }}`. So is a template that includes Django's
+  `django/forms/widgets/attrs.html` inside a tag, which prints attribute names, and a
+  template including itself inside a tag or a script. A page beyond 8 MiB once put
+  together, or a template with more than 64 pages, or 16 MiB in all, through names
+  several files have, is not established and is listed in `Coverage.unread_templates`.
+- Plugin API, additive: `Plugin.rules` maps the rules a plugin reports to their
+  `RuleDescription`, optional, so that third-party plugins describe their own rules;
+  `Finding.related` lists `RelatedLocation`s (a span and a message). `Report.rules` carries the
+  descriptions to the reporters.
+- Plugin API: `coretrace_python.interprocedural.starting_summaries` gives the summaries an
+  import cycle's members start from, and `coretrace_python.engine.MAX_PROJECT_ITERATIONS`
+  is removed, since the project's summaries are iterated until they settle (#261).
+- A summary lists its external calls in source order (#261): a finding naming the first
+  call of a callee's summary may name another of that callee, at the same line, as 4 of
+  saleor's 228 `orm-n-plus-one` findings do.
+
+### Analysis
+
+- `--check` ends on projects whose summaries never settled (#261). Around a call cycle of
+  three functions or more, a caller copied its callee's external calls in an order that
+  rotated forever, so ansible, fastapi, sentry and home-assistant never ended: those calls
+  are now kept in source order. Around an import cycle of three modules or more, the
+  project's iteration shifted by one module each time and stopped at its bound, which
+  could miss a sink reached through the cycle: its members now start from the summaries a
+  module starts from, and the iteration goes on until the project's summaries settle.
+- A lambda or comprehension in the annotation of a parameter or a variable no longer
+  stops the analysis of a directory with a `KeyError` (#262). An assignment expression in
+  an annotation binds its name as Python 3.11 to 3.13 do: in a function, the name is a
+  local whether the annotation is evaluated or not; at module level, it exists only once
+  the annotation is evaluated.
+
+### Templates
+
+- Start tags and their attributes are read as the HTML tokenizer reads them, in time
+  linear in the size of the template (#263): a few dozen characters under a `templates`
+  directory, a PNG or `<a ` followed by 40 letters, stopped the analysis. A tag a template
+  leaves open is read to its end and its attributes are checked; a quoted value left open
+  stays quoted to the end of the template.
+- A `<script>` or `<style>` block or an end tag a template leaves open runs to the end of
+  the template, and a variable inside it is refused, even where the page ends there
+  (#272). A template that ends inside such a construct or a tag is not established where
+  it is included, since the markup after the include continues it, and a template
+  extending another ends where that one ends.
+- A comment ends where a browser ends it (`-->`, `--!>`, at once for `<!-->` and
+  `<!--->`), and one left open hides the rest of the template (#272). A comment holding
+  template tags may end at any of them, since an include there may close it and a
+  condition around its `<!--` may leave it out, or run to its close when they render
+  nothing: it is read on from its first tag, and that reading must be in HTML data right
+  after each later tag and right after the close (#278). Such a comment left open does
+  not end in HTML data.
+
+### Performance
+
+- Collecting the filters a template's render reaches walks each template once for each set
+  of blocks the templates extending it override (#279). It followed every path through
+  nested includes: two includes of one template at each of 20 levels took 31 s.
+  A template in a cycle of includes is still walked again wherever the cycle cuts its
+  walk short.
+
+### Documentation
+
+- `docs/usage.md` documents how templates are read and each page is put together, the
+  modules of a directory without `__init__.py`, assignment expressions in annotations,
+  and the rule descriptions and related locations of the SARIF report; `docs/plugins.md`
+  documents `Plugin.rules` and `Finding.related`.
+
+## 0.23.0 (2026-10-16)
+
+### Compatibility and migration
+
+- The cache format changes (16) and the PyIR schema version is 2: caches built by an earlier version miss once. Every cache key also holds the engine version, so this release reanalyses every module of a cached project, which delivers the fixes below to it.
+- `code-injection` reports new sinks (dynamic imports, `runpy`, the import path, `getattr` calls), so findings may appear once; each keeps the rule id `code-injection` and is titled by what the input chooses (#240). Refutation proves more allowlists and fewer mutated collections (#239), `unsafe-xml` messages and confidences change (#242), and a sink in code that cannot run, or a call to a returned callable, is no longer reported (#258, #240). Baselines recorded before should be recorded again after review.
+- Findings carry new metadata, exported in JSON and as SARIF result properties: `context` (`test`, besides a rule's own `example`) for test code (#244), and `origin: vendored` for code under `third_party`, `third-party`, `vendor` or `vendored` (#243). Vendored code is still analysed and reported; the metadata changes no verdict.
+- A project located under a directory named `test`, `tests`, `testing`, `fixtures` or `__tests__` no longer has every credential at low confidence: `hardcoded-credential` and `config-secrets` judge a file's path in the project (#253).
+- Plugin API, all additive:
+  - `SymbolCallDetector.describe` returns a `CallReport` (message and confidence) per call (#242);
+  - `PluginContext.path(source_id)` gives a file's path from the root of the analysis, and `run_plugins` and `analyze_file` take that root (#253, #244);
+  - `Sink` gains `selects` and `requires_called_result`, and `Sink.applying` (#240);
+  - summaries gain `ExternalCall.invoked_at` and `NonlocalWrite.kept`, flows `TaintFlow.invoked_at`, and `NonlocalResult` its `previous` value (#240, #258);
+  - `coretrace_python.findings.context` classifies paths (`is_test_code`, `code_context`, `code_origin`, `path_in_project`).
+
+### Code injection
+
+- The module `importlib.import_module`, `__import__` or `runpy.run_module` imports and the file `runpy.run_path` runs are sinks, by position or keyword; the package of a relative name, the globals and the run name are data. Titled `Untrusted module import` and `Untrusted script execution`; a constant package prefix gives a hotspot (#240).
+- The directory `sys.path.insert`, `append` or `extend` adds is a hotspot, `Untrusted import path`, as code runs only when a later import resolves there; `site.addsitedir` is a vulnerability, `Untrusted site directory`, its evidence stating that it runs the `.pth` files of a directory the attacker must control. Writes to `sys.path` that are not calls are not covered (#240).
+- `getattr(obj, name)(...)` with a name the input chooses is a hotspot, `Untrusted attribute call`, reported at each call of the result, with `selected_line` and `invoked_line`, in the function and through project functions. A result that is only read, returned or stored before it is called is not reported (#240).
+- These sinks are proven only by a check that limits the value to constants (membership in a constant allowlist, equality with a constant); a check of its shape (`isidentifier()`, `isalnum()`, `re.fullmatch`) leaves a hotspot. For `getattr`, a check where it selects counts past a join, a check before the call counts in the same loop iteration only, and nothing is reported when the selection or the call cannot run (#240).
+- Calling the result of a call, or an item of it, calls what the call returned: its arguments are no longer read as the call's, so `eval('lambda x: x')(data)` or `importlib.import_module('plugins')(data)` reports nothing (#240).
+
+### Refutation
+
+- A membership test proves the value when every member is a constant that cannot change before the test: a tuple, list or set display, or a dict display by its keys, directly or through `.keys()`; `frozenset`, `set`, `tuple` or `list` of one of these; a module-level tuple or `frozenset` of constants bound once and never rebound. A list, set or dict of the function counts only while the function only reads it; a module-level one never does (#239).
+- A lookup in a dict display gives one of its values, of what was stored in it since, or the default, never a key: input that only selects the entry is no longer passed on (#239).
+
+### XML
+
+- `unsafe-xml` states what each parser family risks: entity amplification with Expat older than 2.7.2 for the standard library, which reads no local file; external entities in lxml before 5.0, and in `iterparse` and `ETCompatXMLParser` before 6.1 (CVE-2026-41066); `resolve_entities=True` whatever the version, `'internal'` before 5.0. A parser given `resolve_entities=False` is not reported; XML content the project wrote itself, passed to a function that parses its argument as the document, is reported at low confidence (#242).
+
+### Analysis
+
+- A project function's summary keeps nothing of the code constant propagation proves cannot run: no sink, returned, merged or stored value reaches its callers (#258).
+- A `nonlocal` a nested function may leave unwritten (no write on some path, a write that cannot run, an exception before it) keeps the value the caller's variable held before the call, joined with what may be written (#258).
+- Warnings Python raises while parsing an analysed source (an invalid escape sequence) no longer reach the analyzer's standard error (#241).
+
+### Performance
+
+- `-j N` (`--jobs N`) analyses independent modules in `N` processes, `0` for one per core the process may run on; the report does not depend on it (#205). Each worker prepares the project once and reads the summaries a component imports from the cache or a directory of the run's own, instead of receiving them with every task: on healthchecks, `-j 8` takes 7.9–9.2 s against 15.3–15.5 s with `-j 1` (#245).
+
+### Documentation
+
+- `docs/usage.md` documents the new sinks and their limits, constant allowlists, the `context` and `origin` metadata, unreachable code and `nonlocal` writes, and re-measured performance; `docs/plugins.md` documents `describe`, `PluginContext.path` and the new `Sink` fields.
+
 ## 0.22.0 (2026-10-08)
 
 ### Compatibility and migration

{coretrace_python_analyzer-0.22.0 → coretrace_python_analyzer-0.24.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: coretrace-python-analyzer
-Version: 0.22.0
+Version: 0.24.0
 Summary: Static security analysis infrastructure for Python code
 Project-URL: Homepage, https://github.com/CoreTrace/coretrace-python-analyzer
 Project-URL: Repository, https://github.com/CoreTrace/coretrace-python-analyzer

{coretrace_python_analyzer-0.22.0 → coretrace_python_analyzer-0.24.0}/docs/plugins.md

@@ -85,7 +85,10 @@ class SensitiveLoggingPlugin(SymbolCallDetector):
     message_template: ClassVar[str] = "call to {symbol} may log sensitive data"
 ```
 
-The shipped `dangerous-eval` and `weak-crypto` plugins are of this kind.
+The shipped `dangerous-eval` and `weak-crypto` plugins are of this kind. `accepts(call, symbol, defs)`
+keeps only some calls; `describe(ctx, call, symbol, defs)` returns the `CallReport` (message and
+confidence) of one call, the rule's `message_template` and `confidence` by default, for a rule that
+tells its calls apart, as `unsafe-xml` does.
 
 ### A detector of taint flows: `TaintDetector`
 
@@ -166,7 +169,7 @@ class BottleModels(ModelPlugin):
 | Model | Declares |
 |---|---|
 | `Source(symbol, label, kinds=ALL)` | A call or attribute whose value is attacker-controlled; `label` names it in messages (`http`, `stdin`, `argv`, `http-response`). `kinds` restricts what it can inject. Text input, such as a form field, a header or a URL parameter, carries `TEXT_KINDS`, every kind but `NOSQL`: a string cannot hold a query operator. |
-| `Sink(symbol, kinds, positions=(), keywords=())` | A call dangerous for `kinds`. `positions` limits a kind to the given argument indexes, and `keywords` to the given keyword names; a restricted kind reaches only the arguments listed there. SQL sinks read the statement only, and the SSRF sinks of HTTP clients read the URL, by position or as `url=`. A starred argument or `**kwargs` reaches no restricted kind. |
+| `Sink(symbol, kinds, positions=(), keywords=(), selects=False, requires_called_result=NONE)` | A call dangerous for `kinds`. `positions` limits a kind to the given argument indexes, and `keywords` to the given keyword names; a restricted kind reaches only the arguments listed there. SQL sinks read the statement only, and the SSRF sinks of HTTP clients read the URL, by position or as `url=`. A starred argument or `**kwargs` reaches no restricted kind. `selects` marks an argument that chooses by name the code that runs (a module, a file, a directory, an attribute): only a check limiting it to constants proves it. `requires_called_result` lists the kinds that apply only when the call's result is itself called, as for `getattr`'s name: the engine reports them at each invocation of the result, the other kinds at the call. |
 | `Sanitizer(symbol, kinds)` | A call whose result is safe for `kinds`, including when the call is inside a project function the data goes through. |
 | `EntryPoint(symbol, label, kinds=ALL, inputs=None)` | A decorator or a class base whose functions receive their parameters as `label` input, such as `flask.Flask.route`; or a project function or class named by its own symbol (`python.app.handlers.main`), as an integration reading a deployment declaration names a Lambda handler. `inputs` limits the parameters that are input to those positions, after `self` for a method: a handler's event, not its context. |
 | `TypedParameter(symbol, label, kinds=ALL)` | A class whose annotated parameters carry `label` input, such as Django's `HttpRequest`. |
@@ -184,7 +187,9 @@ class BottleModels(ModelPlugin):
 Two plugins may describe the same symbol. An identical model is registered once; two
 `Sink` models merge their kinds, positions and keywords, so a plugin can add `SQL` to a sink
 another plugin declared for `COMMAND`; any other difference is a conflict that stops
-the analysis with a `ModelError` naming both plugins.
+the analysis with a `ModelError` naming both plugins. A merged sink `selects` when either
+does, and a kind requires the called result only when every declaration carrying it
+does: an unconditional declaration is never weakened, whatever the order.
 
 A plugin may also read models from the project it analyses: `project_models(self, root)`
 returns them, from a file at `root` for instance. It is called once per directory check,
@@ -321,6 +326,10 @@ any analysis result: `SSAAnalysis` (`coretrace_python.ir.ssa`) for the code,
 `TaintAnalysis` (`coretrace_python.taint`) for the flows, `RefutationAnalysis`
 (`coretrace_python.findings.refutation`) for their verdicts, `DependencyAnalysis`
 (`coretrace_python.dependency`) for the requirements. List what you read in `requires`.
+`ctx.path(source_id)` is the path of a file from the root of the analysis, the one
+`coretrace_python.findings.context` classifies (`is_test_code`, `code_context`,
+`code_origin`): judge test code on it, never on the absolute path, which depends on where
+the project sits. It is `None` for a file outside the root.
 
 ## Findings
 
@@ -332,6 +341,32 @@ JSON pointer of a value) for a value of a structured file whose line cannot be:
 that reads a finding's line checks which it has, and never invents a line for a
 `FileLocation` (`coretrace_python.source.positions` places JSON and TOML values).
 
+A plugin describes the rules it reports in its optional `rules` attribute: each rule id to a
+`RuleDescription`, a title and what the rule detects. The SARIF report gives them to its
+consumers (`shortDescription` and `fullDescription`), so that GitHub code scanning names and
+explains the alert; a rule the plugin does not describe is listed by its id alone.
+
+```python
+from collections.abc import Mapping
+
+from coretrace_python.findings import RuleDescription
+
+
+class SensitiveLoggingPlugin(SymbolCallDetector):
+    ...
+    rules: ClassVar[Mapping[str, RuleDescription]] = {
+        "sensitive-logging": RuleDescription(
+            "Sensitive data logged",
+            "A logging call at debug level that may write sensitive data to the logs.",
+        ),
+    }
+```
+
+A finding may also point at other places it involves, each a `RelatedLocation` (a span and a
+short message) in its `related` tuple, as a taint flow reaching its sink inside a project
+function points at that sink. The SARIF report lists them as `relatedLocations` and links to
+them from the message.
+
 ## Testing a plugin
 
 Load the plugin directory with the engine and check one source or one project:

{coretrace_python_analyzer-0.22.0 → coretrace_python_analyzer-0.24.0}/docs/usage.md

@@ -48,7 +48,7 @@ coretrace-python-analyzer [--check | --emit-ir [--ssa]] [options] [path]
 | `--plugins DIR` | Load the plugins found under `DIR` on top of the bundled ones. Repeatable. |
 | `--no-bundled-plugins` | Do not load the plugins shipped with the package. |
 | `--cache DIR` | Keep per-module results under `DIR` and reuse them for unchanged modules. |
-| `--jobs N` | Analyse independent modules in `N` processes. |
+| `-j N`, `--jobs N` | Analyse independent modules in `N` processes, `0` for one per available core. Default: `1`. |
 | `--sbom PATH` | Write a CycloneDX bill of materials of the dependencies to `PATH`. |
 | `--vex PATH` | Write an OpenVEX document to `PATH`: whether each advisory affecting a requirement affects the project. |
 | `--advisories FILE` | Read a local advisory file in addition to `advisories.json` at the root. Repeatable. |
@@ -67,9 +67,16 @@ coretrace-python-analyzer [--check | --emit-ir [--ssa]] [options] [path]
 ## What is analysed
 
 Given a directory, every `.py` file below it is a module of one project, named after
-its package (`app/views.py` is `app.views`). Two files that would share a name, `a/app.py`
-and `z/app.py` outside any package, are told apart by their path from the root, `a.app`
-and `z.app`, and reported: each carries an `ambiguous-module` note, since a symbol
+its package (`app/views.py` is `app.views`). A directory without `__init__.py` inside a
+package is a portion of it, as Python reads one: `pkg/sub/app.py` is `pkg.sub.app`, and
+its relative imports resolve against `pkg`. A module there can also be imported by the
+name a script run from its directory gives it, `app`: an absolute import from that
+directory reaches what lies beside it first, then what the package's root holds, and
+files sharing only such a name are told apart by the importer's directory. From another
+directory that name answers nothing, as Python finds no such module there: at the
+project root, `import os` is the library even beside a `pkg/sub/os.py`. Two files
+that would share a module name, `a/app.py` and `z/app.py` outside any package, are told
+apart by their path from the root, `a.app` and `z.app`, and reported: each carries an `ambiguous-module` note, since a symbol
 `python.app.main` may name a function of either and an import of `app` resolves only
 from the importer's own directory, and each is covered as `ambiguous` rather than
 analysed. An import of such a name that no file of the importer's own directory answers
@@ -120,8 +127,8 @@ variable is not treated that way yet.
 
 | Rule | Reached sink |
 |---|---|
-| `command-injection` | Shell or process execution (`os.system`, `subprocess` with a string, …), judged on what the input controls in the command. A string a shell runs, a string naming the program, the program itself, or an operand the program runs as a command (after the host for `ssh`, the program of `sudo`, `env`, `xargs`, `timeout`…, any argument of a batch file) is `high`. The script an interpreter runs (`bash run.sh`, `python tool.py`, its first operand when no option such as `-c` gives the code), or the module of `python -m`, is `high` and kept for review: execution is possible, and its exploitability depends on the file the input chooses. The arguments after a script or code are the script's own: options of unknown meaning, reported `medium` (a hotspot after an option of the script), refuted behind a constant prefix. Otherwise an element of an argument list run without a shell can only be read as an option: `medium`, or `high` when a documented option of the program runs a command it chooses (`git push --receive-pack=…`, `ssh -o ProxyCommand=…`). A value a documented option takes is judged by what the option does with it, whatever its prefix: run it (`sh -c`, `cmd /c`, `python -c`: `high`), keep it as data (`git commit -m`: refuted). After an option the rule does not know, the finding stays a hotspot to review. A constant prefix that keeps a standalone input from being an option (`f"v{version}"`), or an operand after `--` for a program that honours it, is refuted. Every case keeps the rule id `command-injection`, so suppressions and baselines match whatever the case. When the rule reads the command element by element and establishes what the input is there, the finding's `injection` metadata (in the JSON report, and as the only property of its SARIF result) says so: `command` when the input runs or chooses what runs (a shell string, the program, an operand run as a command, the value of an option that runs it); `option` when it can only reach the program as an option or as the value of an option not established to run it, and the message is then titled `Option injection`. When that cannot be established (an option whose effect is to review, an unknown program or element before the input, a command that is not a list or tuple display, a sink such as `os.system` the rule does not read element by element), the metadata is absent and the title stays `Command injection`. |
-| `code-injection` | The code `eval` or `exec` runs; the globals and locals passed with it are data. `dangerous-eval` reports every such call, this rule the ones attacker input reaches. |
+| `command-injection` | Shell or process execution (`os.system`, `subprocess` with a string, …), judged on what the input controls in the command. A string a shell runs, a string naming the program, the program itself, or an operand the program runs as a command (after the host for `ssh`, the program of `sudo`, `env`, `xargs`, `timeout`…, any argument of a batch file) is `high`. The script an interpreter runs (`bash run.sh`, `python tool.py`, its first operand when no option such as `-c` gives the code), or the module of `python -m`, is `high` and kept for review: execution is possible, and its exploitability depends on the file the input chooses. The arguments after a script or code are the script's own: options of unknown meaning, reported `medium` (a hotspot after an option of the script), refuted behind a constant prefix. Otherwise an element of an argument list run without a shell can only be read as an option: `medium`, or `high` when a documented option of the program runs a command it chooses (`git push --receive-pack=…`, `ssh -o ProxyCommand=…`). A value a documented option takes is judged by what the option does with it, whatever its prefix: run it (`sh -c`, `cmd /c`, `python -c`: `high`), keep it as data (`git commit -m`: refuted). After an option the rule does not know, the finding stays a hotspot to review. A constant prefix that keeps a standalone input from being an option (`f"v{version}"`), or an operand after `--` for a program that honours it, is refuted. Every case keeps the rule id `command-injection`, so suppressions and baselines match whatever the case. When the rule reads the command element by element and establishes what the input is there, the finding's `injection` metadata (in the JSON report, and as a property of its SARIF result) says so: `command` when the input runs or chooses what runs (a shell string, the program, an operand run as a command, the value of an option that runs it); `option` when it can only reach the program as an option or as the value of an option not established to run it, and the message is then titled `Option injection`. When that cannot be established (an option whose effect is to review, an unknown program or element before the input, a command that is not a list or tuple display, a sink such as `os.system` the rule does not read element by element), the metadata is absent and the title stays `Command injection`. |
+| `code-injection` | The code `eval` or `exec` runs; the globals and locals passed with it are data. `dangerous-eval` reports every such call, this rule the ones attacker input reaches. Also the module `importlib.import_module`, `__import__` or `runpy.run_module` imports, and the file `runpy.run_path` runs, by position or by keyword (`name`, `mod_name`, `path_name`); the package of a relative name, the globals and the run name are data. An import runs the top-level code of a module the input chooses among those that can be found, a script path the file it chooses: execution is possible, and how far it goes depends on what the input can reach, so these findings are titled `Untrusted module import` and `Untrusted script execution`. A constant package prefix (`"app.plugins." + name`) confines an import to that package: a hotspot. The environment and the command line reach these sinks as they reach `eval`. Also the directory `sys.path.insert` (its second argument), `sys.path.append` and `sys.path.extend` add to the import path, and the directory `site.addsitedir` adds (`sitedir`), judged the same way. A directory on the import path runs code only when a later import resolves to a module the attacker placed there: a hotspot, titled `Untrusted import path`. `site.addsitedir` runs the import lines of the directory's `.pth` files during the call: a vulnerability, titled `Untrusted site directory`, exploitable when the directory holds files the attacker controls, as the finding's evidence states. Coverage of the import path is partial: writes to `sys.path` that are not calls, a slice assignment (`sys.path[:0] = [d]`), `sys.path += [d]` or rebinding `sys.path`, are not reported. Also the name `getattr(obj, name)` receives, when the attribute it selects is called: the finding, titled `Untrusted attribute call`, is placed at each call of the result, one per call, with the lines of the selection and of the call in its metadata (`selected_line`, `invoked_line`), and through project functions at the call that reaches them. What runs depends on what the object exposes, so it is a hotspot, on a module too and behind a constant prefix; a check proves the name only when it limits the value `getattr` received to constants, where `getattr` selects (even past a join) or before the call, and a check of a name rebound since proves nothing; a check before the call does not prove a callable selected in an earlier iteration of a loop, which reaches the call across the loop's back edge: only a check where it was selected does (even when the name never changes, which this version does not establish); nothing is reported when the selection or the call cannot run, in the function and inside a project function. Calling the result of a call, or an item of it, calls what the call returned: its arguments are not read as the call's, so `getattr(self, 'factory')(None, data)` or `eval('lambda x: x')(data)` reports nothing. A `getattr` whose result is only read is not reported, nor, in this version, one whose result is returned to the caller, stored in an attribute or a container, or passed on before it is called, nor `operator.attrgetter`, `operator.methodcaller`, `obj.__getattribute__(name)` or `vars(obj)[name]`; checks made inside a project function are not seen from its callers, as for every sink. |
 | `sql-injection` | A database statement (`cursor.execute`, SQLAlchemy `text`, Django `raw`, …). Parameters of a parameterised query are not statements. |
 | `nosql-injection` | A NoSQL query, such as a MongoDB filter, given attacker input that may hold a structure: a JSON body giving `{"$ne": null}` matches any value. The input that may hold a structure is JSON, the raw bodies a `json.loads` decodes, files, WebSocket messages and standard input. Text cannot hold an operator: form fields, query parameters, headers, cookies, URL parameters, the command line, the environment, and an entry-point parameter annotated with scalars (`str`, `int`, `Optional[str]`, `Annotated[str, Query()]`, `list[str]`, …), which FastAPI validates. A Django, REST framework, aiohttp or tornado view reads a request object: its text attributes (`GET`, `POST`, `headers`, `query_params`, `query`, `match_info`, `arguments`, …) and the URL parameters after it are text, its body, files and `data` are not. Limits: a helper function the request is passed to reads it as a whole; the keys of a dict are not followed, so a string concatenated into `$where` is not reported. The analyzer ships no NoSQL sink: model plugins declare them. |
 | `path-traversal` | A file system path (`open`, `send_file`, `os.remove`, …). |
@@ -132,10 +139,19 @@ variable is not treated that way yet.
 | `plaintext-credential-storage` | A parameter named like a password stored in a database without hashing. Medium confidence, since a name is a hint. |
 | `exploitable-vulnerability` | An API affected by an advisory of a vulnerable requirement, see Dependencies. |
 
-Each flow is judged before it is reported. A dominating guard that proves the value
-safe (`isdigit()`, membership in a constant collection, equality with a constant, a
+Each flow is judged before it is reported. Code constant propagation proves can never
+run (under `if False:`, `while False:`) reports nothing, in the function and through the
+project functions that contain it: their summaries keep nothing of such code, neither a
+sink nor a returned, merged or stored value. Code that may run is kept, whatever decides
+it at run time, a parameter or a module-level name. A dominating guard that proves the value
+safe (`isdigit()`, membership in a constant allowlist, equality with a constant, a
 numeric value proven by `int()`, `len()` or bounded arithmetic, a validator such as
-`re.fullmatch`) refutes it and nothing is reported. A hotspot is reported at medium
+`re.fullmatch`) refutes it and nothing is reported. A sink that selects code by name,
+the module of `importlib.import_module`, `__import__` or `runpy.run_module` and the file
+of `runpy.run_path`, is proven only by a check that limits the value to constants:
+membership in a constant allowlist or equality with a constant. A check of its shape
+(`isidentifier()`, `isalnum()`, `re.fullmatch`) limits its characters, not the modules
+or files it can name, and leaves a hotspot. A hotspot is reported at medium
 confidence. Two things make a flow a hotspot:
 
 - A guard that examines the data without proving it safe. That covers a test of the value
@@ -149,6 +165,28 @@ a vulnerability. When the flow passes the object whole, to the sink or to a func
 a guard on any of its attributes still counts. The verdict and its evidence are in the
 finding's metadata.
 
+A membership test (`in`, `not in`) is a constant allowlist when every member is a constant
+that cannot change before the test:
+
+- a tuple, list or set display written in the function, or a dict display by its keys,
+  tested directly or through `.keys()`;
+- `frozenset`, `set`, `tuple` or `list` of one of these;
+- a module-level tuple, or `frozenset` of a display, of constants, bound once in its
+  module and never rebound there, by another assignment, an augmented assignment or a
+  `global` statement.
+
+A list, set or dict of the function, a display or the `set` or `list` of one, counts only
+while the function only reads it: changed by a method, stored, or passed to a call before
+the test, it may hold anything. A
+module-level list, set or dict can be changed from any module, so it never counts, and
+the guard leaves a hotspot. Rebinding a module-level name from another module
+(`settings.ALLOWED = ...`) is not seen, as for the module-level strings that fix a URL.
+
+A lookup in a dict display, `{"list": "ls"}[key]` or `.get(key, default)`, gives one of
+its values, of what was stored in it since, or the default, never a key: input that only
+selects the entry, or that is itself a key of the display, is not passed on. The same lookup in a module-level dict passes the key on, as any other
+subscript does.
+
 Django's `render_to_string(name, context)` returns HTML in which autoescaping escaped
 every variable, so data reaching a response through it is no `xss`, if the template really
 escapes. On a directory check, the analyzer reads the project's templates to decide it.
@@ -166,6 +204,39 @@ It establishes escaping only when all of the following hold:
   or an attribute name.
 - No Python file of the project sets `autoescape` to `False`.
 
+Templates are read as a browser's HTML tokenizer reads them, in time linear in the size
+of the template. What a template leaves open at its end is checked too, since once the
+template is included it runs into the markup that follows: a tag's attributes are
+checked, a quoted value stays quoted until its quote closes, and a variable in a
+`<script>` or `<style>` block or an end tag left open is refused, even where the page
+ends there and the browser would never run it. A template including one that ends inside
+such a block, a tag or an end tag is not established either: the markup after the
+include continues it. A template extending another ends where that one ends. A comment
+holding template tags may end at any of them, since an include there may close it and a
+condition around its `<!--` may leave it out, or run to its close when they render
+nothing (#278). The markup is read on from its first tag, and that reading must be in
+HTML data right after each later tag and right after the close, where the others
+resume; where it is not, the template is not established. Such a comment left open does
+not end in HTML data, since it may hide what follows.
+
+Each page is also put together as Django renders it, and read once more (#273): the
+templates it extends, each block taking the definition of the most derived one and `{{
+block.super }}` the next one up, and each template it includes in place. What a child's
+block prints inside its parent's `<script>`, or what an include puts inside an
+attribute, is read where it lands. `{{ block.super }}` is put in place only bare: the
+variable `block` named anywhere else, through a filter, a tag or an alias (`{% with
+parent=block %}{{ parent.super }}`), may still print the parent's markup, marked safe,
+and the template is not established. A name several files have, which Django resolves
+through loaders the analyzer does not read, counts as each of them: a page extending it
+must escape with each file, except those already in its chain, which Django leaves out,
+so that a template extending its own name extends another file of it; an include of it
+is accepted only in HTML data, never inside an attribute, a tag, a `<script>` or
+`<style>` block or a comment, and only when each file ends in HTML data, nothing of it
+left open. A template including itself is read the same way. A page beyond 8 MiB once
+put together, or a template with more than 64 pages, or 16 MiB in all, through the files
+the templates it extends may be, is not established, and is listed among the templates
+the analyzer could not read.
+
 The analyzer never assumes escaping where it cannot read the template. That covers a
 template file it cannot read, a template named by an expression, a template found
 through a `DIRS` entry not named `templates`, one shipped by an installed package, one
@@ -181,7 +252,7 @@ reads no template.
 | `debug-enabled` | `app.run(debug=True)` on a Flask application. |
 | `missing-timeout` | A Requests or httpx call without a `timeout`. |
 | `insecure-tls` | `verify=False` on a Requests or httpx call, or `ssl._create_unverified_context()`. |
-| `unsafe-xml` | A standard-library or lxml XML parser, which expands entities; `defusedxml` is the safe replacement. |
+| `unsafe-xml` | An XML parser of the standard library or of lxml, reported for what it risks; `defusedxml` is the safe replacement. The standard library's parsers run on Expat, which reads no local file but expands the entities a document declares: a crafted document can exhaust memory with Expat older than 2.7.2, which depends on the Python build at run time (`pyexpat.EXPAT_VERSION`). lxml resolves external entities by default before 5.0, and in `iterparse` and `ETCompatXMLParser` before 6.1 (CVE-2026-41066), which lets a crafted document read local files; `resolve_entities=True` does so whatever the version, and `'internal'` keeps external entities out only from lxml 5.0 on, an earlier lxml reading the string as true; a parser given `resolve_entities=False` is not reported. XML content the project wrote itself, a string constant or a module-level string bound once passed to a function that parses its argument as the document (`fromstring`, `parseString`, `XML`), is reported at low confidence; a constant path keeps its confidence, since whoever can write the file chooses the document. |
 | `unsafe-archive-extraction` | `tarfile` or `zipfile` extraction, or `shutil.unpack_archive`, without a member `filter`. |
 | `insecure-temp-file` | `tempfile.mktemp`, which names a file without creating it. |
 | `weak-random` | A `random` function whose result is assigned to a credential-like name, or produced by a function named like one. Medium confidence. |
@@ -194,7 +265,7 @@ sources and in configuration files.
 | Rule | Confidence | Trigger |
 |---|---|---|
 | `hardcoded-secret` | high | A provider-specific format: AWS, GitHub, Slack, Stripe, Google, private keys, JWTs, SendGrid, Twilio. |
-| `hardcoded-credential` | medium | A credential-like name (`password`, `token`, `api_key`, `app.config['SECRET_KEY']`, …) bound to a real value. Placeholders are excluded. Low confidence in test code, fixtures and template files such as `.env.example`, with the context in the metadata. |
+| `hardcoded-credential` | medium | A credential-like name (`password`, `token`, `api_key`, `app.config['SECRET_KEY']`, …) bound to a real value. Placeholders are excluded. Low confidence in test code, fixtures and template files such as `.env.example`, with the context in the metadata, judged on the file's path in the project, not on where the project sits. |
 | `high-entropy-string` | low | An opaque high-entropy token. Hex digests (MD5, SHA-1, SHA-256, SHA-512 lengths, or a name such as `hash`, `checksum`, `commit`), subresource-integrity hashes (`sha512-…`) and character-set constants are not secrets unless a credential name says so. |
 
 ### Dependencies
@@ -303,7 +374,7 @@ the directory of the checked file; a path outside it is printed as it is.
 ```json
 {
   "schema_version": 1,
-  "tool": {"name": "coretrace-python-analyzer", "version": "0.22.0"},
+  "tool": {"name": "coretrace-python-analyzer", "version": "0.24.0"},
   "root": "/home/me/project",
   "findings": [
     {
@@ -334,6 +405,25 @@ in the file (`"/tool/app/api_token"`). The SARIF log gives such a result no `reg
 with the pointer under the location's `properties`, and the text report shows the file
 alone (`settings.toml: high hardcoded-credential: …`).
 
+A finding in test code carries `context: test` in its metadata, in the JSON report and as
+a property of its SARIF result. Test code is a file under a `test`, `tests`, `testing`,
+`fixtures` or `__tests__` directory of the project, or a file named `test_*.py`,
+`*_test.py` or `conftest.py`, judged on its path from the root of the report, not on
+where the project sits. For a file checked alone, that root is the file's own directory,
+as the report gives it, so a `test_*.py` module is test code either way, while a test
+directory above the file is seen only when its directory is checked. The context
+changes no verdict: it is there for baselines, `--fail-on`
+wrappers, readers and refiner plugins, which see it. A rule may still give its own
+context, as `hardcoded-credential` does with `example` for template files, and the
+SARIF result carries that context too.
+
+A finding in vendored code, under a `third_party`, `third-party`, `vendor` or `vendored`
+directory, carries `origin: vendored`, in the JSON report and as a property of its SARIF
+result. The origin is told apart from the context: the tests of a vendored project
+carry both `context: test` and `origin: vendored`. Vendored code is still analysed and
+its findings reported, so that a flow from the project through a vendored module is
+not lost; the origin changes no verdict either.
+
 `root` is the directory the paths are relative to. A file's `status` is `analysed`,
 `ambiguous` (analysed, but another file has its module name, see above), `syntax-error`
 or `unreadable`; only `analysed` files count in `files_analysed`.
@@ -348,7 +438,7 @@ bytes. A single-file check lists no components.
 
 ```json
 "tool": {
-  "name": "coretrace-python-analyzer", "version": "0.22.0",
+  "name": "coretrace-python-analyzer", "version": "0.24.0",
   "components": [
     {"kind": "plugin", "name": "curated-advisories", "version": "2026.09.25", "digest": "sha256:…"},
     {"kind": "advisories", "name": "advisories.json", "digest": "sha256:…"}
@@ -362,7 +452,14 @@ several sources define the same advisory, it names the one whose definition is u
 the next section describes.
 
 `--format sarif` prints a SARIF 2.1.0 log, one run with the tool, its rules and one
-result per finding. The root is declared once as the `SRCROOT` original URI base and
+result per finding. Each rule carries a title (`shortDescription`) and what it detects
+(`fullDescription`), so that a code scanning service can explain a finding: the plugin that
+reports a rule describes it, the engine its own notes, and a rule a plugin does not describe
+keeps its id alone. Each result carries the finding's confidence as the `confidence`
+property. A flow that reaches its sink inside a project function is reported at the call, and
+points at the sink as a related location (`relatedLocations`), with the message
+`sink <sink>, reached via <function>`, which the result's message links to. The root is
+declared once as the `SRCROOT` original URI base and
 every location under it is relative to that base, which is what code scanning services
 need to attach results to files. The components of a directory check are the tool's
 `extensions`, their kind and digest in `properties`:
@@ -718,7 +815,7 @@ project's behalf would go unseen. Without a lock file, a package is never
 `not_affected`. The document's `@id` derives from its statements, so two runs with the
 same result share it; its `author` is `Unknown Author`, as OpenVEX tools write when they
 cannot know it. Its `tooling` names the analyzer and each component with its digest
-(`coretrace-python-analyzer 0.22.0; curated-advisories 2026.09.25 (sha256:…)`), so the
+(`coretrace-python-analyzer 0.24.0; curated-advisories 2026.09.25 (sha256:…)`), so the
 `@id` changes with the advisory data a statement was decided with.
 
 ## Large projects
@@ -730,8 +827,15 @@ graph and the modules it imports. On the next run an unchanged
 module is served from the cache, so editing one file re-analyses that file and its
 importers only. Entries are plain data; an unreadable entry is recomputed.
 
-`--jobs N` analyses independent modules in `N` processes. Modules are scheduled imports
-first, so the result is the same whatever `N`.
+`--jobs N` (`-j N`) analyses independent modules in `N` processes; `0` takes one per core
+the process may run on, as `runtime.jobs` of `ctrace` does (its CPU affinity, read from
+Python 3.13 on, and on Linux before; the cores of the machine otherwise), and the default
+is one. On Windows, a pool holds at most 61 processes, the limit of Python's process pool
+there, whatever `N`.
+Modules are scheduled imports first, so the result is the same whatever `N`: the report,
+the order of its findings, `--fail-on`, the cache, the baseline and the SBOM and VEX
+documents. The command always waits for the complete report. There is no `--async`
+option: `ctrace` deprecated its own for `-j`/`--jobs`, which this follows.
 
 ```bash
 coretrace-python-analyzer --check src/ --cache .coretrace --jobs 4
@@ -750,11 +854,24 @@ Measured on one core of a laptop, single process, cold start, with the bundled p
 
 Time grows a little faster than linearly with the size of the project, because the
 interprocedural summaries are iterated to a fixpoint over the module graph. A warm
-`--cache` brings an unchanged healthchecks to about one second, `--jobs 4` saves a
-quarter of the cold time at the cost of one process's memory per job. The
+`--cache` brings an unchanged healthchecks to about one second. The
 `healthchecks` repository is part of the regression corpus, so the analysis time of a
 real 45 000-line project is exercised on every change.
 
+`--jobs` pays when a wave of the module graph holds many components: each process
+prepares the project once and reads the summaries of the modules a component imports
+from a directory of finished modules, the cache or one of the run's own, at the cost of
+one process's memory per job. Measured on a laptop with 8 cores (4 performance, 4
+efficiency), macOS, Python 3.14, bundled plugins, cold start (October 2026):
+
+| Project | Python files | `-j 1` | `-j 8` |
+|---|---:|---:|---:|
+| healthchecks | 653 | 15.3–15.5 s | 7.9–9.2 s |
+| mitmproxy | 583 | 51.6 s | 45.6 s |
+
+The modules of a strongly connected import cycle are analysed together by one process,
+and a wave waits for its slowest component: a project whose cycles are large gains less.
+
 ## Plugins
 
 The package ships 35 plugins, loaded by default. Security models for the standard library
@@ -844,10 +961,17 @@ comprehensions, lambdas, nested functions and classes, `global`, `nonlocal`, `de
 assignment expressions (`(y := f(x))`, laid out as an assignment before the statement),
 `yield from`, starred assignment targets (`first, *rest = items`), class keyword arguments
 such as `metaclass=`, `with` targets of any assignable form and `match` with every
-pattern kind. `finally` is modelled on the normal path only. A
+pattern kind. An assignment expression in an annotation binds its name as Python 3.11 to
+3.13 do, where the annotation stands: in a function, the name is a local whether the
+annotation is evaluated or not; at module level, it exists only once the annotation is
+evaluated, so under `from __future__ import annotations` a builtin it names stays the
+builtin. `finally` is modelled on the normal path only. A
 `nonlocal` write made by a nested function flows back to the enclosing variable when the
 enclosing function calls that nested function directly by name; a nested function passed
-around and called elsewhere keeps its writes to itself. Methods of module-level classes are
+around and called elsewhere keeps its writes to itself. Where the nested function may
+return without writing the name (no write on some path, a write constant propagation
+proves cannot run, or an exception before it), the variable keeps the value it had
+before the call, joined with what may be written; only a write on every path replaces it. Methods of module-level classes are
 analysed like functions.
 
 ## Looking at the intermediate representation

{coretrace_python_analyzer-0.22.0 → coretrace_python_analyzer-0.24.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "coretrace-python-analyzer"
-version = "0.22.0"
+version = "0.24.0"
 description = "Static security analysis infrastructure for Python code"
 readme = "README.md"
 requires-python = ">=3.11"

{coretrace_python_analyzer-0.22.0 → coretrace_python_analyzer-0.24.0}/src/coretrace_python/__init__.py

@@ -1,4 +1,4 @@
 """CoreTrace's Python static analysis frontend."""
 
-__version__ = "0.22.0"
+__version__ = "0.24.0"