modelaudit 0.2.34__tar.gz → 0.2.35__tar.gz

modelaudit-0.2.35/.release-please-manifest.json

@@ -0,0 +1,3 @@
+{
+  ".": "0.2.35"
+}

{modelaudit-0.2.34 → modelaudit-0.2.35}/CHANGELOG.md

@@ -5,10 +5,52 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.2.35](https://github.com/promptfoo/modelaudit/compare/v0.2.34...v0.2.35) (2026-04-11)
+
+### Bug Fixes
+
+- clean up oversized zip entry temps ([#911](https://github.com/promptfoo/modelaudit/issues/911)) ([66b4871](https://github.com/promptfoo/modelaudit/commit/66b4871f49e367dea545f36af85c9cc75303d615))
+- flag Paddle code patterns as warnings ([#925](https://github.com/promptfoo/modelaudit/issues/925)) ([32fa0b7](https://github.com/promptfoo/modelaudit/commit/32fa0b7551c13059515c464b0118851fa1fbe671))
+- harden manifest parse boundaries ([#922](https://github.com/promptfoo/modelaudit/issues/922)) ([6f5b516](https://github.com/promptfoo/modelaudit/commit/6f5b516bec8492b2f062ba5ea10498c705d972ca))
+- harden standalone pickle scanner ([#901](https://github.com/promptfoo/modelaudit/issues/901)) ([31f7dd3](https://github.com/promptfoo/modelaudit/commit/31f7dd38c6bd77631ccdca90438312c4db2ac857))
+- mark corrupt NumPy object payloads inconclusive ([#912](https://github.com/promptfoo/modelaudit/issues/912)) ([ecba19d](https://github.com/promptfoo/modelaudit/commit/ecba19dc585d5bfbfbfbd687e81cd734a7b0103b))
+- mark incomplete MXNet scans inconclusive ([#923](https://github.com/promptfoo/modelaudit/issues/923)) ([a928ed7](https://github.com/promptfoo/modelaudit/commit/a928ed723a220185c3c0ea4b046b8885c74e8f62))
+- mark incomplete sharded scans inconclusive ([#909](https://github.com/promptfoo/modelaudit/issues/909)) ([510d0fb](https://github.com/promptfoo/modelaudit/commit/510d0fbe45ae9f1b7e213227ebb1210b15a35991))
+- mark malformed GGUF scans inconclusive ([#914](https://github.com/promptfoo/modelaudit/issues/914)) ([9b3e216](https://github.com/promptfoo/modelaudit/commit/9b3e21607309b846b15f809af6fd1bef31268b6a))
+- mark malformed Keras H5 configs inconclusive ([#917](https://github.com/promptfoo/modelaudit/issues/917)) ([23671c3](https://github.com/promptfoo/modelaudit/commit/23671c38796293978b0538eb4c7ce30c8cfa5160))
+- mark malformed Keras ZIP configs inconclusive ([#918](https://github.com/promptfoo/modelaudit/issues/918)) ([d4ad8d8](https://github.com/promptfoo/modelaudit/commit/d4ad8d8717c4f1ca647b292035f68bbf570d9904))
+- mark malformed SafeTensors scans inconclusive ([#913](https://github.com/promptfoo/modelaudit/issues/913)) ([43913d6](https://github.com/promptfoo/modelaudit/commit/43913d65c5eb89014d1bb137768f89e93b8d0d41))
+- mark malformed tflite scans inconclusive ([#916](https://github.com/promptfoo/modelaudit/issues/916)) ([07c871a](https://github.com/promptfoo/modelaudit/commit/07c871a8d19e9181bdcd568fffa9a165883585de))
+- mark partial archive scans inconclusive ([#907](https://github.com/promptfoo/modelaudit/issues/907)) ([c8eb918](https://github.com/promptfoo/modelaudit/commit/c8eb918b8d0a717460be93097cfc1cf0a47e6689))
+- mark partial streaming scans inconclusive ([#908](https://github.com/promptfoo/modelaudit/issues/908)) ([3d47a10](https://github.com/promptfoo/modelaudit/commit/3d47a1055d09c20995c21ebe75a50a2c3d1105f0))
+- mark unknown ONNX tensor dtypes inconclusive ([#915](https://github.com/promptfoo/modelaudit/issues/915)) ([35661b6](https://github.com/promptfoo/modelaudit/commit/35661b6ac166f38f7642ac9a3ea89b6cea538928))
+- preserve picklescan stack state ([#910](https://github.com/promptfoo/modelaudit/issues/910)) ([fabac5c](https://github.com/promptfoo/modelaudit/commit/fabac5c9ead49c2ed5f8357dfa53ccdcce946527))
+- recover malformed Jinja template configs ([#920](https://github.com/promptfoo/modelaudit/issues/920)) ([d619c8f](https://github.com/promptfoo/modelaudit/commit/d619c8f185040c7b3c772a4b94631edddde9d8a8))
+- route corrupt catboost scans fail closed ([#924](https://github.com/promptfoo/modelaudit/issues/924)) ([052bb5f](https://github.com/promptfoo/modelaudit/commit/052bb5f4e6dbc5e48a3fe5d134e0ec8d9605e292))
+- traverse nemo yaml list configs ([#919](https://github.com/promptfoo/modelaudit/issues/919)) ([0d8d4fd](https://github.com/promptfoo/modelaudit/commit/0d8d4fd4dc2ef774db093fb9e7daf27c32b5a0a8))
+- **zip:** fail closed on MAR handler parse errors ([#896](https://github.com/promptfoo/modelaudit/issues/896)) ([a06a620](https://github.com/promptfoo/modelaudit/commit/a06a620f011d120072b1e8619e543a7306d5a4fc))
+
+### Documentation
+
+- add repo correctness audit ledger ([#921](https://github.com/promptfoo/modelaudit/issues/921)) ([06be0b6](https://github.com/promptfoo/modelaudit/commit/06be0b6eaeb53f5f238612a386665c45f3c27dc2))
+
 ## [0.2.34](https://github.com/promptfoo/modelaudit/compare/v0.2.33...v0.2.34) (2026-04-10)
 
 ### Bug Fixes
 
+- flag Paddle code patterns as warnings instead of failing benign scans
+- route corrupt CatBoost scans to fail closed outcomes
+- mark incomplete MXNet scans inconclusive instead of clean
+- harden manifest parse boundaries around malformed metadata
+- recover malformed Jinja template configs as inconclusive scan outcomes
+- traverse NeMo YAML list configs when checking suspicious targets
+- mark malformed Keras ZIP configs inconclusive instead of clean
+- mark malformed Keras H5 scans inconclusive instead of clean
+- mark malformed TFLite scans inconclusive instead of clean
+- mark malformed GGUF scans inconclusive instead of clean
+- mark malformed SafeTensors scans inconclusive instead of clean
+- preserve picklescan stack state across reused scanner runs
+- mark partial streaming scans inconclusive when large-file streaming coverage is incomplete
 - harden native code detection in model scanners ([#897](https://github.com/promptfoo/modelaudit/issues/897)) ([f4f661a](https://github.com/promptfoo/modelaudit/commit/f4f661a09be0032e15aa8895864413e3878233f8))
 
 ## [0.2.33](https://github.com/promptfoo/modelaudit/compare/v0.2.32...v0.2.33) (2026-04-09)
@@ -41,17 +83,23 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Bug Fixes
 
+- mark trailing bytes after NumPy object-array pickle payloads inconclusive without escalating to security findings
 - avoid CoreML nested parse failures on bounded-read truncation
+- mark incomplete sharded-model scans as inconclusive, ignore shard-name prefix matches, and skip caching explicit incomplete outcomes
 - flag TensorFlow `LoadLibrary` and `LoadLibraryV2` graph ops as dangerous native-library loading
 - detect split CNTK native-user-function and native-library references
 - detect Linux/macOS native-library members in Keras archives and uppercase native-library members in PyTorch ZIPs
 - detect embedded Windows DLL/PE, Linux ELF shared-object, and TensorRT plugin entry-point markers in TensorRT engines
 - detect punctuation-delimited TensorRT `/tmp` plugin paths
+- clean up temporary ZIP entry files when extraction fails on entry size limits
 - preserve HuggingFace cache provenance for symlinked custom cache roots
+- mark ONNX tensor dtype validation failures inconclusive instead of allowing clean scans
 - ignore remote OCI `layers[].urls` entries during local layer discovery
 - fail closed on unterminated OpenVINO DOCTYPE declarations
 - avoid PMML `<Extension>` false positives for benign `subprocess` prose while preserving `subprocess.getoutput()`, `subprocess.getstatusoutput()`, and `importlib.import_module("subprocess")` detections
+- mark incomplete ZIP, TAR, and 7z archive traversals as inconclusive in scan metadata
 - route helper-level ZIP-backed `.ckpt`/`.pkl` checkpoints through archive scanners
+- harden standalone pickle scanner dangerous global coverage, nested payload bounds, incomplete-scan reporting, and standalone-primary migration behavior
 
 ## [0.2.31](https://github.com/promptfoo/modelaudit/compare/v0.2.30...v0.2.31) (2026-04-04)
 

{modelaudit-0.2.34 → modelaudit-0.2.35}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: modelaudit
-Version: 0.2.34
+Version: 0.2.35
 Summary: Static scanning library for detecting malicious code, backdoors, and other security risks in ML model files
 Project-URL: Repository, https://github.com/promptfoo/modelaudit
 Project-URL: Homepage, https://github.com/promptfoo/modelaudit

{modelaudit-0.2.34 → modelaudit-0.2.35}/docs/agents/picklescan-package-split.md

@@ -36,6 +36,10 @@ modelaudit/
 - `modelaudit` owns file routing, archive/container orchestration, CLI, cache,
   telemetry, SARIF/export integrations, and `PickleReport -> ScanResult`
   adaptation.
+- During the migration period, `modelaudit.scanners.pickle_scanner.PickleScanner`
+  still merges legacy-only checks after the standalone pass. Keep this fallback
+  until the parity harness shows that standalone verdict, status, and required
+  rule coverage are sufficient for the root scanner to depend on it alone.
 - Wrapper scanners in `modelaudit` pass embedded pickle streams into
   `modelaudit-picklescan`; archive parsing stays in `modelaudit`.
 - The root `modelaudit` wheel bundles `modelaudit_picklescan` as a second import
@@ -56,6 +60,9 @@ scanner = PickleScanner(options=ScanOptions(timeout_s=30.0, max_opcodes=1_000_00
 report = scanner.scan_stream(stream, source="archive.pt:data.pkl", size=pickle_size)
 ```
 
+Resource controls include opcode and wall-clock limits, post-budget tail bytes,
+string-literal scan characters, nested-pickle bytes, and nested scan depth.
+
 Report semantics keep these concepts separate:
 
 - `status`: scan completeness (`complete`, `inconclusive`, `error`)
@@ -63,12 +70,17 @@ Report semantics keep these concepts separate:
 - `findings`: `WARNING`/`CRITICAL` security findings only
 - `notices`: `DEBUG`/`INFO` coverage or explainability notes
 - `errors`: operational failures
+- report mappings are read-only after construction; call `to_dict()` for mutable
+  serialized data
 
 ## Current Integration
 
-- `modelaudit.scanners.pickle_scanner.PickleScanner` scans through the
-  standalone package first, adapts the `PickleReport` into a `ScanResult`, and
-  merges in any legacy-only checks that are still needed for compatibility.
+- `modelaudit.scanners.pickle_scanner.PickleScanner` scans through both engines.
+  The default root result is still legacy-primary for compatibility while the
+  migration is in progress. Set `use_standalone_pickle_primary=True` in scanner
+  config to exercise the intended standalone-primary merge path, where the
+  adapted `PickleReport` owns the result and legacy-only checks are merged as
+  compatibility evidence.
 - Embedded-pickle wrapper scanners (`pytorch_zip`, `joblib`, `numpy`, and
   `executorch`) call the public `scan_stream(..., source=...)` API and preserve
   archive-member context in result locations/details.

modelaudit-0.2.35/docs/agents/repo-correctness-audit.md

@@ -0,0 +1,209 @@
+# ModelAudit Correctness Audit Ledger
+
+Status: active
+Owner: agent-maintained
+Started: 2026-04-10
+
+This file is the running plan, evidence log, and findings ledger for a repo-wide
+correctness audit. It is intentionally conservative: a finite audit cannot prove
+that every future input is safe or that the code is "perfect." The practical goal
+is stronger than ad hoc review: define explicit proof obligations, record which
+ones have evidence, and turn every concrete gap into a small PR with regression
+tests.
+
+## Correctness Standard
+
+A component is not considered proven until it has evidence for all relevant
+obligations below.
+
+1. Routing correctness
+   - Content/structure wins over suffix-only routing where feasible.
+   - Spoofed extensions and nested archive members route to the intended scanner.
+   - Benign near-matches stay clean.
+   - Malicious positives hit the security scanner that owns the format.
+
+2. Parser and structure boundaries
+   - Malformed input does not crash.
+   - Malformed or unsupported structure is not reported as clean when coverage is
+     incomplete.
+   - If scanning cannot cover the intended security surface, the result is
+     operationally explicit: `scan_outcome=inconclusive`, `success=False`, or an
+     operational error depending on the component contract.
+   - If a bounded raw fallback can safely recover security evidence, it runs
+     before returning inconclusive.
+
+3. Security precedence
+   - Warning/critical security findings keep exit code 1 even when the scan is
+     also inconclusive.
+   - Operational or coverage failures without security findings return exit code 2.
+   - INFO-only review notes do not create security failures.
+
+4. Bounded resource use
+   - Reads are size-limited for metadata, archive members, tensors, and embedded
+     payloads.
+   - Recursive archive scans have depth, file count, byte, and timeout budgets.
+   - Temporary extraction paths are sanitized, contained, and cleaned up.
+
+5. Cache and repeatability
+   - Cached results preserve `scan_outcome`, issue severity, scanner name, and
+     exit-code semantics.
+   - Deterministic fixtures do not depend on host paths, global temp names, or
+     installed heavyweight frameworks unless explicitly gated.
+
+6. Optional dependency behavior
+   - Missing optional dependencies fail gracefully.
+   - A missing parser cannot silently turn security coverage into a clean pass.
+   - Tests cover at least one missing-dependency path for dependency-sensitive
+     scanners.
+
+7. Output and integration consistency
+   - CLI, JSON, SARIF, asset inventory, cache, and programmatic APIs agree on
+     success, issue severity, scanner name, and exit code.
+   - File metadata can round-trip through Pydantic models without dropping
+     safety-relevant fields.
+
+## Evidence Levels
+
+- E0: Inventory only. No current audit evidence.
+- E1: Existing test coverage observed, but proof obligations not fully checked.
+- E2: Focused audit found no defect for the selected obligations.
+- E3: Focused audit found a defect and a PR was opened with regression tests.
+- E4: Full obligation suite implemented and passing for the component.
+
+E4 is the target. Most components start below E4.
+
+## Audit Scope Map
+
+### Core and Cross-Cutting Layers
+
+| Area                      | Files                                                                                                              | Initial risks                                                                                 | Evidence                         |
+| ------------------------- | ------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------- | -------------------------------- |
+| Core orchestration        | `modelaudit/core.py`, `modelaudit/models.py`, `modelaudit/scanner_results.py`                                      | exit-code precedence, scan metadata preservation, directory dedupe, cache and stream behavior | E1                               |
+| Scanner registry/routing  | `modelaudit/scanners/__init__.py`, `modelaudit/scanner_registry_metadata.py`, `modelaudit/utils/file/detection.py` | suffix routing, header aliases, optional dependency fallback, lazy loading                    | E1                               |
+| CLI/output                | `modelaudit/cli.py`, output helpers, SARIF/JFrog integrations                                                      | JSON/SARIF consistency, exit codes, partial scans                                             | E0                               |
+| Cache                     | `modelaudit/cache/`, `modelaudit/utils/helpers/cache_decorator.py`                                                 | stale safety metadata, inconclusive persistence, config-sensitive keys                        | E1                               |
+| Archive recursion         | `zip`, `tar`, `sevenzip`, `compressed`, `oci_layer`, `torchserve_mar` scanners                                     | traversal, temp cleanup, nested routing, partial coverage                                     | E3 through recent PRs            |
+| Standalone pickle package | `packages/modelaudit-picklescan/`                                                                                  | parity with adapter, opcode budgets, immutable results                                        | E3 through recent PRs            |
+| Test infrastructure       | `tests/conftest.py`, CI allowlists                                                                                 | regression tests skipped in reduced Python lanes                                              | E3 for current allowlist updates |
+
+### Scanner Inventory
+
+| Scanner               | Primary files/formats                                      | Current evidence | Next proof target                                               |
+| --------------------- | ---------------------------------------------------------- | ---------------- | --------------------------------------------------------------- |
+| `pickle`              | `.pkl`, `.pickle`, `.dill`, `.bin`, `.pt`, `.pth`, `.ckpt` | E3               | post-budget and malformed opcode corpus parity                  |
+| `picklescan_adapter`  | standalone picklescan bridge                               | E3               | adapter/cache equivalence for inconclusive reports              |
+| `pytorch_zip`         | ZIP-backed PyTorch checkpoints                             | E3               | ZIP metadata parse boundaries and nested pickle cache semantics |
+| `pytorch_binary`      | raw `.bin` PyTorch-like blobs                              | E1               | bounded binary fallback and benign weight near-matches          |
+| `joblib`              | `.joblib`, compressed/raw pickle wrappers                  | E3               | codec failure semantics and cache preservation                  |
+| `jax_checkpoint`      | JAX/Orbax/checkpoint pickles                               | E1               | index/metadata structure failures and nested pickle routing     |
+| `flax_msgpack`        | `.msgpack`, `.flax`, `.orbax`, `.jax`                      | E1               | msgpack extension types, depth, and partial unpack coverage     |
+| `numpy`               | `.npy`, `.npz`                                             | E3               | object-array pickle failures and `.npz` member routing          |
+| `safetensors`         | `.safetensors`                                             | E3               | malformed header/schema and dtype consistency                   |
+| `keras_h5`            | HDF5 Keras models                                          | E3, PR #917      | cache and aggregate semantics after malformed config fixes      |
+| `keras_zip`           | `.keras` ZIP models                                        | E3, PR #918      | metadata/weights alias ambiguity after malformed config fixes   |
+| `tf_savedmodel`       | SavedModel dirs, `.pb`                                     | E1               | protobuf parse budgets and function library edges               |
+| `tf_metagraph`        | `.meta`                                                    | E1               | protobuf parse budgets and attr truncation semantics            |
+| `tflite`              | `.tflite`, routed `.bin`                                   | E3, PR #916      | flatbuffer table bounds and custom-op recovery                  |
+| `onnx`                | `.onnx`                                                    | E3, PR #915      | external data path policy and dtype coverage                    |
+| `coreml`              | `.mlmodel`                                                 | E3               | protobuf truncation, linked model paths, custom layer strings   |
+| `openvino`            | `.xml` IR                                                  | E3               | XML parse failures, entity/DOCTYPE boundaries, companion `.bin` |
+| `gguf`                | `.gguf`, `.ggml`, related                                  | E3, PR #914      | metadata value type matrix and tensor offset checks             |
+| `xgboost`             | `.bst`, `.model`, `.json`, `.ubj`                          | E1               | JSON/UBJSON malformed root, subprocess isolation                |
+| `lightgbm`            | `.model`, `.txt`, `.lgb`, `.lightgbm`                      | E1               | text parser bounds and native-library indicators                |
+| `catboost`            | `.cbm`                                                     | E3, PR #924      | binary marker bounds and metadata strings                       |
+| `mxnet`               | `*-symbol.json`, `*-NNNN.params`                           | E3, PR #923      | graph reference traversal and metadata payload recovery         |
+| `nemo`                | `.nemo` tar archives                                       | E3, PR #919      | multi-config precedence and malformed member combinations       |
+| `jinja2_template`     | tokenizer configs, YAML, templates, GGUF metadata          | E3, PR #920      | cache preservation and GGUF metadata extraction failures        |
+| `skops`               | `.skops` ZIP archives                                      | E3               | JSON schema variations and duplicate member precedence          |
+| `torchserve_mar`      | `.mar` archives                                            | E3               | manifest schema roots and handler AST edge cases                |
+| `oci_layer`           | OCI `.manifest`                                            | E3               | manifest schema roots, local-vs-remote layer resolution         |
+| `zip`                 | generic ZIP/NPZ/MAR fallback                               | E3               | unsupported member failure semantics and cleanup                |
+| `tar`                 | tar families                                               | E3               | unsupported member failure semantics and cleanup                |
+| `sevenzip`            | `.7z`                                                      | E3               | nested routing parity with ZIP/TAR                              |
+| `compressed`          | `.gz`, `.bz2`, `.xz`, `.lz4`, `.zlib`                      | E3               | wrapper extension inference and temporary cleanup               |
+| `manifest`            | model/config manifests                                     | E3, PR #922      | JSON/YAML/TOML malformed roots and nested scanning              |
+| `metadata`            | model cards/docs/text                                      | E1               | secret/security pattern false positives and truncation          |
+| `text`                | general text docs                                          | E0               | duplicate responsibility with metadata/manifest                 |
+| `pmml`                | `.pmml`                                                    | E3               | XML parse boundaries and extension payload recovery             |
+| `paddle`              | `.pdmodel`, `.pdiparams`                                   | E3, PR #925      | protobuf/op descriptor parse failures                           |
+| `cntk`                | `.dnn`, `.cmf`                                             | E3               | split reference tracking and malformed binary handling          |
+| `rknn`                | `.rknn`                                                    | E1               | marker and string extraction bounds                             |
+| `torch7`              | `.t7`, `.th`, `.net`                                       | E1               | legacy serialization parse failures                             |
+| `r_serialized`        | `.rds`, `.rda`, `.rdata`                                   | E1               | format header variants and string extraction bounds             |
+| `executorch`          | `.ptl`, `.pte`                                             | E1               | archive/table parse failures and nested payloads                |
+| `tensorrt`            | `.engine`, `.plan`, `.trt`                                 | E3               | plugin marker matrix and binary truncation                      |
+| `llamafile`           | `.llamafile`, `.exe`, extensionless                        | E1               | executable header routing and model payload boundaries          |
+| `weight_distribution` | optional secondary analysis                                | E0               | optional dependency isolation and non-security failure behavior |
+
+## Current Findings and PR Ledger
+
+Recent concrete fixes from this audit stream:
+
+| PR   | Component         | Finding                                                                                                                            | Status               |
+| ---- | ----------------- | ---------------------------------------------------------------------------------------------------------------------------------- | -------------------- |
+| #917 | Keras H5          | Malformed config/training config could be treated as clean or wrong security failure instead of inconclusive coverage              | Open, review pending |
+| #918 | Keras ZIP         | Malformed `config.json` structures could scan clean or crash as the wrong failure type                                             | Open, review pending |
+| #919 | NeMo              | Top-level YAML lists were not traversed for Hydra `_target_`; malformed/scalar configs looked like missing config                  | Open, review pending |
+| #920 | Jinja2 template   | Malformed tokenizer/YAML configs swallowed parse failures and returned "No templates found"; raw visible SSTI payloads were missed | Open, review pending |
+| #922 | Manifest          | `.config` INI manifests with section headers could skip structured parsing and lose URL/hash checks                                | Open, review pending |
+| #923 | MXNet             | Malformed symbol artifacts needed routing into fail-closed scanner outcomes instead of aggregate clean/unknown results             | Open, review pending |
+| #924 | CatBoost          | Corrupt declared-section scans fail closed as inconclusive instead of returning incomplete coverage as clean                       | Open, review pending |
+| #925 | Paddle            | Suspicious Paddle code indicators are warnings, preserving signal without escalating review-only findings to errors                | Open, review pending |
+| #926 | Native code tests | Expanded native-code detection regression coverage and benign executable-suffix near-match negatives                               | Open, review pending |
+
+Earlier open PRs from the same boundary-hardening campaign include #901 and
+#907 through #916. All open PR entries remain provisional until CI and review
+complete; treat them as evidence of audited findings, not landed behavior.
+
+## Audit Workflow
+
+Each iteration should do the following:
+
+1. Sync from `main` and check open PR CI/review state.
+2. Pick one high-risk component from the inventory, preferring E0/E1 items and
+   parser/routing boundaries that can create clean false negatives.
+3. Reproduce a concrete failure before editing.
+4. Patch the narrow behavior with existing architecture patterns.
+5. Add typed, deterministic tests:
+   - malicious positive
+   - benign near-match negative
+   - malformed/unsupported structure
+   - aggregate exit-code semantics
+   - cache semantics when safety metadata is involved
+6. Run targeted tests, then the canonical validation gate from `AGENTS.md`.
+7. Update this ledger with the finding, evidence level, residual risk, and PR.
+8. Open or update a PR.
+
+## High-Risk Backlog
+
+1. `manifest_scanner`: structured config parser failures and unsupported roots.
+   Check JSON/YAML/TOML/INI parse behavior, nested list roots, and whether
+   malformed AIML manifests can return clean.
+2. `torchserve_mar_scanner`: manifest schema roots and parse errors. Verify that
+   handler AST findings and manifest failures preserve security precedence.
+3. `oci_layer_scanner`: manifest schema roots and local-vs-remote layer
+   resolution. Confirm remote URLs cannot be treated as local layers and local
+   malformed manifests fail closed.
+4. `mxnet_scanner`: symbol JSON schema and metadata payload traversal. Verify
+   list/dict roots, malformed JSON, and encoded payload recovery.
+5. `xgboost_scanner`: JSON/UBJSON parse failures and subprocess isolation.
+   Confirm malformed model configs cannot hide suspicious attributes.
+6. `tf_savedmodel` and `tf_metagraph`: protobuf parse budgets, attr truncation,
+   and function references.
+7. `llamafile`, `rknn`, `torch7`, `r_serialized`: binary string extraction
+   bounds and malformed file semantics.
+8. Cross-cutting cache tests for all inconclusive scanner families.
+9. SARIF/CLI/asset output consistency for inconclusive scans with and without
+   security findings.
+
+## Notes Log
+
+### 2026-04-10
+
+- Established this repo-wide audit ledger and proof standard.
+- Confirmed loop mode is enabled for continued audit iterations.
+- Current strongest recurring defect class: structured parser failures or
+  unsupported root shapes collapsing into clean scans. Fixed examples exist in
+  Keras H5, Keras ZIP, NeMo, and Jinja2 template scanners.
+- Next recommended target: `manifest_scanner`, because it owns many AIML config
+  filenames and parses several structured formats.

{modelaudit-0.2.34 → modelaudit-0.2.35}/modelaudit/cache/cache_policy.py

@@ -2,6 +2,8 @@
 
 from typing import Any
 
+from modelaudit.scanner_results import INCONCLUSIVE_SCAN_OUTCOME
+
 _OPERATIONAL_ERROR_INDICATORS = (
     "error during scan",
     "error checking file size",
@@ -33,6 +35,14 @@ _OPERATIONAL_ERROR_INDICATORS = (
 
 def should_cache_scan_result(scan_result: dict[str, Any]) -> bool:
     """Return True when a scan result is stable enough to cache safely."""
+    metadata = scan_result.get("metadata")
+    if isinstance(metadata, dict) and (
+        bool(metadata.get("operational_error"))
+        or bool(metadata.get("analysis_incomplete"))
+        or metadata.get("scan_outcome") == INCONCLUSIVE_SCAN_OUTCOME
+    ):
+        return False
+
     for collection_name in ("issues", "checks"):
         collection = scan_result.get(collection_name)
         if not isinstance(collection, list):

{modelaudit-0.2.34 → modelaudit-0.2.35}/modelaudit/core.py

@@ -58,6 +58,8 @@ _OPERATIONAL_ERROR_REASON_METADATA_KEY = "operational_error_reason"
 _SCAN_OUTCOME_METADATA_KEY = "scan_outcome"
 
 HEADER_FORMAT_TO_SCANNER_ID = _registry.get_header_format_to_scanner_ids()
+_COMPRESSED_HEADER_FORMATS = frozenset({"compressed", "gzip", "bzip2", "xz", "lz4", "zlib"})
+_R_SERIALIZED_EXTENSIONS = frozenset({".rds", ".rda", ".rdata"})
 
 
 def _mark_operational_scan_error(scan_result: ScanResult, reason: str) -> None:
@@ -66,6 +68,22 @@ def _mark_operational_scan_error(scan_result: ScanResult, reason: str) -> None:
     scan_result.metadata[_OPERATIONAL_ERROR_REASON_METADATA_KEY] = reason
 
 
+def _mark_inconclusive_scan_outcome(scan_result: ScanResult, reason: str) -> None:
+    """Mark a scan result as explicitly inconclusive for exit-code aggregation."""
+    scan_result.metadata["analysis_incomplete"] = True
+    scan_result.metadata[_SCAN_OUTCOME_METADATA_KEY] = INCONCLUSIVE_SCAN_OUTCOME
+    scan_result.metadata.setdefault(
+        "scan_outcome_message",
+        "Scan analysis incomplete; failed closed because full coverage was not available.",
+    )
+
+    existing_reasons = scan_result.metadata.get("scan_outcome_reasons")
+    reasons = existing_reasons if isinstance(existing_reasons, list) else []
+    if reason not in reasons:
+        reasons.append(reason)
+    scan_result.metadata["scan_outcome_reasons"] = reasons
+
+
 def _scan_result_has_operational_error(scan_result: ScanResult) -> bool:
     """Return True when a scan result represents an operational failure."""
     metadata = scan_result.metadata or {}
@@ -238,9 +256,12 @@ def _select_preferred_scanner_id(path: str, header_format: str, ext: str) -> str
             return "pickle"
         return "zip"
 
-    if ext == ".joblib" and header_format in {"compressed", "pickle"}:
+    if ext == ".joblib" and header_format in _COMPRESSED_HEADER_FORMATS | {"pickle"}:
         return "joblib"
 
+    if ext in _R_SERIALIZED_EXTENSIONS and header_format in _COMPRESSED_HEADER_FORMATS | {"r_serialized"}:
+        return "r_serialized"
+
     if header_format == "tar" and ext == ".nemo":
         return "nemo"
 
@@ -787,6 +808,8 @@ def scan_model_directory_or_file(
             if scanner:
                 scan_result, was_complete = stream_analyze_file(stream_url, scanner)
                 if scan_result:
+                    if not was_complete:
+                        _mark_inconclusive_scan_outcome(scan_result, "streaming_analysis_incomplete")
                     results.files_scanned += 1
 
                     # Use helper function to add scan result to Pydantic model

{modelaudit-0.2.34 → modelaudit-0.2.35}/modelaudit/detectors/suspicious_symbols.py

@@ -175,8 +175,17 @@ SUSPICIOUS_GLOBALS = {
     # Dynamic resolution / import trampolines
     "pkgutil": ["resolve_name", "get_importer", "walk_packages"],
     "zipimport": "*",
-    # uuid — _get_command_stdout/_popen internally call subprocess.Popen
-    "uuid": ["_get_command_stdout", "_popen"],
+    # uuid private helpers that invoke platform commands through subprocess wrappers
+    "uuid": [
+        "_arp_getnode",
+        "_get_command_stdout",
+        "_ifconfig_getnode",
+        "_ip_getnode",
+        "_lanscan_getnode",
+        "_netstat_getnode",
+        "_popen",
+        "getnode",
+    ],
     # Network / exfiltration
     "smtplib": "*",
     "xmlrpc": "*",
@@ -211,7 +220,7 @@ SUSPICIOUS_GLOBALS = {
     "timeit": ["timeit", "repeat"],
     "trace": "*",
     # Operator / functools bypasses
-    "functools": ["reduce", "partial"],
+    "functools": ["reduce", "partial", "partialmethod"],
     "_operator": "*",
     # Pickle recursion
     "cloudpickle": "*",
@@ -237,10 +246,10 @@ SUSPICIOUS_GLOBALS = {
     "_sqlite3": "*",
     "select": "*",
     "selectors": "*",
-    "logging": ["config"],
+    "logging.config": ["dictConfig", "fileConfig", "listen"],
     "syslog": "*",
-    "tarfile": "*",
-    "zipfile": "*",
+    "tarfile": ["open"],
+    "zipfile": ["ZipFile", "PyZipFile"],
     "shelve": "*",
     # Documentation / tooling (can execute code)
     "doctest": "*",

{modelaudit-0.2.34 → modelaudit-0.2.35}/modelaudit/scanner_registry_metadata.py

@@ -422,6 +422,7 @@ SCANNER_REGISTRY_METADATA: dict[str, dict[str, Any]] = {
         "class": "CompressedScanner",
         "description": "Scans standalone compressed wrappers and routes inner payloads to existing scanners",
         "extensions": [".gz", ".bz2", ".xz", ".lz4", ".zlib"],
+        "header_formats": ["gzip", "bzip2", "xz", "lz4", "zlib"],
         "priority": 95,
         "dependencies": [],
         "numpy_sensitive": False,

modelaudit-0.2.35/modelaudit/scanners/_archive_outcomes.py

@@ -0,0 +1,26 @@
+"""Shared metadata helpers for archive scans that intentionally stop early."""
+
+from __future__ import annotations
+
+from .base import INCONCLUSIVE_SCAN_OUTCOME, ScanResult
+
+
+def mark_archive_scan_incomplete(result: ScanResult, reason: str) -> None:
+    """Mark an archive result as explicitly inconclusive without changing findings."""
+    result.metadata["analysis_incomplete"] = True
+    result.metadata["scan_outcome"] = INCONCLUSIVE_SCAN_OUTCOME
+
+    existing_reasons = result.metadata.get("scan_outcome_reasons")
+    reasons = existing_reasons if isinstance(existing_reasons, list) else []
+    if reason not in reasons:
+        reasons.append(reason)
+    result.metadata["scan_outcome_reasons"] = reasons
+
+
+def member_scan_incomplete(result: ScanResult) -> bool:
+    """Return whether a nested archive member scan stopped before complete analysis."""
+    return (
+        result.metadata.get("analysis_incomplete") is True
+        or result.metadata.get("scan_outcome") == INCONCLUSIVE_SCAN_OUTCOME
+        or (not result.success and not result.has_errors)
+    )

{modelaudit-0.2.34 → modelaudit-0.2.35}/modelaudit/scanners/catboost_scanner.py

@@ -10,7 +10,7 @@ import struct
 from typing import Any, ClassVar
 from urllib.parse import urlparse
 
-from .base import BaseScanner, IssueSeverity, ScanResult
+from .base import INCONCLUSIVE_SCAN_OUTCOME, BaseScanner, IssueSeverity, ScanResult
 
 CATBOOST_MAGIC = b"CBM1"
 _SIZE_SENTINEL = 0xFFFFFFFF
@@ -99,22 +99,16 @@ class CatBoostScanner(BaseScanner):
 
         try:
             file_size = os.path.getsize(path)
-            if file_size < 8:
+            if file_size < 4:
                 return False
 
             with open(path, "rb") as f:
                 if f.read(4) != CATBOOST_MAGIC:
                     return False
-                core_size, header_size = cls._read_core_size(f)
 
-            if core_size <= 0:
-                return False
-
-            return header_size + core_size <= file_size
+            return True
         except OSError:
             return False
-        except (_CatBoostParseError, struct.error):
-            return False
 
     @staticmethod
     def _read_core_size(file_obj: Any) -> tuple[int, int]:
@@ -160,6 +154,7 @@ class CatBoostScanner(BaseScanner):
                 details={"error_type": type(error).__name__},
                 why="Corrupted or truncated model files should be treated as suspicious input.",
             )
+            self._mark_inconclusive_scan_result(result, "catboost_structure_parse_failed")
             result.finish(success=False)
             return result
         except OSError as error:
@@ -171,6 +166,7 @@ class CatBoostScanner(BaseScanner):
                 location=path,
                 details={"error": str(error), "error_type": type(error).__name__},
             )
+            self._mark_inconclusive_scan_result(result, "catboost_read_failed")
             result.finish(success=False)
             return result
 
@@ -195,6 +191,17 @@ class CatBoostScanner(BaseScanner):
         result.finish(success=not result.has_errors)
         return result
 
+    def _mark_inconclusive_scan_result(self, result: ScanResult, reason: str) -> None:
+        """Mark CatBoost analysis as incomplete for aggregate exit-code handling."""
+        existing_reasons = result.metadata.get("scan_outcome_reasons")
+        reasons = existing_reasons if isinstance(existing_reasons, list) else []
+        if reason not in reasons:
+            reasons.append(reason)
+
+        result.metadata["scan_outcome"] = INCONCLUSIVE_SCAN_OUTCOME
+        result.metadata["scan_outcome_reasons"] = reasons
+        result.metadata["analysis_incomplete"] = True
+
     def _parse_sections(
         self,
         path: str,