mlsec 2.0.0__tar.gz

mlsec-2.0.0/.github/workflows/ci.yml

@@ -0,0 +1,55 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+    branches: [main]
+
+permissions:
+  contents: read
+
+jobs:
+  lint:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+      - run: pip install ruff
+      - run: ruff check .
+      - run: ruff format --check .
+
+  test:
+    runs-on: ${{ matrix.os }}
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [ubuntu-latest, macos-latest]
+        python-version: ["3.10", "3.11", "3.12"]
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+      - name: Install dependencies
+        run: pip install -e ".[dev,all]"
+      - name: Run tests
+        run: python -m pytest tests/ --ignore=tests/test_model_inspection.py -v --tb=short --cov=. --cov-report=xml
+      - name: Upload coverage
+        if: matrix.python-version == '3.12' && matrix.os == 'ubuntu-latest'
+        uses: codecov/codecov-action@v4
+        with:
+          file: coverage.xml
+          fail_ci_if_error: false
+
+  typecheck:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+      - run: pip install -e ".[dev,all]"
+      - run: mypy src/mlsec/ --ignore-missing-imports

mlsec-2.0.0/.github/workflows/publish.yml

@@ -0,0 +1,24 @@
+name: Publish to PyPI
+
+on:
+  release:
+    types: [published]
+
+permissions:
+  id-token: write
+
+jobs:
+  publish:
+    runs-on: ubuntu-latest
+    environment: pypi
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+      - name: Install build tools
+        run: pip install build
+      - name: Build package
+        run: python -m build
+      - name: Publish to PyPI
+        uses: pypa/gh-action-pypi-publish@release/v1

mlsec-2.0.0/.gitignore

@@ -0,0 +1,50 @@
+# Python
+__pycache__/
+*.py[cod]
+*$py.class
+*.so
+.Python
+*.egg-info/
+dist/
+build/
+
+# Virtual environments
+venv/
+env/
+ENV/
+.venv
+
+# IDEs
+.vscode/
+.idea/
+*.swp
+*.swo
+*~
+
+# OS
+.DS_Store
+Thumbs.db
+
+# Test artifacts
+.coverage
+htmlcov/
+.pytest_cache/
+.mypy_cache/
+.ruff_cache/
+
+# Content assets
+linkedin-article.md
+linkedin-post.md
+mlsecops-gap-header.png
+mlsec-toolkit-overview.png
+
+# AI tool artifacts
+.claude/
+CLAUDE.md
+claude.md
+planning.md
+status.md
+notes.md
+todo.md
+*.chat
+*.conversation

mlsec-2.0.0/CHANGELOG.md

@@ -0,0 +1,39 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [2.0.0] - 2026-03-30
+
+### Added
+- Proper Python package structure (`pip install mlsec`)
+- Unified CLI entry point (`mlsec <tool>`) with subcommands and aliases
+- `pyproject.toml` with optional dependency groups (`vision`, `transformers`, `onnx`, `safetensors`, `all`, `dev`)
+- Comprehensive test suite (289 tests, 79% coverage)
+- GitHub Actions CI pipeline (lint, test matrix, type checking)
+- All tools now accept `argv` parameter for programmatic use
+
+### Fixed
+- **PGD attack gradient crash** — `adv.grad.zero_()` on `None` after tensor reassignment in projection step
+- **Checkpoint NaN masking** — `tensor.abs().max()` returned NaN when tensor contained NaN, masking extreme finite values. Now uses finite-only filtering.
+- **Checkpoint histogram crash** — `torch.histc` crashed on tensors containing NaN/Inf values. Now filters non-finite values before computing histograms.
+- **Dead code in `extract_state_dict`** — `"state_dict"` key check was unreachable because generic dict check matched first. Reordered to check `"state_dict"` key before generic dict extraction.
+- **Triton config quote stripping** — Parameter keys from pbtxt retained surrounding quotes, causing auth/guard/redact pattern matching to fail.
+- **Triton auth check** — Changed from exact match to substring match for auth parameter detection (consistent with guard and redact checks).
+
+### Changed
+- Restructured from 6 loose scripts to `src/mlsec/` package layout
+- Original standalone scripts preserved at root for backward compatibility
+
+## [1.0.0] - 2025-10-30
+
+### Added
+- Initial release of ML security tools
+- Six specialized security analysis scripts
+- Support for PyTorch, TensorRT, and Triton models
+- Model inspection and auditing capabilities
+
+[2.0.0]: https://github.com/scthornton/ml-security-tools/compare/v1.0.0...v2.0.0
+[1.0.0]: https://github.com/scthornton/ml-security-tools/releases/tag/v1.0.0

mlsec-2.0.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Scott Thornton
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

mlsec-2.0.0/PKG-INFO

@@ -0,0 +1,261 @@
+Metadata-Version: 2.4
+Name: mlsec
+Version: 2.0.0
+Summary: Security analysis toolkit for machine learning models and infrastructure
+Project-URL: Homepage, https://github.com/scthornton/ml-security-tools
+Project-URL: Documentation, https://github.com/scthornton/ml-security-tools#readme
+Project-URL: Repository, https://github.com/scthornton/ml-security-tools
+Project-URL: Issues, https://github.com/scthornton/ml-security-tools/issues
+Author-email: Scott Thornton <scott@perfecxion.ai>
+License-Expression: MIT
+License-File: LICENSE
+Keywords: adversarial,ai-safety,machine-learning,mlops,model-security,security
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Security
+Classifier: Typing :: Typed
+Requires-Python: >=3.10
+Requires-Dist: torch>=2.0
+Provides-Extra: all
+Requires-Dist: onnx>=1.14; extra == 'all'
+Requires-Dist: onnxruntime>=1.15; extra == 'all'
+Requires-Dist: safetensors>=0.3; extra == 'all'
+Requires-Dist: torchvision>=0.15; extra == 'all'
+Requires-Dist: transformers>=4.30; extra == 'all'
+Provides-Extra: dev
+Requires-Dist: mypy>=1.8; extra == 'dev'
+Requires-Dist: pytest-cov>=4.0; extra == 'dev'
+Requires-Dist: pytest>=7.0; extra == 'dev'
+Requires-Dist: ruff>=0.4; extra == 'dev'
+Provides-Extra: onnx
+Requires-Dist: onnx>=1.14; extra == 'onnx'
+Requires-Dist: onnxruntime>=1.15; extra == 'onnx'
+Provides-Extra: safetensors
+Requires-Dist: safetensors>=0.3; extra == 'safetensors'
+Provides-Extra: transformers
+Requires-Dist: transformers>=4.30; extra == 'transformers'
+Provides-Extra: vision
+Requires-Dist: torchvision>=0.15; extra == 'vision'
+Description-Content-Type: text/markdown
+
+# mlsec — ML Security Toolkit
+
+[![CI](https://github.com/scthornton/ml-security-tools/actions/workflows/ci.yml/badge.svg)](https://github.com/scthornton/ml-security-tools/actions/workflows/ci.yml)
+[![Python 3.10+](https://img.shields.io/badge/python-3.10%2B-blue.svg)](https://python.org)
+[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
+
+Security analysis toolkit for machine learning models and infrastructure. Covers the full ML lifecycle — from adversarial robustness testing through training-time poisoning detection to deployment hardening.
+
+```
+pip install mlsec          # core (torch only)
+pip install mlsec[all]     # all optional dependencies
+```
+
+## Why mlsec?
+
+Most ML security tools focus on a single attack surface. **mlsec covers six critical areas in one toolkit**, designed to integrate into existing MLOps pipelines:
+
+| Tool | What it catches | When to use it |
+|------|----------------|----------------|
+| `mlsec adversarial` | Robustness regressions from FGSM, PGD, CW attacks | Before every model release — CI gate |
+| `mlsec inspect` | Suspicious weights, activation anomalies, FGSM vulnerability | Quick triage of new/third-party models |
+| `mlsec poison` | Gradient divergence in distributed training (DDP) | During training — live or post-hoc |
+| `mlsec export-guard` | ONNX graph tampering, supply-chain integrity | Before deploying optimized models |
+| `mlsec checkpoint` | NaN/Inf injection, weight magnitude anomalies, backdoor fingerprints | Before loading any checkpoint |
+| `mlsec triton` | Missing rate limits, auth gaps, DoS vectors | Before deploying inference servers |
+
+## Quick Start
+
+### Adversarial Robustness Testing
+
+Track robustness drift across model versions with baseline regression:
+
+```bash
+mlsec adversarial \
+  --model-script models.py \
+  --factory create_resnet \
+  --input-shape 1,3,224,224 \
+  --num-classes 1000 \
+  --attacks fgsm pgd cw \
+  --epsilon 0.01 \
+  --baseline-file baselines.json \
+  --update-baseline
+```
+
+### Checkpoint Security Triage
+
+Scan checkpoints for anomalies, generate fingerprints, convert to safe formats:
+
+```bash
+mlsec checkpoint /path/to/checkpoints/ \
+  --convert-safetensors \
+  --write-fingerprint fingerprints/ \
+  --reference-fingerprint reference.json \
+  --json
+```
+
+### Distributed Poisoning Detection
+
+Monitor gradient divergence during distributed training:
+
+```bash
+# Generate demo data
+mlsec poison simulate --log-dir logs/ --workers 8 --steps 500
+
+# Analyze for poisoning signals
+mlsec poison monitor --log-dir logs/ --threshold 3.0
+
+# Live monitoring via UDP broadcast
+mlsec poison listen --port 5454 --expected-workers 8
+```
+
+### ONNX/TensorRT Export Validation
+
+Validate the full export pipeline with provenance tracking:
+
+```bash
+mlsec export-guard \
+  --model-script models.py \
+  --input-shape 1,3,224,224 \
+  --enable-onnxruntime \
+  --build-engine \
+  --hash-record attestation.json
+```
+
+### Triton Server Hardening
+
+Audit inference server configs for security gaps:
+
+```bash
+mlsec triton models/**/config.pbtxt --summary
+```
+
+### Model Inspection
+
+Quick security triage of HuggingFace models:
+
+```bash
+mlsec inspect --allow-downloads
+```
+
+## Architecture
+
+```
+mlsec/
+├── adversarial    — FGSM, PGD, CW attacks with baseline regression tracking
+├── inspect        — Weight anomaly detection + activation monitoring hooks
+├── poison         — Gradient snapshotter + CUSUM change-point detection
+├── export-guard   — ONNX lint + SHA-256 provenance chain + trtexec integration
+├── checkpoint     — KL-divergence fingerprinting + safetensors conversion
+└── triton         — Heuristic config.pbtxt parser + security rule engine
+```
+
+Each tool is **standalone** (no cross-dependencies) and works both as a CLI command and as a Python library:
+
+```python
+from mlsec.tools.checkpoint_triage import inspect_state_dict, compute_fingerprint
+
+anomalies = inspect_state_dict(model.state_dict(), threshold=100.0)
+fingerprint = compute_fingerprint(model.state_dict())
+```
+
+## Installation
+
+```bash
+# Minimal (PyTorch only)
+pip install mlsec
+
+# With specific extras
+pip install mlsec[vision]        # + torchvision
+pip install mlsec[transformers]  # + HuggingFace transformers
+pip install mlsec[onnx]          # + onnx + onnxruntime
+pip install mlsec[safetensors]   # + safetensors format support
+pip install mlsec[all]           # everything
+
+# Development
+pip install -e ".[dev,all]"
+pytest tests/ -v
+```
+
+## CI/CD Integration
+
+mlsec is designed to run in automated pipelines. Every tool returns structured exit codes:
+
+| Exit Code | Meaning |
+|-----------|---------|
+| `0` | Clean — no anomalies detected |
+| `1` | Error — invalid input or missing files |
+| `2` | Alert — anomalies detected (investigate) |
+
+Example GitHub Actions step:
+
+```yaml
+- name: Adversarial regression gate
+  run: |
+    mlsec adversarial \
+      --model-script models.py \
+      --factory create_model \
+      --input-shape 1,3,224,224 \
+      --num-classes 1000 \
+      --attacks fgsm pgd \
+      --baseline-file baselines.json
+```
+
+JSON output is available for checkpoint triage (`--json` flag) for machine-readable results.
+
+## Key Detection Capabilities
+
+### Adversarial Attacks
+- **FGSM** — Fast Gradient Sign Method (single-step)
+- **PGD** — Projected Gradient Descent (multi-step with random restarts)
+- **CW** — Carlini-Wagner L-inf (optimization-based)
+- Mixed-precision testing (float32, float16, bfloat16)
+- Baseline tracking with configurable regression thresholds
+
+### Poisoning Detection
+- Per-worker gradient L2/L-inf norm monitoring
+- Cross-worker divergence ratio computation
+- **CUSUM** change-point detection for slow-burn poisoning
+- Live UDP broadcast aggregation for real-time alerts
+
+### Supply Chain Security
+- SHA-256 provenance hashing at every pipeline stage
+- ONNX graph linting (custom domains, control-flow ops, large constants, embedded subgraphs)
+- KL-divergence fingerprinting for checkpoint drift detection
+- Automatic safetensors conversion for pickle-free distribution
+
+### Deployment Hardening
+- Triton config auditing without protobuf dependency
+- Checks for: rate limiting, dynamic batching, auth controls, input bounds, logging redaction
+- ONNX Runtime numerical drift comparison
+
+## Responsible Use
+
+These tools are for **authorized security testing only**. See [SECURITY.md](SECURITY.md) for the full responsible use policy.
+
+## Contributing
+
+```bash
+git clone https://github.com/scthornton/ml-security-tools.git
+cd ml-security-tools
+pip install -e ".[dev,all]"
+pytest tests/ -v
+ruff check .
+```
+
+## License
+
+MIT — see [LICENSE](LICENSE).
+
+## Contact
+
+- **Email:** scott@perfecxion.ai
+- **Security issues:** See [SECURITY.md](SECURITY.md)

mlsec-2.0.0/README.md

@@ -0,0 +1,213 @@
+# mlsec — ML Security Toolkit
+
+[![CI](https://github.com/scthornton/ml-security-tools/actions/workflows/ci.yml/badge.svg)](https://github.com/scthornton/ml-security-tools/actions/workflows/ci.yml)
+[![Python 3.10+](https://img.shields.io/badge/python-3.10%2B-blue.svg)](https://python.org)
+[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
+
+Security analysis toolkit for machine learning models and infrastructure. Covers the full ML lifecycle — from adversarial robustness testing through training-time poisoning detection to deployment hardening.
+
+```
+pip install mlsec          # core (torch only)
+pip install mlsec[all]     # all optional dependencies
+```
+
+## Why mlsec?
+
+Most ML security tools focus on a single attack surface. **mlsec covers six critical areas in one toolkit**, designed to integrate into existing MLOps pipelines:
+
+| Tool | What it catches | When to use it |
+|------|----------------|----------------|
+| `mlsec adversarial` | Robustness regressions from FGSM, PGD, CW attacks | Before every model release — CI gate |
+| `mlsec inspect` | Suspicious weights, activation anomalies, FGSM vulnerability | Quick triage of new/third-party models |
+| `mlsec poison` | Gradient divergence in distributed training (DDP) | During training — live or post-hoc |
+| `mlsec export-guard` | ONNX graph tampering, supply-chain integrity | Before deploying optimized models |
+| `mlsec checkpoint` | NaN/Inf injection, weight magnitude anomalies, backdoor fingerprints | Before loading any checkpoint |
+| `mlsec triton` | Missing rate limits, auth gaps, DoS vectors | Before deploying inference servers |
+
+## Quick Start
+
+### Adversarial Robustness Testing
+
+Track robustness drift across model versions with baseline regression:
+
+```bash
+mlsec adversarial \
+  --model-script models.py \
+  --factory create_resnet \
+  --input-shape 1,3,224,224 \
+  --num-classes 1000 \
+  --attacks fgsm pgd cw \
+  --epsilon 0.01 \
+  --baseline-file baselines.json \
+  --update-baseline
+```
+
+### Checkpoint Security Triage
+
+Scan checkpoints for anomalies, generate fingerprints, convert to safe formats:
+
+```bash
+mlsec checkpoint /path/to/checkpoints/ \
+  --convert-safetensors \
+  --write-fingerprint fingerprints/ \
+  --reference-fingerprint reference.json \
+  --json
+```
+
+### Distributed Poisoning Detection
+
+Monitor gradient divergence during distributed training:
+
+```bash
+# Generate demo data
+mlsec poison simulate --log-dir logs/ --workers 8 --steps 500
+
+# Analyze for poisoning signals
+mlsec poison monitor --log-dir logs/ --threshold 3.0
+
+# Live monitoring via UDP broadcast
+mlsec poison listen --port 5454 --expected-workers 8
+```
+
+### ONNX/TensorRT Export Validation
+
+Validate the full export pipeline with provenance tracking:
+
+```bash
+mlsec export-guard \
+  --model-script models.py \
+  --input-shape 1,3,224,224 \
+  --enable-onnxruntime \
+  --build-engine \
+  --hash-record attestation.json
+```
+
+### Triton Server Hardening
+
+Audit inference server configs for security gaps:
+
+```bash
+mlsec triton models/**/config.pbtxt --summary
+```
+
+### Model Inspection
+
+Quick security triage of HuggingFace models:
+
+```bash
+mlsec inspect --allow-downloads
+```
+
+## Architecture
+
+```
+mlsec/
+├── adversarial    — FGSM, PGD, CW attacks with baseline regression tracking
+├── inspect        — Weight anomaly detection + activation monitoring hooks
+├── poison         — Gradient snapshotter + CUSUM change-point detection
+├── export-guard   — ONNX lint + SHA-256 provenance chain + trtexec integration
+├── checkpoint     — KL-divergence fingerprinting + safetensors conversion
+└── triton         — Heuristic config.pbtxt parser + security rule engine
+```
+
+Each tool is **standalone** (no cross-dependencies) and works both as a CLI command and as a Python library:
+
+```python
+from mlsec.tools.checkpoint_triage import inspect_state_dict, compute_fingerprint
+
+anomalies = inspect_state_dict(model.state_dict(), threshold=100.0)
+fingerprint = compute_fingerprint(model.state_dict())
+```
+
+## Installation
+
+```bash
+# Minimal (PyTorch only)
+pip install mlsec
+
+# With specific extras
+pip install mlsec[vision]        # + torchvision
+pip install mlsec[transformers]  # + HuggingFace transformers
+pip install mlsec[onnx]          # + onnx + onnxruntime
+pip install mlsec[safetensors]   # + safetensors format support
+pip install mlsec[all]           # everything
+
+# Development
+pip install -e ".[dev,all]"
+pytest tests/ -v
+```
+
+## CI/CD Integration
+
+mlsec is designed to run in automated pipelines. Every tool returns structured exit codes:
+
+| Exit Code | Meaning |
+|-----------|---------|
+| `0` | Clean — no anomalies detected |
+| `1` | Error — invalid input or missing files |
+| `2` | Alert — anomalies detected (investigate) |
+
+Example GitHub Actions step:
+
+```yaml
+- name: Adversarial regression gate
+  run: |
+    mlsec adversarial \
+      --model-script models.py \
+      --factory create_model \
+      --input-shape 1,3,224,224 \
+      --num-classes 1000 \
+      --attacks fgsm pgd \
+      --baseline-file baselines.json
+```
+
+JSON output is available for checkpoint triage (`--json` flag) for machine-readable results.
+
+## Key Detection Capabilities
+
+### Adversarial Attacks
+- **FGSM** — Fast Gradient Sign Method (single-step)
+- **PGD** — Projected Gradient Descent (multi-step with random restarts)
+- **CW** — Carlini-Wagner L-inf (optimization-based)
+- Mixed-precision testing (float32, float16, bfloat16)
+- Baseline tracking with configurable regression thresholds
+
+### Poisoning Detection
+- Per-worker gradient L2/L-inf norm monitoring
+- Cross-worker divergence ratio computation
+- **CUSUM** change-point detection for slow-burn poisoning
+- Live UDP broadcast aggregation for real-time alerts
+
+### Supply Chain Security
+- SHA-256 provenance hashing at every pipeline stage
+- ONNX graph linting (custom domains, control-flow ops, large constants, embedded subgraphs)
+- KL-divergence fingerprinting for checkpoint drift detection
+- Automatic safetensors conversion for pickle-free distribution
+
+### Deployment Hardening
+- Triton config auditing without protobuf dependency
+- Checks for: rate limiting, dynamic batching, auth controls, input bounds, logging redaction
+- ONNX Runtime numerical drift comparison
+
+## Responsible Use
+
+These tools are for **authorized security testing only**. See [SECURITY.md](SECURITY.md) for the full responsible use policy.
+
+## Contributing
+
+```bash
+git clone https://github.com/scthornton/ml-security-tools.git
+cd ml-security-tools
+pip install -e ".[dev,all]"
+pytest tests/ -v
+ruff check .
+```
+
+## License
+
+MIT — see [LICENSE](LICENSE).
+
+## Contact
+
+- **Email:** scott@perfecxion.ai
+- **Security issues:** See [SECURITY.md](SECURITY.md)