flawed 0.0.1__tar.gz

flawed-0.0.1/.gitignore

@@ -0,0 +1,12 @@
+__pycache__/
+*.pyc
+*.pyo
+*.egg-info/
+dist/
+build/
+.mypy_cache/
+.ruff_cache/
+.pytest_cache/
+*.egg
+.venv/
+venv/

flawed-0.0.1/PKG-INFO

@@ -0,0 +1,4 @@
+Metadata-Version: 2.4
+Name: flawed
+Version: 0.0.1
+Requires-Python: >=3.11

flawed-0.0.1/README.md

@@ -0,0 +1,25 @@
+# flawed
+
+Static analysis engine for confusion vulnerability research in Python web applications.
+
+```python
+from flawed import open_repo, detector
+from flawed.inputs import Query, Form, Json
+from flawed.effects import Mutation, Session
+from flawed.checks import Crypto, Token
+from flawed.route import Route, POST, accepting
+```
+
+## Status
+
+Early development. API contracts are defined; engine runtime is not yet implemented.
+
+## Structure
+
+- `src/flawed/` — Rule API (Layer 3) — the primary user-facing surface
+- `src/flawed/semantic/` — Semantic Layer (Layer 2) — framework interpretation (planned)
+- `src/flawed/index/` — Code Index (Layer 1) — structural extraction (planned)
+
+## Design
+
+See `docs/` for architecture and design documents.

flawed-0.0.1/docs/architecture-overview.md

@@ -0,0 +1,131 @@
+# Confusion Analysis Engine
+
+Specification for the standalone analysis engine for researching confusion
+vulnerabilities in Python web applications.
+
+## Problem Statement
+
+> Given a local repository snapshot, compile source code into stable
+> structural facts, interpretable framework-specific domain objects, and
+> composable detection-rule primitives for confusion vulnerability research.
+
+## Architecture
+
+Three layers. Each owns a distinct concern. Dependencies flow in one
+direction: Layer 3 -> Layer 2 -> Layer 1. No skipping, no reverse.
+
+```
+  repository snapshot
+        |
+        v
+  Layer 1: Code Index
+    Python-generic structural extraction.
+    Runs once per repo snapshot. Persists artifacts.
+    Exposes: functions, classes, call graph, CFGs, value-flow,
+             symbols, decorators, imports, class hierarchy.
+        |
+        |  CodeIndex API (frozen Python objects)
+        v
+  Layer 2: Semantic Layer
+    Framework/library/convention interpretation.
+    Runs per detection. Extensible by users.
+    Exposes: routes, request reads, gates, effects,
+             lifecycle hooks, on-demand flow traces.
+        |
+        |  WebApp API (frozen domain objects)
+        v
+  Layer 3: Rule API
+    Typed domain navigation, scoped queries,
+    evidence building, detector framework, dev tools.
+    Consumers: rule authors, notebooks, batch detection.
+```
+
+## Directory Structure
+
+```
+L6/
+  README.md                  <- You are here
+  principles.md              <- Vision, non-negotiable principles, API design principles
+  architecture.md            <- Three-layer overview, data flow, execution model
+  boundaries.md              <- Normative boundary spec: what belongs where
+
+  API-L1-Code-Index/         <- Layer 1: Python-generic structural extraction
+    README.md                  Extraction pipeline, CodeIndex API, data types
+
+  API-L2-Semantic-Layer/     <- Layer 2: Framework/library interpretation
+    README.md                  Interpreter architecture, WebApp API, flow tracing
+
+  API-L3-Rule-API/           <- Layer 3: Rule authoring and exploration surface
+    README.md                  Domain objects, queries, evidence, detector framework
+```
+
+## Reading Order
+
+### For rule authors (start here)
+
+1. `principles.md` sections 1-3 -- vision and API design principles
+2. `API-L3-Rule-API/README.md` -- domain objects, queries, examples
+3. Start writing rules
+
+### For framework adapter authors
+
+1. `principles.md` -- non-negotiable design principles
+2. `boundaries.md` -- what belongs in which layer
+3. `API-L2-Semantic-Layer/README.md` -- interpreter protocol, extension model
+
+### For pipeline developers
+
+1. `principles.md` -- full document
+2. `architecture.md` -- three-layer overview, data flow, execution model
+3. `boundaries.md` -- normative boundary spec
+4. `API-L1-Code-Index/README.md` -- extraction pipeline, CodeIndex API
+5. `API-L2-Semantic-Layer/README.md` -- interpreter architecture, WebApp API
+6. `API-L3-Rule-API/README.md` -- rule authoring surface
+
+## Key Design Decisions
+
+1. **Framework knowledge is never in the run-once pipeline.** The Code
+   Index knows Python the language. Flask, SQLAlchemy, marshmallow, and
+   every other framework live exclusively in the Semantic Layer.
+
+2. **Extensibility without re-extraction.** Adding a new framework,
+   fixing a route-detection pattern, or recognizing a new ORM call
+   requires zero re-extraction. User extensions have full parity with
+   built-in defaults.
+
+3. **On-demand flow tracing replaces pre-computed taint.** Semgrep's
+   taint signatures are removed from the internal pipeline. Data-flow
+   tracing is a runtime Semantic Layer service that accepts arbitrary
+   sources, not just `request`.
+
+4. **Decorators split: syntax in Layer 1, meaning in Layer 2.** The Code
+   Index records decorator name, arguments, and location. The Semantic
+   Layer decides whether a decorator is an auth gate, a route
+   registration, or a CSRF exemption.
+
+5. **Strict unidirectional layering.** Layer 3 -> Layer 2 -> Layer 1.
+   No skipping. No reverse dependencies. No exceptions.
+
+## Supersession
+
+This directory supersedes `wip/round2/L5/confusion-analysis/`.
+
+What changed from L5 to L6:
+
+| L5 | L6 | Why |
+|----|-----|-----|
+| 7-stage monolithic pipeline | 3-layer architecture | Clear responsibility boundaries, strict layering |
+| Flask-specific extraction in run-once pipeline | All framework logic in Semantic Layer | Extensibility without re-extraction |
+| No middle layer | Semantic Layer (Layer 2) | Framework interpretation must be dynamic and extensible |
+| Pre-computed taint via Semgrep rule | On-demand flow tracing at Layer 2 | Arbitrary sources, not just `request`; no Semgrep dependency |
+| Extension via model libraries only | Full interpreter replacement/extension | First-class framework adapter protocol |
+| Mixed generic/Flask components per stage | Pure generic pipeline + pure framework layer | Each layer testable in isolation |
+
+## Relationship to Other Documentation
+
+| Document | Role |
+|----------|------|
+| `wip/round2/L5/confusion-analysis/` | Previous iteration (superseded by this directory) |
+| `wip/round2/L4/confusion-analysis-pipeline.md` | Historical: original monolithic spec |
+| `docs/analysis-pipeline/` | Empirical research: Semgrep capabilities, tool evaluations |
+| `.claude/scratch/semgrep-pro-investigation/` | Research artifacts: extraction outputs, wave test results |

flawed-0.0.1/docs/architecture.md

@@ -0,0 +1,330 @@
+# Architecture
+
+Three layers. Each owns a distinct concern. Dependencies flow in one
+direction: Layer 3 → Layer 2 → Layer 1. No skipping, no reverse references.
+
+```
+  repository snapshot
+        │
+        ▼
+  ┌─────────────────────────────────────────────┐
+  │  Layer 1: Code Index                        │
+  │  Python-generic structural extraction.      │
+  │  Runs once per repo snapshot.               │
+  │  Persists artifacts to disk.                │
+  │  Exposes a typed Python API over the        │
+  │  pre-computed code structure.               │
+  │                                             │
+  │  Produces: AST entities, call graph, CFGs,  │
+  │  value-flow, symbols, class hierarchy,      │
+  │  imports, decorators, attribute accesses.    │
+  └────────────────────┬────────────────────────┘
+                       │  CodeIndex API (Python objects, frozen)
+                       ▼
+  ┌─────────────────────────────────────────────┐
+  │  Layer 2: Semantic Layer                    │
+  │  Framework/library/convention-specific      │
+  │  interpretation. Runs per detection.        │
+  │  Extensible by rule authors.               │
+  │                                             │
+  │  Produces: Routes, RequestReads, Gates,     │
+  │  Effects, Lifecycle, method branches,       │
+  │  classified mutations, enhanced dispatch,   │
+  │  on-demand data-flow traces.               │
+  └────────────────────┬────────────────────────┘
+                       │  WebApp API (domain objects, frozen)
+                       ▼
+  ┌─────────────────────────────────────────────┐
+  │  Layer 3: Rule API                          │
+  │  Typed domain navigation, scoped queries,   │
+  │  collection API, evidence building,         │
+  │  detector framework, dev tools.             │
+  │                                             │
+  │  Consumers: rule authors, notebooks,        │
+  │  interactive exploration, batch detection.  │
+  └─────────────────────────────────────────────┘
+```
+
+---
+
+## Layer 1: Code Index
+
+### Responsibility
+
+Given a local Python repository snapshot, extract structural facts that are
+true of the code itself, independent of any web framework, library, or
+coding convention. Store those facts. Expose them through a typed Python API
+that hides file layout, serialization format, and internal ID schemes.
+
+### What it produces
+
+| Fact kind | Source | Notes |
+|-----------|--------|-------|
+| Functions | LibCST + astroid | name, FQN, params, location, kind (top-level/method/nested/lambda) |
+| Classes | LibCST + astroid | name, FQN, bases, location, MRO, hierarchy |
+| Imports | LibCST + Semgrep symbols | resolved names, aliases, re-exports |
+| Decorators | LibCST + astroid | name, resolved FQN, args, target function, location — no semantic interpretation |
+| Call edges | Semgrep call graph + AST | merged cross-file, per-file, with resolution status; includes unresolved dispatch patterns (getattr, dict dispatch) as edges |
+| CFGs | LibCST + NetworkX | per-function blocks, edges, dominance, branch structure |
+| Value-flow edges | LibCST + astroid + Semgrep svalues | assignments, arguments, returns, aliases, unpacking |
+| Symbol resolution | LibCST + astroid + Semgrep symbols | FQNs, resolved names, import chains |
+| Class hierarchy | LibCST + astroid | MRO, base classes, method resolution — stored on Class objects |
+| Attribute accesses | LibCST + astroid | reads and writes on any object; includes dict operations, list/set mutations, del statements |
+| Source locations | all passes | file, line, column for every fact |
+| Errors and gaps | all passes | per-file parse failures, per-function CFG failures, unresolved symbols |
+
+### What it does NOT produce
+
+- Routes or endpoint registrations.
+- Request input classifications (`Query`, `Form`, `Json`).
+- Security gate or effect observations.
+- Lifecycle hooks or framework state access.
+- Taint summaries or request-data flow traces.
+- Semantic tags (`auth_gate`, `db_write`, `lifecycle_state`).
+- Anything that requires knowing which framework the code uses.
+
+### API shape
+
+Layer 1 exposes a `CodeIndex` object. All access goes through it. No
+consumer reads JSONL files, raw Semgrep output, or internal indexes
+directly.
+
+Built on mandatory dependencies: LibCST (parsing, scope analysis), astroid
+(inference, MRO), Semgrep Pro (cross-file call graph, symbols), basedpyright
+(type oracle), and NetworkX (graph algorithms).
+
+```python
+idx = CodeIndex.open("/path/to/analysis-store/repos/slug/snapshot")
+
+idx.functions          # FunctionCollection — filter, iterate, lookup by FQN
+idx.classes            # ClassCollection — includes hierarchy (MRO, subclasses)
+idx.imports            # ImportCollection
+idx.decorators         # DecoratorCollection — syntactic facts with resolved FQNs
+idx.call_graph         # CallGraph — nodes, edges, reachability; includes
+                       #   unresolved dispatch edges (getattr, dict dispatch)
+idx.cfg(fn_fqn)        # per-function CFG with dominance
+idx.value_flow         # ValueFlowGraph — assignment chains, argument binding
+idx.symbols            # SymbolIndex — FQN resolution, import chains
+idx.attributes         # AttributeAccessCollection — reads/writes/mutations on any object
+idx.source(location)   # source text for a location
+idx.errors             # extraction errors and gaps
+```
+
+Every object returned is frozen. Layer 2 cannot mutate Layer 1 data.
+
+### Execution
+
+Runs once per repository snapshot via CLI. Produces a deterministic artifact
+tree in the analysis store. Idempotent: re-running with the same source and
+tool versions produces identical output.
+
+---
+
+## Layer 2: Semantic Layer
+
+### Responsibility
+
+Interpret Layer 1's Python-structural facts through the lens of a specific
+web framework, set of libraries, and project conventions. Produce typed
+domain objects that encode what the code *means* in a web application
+security context.
+
+This is where all framework-specific knowledge lives. Flask route
+registration patterns, request container mappings, ORM write detection,
+session and context state identification, lifecycle hooks — all of it.
+
+### What it produces
+
+| Domain concept | Built from (Layer 1 facts) | Framework knowledge required |
+|----------------|---------------------------|------------------------------|
+| Routes | decorators + functions + class hierarchy | Route registration patterns (`@app.route`, MethodView, etc.) |
+| Request reads | call sites + symbol resolution + attribute accesses | Request container API (`request.args.get`, marshmallow schema, etc.) |
+| Context accesses | attribute accesses + import resolution | Context proxies (`g`, `session`, `current_user`) |
+| Gates | decorators + call sites + conditionals | Auth decorator patterns, permission check functions |
+| Effects | call sites + attribute writes + symbol resolution | ORM write patterns, external request patterns |
+| Lifecycle hooks | decorators + functions + import resolution | Hook registration (`before_request`, `after_request`) |
+| Method branches | CFG blocks + conditionals + value-flow | Method dispatch patterns (`request.method == "POST"`) |
+| Classified mutations | attribute accesses + symbol resolution | Framework-specific state mutation patterns |
+| Enhanced dispatch | call graph (unresolved edges) + class hierarchy | Framework-specific dispatch (MethodView, signals) |
+| On-demand flow traces | call graph + value-flow + CFGs | Source/sink definitions, propagation rules |
+
+### Extension model
+
+Layer 2 is composed of **interpreters** — ordinary Python modules that
+encode reusable framework/library knowledge:
+
+- **Route interpreters**: recognize registration patterns, produce Route objects.
+- **Input interpreters**: map code expressions to typed request-read observations.
+- **Gate interpreters**: classify decorators, calls, conditionals as security gates.
+- **Effect interpreters**: classify calls and writes as security-relevant effects.
+- **State interpreters**: identify framework-specific context access patterns.
+- **Flow interpreters**: define source/sink/propagator rules for data-flow tracing.
+
+Users extend or replace any interpreter. A project using Flask-RESTful adds
+an interpreter that recognizes `Resource` subclasses as route handlers. A
+project with custom auth adds an interpreter that recognizes its
+`@requires_permission` decorator as a gate. These extensions have full
+parity with the built-in Flask interpreters — the built-ins are just
+default interpreter modules, not privileged internal code.
+
+### API shape
+
+Layer 2 exposes a `WebApp` object built from a `CodeIndex` plus a set of
+active interpreters:
+
+```python
+app = WebApp.from_index(idx, interpreters=flask_defaults())
+
+app.routes             # RouteCollection — framework-specific
+app.request_reads      # RequestReadCollection
+app.gates              # GateCollection
+app.effects            # EffectCollection
+app.hooks              # HookCollection
+app.context_accesses   # ContextAccessCollection
+app.classified_mutations  # ClassifiedMutationCollection
+
+app.trace_flow(source, sink)  # on-demand data-flow traversal
+app.reachable_from(function)  # transitive closure with framework edges
+app.enhanced_call_graph       # call graph with framework-injected edges
+```
+
+Every object returned is frozen. Layer 3 cannot mutate Layer 2 data.
+
+### Execution
+
+Runs per detection invocation. Layer 2 code executes each time a rule runs,
+so interpreter changes take effect immediately without re-running the
+internal pipeline.
+
+---
+
+## Layer 3: Rule API
+
+### Responsibility
+
+Provide rule authors with typed, composable, notebook-friendly tools for
+navigating analyzed codebases, expressing security invariants, building
+evidence chains, and producing vulnerability candidates.
+
+### What it provides
+
+- **Domain navigation**: `Route`, `Function`, `CodeScope` with fluent queries.
+- **Scoped queries**: `.reads()`, `.effects()`, `.gates()`, `.calls()` within
+  a code scope (route body, reachable code, function body).
+- **Control flow**: `.cfg.dominates()`, `.cfg.precedes()`, `.cfg.ordered()`.
+- **Value flow**: `.value.flows_to()`, `.value_bindings.to()`.
+- **Collections**: filtering, grouping, chaining over domain objects.
+- **Evidence building**: `CandidateBuilder` with evidence, missing evidence,
+  analysis gaps, severity, tags.
+- **Detector framework**: `@detector`, `@invariant`, `@for_routes`, config.
+- **Dev tools**: `trace_detector()`, `dry_run()`, `explain()`.
+
+### Consumers
+
+- Security researchers writing detection rules.
+- Notebook-driven interactive code exploration.
+- Batch detection across a repository corpus.
+- Hypothesis testing and rule prototyping.
+
+Layer 3 consumers never import Layer 1 types, never read raw files, and
+never need to understand static analysis internals.
+
+---
+
+## Data Flow
+
+```
+Source files
+    │
+    ▼
+Layer 1: Semgrep extraction → normalization → AST passes → merge → index
+    │
+    │  Persisted artifacts: analysis-store/repos/<slug>/<snapshot>/
+    │  - raw/semgrep/        (call graph, symbols, svalues)
+    │  - normalized/         (entities, calls, cfgs, hierarchy, symbols,
+    │                         value_flow, attributes, decorators)
+    │  - projections/        (merged_call_graph, indexes)
+    │  - manifest.json       (version hashes, step status)
+    │  - errors.jsonl        (extraction failures, gaps)
+    │
+    ▼
+Layer 2: Framework interpreters → domain construction → flow tracing
+    │
+    │  Runtime objects (not persisted by default):
+    │  - Route, RequestRead, Gate, Effect, Hook, ContextAccess
+    │  - Enhanced call graph (with framework-specific edges)
+    │  - Classified mutations, method branches
+    │  - On-demand flow traces
+    │
+    ▼
+Layer 3: Rule execution → candidate emission
+    │
+    │  Output: candidates.jsonl, reports, evidence packs
+    │
+    ▼
+Triage and reporting
+```
+
+---
+
+## Execution Model
+
+| Concern | When it runs | Persisted? | Invalidated by |
+|---------|-------------|------------|----------------|
+| Layer 1 extraction | Once per repo snapshot | Yes — artifact tree | Source change, tool version change |
+| Layer 2 interpretation | Per detection run | No (optionally cached) | Interpreter change, Layer 1 change |
+| Layer 3 detection | Per rule invocation | Candidates persisted | Rule change, config change, Layer 2 change |
+
+Layer 1 is the expensive step. Layer 2 and Layer 3 are cheap and
+re-run freely. This split means that adding support for a new
+framework, fixing a gate recognition pattern, or adjusting effect
+classification requires zero re-extraction.
+
+---
+
+## Extension Points
+
+| Extension | Layer | Mechanism | Requires re-extraction? |
+|-----------|-------|-----------|------------------------|
+| New route registration pattern | 2 | Route interpreter module | No |
+| New request parser library | 2 | Input interpreter module | No |
+| New auth decorator pattern | 2 | Gate interpreter module | No |
+| New ORM write pattern | 2 | Effect interpreter module | No |
+| New context proxy | 2 | State interpreter module | No |
+| Custom flow source/sink | 2 | Flow interpreter module | No |
+| New detection rule | 3 | Detector module | No |
+| New model library | 3 | Ordinary Python module | No |
+| New extraction pass | 1 | Fact builder module | Yes |
+
+Extension through Layer 2 interpreters is the expected path. Layer 1
+extraction changes only when the structural fact base is missing a
+primitive observation that no interpreter can recover from existing
+facts.
+
+---
+
+## Dependency Rules
+
+1. **Layer 3 depends only on Layer 2.** Rule code imports Layer 2's domain
+   types and API. It never imports Layer 1 types, never reads analysis
+   store files, never instantiates code index objects.
+
+2. **Layer 2 depends only on Layer 1.** Interpreter code imports Layer 1's
+   `CodeIndex` API. It never reads raw Semgrep output, never parses JSONL
+   files, never accesses internal storage paths.
+
+3. **Layer 1 depends on nothing above.** It has no knowledge of frameworks,
+   routes, security semantics, or vulnerability patterns.
+
+4. **No skipping.** Layer 3 cannot reach into Layer 1 for "just this one
+   query." If Layer 3 needs something, Layer 2 must expose it. If
+   Layer 2 needs something, Layer 1 must expose it.
+
+5. **Boundary immutability.** Objects crossing layer boundaries are frozen.
+   The receiving layer cannot modify what the producing layer tracks.
+
+6. **Vocabulary matches the consumer.** Layer 1 speaks Python structural
+   vocabulary (functions, calls, CFG blocks). Layer 2 speaks web
+   application security vocabulary (routes, inputs, gates, effects).
+   Layer 3 speaks vulnerability research vocabulary (invariants,
+   confusion, candidates, evidence).