envshield 4.7.1__tar.gz → 4.7.4__tar.gz

{envshield-4.7.1 → envshield-4.7.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: envshield
-Version: 4.7.1
+Version: 4.7.4
 Summary: EnvShield: environment variables as a version-controlled configuration contract.
 Author-email: Rabbil Yasar Sajal <rabbilyasar@gmail.com>
 License: MIT License
@@ -270,7 +270,7 @@ Every command above documents its own options with `--help`; the full reference
 
 ## Git / CI workflow
 
-This is where the schema stops being documentation and starts being enforcement. Three concrete things EnvShield catches in a PR:
+This is where the schema stops being documentation and starts being enforcement. Three concrete things EnvShield catches before a change reaches production:
 
 **A new environment variable used in code but missing from the schema:**
 
@@ -284,12 +284,14 @@ envshield undeclared origin/main HEAD --service api
 envshield schema diff origin/main HEAD --service api
 ```
 
-**A secret accidentally introduced into a commit** — the pre-commit hook runs `scan --staged` automatically:
+**A secret accidentally introduced into a commit:**
 
 ```bash
 envshield hook install
 ```
 
+The pre-commit hook runs `scan --staged --enforce`, applying classification-based enforcement with interactive override for high-confidence findings on local commits, and strict blocking in CI. See [Secret safety](#secret-safety) for enforcement behavior details.
+
 In CI, diff against the merge-base, not the base branch's current tip (which can move while the PR is open):
 
 ```yaml
@@ -308,7 +310,25 @@ jobs:
           envshield undeclared "$BASE" HEAD
 ```
 
-Both commands exit non-zero on a real finding — nothing beyond the exit code is required to gate a PR. Add `--json` if you want to post a custom summary instead.
+Both commands exit non-zero on a real finding, and also on an error that prevented evaluation (a bad revision, a schema that failed to parse) — never a silent "clean" for something it couldn't actually check. Add `--json` if you want to post a custom summary instead.
+
+### SARIF output (`undeclared --sarif`)
+
+`envshield undeclared` also supports `--sarif` — a standard [SARIF 2.1.0](https://docs.oasis-open.org/sarif/sarif/v2.1.0/) log, for GitHub Code Scanning or any other SARIF-consuming CI tool, so a missing declaration is annotated on its exact file and line rather than only appearing in a CI log:
+
+```yaml
+      - run: |
+          BASE=$(git merge-base "origin/${{ github.base_ref }}" HEAD)
+          envshield undeclared "$BASE" HEAD --sarif > envshield.sarif
+        continue-on-error: true   # upload the report even when it found something
+      - uses: github/codeql-action/upload-sarif@v3
+        with:
+          sarif_file: envshield.sarif
+```
+
+`--sarif` only exists for `undeclared`, deliberately: each finding there has a real file and line to point at. `schema diff` findings describe the *contract* (a variable added, removed, or changed) rather than a source location, so forcing them into SARIF would mean inventing a line number that isn't real — use its `--json`/exit-code output instead, or annotate it yourself (e.g. a `jq` step over the JSON, emitting `::error::` workflow commands).
+
+An error that prevents evaluating a service (a bad revision, a schema that fails to parse) is never turned into a fabricated SARIF result — it's reported as a SARIF `invocation` with `executionSuccessful: false`, so a consumer can't mistake "couldn't check" for "checked and clean."
 
 ---
 
@@ -410,6 +430,34 @@ A few things worth knowing:
 
 ---
 
+## Docker Compose base + override layers (`files:`)
+
+A manifest registration normally points at one file:
+
+```yaml
+manifests:
+  - file: docker-compose.yml
+    containers:
+      api: api
+```
+
+A real Compose project commonly splits its configuration across a base file plus an override that Compose applies automatically (`docker-compose.yml` + `docker-compose.override.yml`, or any other `-f a -f b` combination). Registering only the base misses whatever the override adds; registering the override alone reports the base's own variables as falsely missing, since an override file is deliberately just a delta. `files:` registers the whole ordered layer set as **one** logical manifest instead:
+
+```yaml
+manifests:
+  - files:
+      - docker-compose.yml
+      - docker-compose.override.yml
+    containers:
+      api: api
+```
+
+Order matters — later files override earlier ones, exactly like `docker compose -f docker-compose.yml -f docker-compose.override.yml` itself. `check`/`doctor`/`explain` all validate the merged result as a single source, under one combined label (`"docker-compose.yml + docker-compose.override.yml"`).
+
+This is Compose-specific merge behavior for the fields EnvShield already understands (`environment:`, `env_file:`) — **not** a general YAML merge engine. Volumes, networks, ports, build, and every other Compose field are never inspected or merged. `files:` is unrelated to `completeness: union` above: union combines *presence across different kinds of source* (a local file and a manifest); `files:` merges *values across ordered layers of one Compose source*. A manifest entry can declare `file:` or `files:`, never both.
+
+---
+
 ## Supported languages / discovery
 
 Source-code discovery (what powers `undeclared` and `explain`'s "used in source" section) is AST-based for **Python** (`os.environ.get`, `os.getenv`, `os.environ[...]`) and pattern-based for **JavaScript/TypeScript** (`process.env.X`, `process.env["X"]`, `import.meta.env.X`, including single-level destructuring). No other language is discovered yet — the schema and validation commands work with any stack, but `undeclared`/`explain` can only see what's read from these two.
@@ -420,10 +468,84 @@ Source-code discovery (what powers `undeclared` and `explain`'s "used in source"
 
 ## Secret safety
 
-A supporting check alongside the contract, not the product itself. `secret = true` on a field tells EnvShield (and `setup`, and code generation) that it's sensitive — the schema records that fact, never the actual value. On top of that:
+A supporting check alongside the contract, not the product itself. `secret = true` on a field tells EnvShield (and `setup`, and code generation) that it's sensitive — the schema records that fact, never the actual value.
+
+### Secret scanning
+
+**`envshield scan`** looks for hardcoded secrets by pattern (Stripe, AWS, GitHub tokens, and more), and for env-var reads the schema doesn't declare, in one pass. Values are always redacted in output — a Stripe *publishable* key (`pk_...`) is never flagged, since detection matches the secret-key pattern (`sk_...`) by shape, not the variable's name.
+
+### Context-aware classification
+
+Python candidates are analyzed for syntactic context to distinguish code patterns from secret values:
+
+```python
+# Classified as code (suppressed):
+client(key=CONFIG_CONSTANT)
+
+
+def handler(api_key: str): ...
+
+
+# Classified as likely secret (detected):
+client(key="sk_live_abc123...")
+```
+
+This reduces false positives on ordinary code patterns without broadly suppressing secret-shaped values. Classification is lightweight and syntactic — it recognizes type annotations, function arguments, and identifier references, but doesn't perform semantic analysis beyond what the tokenizer can see.
+
+### Git hook enforcement
+
+**`envshield hook install`** wires secret scanning into your Git workflow with classification-based enforcement:
+
+```bash
+envshield hook install
+```
+
+The pre-commit hook runs `scan --staged --enforce`, which scans only staged content and applies an enforcement policy based on classification confidence.
+
+#### Enforcement behavior
+
+| Finding type | Interactive (local commit) | Non-interactive (CI) |
+|---|---|---|
+| No findings | Allow | Allow |
+| Clearly code-shaped | Suppressed | Suppressed |
+| Ambiguous | Block | Block |
+| High-confidence secret | Block + explicit override | Block |
+
+**Clearly code-shaped** candidates (type annotations, function keyword arguments, etc.) are suppressed by the classifier and don't produce findings.
+
+**Ambiguous** findings — insufficient context to confidently classify — block without override in both modes.
+
+**High-confidence secrets** (string literals matching secret patterns) block by default but offer an interactive override for local commits:
+
+```
+⚠ EnvShield found potential secrets
+
+HIGH CONFIDENCE
+  src/config.py:42    Generic API Key    [REDACTED (30 chars)]
+
+[1] Abort commit
+[2] Commit anyway
+
+Choice: 2
+
+Type COMMIT ANYWAY to confirm: COMMIT ANYWAY
+
+Proceeding with commit. The finding remains in the repository history.
+```
+
+The override requires exact confirmation text — `yes` or `Y` won't work. EOF (Ctrl+D) or Ctrl+C aborts. Raw secret values are never displayed in enforcement output.
+
+In CI or other non-interactive contexts, high-confidence findings block with no prompt — the commit is refused, and the scan exits non-zero.
+
+#### Scanning modes
+
+```bash
+envshield scan                      # Audit entire codebase
+envshield scan --staged             # Scan only staged Git content
+envshield scan --staged --enforce   # Staged scan + enforcement policy
+```
 
-- **`envshield scan`** looks for hardcoded secrets by pattern (Stripe, AWS, GitHub tokens, and more), and for env-var reads the schema doesn't declare, in one pass. Values are always redacted in output — a Stripe *publishable* key (`pk_...`) is never flagged, since detection matches the secret-key pattern (`sk_...`) by shape, not the variable's name.
-- **`envshield hook install`** wires `scan --staged` into a pre-commit hook, so a real secret is caught before it's committed, not after.
+The pre-commit hook uses the third form. `scan` without flags is for repository-wide audits, not commit-time enforcement.
 
 ---
 

{envshield-4.7.1 → envshield-4.7.4}/README.md

@@ -211,7 +211,7 @@ Every command above documents its own options with `--help`; the full reference
 
 ## Git / CI workflow
 
-This is where the schema stops being documentation and starts being enforcement. Three concrete things EnvShield catches in a PR:
+This is where the schema stops being documentation and starts being enforcement. Three concrete things EnvShield catches before a change reaches production:
 
 **A new environment variable used in code but missing from the schema:**
 
@@ -225,12 +225,14 @@ envshield undeclared origin/main HEAD --service api
 envshield schema diff origin/main HEAD --service api
 ```
 
-**A secret accidentally introduced into a commit** — the pre-commit hook runs `scan --staged` automatically:
+**A secret accidentally introduced into a commit:**
 
 ```bash
 envshield hook install
 ```
 
+The pre-commit hook runs `scan --staged --enforce`, applying classification-based enforcement with interactive override for high-confidence findings on local commits, and strict blocking in CI. See [Secret safety](#secret-safety) for enforcement behavior details.
+
 In CI, diff against the merge-base, not the base branch's current tip (which can move while the PR is open):
 
 ```yaml
@@ -249,7 +251,25 @@ jobs:
           envshield undeclared "$BASE" HEAD
 ```
 
-Both commands exit non-zero on a real finding — nothing beyond the exit code is required to gate a PR. Add `--json` if you want to post a custom summary instead.
+Both commands exit non-zero on a real finding, and also on an error that prevented evaluation (a bad revision, a schema that failed to parse) — never a silent "clean" for something it couldn't actually check. Add `--json` if you want to post a custom summary instead.
+
+### SARIF output (`undeclared --sarif`)
+
+`envshield undeclared` also supports `--sarif` — a standard [SARIF 2.1.0](https://docs.oasis-open.org/sarif/sarif/v2.1.0/) log, for GitHub Code Scanning or any other SARIF-consuming CI tool, so a missing declaration is annotated on its exact file and line rather than only appearing in a CI log:
+
+```yaml
+      - run: |
+          BASE=$(git merge-base "origin/${{ github.base_ref }}" HEAD)
+          envshield undeclared "$BASE" HEAD --sarif > envshield.sarif
+        continue-on-error: true   # upload the report even when it found something
+      - uses: github/codeql-action/upload-sarif@v3
+        with:
+          sarif_file: envshield.sarif
+```
+
+`--sarif` only exists for `undeclared`, deliberately: each finding there has a real file and line to point at. `schema diff` findings describe the *contract* (a variable added, removed, or changed) rather than a source location, so forcing them into SARIF would mean inventing a line number that isn't real — use its `--json`/exit-code output instead, or annotate it yourself (e.g. a `jq` step over the JSON, emitting `::error::` workflow commands).
+
+An error that prevents evaluating a service (a bad revision, a schema that fails to parse) is never turned into a fabricated SARIF result — it's reported as a SARIF `invocation` with `executionSuccessful: false`, so a consumer can't mistake "couldn't check" for "checked and clean."
 
 ---
 
@@ -351,6 +371,34 @@ A few things worth knowing:
 
 ---
 
+## Docker Compose base + override layers (`files:`)
+
+A manifest registration normally points at one file:
+
+```yaml
+manifests:
+  - file: docker-compose.yml
+    containers:
+      api: api
+```
+
+A real Compose project commonly splits its configuration across a base file plus an override that Compose applies automatically (`docker-compose.yml` + `docker-compose.override.yml`, or any other `-f a -f b` combination). Registering only the base misses whatever the override adds; registering the override alone reports the base's own variables as falsely missing, since an override file is deliberately just a delta. `files:` registers the whole ordered layer set as **one** logical manifest instead:
+
+```yaml
+manifests:
+  - files:
+      - docker-compose.yml
+      - docker-compose.override.yml
+    containers:
+      api: api
+```
+
+Order matters — later files override earlier ones, exactly like `docker compose -f docker-compose.yml -f docker-compose.override.yml` itself. `check`/`doctor`/`explain` all validate the merged result as a single source, under one combined label (`"docker-compose.yml + docker-compose.override.yml"`).
+
+This is Compose-specific merge behavior for the fields EnvShield already understands (`environment:`, `env_file:`) — **not** a general YAML merge engine. Volumes, networks, ports, build, and every other Compose field are never inspected or merged. `files:` is unrelated to `completeness: union` above: union combines *presence across different kinds of source* (a local file and a manifest); `files:` merges *values across ordered layers of one Compose source*. A manifest entry can declare `file:` or `files:`, never both.
+
+---
+
 ## Supported languages / discovery
 
 Source-code discovery (what powers `undeclared` and `explain`'s "used in source" section) is AST-based for **Python** (`os.environ.get`, `os.getenv`, `os.environ[...]`) and pattern-based for **JavaScript/TypeScript** (`process.env.X`, `process.env["X"]`, `import.meta.env.X`, including single-level destructuring). No other language is discovered yet — the schema and validation commands work with any stack, but `undeclared`/`explain` can only see what's read from these two.
@@ -361,10 +409,84 @@ Source-code discovery (what powers `undeclared` and `explain`'s "used in source"
 
 ## Secret safety
 
-A supporting check alongside the contract, not the product itself. `secret = true` on a field tells EnvShield (and `setup`, and code generation) that it's sensitive — the schema records that fact, never the actual value. On top of that:
+A supporting check alongside the contract, not the product itself. `secret = true` on a field tells EnvShield (and `setup`, and code generation) that it's sensitive — the schema records that fact, never the actual value.
+
+### Secret scanning
+
+**`envshield scan`** looks for hardcoded secrets by pattern (Stripe, AWS, GitHub tokens, and more), and for env-var reads the schema doesn't declare, in one pass. Values are always redacted in output — a Stripe *publishable* key (`pk_...`) is never flagged, since detection matches the secret-key pattern (`sk_...`) by shape, not the variable's name.
+
+### Context-aware classification
+
+Python candidates are analyzed for syntactic context to distinguish code patterns from secret values:
+
+```python
+# Classified as code (suppressed):
+client(key=CONFIG_CONSTANT)
+
+
+def handler(api_key: str): ...
+
+
+# Classified as likely secret (detected):
+client(key="sk_live_abc123...")
+```
+
+This reduces false positives on ordinary code patterns without broadly suppressing secret-shaped values. Classification is lightweight and syntactic — it recognizes type annotations, function arguments, and identifier references, but doesn't perform semantic analysis beyond what the tokenizer can see.
+
+### Git hook enforcement
+
+**`envshield hook install`** wires secret scanning into your Git workflow with classification-based enforcement:
+
+```bash
+envshield hook install
+```
+
+The pre-commit hook runs `scan --staged --enforce`, which scans only staged content and applies an enforcement policy based on classification confidence.
+
+#### Enforcement behavior
+
+| Finding type | Interactive (local commit) | Non-interactive (CI) |
+|---|---|---|
+| No findings | Allow | Allow |
+| Clearly code-shaped | Suppressed | Suppressed |
+| Ambiguous | Block | Block |
+| High-confidence secret | Block + explicit override | Block |
+
+**Clearly code-shaped** candidates (type annotations, function keyword arguments, etc.) are suppressed by the classifier and don't produce findings.
+
+**Ambiguous** findings — insufficient context to confidently classify — block without override in both modes.
+
+**High-confidence secrets** (string literals matching secret patterns) block by default but offer an interactive override for local commits:
+
+```
+⚠ EnvShield found potential secrets
+
+HIGH CONFIDENCE
+  src/config.py:42    Generic API Key    [REDACTED (30 chars)]
+
+[1] Abort commit
+[2] Commit anyway
+
+Choice: 2
+
+Type COMMIT ANYWAY to confirm: COMMIT ANYWAY
+
+Proceeding with commit. The finding remains in the repository history.
+```
+
+The override requires exact confirmation text — `yes` or `Y` won't work. EOF (Ctrl+D) or Ctrl+C aborts. Raw secret values are never displayed in enforcement output.
+
+In CI or other non-interactive contexts, high-confidence findings block with no prompt — the commit is refused, and the scan exits non-zero.
+
+#### Scanning modes
+
+```bash
+envshield scan                      # Audit entire codebase
+envshield scan --staged             # Scan only staged Git content
+envshield scan --staged --enforce   # Staged scan + enforcement policy
+```
 
-- **`envshield scan`** looks for hardcoded secrets by pattern (Stripe, AWS, GitHub tokens, and more), and for env-var reads the schema doesn't declare, in one pass. Values are always redacted in output — a Stripe *publishable* key (`pk_...`) is never flagged, since detection matches the secret-key pattern (`sk_...`) by shape, not the variable's name.
-- **`envshield hook install`** wires `scan --staged` into a pre-commit hook, so a real secret is caught before it's committed, not after.
+The pre-commit hook uses the third form. `scan` without flags is for repository-wide audits, not commit-time enforcement.
 
 ---
 

envshield-4.7.4/envshield/__init__.py

@@ -0,0 +1 @@
+__version__ = "4.7.4"

envshield-4.7.4/envshield/classifier/__init__.py

@@ -0,0 +1,12 @@
+# envshield/classifier/__init__.py
+"""
+Context classifier for false-positive reduction in secret scanning.
+
+This module provides the tokenizer and context classifier used to distinguish
+code patterns (function keyword arguments, type annotations, destructuring)
+from actual secret values, reducing false positives in the Generic API Key
+detector.
+
+See Milestone 2 (tokenizer) and Milestone 3 (classifier) documentation for
+design rationale and limitations.
+"""