envshield 4.7.3__tar.gz → 4.8.0__tar.gz

{envshield-4.7.3 → envshield-4.8.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: envshield
-Version: 4.7.3
+Version: 4.8.0
 Summary: EnvShield: environment variables as a version-controlled configuration contract.
 Author-email: Rabbil Yasar Sajal <rabbilyasar@gmail.com>
 License: MIT License
@@ -165,10 +165,11 @@ What each field actually means:
 | `enum` | Value must be one of this list. Implies an enum type regardless of `type`. |
 | `pattern` | A regex the value must also match, e.g. `pattern = "^v\\d+\\.\\d+\\.\\d+$"`. |
 | `secret` | Marks the variable sensitive — masked input in `setup`, masked in generated code. `import` auto-suggests it from the variable's name/value; it never overwrites a `secret` value you've already committed. |
-| `defaultValue` | What `setup` writes automatically. A variable still has to be explicitly present in your local file even with a default — `check`/`doctor` name the default inline so a missing one is obvious. |
+| `defaultValue` | What `setup` writes automatically (and `.env.example` and generated code show). A default never makes a variable optional: it still has to be explicitly present in your local file — `check`/`doctor` name the default inline so a missing one is obvious. |
+| `required` | Optional. `required = false` makes a variable optional: it may be absent or blank, but a value that is set must still be valid. `required = true` is the default. Can't be combined with `requiredIf`. |
 | `description` | Shown in `setup`, copied into generated code. |
 
-**There is no separate `required = true/false` field.** Requiredness is derived: unconditional by default, waived by a `defaultValue`, or made conditional by `requiredIf` — see below.
+Requiredness: every variable is required unless `required = false`, or `requiredIf` makes it conditional (see below). Any other key, or an unknown `type`, is an error — a typo like `default = "8000"` is reported (`did you mean 'defaultValue'?`) instead of silently ignored.
 
 ### Conditional requirements (`requiredIf`)
 
@@ -210,7 +211,7 @@ Four stages, in order: define the contract, validate against it, discover what c
 
 ### Validate
 
-**`envshield check`** — validate local configuration, and commonly-used Docker Compose and Kubernetes deployment-manifest patterns, against the same schema (see [Known limitations](#known-limitations)).
+**`envshield check`** — validate local configuration, and commonly-used Docker Compose and Kubernetes deployment-manifest patterns, against the same schema (see [Known limitations](#known-limitations)). It also scans the service's own source code and fails on a variable the code reads but the schema doesn't declare (the same files the pre-commit `scan` checks). `--process-env` also counts variables set in your shell (for a name the schema declares, the shell's value wins over the file's). `--json` adds a versioned, value-free report per service.
 
 ```bash
 $ envshield check
@@ -270,7 +271,7 @@ Every command above documents its own options with `--help`; the full reference
 
 ## Git / CI workflow
 
-This is where the schema stops being documentation and starts being enforcement. Three concrete things EnvShield catches in a PR:
+This is where the schema stops being documentation and starts being enforcement. Three concrete things EnvShield catches before a change reaches production:
 
 **A new environment variable used in code but missing from the schema:**
 
@@ -284,12 +285,14 @@ envshield undeclared origin/main HEAD --service api
 envshield schema diff origin/main HEAD --service api
 ```
 
-**A secret accidentally introduced into a commit** — the pre-commit hook runs `scan --staged` automatically:
+**A secret accidentally introduced into a commit:**
 
 ```bash
 envshield hook install
 ```
 
+The pre-commit hook runs `scan --staged --enforce`, applying classification-based enforcement with interactive override for high-confidence findings on local commits, and strict blocking in CI. See [Secret safety](#secret-safety) for enforcement behavior details.
+
 In CI, diff against the merge-base, not the base branch's current tip (which can move while the PR is open):
 
 ```yaml
@@ -466,10 +469,84 @@ Source-code discovery (what powers `undeclared` and `explain`'s "used in source"
 
 ## Secret safety
 
-A supporting check alongside the contract, not the product itself. `secret = true` on a field tells EnvShield (and `setup`, and code generation) that it's sensitive — the schema records that fact, never the actual value. On top of that:
+A supporting check alongside the contract, not the product itself. `secret = true` on a field tells EnvShield (and `setup`, and code generation) that it's sensitive — the schema records that fact, never the actual value.
+
+### Secret scanning
+
+**`envshield scan`** looks for hardcoded secrets by pattern (Stripe, AWS, GitHub tokens, and more), and for env-var reads the schema doesn't declare, in one pass. Values are always redacted in output — a Stripe *publishable* key (`pk_...`) is never flagged, since detection matches the secret-key pattern (`sk_...`) by shape, not the variable's name.
+
+### Context-aware classification
+
+Python candidates are analyzed for syntactic context to distinguish code patterns from secret values:
+
+```python
+# Classified as code (suppressed):
+client(key=CONFIG_CONSTANT)
+
+
+def handler(api_key: str): ...
+
+
+# Classified as likely secret (detected):
+client(key="sk_live_abc123...")
+```
+
+This reduces false positives on ordinary code patterns without broadly suppressing secret-shaped values. Classification is lightweight and syntactic — it recognizes type annotations, function arguments, and identifier references, but doesn't perform semantic analysis beyond what the tokenizer can see.
+
+### Git hook enforcement
+
+**`envshield hook install`** wires secret scanning into your Git workflow with classification-based enforcement:
+
+```bash
+envshield hook install
+```
+
+The pre-commit hook runs `scan --staged --enforce`, which scans only staged content and applies an enforcement policy based on classification confidence.
+
+#### Enforcement behavior
+
+| Finding type | Interactive (local commit) | Non-interactive (CI) |
+|---|---|---|
+| No findings | Allow | Allow |
+| Clearly code-shaped | Suppressed | Suppressed |
+| Ambiguous | Block | Block |
+| High-confidence secret | Block + explicit override | Block |
+
+**Clearly code-shaped** candidates (type annotations, function keyword arguments, etc.) are suppressed by the classifier and don't produce findings.
+
+**Ambiguous** findings — insufficient context to confidently classify — block without override in both modes.
+
+**High-confidence secrets** (string literals matching secret patterns) block by default but offer an interactive override for local commits:
+
+```
+⚠ EnvShield found potential secrets
+
+HIGH CONFIDENCE
+  src/config.py:42    Generic API Key    [REDACTED (30 chars)]
+
+[1] Abort commit
+[2] Commit anyway
+
+Choice: 2
+
+Type COMMIT ANYWAY to confirm: COMMIT ANYWAY
+
+Proceeding with commit. The finding remains in the repository history.
+```
+
+The override requires exact confirmation text — `yes` or `Y` won't work. EOF (Ctrl+D) or Ctrl+C aborts. Raw secret values are never displayed in enforcement output.
+
+In CI or other non-interactive contexts, high-confidence findings block with no prompt — the commit is refused, and the scan exits non-zero.
+
+#### Scanning modes
+
+```bash
+envshield scan                      # Audit entire codebase
+envshield scan --staged             # Scan only staged Git content
+envshield scan --staged --enforce   # Staged scan + enforcement policy
+```
 
-- **`envshield scan`** looks for hardcoded secrets by pattern (Stripe, AWS, GitHub tokens, and more), and for env-var reads the schema doesn't declare, in one pass. Values are always redacted in output — a Stripe *publishable* key (`pk_...`) is never flagged, since detection matches the secret-key pattern (`sk_...`) by shape, not the variable's name. Python candidates are context-aware: code-shaped patterns like `key=CONSTANT` or `api_key: str` may be suppressed when they're clearly not secret values, while secret-shaped string literals remain detectable regardless of context.
-- **`envshield hook install`** wires `scan --staged` into a pre-commit hook, so a real secret is caught before it's committed, not after.
+The pre-commit hook uses the third form. `scan` without flags is for repository-wide audits, not commit-time enforcement.
 
 ---
 
@@ -493,12 +570,15 @@ envshield uninstall --yes    # skip the confirmation prompt
 
 ## Known limitations
 
-EnvShield is built around the patterns real projects actually use. A few narrower cases aren't handled yet — none of them let a secret leak or let a genuinely missing required variable pass as clean:
+EnvShield is built around the patterns real projects actually use. A few narrower cases aren't handled yet. None of them let a secret leak; the first one can let a value EnvShield couldn't see pass `check`:
 
+- **Unverified manifest values pass the v4 gate.** When a deployment manifest only *points* at a value — a Kubernetes `secretKeyRef`/`configMapKeyRef` to an object outside the file, a Secret key in the same file, a Compose `${VAR}` or bare `KEY`, or a `${VAR-x}`/`${VAR:-x}` fallback — EnvShield can't see the deployed value. `check` reports these variables as unverified (`valid: null`, status `not_visible`, listed in `summary.unverified`, and named in the human output), but they still pass: exit code and `success` are unchanged. A fallback that itself violates the contract still fails. Making the gate act on unverified values is planned for v5.
 - **Docker Compose:** `${VAR}`, `${VAR:-default}`, and the no-colon `${VAR-default}` form are all supported. The `${VAR:?}`/`${VAR:+}` forms and embedded/multiple interpolation references in one value aren't evaluated yet.
 - **Kubernetes:** `env[].valueFrom.secretKeyRef`/`configMapKeyRef` are matched by the container's environment-variable name (`env[].name`) — name your schema variable after the env var, not the key — and, when the referenced Secret/ConfigMap is defined in the same manifest, the referenced key is validated to actually exist there. `envFrom.prefix` is applied. `initContainers` are selectable via `--container` but never auto-selected by default.
 - **Source discovery:** Python is AST-based; JavaScript/TypeScript is pattern-based. No other language is discovered yet.
 - **.env:** multiline quoted values aren't supported.
+- **Windows:** the write-safety fix for a failed permission change (BL-162) hasn't been run on Windows; where `os.fchmod` is missing (Windows before Python 3.13) it's skipped, which is tested on Linux by simulation only.
+- **CI:** runs on Ubuntu only (Python 3.10–3.12).
 
 These are bounded, tracked engineering work — see the [roadmap](ROADMAP.md) for what's next and what EnvShield deliberately isn't building.
 

{envshield-4.7.3 → envshield-4.8.0}/README.md

@@ -106,10 +106,11 @@ What each field actually means:
 | `enum` | Value must be one of this list. Implies an enum type regardless of `type`. |
 | `pattern` | A regex the value must also match, e.g. `pattern = "^v\\d+\\.\\d+\\.\\d+$"`. |
 | `secret` | Marks the variable sensitive — masked input in `setup`, masked in generated code. `import` auto-suggests it from the variable's name/value; it never overwrites a `secret` value you've already committed. |
-| `defaultValue` | What `setup` writes automatically. A variable still has to be explicitly present in your local file even with a default — `check`/`doctor` name the default inline so a missing one is obvious. |
+| `defaultValue` | What `setup` writes automatically (and `.env.example` and generated code show). A default never makes a variable optional: it still has to be explicitly present in your local file — `check`/`doctor` name the default inline so a missing one is obvious. |
+| `required` | Optional. `required = false` makes a variable optional: it may be absent or blank, but a value that is set must still be valid. `required = true` is the default. Can't be combined with `requiredIf`. |
 | `description` | Shown in `setup`, copied into generated code. |
 
-**There is no separate `required = true/false` field.** Requiredness is derived: unconditional by default, waived by a `defaultValue`, or made conditional by `requiredIf` — see below.
+Requiredness: every variable is required unless `required = false`, or `requiredIf` makes it conditional (see below). Any other key, or an unknown `type`, is an error — a typo like `default = "8000"` is reported (`did you mean 'defaultValue'?`) instead of silently ignored.
 
 ### Conditional requirements (`requiredIf`)
 
@@ -151,7 +152,7 @@ Four stages, in order: define the contract, validate against it, discover what c
 
 ### Validate
 
-**`envshield check`** — validate local configuration, and commonly-used Docker Compose and Kubernetes deployment-manifest patterns, against the same schema (see [Known limitations](#known-limitations)).
+**`envshield check`** — validate local configuration, and commonly-used Docker Compose and Kubernetes deployment-manifest patterns, against the same schema (see [Known limitations](#known-limitations)). It also scans the service's own source code and fails on a variable the code reads but the schema doesn't declare (the same files the pre-commit `scan` checks). `--process-env` also counts variables set in your shell (for a name the schema declares, the shell's value wins over the file's). `--json` adds a versioned, value-free report per service.
 
 ```bash
 $ envshield check
@@ -211,7 +212,7 @@ Every command above documents its own options with `--help`; the full reference
 
 ## Git / CI workflow
 
-This is where the schema stops being documentation and starts being enforcement. Three concrete things EnvShield catches in a PR:
+This is where the schema stops being documentation and starts being enforcement. Three concrete things EnvShield catches before a change reaches production:
 
 **A new environment variable used in code but missing from the schema:**
 
@@ -225,12 +226,14 @@ envshield undeclared origin/main HEAD --service api
 envshield schema diff origin/main HEAD --service api
 ```
 
-**A secret accidentally introduced into a commit** — the pre-commit hook runs `scan --staged` automatically:
+**A secret accidentally introduced into a commit:**
 
 ```bash
 envshield hook install
 ```
 
+The pre-commit hook runs `scan --staged --enforce`, applying classification-based enforcement with interactive override for high-confidence findings on local commits, and strict blocking in CI. See [Secret safety](#secret-safety) for enforcement behavior details.
+
 In CI, diff against the merge-base, not the base branch's current tip (which can move while the PR is open):
 
 ```yaml
@@ -407,10 +410,84 @@ Source-code discovery (what powers `undeclared` and `explain`'s "used in source"
 
 ## Secret safety
 
-A supporting check alongside the contract, not the product itself. `secret = true` on a field tells EnvShield (and `setup`, and code generation) that it's sensitive — the schema records that fact, never the actual value. On top of that:
+A supporting check alongside the contract, not the product itself. `secret = true` on a field tells EnvShield (and `setup`, and code generation) that it's sensitive — the schema records that fact, never the actual value.
+
+### Secret scanning
+
+**`envshield scan`** looks for hardcoded secrets by pattern (Stripe, AWS, GitHub tokens, and more), and for env-var reads the schema doesn't declare, in one pass. Values are always redacted in output — a Stripe *publishable* key (`pk_...`) is never flagged, since detection matches the secret-key pattern (`sk_...`) by shape, not the variable's name.
+
+### Context-aware classification
+
+Python candidates are analyzed for syntactic context to distinguish code patterns from secret values:
+
+```python
+# Classified as code (suppressed):
+client(key=CONFIG_CONSTANT)
+
+
+def handler(api_key: str): ...
+
+
+# Classified as likely secret (detected):
+client(key="sk_live_abc123...")
+```
+
+This reduces false positives on ordinary code patterns without broadly suppressing secret-shaped values. Classification is lightweight and syntactic — it recognizes type annotations, function arguments, and identifier references, but doesn't perform semantic analysis beyond what the tokenizer can see.
+
+### Git hook enforcement
+
+**`envshield hook install`** wires secret scanning into your Git workflow with classification-based enforcement:
+
+```bash
+envshield hook install
+```
+
+The pre-commit hook runs `scan --staged --enforce`, which scans only staged content and applies an enforcement policy based on classification confidence.
+
+#### Enforcement behavior
+
+| Finding type | Interactive (local commit) | Non-interactive (CI) |
+|---|---|---|
+| No findings | Allow | Allow |
+| Clearly code-shaped | Suppressed | Suppressed |
+| Ambiguous | Block | Block |
+| High-confidence secret | Block + explicit override | Block |
+
+**Clearly code-shaped** candidates (type annotations, function keyword arguments, etc.) are suppressed by the classifier and don't produce findings.
+
+**Ambiguous** findings — insufficient context to confidently classify — block without override in both modes.
+
+**High-confidence secrets** (string literals matching secret patterns) block by default but offer an interactive override for local commits:
+
+```
+⚠ EnvShield found potential secrets
+
+HIGH CONFIDENCE
+  src/config.py:42    Generic API Key    [REDACTED (30 chars)]
+
+[1] Abort commit
+[2] Commit anyway
+
+Choice: 2
+
+Type COMMIT ANYWAY to confirm: COMMIT ANYWAY
+
+Proceeding with commit. The finding remains in the repository history.
+```
+
+The override requires exact confirmation text — `yes` or `Y` won't work. EOF (Ctrl+D) or Ctrl+C aborts. Raw secret values are never displayed in enforcement output.
+
+In CI or other non-interactive contexts, high-confidence findings block with no prompt — the commit is refused, and the scan exits non-zero.
+
+#### Scanning modes
+
+```bash
+envshield scan                      # Audit entire codebase
+envshield scan --staged             # Scan only staged Git content
+envshield scan --staged --enforce   # Staged scan + enforcement policy
+```
 
-- **`envshield scan`** looks for hardcoded secrets by pattern (Stripe, AWS, GitHub tokens, and more), and for env-var reads the schema doesn't declare, in one pass. Values are always redacted in output — a Stripe *publishable* key (`pk_...`) is never flagged, since detection matches the secret-key pattern (`sk_...`) by shape, not the variable's name. Python candidates are context-aware: code-shaped patterns like `key=CONSTANT` or `api_key: str` may be suppressed when they're clearly not secret values, while secret-shaped string literals remain detectable regardless of context.
-- **`envshield hook install`** wires `scan --staged` into a pre-commit hook, so a real secret is caught before it's committed, not after.
+The pre-commit hook uses the third form. `scan` without flags is for repository-wide audits, not commit-time enforcement.
 
 ---
 
@@ -434,12 +511,15 @@ envshield uninstall --yes    # skip the confirmation prompt
 
 ## Known limitations
 
-EnvShield is built around the patterns real projects actually use. A few narrower cases aren't handled yet — none of them let a secret leak or let a genuinely missing required variable pass as clean:
+EnvShield is built around the patterns real projects actually use. A few narrower cases aren't handled yet. None of them let a secret leak; the first one can let a value EnvShield couldn't see pass `check`:
 
+- **Unverified manifest values pass the v4 gate.** When a deployment manifest only *points* at a value — a Kubernetes `secretKeyRef`/`configMapKeyRef` to an object outside the file, a Secret key in the same file, a Compose `${VAR}` or bare `KEY`, or a `${VAR-x}`/`${VAR:-x}` fallback — EnvShield can't see the deployed value. `check` reports these variables as unverified (`valid: null`, status `not_visible`, listed in `summary.unverified`, and named in the human output), but they still pass: exit code and `success` are unchanged. A fallback that itself violates the contract still fails. Making the gate act on unverified values is planned for v5.
 - **Docker Compose:** `${VAR}`, `${VAR:-default}`, and the no-colon `${VAR-default}` form are all supported. The `${VAR:?}`/`${VAR:+}` forms and embedded/multiple interpolation references in one value aren't evaluated yet.
 - **Kubernetes:** `env[].valueFrom.secretKeyRef`/`configMapKeyRef` are matched by the container's environment-variable name (`env[].name`) — name your schema variable after the env var, not the key — and, when the referenced Secret/ConfigMap is defined in the same manifest, the referenced key is validated to actually exist there. `envFrom.prefix` is applied. `initContainers` are selectable via `--container` but never auto-selected by default.
 - **Source discovery:** Python is AST-based; JavaScript/TypeScript is pattern-based. No other language is discovered yet.
 - **.env:** multiline quoted values aren't supported.
+- **Windows:** the write-safety fix for a failed permission change (BL-162) hasn't been run on Windows; where `os.fchmod` is missing (Windows before Python 3.13) it's skipped, which is tested on Linux by simulation only.
+- **CI:** runs on Ubuntu only (Python 3.10–3.12).
 
 These are bounded, tracked engineering work — see the [roadmap](ROADMAP.md) for what's next and what EnvShield deliberately isn't building.
 

envshield-4.8.0/envshield/__init__.py

@@ -0,0 +1 @@
+__version__ = "4.8.0"