dumpling-cli 0.4.0__tar.gz → 0.4.2__tar.gz

{dumpling_cli-0.4.0 → dumpling_cli-0.4.2}/CHANGELOG.md

@@ -7,6 +7,19 @@ and this project follows [Semantic Versioning](https://semver.org/spec/v2.0.0.ht
 
 ## [Unreleased]
 
+## [0.4.2] - 2026-05-03
+
+### Fixed
+
+- **JSON path rules on non-JSON cells**: Path-based `[rules]` anonymization is skipped when the cell is not strict JSON, leaving the original value unchanged (consistent with row-filter JSON path behavior).
+- **JSON scalar types in path-based anonymization**: Replacements at JSON paths preserve number and boolean leaf types where possible (numeric and boolean coercion from generated text).
+
+## [0.4.1] - 2026-05-03
+
+### Fixed
+
+- **INSERT row parsing with JSON casts**: Values such as `'{"k":1}'::jsonb` are parsed so the cell’s unescaped payload is valid JSON for JSON path rules and anonymization; trailing casts like `::jsonb` / `::text` are preserved on output.
+
 ## [0.4.0] - 2026-05-02
 
 ### Added
@@ -55,6 +68,8 @@ and this project follows [Semantic Versioning](https://semver.org/spec/v2.0.0.ht
 - Configurable output scan severities and per-category thresholds via `[output_scan]`.
 - JSON report section for output scan findings including category, count, threshold, severity, and sample locations.
 
+[0.4.2]: https://github.com/ababic/dumpling/compare/v0.4.1...v0.4.2
+[0.4.1]: https://github.com/ababic/dumpling/compare/v0.4.0...v0.4.1
 [0.4.0]: https://github.com/ababic/dumpling/compare/v0.3.0...v0.4.0
 [0.3.0]: https://github.com/ababic/dumpling/compare/v0.2.0...v0.3.0
 [0.2.0]: https://github.com/ababic/dumpling/compare/v0.1.0...v0.2.0

{dumpling_cli-0.4.0 → dumpling_cli-0.4.2}/Cargo.lock

@@ -262,7 +262,7 @@ dependencies = [
 
 [[package]]
 name = "dumpling"
-version = "0.4.0"
+version = "0.4.2"
 dependencies = [
  "anyhow",
  "chrono",

{dumpling_cli-0.4.0 → dumpling_cli-0.4.2}/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "dumpling"
-version = "0.4.0"
+version = "0.4.2"
 edition = "2021"
 readme = "README.md"
 

{dumpling_cli-0.4.0 → dumpling_cli-0.4.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: dumpling-cli
-Version: 0.4.0
+Version: 0.4.2
 Classifier: Development Status :: 4 - Beta
 Classifier: Environment :: Console
 Classifier: Intended Audience :: Developers
@@ -19,18 +19,48 @@ Keywords: postgres,sqlite,mssql,sql,anonymization,cli,rust
 Requires-Python: >=3.8
 Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
 
-# Dumpling
+<p align="center">
+  <img src="assets/logo.svg" width="140" height="140" alt="Dumpling logo: a dumpling with steam" />
+</p>
 
-**Dumpling** is a static anonymizer for plain SQL dumps. It supports PostgreSQL (`pg_dump` plain format), SQLite (`.dump`), and SQL Server / MSSQL (SSMS / mssql-scripter output). It lets you safely share, test with, or store database snapshots by replacing sensitive column data according to configurable rules — without ever touching a live database.
+<h1 align="center">Dumpling</h1>
+
+<p align="center">
+  <strong>Sanitize SQL dumps before they go anywhere.</strong><br />
+  Turn huge <code>pg_dump</code> / SQLite / SQL Server exports into shareable, test-friendly snapshots — no DB connection, no secrets left by accident.
+</p>
+
+<p align="center">
+  <a href="https://pypi.org/project/dumpling-cli/"><img src="https://img.shields.io/pypi/v/dumpling-cli.svg" alt="PyPI version" /></a>
+  <a href="https://pypi.org/project/dumpling-cli/"><img src="https://img.shields.io/pypi/pyversions/dumpling-cli.svg" alt="Python versions" /></a>
+  <a href="https://pypi.org/project/dumpling-cli/"><img src="https://img.shields.io/pypi/l/dumpling-cli.svg" alt="PyPI license" /></a>
+  <a href="https://github.com/ababic/dumpling/actions/workflows/tests.yml"><img src="https://github.com/ababic/dumpling/actions/workflows/tests.yml/badge.svg" alt="Tests" /></a>
+  <a href="https://github.com/ababic/dumpling/actions/workflows/ci.yml"><img src="https://github.com/ababic/dumpling/actions/workflows/ci.yml/badge.svg" alt="Lint" /></a>
+  <img src="https://img.shields.io/badge/rust-stable-orange?logo=rust" alt="Rust stable" />
+</p>
+
+<p align="center">
+  <a href="https://ababic.github.io/dumpling/"><strong>Documentation</strong></a>
+  &nbsp;·&nbsp;
+  <a href="https://github.com/ababic/dumpling"><strong>GitHub</strong></a>
+</p>
+
+<p align="center">
+  <sub><em>Disclaimer: This project is entirely vibe-coded, but with strong human guidance, review, and attention to quality and safety.</em></sub>
+</p>
+
+---
+
+**Dumpling** reads plain-text SQL dumps (PostgreSQL `pg_dump`, SQLite `.dump`, SQL Server / MSSQL scripts) and rewrites sensitive columns using rules you define in TOML. Everything runs offline on files — ideal for CI, staging share-outs, and compliance-minded workflows.
 
 ## Why Dumpling?
 
-- **No live database required.** Works entirely on dump files; nothing connects to your database.
-- **Streaming and memory-efficient.** Processes dumps line by line, so even multi-gigabyte files stay manageable.
-- **Fail-safe by default.** If no configuration is found, Dumpling exits non-zero and tells you exactly where it looked. Silence is never mistaken for success.
-- **Deterministic anonymization.** Domain mappings ensure the same source value always produces the same pseudonym, keeping foreign-key relationships intact across tables.
-- **CI/CD ready.** `--check` mode, strict-coverage enforcement, JSON reports, and residual-PII scan gates plug cleanly into any pipeline.
-- **Flexible configuration.** Rules live in a `.dumplingconf` file or directly in `pyproject.toml` — no extra tooling needed.
+- **Offline by design** — works on dump files only; nothing connects to your database.
+- **Streams giant files** — line-by-line processing keeps multi‑GB dumps reasonable on modest hardware.
+- **Fails loud, not silent** — missing config exits non‑zero and lists where Dumpling looked; use `--allow-noop` only when you mean it.
+- **Stable pseudonyms** — optional domain mappings keep the same source value as the same fake value across tables (foreign keys stay consistent).
+- **Pipeline-ready** — `--check`, strict coverage, JSON reports, and residual PII scans fit pre-merge gates and release automation.
+- **Configure once** — `.dumplingconf` or `[tool.dumpling]` in `pyproject.toml`; install via **Rust** (`cargo`) or **`pip install dumpling-cli`**.
 
 ---
 
@@ -344,6 +374,22 @@ Supported predicate operators:
 
 Predicates can target nested JSON values using dot notation (`payload.profile.tier`) or Django-style notation (`payload__profile__tier`). For JSON arrays, path segments are evaluated against each element, so list-of-dicts structures can be matched naturally.
 
+### JSON path list targeting
+
+JSON list/array traversal is automatic once a path segment resolves to an array.
+
+- **All elements in an array**: use the next field name directly.
+  - `payload.items.kind` or `payload__items__kind`
+  - Matches/rewrites `kind` for every object in `items`.
+- **Specific array index**: use a numeric segment.
+  - `payload.items.0.kind` or `payload__items__0__kind`
+  - Targets only the first element.
+- **Nested arrays**: combine field and index segments as needed.
+  - `payload.groups.members.email`
+  - `payload.groups.1.members.0.email`
+
+This path behavior is shared by both `row_filters` predicates and JSON-path anonymization rules in `[rules]`.
+
 ```toml
 [row_filters."public.users"]
 retain = [

{dumpling_cli-0.4.0 → dumpling_cli-0.4.2}/README.md

@@ -1,15 +1,45 @@
-# Dumpling
+<p align="center">
+  <img src="assets/logo.svg" width="140" height="140" alt="Dumpling logo: a dumpling with steam" />
+</p>
+
+<h1 align="center">Dumpling</h1>
+
+<p align="center">
+  <strong>Sanitize SQL dumps before they go anywhere.</strong><br />
+  Turn huge <code>pg_dump</code> / SQLite / SQL Server exports into shareable, test-friendly snapshots — no DB connection, no secrets left by accident.
+</p>
+
+<p align="center">
+  <a href="https://pypi.org/project/dumpling-cli/"><img src="https://img.shields.io/pypi/v/dumpling-cli.svg" alt="PyPI version" /></a>
+  <a href="https://pypi.org/project/dumpling-cli/"><img src="https://img.shields.io/pypi/pyversions/dumpling-cli.svg" alt="Python versions" /></a>
+  <a href="https://pypi.org/project/dumpling-cli/"><img src="https://img.shields.io/pypi/l/dumpling-cli.svg" alt="PyPI license" /></a>
+  <a href="https://github.com/ababic/dumpling/actions/workflows/tests.yml"><img src="https://github.com/ababic/dumpling/actions/workflows/tests.yml/badge.svg" alt="Tests" /></a>
+  <a href="https://github.com/ababic/dumpling/actions/workflows/ci.yml"><img src="https://github.com/ababic/dumpling/actions/workflows/ci.yml/badge.svg" alt="Lint" /></a>
+  <img src="https://img.shields.io/badge/rust-stable-orange?logo=rust" alt="Rust stable" />
+</p>
+
+<p align="center">
+  <a href="https://ababic.github.io/dumpling/"><strong>Documentation</strong></a>
+  &nbsp;·&nbsp;
+  <a href="https://github.com/ababic/dumpling"><strong>GitHub</strong></a>
+</p>
+
+<p align="center">
+  <sub><em>Disclaimer: This project is entirely vibe-coded, but with strong human guidance, review, and attention to quality and safety.</em></sub>
+</p>
 
-**Dumpling** is a static anonymizer for plain SQL dumps. It supports PostgreSQL (`pg_dump` plain format), SQLite (`.dump`), and SQL Server / MSSQL (SSMS / mssql-scripter output). It lets you safely share, test with, or store database snapshots by replacing sensitive column data according to configurable rules — without ever touching a live database.
+---
+
+**Dumpling** reads plain-text SQL dumps (PostgreSQL `pg_dump`, SQLite `.dump`, SQL Server / MSSQL scripts) and rewrites sensitive columns using rules you define in TOML. Everything runs offline on files — ideal for CI, staging share-outs, and compliance-minded workflows.
 
 ## Why Dumpling?
 
-- **No live database required.** Works entirely on dump files; nothing connects to your database.
-- **Streaming and memory-efficient.** Processes dumps line by line, so even multi-gigabyte files stay manageable.
-- **Fail-safe by default.** If no configuration is found, Dumpling exits non-zero and tells you exactly where it looked. Silence is never mistaken for success.
-- **Deterministic anonymization.** Domain mappings ensure the same source value always produces the same pseudonym, keeping foreign-key relationships intact across tables.
-- **CI/CD ready.** `--check` mode, strict-coverage enforcement, JSON reports, and residual-PII scan gates plug cleanly into any pipeline.
-- **Flexible configuration.** Rules live in a `.dumplingconf` file or directly in `pyproject.toml` — no extra tooling needed.
+- **Offline by design** — works on dump files only; nothing connects to your database.
+- **Streams giant files** — line-by-line processing keeps multi‑GB dumps reasonable on modest hardware.
+- **Fails loud, not silent** — missing config exits non‑zero and lists where Dumpling looked; use `--allow-noop` only when you mean it.
+- **Stable pseudonyms** — optional domain mappings keep the same source value as the same fake value across tables (foreign keys stay consistent).
+- **Pipeline-ready** — `--check`, strict coverage, JSON reports, and residual PII scans fit pre-merge gates and release automation.
+- **Configure once** — `.dumplingconf` or `[tool.dumpling]` in `pyproject.toml`; install via **Rust** (`cargo`) or **`pip install dumpling-cli`**.
 
 ---
 
@@ -323,6 +353,22 @@ Supported predicate operators:
 
 Predicates can target nested JSON values using dot notation (`payload.profile.tier`) or Django-style notation (`payload__profile__tier`). For JSON arrays, path segments are evaluated against each element, so list-of-dicts structures can be matched naturally.
 
+### JSON path list targeting
+
+JSON list/array traversal is automatic once a path segment resolves to an array.
+
+- **All elements in an array**: use the next field name directly.
+  - `payload.items.kind` or `payload__items__kind`
+  - Matches/rewrites `kind` for every object in `items`.
+- **Specific array index**: use a numeric segment.
+  - `payload.items.0.kind` or `payload__items__0__kind`
+  - Targets only the first element.
+- **Nested arrays**: combine field and index segments as needed.
+  - `payload.groups.members.email`
+  - `payload.groups.1.members.0.email`
+
+This path behavior is shared by both `row_filters` predicates and JSON-path anonymization rules in `[rules]`.
+
 ```toml
 [row_filters."public.users"]
 retain = [

dumpling_cli-0.4.2/assets/logo.svg

@@ -0,0 +1,33 @@
+<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 128 128" role="img" aria-label="Dumpling logo">
+  <defs>
+    <linearGradient id="steam" x1="0%" y1="100%" x2="0%" y2="0%">
+      <stop offset="0%" stop-color="#e8f4fc" stop-opacity="0"/>
+      <stop offset="50%" stop-color="#cfe9fb" stop-opacity="0.85"/>
+      <stop offset="100%" stop-color="#b8dff8" stop-opacity="0"/>
+    </linearGradient>
+    <linearGradient id="dough" x1="0%" y1="0%" x2="100%" y2="100%">
+      <stop offset="0%" stop-color="#fff8ef"/>
+      <stop offset="45%" stop-color="#f4dcc4"/>
+      <stop offset="100%" stop-color="#e8b896"/>
+    </linearGradient>
+    <linearGradient id="shadow" x1="0%" y1="0%" x2="0%" y2="100%">
+      <stop offset="0%" stop-color="#c4865a" stop-opacity="0.35"/>
+      <stop offset="100%" stop-color="#8b5a3c" stop-opacity="0.15"/>
+    </linearGradient>
+  </defs>
+  <!-- Steam -->
+  <path fill="url(#steam)" d="M44 18c2-6 8-10 14-8s8 10 4 15c-3 4-2 9 2 12 5 4 5 12 0 16-6 5-16 4-20-3-2-4 0-9 4-11 3-2 3-6 0-9-4-4-4-10 0-12z"/>
+  <path fill="url(#steam)" opacity="0.75" d="M64 14c3-5 9-7 14-4 5 3 6 10 2 14-4 4-3 10 2 13 6 4 7 13 1 18-7 6-19 4-24-5-2-5 1-11 6-13 4-2 4-7 1-10-5-4-5-11 0-13z"/>
+  <path fill="url(#steam)" opacity="0.6" d="M82 20c2-5 8-8 13-5 5 3 7 10 3 15-3 4-2 9 3 12 5 3 7 11 2 16-6 6-17 5-22-3-2-4 0-9 5-11 3-2 4-6 1-9-4-4-4-10 0-13z"/>
+  <!-- Plate -->
+  <ellipse cx="64" cy="108" rx="52" ry="10" fill="#dfe8ef"/>
+  <ellipse cx="64" cy="106" rx="48" ry="8" fill="#eef4f8"/>
+  <!-- Dumpling body -->
+  <ellipse cx="64" cy="82" rx="42" ry="28" fill="url(#dough)" stroke="#d4a574" stroke-width="2"/>
+  <ellipse cx="64" cy="96" rx="38" ry="12" fill="url(#shadow)"/>
+  <!-- Pleats -->
+  <path fill="none" stroke="#c9956a" stroke-width="1.8" stroke-linecap="round" d="M34 58c6 10 14 16 30 16s24-6 30-16"/>
+  <path fill="none" stroke="#d9b08a" stroke-width="1.2" stroke-linecap="round" opacity="0.9" d="M42 54c5 8 13 13 22 13s17-5 22-13"/>
+  <!-- Highlight -->
+  <ellipse cx="48" cy="76" rx="10" ry="6" fill="#ffffff" opacity="0.35"/>
+</svg>

{dumpling_cli-0.4.0 → dumpling_cli-0.4.2}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "maturin"
 
 [project]
 name = "dumpling-cli"
-version = "0.4.0"
+version = "0.4.2"
 description = "Static anonymizer for plain SQL dumps (PostgreSQL, SQLite, SQL Server)."
 readme = "README.md"
 requires-python = ">=3.8"

{dumpling_cli-0.4.0 → dumpling_cli-0.4.2}/src/filter.rs

@@ -224,25 +224,118 @@ fn replacement_to_json_value(repl: &Replacement) -> serde_json::Value {
         .unwrap_or_else(|_| serde_json::Value::String(repl.value.clone()))
 }
 
+/// When rewriting JSON at a path, map `Replacement` back into [`serde_json::Value`] while keeping
+/// the leaf's JSON type when the strategy still returns text (e.g. `Replacement::quoted` for
+/// `string`, `hash`, etc.): numeric and boolean leaves stay JSON numbers/bools if the replacement
+/// text parses as such.
+fn coerce_json_path_replacement(
+    original: &serde_json::Value,
+    repl: &Replacement,
+) -> serde_json::Value {
+    if repl.is_null {
+        return serde_json::Value::Null;
+    }
+    match original {
+        serde_json::Value::Bool(_) => {
+            if let Some(b) = parse_loose_json_bool(&repl.value) {
+                return serde_json::Value::Bool(b);
+            }
+            if !repl.force_quoted {
+                if let Ok(v) = serde_json::from_str::<serde_json::Value>(&repl.value) {
+                    match v {
+                        serde_json::Value::Bool(b) => return serde_json::Value::Bool(b),
+                        serde_json::Value::Number(n) => {
+                            if n.as_u64() == Some(0) || n.as_i64() == Some(0) {
+                                return serde_json::Value::Bool(false);
+                            }
+                            if n.as_u64() == Some(1) || n.as_i64() == Some(1) {
+                                return serde_json::Value::Bool(true);
+                            }
+                        }
+                        _ => {}
+                    }
+                }
+            }
+            serde_json::Value::String(repl.value.clone())
+        }
+        serde_json::Value::Number(_) => {
+            if let Some(n) = parse_loose_json_number(&repl.value) {
+                return serde_json::Value::Number(n);
+            }
+            if !repl.force_quoted {
+                if let Ok(serde_json::Value::Number(n)) =
+                    serde_json::from_str::<serde_json::Value>(&repl.value)
+                {
+                    return serde_json::Value::Number(n);
+                }
+            }
+            serde_json::Value::String(repl.value.clone())
+        }
+        serde_json::Value::String(_) => {
+            if repl.force_quoted {
+                serde_json::Value::String(repl.value.clone())
+            } else {
+                serde_json::from_str(&repl.value)
+                    .unwrap_or_else(|_| serde_json::Value::String(repl.value.clone()))
+            }
+        }
+        serde_json::Value::Null => replacement_to_json_value(repl),
+        serde_json::Value::Array(_) | serde_json::Value::Object(_) => {
+            replacement_to_json_value(repl)
+        }
+    }
+}
+
+fn parse_loose_json_bool(s: &str) -> Option<bool> {
+    match s.trim().to_ascii_lowercase().as_str() {
+        "true" => Some(true),
+        "false" => Some(false),
+        _ => None,
+    }
+}
+
+fn parse_loose_json_number(s: &str) -> Option<serde_json::Number> {
+    let t = s.trim();
+    if t.is_empty() {
+        return None;
+    }
+    if let Ok(i) = t.parse::<i64>() {
+        return Some(i.into());
+    }
+    if let Ok(u) = t.parse::<u64>() {
+        return Some(u.into());
+    }
+    let f = t.parse::<f64>().ok()?;
+    serde_json::Number::from_f64(f)
+}
+
 fn apply_leaf_replacement(target: &mut serde_json::Value, repl: &Replacement) {
-    *target = replacement_to_json_value(repl);
+    let original = target.clone();
+    *target = coerce_json_path_replacement(&original, repl);
 }
 
 /// Mutate JSON document strings at configured paths using the same path semantics as predicates.
+///
+/// Returns [`None`] when `raw_json` is not valid strict JSON (same tolerance as row-filter JSON
+/// path extraction): path rules are skipped for that cell and callers should passthrough the
+/// original value unchanged.
 pub fn rewrite_json_paths_with_rules(
     registry: &AnonymizerRegistry,
     column_max_len: Option<usize>,
     json_rules: &[(Vec<String>, AnonymizerSpec)],
     raw_json: &str,
-) -> anyhow::Result<String> {
-    let mut root = serde_json::from_str::<serde_json::Value>(raw_json)?;
+) -> anyhow::Result<Option<String>> {
+    let mut root = match serde_json::from_str::<serde_json::Value>(raw_json) {
+        Ok(v) => v,
+        Err(_) => return Ok(None),
+    };
     for (path, spec) in json_rules {
         let mut apply = |original_cell: Option<String>| {
             apply_anonymizer(registry, spec, original_cell.as_deref(), column_max_len)
         };
         mutate_json_at_path(&mut root, path, &mut apply)?;
     }
-    Ok(root.to_string())
+    Ok(Some(root.to_string()))
 }
 
 fn mutate_json_at_path<F>(
@@ -457,7 +550,8 @@ fn get_cached_regex(pat: &str, case_insensitive: bool) -> regex::Regex {
 #[cfg(test)]
 mod tests {
     use super::*;
-    use crate::settings::{ResolvedConfig, RowFilterSet};
+    use crate::settings::{AnonymizerSpec, ResolvedConfig, RowFilterSet};
+    use crate::transform::AnonymizerRegistry;
     use std::collections::HashMap;
 
     #[test]
@@ -630,4 +724,139 @@ mod tests {
             &[Some(r#"{"items":[{"kind":"secondary"}]}"#.to_string())]
         ));
     }
+
+    #[test]
+    fn rewrite_json_paths_skips_non_json_cells_like_row_filters() {
+        let mut rules: HashMap<String, HashMap<String, AnonymizerSpec>> = HashMap::new();
+        let spec = AnonymizerSpec {
+            strategy: "string".to_string(),
+            salt: None,
+            min: None,
+            max: None,
+            length: Some(4),
+            min_days: None,
+            max_days: None,
+            min_seconds: None,
+            max_seconds: None,
+            domain: None,
+            unique_within_domain: None,
+            as_string: Some(true),
+            locale: None,
+            faker: None,
+            format: None,
+        };
+        rules.insert("public.t".to_string(), HashMap::new());
+        let cfg = ResolvedConfig {
+            salt: None,
+            rules,
+            row_filters: HashMap::new(),
+            column_cases: HashMap::new(),
+            sensitive_columns: HashMap::new(),
+            output_scan: crate::settings::OutputScanConfig::default(),
+            source_path: None,
+        };
+        let registry = AnonymizerRegistry::from_config(&cfg);
+        let json_rules: Vec<(Vec<String>, AnonymizerSpec)> = vec![(
+            vec!["profile".to_string(), "secret".to_string()],
+            spec.clone(),
+        )];
+        assert!(
+            rewrite_json_paths_with_rules(&registry, None, &json_rules, "{not json")
+                .unwrap()
+                .is_none()
+        );
+        let out = rewrite_json_paths_with_rules(
+            &registry,
+            None,
+            &json_rules,
+            r#"{"profile":{"secret":"x"}}"#,
+        )
+        .unwrap()
+        .expect("valid JSON should rewrite");
+        let v: serde_json::Value = serde_json::from_str(&out).unwrap();
+        assert_ne!(v["profile"]["secret"], "x");
+    }
+
+    #[test]
+    fn rewrite_json_paths_preserves_number_and_bool_leaf_types_for_quoted_replacements() {
+        let mut rules: HashMap<String, HashMap<String, AnonymizerSpec>> = HashMap::new();
+        rules.insert("public.t".to_string(), HashMap::new());
+        let cfg = ResolvedConfig {
+            salt: None,
+            rules,
+            row_filters: HashMap::new(),
+            column_cases: HashMap::new(),
+            sensitive_columns: HashMap::new(),
+            output_scan: crate::settings::OutputScanConfig::default(),
+            source_path: None,
+        };
+        let registry = AnonymizerRegistry::from_config(&cfg);
+
+        let int_spec = AnonymizerSpec {
+            strategy: "int_range".to_string(),
+            salt: None,
+            min: Some(0),
+            max: Some(9),
+            length: None,
+            min_days: None,
+            max_days: None,
+            min_seconds: None,
+            max_seconds: None,
+            domain: Some("coerce_int_leaf".to_string()),
+            unique_within_domain: None,
+            as_string: None,
+            locale: None,
+            faker: None,
+            format: None,
+        };
+        let out = rewrite_json_paths_with_rules(
+            &registry,
+            None,
+            &[(vec!["n".to_string()], int_spec)],
+            r#"{"n":1,"b":true,"s":"x"}"#,
+        )
+        .unwrap()
+        .unwrap();
+        let v: serde_json::Value = serde_json::from_str(&out).unwrap();
+        assert!(
+            v["n"].is_number(),
+            "int_range replacement should stay JSON number, got {:?}",
+            v["n"]
+        );
+        assert_eq!(v["b"], true);
+        assert_eq!(v["s"], "x");
+
+        let string_spec = AnonymizerSpec {
+            strategy: "int_range".to_string(),
+            salt: None,
+            min: Some(0),
+            max: Some(0),
+            length: None,
+            min_days: None,
+            max_days: None,
+            min_seconds: None,
+            max_seconds: None,
+            domain: Some("coerce_bool_leaf".to_string()),
+            unique_within_domain: None,
+            as_string: None,
+            locale: None,
+            faker: None,
+            format: None,
+        };
+        let out2 = rewrite_json_paths_with_rules(
+            &registry,
+            None,
+            &[(vec!["b".to_string()], string_spec)],
+            r#"{"b":false}"#,
+        )
+        .unwrap()
+        .unwrap();
+        let v2: serde_json::Value = serde_json::from_str(&out2).unwrap();
+        assert!(
+            v2["b"].is_boolean(),
+            "unquoted 0 from int_range should coerce to bool at bool leaf, got {:?}",
+            v2["b"]
+        );
+        assert_eq!(v2["b"], false);
+    }
 }

{dumpling_cli-0.4.0 → dumpling_cli-0.4.2}/src/sql.rs

@@ -562,7 +562,11 @@ impl SqlStreamProcessor {
             None => return Ok(None),
         };
         let specs: Vec<AnonymizerSpec> = json_owned.iter().map(|(_, s)| s.clone()).collect();
-        let out = rewrite_json_paths_with_rules(&self.anonymizers, col_len, &json_owned, raw)?;
+        let Some(out) =
+            rewrite_json_paths_with_rules(&self.anonymizers, col_len, &json_owned, raw)?
+        else {
+            return Ok(None);
+        };
         let repl = Replacement::quoted(out);
         Ok(Some((repl, specs)))
     }
@@ -1155,20 +1159,22 @@ struct Cell {
     original: Option<String>, // None for NULL
     was_quoted: bool,
     was_default: bool,
+    trailing_expr: Option<String>,
 }
 
 impl Cell {
     fn render_original(&self) -> String {
+        let trailing = self.trailing_expr.as_deref().unwrap_or("");
         if self.was_default {
-            return "DEFAULT".to_string();
+            return format!("DEFAULT{trailing}");
         }
         match &self.original {
-            None => "NULL".to_string(),
+            None => format!("NULL{trailing}"),
             Some(s) => {
                 if self.was_quoted {
-                    format!("'{}'", s.replace('\'', "''"))
+                    format!("'{}'{trailing}", s.replace('\'', "''"))
                 } else {
-                    s.clone()
+                    format!("{s}{trailing}")
                 }
             }
         }
@@ -1176,14 +1182,15 @@ impl Cell {
 }
 
 fn render_cell(repl: &Replacement, original: &Cell) -> String {
+    let trailing = original.trailing_expr.as_deref().unwrap_or("");
     if repl.is_null {
-        return "NULL".to_string();
+        return format!("NULL{trailing}");
     }
     let should_quote = repl.force_quoted || original.was_quoted;
     if should_quote {
-        format!("'{}'", repl.value.replace('\'', "''"))
+        format!("'{}'{trailing}", repl.value.replace('\'', "''"))
     } else {
-        repl.value.clone()
+        format!("{}{trailing}", repl.value)
     }
 }
 
@@ -1243,7 +1250,9 @@ fn parse_parenthesized_values(s: &str) -> anyhow::Result<(Vec<Cell>, usize)> {
     let mut cells: Vec<Cell> = Vec::new();
     let mut in_single = false;
     let mut buf = String::new();
+    let mut trailing_expr = String::new();
     let mut was_quoted = false;
+    let mut closed_quoted_literal = false;
     while i < chs.len() {
         let c = chs[i];
         if in_single {
@@ -1255,6 +1264,7 @@ fn parse_parenthesized_values(s: &str) -> anyhow::Result<(Vec<Cell>, usize)> {
                     continue;
                 } else {
                     in_single = false;
+                    closed_quoted_literal = true;
                     i += 1;
                     continue;
                 }
@@ -1282,17 +1292,19 @@ fn parse_parenthesized_values(s: &str) -> anyhow::Result<(Vec<Cell>, usize)> {
                 }
                 ')' => {
                     // end cell, end row
-                    let cell = finalize_cell(&buf, was_quoted);
+                    let cell = finalize_cell(&buf, was_quoted, &trailing_expr);
                     cells.push(cell);
                     i += 1;
                     return Ok((cells, i));
                 }
                 ',' => {
                     // end cell
-                    let cell = finalize_cell(&buf, was_quoted);
+                    let cell = finalize_cell(&buf, was_quoted, &trailing_expr);
                     cells.push(cell);
                     buf.clear();
+                    trailing_expr.clear();
                     was_quoted = false;
+                    closed_quoted_literal = false;
                     i += 1;
                     // consume following spaces
                     while i < chs.len() && chs[i].is_whitespace() {
@@ -1300,11 +1312,19 @@ fn parse_parenthesized_values(s: &str) -> anyhow::Result<(Vec<Cell>, usize)> {
                     }
                 }
                 c if c.is_whitespace() => {
-                    // skip insignificant whitespace between tokens when unquoted
+                    // Preserve whitespace after a quoted literal so explicit SQL casts stay intact.
+                    if was_quoted && closed_quoted_literal {
+                        trailing_expr.push(c);
+                    }
+                    // Skip insignificant whitespace between tokens when unquoted.
                     i += 1;
                 }
                 other => {
-                    buf.push(other);
+                    if was_quoted && closed_quoted_literal {
+                        trailing_expr.push(other);
+                    } else {
+                        buf.push(other);
+                    }
                     i += 1;
                 }
             }
@@ -1313,12 +1333,21 @@ fn parse_parenthesized_values(s: &str) -> anyhow::Result<(Vec<Cell>, usize)> {
     anyhow::bail!("unterminated values row")
 }
 
-fn finalize_cell(buf: &str, was_quoted: bool) -> Cell {
+fn finalize_cell(buf: &str, was_quoted: bool, trailing_expr: &str) -> Cell {
+    let trailing = {
+        let t = trailing_expr.trim();
+        if t.is_empty() {
+            None
+        } else {
+            Some(t.to_string())
+        }
+    };
     if was_quoted {
         Cell {
             original: Some(buf.to_string()),
             was_quoted: true,
             was_default: false,
+            trailing_expr: trailing,
         }
     } else {
         let t = buf.trim();
@@ -1327,18 +1356,21 @@ fn finalize_cell(buf: &str, was_quoted: bool) -> Cell {
                 original: None,
                 was_quoted: false,
                 was_default: false,
+                trailing_expr: None,
             }
         } else if t.eq_ignore_ascii_case("default") {
             Cell {
                 original: None,
                 was_quoted: false,
                 was_default: true,
+                trailing_expr: None,
             }
         } else {
             Cell {
                 original: Some(t.to_string()),
                 was_quoted: false,
                 was_default: false,
+                trailing_expr: None,
             }
         }
     }
@@ -2370,6 +2402,240 @@ COPY public.events (id, payload) FROM stdin;
         );
     }
 
+    #[test]
+    fn pipeline_json_path_rules_passthrough_non_json_cells() {
+        let mut rules: HashMap<String, HashMap<String, AnonymizerSpec>> = HashMap::new();
+        let mut cols: HashMap<String, AnonymizerSpec> = HashMap::new();
+        cols.insert(
+            "payload.profile.secret".to_string(),
+            AnonymizerSpec {
+                strategy: "string".to_string(),
+                salt: None,
+                min: None,
+                max: None,
+                length: Some(8),
+                min_days: None,
+                max_days: None,
+                min_seconds: None,
+                max_seconds: None,
+                domain: Some("secrets".to_string()),
+                unique_within_domain: None,
+                as_string: Some(true),
+                locale: None,
+                faker: None,
+                format: None,
+            },
+        );
+        rules.insert("public.events".to_string(), cols);
+        let cfg = ResolvedConfig {
+            salt: None,
+            rules,
+            row_filters: HashMap::new(),
+            column_cases: HashMap::new(),
+            sensitive_columns: HashMap::new(),
+            output_scan: crate::settings::OutputScanConfig::default(),
+            source_path: None,
+        };
+        let reg = AnonymizerRegistry::from_config(&cfg);
+        let mut proc =
+            SqlStreamProcessor::new(reg, cfg, Vec::new(), Vec::new(), None, DumpFormat::Postgres);
+        let input = r#"
+CREATE TABLE public.events (id int, payload jsonb);
+INSERT INTO public.events (id, payload) VALUES
+  (1, '{not strict json}'),
+  (2, '{"profile":{"tier":"gold","secret":"alpha"}}');
+
+COPY public.events (id, payload) FROM stdin;
+3	{not strict json}
+4	{"profile":{"tier":"gold","secret":"alpha"}}
+\.
+"#;
+        let mut reader = std::io::BufReader::new(input.as_bytes());
+        let mut out = Vec::new();
+        proc.process(&mut reader, &mut out).unwrap();
+        let s = String::from_utf8(out).unwrap();
+        assert!(
+            s.contains("(1, '{not strict json}')"),
+            "non-JSON INSERT cell should passthrough unchanged, got:\n{s}"
+        );
+        assert!(
+            !s.contains("alpha"),
+            "valid JSON INSERT row should still anonymize nested paths, got:\n{s}"
+        );
+        assert!(
+            s.contains("\n3\t{not strict json}\n"),
+            "non-JSON COPY cell should passthrough unchanged, got:\n{s}"
+        );
+        assert!(
+            !s.contains("\n4\t{\"profile\":{\"tier\":\"gold\",\"secret\":\"alpha\"}}\n"),
+            "valid JSON COPY row should anonymize nested secret, got:\n{s}"
+        );
+    }
+
+    #[test]
+    fn pipeline_json_path_int_range_preserves_json_number_type() {
+        let mut rules: HashMap<String, HashMap<String, AnonymizerSpec>> = HashMap::new();
+        let mut cols: HashMap<String, AnonymizerSpec> = HashMap::new();
+        cols.insert(
+            "payload.score".to_string(),
+            AnonymizerSpec {
+                strategy: "int_range".to_string(),
+                salt: None,
+                min: Some(0),
+                max: Some(100),
+                length: None,
+                min_days: None,
+                max_days: None,
+                min_seconds: None,
+                max_seconds: None,
+                domain: Some("pipeline_json_num".to_string()),
+                unique_within_domain: None,
+                as_string: None,
+                locale: None,
+                faker: None,
+                format: None,
+            },
+        );
+        rules.insert("public.events".to_string(), cols);
+        let cfg = ResolvedConfig {
+            salt: None,
+            rules,
+            row_filters: HashMap::new(),
+            column_cases: HashMap::new(),
+            sensitive_columns: HashMap::new(),
+            output_scan: crate::settings::OutputScanConfig::default(),
+            source_path: None,
+        };
+        let reg = AnonymizerRegistry::from_config(&cfg);
+        let mut proc =
+            SqlStreamProcessor::new(reg, cfg, Vec::new(), Vec::new(), None, DumpFormat::Postgres);
+        let input = r#"
+CREATE TABLE public.events (id int, payload jsonb);
+INSERT INTO public.events (id, payload) VALUES
+  (1, '{"score":42,"label":"x"}');
+
+COPY public.events (id, payload) FROM stdin;
+2	{"score":42,"label":"x"}
+\.
+"#;
+        let mut reader = std::io::BufReader::new(input.as_bytes());
+        let mut out = Vec::new();
+        proc.process(&mut reader, &mut out).unwrap();
+        let s = String::from_utf8(out).unwrap();
+        let insert_pos = s.find("INSERT INTO public.events").unwrap();
+        let insert_tail = &s[insert_pos..];
+        let insert_end = insert_tail.find(";\n").unwrap() + insert_pos;
+        let ins_stmt = &s[insert_pos..=insert_end];
+        let vals_idx = ins_stmt.to_uppercase().find("VALUES").unwrap();
+        let ins_block = strip_trailing_semicolon(ins_stmt[vals_idx + "VALUES".len()..].trim());
+        let ins_rows = parse_values_rows(ins_block).unwrap();
+        let copy_line = s
+            .lines()
+            .find(|l| l.starts_with("2\t{"))
+            .expect("expected COPY data row");
+        let copy_json = copy_line.split_once('\t').unwrap().1;
+        let v_ins =
+            serde_json::from_str::<serde_json::Value>(ins_rows[0][1].original.as_ref().unwrap())
+                .unwrap();
+        let v_copy = serde_json::from_str::<serde_json::Value>(copy_json).unwrap();
+        assert!(
+            v_ins["score"].is_number(),
+            "INSERT payload.score should remain JSON number, got {:?}",
+            v_ins["score"]
+        );
+        assert!(
+            v_copy["score"].is_number(),
+            "COPY payload.score should remain JSON number, got {:?}",
+            v_copy["score"]
+        );
+        assert_eq!(v_ins["score"], v_copy["score"]);
+        assert_eq!(v_ins["label"], "x");
+    }
+
+    #[test]
+    fn parse_values_rows_tracks_trailing_cast_for_quoted_literals() {
+        let rows =
+            parse_values_rows("(1, '{\"profile\":{\"secret\":\"alpha\"}}'::jsonb, 'note'::text)")
+                .unwrap();
+        assert_eq!(rows.len(), 1);
+        assert_eq!(rows[0].len(), 3);
+        assert_eq!(
+            rows[0][1].original.as_deref(),
+            Some("{\"profile\":{\"secret\":\"alpha\"}}")
+        );
+        assert_eq!(rows[0][1].trailing_expr.as_deref(), Some("::jsonb"));
+        assert_eq!(rows[0][2].original.as_deref(), Some("note"));
+        assert_eq!(rows[0][2].trailing_expr.as_deref(), Some("::text"));
+    }
+
+    #[test]
+    fn pipeline_anonymizes_nested_json_paths_for_jsonb_cast_insert_rows() {
+        let mut rules: HashMap<String, HashMap<String, AnonymizerSpec>> = HashMap::new();
+        let mut cols: HashMap<String, AnonymizerSpec> = HashMap::new();
+        cols.insert(
+            "payload.profile.secret".to_string(),
+            AnonymizerSpec {
+                strategy: "string".to_string(),
+                salt: None,
+                min: None,
+                max: None,
+                length: Some(8),
+                min_days: None,
+                max_days: None,
+                min_seconds: None,
+                max_seconds: None,
+                domain: Some("secrets".to_string()),
+                unique_within_domain: None,
+                as_string: Some(true),
+                locale: None,
+                faker: None,
+                format: None,
+            },
+        );
+        rules.insert("public.events".to_string(), cols);
+        let cfg = ResolvedConfig {
+            salt: None,
+            rules,
+            row_filters: HashMap::new(),
+            column_cases: HashMap::new(),
+            sensitive_columns: HashMap::new(),
+            output_scan: crate::settings::OutputScanConfig::default(),
+            source_path: None,
+        };
+        let reg = AnonymizerRegistry::from_config(&cfg);
+        let mut proc =
+            SqlStreamProcessor::new(reg, cfg, Vec::new(), Vec::new(), None, DumpFormat::Postgres);
+        let input = r#"
+CREATE TABLE public.events (id int, payload jsonb);
+INSERT INTO public.events (id, payload) VALUES
+  (1, '{"profile":{"tier":"gold","secret":"alpha"}}'::jsonb),
+  (2, '{"profile":{"tier":"gold","secret":"alpha"}}'::jsonb);
+"#;
+        let mut reader = std::io::BufReader::new(input.as_bytes());
+        let mut out = Vec::new();
+        proc.process(&mut reader, &mut out).unwrap();
+        let s = String::from_utf8(out).unwrap();
+        assert!(!s.contains("alpha"), "nested secret should be anonymized");
+        assert!(s.contains("::jsonb"), "jsonb cast should be preserved");
+
+        let insert_pos = s.find("INSERT INTO public.events").unwrap();
+        let insert_tail = &s[insert_pos..];
+        let insert_end = insert_tail.find(";\n").unwrap() + insert_pos;
+        let ins_stmt = &s[insert_pos..=insert_end];
+        let vals_idx = ins_stmt.to_uppercase().find("VALUES").unwrap();
+        let ins_block = strip_trailing_semicolon(ins_stmt[vals_idx + "VALUES".len()..].trim());
+        let ins_rows = parse_values_rows(ins_block).unwrap();
+        assert_eq!(ins_rows[0][1].trailing_expr.as_deref(), Some("::jsonb"));
+        assert_eq!(ins_rows[1][1].trailing_expr.as_deref(), Some("::jsonb"));
+        let v0 =
+            serde_json::from_str::<serde_json::Value>(ins_rows[0][1].original.as_ref().unwrap())
+                .unwrap();
+        let v1 =
+            serde_json::from_str::<serde_json::Value>(ins_rows[1][1].original.as_ref().unwrap())
+                .unwrap();
+        assert_eq!(v0["profile"]["secret"], v1["profile"]["secret"]);
+    }
+
     #[test]
     fn generated_values_fit_length_restricted_columns_from_create_table() {
         let mut rules: HashMap<String, HashMap<String, AnonymizerSpec>> = HashMap::new();