csvnorm 1.2.14__tar.gz → 1.2.16__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (27) hide show
  1. {csvnorm-1.2.14/src/csvnorm.egg-info → csvnorm-1.2.16}/PKG-INFO +20 -2
  2. {csvnorm-1.2.14 → csvnorm-1.2.16}/README.md +19 -1
  3. {csvnorm-1.2.14 → csvnorm-1.2.16}/pyproject.toml +1 -1
  4. {csvnorm-1.2.14 → csvnorm-1.2.16}/src/csvnorm/cli.py +73 -28
  5. {csvnorm-1.2.14 → csvnorm-1.2.16}/src/csvnorm/core.py +3 -0
  6. {csvnorm-1.2.14 → csvnorm-1.2.16}/src/csvnorm/ui.py +12 -5
  7. {csvnorm-1.2.14 → csvnorm-1.2.16/src/csvnorm.egg-info}/PKG-INFO +20 -2
  8. {csvnorm-1.2.14 → csvnorm-1.2.16}/LICENSE +0 -0
  9. {csvnorm-1.2.14 → csvnorm-1.2.16}/setup.cfg +0 -0
  10. {csvnorm-1.2.14 → csvnorm-1.2.16}/src/csvnorm/__init__.py +0 -0
  11. {csvnorm-1.2.14 → csvnorm-1.2.16}/src/csvnorm/__main__.py +0 -0
  12. {csvnorm-1.2.14 → csvnorm-1.2.16}/src/csvnorm/encoding.py +0 -0
  13. {csvnorm-1.2.14 → csvnorm-1.2.16}/src/csvnorm/mojibake.py +0 -0
  14. {csvnorm-1.2.14 → csvnorm-1.2.16}/src/csvnorm/utils.py +0 -0
  15. {csvnorm-1.2.14 → csvnorm-1.2.16}/src/csvnorm/validation.py +0 -0
  16. {csvnorm-1.2.14 → csvnorm-1.2.16}/src/csvnorm.egg-info/SOURCES.txt +0 -0
  17. {csvnorm-1.2.14 → csvnorm-1.2.16}/src/csvnorm.egg-info/dependency_links.txt +0 -0
  18. {csvnorm-1.2.14 → csvnorm-1.2.16}/src/csvnorm.egg-info/entry_points.txt +0 -0
  19. {csvnorm-1.2.14 → csvnorm-1.2.16}/src/csvnorm.egg-info/requires.txt +0 -0
  20. {csvnorm-1.2.14 → csvnorm-1.2.16}/src/csvnorm.egg-info/top_level.txt +0 -0
  21. {csvnorm-1.2.14 → csvnorm-1.2.16}/tests/test_cli.py +0 -0
  22. {csvnorm-1.2.14 → csvnorm-1.2.16}/tests/test_core_helpers.py +0 -0
  23. {csvnorm-1.2.14 → csvnorm-1.2.16}/tests/test_encoding.py +0 -0
  24. {csvnorm-1.2.14 → csvnorm-1.2.16}/tests/test_integration.py +0 -0
  25. {csvnorm-1.2.14 → csvnorm-1.2.16}/tests/test_mojibake.py +0 -0
  26. {csvnorm-1.2.14 → csvnorm-1.2.16}/tests/test_utils.py +0 -0
  27. {csvnorm-1.2.14 → csvnorm-1.2.16}/tests/test_validation.py +0 -0
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: csvnorm
3
- Version: 1.2.14
3
+ Version: 1.2.16
4
4
  Summary: A command-line utility to validate and normalize CSV files
5
5
  Author-email: aborruso <aborruso@gmail.com>
6
6
  License: MIT License
@@ -126,9 +126,10 @@ This tool prepares CSV files for **basic exploratory data analysis (EDA)**, not
126
126
 
127
127
  ```bash
128
128
  csvnorm input.csv [options]
129
+ csvnorm - # read from stdin
129
130
  ```
130
131
 
131
- **By default, csvnorm writes to stdout** for easy piping and composability with other Unix tools.
132
+ **By default, csvnorm writes to stdout** for easy piping and composability with other Unix tools. Use `-` as input to read from stdin.
132
133
 
133
134
  ### Options
134
135
 
@@ -153,6 +154,11 @@ csvnorm input.csv [options]
153
154
  # Default: output to stdout
154
155
  csvnorm data.csv
155
156
 
157
+ # Read from stdin
158
+ cat data.csv | csvnorm -
159
+ curl -s https://example.com/data.csv | csvnorm - -o clean.csv
160
+ csvnorm - --check < data.csv
161
+
156
162
  # Preview first rows
157
163
  csvnorm data.csv | head -20
158
164
 
@@ -373,3 +379,15 @@ See [docs/COMMUNICATION.md](docs/COMMUNICATION.md) for details.
373
379
  ## License
374
380
 
375
381
  MIT License (c) 2026 aborruso@gmail.com - See LICENSE file for details
382
+
383
+ ## Acknowledgments
384
+
385
+ csvnorm is built on top of excellent open-source libraries:
386
+
387
+ - [**charset-normalizer**](https://github.com/Ousret/charset_normalizer) - Universal encoding detection
388
+ - [**DuckDB**](https://duckdb.org/) - Fast in-process analytical database for CSV processing
389
+ - [**ftfy**](https://github.com/rspeer/python-ftfy) - Fixes mojibake and other text encoding issues
390
+ - [**Rich**](https://github.com/Textualize/rich) - Beautiful terminal output formatting
391
+ - [**rich-argparse**](https://github.com/hamdanal/rich-argparse) - Enhanced CLI help formatting
392
+
393
+ We are grateful to the creators and maintainers of these libraries, without whom csvnorm would not exist and function.
@@ -66,9 +66,10 @@ This tool prepares CSV files for **basic exploratory data analysis (EDA)**, not
66
66
 
67
67
  ```bash
68
68
  csvnorm input.csv [options]
69
+ csvnorm - # read from stdin
69
70
  ```
70
71
 
71
- **By default, csvnorm writes to stdout** for easy piping and composability with other Unix tools.
72
+ **By default, csvnorm writes to stdout** for easy piping and composability with other Unix tools. Use `-` as input to read from stdin.
72
73
 
73
74
  ### Options
74
75
 
@@ -93,6 +94,11 @@ csvnorm input.csv [options]
93
94
  # Default: output to stdout
94
95
  csvnorm data.csv
95
96
 
97
+ # Read from stdin
98
+ cat data.csv | csvnorm -
99
+ curl -s https://example.com/data.csv | csvnorm - -o clean.csv
100
+ csvnorm - --check < data.csv
101
+
96
102
  # Preview first rows
97
103
  csvnorm data.csv | head -20
98
104
 
@@ -313,3 +319,15 @@ See [docs/COMMUNICATION.md](docs/COMMUNICATION.md) for details.
313
319
  ## License
314
320
 
315
321
  MIT License (c) 2026 aborruso@gmail.com - See LICENSE file for details
322
+
323
+ ## Acknowledgments
324
+
325
+ csvnorm is built on top of excellent open-source libraries:
326
+
327
+ - [**charset-normalizer**](https://github.com/Ousret/charset_normalizer) - Universal encoding detection
328
+ - [**DuckDB**](https://duckdb.org/) - Fast in-process analytical database for CSV processing
329
+ - [**ftfy**](https://github.com/rspeer/python-ftfy) - Fixes mojibake and other text encoding issues
330
+ - [**Rich**](https://github.com/Textualize/rich) - Beautiful terminal output formatting
331
+ - [**rich-argparse**](https://github.com/hamdanal/rich-argparse) - Enhanced CLI help formatting
332
+
333
+ We are grateful to the creators and maintainers of these libraries, without whom csvnorm would not exist and function.
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
4
4
 
5
5
  [project]
6
6
  name = "csvnorm"
7
- version = "1.2.14"
7
+ version = "1.2.16"
8
8
  description = "A command-line utility to validate and normalize CSV files"
9
9
  readme = "README.md"
10
10
  requires-python = ">=3.9"
@@ -6,12 +6,12 @@ from pathlib import Path
6
6
  from typing import Optional
7
7
 
8
8
  from rich.console import Console
9
- from rich_argparse import RichHelpFormatter
9
+ from rich_argparse import RawDescriptionRichHelpFormatter
10
10
 
11
11
  from importlib.metadata import version
12
12
  from csvnorm.core import process_csv
13
13
  from csvnorm.mojibake import DEFAULT_MOJIBAKE_SAMPLE
14
- from csvnorm.utils import setup_logger
14
+ from csvnorm.utils import setup_logger, to_snake_case
15
15
 
16
16
  console = Console()
17
17
 
@@ -23,30 +23,6 @@ def show_banner() -> None:
23
23
  console.print()
24
24
 
25
25
 
26
- def show_examples() -> None:
27
- console.print()
28
- console.print("[orange1]Examples:[/orange1]")
29
- console.print(" # Base command with stdout output")
30
- console.print(" [cyan]csvnorm data.csv[/cyan]")
31
- console.print(" # Output to stdout")
32
- console.print(" [cyan]csvnorm data.csv -o output.csv[/cyan]")
33
- console.print(" # Write to file")
34
- console.print(" [cyan]csvnorm data.csv > output.csv[/cyan]")
35
- console.print(" # Shell redirect")
36
- console.print(" [cyan]csvnorm data.csv | head -20[/cyan]")
37
- console.print(" # Preview with pipe")
38
- console.print(" [cyan]csvnorm data.csv --check[/cyan]")
39
- console.print(" # Validate without processing")
40
- console.print(" [cyan]csvnorm data.csv --strict | process.sh[/cyan]")
41
- console.print(" # Fail-fast mode for pipelines")
42
- console.print(" [cyan]csvnorm data.csv -d ';' -o output.csv[/cyan]")
43
- console.print(" # Custom delimiter")
44
- console.print(" [cyan]csvnorm data.csv --skip-rows 2 -o out.csv[/cyan]")
45
- console.print(" # Skip first 2 rows")
46
- console.print(" [cyan]csvnorm https://example.com/data.csv -o processed.csv[/cyan]")
47
- console.print(" # Read from stdin")
48
- console.print(" [cyan]cat data.csv | csvnorm -[/cyan]")
49
-
50
26
 
51
27
  class VersionAction(argparse.Action):
52
28
  """Custom action to show banner with version."""
@@ -74,12 +50,28 @@ class VersionAction(argparse.Action):
74
50
  parser.exit()
75
51
 
76
52
 
53
+ _EXAMPLES = """
54
+ Examples:
55
+ csvnorm data.csv Output to stdout (default)
56
+ csvnorm data.csv -o output.csv Write to file
57
+ csvnorm data.csv > output.csv Shell redirect
58
+ csvnorm data.csv | head -20 Pipe to another command
59
+ csvnorm data.csv --check Validate only, no output
60
+ csvnorm data.csv --strict | process.sh Fail-fast pipeline
61
+ csvnorm data.csv -d ';' -o output.csv Custom delimiter
62
+ csvnorm data.csv --skip-rows 2 -o out.csv Skip metadata rows
63
+ cat data.csv | csvnorm - Read from stdin
64
+ csvnorm https://example.com/data.csv -o out.csv Remote URL
65
+ """
66
+
67
+
77
68
  def create_parser() -> argparse.ArgumentParser:
78
69
  """Create and return the argument parser."""
79
70
  parser = argparse.ArgumentParser(
80
71
  prog="csvnorm",
81
72
  description="Validate and normalize CSV files for exploratory data analysis",
82
- formatter_class=RichHelpFormatter,
73
+ formatter_class=RawDescriptionRichHelpFormatter,
74
+ epilog=_EXAMPLES,
83
75
  )
84
76
 
85
77
  parser.add_argument(
@@ -131,6 +123,17 @@ def create_parser() -> argparse.ArgumentParser:
131
123
  help="Write to file instead of stdout (default: stdout)",
132
124
  )
133
125
 
126
+ parser.add_argument(
127
+ "-O",
128
+ "--auto-output",
129
+ action="store_true",
130
+ help=(
131
+ "Auto-derive output filename in snake_case from input filename, "
132
+ "saved in the same directory. Example: 'My Data.csv' → 'my_data.csv'. "
133
+ "Asks confirmation if output already exists (unless -f is set)."
134
+ ),
135
+ )
136
+
134
137
  parser.add_argument(
135
138
  "--fix-mojibake",
136
139
  nargs="?",
@@ -213,7 +216,6 @@ def main(argv: Optional[list[str]] = None) -> int:
213
216
  )
214
217
  console.print()
215
218
  parser.print_help()
216
- show_examples()
217
219
  return 0 if argv else 2
218
220
 
219
221
  args = parser.parse_args(argv)
@@ -230,6 +232,49 @@ def main(argv: Optional[list[str]] = None) -> int:
230
232
  console.print("[red]Error:[/red] --skip-rows must be non-negative", style="red")
231
233
  return 1
232
234
 
235
+ # Resolve --auto-output into output_file
236
+ if args.auto_output:
237
+ if args.output_file:
238
+ console.print("[red]Error:[/red] -O/--auto-output cannot be used with -o", style="red")
239
+ return 1
240
+ if args.input_file == "-":
241
+ console.print("[red]Error:[/red] -O/--auto-output cannot be used with stdin input", style="red")
242
+ return 1
243
+ if args.input_file.startswith("http://") or args.input_file.startswith("https://"):
244
+ console.print("[red]Error:[/red] -O/--auto-output cannot be used with URL input", style="red")
245
+ return 1
246
+
247
+ input_path = Path(args.input_file)
248
+ snake_stem = to_snake_case(input_path.stem)
249
+ auto_output = input_path.parent / f"{snake_stem}.csv"
250
+
251
+ if auto_output.resolve() == input_path.resolve():
252
+ console.print(
253
+ f"[yellow]Warning:[/yellow] filename is already in snake_case ({input_path.name}). "
254
+ "Use -o to specify a different output path.",
255
+ style="yellow",
256
+ )
257
+ return 1
258
+
259
+ if auto_output.exists() and not args.force:
260
+ if not sys.stdin.isatty():
261
+ console.print(
262
+ f"[red]Error:[/red] Output file already exists: {auto_output}\n"
263
+ "Use --force to overwrite."
264
+ )
265
+ return 1
266
+ console.print(f"[yellow]Output file already exists:[/yellow] {auto_output}")
267
+ try:
268
+ confirm = input("Overwrite? [y/N] ").strip().lower()
269
+ except (EOFError, KeyboardInterrupt):
270
+ console.print("\nAborted.")
271
+ return 1
272
+ if confirm != "y":
273
+ console.print("Aborted.")
274
+ return 0
275
+
276
+ args.output_file = auto_output
277
+
233
278
  # Validate incompatible flag combinations
234
279
  if args.check and args.output_file:
235
280
  console.print(
@@ -370,6 +370,7 @@ def _emit_stdout_output(
370
370
  keep_names=summary["keep_names"],
371
371
  output_display=summary["output_display"],
372
372
  out_console=stderr_console,
373
+ reject_count=summary.get("reject_count", 0),
373
374
  )
374
375
 
375
376
  # Return exit code 1 if validation errors occurred (warning already shown before output)
@@ -576,6 +577,7 @@ def _compute_and_show_output(
576
577
  "delimiter": delimiter,
577
578
  "keep_names": keep_names,
578
579
  "output_display": "stdout",
580
+ "reject_count": reject_count - 1 if has_validation_errors else 0,
579
581
  }
580
582
  return _emit_stdout_output(
581
583
  actual_output_file,
@@ -597,6 +599,7 @@ def _compute_and_show_output(
597
599
  output_size=output_size,
598
600
  delimiter=delimiter,
599
601
  keep_names=keep_names,
602
+ reject_count=reject_count - 1 if has_validation_errors else 0,
600
603
  )
601
604
 
602
605
  if has_validation_errors:
@@ -11,6 +11,7 @@ from csvnorm.encoding import needs_conversion
11
11
  from csvnorm.utils import format_file_size
12
12
 
13
13
  console = Console()
14
+ err_console = Console(stderr=True)
14
15
 
15
16
 
16
17
  def show_error_panel(message: str, title: str = "Error") -> None:
@@ -20,7 +21,7 @@ def show_error_panel(message: str, title: str = "Error") -> None:
20
21
  message: Error message to display.
21
22
  title: Panel title (default: "Error").
22
23
  """
23
- console.print(Panel(f"[bold red]{title}:[/bold red] {message}", border_style="red"))
24
+ err_console.print(Panel(f"[bold red]{title}:[/bold red] {message}", border_style="red"))
24
25
 
25
26
 
26
27
  def show_warning_panel(message: str, title: str = "Warning") -> None:
@@ -30,7 +31,7 @@ def show_warning_panel(message: str, title: str = "Warning") -> None:
30
31
  message: Warning message to display.
31
32
  title: Panel title (default: "Warning").
32
33
  """
33
- console.print(
34
+ err_console.print(
34
35
  Panel(f"[bold yellow]{title}:[/bold yellow] {message}", border_style="yellow")
35
36
  )
36
37
 
@@ -49,6 +50,7 @@ def show_success_table(
49
50
  keep_names: bool,
50
51
  output_display: Optional[str] = None,
51
52
  out_console: Optional[Console] = None,
53
+ reject_count: int = 0,
52
54
  ) -> None:
53
55
  """Display success summary table with processing results.
54
56
 
@@ -66,7 +68,10 @@ def show_success_table(
66
68
  keep_names: Whether original column names were kept.
67
69
  """
68
70
  table = Table(show_header=False, box=None, padding=(0, 1))
69
- table.add_row("[green]✓[/green] Success", "")
71
+ if reject_count > 0:
72
+ table.add_row("[yellow]⚠[/yellow] Done with errors", "")
73
+ else:
74
+ table.add_row("[green]✓[/green] Success", "")
70
75
  table.add_row("Input:", f"[cyan]{input_file}[/cyan]")
71
76
  output_value = output_display if output_display is not None else str(output_file)
72
77
  table.add_row("Output:", f"[cyan]{output_value}[/cyan]")
@@ -95,6 +100,8 @@ def show_success_table(
95
100
  # Optional fields
96
101
  if delimiter != ",":
97
102
  table.add_row("Delimiter:", repr(delimiter))
103
+ if reject_count > 0:
104
+ table.add_row("Rejected rows:", f"[yellow]{reject_count}[/yellow]")
98
105
  if not keep_names:
99
106
  table.add_row("Headers:", "normalized to snake_case")
100
107
 
@@ -112,9 +119,9 @@ def show_validation_error_panel(
112
119
  reject_count: Number of rejected rows (including header).
113
120
  error_types: List of error type descriptions.
114
121
  reject_file: Path to reject errors CSV file.
115
- console_out: Optional console to use (defaults to module console).
122
+ console_out: Optional console to use (defaults to stderr).
116
123
  """
117
- target_console = console_out or console
124
+ target_console = console_out or err_console
118
125
  target_console.print()
119
126
  error_lines = []
120
127
  error_lines.append("[bold red]Validation Errors:[/bold red]")
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: csvnorm
3
- Version: 1.2.14
3
+ Version: 1.2.16
4
4
  Summary: A command-line utility to validate and normalize CSV files
5
5
  Author-email: aborruso <aborruso@gmail.com>
6
6
  License: MIT License
@@ -126,9 +126,10 @@ This tool prepares CSV files for **basic exploratory data analysis (EDA)**, not
126
126
 
127
127
  ```bash
128
128
  csvnorm input.csv [options]
129
+ csvnorm - # read from stdin
129
130
  ```
130
131
 
131
- **By default, csvnorm writes to stdout** for easy piping and composability with other Unix tools.
132
+ **By default, csvnorm writes to stdout** for easy piping and composability with other Unix tools. Use `-` as input to read from stdin.
132
133
 
133
134
  ### Options
134
135
 
@@ -153,6 +154,11 @@ csvnorm input.csv [options]
153
154
  # Default: output to stdout
154
155
  csvnorm data.csv
155
156
 
157
+ # Read from stdin
158
+ cat data.csv | csvnorm -
159
+ curl -s https://example.com/data.csv | csvnorm - -o clean.csv
160
+ csvnorm - --check < data.csv
161
+
156
162
  # Preview first rows
157
163
  csvnorm data.csv | head -20
158
164
 
@@ -373,3 +379,15 @@ See [docs/COMMUNICATION.md](docs/COMMUNICATION.md) for details.
373
379
  ## License
374
380
 
375
381
  MIT License (c) 2026 aborruso@gmail.com - See LICENSE file for details
382
+
383
+ ## Acknowledgments
384
+
385
+ csvnorm is built on top of excellent open-source libraries:
386
+
387
+ - [**charset-normalizer**](https://github.com/Ousret/charset_normalizer) - Universal encoding detection
388
+ - [**DuckDB**](https://duckdb.org/) - Fast in-process analytical database for CSV processing
389
+ - [**ftfy**](https://github.com/rspeer/python-ftfy) - Fixes mojibake and other text encoding issues
390
+ - [**Rich**](https://github.com/Textualize/rich) - Beautiful terminal output formatting
391
+ - [**rich-argparse**](https://github.com/hamdanal/rich-argparse) - Enhanced CLI help formatting
392
+
393
+ We are grateful to the creators and maintainers of these libraries, without whom csvnorm would not exist and function.
File without changes
File without changes
File without changes
File without changes
File without changes