csvnorm 1.2.14__tar.gz → 1.2.15__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {csvnorm-1.2.14/src/csvnorm.egg-info → csvnorm-1.2.15}/PKG-INFO +20 -2
- {csvnorm-1.2.14 → csvnorm-1.2.15}/README.md +19 -1
- {csvnorm-1.2.14 → csvnorm-1.2.15}/pyproject.toml +1 -1
- {csvnorm-1.2.14 → csvnorm-1.2.15}/src/csvnorm/cli.py +51 -3
- {csvnorm-1.2.14 → csvnorm-1.2.15}/src/csvnorm/core.py +3 -0
- {csvnorm-1.2.14 → csvnorm-1.2.15}/src/csvnorm/ui.py +7 -1
- {csvnorm-1.2.14 → csvnorm-1.2.15/src/csvnorm.egg-info}/PKG-INFO +20 -2
- {csvnorm-1.2.14 → csvnorm-1.2.15}/LICENSE +0 -0
- {csvnorm-1.2.14 → csvnorm-1.2.15}/setup.cfg +0 -0
- {csvnorm-1.2.14 → csvnorm-1.2.15}/src/csvnorm/__init__.py +0 -0
- {csvnorm-1.2.14 → csvnorm-1.2.15}/src/csvnorm/__main__.py +0 -0
- {csvnorm-1.2.14 → csvnorm-1.2.15}/src/csvnorm/encoding.py +0 -0
- {csvnorm-1.2.14 → csvnorm-1.2.15}/src/csvnorm/mojibake.py +0 -0
- {csvnorm-1.2.14 → csvnorm-1.2.15}/src/csvnorm/utils.py +0 -0
- {csvnorm-1.2.14 → csvnorm-1.2.15}/src/csvnorm/validation.py +0 -0
- {csvnorm-1.2.14 → csvnorm-1.2.15}/src/csvnorm.egg-info/SOURCES.txt +0 -0
- {csvnorm-1.2.14 → csvnorm-1.2.15}/src/csvnorm.egg-info/dependency_links.txt +0 -0
- {csvnorm-1.2.14 → csvnorm-1.2.15}/src/csvnorm.egg-info/entry_points.txt +0 -0
- {csvnorm-1.2.14 → csvnorm-1.2.15}/src/csvnorm.egg-info/requires.txt +0 -0
- {csvnorm-1.2.14 → csvnorm-1.2.15}/src/csvnorm.egg-info/top_level.txt +0 -0
- {csvnorm-1.2.14 → csvnorm-1.2.15}/tests/test_cli.py +0 -0
- {csvnorm-1.2.14 → csvnorm-1.2.15}/tests/test_core_helpers.py +0 -0
- {csvnorm-1.2.14 → csvnorm-1.2.15}/tests/test_encoding.py +0 -0
- {csvnorm-1.2.14 → csvnorm-1.2.15}/tests/test_integration.py +0 -0
- {csvnorm-1.2.14 → csvnorm-1.2.15}/tests/test_mojibake.py +0 -0
- {csvnorm-1.2.14 → csvnorm-1.2.15}/tests/test_utils.py +0 -0
- {csvnorm-1.2.14 → csvnorm-1.2.15}/tests/test_validation.py +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: csvnorm
|
|
3
|
-
Version: 1.2.
|
|
3
|
+
Version: 1.2.15
|
|
4
4
|
Summary: A command-line utility to validate and normalize CSV files
|
|
5
5
|
Author-email: aborruso <aborruso@gmail.com>
|
|
6
6
|
License: MIT License
|
|
@@ -126,9 +126,10 @@ This tool prepares CSV files for **basic exploratory data analysis (EDA)**, not
|
|
|
126
126
|
|
|
127
127
|
```bash
|
|
128
128
|
csvnorm input.csv [options]
|
|
129
|
+
csvnorm - # read from stdin
|
|
129
130
|
```
|
|
130
131
|
|
|
131
|
-
**By default, csvnorm writes to stdout** for easy piping and composability with other Unix tools.
|
|
132
|
+
**By default, csvnorm writes to stdout** for easy piping and composability with other Unix tools. Use `-` as input to read from stdin.
|
|
132
133
|
|
|
133
134
|
### Options
|
|
134
135
|
|
|
@@ -153,6 +154,11 @@ csvnorm input.csv [options]
|
|
|
153
154
|
# Default: output to stdout
|
|
154
155
|
csvnorm data.csv
|
|
155
156
|
|
|
157
|
+
# Read from stdin
|
|
158
|
+
cat data.csv | csvnorm -
|
|
159
|
+
curl -s https://example.com/data.csv | csvnorm - -o clean.csv
|
|
160
|
+
csvnorm - --check < data.csv
|
|
161
|
+
|
|
156
162
|
# Preview first rows
|
|
157
163
|
csvnorm data.csv | head -20
|
|
158
164
|
|
|
@@ -373,3 +379,15 @@ See [docs/COMMUNICATION.md](docs/COMMUNICATION.md) for details.
|
|
|
373
379
|
## License
|
|
374
380
|
|
|
375
381
|
MIT License (c) 2026 aborruso@gmail.com - See LICENSE file for details
|
|
382
|
+
|
|
383
|
+
## Acknowledgments
|
|
384
|
+
|
|
385
|
+
csvnorm is built on top of excellent open-source libraries:
|
|
386
|
+
|
|
387
|
+
- [**charset-normalizer**](https://github.com/Ousret/charset_normalizer) - Universal encoding detection
|
|
388
|
+
- [**DuckDB**](https://duckdb.org/) - Fast in-process analytical database for CSV processing
|
|
389
|
+
- [**ftfy**](https://github.com/rspeer/python-ftfy) - Fixes mojibake and other text encoding issues
|
|
390
|
+
- [**Rich**](https://github.com/Textualize/rich) - Beautiful terminal output formatting
|
|
391
|
+
- [**rich-argparse**](https://github.com/hamdanal/rich-argparse) - Enhanced CLI help formatting
|
|
392
|
+
|
|
393
|
+
We are grateful to the creators and maintainers of these libraries, without whom csvnorm would not exist and function.
|
|
@@ -66,9 +66,10 @@ This tool prepares CSV files for **basic exploratory data analysis (EDA)**, not
|
|
|
66
66
|
|
|
67
67
|
```bash
|
|
68
68
|
csvnorm input.csv [options]
|
|
69
|
+
csvnorm - # read from stdin
|
|
69
70
|
```
|
|
70
71
|
|
|
71
|
-
**By default, csvnorm writes to stdout** for easy piping and composability with other Unix tools.
|
|
72
|
+
**By default, csvnorm writes to stdout** for easy piping and composability with other Unix tools. Use `-` as input to read from stdin.
|
|
72
73
|
|
|
73
74
|
### Options
|
|
74
75
|
|
|
@@ -93,6 +94,11 @@ csvnorm input.csv [options]
|
|
|
93
94
|
# Default: output to stdout
|
|
94
95
|
csvnorm data.csv
|
|
95
96
|
|
|
97
|
+
# Read from stdin
|
|
98
|
+
cat data.csv | csvnorm -
|
|
99
|
+
curl -s https://example.com/data.csv | csvnorm - -o clean.csv
|
|
100
|
+
csvnorm - --check < data.csv
|
|
101
|
+
|
|
96
102
|
# Preview first rows
|
|
97
103
|
csvnorm data.csv | head -20
|
|
98
104
|
|
|
@@ -313,3 +319,15 @@ See [docs/COMMUNICATION.md](docs/COMMUNICATION.md) for details.
|
|
|
313
319
|
## License
|
|
314
320
|
|
|
315
321
|
MIT License (c) 2026 aborruso@gmail.com - See LICENSE file for details
|
|
322
|
+
|
|
323
|
+
## Acknowledgments
|
|
324
|
+
|
|
325
|
+
csvnorm is built on top of excellent open-source libraries:
|
|
326
|
+
|
|
327
|
+
- [**charset-normalizer**](https://github.com/Ousret/charset_normalizer) - Universal encoding detection
|
|
328
|
+
- [**DuckDB**](https://duckdb.org/) - Fast in-process analytical database for CSV processing
|
|
329
|
+
- [**ftfy**](https://github.com/rspeer/python-ftfy) - Fixes mojibake and other text encoding issues
|
|
330
|
+
- [**Rich**](https://github.com/Textualize/rich) - Beautiful terminal output formatting
|
|
331
|
+
- [**rich-argparse**](https://github.com/hamdanal/rich-argparse) - Enhanced CLI help formatting
|
|
332
|
+
|
|
333
|
+
We are grateful to the creators and maintainers of these libraries, without whom csvnorm would not exist and function.
|
|
@@ -11,7 +11,7 @@ from rich_argparse import RichHelpFormatter
|
|
|
11
11
|
from importlib.metadata import version
|
|
12
12
|
from csvnorm.core import process_csv
|
|
13
13
|
from csvnorm.mojibake import DEFAULT_MOJIBAKE_SAMPLE
|
|
14
|
-
from csvnorm.utils import setup_logger
|
|
14
|
+
from csvnorm.utils import setup_logger, to_snake_case
|
|
15
15
|
|
|
16
16
|
console = Console()
|
|
17
17
|
|
|
@@ -43,9 +43,9 @@ def show_examples() -> None:
|
|
|
43
43
|
console.print(" # Custom delimiter")
|
|
44
44
|
console.print(" [cyan]csvnorm data.csv --skip-rows 2 -o out.csv[/cyan]")
|
|
45
45
|
console.print(" # Skip first 2 rows")
|
|
46
|
-
console.print(" [cyan]csvnorm https://example.com/data.csv -o processed.csv[/cyan]")
|
|
47
|
-
console.print(" # Read from stdin")
|
|
48
46
|
console.print(" [cyan]cat data.csv | csvnorm -[/cyan]")
|
|
47
|
+
console.print(" # Read from stdin")
|
|
48
|
+
console.print(" [cyan]csvnorm https://example.com/data.csv -o processed.csv[/cyan]")
|
|
49
49
|
|
|
50
50
|
|
|
51
51
|
class VersionAction(argparse.Action):
|
|
@@ -131,6 +131,17 @@ def create_parser() -> argparse.ArgumentParser:
|
|
|
131
131
|
help="Write to file instead of stdout (default: stdout)",
|
|
132
132
|
)
|
|
133
133
|
|
|
134
|
+
parser.add_argument(
|
|
135
|
+
"-O",
|
|
136
|
+
"--auto-output",
|
|
137
|
+
action="store_true",
|
|
138
|
+
help=(
|
|
139
|
+
"Auto-derive output filename in snake_case from input filename, "
|
|
140
|
+
"saved in the same directory. Example: 'My Data.csv' → 'my_data.csv'. "
|
|
141
|
+
"Asks confirmation if output already exists (unless -f is set)."
|
|
142
|
+
),
|
|
143
|
+
)
|
|
144
|
+
|
|
134
145
|
parser.add_argument(
|
|
135
146
|
"--fix-mojibake",
|
|
136
147
|
nargs="?",
|
|
@@ -230,6 +241,43 @@ def main(argv: Optional[list[str]] = None) -> int:
|
|
|
230
241
|
console.print("[red]Error:[/red] --skip-rows must be non-negative", style="red")
|
|
231
242
|
return 1
|
|
232
243
|
|
|
244
|
+
# Resolve --auto-output into output_file
|
|
245
|
+
if args.auto_output:
|
|
246
|
+
if args.output_file:
|
|
247
|
+
console.print("[red]Error:[/red] -O/--auto-output cannot be used with -o", style="red")
|
|
248
|
+
return 1
|
|
249
|
+
if args.input_file == "-":
|
|
250
|
+
console.print("[red]Error:[/red] -O/--auto-output cannot be used with stdin input", style="red")
|
|
251
|
+
return 1
|
|
252
|
+
if args.input_file.startswith("http://") or args.input_file.startswith("https://"):
|
|
253
|
+
console.print("[red]Error:[/red] -O/--auto-output cannot be used with URL input", style="red")
|
|
254
|
+
return 1
|
|
255
|
+
|
|
256
|
+
input_path = Path(args.input_file)
|
|
257
|
+
snake_stem = to_snake_case(input_path.stem)
|
|
258
|
+
auto_output = input_path.parent / f"{snake_stem}.csv"
|
|
259
|
+
|
|
260
|
+
if auto_output.resolve() == input_path.resolve():
|
|
261
|
+
console.print(
|
|
262
|
+
f"[yellow]Warning:[/yellow] filename is already in snake_case ({input_path.name}). "
|
|
263
|
+
"Use -o to specify a different output path.",
|
|
264
|
+
style="yellow",
|
|
265
|
+
)
|
|
266
|
+
return 1
|
|
267
|
+
|
|
268
|
+
if auto_output.exists() and not args.force:
|
|
269
|
+
console.print(f"[yellow]Output file already exists:[/yellow] {auto_output}")
|
|
270
|
+
try:
|
|
271
|
+
confirm = input("Overwrite? [y/N] ").strip().lower()
|
|
272
|
+
except (EOFError, KeyboardInterrupt):
|
|
273
|
+
console.print("\nAborted.")
|
|
274
|
+
return 0
|
|
275
|
+
if confirm != "y":
|
|
276
|
+
console.print("Aborted.")
|
|
277
|
+
return 0
|
|
278
|
+
|
|
279
|
+
args.output_file = auto_output
|
|
280
|
+
|
|
233
281
|
# Validate incompatible flag combinations
|
|
234
282
|
if args.check and args.output_file:
|
|
235
283
|
console.print(
|
|
@@ -370,6 +370,7 @@ def _emit_stdout_output(
|
|
|
370
370
|
keep_names=summary["keep_names"],
|
|
371
371
|
output_display=summary["output_display"],
|
|
372
372
|
out_console=stderr_console,
|
|
373
|
+
reject_count=summary.get("reject_count", 0),
|
|
373
374
|
)
|
|
374
375
|
|
|
375
376
|
# Return exit code 1 if validation errors occurred (warning already shown before output)
|
|
@@ -576,6 +577,7 @@ def _compute_and_show_output(
|
|
|
576
577
|
"delimiter": delimiter,
|
|
577
578
|
"keep_names": keep_names,
|
|
578
579
|
"output_display": "stdout",
|
|
580
|
+
"reject_count": reject_count - 1 if has_validation_errors else 0,
|
|
579
581
|
}
|
|
580
582
|
return _emit_stdout_output(
|
|
581
583
|
actual_output_file,
|
|
@@ -597,6 +599,7 @@ def _compute_and_show_output(
|
|
|
597
599
|
output_size=output_size,
|
|
598
600
|
delimiter=delimiter,
|
|
599
601
|
keep_names=keep_names,
|
|
602
|
+
reject_count=reject_count - 1 if has_validation_errors else 0,
|
|
600
603
|
)
|
|
601
604
|
|
|
602
605
|
if has_validation_errors:
|
|
@@ -49,6 +49,7 @@ def show_success_table(
|
|
|
49
49
|
keep_names: bool,
|
|
50
50
|
output_display: Optional[str] = None,
|
|
51
51
|
out_console: Optional[Console] = None,
|
|
52
|
+
reject_count: int = 0,
|
|
52
53
|
) -> None:
|
|
53
54
|
"""Display success summary table with processing results.
|
|
54
55
|
|
|
@@ -66,7 +67,10 @@ def show_success_table(
|
|
|
66
67
|
keep_names: Whether original column names were kept.
|
|
67
68
|
"""
|
|
68
69
|
table = Table(show_header=False, box=None, padding=(0, 1))
|
|
69
|
-
|
|
70
|
+
if reject_count > 0:
|
|
71
|
+
table.add_row("[yellow]⚠[/yellow] Done with errors", "")
|
|
72
|
+
else:
|
|
73
|
+
table.add_row("[green]✓[/green] Success", "")
|
|
70
74
|
table.add_row("Input:", f"[cyan]{input_file}[/cyan]")
|
|
71
75
|
output_value = output_display if output_display is not None else str(output_file)
|
|
72
76
|
table.add_row("Output:", f"[cyan]{output_value}[/cyan]")
|
|
@@ -95,6 +99,8 @@ def show_success_table(
|
|
|
95
99
|
# Optional fields
|
|
96
100
|
if delimiter != ",":
|
|
97
101
|
table.add_row("Delimiter:", repr(delimiter))
|
|
102
|
+
if reject_count > 0:
|
|
103
|
+
table.add_row("Rejected rows:", f"[yellow]{reject_count}[/yellow]")
|
|
98
104
|
if not keep_names:
|
|
99
105
|
table.add_row("Headers:", "normalized to snake_case")
|
|
100
106
|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: csvnorm
|
|
3
|
-
Version: 1.2.
|
|
3
|
+
Version: 1.2.15
|
|
4
4
|
Summary: A command-line utility to validate and normalize CSV files
|
|
5
5
|
Author-email: aborruso <aborruso@gmail.com>
|
|
6
6
|
License: MIT License
|
|
@@ -126,9 +126,10 @@ This tool prepares CSV files for **basic exploratory data analysis (EDA)**, not
|
|
|
126
126
|
|
|
127
127
|
```bash
|
|
128
128
|
csvnorm input.csv [options]
|
|
129
|
+
csvnorm - # read from stdin
|
|
129
130
|
```
|
|
130
131
|
|
|
131
|
-
**By default, csvnorm writes to stdout** for easy piping and composability with other Unix tools.
|
|
132
|
+
**By default, csvnorm writes to stdout** for easy piping and composability with other Unix tools. Use `-` as input to read from stdin.
|
|
132
133
|
|
|
133
134
|
### Options
|
|
134
135
|
|
|
@@ -153,6 +154,11 @@ csvnorm input.csv [options]
|
|
|
153
154
|
# Default: output to stdout
|
|
154
155
|
csvnorm data.csv
|
|
155
156
|
|
|
157
|
+
# Read from stdin
|
|
158
|
+
cat data.csv | csvnorm -
|
|
159
|
+
curl -s https://example.com/data.csv | csvnorm - -o clean.csv
|
|
160
|
+
csvnorm - --check < data.csv
|
|
161
|
+
|
|
156
162
|
# Preview first rows
|
|
157
163
|
csvnorm data.csv | head -20
|
|
158
164
|
|
|
@@ -373,3 +379,15 @@ See [docs/COMMUNICATION.md](docs/COMMUNICATION.md) for details.
|
|
|
373
379
|
## License
|
|
374
380
|
|
|
375
381
|
MIT License (c) 2026 aborruso@gmail.com - See LICENSE file for details
|
|
382
|
+
|
|
383
|
+
## Acknowledgments
|
|
384
|
+
|
|
385
|
+
csvnorm is built on top of excellent open-source libraries:
|
|
386
|
+
|
|
387
|
+
- [**charset-normalizer**](https://github.com/Ousret/charset_normalizer) - Universal encoding detection
|
|
388
|
+
- [**DuckDB**](https://duckdb.org/) - Fast in-process analytical database for CSV processing
|
|
389
|
+
- [**ftfy**](https://github.com/rspeer/python-ftfy) - Fixes mojibake and other text encoding issues
|
|
390
|
+
- [**Rich**](https://github.com/Textualize/rich) - Beautiful terminal output formatting
|
|
391
|
+
- [**rich-argparse**](https://github.com/hamdanal/rich-argparse) - Enhanced CLI help formatting
|
|
392
|
+
|
|
393
|
+
We are grateful to the creators and maintainers of these libraries, without whom csvnorm would not exist and function.
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|