nm-tool-forge 0.2.3__tar.gz → 0.2.4__tar.gz

{nm_tool_forge-0.2.3 → nm_tool_forge-0.2.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: nm-tool-forge
-Version: 0.2.3
+Version: 0.2.4
 Summary: Analyze MigMan log files and generate aggregated CSV, Markdown, HTML, and optional PDF reports.
 Author-email: Stefan Ewald <s.ew@outlook.de>
 License-Expression: MIT
@@ -31,7 +31,7 @@ Dynamic: license-file
 
 # nm-tool-forge
 
-`nm-tool-forge` analyzes MigMan text log files with severity tokens such as `INFO`, `ERROR`, and `WARNING` and generates aggregated CSV, Markdown, HTML, and optional PDF reports.
+`nm-tool-forge` analyzes MigMan text log files with severity tokens such as `INFO`, `ERROR`, and `WARNING` and generates aggregated CSV, Markdown, HTML, and optional PDF reports. The package also includes `csvchunking`, a small helper for splitting large CSV files into migration-friendly chunks.
 
 The project uses a package-ready `src` layout. The legacy `log_analysis.py` file remains available as a thin compatibility entry point for older local setups.
 
@@ -43,6 +43,7 @@ The project uses a package-ready `src` layout. The legacy `log_analysis.py` file
 - Generate Markdown summary reports
 - Optionally convert reports to HTML and PDF
 - Keep a backup copy of analyzed log files
+- Split large CSV files into numbered chunks while preserving the header row
 - Run built-in self-tests from the CLI
 
 ## Installation
@@ -61,12 +62,14 @@ python -m pip install .[pdf,dev]
 
 ## Command-line usage
 
-After installation, both entry points are available:
+After installation, the CLI entry points are available:
 
 ```powershell
 python -m loganalysis --help
+python -m csvchunking --help
 loganalysis --help
 nm-tool-forge --help
+csvchunking --help
 ```
 
 Typical analysis run:
@@ -89,29 +92,30 @@ python -m loganalysis --self-test
 
 Legacy compatibility call:
 
+```powershell
+python .\log_analysis.py --convert
+```
 
-## Release process
-
-To publish a new release, always test on TestPyPI first, then upload to PyPI only after successful Conda/Smoke-Tests:
-
-```bash
-export TWINE_USERNAME="__token__"
-export TWINE_PASSWORD="pypi-..."
+CSV chunking run:
 
-bash scripts/release_testpypi.sh --bump patch
-bash scripts/release_pypi.sh --yes
+```powershell
+csvchunking "data\large_export.csv" --chunk-size 5000
 ```
 
-**Hinweise:**
-- Erst TestPyPI ausführen und testen, dann final nach PyPI hochladen.
-- Versionen auf PyPI können nicht überschrieben oder erneut verwendet werden.
+The command creates an output directory next to the input file named after the CSV stem. For example, `data\large_export.csv` is split into files such as `data\large_export\large_export_01.csv`, `data\large_export\large_export_02.csv`, and so on.
+
+CSV chunking with an explicit encoding:
 
 ```powershell
-python .\log_analysis.py --convert
+python -m csvchunking "data\large_export.csv" --chunk-size 5000 --encoding utf-8-sig
 ```
 
+Each chunk contains the original header row plus up to `--chunk-size` data rows. The delimiter is detected automatically; if detection fails, semicolon-separated CSV is used.
+
 ## Supported CLI options
 
+Log analysis options:
+
 - `--logs-dir`
 - `--out-dir`
 - `--backup-dir`
@@ -119,6 +123,28 @@ python .\log_analysis.py --convert
 - `--convert`
 - `--self-test`
 
+CSV chunking options:
+
+- `input_file` - path to the CSV file to split
+- `--chunk-size` - required number of data rows per output file; must be greater than zero
+- `--encoding` - input and output encoding; defaults to `utf-8-sig`
+
+## Release process
+
+To publish a new release, always test on TestPyPI first, then upload to PyPI only after successful Conda smoke tests:
+
+```bash
+export TWINE_USERNAME="__token__"
+export TWINE_PASSWORD="pypi-..."
+
+bash scripts/release_testpypi.sh --bump patch
+bash scripts/release_pypi.sh --yes
+```
+
+**Notes:**
+- Run and verify the TestPyPI release first, then upload the final package to PyPI.
+- PyPI versions cannot be overwritten or reused.
+
 ## Library usage
 
 ```python
@@ -130,6 +156,7 @@ from loganalysis import (
     iter_logical_entries,
     normalize_message,
 )
+from csvchunking import split_csv
 
 result = analyze_file(Path("logs/app.txt"))
 print(result["norm_counts"])
@@ -146,14 +173,21 @@ convert_report_md_to_html_pdf(
     Path("log_analyse_out/report.html"),
     Path("log_analyse_out/report.pdf"),
 )
+
+chunk_result = split_csv(Path("data/large_export.csv"), chunk_size=5000)
+print(chunk_result.output_dir)
+print(chunk_result.output_files)
 ```
 
+`split_csv()` returns a `ChunkResult` with the input file, output directory, chunk size, processed data-row count, created file count, and generated output file paths.
+
 ## Project structure
 
 ```text
 .
 ├─ pyproject.toml
 ├─ src/loganalysis/
+├─ src/csvchunking/
 ├─ tests/
 ├─ docs/
 └─ log_analysis.py
@@ -168,7 +202,9 @@ Important modules:
 - `report_html.py` - HTML/CSS rendering
 - `report_pdf.py` - PDF engine selection and fallback handling
 - `converters.py` - Markdown-to-HTML/PDF conversion
-- `cli.py` - command-line entry point
+- `loganalysis/cli.py` - log analysis command-line entry point
+- `csvchunking/chunker.py` - CSV splitting logic and `ChunkResult`
+- `csvchunking/cli.py` - CSV chunking command-line entry point
 
 ## HTML/PDF conversion
 

{nm_tool_forge-0.2.3 → nm_tool_forge-0.2.4}/README.md

@@ -1,6 +1,6 @@
 # nm-tool-forge
 
-`nm-tool-forge` analyzes MigMan text log files with severity tokens such as `INFO`, `ERROR`, and `WARNING` and generates aggregated CSV, Markdown, HTML, and optional PDF reports.
+`nm-tool-forge` analyzes MigMan text log files with severity tokens such as `INFO`, `ERROR`, and `WARNING` and generates aggregated CSV, Markdown, HTML, and optional PDF reports. The package also includes `csvchunking`, a small helper for splitting large CSV files into migration-friendly chunks.
 
 The project uses a package-ready `src` layout. The legacy `log_analysis.py` file remains available as a thin compatibility entry point for older local setups.
 
@@ -9,10 +9,11 @@ The project uses a package-ready `src` layout. The legacy `log_analysis.py` file
 - Parse logical log entries from multi-line text logs
 - Normalize recurring error patterns for better aggregation
 - Generate aggregated CSV reports
-- Generate Markdown summary reports
-- Optionally convert reports to HTML and PDF
-- Keep a backup copy of analyzed log files
-- Run built-in self-tests from the CLI
+- Generate Markdown summary reports
+- Optionally convert reports to HTML and PDF
+- Keep a backup copy of analyzed log files
+- Split large CSV files into numbered chunks while preserving the header row
+- Run built-in self-tests from the CLI
 
 ## Installation
 
@@ -30,13 +31,15 @@ python -m pip install .[pdf,dev]
 
 ## Command-line usage
 
-After installation, both entry points are available:
-
-```powershell
-python -m loganalysis --help
-loganalysis --help
-nm-tool-forge --help
-```
+After installation, the CLI entry points are available:
+
+```powershell
+python -m loganalysis --help
+python -m csvchunking --help
+loganalysis --help
+nm-tool-forge --help
+csvchunking --help
+```
 
 Typical analysis run:
 
@@ -50,18 +53,54 @@ Analysis with HTML/PDF conversion:
 nm-tool-forge --logs-dir logs --out-dir log_analyse_out --convert
 ```
 
-Self-test mode:
-
-```powershell
-python -m loganalysis --self-test
-```
-
-Legacy compatibility call:
-
-
-## Release process
-
-To publish a new release, always test on TestPyPI first, then upload to PyPI only after successful Conda/Smoke-Tests:
+Self-test mode:
+
+```powershell
+python -m loganalysis --self-test
+```
+
+Legacy compatibility call:
+
+```powershell
+python .\log_analysis.py --convert
+```
+
+CSV chunking run:
+
+```powershell
+csvchunking "data\large_export.csv" --chunk-size 5000
+```
+
+The command creates an output directory next to the input file named after the CSV stem. For example, `data\large_export.csv` is split into files such as `data\large_export\large_export_01.csv`, `data\large_export\large_export_02.csv`, and so on.
+
+CSV chunking with an explicit encoding:
+
+```powershell
+python -m csvchunking "data\large_export.csv" --chunk-size 5000 --encoding utf-8-sig
+```
+
+Each chunk contains the original header row plus up to `--chunk-size` data rows. The delimiter is detected automatically; if detection fails, semicolon-separated CSV is used.
+
+## Supported CLI options
+
+Log analysis options:
+
+- `--logs-dir`
+- `--out-dir`
+- `--backup-dir`
+- `--top-examples`
+- `--convert`
+- `--self-test`
+
+CSV chunking options:
+
+- `input_file` - path to the CSV file to split
+- `--chunk-size` - required number of data rows per output file; must be greater than zero
+- `--encoding` - input and output encoding; defaults to `utf-8-sig`
+
+## Release process
+
+To publish a new release, always test on TestPyPI first, then upload to PyPI only after successful Conda smoke tests:
 
 ```bash
 export TWINE_USERNAME="__token__"
@@ -71,37 +110,25 @@ bash scripts/release_testpypi.sh --bump patch
 bash scripts/release_pypi.sh --yes
 ```
 
-**Hinweise:**
-- Erst TestPyPI ausführen und testen, dann final nach PyPI hochladen.
-- Versionen auf PyPI können nicht überschrieben oder erneut verwendet werden.
-
-```powershell
-python .\log_analysis.py --convert
-```
-
-## Supported CLI options
-
-- `--logs-dir`
-- `--out-dir`
-- `--backup-dir`
-- `--top-examples`
-- `--convert`
-- `--self-test`
-
-## Library usage
-
-```python
+**Notes:**
+- Run and verify the TestPyPI release first, then upload the final package to PyPI.
+- PyPI versions cannot be overwritten or reused.
+
+## Library usage
+
+```python
 from pathlib import Path
 
 from loganalysis import (
     analyze_file,
     convert_report_md_to_html_pdf,
-    iter_logical_entries,
-    normalize_message,
-)
-
-result = analyze_file(Path("logs/app.txt"))
-print(result["norm_counts"])
+    iter_logical_entries,
+    normalize_message,
+)
+from csvchunking import split_csv
+
+result = analyze_file(Path("logs/app.txt"))
+print(result["norm_counts"])
 
 print(normalize_message(
     'Conversion: X =3100110. 138 The record was not found in table "Teile".'
@@ -112,20 +139,27 @@ for entry in iter_logical_entries(Path("logs/app.txt")):
 
 convert_report_md_to_html_pdf(
     Path("log_analyse_out/report.md"),
-    Path("log_analyse_out/report.html"),
-    Path("log_analyse_out/report.pdf"),
-)
-```
+    Path("log_analyse_out/report.html"),
+    Path("log_analyse_out/report.pdf"),
+)
+
+chunk_result = split_csv(Path("data/large_export.csv"), chunk_size=5000)
+print(chunk_result.output_dir)
+print(chunk_result.output_files)
+```
+
+`split_csv()` returns a `ChunkResult` with the input file, output directory, chunk size, processed data-row count, created file count, and generated output file paths.
 
 ## Project structure
 
 ```text
 .
-├─ pyproject.toml
-├─ src/loganalysis/
-├─ tests/
-├─ docs/
-└─ log_analysis.py
+├─ pyproject.toml
+├─ src/loganalysis/
+├─ src/csvchunking/
+├─ tests/
+├─ docs/
+└─ log_analysis.py
 ```
 
 Important modules:
@@ -135,9 +169,11 @@ Important modules:
 - `normalization.py` - message normalization
 - `report_markdown.py` - Markdown report model and rendering
 - `report_html.py` - HTML/CSS rendering
-- `report_pdf.py` - PDF engine selection and fallback handling
-- `converters.py` - Markdown-to-HTML/PDF conversion
-- `cli.py` - command-line entry point
+- `report_pdf.py` - PDF engine selection and fallback handling
+- `converters.py` - Markdown-to-HTML/PDF conversion
+- `loganalysis/cli.py` - log analysis command-line entry point
+- `csvchunking/chunker.py` - CSV splitting logic and `ChunkResult`
+- `csvchunking/cli.py` - CSV chunking command-line entry point
 
 ## HTML/PDF conversion
 

{nm_tool_forge-0.2.3 → nm_tool_forge-0.2.4}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "nm-tool-forge"
-version = "0.2.3"
+version = "0.2.4"
 description = "Analyze MigMan log files and generate aggregated CSV, Markdown, HTML, and optional PDF reports."
 readme = { file = "README.md", content-type = "text/markdown" }
 requires-python = ">=3.10"

{nm_tool_forge-0.2.3 → nm_tool_forge-0.2.4}/src/csvchunking/__init__.py

@@ -1,4 +1,4 @@
 from .chunker import ChunkResult, split_csv
 
 __all__ = ["ChunkResult", "split_csv"]
-__version__ = "0.2.3"
+__version__ = "0.2.4"

{nm_tool_forge-0.2.3 → nm_tool_forge-0.2.4}/src/csvchunking/chunker.py

@@ -18,16 +18,16 @@ def split_csv(
     chunk_size: int,
     encoding: str = "utf-8-sig",
 ) -> ChunkResult:
-    if not Path(input_file).is_file():
-        raise FileNotFoundError(f"Eingabedatei nicht gefunden: {input_file}")
-    if chunk_size <= 0:
-        raise ValueError("chunk_size muss > 0 sein")
+    if not Path(input_file).is_file():
+        raise FileNotFoundError(f"Input file not found: {input_file}")
+    if chunk_size <= 0:
+        raise ValueError("chunk_size must be greater than 0")
 
     input_file = Path(input_file)
     output_dir = input_file.parent / input_file.stem
     output_dir.mkdir(exist_ok=True)
 
-    # Delimiter automatisch erkennen
+    # Detect the delimiter automatically.
     with open(input_file, encoding=encoding, newline="") as f:
         sample = f.read(4096)
         f.seek(0)
@@ -41,7 +41,7 @@ def split_csv(
         try:
             header = next(reader)
         except StopIteration as exc:
-            raise ValueError("Eingabedatei ist leer.") from exc
+            raise ValueError("Input file is empty.") from exc
         chunk = []
         file_count = 0
         data_rows = 0

{nm_tool_forge-0.2.3 → nm_tool_forge-0.2.4}/src/csvchunking/cli.py

@@ -7,22 +7,22 @@ from .chunker import split_csv
 
 def main() -> None:
     parser = argparse.ArgumentParser(
-        description="Teilt eine große CSV-Datei in kleinere Chunks mit Header.",
+        description="Split a large CSV file into smaller chunks with a header row.",
     )
-    parser.add_argument("input_file", help="Pfad zur CSV-Datei")
+    parser.add_argument("input_file", help="Path to the CSV file")
     parser.add_argument(
         "--chunk-size",
         type=int,
         required=True,
-        help="Anzahl Datenzeilen pro Ausgabedatei, muss > 0 sein",
+        help="Number of data rows per output file; must be greater than 0",
     )
-    parser.add_argument("--encoding", default="utf-8-sig", help="Encoding für Ein- und Ausgabe (Standard: utf-8-sig)")
+    parser.add_argument("--encoding", default="utf-8-sig", help="Input and output encoding (Default: utf-8-sig)")
     args = parser.parse_args()
-    try:
-        result = split_csv(Path(args.input_file), args.chunk_size, encoding=args.encoding)
-    except Exception as e:
-        print(f"Fehler: {e}", file=sys.stderr)
-        sys.exit(1)
+    try:
+        result = split_csv(Path(args.input_file), args.chunk_size, encoding=args.encoding)
+    except Exception as e:
+        print(f"Error: {e}", file=sys.stderr)
+        sys.exit(1)
     print("CSV chunking completed.")
     print(f"- Input: {result.input_file}")
     print(f"- Output directory: {result.output_dir}")

{nm_tool_forge-0.2.3 → nm_tool_forge-0.2.4}/src/loganalysis/__init__.py

@@ -13,4 +13,4 @@ __all__ = [
     "run_analysis",
 ]
 
-__version__ = "0.2.3"
+__version__ = "0.2.4"

{nm_tool_forge-0.2.3 → nm_tool_forge-0.2.4}/src/nm_tool_forge.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: nm-tool-forge
-Version: 0.2.3
+Version: 0.2.4
 Summary: Analyze MigMan log files and generate aggregated CSV, Markdown, HTML, and optional PDF reports.
 Author-email: Stefan Ewald <s.ew@outlook.de>
 License-Expression: MIT
@@ -31,7 +31,7 @@ Dynamic: license-file
 
 # nm-tool-forge
 
-`nm-tool-forge` analyzes MigMan text log files with severity tokens such as `INFO`, `ERROR`, and `WARNING` and generates aggregated CSV, Markdown, HTML, and optional PDF reports.
+`nm-tool-forge` analyzes MigMan text log files with severity tokens such as `INFO`, `ERROR`, and `WARNING` and generates aggregated CSV, Markdown, HTML, and optional PDF reports. The package also includes `csvchunking`, a small helper for splitting large CSV files into migration-friendly chunks.
 
 The project uses a package-ready `src` layout. The legacy `log_analysis.py` file remains available as a thin compatibility entry point for older local setups.
 
@@ -43,6 +43,7 @@ The project uses a package-ready `src` layout. The legacy `log_analysis.py` file
 - Generate Markdown summary reports
 - Optionally convert reports to HTML and PDF
 - Keep a backup copy of analyzed log files
+- Split large CSV files into numbered chunks while preserving the header row
 - Run built-in self-tests from the CLI
 
 ## Installation
@@ -61,12 +62,14 @@ python -m pip install .[pdf,dev]
 
 ## Command-line usage
 
-After installation, both entry points are available:
+After installation, the CLI entry points are available:
 
 ```powershell
 python -m loganalysis --help
+python -m csvchunking --help
 loganalysis --help
 nm-tool-forge --help
+csvchunking --help
 ```
 
 Typical analysis run:
@@ -89,29 +92,30 @@ python -m loganalysis --self-test
 
 Legacy compatibility call:
 
+```powershell
+python .\log_analysis.py --convert
+```
 
-## Release process
-
-To publish a new release, always test on TestPyPI first, then upload to PyPI only after successful Conda/Smoke-Tests:
-
-```bash
-export TWINE_USERNAME="__token__"
-export TWINE_PASSWORD="pypi-..."
+CSV chunking run:
 
-bash scripts/release_testpypi.sh --bump patch
-bash scripts/release_pypi.sh --yes
+```powershell
+csvchunking "data\large_export.csv" --chunk-size 5000
 ```
 
-**Hinweise:**
-- Erst TestPyPI ausführen und testen, dann final nach PyPI hochladen.
-- Versionen auf PyPI können nicht überschrieben oder erneut verwendet werden.
+The command creates an output directory next to the input file named after the CSV stem. For example, `data\large_export.csv` is split into files such as `data\large_export\large_export_01.csv`, `data\large_export\large_export_02.csv`, and so on.
+
+CSV chunking with an explicit encoding:
 
 ```powershell
-python .\log_analysis.py --convert
+python -m csvchunking "data\large_export.csv" --chunk-size 5000 --encoding utf-8-sig
 ```
 
+Each chunk contains the original header row plus up to `--chunk-size` data rows. The delimiter is detected automatically; if detection fails, semicolon-separated CSV is used.
+
 ## Supported CLI options
 
+Log analysis options:
+
 - `--logs-dir`
 - `--out-dir`
 - `--backup-dir`
@@ -119,6 +123,28 @@ python .\log_analysis.py --convert
 - `--convert`
 - `--self-test`
 
+CSV chunking options:
+
+- `input_file` - path to the CSV file to split
+- `--chunk-size` - required number of data rows per output file; must be greater than zero
+- `--encoding` - input and output encoding; defaults to `utf-8-sig`
+
+## Release process
+
+To publish a new release, always test on TestPyPI first, then upload to PyPI only after successful Conda smoke tests:
+
+```bash
+export TWINE_USERNAME="__token__"
+export TWINE_PASSWORD="pypi-..."
+
+bash scripts/release_testpypi.sh --bump patch
+bash scripts/release_pypi.sh --yes
+```
+
+**Notes:**
+- Run and verify the TestPyPI release first, then upload the final package to PyPI.
+- PyPI versions cannot be overwritten or reused.
+
 ## Library usage
 
 ```python
@@ -130,6 +156,7 @@ from loganalysis import (
     iter_logical_entries,
     normalize_message,
 )
+from csvchunking import split_csv
 
 result = analyze_file(Path("logs/app.txt"))
 print(result["norm_counts"])
@@ -146,14 +173,21 @@ convert_report_md_to_html_pdf(
     Path("log_analyse_out/report.html"),
     Path("log_analyse_out/report.pdf"),
 )
+
+chunk_result = split_csv(Path("data/large_export.csv"), chunk_size=5000)
+print(chunk_result.output_dir)
+print(chunk_result.output_files)
 ```
 
+`split_csv()` returns a `ChunkResult` with the input file, output directory, chunk size, processed data-row count, created file count, and generated output file paths.
+
 ## Project structure
 
 ```text
 .
 ├─ pyproject.toml
 ├─ src/loganalysis/
+├─ src/csvchunking/
 ├─ tests/
 ├─ docs/
 └─ log_analysis.py
@@ -168,7 +202,9 @@ Important modules:
 - `report_html.py` - HTML/CSS rendering
 - `report_pdf.py` - PDF engine selection and fallback handling
 - `converters.py` - Markdown-to-HTML/PDF conversion
-- `cli.py` - command-line entry point
+- `loganalysis/cli.py` - log analysis command-line entry point
+- `csvchunking/chunker.py` - CSV splitting logic and `ChunkResult`
+- `csvchunking/cli.py` - CSV chunking command-line entry point
 
 ## HTML/PDF conversion
 

{nm_tool_forge-0.2.3 → nm_tool_forge-0.2.4}/tests/test_csvchunking.py

@@ -12,7 +12,7 @@ def make_csv(tmp_path, name, header, rows, encoding="utf-8-sig", delimiter=";"):
     return file
 
 
-def test_normale_aufteilung(tmp_path):
+def test_regular_split(tmp_path):
     header = ["col1", "col2"]
     rows = [["A", "1"], ["B", "2"], ["C", "3"], ["D", "4"], ["E", "5"]]
     file = make_csv(tmp_path, "sample.csv", header, rows)
@@ -27,7 +27,7 @@ def test_normale_aufteilung(tmp_path):
     assert (result.output_dir / "sample_03.csv").exists()
 
 
-def test_header_in_jeder_datei(tmp_path):
+def test_header_in_each_file(tmp_path):
     header = ["foo", "bar"]
     rows = [["x", "1"], ["y", "2"], ["z", "3"]]
     file = make_csv(tmp_path, "test.csv", header, rows)
@@ -37,7 +37,7 @@ def test_header_in_jeder_datei(tmp_path):
             assert f.readline().strip() == "foo;bar"
 
 
-def test_dateiname_mit_leerzeichen(tmp_path):
+def test_filename_with_spaces(tmp_path):
     header = ["a", "b"]
     rows = [["1", "2"]]
     file = make_csv(tmp_path, "Part-Storage Areas Relationships.csv", header, rows)
@@ -46,7 +46,7 @@ def test_dateiname_mit_leerzeichen(tmp_path):
     assert (result.output_dir / "Part-Storage Areas Relationships_01.csv").exists()
 
 
-def test_ungueltige_chunkgroesse(tmp_path):
+def test_invalid_chunk_size(tmp_path):
     header = ["a", "b"]
     rows = [["1", "2"]]
     file = make_csv(tmp_path, "fail.csv", header, rows)
@@ -56,7 +56,7 @@ def test_ungueltige_chunkgroesse(tmp_path):
         split_csv(file, chunk_size=-1)
 
 
-def test_leere_datei(tmp_path):
+def test_empty_file(tmp_path):
     file = tmp_path / "empty.csv"
     file.write_text("")
     with pytest.raises(ValueError):