nm-tool-forge 0.1.0__tar.gz → 0.2.4__tar.gz

{nm_tool_forge-0.1.0 → nm_tool_forge-0.2.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: nm-tool-forge
-Version: 0.1.0
+Version: 0.2.4
 Summary: Analyze MigMan log files and generate aggregated CSV, Markdown, HTML, and optional PDF reports.
 Author-email: Stefan Ewald <s.ew@outlook.de>
 License-Expression: MIT
@@ -31,7 +31,7 @@ Dynamic: license-file
 
 # nm-tool-forge
 
-`nm-tool-forge` analyzes MigMan text log files with severity tokens such as `INFO`, `ERROR`, and `WARNING` and generates aggregated CSV, Markdown, HTML, and optional PDF reports.
+`nm-tool-forge` analyzes MigMan text log files with severity tokens such as `INFO`, `ERROR`, and `WARNING` and generates aggregated CSV, Markdown, HTML, and optional PDF reports. The package also includes `csvchunking`, a small helper for splitting large CSV files into migration-friendly chunks.
 
 The project uses a package-ready `src` layout. The legacy `log_analysis.py` file remains available as a thin compatibility entry point for older local setups.
 
@@ -43,6 +43,7 @@ The project uses a package-ready `src` layout. The legacy `log_analysis.py` file
 - Generate Markdown summary reports
 - Optionally convert reports to HTML and PDF
 - Keep a backup copy of analyzed log files
+- Split large CSV files into numbered chunks while preserving the header row
 - Run built-in self-tests from the CLI
 
 ## Installation
@@ -61,12 +62,14 @@ python -m pip install .[pdf,dev]
 
 ## Command-line usage
 
-After installation, both entry points are available:
+After installation, the CLI entry points are available:
 
 ```powershell
 python -m loganalysis --help
+python -m csvchunking --help
 loganalysis --help
 nm-tool-forge --help
+csvchunking --help
 ```
 
 Typical analysis run:
@@ -93,8 +96,26 @@ Legacy compatibility call:
 python .\log_analysis.py --convert
 ```
 
+CSV chunking run:
+
+```powershell
+csvchunking "data\large_export.csv" --chunk-size 5000
+```
+
+The command creates an output directory next to the input file named after the CSV stem. For example, `data\large_export.csv` is split into files such as `data\large_export\large_export_01.csv`, `data\large_export\large_export_02.csv`, and so on.
+
+CSV chunking with an explicit encoding:
+
+```powershell
+python -m csvchunking "data\large_export.csv" --chunk-size 5000 --encoding utf-8-sig
+```
+
+Each chunk contains the original header row plus up to `--chunk-size` data rows. The delimiter is detected automatically; if detection fails, semicolon-separated CSV is used.
+
 ## Supported CLI options
 
+Log analysis options:
+
 - `--logs-dir`
 - `--out-dir`
 - `--backup-dir`
@@ -102,6 +123,28 @@ python .\log_analysis.py --convert
 - `--convert`
 - `--self-test`
 
+CSV chunking options:
+
+- `input_file` - path to the CSV file to split
+- `--chunk-size` - required number of data rows per output file; must be greater than zero
+- `--encoding` - input and output encoding; defaults to `utf-8-sig`
+
+## Release process
+
+To publish a new release, always test on TestPyPI first, then upload to PyPI only after successful Conda smoke tests:
+
+```bash
+export TWINE_USERNAME="__token__"
+export TWINE_PASSWORD="pypi-..."
+
+bash scripts/release_testpypi.sh --bump patch
+bash scripts/release_pypi.sh --yes
+```
+
+**Notes:**
+- Run and verify the TestPyPI release first, then upload the final package to PyPI.
+- PyPI versions cannot be overwritten or reused.
+
 ## Library usage
 
 ```python
@@ -113,6 +156,7 @@ from loganalysis import (
     iter_logical_entries,
     normalize_message,
 )
+from csvchunking import split_csv
 
 result = analyze_file(Path("logs/app.txt"))
 print(result["norm_counts"])
@@ -129,14 +173,21 @@ convert_report_md_to_html_pdf(
     Path("log_analyse_out/report.html"),
     Path("log_analyse_out/report.pdf"),
 )
+
+chunk_result = split_csv(Path("data/large_export.csv"), chunk_size=5000)
+print(chunk_result.output_dir)
+print(chunk_result.output_files)
 ```
 
+`split_csv()` returns a `ChunkResult` with the input file, output directory, chunk size, processed data-row count, created file count, and generated output file paths.
+
 ## Project structure
 
 ```text
 .
 ├─ pyproject.toml
 ├─ src/loganalysis/
+├─ src/csvchunking/
 ├─ tests/
 ├─ docs/
 └─ log_analysis.py
@@ -151,7 +202,9 @@ Important modules:
 - `report_html.py` - HTML/CSS rendering
 - `report_pdf.py` - PDF engine selection and fallback handling
 - `converters.py` - Markdown-to-HTML/PDF conversion
-- `cli.py` - command-line entry point
+- `loganalysis/cli.py` - log analysis command-line entry point
+- `csvchunking/chunker.py` - CSV splitting logic and `ChunkResult`
+- `csvchunking/cli.py` - CSV chunking command-line entry point
 
 ## HTML/PDF conversion
 

{nm_tool_forge-0.1.0 → nm_tool_forge-0.2.4}/README.md

@@ -1,6 +1,6 @@
 # nm-tool-forge
 
-`nm-tool-forge` analyzes MigMan text log files with severity tokens such as `INFO`, `ERROR`, and `WARNING` and generates aggregated CSV, Markdown, HTML, and optional PDF reports.
+`nm-tool-forge` analyzes MigMan text log files with severity tokens such as `INFO`, `ERROR`, and `WARNING` and generates aggregated CSV, Markdown, HTML, and optional PDF reports. The package also includes `csvchunking`, a small helper for splitting large CSV files into migration-friendly chunks.
 
 The project uses a package-ready `src` layout. The legacy `log_analysis.py` file remains available as a thin compatibility entry point for older local setups.
 
@@ -9,10 +9,11 @@ The project uses a package-ready `src` layout. The legacy `log_analysis.py` file
 - Parse logical log entries from multi-line text logs
 - Normalize recurring error patterns for better aggregation
 - Generate aggregated CSV reports
-- Generate Markdown summary reports
-- Optionally convert reports to HTML and PDF
-- Keep a backup copy of analyzed log files
-- Run built-in self-tests from the CLI
+- Generate Markdown summary reports
+- Optionally convert reports to HTML and PDF
+- Keep a backup copy of analyzed log files
+- Split large CSV files into numbered chunks while preserving the header row
+- Run built-in self-tests from the CLI
 
 ## Installation
 
@@ -30,13 +31,15 @@ python -m pip install .[pdf,dev]
 
 ## Command-line usage
 
-After installation, both entry points are available:
-
-```powershell
-python -m loganalysis --help
-loganalysis --help
-nm-tool-forge --help
-```
+After installation, the CLI entry points are available:
+
+```powershell
+python -m loganalysis --help
+python -m csvchunking --help
+loganalysis --help
+nm-tool-forge --help
+csvchunking --help
+```
 
 Typical analysis run:
 
@@ -50,41 +53,82 @@ Analysis with HTML/PDF conversion:
 nm-tool-forge --logs-dir logs --out-dir log_analyse_out --convert
 ```
 
-Self-test mode:
-
-```powershell
-python -m loganalysis --self-test
-```
-
-Legacy compatibility call:
-
-```powershell
-python .\log_analysis.py --convert
+Self-test mode:
+
+```powershell
+python -m loganalysis --self-test
+```
+
+Legacy compatibility call:
+
+```powershell
+python .\log_analysis.py --convert
+```
+
+CSV chunking run:
+
+```powershell
+csvchunking "data\large_export.csv" --chunk-size 5000
+```
+
+The command creates an output directory next to the input file named after the CSV stem. For example, `data\large_export.csv` is split into files such as `data\large_export\large_export_01.csv`, `data\large_export\large_export_02.csv`, and so on.
+
+CSV chunking with an explicit encoding:
+
+```powershell
+python -m csvchunking "data\large_export.csv" --chunk-size 5000 --encoding utf-8-sig
+```
+
+Each chunk contains the original header row plus up to `--chunk-size` data rows. The delimiter is detected automatically; if detection fails, semicolon-separated CSV is used.
+
+## Supported CLI options
+
+Log analysis options:
+
+- `--logs-dir`
+- `--out-dir`
+- `--backup-dir`
+- `--top-examples`
+- `--convert`
+- `--self-test`
+
+CSV chunking options:
+
+- `input_file` - path to the CSV file to split
+- `--chunk-size` - required number of data rows per output file; must be greater than zero
+- `--encoding` - input and output encoding; defaults to `utf-8-sig`
+
+## Release process
+
+To publish a new release, always test on TestPyPI first, then upload to PyPI only after successful Conda smoke tests:
+
+```bash
+export TWINE_USERNAME="__token__"
+export TWINE_PASSWORD="pypi-..."
+
+bash scripts/release_testpypi.sh --bump patch
+bash scripts/release_pypi.sh --yes
 ```
 
-## Supported CLI options
-
-- `--logs-dir`
-- `--out-dir`
-- `--backup-dir`
-- `--top-examples`
-- `--convert`
-- `--self-test`
-
-## Library usage
-
-```python
+**Notes:**
+- Run and verify the TestPyPI release first, then upload the final package to PyPI.
+- PyPI versions cannot be overwritten or reused.
+
+## Library usage
+
+```python
 from pathlib import Path
 
 from loganalysis import (
     analyze_file,
     convert_report_md_to_html_pdf,
-    iter_logical_entries,
-    normalize_message,
-)
-
-result = analyze_file(Path("logs/app.txt"))
-print(result["norm_counts"])
+    iter_logical_entries,
+    normalize_message,
+)
+from csvchunking import split_csv
+
+result = analyze_file(Path("logs/app.txt"))
+print(result["norm_counts"])
 
 print(normalize_message(
     'Conversion: X =3100110. 138 The record was not found in table "Teile".'
@@ -95,20 +139,27 @@ for entry in iter_logical_entries(Path("logs/app.txt")):
 
 convert_report_md_to_html_pdf(
     Path("log_analyse_out/report.md"),
-    Path("log_analyse_out/report.html"),
-    Path("log_analyse_out/report.pdf"),
-)
-```
+    Path("log_analyse_out/report.html"),
+    Path("log_analyse_out/report.pdf"),
+)
+
+chunk_result = split_csv(Path("data/large_export.csv"), chunk_size=5000)
+print(chunk_result.output_dir)
+print(chunk_result.output_files)
+```
+
+`split_csv()` returns a `ChunkResult` with the input file, output directory, chunk size, processed data-row count, created file count, and generated output file paths.
 
 ## Project structure
 
 ```text
 .
-├─ pyproject.toml
-├─ src/loganalysis/
-├─ tests/
-├─ docs/
-└─ log_analysis.py
+├─ pyproject.toml
+├─ src/loganalysis/
+├─ src/csvchunking/
+├─ tests/
+├─ docs/
+└─ log_analysis.py
 ```
 
 Important modules:
@@ -118,9 +169,11 @@ Important modules:
 - `normalization.py` - message normalization
 - `report_markdown.py` - Markdown report model and rendering
 - `report_html.py` - HTML/CSS rendering
-- `report_pdf.py` - PDF engine selection and fallback handling
-- `converters.py` - Markdown-to-HTML/PDF conversion
-- `cli.py` - command-line entry point
+- `report_pdf.py` - PDF engine selection and fallback handling
+- `converters.py` - Markdown-to-HTML/PDF conversion
+- `loganalysis/cli.py` - log analysis command-line entry point
+- `csvchunking/chunker.py` - CSV splitting logic and `ChunkResult`
+- `csvchunking/cli.py` - CSV chunking command-line entry point
 
 ## HTML/PDF conversion
 

{nm_tool_forge-0.1.0 → nm_tool_forge-0.2.4}/pyproject.toml

@@ -1,67 +1,68 @@
-[build-system]
-requires = ["setuptools>=69", "wheel"]
-build-backend = "setuptools.build_meta"
-
-[project]
-name = "nm-tool-forge"
-version = "0.1.0"
-description = "Analyze MigMan log files and generate aggregated CSV, Markdown, HTML, and optional PDF reports."
-readme = { file = "README.md", content-type = "text/markdown" }
-requires-python = ">=3.10"
-license = "MIT"
-license-files = ["LICENSE"]
-authors = [
-  { name = "Stefan Ewald", email = "s.ew@outlook.de" }
-]
-keywords = ["migman", "logs", "analysis", "reporting", "csv", "markdown", "pdf"]
-classifiers = [
-  "Development Status :: 4 - Beta",
-  "Intended Audience :: Developers",
-  "Programming Language :: Python :: 3",
-  "Programming Language :: Python :: 3.10",
-  "Programming Language :: Python :: 3.11",
-  "Programming Language :: Python :: 3.12",
-  "Programming Language :: Python :: 3.13",
-  "Topic :: Software Development :: Libraries :: Python Modules",
-  "Topic :: Utilities",
-]
-
-dependencies = [
-  "chardet>=5.0",
-]
-
-[project.optional-dependencies]
-pdf = [
-  "weasyprint>=62",
-]
-dev = [
-  "pytest>=8.0",
-  "build>=1.2",
-  "twine>=5.0",
-  "ruff>=0.11",
-]
-
-[project.urls]
-Homepage = "https://github.com/Jack736-ui/migman_log"
-Issues = "https://github.com/Jack736-ui/migman_log/issues"
-
-[project.scripts]
-nm-tool-forge = "loganalysis.cli:main"
-loganalysis = "loganalysis.cli:main"
-
-[tool.setuptools]
-package-dir = { "" = "src" }
-
-[tool.setuptools.packages.find]
-where = ["src"]
-
-[tool.pytest.ini_options]
-testpaths = ["tests"]
-addopts = "--basetemp=tests_tmp"
-
-[tool.ruff]
-line-length = 120
-target-version = "py310"
-
-[tool.ruff.lint]
+[build-system]
+requires = ["setuptools>=69", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "nm-tool-forge"
+version = "0.2.4"
+description = "Analyze MigMan log files and generate aggregated CSV, Markdown, HTML, and optional PDF reports."
+readme = { file = "README.md", content-type = "text/markdown" }
+requires-python = ">=3.10"
+license = "MIT"
+license-files = ["LICENSE"]
+authors = [
+  { name = "Stefan Ewald", email = "s.ew@outlook.de" }
+]
+keywords = ["migman", "logs", "analysis", "reporting", "csv", "markdown", "pdf"]
+classifiers = [
+  "Development Status :: 4 - Beta",
+  "Intended Audience :: Developers",
+  "Programming Language :: Python :: 3",
+  "Programming Language :: Python :: 3.10",
+  "Programming Language :: Python :: 3.11",
+  "Programming Language :: Python :: 3.12",
+  "Programming Language :: Python :: 3.13",
+  "Topic :: Software Development :: Libraries :: Python Modules",
+  "Topic :: Utilities",
+]
+
+dependencies = [
+  "chardet>=5.0",
+]
+
+[project.optional-dependencies]
+pdf = [
+  "weasyprint>=62",
+]
+dev = [
+  "pytest>=8.0",
+  "build>=1.2",
+  "twine>=5.0",
+  "ruff>=0.11",
+]
+
+[project.urls]
+Homepage = "https://github.com/Jack736-ui/migman_log"
+Issues = "https://github.com/Jack736-ui/migman_log/issues"
+
+[project.scripts]
+nm-tool-forge = "loganalysis.cli:main"
+loganalysis = "loganalysis.cli:main"
+csvchunking = "csvchunking.cli:main"
+
+[tool.setuptools]
+package-dir = { "" = "src" }
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+addopts = "--basetemp=tests_tmp"
+
+[tool.ruff]
+line-length = 120
+target-version = "py310"
+
+[tool.ruff.lint]
 select = ["E", "F", "I", "B", "UP"]

nm_tool_forge-0.2.4/src/csvchunking/__init__.py

@@ -0,0 +1,4 @@
+from .chunker import ChunkResult, split_csv
+
+__all__ = ["ChunkResult", "split_csv"]
+__version__ = "0.2.4"

nm_tool_forge-0.2.4/src/csvchunking/__main__.py

@@ -0,0 +1,4 @@
+from .cli import main
+
+if __name__ == "__main__":
+    main()

nm_tool_forge-0.2.4/src/csvchunking/chunker.py

@@ -0,0 +1,76 @@
+import csv
+from dataclasses import dataclass
+from pathlib import Path
+
+
+@dataclass(frozen=True)
+class ChunkResult:
+    input_file: Path
+    output_dir: Path
+    chunk_size: int
+    data_rows_processed: int
+    files_created: int
+    output_files: tuple[Path, ...]
+
+
+def split_csv(
+    input_file: Path,
+    chunk_size: int,
+    encoding: str = "utf-8-sig",
+) -> ChunkResult:
+    if not Path(input_file).is_file():
+        raise FileNotFoundError(f"Input file not found: {input_file}")
+    if chunk_size <= 0:
+        raise ValueError("chunk_size must be greater than 0")
+
+    input_file = Path(input_file)
+    output_dir = input_file.parent / input_file.stem
+    output_dir.mkdir(exist_ok=True)
+
+    # Detect the delimiter automatically.
+    with open(input_file, encoding=encoding, newline="") as f:
+        sample = f.read(4096)
+        f.seek(0)
+        sniffer = csv.Sniffer()
+        try:
+            dialect = sniffer.sniff(sample)
+        except Exception:
+            dialect = csv.excel
+            dialect.delimiter = ";"
+        reader = csv.reader(f, dialect)
+        try:
+            header = next(reader)
+        except StopIteration as exc:
+            raise ValueError("Input file is empty.") from exc
+        chunk = []
+        file_count = 0
+        data_rows = 0
+        output_files = []
+        for row in reader:
+            chunk.append(row)
+            data_rows += 1
+            if len(chunk) == chunk_size:
+                file_count += 1
+                out_path = output_dir / f"{input_file.stem}_{file_count:02d}{input_file.suffix}"
+                with open(out_path, "w", encoding=encoding, newline="") as out:
+                    writer = csv.writer(out, dialect)
+                    writer.writerow(header)
+                    writer.writerows(chunk)
+                output_files.append(out_path)
+                chunk = []
+        if chunk:
+            file_count += 1
+            out_path = output_dir / f"{input_file.stem}_{file_count:02d}{input_file.suffix}"
+            with open(out_path, "w", encoding=encoding, newline="") as out:
+                writer = csv.writer(out, dialect)
+                writer.writerow(header)
+                writer.writerows(chunk)
+            output_files.append(out_path)
+    return ChunkResult(
+        input_file=input_file,
+        output_dir=output_dir,
+        chunk_size=chunk_size,
+        data_rows_processed=data_rows,
+        files_created=file_count,
+        output_files=tuple(output_files),
+    )

nm_tool_forge-0.2.4/src/csvchunking/cli.py

@@ -0,0 +1,31 @@
+import argparse
+import sys
+from pathlib import Path
+
+from .chunker import split_csv
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(
+        description="Split a large CSV file into smaller chunks with a header row.",
+    )
+    parser.add_argument("input_file", help="Path to the CSV file")
+    parser.add_argument(
+        "--chunk-size",
+        type=int,
+        required=True,
+        help="Number of data rows per output file; must be greater than 0",
+    )
+    parser.add_argument("--encoding", default="utf-8-sig", help="Input and output encoding (Default: utf-8-sig)")
+    args = parser.parse_args()
+    try:
+        result = split_csv(Path(args.input_file), args.chunk_size, encoding=args.encoding)
+    except Exception as e:
+        print(f"Error: {e}", file=sys.stderr)
+        sys.exit(1)
+    print("CSV chunking completed.")
+    print(f"- Input: {result.input_file}")
+    print(f"- Output directory: {result.output_dir}")
+    print(f"- Chunk size: {result.chunk_size}")
+    print(f"- Data rows processed: {result.data_rows_processed}")
+    print(f"- Files created: {result.files_created}")

{nm_tool_forge-0.1.0 → nm_tool_forge-0.2.4}/src/loganalysis/__init__.py

@@ -1,16 +1,16 @@
-from __future__ import annotations
-
-from .analysis import analyze_file, run_analysis
-from .converters import convert_report_md_to_html_pdf
-from .normalization import normalize_message
-from .parsing import iter_logical_entries
-
-__all__ = [
-    "analyze_file",
-    "convert_report_md_to_html_pdf",
-    "iter_logical_entries",
-    "normalize_message",
-    "run_analysis",
-]
-
-__version__ = "0.1.0"
+from __future__ import annotations
+
+from .analysis import analyze_file, run_analysis
+from .converters import convert_report_md_to_html_pdf
+from .normalization import normalize_message
+from .parsing import iter_logical_entries
+
+__all__ = [
+    "analyze_file",
+    "convert_report_md_to_html_pdf",
+    "iter_logical_entries",
+    "normalize_message",
+    "run_analysis",
+]
+
+__version__ = "0.2.4"

{nm_tool_forge-0.1.0 → nm_tool_forge-0.2.4}/src/loganalysis/cli.py

@@ -1,8 +1,8 @@
 from __future__ import annotations
 
-import argparse
-from pathlib import Path
-from typing import Sequence
+import argparse
+from collections.abc import Sequence
+from pathlib import Path
 
 from .analysis import NoLogFilesError, run_analysis
 from .constants import DEFAULT_LOGS_DIR, DEFAULT_OUT_DIR, DEFAULT_TOP_EXAMPLES, EXIT_SUCCESS
@@ -16,7 +16,11 @@ def build_parser() -> argparse.ArgumentParser:
     parser = argparse.ArgumentParser(
         description="Aggregated analysis of log files (INFO/ERROR/WARNING) in logs/*.txt",
     )
-    parser.add_argument("--logs-dir", default=DEFAULT_LOGS_DIR, help=f"Subdirectory with log files (Default: {DEFAULT_LOGS_DIR})")
+    parser.add_argument(
+        "--logs-dir",
+        default=DEFAULT_LOGS_DIR,
+        help=f"Subdirectory with log files (Default: {DEFAULT_LOGS_DIR})",
+    )
     parser.add_argument("--out-dir", default=DEFAULT_OUT_DIR, help=f"Output directory (Default: {DEFAULT_OUT_DIR})")
     parser.add_argument("--backup-dir", default=None, help="Backup directory (Default: <out-dir>/backup)")
     parser.add_argument(

{nm_tool_forge-0.1.0 → nm_tool_forge-0.2.4}/src/loganalysis/csv_export.py

@@ -1,10 +1,10 @@
-from __future__ import annotations
-
-import csv
-from pathlib import Path
-from typing import Iterable
-
-from .filesystem import ensure_dir
+from __future__ import annotations
+
+import csv
+from collections.abc import Iterable
+from pathlib import Path
+
+from .filesystem import ensure_dir
 
 
 def write_csv(path: Path, rows: Iterable[tuple[str, str, int]], headers: list[str]) -> None:

{nm_tool_forge-0.1.0 → nm_tool_forge-0.2.4}/src/loganalysis/parsing.py

@@ -1,9 +1,9 @@
-from __future__ import annotations
-
-from pathlib import Path
-from typing import Iterable
-
-from .constants import RE_ENTRY_START, RE_LINE_PREFIX, RE_TRAILING_DATASET, RE_WHITESPACE, SEVERITY_ALIASES
+from __future__ import annotations
+
+from collections.abc import Iterable
+from pathlib import Path
+
+from .constants import RE_ENTRY_START, RE_LINE_PREFIX, RE_TRAILING_DATASET, RE_WHITESPACE, SEVERITY_ALIASES
 from .encoding import detect_encoding
 from .models import ParsedLine
 

{nm_tool_forge-0.1.0 → nm_tool_forge-0.2.4}/src/loganalysis/report_markdown.py

@@ -52,12 +52,14 @@ def build_markdown_report(
             continue
 
         lines.append("| Severity | Count | Normalized message | Examples |")
-        lines.append("|---|---:|---|---|")
-        for (severity, normalized_message), count in top_norm:
-            examples_counter = analysis.norm_examples[(severity, normalized_message)]
-            examples = [f"{message} ({amount})" for message, amount in examples_counter.most_common(config.top_examples)]
-            examples_text = "<br>".join(examples) if examples else ""
-            lines.append(f"| {severity} | {count} | {normalized_message} | {examples_text} |")
+        lines.append("|---|---:|---|---|")
+        for (severity, normalized_message), count in top_norm:
+            examples_counter = analysis.norm_examples[(severity, normalized_message)]
+            examples = [
+                f"{message} ({amount})" for message, amount in examples_counter.most_common(config.top_examples)
+            ]
+            examples_text = "<br>".join(examples) if examples else ""
+            lines.append(f"| {severity} | {count} | {normalized_message} | {examples_text} |")
         lines.append("")
 
     lines.append("## Overall summary (all files)")
@@ -65,12 +67,14 @@ def build_markdown_report(
     top_global = _top_counter_items(summary.global_norm, REPORT_TOP_GLOBAL)
     if top_global:
         lines.append("| Severity | Count | Normalized message | Examples |")
-        lines.append("|---|---:|---|---|")
-        for (severity, normalized_message), count in top_global:
-            examples_counter = summary.global_norm_examples[(severity, normalized_message)]
-            examples = [f"{message} ({amount})" for message, amount in examples_counter.most_common(config.top_examples)]
-            examples_text = "<br>".join(examples) if examples else ""
-            lines.append(f"| {severity} | {count} | {normalized_message} | {examples_text} |")
+        lines.append("|---|---:|---|---|")
+        for (severity, normalized_message), count in top_global:
+            examples_counter = summary.global_norm_examples[(severity, normalized_message)]
+            examples = [
+                f"{message} ({amount})" for message, amount in examples_counter.most_common(config.top_examples)
+            ]
+            examples_text = "<br>".join(examples) if examples else ""
+            lines.append(f"| {severity} | {count} | {normalized_message} | {examples_text} |")
         lines.append("")
     else:
         lines.append("_No messages found._")

{nm_tool_forge-0.1.0 → nm_tool_forge-0.2.4}/src/loganalysis/report_pdf.py

@@ -1,10 +1,15 @@
 from __future__ import annotations
 
 import shutil
-from contextlib import redirect_stderr, redirect_stdout
-from io import StringIO
-
-from .constants import COMMON_MOJIBAKE_TOKENS, LATEX_PDF_ENGINES, LATEX_SPECIAL_CHAR_REPLACEMENTS, RE_MARKDOWN_TABLE_SEPARATOR
+from contextlib import redirect_stderr, redirect_stdout
+from io import StringIO
+
+from .constants import (
+    COMMON_MOJIBAKE_TOKENS,
+    LATEX_PDF_ENGINES,
+    LATEX_SPECIAL_CHAR_REPLACEMENTS,
+    RE_MARKDOWN_TABLE_SEPARATOR,
+)
 
 
 def select_pdf_engine() -> str | None:

{nm_tool_forge-0.1.0 → nm_tool_forge-0.2.4}/src/loganalysis/selftest.py

@@ -9,17 +9,23 @@ from .report_pdf import build_pdf_safe_markdown, escape_latex_text, make_markdow
 
 
 def run_self_tests() -> None:
-    """Run deterministic built-in assertions for quick local verification."""
-
-    for raw_message, expected in NORMALIZATION_SELF_TEST_CASES:
-        actual = normalize_message(raw_message)
-        assert actual == expected, f"normalize_message({raw_message!r}) -> {actual!r}, expected {expected!r}"
-
-    assert is_entry_start("ERROR\tLine 1: tab-separated severity")
-    assert escape_latex_text(r"D:\DATEN_UEBERNAHME\A&B") == r"D:\textbackslash{}DATEN\_UEBERNAHME\textbackslash{}A\&B"
-    assert make_markdown_table_line_pdf_safe("|---|---:|---|") == "|---|---:|---|"
-    assert make_markdown_table_line_pdf_safe(r"| ERROR | D:\DATEN_1<br>foo |") == r"| ERROR | D:\textbackslash{}DATEN\_1 ; foo |"
-    assert build_pdf_safe_markdown("plain\n| A | B |\n").endswith("\n")
+    """Run deterministic built-in assertions for quick local verification."""
+
+    for raw_message, expected in NORMALIZATION_SELF_TEST_CASES:
+        actual = normalize_message(raw_message)
+        assert actual == expected, f"normalize_message({raw_message!r}) -> {actual!r}, expected {expected!r}"
+
+    assert is_entry_start("ERROR\tLine 1: tab-separated severity")
+    assert (
+        escape_latex_text(r"D:\DATEN_UEBERNAHME\A&B")
+        == r"D:\textbackslash{}DATEN\_UEBERNAHME\textbackslash{}A\&B"
+    )
+    assert make_markdown_table_line_pdf_safe("|---|---:|---|") == "|---|---:|---|"
+    assert (
+        make_markdown_table_line_pdf_safe(r"| ERROR | D:\DATEN_1<br>foo |")
+        == r"| ERROR | D:\textbackslash{}DATEN\_1 ; foo |"
+    )
+    assert build_pdf_safe_markdown("plain\n| A | B |\n").endswith("\n")
 
     sample_report_markdown = """# Log Analysis Report
 

{nm_tool_forge-0.1.0 → nm_tool_forge-0.2.4}/src/nm_tool_forge.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: nm-tool-forge
-Version: 0.1.0
+Version: 0.2.4
 Summary: Analyze MigMan log files and generate aggregated CSV, Markdown, HTML, and optional PDF reports.
 Author-email: Stefan Ewald <s.ew@outlook.de>
 License-Expression: MIT
@@ -31,7 +31,7 @@ Dynamic: license-file
 
 # nm-tool-forge
 
-`nm-tool-forge` analyzes MigMan text log files with severity tokens such as `INFO`, `ERROR`, and `WARNING` and generates aggregated CSV, Markdown, HTML, and optional PDF reports.
+`nm-tool-forge` analyzes MigMan text log files with severity tokens such as `INFO`, `ERROR`, and `WARNING` and generates aggregated CSV, Markdown, HTML, and optional PDF reports. The package also includes `csvchunking`, a small helper for splitting large CSV files into migration-friendly chunks.
 
 The project uses a package-ready `src` layout. The legacy `log_analysis.py` file remains available as a thin compatibility entry point for older local setups.
 
@@ -43,6 +43,7 @@ The project uses a package-ready `src` layout. The legacy `log_analysis.py` file
 - Generate Markdown summary reports
 - Optionally convert reports to HTML and PDF
 - Keep a backup copy of analyzed log files
+- Split large CSV files into numbered chunks while preserving the header row
 - Run built-in self-tests from the CLI
 
 ## Installation
@@ -61,12 +62,14 @@ python -m pip install .[pdf,dev]
 
 ## Command-line usage
 
-After installation, both entry points are available:
+After installation, the CLI entry points are available:
 
 ```powershell
 python -m loganalysis --help
+python -m csvchunking --help
 loganalysis --help
 nm-tool-forge --help
+csvchunking --help
 ```
 
 Typical analysis run:
@@ -93,8 +96,26 @@ Legacy compatibility call:
 python .\log_analysis.py --convert
 ```
 
+CSV chunking run:
+
+```powershell
+csvchunking "data\large_export.csv" --chunk-size 5000
+```
+
+The command creates an output directory next to the input file named after the CSV stem. For example, `data\large_export.csv` is split into files such as `data\large_export\large_export_01.csv`, `data\large_export\large_export_02.csv`, and so on.
+
+CSV chunking with an explicit encoding:
+
+```powershell
+python -m csvchunking "data\large_export.csv" --chunk-size 5000 --encoding utf-8-sig
+```
+
+Each chunk contains the original header row plus up to `--chunk-size` data rows. The delimiter is detected automatically; if detection fails, semicolon-separated CSV is used.
+
 ## Supported CLI options
 
+Log analysis options:
+
 - `--logs-dir`
 - `--out-dir`
 - `--backup-dir`
@@ -102,6 +123,28 @@ python .\log_analysis.py --convert
 - `--convert`
 - `--self-test`
 
+CSV chunking options:
+
+- `input_file` - path to the CSV file to split
+- `--chunk-size` - required number of data rows per output file; must be greater than zero
+- `--encoding` - input and output encoding; defaults to `utf-8-sig`
+
+## Release process
+
+To publish a new release, always test on TestPyPI first, then upload to PyPI only after successful Conda smoke tests:
+
+```bash
+export TWINE_USERNAME="__token__"
+export TWINE_PASSWORD="pypi-..."
+
+bash scripts/release_testpypi.sh --bump patch
+bash scripts/release_pypi.sh --yes
+```
+
+**Notes:**
+- Run and verify the TestPyPI release first, then upload the final package to PyPI.
+- PyPI versions cannot be overwritten or reused.
+
 ## Library usage
 
 ```python
@@ -113,6 +156,7 @@ from loganalysis import (
     iter_logical_entries,
     normalize_message,
 )
+from csvchunking import split_csv
 
 result = analyze_file(Path("logs/app.txt"))
 print(result["norm_counts"])
@@ -129,14 +173,21 @@ convert_report_md_to_html_pdf(
     Path("log_analyse_out/report.html"),
     Path("log_analyse_out/report.pdf"),
 )
+
+chunk_result = split_csv(Path("data/large_export.csv"), chunk_size=5000)
+print(chunk_result.output_dir)
+print(chunk_result.output_files)
 ```
 
+`split_csv()` returns a `ChunkResult` with the input file, output directory, chunk size, processed data-row count, created file count, and generated output file paths.
+
 ## Project structure
 
 ```text
 .
 ├─ pyproject.toml
 ├─ src/loganalysis/
+├─ src/csvchunking/
 ├─ tests/
 ├─ docs/
 └─ log_analysis.py
@@ -151,7 +202,9 @@ Important modules:
 - `report_html.py` - HTML/CSS rendering
 - `report_pdf.py` - PDF engine selection and fallback handling
 - `converters.py` - Markdown-to-HTML/PDF conversion
-- `cli.py` - command-line entry point
+- `loganalysis/cli.py` - log analysis command-line entry point
+- `csvchunking/chunker.py` - CSV splitting logic and `ChunkResult`
+- `csvchunking/cli.py` - CSV chunking command-line entry point
 
 ## HTML/PDF conversion
 

{nm_tool_forge-0.1.0 → nm_tool_forge-0.2.4}/src/nm_tool_forge.egg-info/SOURCES.txt

@@ -1,6 +1,10 @@
 LICENSE
 README.md
 pyproject.toml
+src/csvchunking/__init__.py
+src/csvchunking/__main__.py
+src/csvchunking/chunker.py
+src/csvchunking/cli.py
 src/loganalysis/__init__.py
 src/loganalysis/__main__.py
 src/loganalysis/analysis.py
@@ -25,6 +29,7 @@ src/nm_tool_forge.egg-info/entry_points.txt
 src/nm_tool_forge.egg-info/requires.txt
 src/nm_tool_forge.egg-info/top_level.txt
 tests/test_analysis.py
+tests/test_csvchunking.py
 tests/test_normalization.py
 tests/test_parsing.py
 tests/test_report_html.py

{nm_tool_forge-0.1.0 → nm_tool_forge-0.2.4}/src/nm_tool_forge.egg-info/entry_points.txt

@@ -1,3 +1,4 @@
 [console_scripts]
+csvchunking = csvchunking.cli:main
 loganalysis = loganalysis.cli:main
 nm-tool-forge = loganalysis.cli:main

{nm_tool_forge-0.1.0 → nm_tool_forge-0.2.4}/src/nm_tool_forge.egg-info/top_level.txt

@@ -1 +1,2 @@
+csvchunking
 loganalysis

{nm_tool_forge-0.1.0 → nm_tool_forge-0.2.4}/tests/test_analysis.py

@@ -18,11 +18,12 @@ def test_analyze_file_aggregates_raw_and_normalized_counts(tmp_path: Path) -> No
     result = analyze_file(log_path)
 
     assert result.total_lines == 3
-    assert result.total_entries == 3
-    assert result.unknown_lines == 0
-    assert result.raw_counts[("WARNING", "Different issue")] == 1
-    assert result.norm_counts[("ERROR", 'Conversion: X =<VALUE> The record was not found in table "Teile".')] == 2
-    assert len(result.norm_examples[("ERROR", 'Conversion: X =<VALUE> The record was not found in table "Teile".')]) == 2
+    assert result.total_entries == 3
+    assert result.unknown_lines == 0
+    assert result.raw_counts[("WARNING", "Different issue")] == 1
+    normalized_key = ("ERROR", 'Conversion: X =<VALUE> The record was not found in table "Teile".')
+    assert result.norm_counts[normalized_key] == 2
+    assert len(result.norm_examples[normalized_key]) == 2
 
 
 def test_run_analysis_writes_outputs_and_report(tmp_path: Path) -> None:

nm_tool_forge-0.2.4/tests/test_csvchunking.py

@@ -0,0 +1,63 @@
+import pytest
+
+from csvchunking.chunker import split_csv
+
+
+def make_csv(tmp_path, name, header, rows, encoding="utf-8-sig", delimiter=";"):
+    file = tmp_path / name
+    with open(file, "w", encoding=encoding, newline="") as f:
+        f.write(delimiter.join(header) + "\n")
+        for row in rows:
+            f.write(delimiter.join(row) + "\n")
+    return file
+
+
+def test_regular_split(tmp_path):
+    header = ["col1", "col2"]
+    rows = [["A", "1"], ["B", "2"], ["C", "3"], ["D", "4"], ["E", "5"]]
+    file = make_csv(tmp_path, "sample.csv", header, rows)
+    result = split_csv(file, chunk_size=2)
+    assert result.files_created == 3
+    for out in result.output_files:
+        with open(out, encoding="utf-8-sig") as f:
+            lines = f.read().splitlines()
+            assert lines[0] == "col1;col2"
+    assert (result.output_dir / "sample_01.csv").exists()
+    assert (result.output_dir / "sample_02.csv").exists()
+    assert (result.output_dir / "sample_03.csv").exists()
+
+
+def test_header_in_each_file(tmp_path):
+    header = ["foo", "bar"]
+    rows = [["x", "1"], ["y", "2"], ["z", "3"]]
+    file = make_csv(tmp_path, "test.csv", header, rows)
+    result = split_csv(file, chunk_size=1)
+    for out in result.output_files:
+        with open(out, encoding="utf-8-sig") as f:
+            assert f.readline().strip() == "foo;bar"
+
+
+def test_filename_with_spaces(tmp_path):
+    header = ["a", "b"]
+    rows = [["1", "2"]]
+    file = make_csv(tmp_path, "Part-Storage Areas Relationships.csv", header, rows)
+    result = split_csv(file, chunk_size=1)
+    assert result.output_dir.name == "Part-Storage Areas Relationships"
+    assert (result.output_dir / "Part-Storage Areas Relationships_01.csv").exists()
+
+
+def test_invalid_chunk_size(tmp_path):
+    header = ["a", "b"]
+    rows = [["1", "2"]]
+    file = make_csv(tmp_path, "fail.csv", header, rows)
+    with pytest.raises(ValueError):
+        split_csv(file, chunk_size=0)
+    with pytest.raises(ValueError):
+        split_csv(file, chunk_size=-1)
+
+
+def test_empty_file(tmp_path):
+    file = tmp_path / "empty.csv"
+    file.write_text("")
+    with pytest.raises(ValueError):
+        split_csv(file, chunk_size=1)

{nm_tool_forge-0.1.0 → nm_tool_forge-0.2.4}/tests/test_report_markdown.py

@@ -16,7 +16,9 @@ def test_build_and_parse_markdown_report_roundtrip(tmp_path: Path) -> None:
         unknown_lines=0,
         raw_counts=Counter({("ERROR", 'Conversion: X =3100110. 138 The record was not found in table "Teile".'): 2}),
         norm_counts=Counter({normalized_key: 2}),
-        norm_examples={normalized_key: Counter({'Conversion: X =3100110. 138 The record was not found in table "Teile".': 2})},
+        norm_examples={
+            normalized_key: Counter({'Conversion: X =3100110. 138 The record was not found in table "Teile".': 2})
+        },
         backup_path=tmp_path / "backup" / "demo.txt.bak",
     )
     summary = AnalysisSummary(