nm-tool-forge 0.1.0__tar.gz

nm_tool_forge-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

nm_tool_forge-0.1.0/PKG-INFO

@@ -0,0 +1,198 @@
+Metadata-Version: 2.4
+Name: nm-tool-forge
+Version: 0.1.0
+Summary: Analyze MigMan log files and generate aggregated CSV, Markdown, HTML, and optional PDF reports.
+Author-email: Stefan Ewald <s.ew@outlook.de>
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/Jack736-ui/migman_log
+Project-URL: Issues, https://github.com/Jack736-ui/migman_log/issues
+Keywords: migman,logs,analysis,reporting,csv,markdown,pdf
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Topic :: Utilities
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: chardet>=5.0
+Provides-Extra: pdf
+Requires-Dist: weasyprint>=62; extra == "pdf"
+Provides-Extra: dev
+Requires-Dist: pytest>=8.0; extra == "dev"
+Requires-Dist: build>=1.2; extra == "dev"
+Requires-Dist: twine>=5.0; extra == "dev"
+Requires-Dist: ruff>=0.11; extra == "dev"
+Dynamic: license-file
+
+# nm-tool-forge
+
+`nm-tool-forge` analyzes MigMan text log files with severity tokens such as `INFO`, `ERROR`, and `WARNING` and generates aggregated CSV, Markdown, HTML, and optional PDF reports.
+
+The project uses a package-ready `src` layout. The legacy `log_analysis.py` file remains available as a thin compatibility entry point for older local setups.
+
+## Features
+
+- Parse logical log entries from multi-line text logs
+- Normalize recurring error patterns for better aggregation
+- Generate aggregated CSV reports
+- Generate Markdown summary reports
+- Optionally convert reports to HTML and PDF
+- Keep a backup copy of analyzed log files
+- Run built-in self-tests from the CLI
+
+## Installation
+
+Basic installation from a local checkout:
+
+```powershell
+python -m pip install .
+```
+
+Installation with optional PDF support and developer tools:
+
+```powershell
+python -m pip install .[pdf,dev]
+```
+
+## Command-line usage
+
+After installation, both entry points are available:
+
+```powershell
+python -m loganalysis --help
+loganalysis --help
+nm-tool-forge --help
+```
+
+Typical analysis run:
+
+```powershell
+nm-tool-forge --logs-dir logs --out-dir log_analyse_out
+```
+
+Analysis with HTML/PDF conversion:
+
+```powershell
+nm-tool-forge --logs-dir logs --out-dir log_analyse_out --convert
+```
+
+Self-test mode:
+
+```powershell
+python -m loganalysis --self-test
+```
+
+Legacy compatibility call:
+
+```powershell
+python .\log_analysis.py --convert
+```
+
+## Supported CLI options
+
+- `--logs-dir`
+- `--out-dir`
+- `--backup-dir`
+- `--top-examples`
+- `--convert`
+- `--self-test`
+
+## Library usage
+
+```python
+from pathlib import Path
+
+from loganalysis import (
+    analyze_file,
+    convert_report_md_to_html_pdf,
+    iter_logical_entries,
+    normalize_message,
+)
+
+result = analyze_file(Path("logs/app.txt"))
+print(result["norm_counts"])
+
+print(normalize_message(
+    'Conversion: X =3100110. 138 The record was not found in table "Teile".'
+))
+
+for entry in iter_logical_entries(Path("logs/app.txt")):
+    print(entry)
+
+convert_report_md_to_html_pdf(
+    Path("log_analyse_out/report.md"),
+    Path("log_analyse_out/report.html"),
+    Path("log_analyse_out/report.pdf"),
+)
+```
+
+## Project structure
+
+```text
+.
+├─ pyproject.toml
+├─ src/loganalysis/
+├─ tests/
+├─ docs/
+└─ log_analysis.py
+```
+
+Important modules:
+
+- `analysis.py` - file-level and overall aggregation
+- `parsing.py` - logical entry detection and parsing
+- `normalization.py` - message normalization
+- `report_markdown.py` - Markdown report model and rendering
+- `report_html.py` - HTML/CSS rendering
+- `report_pdf.py` - PDF engine selection and fallback handling
+- `converters.py` - Markdown-to-HTML/PDF conversion
+- `cli.py` - command-line entry point
+
+## HTML/PDF conversion
+
+Report conversion is intentionally optional:
+
+- `report.md` remains the primary human-readable output
+- `report.html` is generated from the internal report model
+- `report.pdf` is created when supported PDF tooling is available
+
+PDF engine preference order:
+
+1. `weasyprint`
+2. `wkhtmltopdf`
+3. `pandoc` + `xelatex` or `pdflatex`
+
+If no supported PDF engine is available, the analysis still succeeds and generates Markdown and HTML output.
+
+Windows-specific setup notes:
+
+- `docs/install_gtk_weasyprint_windows.md`
+- `docs/install_xelatex_windows.md`
+
+## Tests
+
+```powershell
+pytest
+```
+
+## Local build
+
+```powershell
+python -m build
+```
+
+Expected artifacts:
+
+- `dist/*.tar.gz`
+- `dist/*.whl`
+
+## Notes
+
+The package name on PyPI/TestPyPI is `nm-tool-forge`, while the current Python import package remains `loganalysis`.
+
+This keeps the first public release small and low-risk. A later follow-up release can still rename the import package if desired.

nm_tool_forge-0.1.0/README.md

@@ -0,0 +1,167 @@
+# nm-tool-forge
+
+`nm-tool-forge` analyzes MigMan text log files with severity tokens such as `INFO`, `ERROR`, and `WARNING` and generates aggregated CSV, Markdown, HTML, and optional PDF reports.
+
+The project uses a package-ready `src` layout. The legacy `log_analysis.py` file remains available as a thin compatibility entry point for older local setups.
+
+## Features
+
+- Parse logical log entries from multi-line text logs
+- Normalize recurring error patterns for better aggregation
+- Generate aggregated CSV reports
+- Generate Markdown summary reports
+- Optionally convert reports to HTML and PDF
+- Keep a backup copy of analyzed log files
+- Run built-in self-tests from the CLI
+
+## Installation
+
+Basic installation from a local checkout:
+
+```powershell
+python -m pip install .
+```
+
+Installation with optional PDF support and developer tools:
+
+```powershell
+python -m pip install .[pdf,dev]
+```
+
+## Command-line usage
+
+After installation, both entry points are available:
+
+```powershell
+python -m loganalysis --help
+loganalysis --help
+nm-tool-forge --help
+```
+
+Typical analysis run:
+
+```powershell
+nm-tool-forge --logs-dir logs --out-dir log_analyse_out
+```
+
+Analysis with HTML/PDF conversion:
+
+```powershell
+nm-tool-forge --logs-dir logs --out-dir log_analyse_out --convert
+```
+
+Self-test mode:
+
+```powershell
+python -m loganalysis --self-test
+```
+
+Legacy compatibility call:
+
+```powershell
+python .\log_analysis.py --convert
+```
+
+## Supported CLI options
+
+- `--logs-dir`
+- `--out-dir`
+- `--backup-dir`
+- `--top-examples`
+- `--convert`
+- `--self-test`
+
+## Library usage
+
+```python
+from pathlib import Path
+
+from loganalysis import (
+    analyze_file,
+    convert_report_md_to_html_pdf,
+    iter_logical_entries,
+    normalize_message,
+)
+
+result = analyze_file(Path("logs/app.txt"))
+print(result["norm_counts"])
+
+print(normalize_message(
+    'Conversion: X =3100110. 138 The record was not found in table "Teile".'
+))
+
+for entry in iter_logical_entries(Path("logs/app.txt")):
+    print(entry)
+
+convert_report_md_to_html_pdf(
+    Path("log_analyse_out/report.md"),
+    Path("log_analyse_out/report.html"),
+    Path("log_analyse_out/report.pdf"),
+)
+```
+
+## Project structure
+
+```text
+.
+├─ pyproject.toml
+├─ src/loganalysis/
+├─ tests/
+├─ docs/
+└─ log_analysis.py
+```
+
+Important modules:
+
+- `analysis.py` - file-level and overall aggregation
+- `parsing.py` - logical entry detection and parsing
+- `normalization.py` - message normalization
+- `report_markdown.py` - Markdown report model and rendering
+- `report_html.py` - HTML/CSS rendering
+- `report_pdf.py` - PDF engine selection and fallback handling
+- `converters.py` - Markdown-to-HTML/PDF conversion
+- `cli.py` - command-line entry point
+
+## HTML/PDF conversion
+
+Report conversion is intentionally optional:
+
+- `report.md` remains the primary human-readable output
+- `report.html` is generated from the internal report model
+- `report.pdf` is created when supported PDF tooling is available
+
+PDF engine preference order:
+
+1. `weasyprint`
+2. `wkhtmltopdf`
+3. `pandoc` + `xelatex` or `pdflatex`
+
+If no supported PDF engine is available, the analysis still succeeds and generates Markdown and HTML output.
+
+Windows-specific setup notes:
+
+- `docs/install_gtk_weasyprint_windows.md`
+- `docs/install_xelatex_windows.md`
+
+## Tests
+
+```powershell
+pytest
+```
+
+## Local build
+
+```powershell
+python -m build
+```
+
+Expected artifacts:
+
+- `dist/*.tar.gz`
+- `dist/*.whl`
+
+## Notes
+
+The package name on PyPI/TestPyPI is `nm-tool-forge`, while the current Python import package remains `loganalysis`.
+
+This keeps the first public release small and low-risk. A later follow-up release can still rename the import package if desired.

nm_tool_forge-0.1.0/pyproject.toml

@@ -0,0 +1,67 @@
+[build-system]
+requires = ["setuptools>=69", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "nm-tool-forge"
+version = "0.1.0"
+description = "Analyze MigMan log files and generate aggregated CSV, Markdown, HTML, and optional PDF reports."
+readme = { file = "README.md", content-type = "text/markdown" }
+requires-python = ">=3.10"
+license = "MIT"
+license-files = ["LICENSE"]
+authors = [
+  { name = "Stefan Ewald", email = "s.ew@outlook.de" }
+]
+keywords = ["migman", "logs", "analysis", "reporting", "csv", "markdown", "pdf"]
+classifiers = [
+  "Development Status :: 4 - Beta",
+  "Intended Audience :: Developers",
+  "Programming Language :: Python :: 3",
+  "Programming Language :: Python :: 3.10",
+  "Programming Language :: Python :: 3.11",
+  "Programming Language :: Python :: 3.12",
+  "Programming Language :: Python :: 3.13",
+  "Topic :: Software Development :: Libraries :: Python Modules",
+  "Topic :: Utilities",
+]
+
+dependencies = [
+  "chardet>=5.0",
+]
+
+[project.optional-dependencies]
+pdf = [
+  "weasyprint>=62",
+]
+dev = [
+  "pytest>=8.0",
+  "build>=1.2",
+  "twine>=5.0",
+  "ruff>=0.11",
+]
+
+[project.urls]
+Homepage = "https://github.com/Jack736-ui/migman_log"
+Issues = "https://github.com/Jack736-ui/migman_log/issues"
+
+[project.scripts]
+nm-tool-forge = "loganalysis.cli:main"
+loganalysis = "loganalysis.cli:main"
+
+[tool.setuptools]
+package-dir = { "" = "src" }
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+addopts = "--basetemp=tests_tmp"
+
+[tool.ruff]
+line-length = 120
+target-version = "py310"
+
+[tool.ruff.lint]
+select = ["E", "F", "I", "B", "UP"]

nm_tool_forge-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

nm_tool_forge-0.1.0/src/loganalysis/__init__.py

@@ -0,0 +1,16 @@
+from __future__ import annotations
+
+from .analysis import analyze_file, run_analysis
+from .converters import convert_report_md_to_html_pdf
+from .normalization import normalize_message
+from .parsing import iter_logical_entries
+
+__all__ = [
+    "analyze_file",
+    "convert_report_md_to_html_pdf",
+    "iter_logical_entries",
+    "normalize_message",
+    "run_analysis",
+]
+
+__version__ = "0.1.0"

nm_tool_forge-0.1.0/src/loganalysis/__main__.py

@@ -0,0 +1,5 @@
+from __future__ import annotations
+
+from .cli import main
+
+raise SystemExit(main())

nm_tool_forge-0.1.0/src/loganalysis/analysis.py

@@ -0,0 +1,175 @@
+from __future__ import annotations
+
+from collections import Counter, defaultdict
+from pathlib import Path
+
+from .constants import DEFAULT_OUT_DIR, DEFAULT_TOP_EXAMPLES, EXIT_NO_LOG_FILES
+from .csv_export import write_csv
+from .encoding import count_physical_lines, detect_encoding
+from .filesystem import backup_file, ensure_dir
+from .models import AnalysisConfig, AnalysisRunResult, AnalysisSummary, FileAnalysis, MessageKey
+from .normalization import normalize_message
+from .parsing import iter_logical_entries, parse_entry
+from .report_markdown import build_markdown_report
+
+
+class NoLogFilesError(FileNotFoundError):
+    """Raised when no matching log files are found for an analysis run."""
+
+    exit_code = EXIT_NO_LOG_FILES
+
+
+def analyze_file(file_path: Path) -> FileAnalysis:
+    """Analyze one log file and aggregate raw and normalized message counts."""
+
+    raw_counts: Counter[MessageKey] = Counter()
+    norm_counts: Counter[MessageKey] = Counter()
+    norm_examples: dict[MessageKey, Counter[str]] = defaultdict(Counter)
+
+    unknown_lines = 0
+    total_entries = 0
+
+    encoding = detect_encoding(file_path)
+    total_lines = count_physical_lines(file_path, encoding=encoding)
+
+    for entry in iter_logical_entries(file_path, encoding=encoding):
+        total_entries += 1
+        parsed = parse_entry(entry)
+        if not parsed:
+            continue
+
+        severity = parsed.severity
+        message = parsed.message
+
+        if severity == "UNKNOWN":
+            unknown_lines += 1
+
+        raw_counts[(severity, message)] += 1
+
+        normalized_message = normalize_message(message)
+        norm_key = (severity, normalized_message)
+        norm_counts[norm_key] += 1
+        norm_examples[norm_key][message] += 1
+
+    return FileAnalysis(
+        file=file_path,
+        total_lines=total_lines,
+        total_entries=total_entries,
+        unknown_lines=unknown_lines,
+        raw_counts=raw_counts,
+        norm_counts=norm_counts,
+        norm_examples=dict(norm_examples),
+    )
+
+
+def sorted_rows(counter: Counter[MessageKey]) -> list[tuple[str, str, int]]:
+    """Return aggregated rows in a stable severity/count/message order."""
+
+    return [
+        (severity, message, count)
+        for (severity, message), count in sorted(
+            counter.items(),
+            key=lambda item: (item[0][0], -item[1], item[0][1]),
+        )
+    ]
+
+
+def find_log_files(logs_dir: Path) -> list[Path]:
+    """Return all supported log files from the configured logs directory."""
+
+    return sorted(logs_dir.glob("*.txt"))
+
+
+def build_default_config(
+    *,
+    logs_dir: Path,
+    out_dir: Path | None = None,
+    backup_dir: Path | None = None,
+    top_examples: int = DEFAULT_TOP_EXAMPLES,
+    convert: bool = False,
+) -> AnalysisConfig:
+    """Build an analysis configuration with default output locations."""
+
+    resolved_out_dir = out_dir or Path(DEFAULT_OUT_DIR)
+    resolved_backup_dir = backup_dir or (resolved_out_dir / "backup")
+    return AnalysisConfig(
+        logs_dir=logs_dir,
+        out_dir=resolved_out_dir,
+        backup_dir=resolved_backup_dir,
+        top_examples=top_examples,
+        convert=convert,
+    )
+
+
+def run_analysis(config: AnalysisConfig) -> AnalysisRunResult:
+    """Run the full analysis workflow and write CSV and report outputs."""
+
+    logs_dir = config.logs_dir
+    out_dir = config.out_dir
+    backup_dir = config.backup_dir or (out_dir / "backup")
+
+    ensure_dir(out_dir)
+    ensure_dir(backup_dir)
+
+    log_files = find_log_files(logs_dir)
+    if not log_files:
+        raise NoLogFilesError(f"No *.txt files found in: {logs_dir.resolve()}")
+
+    summary = AnalysisSummary(
+        analyses=[],
+        global_raw=Counter(),
+        global_norm=Counter(),
+        global_norm_examples={},
+    )
+
+    for log_file in log_files:
+        backup_path = backup_file(log_file, backup_dir)
+        analysis = analyze_file(log_file)
+        analysis.backup_path = backup_path
+        summary.analyses.append(analysis)
+
+        summary.global_raw.update(analysis.raw_counts)
+        summary.global_norm.update(analysis.norm_counts)
+        for key, counter in analysis.norm_examples.items():
+            summary.global_norm_examples.setdefault(key, Counter()).update(counter)
+
+        write_csv(
+            out_dir / f"{log_file.stem}.aggregated.csv",
+            sorted_rows(analysis.raw_counts),
+            headers=["SEVERITY", "MESSAGE", "COUNT"],
+        )
+        write_csv(
+            out_dir / f"{log_file.stem}.aggregated.normalized.csv",
+            sorted_rows(analysis.norm_counts),
+            headers=["SEVERITY", "MESSAGE_NORMALIZED", "COUNT"],
+        )
+
+    write_csv(
+        out_dir / "summary.all_files.csv",
+        sorted_rows(summary.global_raw),
+        headers=["SEVERITY", "MESSAGE", "COUNT"],
+    )
+    write_csv(
+        out_dir / "summary.all_files.normalized.csv",
+        sorted_rows(summary.global_norm),
+        headers=["SEVERITY", "MESSAGE_NORMALIZED", "COUNT"],
+    )
+
+    report_path = out_dir / "report.md"
+    report_path.write_text(build_markdown_report(summary, config), encoding="utf-8", newline="\n")
+
+    result = AnalysisRunResult(
+        out_dir=out_dir,
+        backup_dir=backup_dir,
+        report_path=report_path,
+        summary=summary,
+    )
+
+    if config.convert:
+        from .converters import convert_report_md_to_html_pdf
+
+        result.html_path = out_dir / "report.html"
+        result.pdf_path = out_dir / "report.pdf"
+        result.convert_status = convert_report_md_to_html_pdf(result.report_path, result.html_path, result.pdf_path)
+
+    return result