cobol-intel 0.3.0__tar.gz

cobol_intel-0.3.0/.dockerignore

@@ -0,0 +1,13 @@
+.venv
+.git
+__pycache__
+*.pyc
+artifacts/
+tests/
+docs/
+.github/
+.mypy_cache/
+.ruff_cache/
+benchmark_*.json
+benchmark_*.md
+tests_runtime_*

cobol_intel-0.3.0/.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,30 @@
+---
+name: Bug Report
+about: Report a bug in cobol-intel
+title: "[Bug] "
+labels: bug
+---
+
+## Description
+
+A clear description of the bug.
+
+## Steps to Reproduce
+
+1. Run `cobol-intel analyze ...`
+2. ...
+
+## Expected Behavior
+
+What you expected to happen.
+
+## Actual Behavior
+
+What actually happened. Include error messages or output.
+
+## Environment
+
+- OS: [e.g. Windows 11, Ubuntu 24.04]
+- Python version: [e.g. 3.11.8]
+- cobol-intel version: [e.g. 0.1.0]
+- LLM backend (if relevant): [e.g. claude, openai, ollama]

cobol_intel-0.3.0/.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,22 @@
+---
+name: Feature Request
+about: Suggest a new feature for cobol-intel
+title: "[Feature] "
+labels: enhancement
+---
+
+## Problem
+
+What problem does this feature solve?
+
+## Proposed Solution
+
+How should this feature work?
+
+## Alternatives Considered
+
+Any alternative approaches you've thought about.
+
+## Additional Context
+
+Any other context, screenshots, or examples.

cobol_intel-0.3.0/.github/workflows/ci.yml

@@ -0,0 +1,29 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+    branches: [main]
+
+jobs:
+  test:
+    strategy:
+      matrix:
+        os: [ubuntu-latest, windows-latest]
+        python-version: ["3.11", "3.12"]
+    runs-on: ${{ matrix.os }}
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+      - name: Install dependencies
+        run: |
+          pip install -e ".[api,dev]"
+      - name: Lint
+        run: ruff check src/ tests/ tools/
+      - name: Check module boundaries
+        run: tach check
+      - name: Run tests with coverage
+        run: pytest --tb=short -q --cov=src/cobol_intel --cov-fail-under=85

cobol_intel-0.3.0/.github/workflows/release.yml

@@ -0,0 +1,47 @@
+name: Release to PyPI
+
+on:
+  push:
+    tags:
+      - "v*"
+
+permissions:
+  contents: read
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+
+      - name: Install build tools
+        run: pip install build
+
+      - name: Build package
+        run: python -m build
+
+      - name: Upload artifacts
+        uses: actions/upload-artifact@v4
+        with:
+          name: dist
+          path: dist/
+
+  publish:
+    needs: build
+    runs-on: ubuntu-latest
+    environment: pypi
+    permissions:
+      id-token: write
+    steps:
+      - name: Download artifacts
+        uses: actions/download-artifact@v4
+        with:
+          name: dist
+          path: dist/
+
+      - name: Publish to PyPI
+        uses: pypa/gh-action-pypi-publish@release/v1

cobol_intel-0.3.0/.gitignore

@@ -0,0 +1,41 @@
+# Python
+__pycache__/
+*.py[cod]
+*.pyo
+.venv/
+venv/
+dist/
+build/
+*.egg-info/
+.eggs/
+
+# Testing
+.pytest_cache/
+.pytest_tmp/
+pytest_tmp_workspace/
+pytest-cache-files-*/
+tests/.cache/
+.coverage
+htmlcov/
+tmp*/
+
+# Artifacts (generated output, tidak di-commit)
+artifacts/
+tests_runtime_artifacts/
+tests_runtime_artifacts_cli/
+.cobol_intel_cache/
+
+# Environment
+.env
+.env.*
+!.env.example
+
+# IDE
+.vscode/
+.idea/
+*.iml
+
+# OS
+.DS_Store
+Thumbs.db
+.claude

cobol_intel-0.3.0/CHANGELOG.md

@@ -0,0 +1,94 @@
+# Changelog
+
+All notable changes to cobol-intel will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/).
+
+## [Unreleased]
+
+### Added
+
+- Fine-tuning dataset builder (`tools/dataset_builder.py`) — generates
+  Alpaca/ShareGPT instruction-tuning pairs from the analysis pipeline
+- LoRA/PEFT fine-tuning script (`tools/finetune.py`) — CodeLlama-7B compatible,
+  QLoRA support, checkpoint resume, reproducible config saving
+- Local fine-tuned model backend (`llm/local_backend.py`) — loads PEFT or
+  standard HuggingFace models for fully offline inference
+- Prompt strategy comparison in benchmark (`tools/benchmark.py --compare`)
+- `py.typed` marker for PEP 561 typed package support
+
+### Fixed
+
+- pyproject.toml TOML ordering: `dependencies` was incorrectly nested under
+  `[project.urls]` instead of `[project]`, causing `uv build` to fail
+- Local backend defaults are now deterministic and install guidance points to
+  package extras for offline inference and training
+- Governance now treats the `local` backend as `local_only` for policy and
+  redaction decisions
+
+## [0.3.0] - 2026-04-01
+
+### Added
+
+- **Control flow graph (CFG) builder**: intra-program CFG with basic blocks, branch/perform/
+  fallthrough edges, and unsupported construct warnings (GO TO, ALTER)
+- **Field reference indexer**: per-statement read/write/condition/call_param classification
+  with aggregated field usage counts
+- **Data flow analyzer**: directed field-to-field flow graph covering MOVE, COMPUTE,
+  READ INTO, WRITE FROM, REWRITE FROM, and CALL USING — with Mermaid diagram output
+- **Dead code detector**: unreachable paragraph detection via BFS reachability, unused
+  data item scanning, and trivially dead branch detection (constant conditions)
+- Pipeline now writes `analysis/` artifacts (CFG, data flow, dead code, references)
+  for every parsed program
+- Doc generator includes data flow diagrams and dead code findings sections
+- `ArtifactIndex` in manifest now tracks `analysis` artifacts
+
+### Changed
+
+- Hardened API ergonomics with a module-level FastAPI app export, version response parity,
+  structured error payloads, and richer run summaries
+- Made explanation cache keys safer against stale outputs by including a context revision
+- Synced progress and fintech-readiness docs with the actual Phase 3 feature set
+- Version bumped to 0.3.0
+
+### Fixed
+
+- Fixed `make serve-api` to point at a real FastAPI app object
+- Made `make clean` portable by using Python stdlib instead of Unix-only shell commands
+
+## [0.2.0] - 2026-04-01
+
+### Added
+
+- Read-only REST API with versioned endpoints (`/api/v1/`)
+- Structured error codes (`ErrorCode` enum) for operational monitoring
+- Cross-platform CI pipeline (Linux + Windows, Python 3.11 + 3.12)
+- Benchmark suite for parse success rate, latency, and token savings
+- Per-program documentation generator (Markdown + HTML)
+- Self-contained HTML report with sidebar navigation, search, and Mermaid graphs
+- Change impact analyzer with call graph traversal and field reference scanning
+- Parallel LLM processing with bounded backend-specific concurrency
+- File-based explanation cache with composite invalidation keys
+- Docker image and docker-compose with optional Ollama sidecar
+- CLI commands: `impact`, `docs`
+- CLI flags: `--parallel`, `--max-workers`, `--cache/--no-cache`, `--format`
+- `Makefile` with common targets: lint, test, bench, build, serve-api
+- PyPI publish workflow on tag push
+- Output gallery and API guide documentation
+- `CHANGELOG.md`, `CONTRIBUTING.md`, and GitHub issue templates
+
+## [0.1.0] - 2026-03-31
+
+### Added
+
+- ANTLR4-based COBOL parser with fixed-format and free-format support
+- COPYBOOK resolver with circular dependency detection
+- Call graph builder and business rules extractor
+- Multi-backend LLM explanation engine (Claude, OpenAI, Ollama)
+- Context builder with smart chunking and token budget awareness
+- Governance layer: audit logging, sensitivity classification, prompt redaction
+- Strict policy enforcement and configurable model registry
+- Backend retry/timeout and token budget controls
+- CLI commands: `analyze`, `explain`, `graph`
+- Versioned JSON artifact contracts with Pydantic v2

cobol_intel-0.3.0/CONTRIBUTING.md

@@ -0,0 +1,76 @@
+# Contributing to cobol-intel
+
+## Development Setup
+
+```bash
+# Clone the repo
+git clone https://github.com/YOUR_USERNAME/llm-cobol-bussiness.git
+cd llm-cobol-bussiness
+
+# Create virtual environment
+python -m venv .venv
+source .venv/bin/activate  # Linux/Mac
+# .venv\Scripts\activate   # Windows
+
+# Install with dev + api dependencies
+pip install -e ".[api,dev]"
+```
+
+## Running Tests
+
+```bash
+# Run all tests
+pytest
+
+# Run with coverage
+pytest --cov=src/cobol_intel
+
+# Run specific test categories
+pytest tests/unit/
+pytest tests/contract/
+pytest tests/integration/
+pytest tests/evaluation/
+```
+
+## Linting
+
+```bash
+ruff check src/ tests/
+```
+
+## Module Boundaries
+
+This project enforces strict module dependency boundaries using [tach](https://docs.gauge.sh/):
+
+```bash
+tach check
+```
+
+The dependency graph:
+
+```text
+contracts (0 deps) <- core <- service <- cli
+parsers <- analysis <- service        <- api
+llm <- service
+outputs <- service
+```
+
+Key rules:
+- `contracts` and `core` must never import from `cli` or `api`
+- `analysis` and `parsers` must not depend on LLM backends
+- `cli` and `api` only call `service`, never access internals directly
+
+## Pull Request Process
+
+1. Fork the repo and create a feature branch
+2. Write tests for new functionality
+3. Ensure `pytest`, `ruff check`, and `tach check` all pass
+4. Keep commits focused and well-described
+5. Open a PR against `main` with a clear description
+
+## Code Style
+
+- Python 3.11+
+- Max line length: 100 characters
+- Use type annotations for function signatures
+- Follow existing patterns in the module you're modifying

cobol_intel-0.3.0/Dockerfile

@@ -0,0 +1,35 @@
+# --- Build stage ---
+FROM python:3.11-slim AS builder
+
+WORKDIR /build
+
+COPY pyproject.toml README.md ./
+COPY src/ src/
+COPY config/ config/
+
+RUN pip install --no-cache-dir --prefix=/install ".[api]"
+
+# --- Runtime stage ---
+FROM python:3.11-slim
+
+LABEL maintainer="WwzFwz" \
+      description="COBOL Intelligence Platform — static analysis + LLM for legacy COBOL"
+
+RUN groupadd --gid 1000 cobol && \
+    useradd --uid 1000 --gid cobol --create-home cobol
+
+COPY --from=builder /install /usr/local
+COPY config/ /app/config/
+
+WORKDIR /app
+RUN mkdir -p /app/artifacts && chown -R cobol:cobol /app
+
+USER cobol
+
+EXPOSE 8000
+
+HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
+    CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8000/api/v1/health')" || exit 1
+
+ENTRYPOINT ["cobol-intel"]
+CMD ["--help"]

cobol_intel-0.3.0/Makefile

@@ -0,0 +1,26 @@
+.PHONY: lint test test-cov bench build serve-api docker clean
+
+lint:
+	ruff check src/ tests/
+	tach check
+
+test:
+	pytest tests/ -x -q --tb=short
+
+test-cov:
+	pytest tests/ --cov=src/cobol_intel --cov-report=term-missing
+
+bench:
+	python tools/benchmark.py
+
+build:
+	python -m build
+
+serve-api:
+	python -m uvicorn cobol_intel.api.app:app --host 0.0.0.0 --port 8000 --reload
+
+docker:
+	docker build -t cobol-intel .
+
+clean:
+	python -c "from pathlib import Path; import shutil; [shutil.rmtree(p, ignore_errors=True) for p in [Path('dist'), Path('build'), Path('.cobol_intel_cache')]]; [shutil.rmtree(p, ignore_errors=True) for p in Path('.').rglob('__pycache__') if p.is_dir()]; [shutil.rmtree(p, ignore_errors=True) for p in Path('.').glob('*.egg-info') if p.is_dir()]"

cobol_intel-0.3.0/PKG-INFO

@@ -0,0 +1,259 @@
+Metadata-Version: 2.4
+Name: cobol-intel
+Version: 0.3.0
+Summary: Open-source platform for understanding, documenting, and analyzing legacy COBOL codebases using static analysis and LLM
+Project-URL: Homepage, https://github.com/WwzFwz/cobol-intel
+Project-URL: Documentation, https://github.com/WwzFwz/cobol-intel/tree/main/docs
+Project-URL: Repository, https://github.com/WwzFwz/cobol-intel
+Project-URL: Issues, https://github.com/WwzFwz/cobol-intel/issues
+Project-URL: Changelog, https://github.com/WwzFwz/cobol-intel/blob/main/CHANGELOG.md
+License: MIT
+Keywords: cobol,fintech,legacy,llm,modernization,static-analysis
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Financial and Insurance Industry
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Software Development :: Code Generators
+Classifier: Topic :: Software Development :: Documentation
+Classifier: Typing :: Typed
+Requires-Python: >=3.11
+Requires-Dist: anthropic>=0.30.0
+Requires-Dist: antlr4-python3-runtime>=4.13.0
+Requires-Dist: lark>=1.2.0
+Requires-Dist: networkx>=3.0
+Requires-Dist: ollama>=0.2.0
+Requires-Dist: openai>=1.0.0
+Requires-Dist: pydantic>=2.0.0
+Requires-Dist: rich>=13.0.0
+Requires-Dist: typer>=0.12.0
+Provides-Extra: api
+Requires-Dist: fastapi>=0.115.0; extra == 'api'
+Requires-Dist: uvicorn[standard]>=0.30.0; extra == 'api'
+Provides-Extra: dev
+Requires-Dist: httpx>=0.27.0; extra == 'dev'
+Requires-Dist: jsonschema>=4.0.0; extra == 'dev'
+Requires-Dist: pytest-cov>=5.0.0; extra == 'dev'
+Requires-Dist: pytest>=8.0.0; extra == 'dev'
+Requires-Dist: ruff>=0.4.0; extra == 'dev'
+Requires-Dist: tach>=0.9.0; extra == 'dev'
+Provides-Extra: local
+Requires-Dist: peft>=0.11.0; extra == 'local'
+Requires-Dist: torch>=2.2.0; extra == 'local'
+Requires-Dist: transformers>=4.40.0; extra == 'local'
+Provides-Extra: train
+Requires-Dist: accelerate>=0.30.0; extra == 'train'
+Requires-Dist: bitsandbytes>=0.43.0; (platform_system != 'Windows') and extra == 'train'
+Requires-Dist: datasets>=2.19.0; extra == 'train'
+Requires-Dist: peft>=0.11.0; extra == 'train'
+Requires-Dist: torch>=2.2.0; extra == 'train'
+Requires-Dist: transformers>=4.40.0; extra == 'train'
+Description-Content-Type: text/markdown
+
+# cobol-intel
+
+[![CI](https://github.com/WwzFwz/cobol-intel/actions/workflows/ci.yml/badge.svg)](https://github.com/WwzFwz/cobol-intel/actions/workflows/ci.yml)
+[![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-blue.svg)](https://www.python.org/downloads/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
+
+Open-source static analysis and LLM explanation platform for legacy COBOL
+codebases. Built for banking, fintech, and regulated modernization workflows.
+
+## Why This Exists
+
+Legacy COBOL systems fail the same way: key maintainers retire, documentation
+goes stale, impact analysis is manual, and regulators still need clear
+explanations. `cobol-intel` fixes that with a structured pipeline:
+
+```
+COBOL source → parser & AST → call graph & business rules → LLM explanation
+                                                           → impact analysis
+                                                           → documentation
+```
+
+The LLM consumes clean, traceable artifacts — not raw COBOL.
+
+## Quickstart
+
+```bash
+pip install cobol-intel
+
+# Optional extras
+pip install "cobol-intel[api]"    # REST API
+pip install "cobol-intel[local]"  # local HuggingFace inference
+pip install "cobol-intel[train]"  # fine-tuning scripts
+
+# Analyze a COBOL directory
+cobol-intel analyze samples/ --copybook-dir copybooks
+
+# Explain with an LLM backend
+cobol-intel explain samples/complex/payment.cbl --model claude --mode business
+
+# Generate documentation
+cobol-intel docs artifacts/samples/run_xxx --format html
+
+# Analyze change impact
+cobol-intel impact artifacts/samples/run_xxx --changed-program PAYMENT --changed-field WS-BALANCE
+```
+
+Output:
+
+```
+[cobol-intel] analyze: samples/
+Run ID: run_20260401_001
+Status: completed
+Artifacts: artifacts/samples/run_20260401_001
+```
+
+## Features
+
+### Static Analysis
+- ANTLR4-based parser (fixed + free format COBOL)
+- COPYBOOK resolution with circular dependency detection
+- Call graph builder and business rules extractor
+- Control flow graph (CFG) with branch, perform, and fallthrough edges
+- Field-level data flow analysis (MOVE, COMPUTE, READ INTO, WRITE FROM, CALL)
+- Dead code detection: unreachable paragraphs, unused data items, dead branches
+- Field reference indexer with read/write/condition classification
+- Data item hierarchy with PIC, COMP-3, REDEFINES, OCCURS, level-88
+
+### LLM Explanation
+- Multi-backend: Claude, OpenAI, Ollama
+- Three modes: `technical`, `business`, `audit`
+- Governance: audit logging, sensitivity classification, prompt redaction
+- Policy enforcement, token budgets, retry/timeout
+- Parallel processing with bounded concurrency
+- File-based cache with composite keys
+
+### Change Impact Analysis
+- "If I change field X, what breaks?"
+- BFS traversal on reverse call graph
+- Field reference scanning across ASTs and business rules
+- Configurable depth limit
+
+### Output & Documentation
+- Versioned JSON artifact contracts (Pydantic v2)
+- Markdown + HTML report generation
+- Self-contained HTML with sidebar nav, search, and Mermaid graphs
+- Structured error codes for operational monitoring
+
+### Fine-Tuning
+- Dataset builder: generates instruction-tuning pairs from pipeline output
+- LoRA/PEFT fine-tuning script for CodeLlama-7B or similar (QLoRA supported)
+- Local fine-tuned model backend for fully offline inference
+- Prompt comparison benchmark: raw source vs structured pipeline prompts
+
+### API & Distribution
+- Versioned REST API (`/api/v1/`) with OpenAPI docs and typed error responses
+- Docker image + docker-compose with optional Ollama sidecar
+- Cross-platform CI (Linux + Windows, Python 3.11 + 3.12)
+- PyPI-ready wheel with PEP 561 type stubs
+
+## CLI Commands
+
+| Command | Description |
+|---------|-------------|
+| `analyze` | Parse COBOL files, build AST, call graph, business rules |
+| `explain` | Run analysis + LLM explanation |
+| `graph` | Build dependency and call graph artifacts |
+| `impact` | Analyze change impact from a completed run |
+| `docs` | Generate documentation (Markdown or HTML) |
+
+Global:
+
+```bash
+cobol-intel --version           # Show version
+```
+
+Key flags:
+
+```bash
+--model claude|openai|ollama|local  # LLM backend
+--mode technical|business|audit # Explanation style
+--parallel                      # Enable parallel LLM processing
+--max-workers N                 # Override concurrency limit
+--cache / --no-cache            # Explanation cache toggle
+--strict-policy                 # Hard block policy violations
+--max-tokens-per-run N          # Token budget cap
+--format markdown|html          # Documentation format
+```
+
+## API Usage
+
+```bash
+pip install "cobol-intel[api]"
+cobol-intel-api  # starts on port 8000
+
+curl http://localhost:8000/api/v1/health
+curl http://localhost:8000/api/v1/runs?output_dir=artifacts
+curl http://localhost:8000/api/v1/version
+```
+
+See [docs/API_GUIDE.md](docs/API_GUIDE.md) for full endpoint reference.
+
+## Output Artifacts
+
+Each run produces a stable artifact tree:
+
+```
+artifacts/<project>/<run_id>/
+  manifest.json          # Run metadata, governance, errors
+  ast/                   # Per-program AST JSON
+  graphs/                # Call graph JSON + Mermaid
+  rules/                 # Business rules JSON + Markdown
+  analysis/              # CFG, data flow, dead code, references
+  docs/                  # Explanations, documentation
+  logs/                  # Audit event log
+```
+
+See [docs/OUTPUT_GALLERY.md](docs/OUTPUT_GALLERY.md) for sample artifacts.
+
+## COBOL Subset Coverage
+
+- Fixed-format and free-format COBOL
+- `COPY`, circular copy detection, `COPY ... REPLACING`
+- `WORKING-STORAGE`, `FILE`, `LINKAGE` sections
+- `PROCEDURE DIVISION USING`
+- `PIC`, `COMP-3`, `REDEFINES`, `OCCURS`, level-88 conditions
+- `IF`, `EVALUATE`, `PERFORM`, `CALL`, `STRING`, `UNSTRING`, `INSPECT`
+- File I/O: `OPEN`, `READ`, `WRITE`, `REWRITE`, `CLOSE`
+- `EXEC SQL` subset for static-analysis context
+
+## Development
+
+```bash
+git clone https://github.com/WwzFwz/cobol-intel.git
+cd cobol-intel
+pip install -e ".[dev]"
+
+make lint    # ruff + tach
+make test    # pytest
+make bench   # benchmark suite
+make build   # build wheel
+```
+
+Offline inference and training extras:
+
+```bash
+pip install -e ".[local]"  # local HuggingFace backend
+pip install -e ".[train]"  # dataset + fine-tuning tooling
+```
+
+See [CONTRIBUTING.md](CONTRIBUTING.md) for full dev setup and guidelines.
+
+## Documentation
+
+- [Architecture](docs/ARCHITECTURE.md)
+- [Architecture Decisions](docs/DECISIONS.md)
+- [API Guide](docs/API_GUIDE.md)
+- [Output Gallery](docs/OUTPUT_GALLERY.md)
+- [Fintech Readiness](docs/FINTECH_READINESS.md)
+- [Parser Evaluation](docs/PARSER_EVALUATION.md)
+- [Project Plan](docs/PLAN.md)
+- [Progress](docs/PROGRESS.md)
+- [Changelog](CHANGELOG.md)
+
+## License
+
+MIT