fwforge 0.1.0__tar.gz

fwforge-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 TallowX92
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

fwforge-0.1.0/PKG-INFO

@@ -0,0 +1,108 @@
+Metadata-Version: 2.4
+Name: fwforge
+Version: 0.1.0
+Summary: Fixed-width file parser for legacy logistics & supply chain data
+Author-email: TallowX92 <tallow072@gmail.com>
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/TallowX92/fwforge
+Project-URL: Repository, https://github.com/TallowX92/fwforge
+Project-URL: Issues, https://github.com/TallowX92/fwforge/issues
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: pyyaml>=6.0
+Dynamic: license-file
+
+# FixedWidth Forge
+
+[![PyPI version](https://img.shields.io/pypi/v/fwforge.svg)](https://pypi.org/project/fwforge/)
+[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/fwforge)](https://pypi.org/project/fwforge/)
+[![GitHub](https://img.shields.io/github/license/TallowX92/fwforge)](https://github.com/TallowX92/fwforge)
+
+**Parse legacy fixed-width files from logistics, warehouses, ERP systems, and mainframes — instantly.**
+
+Turn messy carrier reports, shipment manifests, and COBOL exports into clean CSV or JSON with a simple YAML schema.
+
+## Why Logistics & Supply Chain?
+Legacy formats like **fixed-width flat files**, **EDI**, and proprietary mainframes are the glue of the supply chain. `fwforge` provides the bridge to modern data pipelines without the enterprise bloat.
+
+## Features
+- **Schema-Driven**: Define column layouts in YAML. Supports `start+length` or `start+end` positions.
+- **Ultra-Fast**: Built on Python principles for rapid parsing of multi-gigabyte flat files.
+- **Batch Processing**: Process entire directories of manifest exports with one command.
+- **Inference Engine**: Use `--infer` to automatically generate a baseline schema from a sample data file.
+- **Clean Output**: Transform legacy data to clean CSV or JSON with built-in type casting.
+
+## Installation
+
+```bash
+pip install fwforge
+```
+
+Install from source (latest dev):
+
+```bash
+pip install git+https://github.com/TallowX92/fwforge.git
+```
+
+## Quick Start
+
+### 1. Infer a schema
+Start from scratch with a sample file:
+```bash
+fwforge --infer -i data.txt > my-layout.yaml
+```
+
+### 2. Parse data
+Convert legacy data to CSV (using the included sample):
+```bash
+fwforge -i data.txt -s layout.yaml -f csv -o output.csv
+cat output.csv
+```
+
+### 3. Batch process a folder
+```bash
+fwforge -i ./daily_manifests/ -s manifest.yaml -f json
+```
+
+## Example Schema (`layout.yaml`)
+```yaml
+name: "Freight-Manifest-v1"
+columns:
+  - name: "carrier_code"
+    start: 0
+    length: 5
+    trim: true
+    type: "string"
+  - name: "weight"
+    start: 20
+    length: 10
+    trim: true
+    type: "float"
+```
+
+## Development
+
+```bash
+# Clone and setup
+git clone https://github.com/TallowX92/fwforge.git
+cd fwforge
+uv sync
+uv run pytest -v
+
+# Run CLI
+uv run fwforge --help
+```
+
+## Roadmap
+
+- More robust type casting (dates, currency, custom)
+- Schema validation + strict mode
+- Better inference (header detection, multi-line records)
+- Performance / memory improvements for GB+ files
+- Standalone binary releases
+- Expanded output formats (parquet, etc.)
+
+## Changelog
+
+See [CHANGELOG.md](CHANGELOG.md) for release notes.

fwforge-0.1.0/README.md

@@ -0,0 +1,93 @@
+# FixedWidth Forge
+
+[![PyPI version](https://img.shields.io/pypi/v/fwforge.svg)](https://pypi.org/project/fwforge/)
+[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/fwforge)](https://pypi.org/project/fwforge/)
+[![GitHub](https://img.shields.io/github/license/TallowX92/fwforge)](https://github.com/TallowX92/fwforge)
+
+**Parse legacy fixed-width files from logistics, warehouses, ERP systems, and mainframes — instantly.**
+
+Turn messy carrier reports, shipment manifests, and COBOL exports into clean CSV or JSON with a simple YAML schema.
+
+## Why Logistics & Supply Chain?
+Legacy formats like **fixed-width flat files**, **EDI**, and proprietary mainframes are the glue of the supply chain. `fwforge` provides the bridge to modern data pipelines without the enterprise bloat.
+
+## Features
+- **Schema-Driven**: Define column layouts in YAML. Supports `start+length` or `start+end` positions.
+- **Ultra-Fast**: Built on Python principles for rapid parsing of multi-gigabyte flat files.
+- **Batch Processing**: Process entire directories of manifest exports with one command.
+- **Inference Engine**: Use `--infer` to automatically generate a baseline schema from a sample data file.
+- **Clean Output**: Transform legacy data to clean CSV or JSON with built-in type casting.
+
+## Installation
+
+```bash
+pip install fwforge
+```
+
+Install from source (latest dev):
+
+```bash
+pip install git+https://github.com/TallowX92/fwforge.git
+```
+
+## Quick Start
+
+### 1. Infer a schema
+Start from scratch with a sample file:
+```bash
+fwforge --infer -i data.txt > my-layout.yaml
+```
+
+### 2. Parse data
+Convert legacy data to CSV (using the included sample):
+```bash
+fwforge -i data.txt -s layout.yaml -f csv -o output.csv
+cat output.csv
+```
+
+### 3. Batch process a folder
+```bash
+fwforge -i ./daily_manifests/ -s manifest.yaml -f json
+```
+
+## Example Schema (`layout.yaml`)
+```yaml
+name: "Freight-Manifest-v1"
+columns:
+  - name: "carrier_code"
+    start: 0
+    length: 5
+    trim: true
+    type: "string"
+  - name: "weight"
+    start: 20
+    length: 10
+    trim: true
+    type: "float"
+```
+
+## Development
+
+```bash
+# Clone and setup
+git clone https://github.com/TallowX92/fwforge.git
+cd fwforge
+uv sync
+uv run pytest -v
+
+# Run CLI
+uv run fwforge --help
+```
+
+## Roadmap
+
+- More robust type casting (dates, currency, custom)
+- Schema validation + strict mode
+- Better inference (header detection, multi-line records)
+- Performance / memory improvements for GB+ files
+- Standalone binary releases
+- Expanded output formats (parquet, etc.)
+
+## Changelog
+
+See [CHANGELOG.md](CHANGELOG.md) for release notes.

fwforge-0.1.0/pyproject.toml

@@ -0,0 +1,34 @@
+[build-system]
+requires = ["setuptools>=61.0"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "fwforge"
+version = "0.1.0"
+description = "Fixed-width file parser for legacy logistics & supply chain data"
+readme = "README.md"
+license = "MIT"
+license-files = ["LICENSE"]
+requires-python = ">=3.9"
+authors = [
+    {name = "TallowX92", email = "tallow072@gmail.com"}
+]
+dependencies = [
+    "pyyaml>=6.0",
+]
+
+[project.urls]
+Homepage = "https://github.com/TallowX92/fwforge"
+Repository = "https://github.com/TallowX92/fwforge"
+Issues = "https://github.com/TallowX92/fwforge/issues"
+
+[project.scripts]
+fwforge = "fwforge.cli:main"
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[dependency-groups]
+dev = [
+    "pytest>=8.0",
+]

fwforge-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

fwforge-0.1.0/src/fwforge/__init__.py

@@ -0,0 +1 @@
+from .cli import main

fwforge-0.1.0/src/fwforge/__main__.py

@@ -0,0 +1,4 @@
+from .cli import main
+
+if __name__ == "__main__":
+    main()

fwforge-0.1.0/src/fwforge/cli.py

@@ -0,0 +1,103 @@
+import argparse
+import sys
+import yaml
+import os
+from .fw_parser import FixedWidthParser
+from .writer import write_csv, write_json
+
+def infer_layout(filepath):
+    try:
+        with open(filepath, 'r', encoding='utf-8', errors='replace') as f:
+            line = f.readline().rstrip('\n\r')
+        
+        columns = []
+        import re
+        for match in re.finditer(r'\S+', line):
+            columns.append({
+                "name": f"col_{match.start()}",
+                "start": match.start(),
+                "length": match.end() - match.start(),
+                "trim": True,
+                "type": "string"
+            })
+        return {"name": "InferredLayout", "columns": columns}
+    except FileNotFoundError:
+        print(f"Error: File '{filepath}' not found.", file=sys.stderr)
+        sys.exit(1)
+
+def process_file(input_path, output, layout, format_type):
+    try:
+        column_names = [col['name'] for col in layout['columns']]
+        file_parser = FixedWidthParser(layout)
+        
+        records = []
+        with open(input_path, 'r', encoding='utf-8', errors='replace') as f:
+            for line in f:
+                line = line.rstrip('\n\r')
+                if line.strip():
+                    records.append(file_parser.parse_line(line))
+        
+        if format_type == "csv":
+            write_csv(records, column_names, output)
+        else:
+            write_json(records, output)
+            
+    except Exception as e:
+        print(f"Error processing {input_path}: {e}", file=sys.stderr)
+    finally:
+        if output is not sys.stdout:
+            output.close()
+
+def main():
+    parser = argparse.ArgumentParser(description="FixedWidth Forge: Legacy data parser")
+    parser.add_argument("-i", "--input", required=True, help="Input file or directory")
+    parser.add_argument("-s", "--schema", help="YAML schema file")
+    parser.add_argument("-f", "--format", choices=["csv", "json"], default="csv")
+    parser.add_argument("-o", "--output", help="Output file or directory")
+    parser.add_argument("--infer", action="store_true", help="Infer schema from input file")
+    args = parser.parse_args()
+
+    if args.infer:
+        layout = infer_layout(args.input)
+        print(yaml.dump(layout))
+        return
+
+    if not args.schema:
+        parser.error("the following arguments are required: -s/--schema")
+
+    try:
+        with open(args.schema, 'r') as f:
+            layout = yaml.safe_load(f)
+    except FileNotFoundError:
+        print(f"Error: Schema file '{args.schema}' not found.", file=sys.stderr)
+        sys.exit(1)
+
+    if os.path.isdir(args.input):
+        output_dir = args.output or args.input
+        if not os.path.exists(output_dir):
+            os.makedirs(output_dir)
+        
+        for filename in os.listdir(args.input):
+            if filename.endswith((".txt", ".fwf")):
+                input_path = os.path.join(args.input, filename)
+                output_filename = os.path.splitext(filename)[0] + "." + args.format
+                output_path = os.path.join(output_dir, output_filename)
+                print(f"Processing {input_path} -> {output_path}")
+                try:
+                    with open(output_path, 'w', encoding='utf-8', newline='') as out_file:
+                        process_file(input_path, out_file, layout, args.format)
+                except Exception as e:
+                    print(f"Error writing to {output_path}: {e}", file=sys.stderr)
+    else:
+        try:
+            output = open(args.output, 'w', encoding='utf-8', newline='') if args.output else sys.stdout
+            process_file(args.input, output, layout, args.format)
+        except FileNotFoundError:
+             print(f"Error: Input file '{args.input}' not found.", file=sys.stderr)
+             sys.exit(1)
+        except Exception as e:
+            print(f"Error: {e}", file=sys.stderr)
+            sys.exit(1)
+
+if __name__ == "__main__":
+    main()

fwforge-0.1.0/src/fwforge/fw_parser.py

@@ -0,0 +1,47 @@
+from typing import Any, Dict, List
+
+class FixedWidthParser:
+    def __init__(self, layout: Dict):
+        self.layout = layout
+        self.columns = layout.get('columns', [])
+
+    def parse_line(self, line: str) -> Dict[str, Any]:
+        record = {}
+        for col in self.columns:
+            start = col.get('start', 0)
+            # Support both length and end
+            if 'length' in col:
+                end = start + col['length']
+            elif 'end' in col:
+                end = col['end']
+            else:
+                end = start # Default to zero-length if neither provided
+                
+            trim = col.get('trim', True)
+            
+            if end > len(line):
+                val = line[start:].strip() if trim else line[start:]
+            else:
+                val = line[start:end]
+                if trim:
+                    val = val.strip()
+            
+            # Basic type conversion
+            record[col['name']] = self._cast_value(val, col.get('type', 'string'))
+        
+        return record
+
+    def _cast_value(self, value: str, col_type: str) -> Any:
+        if not value:
+            return None if col_type in ('int', 'float') else ""
+        
+        try:
+            if col_type == 'int':
+                return int(value)
+            elif col_type == 'float':
+                return float(value)
+            elif col_type == 'string':
+                return value
+        except ValueError:
+            pass  # fallback to string
+        return value

fwforge-0.1.0/src/fwforge/writer.py

@@ -0,0 +1,11 @@
+import csv
+import json
+import sys
+
+def write_csv(records, columns, output=sys.stdout):
+    writer = csv.DictWriter(output, fieldnames=columns)
+    writer.writeheader()
+    writer.writerows(records)
+
+def write_json(records, output=sys.stdout):
+    json.dump(records, output, indent=2)

fwforge-0.1.0/src/fwforge.egg-info/PKG-INFO

@@ -0,0 +1,108 @@
+Metadata-Version: 2.4
+Name: fwforge
+Version: 0.1.0
+Summary: Fixed-width file parser for legacy logistics & supply chain data
+Author-email: TallowX92 <tallow072@gmail.com>
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/TallowX92/fwforge
+Project-URL: Repository, https://github.com/TallowX92/fwforge
+Project-URL: Issues, https://github.com/TallowX92/fwforge/issues
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: pyyaml>=6.0
+Dynamic: license-file
+
+# FixedWidth Forge
+
+[![PyPI version](https://img.shields.io/pypi/v/fwforge.svg)](https://pypi.org/project/fwforge/)
+[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/fwforge)](https://pypi.org/project/fwforge/)
+[![GitHub](https://img.shields.io/github/license/TallowX92/fwforge)](https://github.com/TallowX92/fwforge)
+
+**Parse legacy fixed-width files from logistics, warehouses, ERP systems, and mainframes — instantly.**
+
+Turn messy carrier reports, shipment manifests, and COBOL exports into clean CSV or JSON with a simple YAML schema.
+
+## Why Logistics & Supply Chain?
+Legacy formats like **fixed-width flat files**, **EDI**, and proprietary mainframes are the glue of the supply chain. `fwforge` provides the bridge to modern data pipelines without the enterprise bloat.
+
+## Features
+- **Schema-Driven**: Define column layouts in YAML. Supports `start+length` or `start+end` positions.
+- **Ultra-Fast**: Built on Python principles for rapid parsing of multi-gigabyte flat files.
+- **Batch Processing**: Process entire directories of manifest exports with one command.
+- **Inference Engine**: Use `--infer` to automatically generate a baseline schema from a sample data file.
+- **Clean Output**: Transform legacy data to clean CSV or JSON with built-in type casting.
+
+## Installation
+
+```bash
+pip install fwforge
+```
+
+Install from source (latest dev):
+
+```bash
+pip install git+https://github.com/TallowX92/fwforge.git
+```
+
+## Quick Start
+
+### 1. Infer a schema
+Start from scratch with a sample file:
+```bash
+fwforge --infer -i data.txt > my-layout.yaml
+```
+
+### 2. Parse data
+Convert legacy data to CSV (using the included sample):
+```bash
+fwforge -i data.txt -s layout.yaml -f csv -o output.csv
+cat output.csv
+```
+
+### 3. Batch process a folder
+```bash
+fwforge -i ./daily_manifests/ -s manifest.yaml -f json
+```
+
+## Example Schema (`layout.yaml`)
+```yaml
+name: "Freight-Manifest-v1"
+columns:
+  - name: "carrier_code"
+    start: 0
+    length: 5
+    trim: true
+    type: "string"
+  - name: "weight"
+    start: 20
+    length: 10
+    trim: true
+    type: "float"
+```
+
+## Development
+
+```bash
+# Clone and setup
+git clone https://github.com/TallowX92/fwforge.git
+cd fwforge
+uv sync
+uv run pytest -v
+
+# Run CLI
+uv run fwforge --help
+```
+
+## Roadmap
+
+- More robust type casting (dates, currency, custom)
+- Schema validation + strict mode
+- Better inference (header detection, multi-line records)
+- Performance / memory improvements for GB+ files
+- Standalone binary releases
+- Expanded output formats (parquet, etc.)
+
+## Changelog
+
+See [CHANGELOG.md](CHANGELOG.md) for release notes.

fwforge-0.1.0/src/fwforge.egg-info/SOURCES.txt

@@ -0,0 +1,15 @@
+LICENSE
+README.md
+pyproject.toml
+src/fwforge/__init__.py
+src/fwforge/__main__.py
+src/fwforge/cli.py
+src/fwforge/fw_parser.py
+src/fwforge/writer.py
+src/fwforge.egg-info/PKG-INFO
+src/fwforge.egg-info/SOURCES.txt
+src/fwforge.egg-info/dependency_links.txt
+src/fwforge.egg-info/entry_points.txt
+src/fwforge.egg-info/requires.txt
+src/fwforge.egg-info/top_level.txt
+tests/test_parser.py

fwforge-0.1.0/src/fwforge.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

fwforge-0.1.0/src/fwforge.egg-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+fwforge = fwforge.cli:main

fwforge-0.1.0/src/fwforge.egg-info/requires.txt

@@ -0,0 +1 @@
+pyyaml>=6.0

fwforge-0.1.0/src/fwforge.egg-info/top_level.txt

@@ -0,0 +1 @@
+fwforge

fwforge-0.1.0/tests/test_parser.py

@@ -0,0 +1,43 @@
+import pytest
+from fwforge.fw_parser import FixedWidthParser
+
+def test_basic_parsing():
+    layout = {
+        "columns": [
+            {"name": "a", "start": 0, "length": 3, "trim": True, "type": "string"},
+            {"name": "b", "start": 3, "length": 2, "trim": True, "type": "int"}
+        ]
+    }
+    parser = FixedWidthParser(layout)
+    line = "ABC10"
+    assert parser.parse_line(line) == {"a": "ABC", "b": 10}
+
+def test_start_end_parsing():
+    layout = {
+        "columns": [
+            {"name": "a", "start": 0, "end": 3, "trim": True, "type": "string"},
+        ]
+    }
+    parser = FixedWidthParser(layout)
+    line = "ABC10"
+    assert parser.parse_line(line) == {"a": "ABC"}
+
+def test_short_line_handling():
+    layout = {
+        "columns": [
+            {"name": "a", "start": 0, "length": 5, "trim": False, "type": "string"},
+        ]
+    }
+    parser = FixedWidthParser(layout)
+    line = "AB"
+    # Should handle end > len(line)
+    assert parser.parse_line(line) == {"a": "AB"}
+
+def test_float_casting():
+    layout = {
+        "columns": [
+            {"name": "weight", "start": 0, "length": 5, "trim": True, "type": "float"},
+        ]
+    }
+    parser = FixedWidthParser(layout)
+    assert parser.parse_line("10.5 ") == {"weight": 10.5}