pra-effect 1.0.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Genesis410
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1,216 @@
1
+ Metadata-Version: 2.4
2
+ Name: pra-effect
3
+ Version: 1.0.0
4
+ Summary: Zero-dependency local CLI for parsing, deduplicating, and fingerprinting Python & Node.js error stack traces.
5
+ Author: Genesis410
6
+ License-Expression: MIT
7
+ Project-URL: Homepage, https://github.com/Genesis410/pra-effect
8
+ Project-URL: Repository, https://github.com/Genesis410/pra-effect.git
9
+ Project-URL: Issues, https://github.com/Genesis410/pra-effect/issues
10
+ Keywords: stacktrace,deduplication,cli,incident-management,logging,developer-tools
11
+ Classifier: Development Status :: 5 - Production/Stable
12
+ Classifier: Environment :: Console
13
+ Classifier: Intended Audience :: Developers
14
+ Classifier: Programming Language :: Python :: 3
15
+ Classifier: Programming Language :: Python :: 3.9
16
+ Classifier: Programming Language :: Python :: 3.10
17
+ Classifier: Programming Language :: Python :: 3.11
18
+ Classifier: Programming Language :: Python :: 3.12
19
+ Classifier: Programming Language :: Python :: 3.13
20
+ Classifier: Topic :: Software Development :: Debuggers
21
+ Classifier: Topic :: System :: Logging
22
+ Requires-Python: >=3.9
23
+ Description-Content-Type: text/markdown
24
+ License-File: LICENSE
25
+ Dynamic: license-file
26
+
27
+ # Pra-Effect
28
+
29
+ > Zero-dependency local CLI for parsing, deduplicating, and fingerprinting Python & Node.js error stack traces with low-memory streaming ingestion ($O(1)$ reader, $O(G)$ unique-group memory), ReDoS resilience, SQLite persistence, and standalone HTML reporting.
30
+
31
+ [![Python Version](https://img.shields.io/badge/python-3.9+-blue.svg)](https://www.python.org/)
32
+ [![License](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
33
+ [![Dependencies](https://img.shields.io/badge/dependencies-zero-brightgreen.svg)]()
34
+
35
+ ---
36
+
37
+ ## Highlights
38
+
39
+ - **Zero Third-Party Dependencies:** Pure Python standard library (`argparse`, `hashlib`, `html`, `re`, `sqlite3`, `pathlib`).
40
+ - **Resilient Fingerprinting (`fingerprint_v1`):** Normalizes dynamic memory addresses (`0x7f...`), variable line numbers, timestamps, UUIDs, and ephemeral ports so identical incidents aggregate into one canonical hash.
41
+ - **Dual Runtime Support:** Natively parses Python tracebacks (standard, chained exceptions, multiline messages) and Node.js call stacks (V8 engine).
42
+ - **Robust Input Handling:**
43
+ - Hard line truncation at 4,096 characters to prevent catastrophic regex backtracking (ReDoS).
44
+ - Interleaving tolerance up to 2 lines of interleaved non-trace log noise.
45
+ - Bounded message collection: log lines (`INFO ...`, timestamps) never leak into an error message.
46
+ - Chained Python exceptions collapse to a single root incident (causes kept as diagnostic context, never split the fingerprint).
47
+ - **Streaming Architecture & Performance:**
48
+ - Input parsed line-by-line through state machines — reader memory footprint is $O(1)$ (0.0047 MB tracemalloc for pure stream ingestion).
49
+ - Total memory scales with the number of *unique incident groups* $O(G)$, not total input byte size ($G$ distinct error patterns retained in memory).
50
+ - Benchmark (Python 3.14 on Windows, synthetic multi-group logs):
51
+ - *Typical log stream (28.2 MB, ~72k traces, ~6.8k unique groups):* ~6.1 MB Python heap, ~36.5 MB Peak RSS (GATE-MEM-01B PASS).
52
+ - *Adversarial high-cardinality stream (21.4 MB, ~54k traces, ~44.7k unique groups):* ~39.8 MB Python heap, ~106 MB Peak RSS (characterized in DEC-017).
53
+ - Optional `--max-input-mb` backstop rejects oversized raw log files (default 100 MB, 0 disables).
54
+ - **Local Persistence & Export:**
55
+ - WAL-mode SQLite storage for thread-safe concurrent writes and query performance.
56
+ - Self-contained, single-file HTML reports with zero external CDN assets.
57
+ - **Semantic Exit Codes:** POSIX standard compliance (`0` = success/analysis complete, `1` = fatal I/O, `2` = invalid CLI invocation).
58
+
59
+ ---
60
+
61
+ ## Installation
62
+
63
+ ### From Source (Editable / Tool Mode)
64
+ ```bash
65
+ git clone https://github.com/Genesis410/pra-effect.git
66
+ cd pra-effect
67
+ pip install -e .
68
+ ```
69
+
70
+ ### Standalone (Zero Install)
71
+ You can directly invoke the script without installation:
72
+ ```bash
73
+ python pra_effect.py --help
74
+ ```
75
+
76
+ ---
77
+
78
+ ## CLI Usage
79
+
80
+ Pra-Effect provides three primary commands: `ingest`, `scan`, and `history`.
81
+
82
+ ### 1. Ingest via STDIN (Pipe Mode)
83
+ Pipe stdout or stderr from test runners, logs, or services directly into Pra-Effect:
84
+
85
+ ```bash
86
+ # Direct pipe from a failing script
87
+ python app.py 2>&1 | pra-effect ingest
88
+
89
+ # Save run to SQLite and export an HTML dashboard
90
+ pytest 2>&1 | pra-effect ingest --save --html report.html
91
+
92
+ # Check installed version (must match pyproject + git tag)
93
+ pra-effect --version
94
+ ```
95
+
96
+ ### 2. Scan Existing Log Files
97
+ Analyze log dumps or application error logs on disk — a single file or a whole directory:
98
+
99
+ ```bash
100
+ # Explicit file (must be readable, otherwise exit 1)
101
+ pra-effect scan /var/log/application.log --save --html report.html
102
+
103
+ # Directory scan: default pattern *.log,*.txt, comma-separated globs supported
104
+ pra-effect scan /var/log --pattern "*.log,*.txt,*.out"
105
+
106
+ # Files that cannot be read (binary/unreadable) are skipped with a warning on stderr.
107
+ # Exit 1 if zero readable files matched — never a silent "0 errors found".
108
+
109
+ # Input-size backstop (default 100 MB, streaming already bounds the pipeline; 0 disables)
110
+ pra-effect scan /var/log/huge --max-input-mb 250
111
+ ```
112
+
113
+ ### 3. Historical Incident Aggregation
114
+ Query grouped incident metrics stored in your local SQLite database:
115
+
116
+ ```bash
117
+ pra-effect history
118
+ ```
119
+
120
+ Example Output:
121
+ ```text
122
+ === Pra-Effect Historical Incident Groups ===
123
+ Fingerprint: f9e160e1d88bbca1 (v1)
124
+ Runtime: python
125
+ Error Type: ValueError
126
+ Message: bad <script>alert(1)</script>
127
+ Occurrences: 3 total across 2 runs
128
+ First Seen: 2026-10-24 16:30:00 UTC
129
+ Last Seen: 2026-10-24 16:45:12 UTC
130
+ ```
131
+
132
+ ---
133
+
134
+ ## Architecture & Data Flow
135
+
136
+ ```text
137
+ Log Stream (STDIN / File)
138
+ │
139
+ ▼
140
+ Pre-check Sanitizer (4,096 char cutoff, ReDoS guard)
141
+ │
142
+ ▼
143
+ Grammar Parsers (Python Tracebacks / Node.js V8 Stacks)
144
+ │
145
+ ▼
146
+ Message Normalizer (UUID / hex / ID collapse — not PII redaction)
147
+ │
148
+ ▼
149
+ Fingerprint Engine (SHA-256 top-frame + message -> 16-hex ID)
150
+ │
151
+ ├─────────────────────────┬─────────────────────────┐
152
+ ▼ ▼ ▼
153
+ Terminal ANSI Output SQLite Storage (WAL) HTML Dashboard
154
+ ```
155
+
156
+ ---
157
+
158
+ ## Database & Schema
159
+
160
+ When `--save` is passed, runs are stored in `./.pra-effect/traces.db` (override via `--db <path>`):
161
+
162
+ - `runs`: Metadata per execution (duration, line counts, unique group count).
163
+ - `incidents`: Normalized error incidents linked by foreign keys, tracking `is_truncated`, occurrence counts, and project frames.
164
+ - `v_historical_incidents`: High-performance view aggregating cross-run recurrence.
165
+
166
+ ---
167
+
168
+ ## Testing
169
+
170
+ Run the included verification suite (standard `unittest` discovery, works from any directory):
171
+
172
+ ```bash
173
+ python -m unittest discover -s . -p "test_*.py"
174
+ # or
175
+ python test_pra_effect.py
176
+ ```
177
+
178
+ Memory benchmarks for release gates (`GATE-MEM-01A` streaming pipeline & `GATE-MEM-01B` realistic workload):
179
+
180
+ ```bash
181
+ python bench_memory.py # exit 0 = PASS (<= 50 MB Peak RSS), 1 = gate exceeded, 2 = RSS NOT_RUN
182
+ ```
183
+
184
+ Benchmark & Performance Gates:
185
+ - **GATE-MEM-01A (Pipeline Ingestion Stream)**: Peak RSS $\le$ 35 MB, tracemalloc $<$ 1 MB. Empirically measured **24.36 MB Peak RSS** and **0.0047 MB tracemalloc** across 54,000+ tracebacks ($O(1)$ streaming ingestion).
186
+ - **GATE-MEM-01B (Realistic Workload End-to-End)**: Peak RSS $\le$ 50 MB on files $\ge$ 25 MB with realistic deduplication ($\le$ 10,000 unique groups). Empirically measured **36.50 MB Peak RSS** and **6.12 MB tracemalloc** on 28.2 MB / 72,316 tracebacks.
187
+ - **High-Cardinality Boundary**: In-memory grouping memory scales linearly $O(G)$ with unique error groups (~963 bytes per group with complete `representative_trace`). Peak RSS 50 MB is not guaranteed for adversarial logs where every single error is unique. `--max-input-mb` (default 100 MB) provides raw input byte capping.
188
+
189
+ Tests cover:
190
+ - Python & Node.js grouping and line drift resilience.
191
+ - Message normalization and hex address masking.
192
+ - ReDoS guard and bounded multiline collection.
193
+ - Interleaved non-trace log line tolerance.
194
+ - SQLite WAL mode and concurrent transaction safety.
195
+ - Single-file HTML generation with full XSS escaping on every rendered field, plus a PII notice (normalization is not redaction).
196
+ - Chained exception collapse, bounded message collector, directory scan `--pattern` with skip/exit semantics.
197
+ - Fingerprint version isolation in history queries (`FINGERPRINT_SPEC.md`).
198
+ - CLI exit codes (0, 1, 2).
199
+ - Streaming pipeline memory regression ($O(1)$ constant memory ingestion).
200
+ - Raw input file cap backstop (`--max-input-mb`).
201
+ - Golden CLI output parity (byte-identical across refactors).
202
+
203
+ ---
204
+
205
+ ## License
206
+
207
+ MIT License. See [LICENSE](LICENSE) for details.
208
+
209
+ ---
210
+
211
+ ## Release & Changelog
212
+
213
+ See [CHANGELOG.md](CHANGELOG.md) for the version history. Current release: `1.0.0`.
214
+
215
+ Version consistency across the package (`pyproject.toml`), the CLI (`--version`), and
216
+ the git tag is enforced by `test_release_version_consistency`.
@@ -0,0 +1,190 @@
1
+ # Pra-Effect
2
+
3
+ > Zero-dependency local CLI for parsing, deduplicating, and fingerprinting Python & Node.js error stack traces with low-memory streaming ingestion ($O(1)$ reader, $O(G)$ unique-group memory), ReDoS resilience, SQLite persistence, and standalone HTML reporting.
4
+
5
+ [![Python Version](https://img.shields.io/badge/python-3.9+-blue.svg)](https://www.python.org/)
6
+ [![License](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
7
+ [![Dependencies](https://img.shields.io/badge/dependencies-zero-brightgreen.svg)]()
8
+
9
+ ---
10
+
11
+ ## Highlights
12
+
13
+ - **Zero Third-Party Dependencies:** Pure Python standard library (`argparse`, `hashlib`, `html`, `re`, `sqlite3`, `pathlib`).
14
+ - **Resilient Fingerprinting (`fingerprint_v1`):** Normalizes dynamic memory addresses (`0x7f...`), variable line numbers, timestamps, UUIDs, and ephemeral ports so identical incidents aggregate into one canonical hash.
15
+ - **Dual Runtime Support:** Natively parses Python tracebacks (standard, chained exceptions, multiline messages) and Node.js call stacks (V8 engine).
16
+ - **Robust Input Handling:**
17
+ - Hard line truncation at 4,096 characters to prevent catastrophic regex backtracking (ReDoS).
18
+ - Interleaving tolerance up to 2 lines of interleaved non-trace log noise.
19
+ - Bounded message collection: log lines (`INFO ...`, timestamps) never leak into an error message.
20
+ - Chained Python exceptions collapse to a single root incident (causes kept as diagnostic context, never split the fingerprint).
21
+ - **Streaming Architecture & Performance:**
22
+ - Input parsed line-by-line through state machines — reader memory footprint is $O(1)$ (0.0047 MB tracemalloc for pure stream ingestion).
23
+ - Total memory scales with the number of *unique incident groups* $O(G)$, not total input byte size ($G$ distinct error patterns retained in memory).
24
+ - Benchmark (Python 3.14 on Windows, synthetic multi-group logs):
25
+ - *Typical log stream (28.2 MB, ~72k traces, ~6.8k unique groups):* ~6.1 MB Python heap, ~36.5 MB Peak RSS (GATE-MEM-01B PASS).
26
+ - *Adversarial high-cardinality stream (21.4 MB, ~54k traces, ~44.7k unique groups):* ~39.8 MB Python heap, ~106 MB Peak RSS (characterized in DEC-017).
27
+ - Optional `--max-input-mb` backstop rejects oversized raw log files (default 100 MB, 0 disables).
28
+ - **Local Persistence & Export:**
29
+ - WAL-mode SQLite storage for thread-safe concurrent writes and query performance.
30
+ - Self-contained, single-file HTML reports with zero external CDN assets.
31
+ - **Semantic Exit Codes:** POSIX standard compliance (`0` = success/analysis complete, `1` = fatal I/O, `2` = invalid CLI invocation).
32
+
33
+ ---
34
+
35
+ ## Installation
36
+
37
+ ### From Source (Editable / Tool Mode)
38
+ ```bash
39
+ git clone https://github.com/Genesis410/pra-effect.git
40
+ cd pra-effect
41
+ pip install -e .
42
+ ```
43
+
44
+ ### Standalone (Zero Install)
45
+ You can directly invoke the script without installation:
46
+ ```bash
47
+ python pra_effect.py --help
48
+ ```
49
+
50
+ ---
51
+
52
+ ## CLI Usage
53
+
54
+ Pra-Effect provides three primary commands: `ingest`, `scan`, and `history`.
55
+
56
+ ### 1. Ingest via STDIN (Pipe Mode)
57
+ Pipe stdout or stderr from test runners, logs, or services directly into Pra-Effect:
58
+
59
+ ```bash
60
+ # Direct pipe from a failing script
61
+ python app.py 2>&1 | pra-effect ingest
62
+
63
+ # Save run to SQLite and export an HTML dashboard
64
+ pytest 2>&1 | pra-effect ingest --save --html report.html
65
+
66
+ # Check installed version (must match pyproject + git tag)
67
+ pra-effect --version
68
+ ```
69
+
70
+ ### 2. Scan Existing Log Files
71
+ Analyze log dumps or application error logs on disk — a single file or a whole directory:
72
+
73
+ ```bash
74
+ # Explicit file (must be readable, otherwise exit 1)
75
+ pra-effect scan /var/log/application.log --save --html report.html
76
+
77
+ # Directory scan: default pattern *.log,*.txt, comma-separated globs supported
78
+ pra-effect scan /var/log --pattern "*.log,*.txt,*.out"
79
+
80
+ # Files that cannot be read (binary/unreadable) are skipped with a warning on stderr.
81
+ # Exit 1 if zero readable files matched — never a silent "0 errors found".
82
+
83
+ # Input-size backstop (default 100 MB, streaming already bounds the pipeline; 0 disables)
84
+ pra-effect scan /var/log/huge --max-input-mb 250
85
+ ```
86
+
87
+ ### 3. Historical Incident Aggregation
88
+ Query grouped incident metrics stored in your local SQLite database:
89
+
90
+ ```bash
91
+ pra-effect history
92
+ ```
93
+
94
+ Example Output:
95
+ ```text
96
+ === Pra-Effect Historical Incident Groups ===
97
+ Fingerprint: f9e160e1d88bbca1 (v1)
98
+ Runtime: python
99
+ Error Type: ValueError
100
+ Message: bad <script>alert(1)</script>
101
+ Occurrences: 3 total across 2 runs
102
+ First Seen: 2026-10-24 16:30:00 UTC
103
+ Last Seen: 2026-10-24 16:45:12 UTC
104
+ ```
105
+
106
+ ---
107
+
108
+ ## Architecture & Data Flow
109
+
110
+ ```text
111
+ Log Stream (STDIN / File)
112
+ │
113
+ ▼
114
+ Pre-check Sanitizer (4,096 char cutoff, ReDoS guard)
115
+ │
116
+ ▼
117
+ Grammar Parsers (Python Tracebacks / Node.js V8 Stacks)
118
+ │
119
+ ▼
120
+ Message Normalizer (UUID / hex / ID collapse — not PII redaction)
121
+ │
122
+ ▼
123
+ Fingerprint Engine (SHA-256 top-frame + message -> 16-hex ID)
124
+ │
125
+ ├─────────────────────────┬─────────────────────────┐
126
+ ▼ ▼ ▼
127
+ Terminal ANSI Output SQLite Storage (WAL) HTML Dashboard
128
+ ```
129
+
130
+ ---
131
+
132
+ ## Database & Schema
133
+
134
+ When `--save` is passed, runs are stored in `./.pra-effect/traces.db` (override via `--db <path>`):
135
+
136
+ - `runs`: Metadata per execution (duration, line counts, unique group count).
137
+ - `incidents`: Normalized error incidents linked by foreign keys, tracking `is_truncated`, occurrence counts, and project frames.
138
+ - `v_historical_incidents`: High-performance view aggregating cross-run recurrence.
139
+
140
+ ---
141
+
142
+ ## Testing
143
+
144
+ Run the included verification suite (standard `unittest` discovery, works from any directory):
145
+
146
+ ```bash
147
+ python -m unittest discover -s . -p "test_*.py"
148
+ # or
149
+ python test_pra_effect.py
150
+ ```
151
+
152
+ Memory benchmarks for release gates (`GATE-MEM-01A` streaming pipeline & `GATE-MEM-01B` realistic workload):
153
+
154
+ ```bash
155
+ python bench_memory.py # exit 0 = PASS (<= 50 MB Peak RSS), 1 = gate exceeded, 2 = RSS NOT_RUN
156
+ ```
157
+
158
+ Benchmark & Performance Gates:
159
+ - **GATE-MEM-01A (Pipeline Ingestion Stream)**: Peak RSS $\le$ 35 MB, tracemalloc $<$ 1 MB. Empirically measured **24.36 MB Peak RSS** and **0.0047 MB tracemalloc** across 54,000+ tracebacks ($O(1)$ streaming ingestion).
160
+ - **GATE-MEM-01B (Realistic Workload End-to-End)**: Peak RSS $\le$ 50 MB on files $\ge$ 25 MB with realistic deduplication ($\le$ 10,000 unique groups). Empirically measured **36.50 MB Peak RSS** and **6.12 MB tracemalloc** on 28.2 MB / 72,316 tracebacks.
161
+ - **High-Cardinality Boundary**: In-memory grouping memory scales linearly $O(G)$ with unique error groups (~963 bytes per group with complete `representative_trace`). Peak RSS 50 MB is not guaranteed for adversarial logs where every single error is unique. `--max-input-mb` (default 100 MB) provides raw input byte capping.
162
+
163
+ Tests cover:
164
+ - Python & Node.js grouping and line drift resilience.
165
+ - Message normalization and hex address masking.
166
+ - ReDoS guard and bounded multiline collection.
167
+ - Interleaved non-trace log line tolerance.
168
+ - SQLite WAL mode and concurrent transaction safety.
169
+ - Single-file HTML generation with full XSS escaping on every rendered field, plus a PII notice (normalization is not redaction).
170
+ - Chained exception collapse, bounded message collector, directory scan `--pattern` with skip/exit semantics.
171
+ - Fingerprint version isolation in history queries (`FINGERPRINT_SPEC.md`).
172
+ - CLI exit codes (0, 1, 2).
173
+ - Streaming pipeline memory regression ($O(1)$ constant memory ingestion).
174
+ - Raw input file cap backstop (`--max-input-mb`).
175
+ - Golden CLI output parity (byte-identical across refactors).
176
+
177
+ ---
178
+
179
+ ## License
180
+
181
+ MIT License. See [LICENSE](LICENSE) for details.
182
+
183
+ ---
184
+
185
+ ## Release & Changelog
186
+
187
+ See [CHANGELOG.md](CHANGELOG.md) for the version history. Current release: `1.0.0`.
188
+
189
+ Version consistency across the package (`pyproject.toml`), the CLI (`--version`), and
190
+ the git tag is enforced by `test_release_version_consistency`.
@@ -0,0 +1,216 @@
1
+ Metadata-Version: 2.4
2
+ Name: pra-effect
3
+ Version: 1.0.0
4
+ Summary: Zero-dependency local CLI for parsing, deduplicating, and fingerprinting Python & Node.js error stack traces.
5
+ Author: Genesis410
6
+ License-Expression: MIT
7
+ Project-URL: Homepage, https://github.com/Genesis410/pra-effect
8
+ Project-URL: Repository, https://github.com/Genesis410/pra-effect.git
9
+ Project-URL: Issues, https://github.com/Genesis410/pra-effect/issues
10
+ Keywords: stacktrace,deduplication,cli,incident-management,logging,developer-tools
11
+ Classifier: Development Status :: 5 - Production/Stable
12
+ Classifier: Environment :: Console
13
+ Classifier: Intended Audience :: Developers
14
+ Classifier: Programming Language :: Python :: 3
15
+ Classifier: Programming Language :: Python :: 3.9
16
+ Classifier: Programming Language :: Python :: 3.10
17
+ Classifier: Programming Language :: Python :: 3.11
18
+ Classifier: Programming Language :: Python :: 3.12
19
+ Classifier: Programming Language :: Python :: 3.13
20
+ Classifier: Topic :: Software Development :: Debuggers
21
+ Classifier: Topic :: System :: Logging
22
+ Requires-Python: >=3.9
23
+ Description-Content-Type: text/markdown
24
+ License-File: LICENSE
25
+ Dynamic: license-file
26
+
27
+ # Pra-Effect
28
+
29
+ > Zero-dependency local CLI for parsing, deduplicating, and fingerprinting Python & Node.js error stack traces with low-memory streaming ingestion ($O(1)$ reader, $O(G)$ unique-group memory), ReDoS resilience, SQLite persistence, and standalone HTML reporting.
30
+
31
+ [![Python Version](https://img.shields.io/badge/python-3.9+-blue.svg)](https://www.python.org/)
32
+ [![License](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
33
+ [![Dependencies](https://img.shields.io/badge/dependencies-zero-brightgreen.svg)]()
34
+
35
+ ---
36
+
37
+ ## Highlights
38
+
39
+ - **Zero Third-Party Dependencies:** Pure Python standard library (`argparse`, `hashlib`, `html`, `re`, `sqlite3`, `pathlib`).
40
+ - **Resilient Fingerprinting (`fingerprint_v1`):** Normalizes dynamic memory addresses (`0x7f...`), variable line numbers, timestamps, UUIDs, and ephemeral ports so identical incidents aggregate into one canonical hash.
41
+ - **Dual Runtime Support:** Natively parses Python tracebacks (standard, chained exceptions, multiline messages) and Node.js call stacks (V8 engine).
42
+ - **Robust Input Handling:**
43
+ - Hard line truncation at 4,096 characters to prevent catastrophic regex backtracking (ReDoS).
44
+ - Interleaving tolerance up to 2 lines of interleaved non-trace log noise.
45
+ - Bounded message collection: log lines (`INFO ...`, timestamps) never leak into an error message.
46
+ - Chained Python exceptions collapse to a single root incident (causes kept as diagnostic context, never split the fingerprint).
47
+ - **Streaming Architecture & Performance:**
48
+ - Input parsed line-by-line through state machines — reader memory footprint is $O(1)$ (0.0047 MB tracemalloc for pure stream ingestion).
49
+ - Total memory scales with the number of *unique incident groups* $O(G)$, not total input byte size ($G$ distinct error patterns retained in memory).
50
+ - Benchmark (Python 3.14 on Windows, synthetic multi-group logs):
51
+ - *Typical log stream (28.2 MB, ~72k traces, ~6.8k unique groups):* ~6.1 MB Python heap, ~36.5 MB Peak RSS (GATE-MEM-01B PASS).
52
+ - *Adversarial high-cardinality stream (21.4 MB, ~54k traces, ~44.7k unique groups):* ~39.8 MB Python heap, ~106 MB Peak RSS (characterized in DEC-017).
53
+ - Optional `--max-input-mb` backstop rejects oversized raw log files (default 100 MB, 0 disables).
54
+ - **Local Persistence & Export:**
55
+ - WAL-mode SQLite storage for thread-safe concurrent writes and query performance.
56
+ - Self-contained, single-file HTML reports with zero external CDN assets.
57
+ - **Semantic Exit Codes:** POSIX standard compliance (`0` = success/analysis complete, `1` = fatal I/O, `2` = invalid CLI invocation).
58
+
59
+ ---
60
+
61
+ ## Installation
62
+
63
+ ### From Source (Editable / Tool Mode)
64
+ ```bash
65
+ git clone https://github.com/Genesis410/pra-effect.git
66
+ cd pra-effect
67
+ pip install -e .
68
+ ```
69
+
70
+ ### Standalone (Zero Install)
71
+ You can directly invoke the script without installation:
72
+ ```bash
73
+ python pra_effect.py --help
74
+ ```
75
+
76
+ ---
77
+
78
+ ## CLI Usage
79
+
80
+ Pra-Effect provides three primary commands: `ingest`, `scan`, and `history`.
81
+
82
+ ### 1. Ingest via STDIN (Pipe Mode)
83
+ Pipe stdout or stderr from test runners, logs, or services directly into Pra-Effect:
84
+
85
+ ```bash
86
+ # Direct pipe from a failing script
87
+ python app.py 2>&1 | pra-effect ingest
88
+
89
+ # Save run to SQLite and export an HTML dashboard
90
+ pytest 2>&1 | pra-effect ingest --save --html report.html
91
+
92
+ # Check installed version (must match pyproject + git tag)
93
+ pra-effect --version
94
+ ```
95
+
96
+ ### 2. Scan Existing Log Files
97
+ Analyze log dumps or application error logs on disk — a single file or a whole directory:
98
+
99
+ ```bash
100
+ # Explicit file (must be readable, otherwise exit 1)
101
+ pra-effect scan /var/log/application.log --save --html report.html
102
+
103
+ # Directory scan: default pattern *.log,*.txt, comma-separated globs supported
104
+ pra-effect scan /var/log --pattern "*.log,*.txt,*.out"
105
+
106
+ # Files that cannot be read (binary/unreadable) are skipped with a warning on stderr.
107
+ # Exit 1 if zero readable files matched — never a silent "0 errors found".
108
+
109
+ # Input-size backstop (default 100 MB, streaming already bounds the pipeline; 0 disables)
110
+ pra-effect scan /var/log/huge --max-input-mb 250
111
+ ```
112
+
113
+ ### 3. Historical Incident Aggregation
114
+ Query grouped incident metrics stored in your local SQLite database:
115
+
116
+ ```bash
117
+ pra-effect history
118
+ ```
119
+
120
+ Example Output:
121
+ ```text
122
+ === Pra-Effect Historical Incident Groups ===
123
+ Fingerprint: f9e160e1d88bbca1 (v1)
124
+ Runtime: python
125
+ Error Type: ValueError
126
+ Message: bad <script>alert(1)</script>
127
+ Occurrences: 3 total across 2 runs
128
+ First Seen: 2026-10-24 16:30:00 UTC
129
+ Last Seen: 2026-10-24 16:45:12 UTC
130
+ ```
131
+
132
+ ---
133
+
134
+ ## Architecture & Data Flow
135
+
136
+ ```text
137
+ Log Stream (STDIN / File)
138
+ │
139
+ ▼
140
+ Pre-check Sanitizer (4,096 char cutoff, ReDoS guard)
141
+ │
142
+ ▼
143
+ Grammar Parsers (Python Tracebacks / Node.js V8 Stacks)
144
+ │
145
+ ▼
146
+ Message Normalizer (UUID / hex / ID collapse — not PII redaction)
147
+ │
148
+ ▼
149
+ Fingerprint Engine (SHA-256 top-frame + message -> 16-hex ID)
150
+ │
151
+ ├─────────────────────────┬─────────────────────────┐
152
+ ▼ ▼ ▼
153
+ Terminal ANSI Output SQLite Storage (WAL) HTML Dashboard
154
+ ```
155
+
156
+ ---
157
+
158
+ ## Database & Schema
159
+
160
+ When `--save` is passed, runs are stored in `./.pra-effect/traces.db` (override via `--db <path>`):
161
+
162
+ - `runs`: Metadata per execution (duration, line counts, unique group count).
163
+ - `incidents`: Normalized error incidents linked by foreign keys, tracking `is_truncated`, occurrence counts, and project frames.
164
+ - `v_historical_incidents`: High-performance view aggregating cross-run recurrence.
165
+
166
+ ---
167
+
168
+ ## Testing
169
+
170
+ Run the included verification suite (standard `unittest` discovery, works from any directory):
171
+
172
+ ```bash
173
+ python -m unittest discover -s . -p "test_*.py"
174
+ # or
175
+ python test_pra_effect.py
176
+ ```
177
+
178
+ Memory benchmarks for release gates (`GATE-MEM-01A` streaming pipeline & `GATE-MEM-01B` realistic workload):
179
+
180
+ ```bash
181
+ python bench_memory.py # exit 0 = PASS (<= 50 MB Peak RSS), 1 = gate exceeded, 2 = RSS NOT_RUN
182
+ ```
183
+
184
+ Benchmark & Performance Gates:
185
+ - **GATE-MEM-01A (Pipeline Ingestion Stream)**: Peak RSS $\le$ 35 MB, tracemalloc $<$ 1 MB. Empirically measured **24.36 MB Peak RSS** and **0.0047 MB tracemalloc** across 54,000+ tracebacks ($O(1)$ streaming ingestion).
186
+ - **GATE-MEM-01B (Realistic Workload End-to-End)**: Peak RSS $\le$ 50 MB on files $\ge$ 25 MB with realistic deduplication ($\le$ 10,000 unique groups). Empirically measured **36.50 MB Peak RSS** and **6.12 MB tracemalloc** on 28.2 MB / 72,316 tracebacks.
187
+ - **High-Cardinality Boundary**: In-memory grouping memory scales linearly $O(G)$ with unique error groups (~963 bytes per group with complete `representative_trace`). Peak RSS 50 MB is not guaranteed for adversarial logs where every single error is unique. `--max-input-mb` (default 100 MB) provides raw input byte capping.
188
+
189
+ Tests cover:
190
+ - Python & Node.js grouping and line drift resilience.
191
+ - Message normalization and hex address masking.
192
+ - ReDoS guard and bounded multiline collection.
193
+ - Interleaved non-trace log line tolerance.
194
+ - SQLite WAL mode and concurrent transaction safety.
195
+ - Single-file HTML generation with full XSS escaping on every rendered field, plus a PII notice (normalization is not redaction).
196
+ - Chained exception collapse, bounded message collector, directory scan `--pattern` with skip/exit semantics.
197
+ - Fingerprint version isolation in history queries (`FINGERPRINT_SPEC.md`).
198
+ - CLI exit codes (0, 1, 2).
199
+ - Streaming pipeline memory regression ($O(1)$ constant memory ingestion).
200
+ - Raw input file cap backstop (`--max-input-mb`).
201
+ - Golden CLI output parity (byte-identical across refactors).
202
+
203
+ ---
204
+
205
+ ## License
206
+
207
+ MIT License. See [LICENSE](LICENSE) for details.
208
+
209
+ ---
210
+
211
+ ## Release & Changelog
212
+
213
+ See [CHANGELOG.md](CHANGELOG.md) for the version history. Current release: `1.0.0`.
214
+
215
+ Version consistency across the package (`pyproject.toml`), the CLI (`--version`), and
216
+ the git tag is enforced by `test_release_version_consistency`.
@@ -0,0 +1,9 @@
1
+ LICENSE
2
+ README.md
3
+ pra_effect.py
4
+ pyproject.toml
5
+ pra_effect.egg-info/PKG-INFO
6
+ pra_effect.egg-info/SOURCES.txt
7
+ pra_effect.egg-info/dependency_links.txt
8
+ pra_effect.egg-info/entry_points.txt
9
+ pra_effect.egg-info/top_level.txt
@@ -0,0 +1,2 @@
1
+ [console_scripts]
2
+ pra-effect = pra_effect:main
@@ -0,0 +1 @@
1
+ pra_effect