statementproof 0.1.0__tar.gz → 0.2.0__tar.gz

statementproof-0.2.0/PKG-INFO

@@ -0,0 +1,222 @@
+Metadata-Version: 2.4
+Name: statementproof
+Version: 0.2.0
+Summary: Extract transactions from bank statement PDFs -- and prove the extraction is right, or say it isn't.
+License: MIT
+Project-URL: Homepage, https://github.com/OrbitalKeyAi/statementproof
+Project-URL: Documentation, https://orbitalkeyai.github.io/statementproof/
+Project-URL: Source, https://github.com/OrbitalKeyAi/statementproof
+Project-URL: Issues, https://github.com/OrbitalKeyAi/statementproof/issues
+Keywords: bank-statement,pdf,csv,bookkeeping,accounting,reconciliation,extraction,converter,quickbooks,xero
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Financial and Insurance Industry
+Classifier: Topic :: Office/Business :: Financial :: Accounting
+Classifier: Topic :: Text Processing :: Filters
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Operating System :: OS Independent
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: pdfplumber>=0.10
+Provides-Extra: dev
+Requires-Dist: reportlab>=4.0; extra == "dev"
+Dynamic: license-file
+
+# statementproof
+
+**Convert bank statement PDFs to CSV — and find out when the conversion is wrong.**
+
+Most PDF-to-CSV converters hand you rows with no way to check them. The best of them
+(monopoly) does check totals — but tells you only that *something* is wrong, somewhere. The expensive failure isn't a crash you notice — it's one debit read as a credit,
+which reconciles to nothing and turns up weeks later inside a client's books.
+
+A bank statement is one of the few documents that carries its own checksum: the balances.
+statementproof uses them.
+
+```
+$ statementproof october.pdf
+
+  opening balance : 69.96
+  closing balance : 586.71
+  transactions    : 21 extracted, 0 skipped
+
+  VERIFIED  (21 transactions, checks: chain, aggregate)
+```
+
+And when a row is misread:
+
+```
+  FAILED  (21 transactions, checks: chain, aggregate)
+    [chain_break] row 14: running balance does not follow:
+    534.66 +37.07 should give 571.73, statement shows 497.59 (off by -74.14)
+```
+
+## Install
+
+```bash
+pip install statementproof
+```
+
+Python 3.9+. One dependency (`pdfplumber`).
+
+## Usage
+
+```bash
+# check a statement and print the report
+statementproof statement.pdf
+
+# write the transactions to CSV
+statementproof statement.pdf --csv transactions.csv
+
+# layout report you can share — contains no financial data
+statementproof statement.pdf --diagnostic
+
+# a whole folder of statements, one summary
+statementproof ./january-statements/
+
+# write a reconciliation record you can file
+statementproof ./january-statements/ --record january.html
+```
+
+Exit code is `0` when verified, `1` otherwise, so it drops into a script. In batch mode a
+file that could not be opened also exits non-zero — **unreadable is never counted as
+passing.**
+
+## The reconciliation record
+
+`--record` writes a document recording *which exact file was checked, by whom, on what
+date, and what the check found.* It opens in any browser and prints to PDF.
+
+The load-bearing field is the **SHA-256 of the source PDF**. A record that names a file
+proves nothing — filenames change and statements get re-exported. The hash binds the
+record to one exact document, so if the PDF is later altered by a single byte, the record
+visibly no longer describes the file it is attached to.
+
+In batch mode the record also names every file that *could not* be read, in the printed
+copy as well as the JSON. A record that quietly omits the statements it choked on would
+assert a clean review of a folder it never finished reading.
+
+## Three verdicts, and it will not bluff
+
+| verdict | meaning |
+|---|---|
+| `VERIFIED` | the rows reproduce the statement's opening and closing balances, and each running balance follows from the last |
+| `FAILED` | they don't — **with the row number, expected figure, and size of the gap** |
+| `UNVERIFIED` | the statement carries no balances to check against. Output may be perfect; nothing proves it, so nothing is claimed |
+
+A converter that returns a clean-looking CSV it cannot vouch for is the problem this
+exists to solve, so `UNVERIFIED` is never dressed up as success.
+
+## Your statement never leaves your machine
+
+This tool asks you to point it at a document carrying your name, address, account
+number, employer, and every merchant you used last month. "Nothing uploaded" can't be a
+policy — it has to be a property of the code.
+
+**No networking library is imported anywhere in this package**, and
+[`tests/test_privacy.py`](statementproof/tests/test_privacy.py) parses every module's
+AST and **fails the build if one ever is.** No file is written unless you name it.
+
+The `--diagnostic` flag exists because a library of real statement *layouts* is what
+would most improve this tool — and a layout can be described without describing anyone's
+money. It reports column positions, column density, date-token **shapes** (`DD/DD`, not
+`10/02`), and where extraction broke. It contains no amounts, balances, descriptions,
+dates, account numbers, names, or even the filename. It prints to your screen so you can
+read all of it before deciding whether to share it. The tool never sends it anywhere.
+
+Those exclusions are asserted by tests against a known statement, not merely intended.
+
+## How it works
+
+**Geometry, not regex.** The most-cited converter failure is *"columns shift, debit and
+credit values land in the wrong places."* That happens because `extract_text()` flattens
+a two-dimensional page and discards the x-position — the only signal separating a debit
+column from a credit column. statementproof reads each number's coordinates and clusters
+the money columns by their right edges.
+
+**Column roles from arithmetic, not headers.** Which column holds the running balance is
+decided by behaviour, because header wording differs by bank and often vanishes on
+continuation pages.
+
+**Sign from the balance chain.** If the balance went down it was a debit, whether or not
+a minus glyph survived extraction.
+
+## Limitations — read these first
+
+**Text-layer PDFs only.** A statement downloaded from your bank's website normally has a
+text layer; a scan or a phone photo does not. Given one, statementproof says so rather
+than inventing rows. **OCR is not built yet.**
+
+**Arithmetic consistency, not correctness.** It cannot see an error that leaves the
+totals intact:
+
+- a page header picked up as a row with a `0.00` amount
+- a wrong date or a truncated description
+- two errors that cancel out
+
+*"Provably arithmetically consistent"* is the honest claim. *"Provably correct"* is not,
+and is not made here.
+
+**It has not met your bank.** Seven layouts extract exactly — six written by the author,
+one reproduced from a real bank's published specimen. That seventh immediately found
+three bugs the other six could not, including `MM/DD` dates with no year, which took
+extraction from 21/21 to 0/21 while the author's own suite still reported 6/6 passing.
+**Passing tests you wrote yourself is worth very little.**
+
+If it fails on your statement, run `--diagnostic` and
+[open an issue](https://github.com/OrbitalKeyAi/statementproof/issues) with that output.
+It contains no financial data, and each new layout makes the parser better for everyone.
+
+## Tests
+
+```bash
+python statementproof/tests/make_statements.py      # build synthetic corpus
+python statementproof/tests/make_real_derived.py    # build real-bank-derived layout
+python statementproof/tests/score.py                # score extraction vs ground truth
+python statementproof/tests/test_failure_modes.py   # validator behaviour
+python statementproof/tests/test_privacy.py         # privacy promises
+python statementproof/tests/test_record.py          # reconciliation record guarantees
+```
+
+Current: **7/7 layouts extracted exactly · 9/9 failure-mode tests · 8/8 privacy checks ·
+13/13 record guarantees · 0 false assurances.**
+
+"False assurance" — reporting `VERIFIED` on a bad extraction — is the number that
+matters. A validator that green-lights an error is worse than no validator.
+
+## Commercial licence
+
+The tool is free and MIT, and every figure it produces is real whether you pay or not.
+
+What a licence changes is one thing: **it puts your firm's name on the record.** Without
+it the record is stamped `UNLICENSED` — the numbers are still correct, but no one is named
+as having performed the review, and an unsigned document is not evidence that anyone did.
+
+**[Buy a licence — $39, one-time, perpetual](https://buy.stripe.com/4gM8wRe6NdON50s2kjgA802)**
+· unlimited statements · unlimited machines · no subscription
+
+Then set it once per machine:
+
+```bash
+setx STATEMENTPROOF_LICENCE "Your Firm Name"        # Windows
+export STATEMENTPROOF_LICENCE="Your Firm Name"      # macOS / Linux
+```
+
+There is no key server, no activation call, and no check-in — the package still contains
+no network code of any kind. The licence is an attestation, not a lock. Since the source
+is public, any check could be deleted in seconds; pretending otherwise would be exactly
+the kind of bluffing this tool exists to refuse. Editing the stamp out would produce a
+record naming a licence holder who did not buy one, which is forging your own audit
+evidence, and no payment system prevents that either.
+
+## Related
+
+**[sqliteproof](https://github.com/OrbitalKeyAi/sqliteproof)** — the same idea for SQLite
+databases. `PRAGMA integrity_check` tells you a page number; sqliteproof tells you which
+tables survived, how many rows were lost, and what is safe to export. Free, MIT, no
+dependencies.
+
+## License
+
+MIT.

{statementproof-0.1.0 → statementproof-0.2.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "statementproof"
-version = "0.1.0"
+version = "0.2.0"
 description = "Extract transactions from bank statement PDFs -- and prove the extraction is right, or say it isn't."
 readme = "statementproof/README.md"
 requires-python = ">=3.9"
@@ -26,6 +26,12 @@ dependencies = [
     "pdfplumber>=0.10",
 ]
 
+[project.urls]
+Homepage = "https://github.com/OrbitalKeyAi/statementproof"
+Documentation = "https://orbitalkeyai.github.io/statementproof/"
+Source = "https://github.com/OrbitalKeyAi/statementproof"
+Issues = "https://github.com/OrbitalKeyAi/statementproof/issues"
+
 [project.optional-dependencies]
 dev = ["reportlab>=4.0"]
 

statementproof-0.2.0/statementproof/README.md

@@ -0,0 +1,197 @@
+# statementproof
+
+**Convert bank statement PDFs to CSV — and find out when the conversion is wrong.**
+
+Most PDF-to-CSV converters hand you rows with no way to check them. The best of them
+(monopoly) does check totals — but tells you only that *something* is wrong, somewhere. The expensive failure isn't a crash you notice — it's one debit read as a credit,
+which reconciles to nothing and turns up weeks later inside a client's books.
+
+A bank statement is one of the few documents that carries its own checksum: the balances.
+statementproof uses them.
+
+```
+$ statementproof october.pdf
+
+  opening balance : 69.96
+  closing balance : 586.71
+  transactions    : 21 extracted, 0 skipped
+
+  VERIFIED  (21 transactions, checks: chain, aggregate)
+```
+
+And when a row is misread:
+
+```
+  FAILED  (21 transactions, checks: chain, aggregate)
+    [chain_break] row 14: running balance does not follow:
+    534.66 +37.07 should give 571.73, statement shows 497.59 (off by -74.14)
+```
+
+## Install
+
+```bash
+pip install statementproof
+```
+
+Python 3.9+. One dependency (`pdfplumber`).
+
+## Usage
+
+```bash
+# check a statement and print the report
+statementproof statement.pdf
+
+# write the transactions to CSV
+statementproof statement.pdf --csv transactions.csv
+
+# layout report you can share — contains no financial data
+statementproof statement.pdf --diagnostic
+
+# a whole folder of statements, one summary
+statementproof ./january-statements/
+
+# write a reconciliation record you can file
+statementproof ./january-statements/ --record january.html
+```
+
+Exit code is `0` when verified, `1` otherwise, so it drops into a script. In batch mode a
+file that could not be opened also exits non-zero — **unreadable is never counted as
+passing.**
+
+## The reconciliation record
+
+`--record` writes a document recording *which exact file was checked, by whom, on what
+date, and what the check found.* It opens in any browser and prints to PDF.
+
+The load-bearing field is the **SHA-256 of the source PDF**. A record that names a file
+proves nothing — filenames change and statements get re-exported. The hash binds the
+record to one exact document, so if the PDF is later altered by a single byte, the record
+visibly no longer describes the file it is attached to.
+
+In batch mode the record also names every file that *could not* be read, in the printed
+copy as well as the JSON. A record that quietly omits the statements it choked on would
+assert a clean review of a folder it never finished reading.
+
+## Three verdicts, and it will not bluff
+
+| verdict | meaning |
+|---|---|
+| `VERIFIED` | the rows reproduce the statement's opening and closing balances, and each running balance follows from the last |
+| `FAILED` | they don't — **with the row number, expected figure, and size of the gap** |
+| `UNVERIFIED` | the statement carries no balances to check against. Output may be perfect; nothing proves it, so nothing is claimed |
+
+A converter that returns a clean-looking CSV it cannot vouch for is the problem this
+exists to solve, so `UNVERIFIED` is never dressed up as success.
+
+## Your statement never leaves your machine
+
+This tool asks you to point it at a document carrying your name, address, account
+number, employer, and every merchant you used last month. "Nothing uploaded" can't be a
+policy — it has to be a property of the code.
+
+**No networking library is imported anywhere in this package**, and
+[`tests/test_privacy.py`](statementproof/tests/test_privacy.py) parses every module's
+AST and **fails the build if one ever is.** No file is written unless you name it.
+
+The `--diagnostic` flag exists because a library of real statement *layouts* is what
+would most improve this tool — and a layout can be described without describing anyone's
+money. It reports column positions, column density, date-token **shapes** (`DD/DD`, not
+`10/02`), and where extraction broke. It contains no amounts, balances, descriptions,
+dates, account numbers, names, or even the filename. It prints to your screen so you can
+read all of it before deciding whether to share it. The tool never sends it anywhere.
+
+Those exclusions are asserted by tests against a known statement, not merely intended.
+
+## How it works
+
+**Geometry, not regex.** The most-cited converter failure is *"columns shift, debit and
+credit values land in the wrong places."* That happens because `extract_text()` flattens
+a two-dimensional page and discards the x-position — the only signal separating a debit
+column from a credit column. statementproof reads each number's coordinates and clusters
+the money columns by their right edges.
+
+**Column roles from arithmetic, not headers.** Which column holds the running balance is
+decided by behaviour, because header wording differs by bank and often vanishes on
+continuation pages.
+
+**Sign from the balance chain.** If the balance went down it was a debit, whether or not
+a minus glyph survived extraction.
+
+## Limitations — read these first
+
+**Text-layer PDFs only.** A statement downloaded from your bank's website normally has a
+text layer; a scan or a phone photo does not. Given one, statementproof says so rather
+than inventing rows. **OCR is not built yet.**
+
+**Arithmetic consistency, not correctness.** It cannot see an error that leaves the
+totals intact:
+
+- a page header picked up as a row with a `0.00` amount
+- a wrong date or a truncated description
+- two errors that cancel out
+
+*"Provably arithmetically consistent"* is the honest claim. *"Provably correct"* is not,
+and is not made here.
+
+**It has not met your bank.** Seven layouts extract exactly — six written by the author,
+one reproduced from a real bank's published specimen. That seventh immediately found
+three bugs the other six could not, including `MM/DD` dates with no year, which took
+extraction from 21/21 to 0/21 while the author's own suite still reported 6/6 passing.
+**Passing tests you wrote yourself is worth very little.**
+
+If it fails on your statement, run `--diagnostic` and
+[open an issue](https://github.com/OrbitalKeyAi/statementproof/issues) with that output.
+It contains no financial data, and each new layout makes the parser better for everyone.
+
+## Tests
+
+```bash
+python statementproof/tests/make_statements.py      # build synthetic corpus
+python statementproof/tests/make_real_derived.py    # build real-bank-derived layout
+python statementproof/tests/score.py                # score extraction vs ground truth
+python statementproof/tests/test_failure_modes.py   # validator behaviour
+python statementproof/tests/test_privacy.py         # privacy promises
+python statementproof/tests/test_record.py          # reconciliation record guarantees
+```
+
+Current: **7/7 layouts extracted exactly · 9/9 failure-mode tests · 8/8 privacy checks ·
+13/13 record guarantees · 0 false assurances.**
+
+"False assurance" — reporting `VERIFIED` on a bad extraction — is the number that
+matters. A validator that green-lights an error is worse than no validator.
+
+## Commercial licence
+
+The tool is free and MIT, and every figure it produces is real whether you pay or not.
+
+What a licence changes is one thing: **it puts your firm's name on the record.** Without
+it the record is stamped `UNLICENSED` — the numbers are still correct, but no one is named
+as having performed the review, and an unsigned document is not evidence that anyone did.
+
+**[Buy a licence — $39, one-time, perpetual](https://buy.stripe.com/4gM8wRe6NdON50s2kjgA802)**
+· unlimited statements · unlimited machines · no subscription
+
+Then set it once per machine:
+
+```bash
+setx STATEMENTPROOF_LICENCE "Your Firm Name"        # Windows
+export STATEMENTPROOF_LICENCE="Your Firm Name"      # macOS / Linux
+```
+
+There is no key server, no activation call, and no check-in — the package still contains
+no network code of any kind. The licence is an attestation, not a lock. Since the source
+is public, any check could be deleted in seconds; pretending otherwise would be exactly
+the kind of bluffing this tool exists to refuse. Editing the stamp out would produce a
+record naming a licence holder who did not buy one, which is forging your own audit
+evidence, and no payment system prevents that either.
+
+## Related
+
+**[sqliteproof](https://github.com/OrbitalKeyAi/sqliteproof)** — the same idea for SQLite
+databases. `PRAGMA integrity_check` tells you a page number; sqliteproof tells you which
+tables survived, how many rows were lost, and what is safe to export. Free, MIT, no
+dependencies.
+
+## License
+
+MIT.

{statementproof-0.1.0 → statementproof-0.2.0}/statementproof/__init__.py

@@ -1,2 +1,2 @@
 """statementproof - verifiable bank statement extraction."""
-__version__ = "0.1.0"
+__version__ = "0.2.0"

{statementproof-0.1.0 → statementproof-0.2.0}/statementproof/cli.py

@@ -99,17 +99,85 @@ def build_diagnostic(pdf_path, txns, diag, opening, closing, res):
     }
 
 
+def run_batch(args):
+    """Validate every PDF in a folder, then emit one record covering all of them.
+
+    A firm processes many clients' statements a month, and the free single-file path makes
+    that a hundred terminal invocations nobody keeps a trace of. What matters here is that
+    **a file that fails to open is reported, not skipped silently** -- a batch summary that
+    quietly omits the statements it choked on is precisely the false assurance this whole
+    tool exists to refuse.
+    """
+    from . import record as _rec
+
+    pdfs = sorted(os.path.join(args.pdf, f) for f in os.listdir(args.pdf)
+                  if f.lower().endswith(".pdf"))
+    if not pdfs:
+        print("error: no PDFs in %s" % args.pdf, file=sys.stderr)
+        return 2
+
+    if not args.quiet:
+        print(BANNER)
+        print("  %d statement(s) in %s\n" % (len(pdfs), args.pdf))
+
+    records, failed_to_read = [], []
+    for p in pdfs:
+        name = os.path.basename(p)
+        try:
+            opening, closing = find_balances(p)
+            txns, diag = parse(p, balance_hints=(opening, closing))
+            res = validate(txns, opening, closing)
+        except Exception as e:
+            failed_to_read.append((name, "%s: %s" % (type(e).__name__, e)))
+            if not args.quiet:
+                print("  %-38s COULD NOT READ  (%s)" % (name[:38], type(e).__name__))
+            continue
+
+        records.append(_rec.build(p, txns, res, opening, closing, diag))
+        if not args.quiet:
+            print("  %-38s %-13s %3d txns" % (name[:38], res.verdict, len(txns)))
+
+    batch = _rec.build_batch(records)
+    if failed_to_read:
+        batch["summary"]["unreadable"] = [{"file": n, "error": e} for n, e in failed_to_read]
+
+    if not args.quiet:
+        print()
+        for verdict, n in batch["summary"]["by_verdict"].items():
+            print("  %-13s %d" % (verdict, n))
+        if failed_to_read:
+            print("  %-13s %d   <-- not validated; they are not counted as passing"
+                  % ("UNREADABLE", len(failed_to_read)))
+
+    if args.record:
+        _rec.write(batch, args.record)
+        if not args.quiet:
+            print("\n  wrote batch reconciliation record to %s" % args.record)
+            if batch["licence"]["status"] != "licensed":
+                print("  (UNLICENSED -- set %s to name the reviewer.)" % _rec.LICENCE_ENV)
+
+    # Non-zero if anything failed or could not be read, so it drops into a firm's scripts.
+    bad = batch["summary"]["by_verdict"].get("FAILED", 0) + len(failed_to_read)
+    return 1 if bad else 0
+
+
 def main(argv=None):
     ap = argparse.ArgumentParser(
         prog="statementproof",
         description="Validate bank statement PDF extraction. Runs locally; uploads nothing.")
-    ap.add_argument("pdf", help="path to a bank statement PDF")
+    ap.add_argument("pdf", help="path to a bank statement PDF, or a folder of them")
     ap.add_argument("--csv", metavar="OUT", help="write extracted transactions to CSV")
+    ap.add_argument("--record", metavar="OUT",
+                    help="write a reconciliation record (.html or .json) -- the artifact "
+                         "recording which file was checked, by whom, and what was found")
     ap.add_argument("--diagnostic", action="store_true",
                     help="print a shareable layout report containing no financial data")
     ap.add_argument("--quiet", action="store_true", help="verdict line only")
     args = ap.parse_args(argv)
 
+    if os.path.isdir(args.pdf):
+        return run_batch(args)
+
     if not os.path.isfile(args.pdf):
         print("error: no such file: %s" % args.pdf, file=sys.stderr)
         return 2
@@ -173,6 +241,17 @@ def main(argv=None):
         if not args.quiet:
             print("\n  wrote %d rows to %s" % (len(txns), args.csv))
 
+    if args.record:
+        from . import record as _rec
+        r = _rec.build(args.pdf, txns, res, opening, closing, diag)
+        _rec.write(r, args.record)
+        if not args.quiet:
+            print("\n  wrote reconciliation record to %s" % args.record)
+            if r["licence"]["status"] != "licensed":
+                print("  (UNLICENSED -- the figures are real, but no licence holder is named,")
+                print("   so it does not record who performed the review. Set %s.)"
+                      % _rec.LICENCE_ENV)
+
     if args.diagnostic:
         d = build_diagnostic(args.pdf, txns, diag, opening, closing, res)
         print("\n" + "=" * 68)