sqliteproof 0.1.0__tar.gz → 0.1.1__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,12 +1,12 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: sqliteproof
3
- Version: 0.1.0
4
- Summary: Structural validation for SQLite databases -- tells you which tables survived, not which pages broke.
3
+ Version: 0.1.1
4
+ Summary: Recover what survived a corrupt SQLite database: which tables are intact, how many rows are lost, what is safe to export. For 'database disk image is malformed'.
5
5
  License: MIT
6
6
  Project-URL: Homepage, https://github.com/OrbitalKeyAi/sqliteproof
7
7
  Project-URL: Source, https://github.com/OrbitalKeyAi/sqliteproof
8
8
  Project-URL: Issues, https://github.com/OrbitalKeyAi/sqliteproof/issues
9
- Keywords: sqlite,corruption,integrity,database,validation,backup,recovery,cli,devops
9
+ Keywords: sqlite,corruption,corrupt,malformed,database-disk-image-is-malformed,integrity-check,recover,recovery,repair,salvage,forensics,database,validation,backup,restore,dba,sysadmin,devops,cli
10
10
  Classifier: Development Status :: 3 - Alpha
11
11
  Classifier: Intended Audience :: System Administrators
12
12
  Classifier: Intended Audience :: Developers
@@ -122,6 +122,39 @@ Ground truth comes from SQLite's own page allocation: tables are built one at a
122
122
  the pages added between "before" and "after" belong to that table. The file format isn't
123
123
  my invention and the damage lands where SQLite chose to put the data.
124
124
 
125
+ ## If you got here from an error message
126
+
127
+ These are the messages SQLite prints when a database has gone bad. If you pasted one into a
128
+ search engine and landed here, this is what each one means and what this tool does about it.
129
+
130
+ | what SQLite told you | what it means | what sqliteproof adds |
131
+ |---|---|---|
132
+ | `database disk image is malformed` | The B-tree structure is damaged somewhere. SQLite will not say where. | Which tables still read completely, which are damaged, and how many rows are gone |
133
+ | `Tree N page M cell K: Offset … out of range` | `PRAGMA integrity_check` found a specific broken page | Translates the page into the table that owns it |
134
+ | `malformed database schema` | The schema table itself is damaged | Reports UNREADABLE rather than guessing |
135
+ | `no such table` after a crash | Possibly a lost schema page | Reads what is still there, read-only, without writing |
136
+
137
+ Two things worth knowing before you do anything else:
138
+
139
+ 1. **Copy the file first.** Work on the copy. Several recovery approaches write to the
140
+ database, and a failed repair on your only copy is unrecoverable. sqliteproof itself opens
141
+ the file read-only and never writes, which is why it is safe to run first.
142
+ 2. **A clean report is not a guarantee of correct data.** It means every row was structurally
143
+ readable. Corruption that produces valid-looking values cannot be detected this way, and
144
+ this tool says so rather than implying otherwise.
145
+
146
+ If you need to *repair* rather than assess, SQLite's own `.recover` command in the `sqlite3`
147
+ shell is the right next step. This tool tells you whether that is worth doing and what you
148
+ stand to lose — it does not replace it.
149
+
150
+
151
+ ## Related
152
+
153
+ **[statementproof](https://github.com/OrbitalKeyAi/statementproof)** — the same idea for
154
+ bank statement PDFs. Most converters hand you a CSV and leave you to trust it;
155
+ statementproof checks the extracted rows against the statement's own opening and closing
156
+ balances and names the row where the running total stops following.
157
+
125
158
  ## License
126
159
 
127
160
  MIT.
@@ -1,105 +1,138 @@
1
- # sqliteproof
2
-
3
- **Find out which tables survived, not which pages broke.**
4
-
5
- SQLite's own `PRAGMA integrity_check` is good at *detecting* corruption. What it gives you is this:
6
-
7
- ```
8
- *** in database main ***
9
- Tree 25 page 35 cell 18: Offset 57005 out of range 239..4092
10
- database disk image is malformed
11
- ```
12
-
13
- That's a page number. Nobody stores data by page number. What you actually need to know at
14
- 3am is **which tables can I still trust**, **how many rows did I lose**, and **is this worth
15
- restoring from backup**.
16
-
17
- ```
18
- $ sqliteproof app.db
19
-
20
- app.db — DAMAGED
21
-
22
- table verdict rows lost
23
- ------------------------------------------------------------
24
- orders damaged 392/400 8
25
- breaks after row 182: database disk image is malformed
26
- audit_log intact 400/400 0
27
- customers intact 400/400 0
28
-
29
- 1192 rows read, 8 unreadable.
30
- Tables marked intact above are safe to export. Restore the damaged
31
- ones from backup rather than trusting a partial read.
32
- ```
33
-
34
- ## Install
35
-
36
- ```bash
37
- pip install sqliteproof
38
- ```
39
-
40
- Python 3.9+. **No dependencies** — standard library only.
41
-
42
- ## Usage
43
-
44
- ```bash
45
- sqliteproof app.db # full report
46
- sqliteproof app.db --json # machine-readable
47
- sqliteproof app.db --quiet # verdict line only
48
- ```
49
-
50
- Exit codes: **0** intact · **1** damaged · **2** unreadable or undetermined. Drops straight
51
- into a backup script:
52
-
53
- ```bash
54
- sqliteproof app.db --quiet || echo "corruption detected" | mail -s alert me@example.com
55
- ```
56
-
57
- ## It opens the database read-only
58
-
59
- A tool asked to inspect a damaged file must never be able to damage it further. The
60
- connection is opened with `mode=ro` and nothing is written to the database, ever.
61
-
62
- Runs entirely on your machine. No network, nothing uploaded.
63
-
64
- ## It will not bluff
65
-
66
- | verdict | meaning |
67
- |---|---|
68
- | `INTACT` | every table read completely and row counts match |
69
- | `DAMAGED` | some rows are unreadable — **with the table named and the break point located** |
70
- | `UNKNOWN` | the damage prevents a determination |
71
-
72
- `UNKNOWN` is never dressed up as clean. A tool that reports a corrupt database as healthy
73
- is worse than no tool.
74
-
75
- ## Limitations — read these first
76
-
77
- **Structural, not semantic.** It verifies rows can be *read*. It cannot detect corruption
78
- that produces valid-looking values — a flipped bit inside an integer that still parses is
79
- invisible to it.
80
-
81
- **Row counts come from the same damaged btree.** When `COUNT(*)` itself fails, the expected
82
- count is unknown and the verdict degrades to `UNKNOWN` rather than guessing.
83
-
84
- **It does not repair anything.** It tells you what survived so you can export the good
85
- tables and restore the rest. Recovery is a different tool.
86
-
87
- **v0.1.0.** Tested against databases built by SQLite and damaged at known page offsets:
88
- 3/3 corrupted databases localised to the correct table, 0 false alarms, 0 false-clean.
89
- That corpus is deliberate byte corruption, which is one failure mode among several — real
90
- corruption also arrives via truncated files, interrupted writes, and failing disks.
91
-
92
- ## Tests
93
-
94
- ```bash
95
- python sqliteproof/tests/corrupt.py # build the corpus
96
- python sqliteproof/tests/score.py # score localisation vs ground truth
97
- ```
98
-
99
- Ground truth comes from SQLite's own page allocation: tables are built one at a time and
100
- the pages added between "before" and "after" belong to that table. The file format isn't
101
- my invention and the damage lands where SQLite chose to put the data.
102
-
103
- ## License
104
-
105
- MIT.
1
+ # sqliteproof
2
+
3
+ **Find out which tables survived, not which pages broke.**
4
+
5
+ SQLite's own `PRAGMA integrity_check` is good at *detecting* corruption. What it gives you is this:
6
+
7
+ ```
8
+ *** in database main ***
9
+ Tree 25 page 35 cell 18: Offset 57005 out of range 239..4092
10
+ database disk image is malformed
11
+ ```
12
+
13
+ That's a page number. Nobody stores data by page number. What you actually need to know at
14
+ 3am is **which tables can I still trust**, **how many rows did I lose**, and **is this worth
15
+ restoring from backup**.
16
+
17
+ ```
18
+ $ sqliteproof app.db
19
+
20
+ app.db — DAMAGED
21
+
22
+ table verdict rows lost
23
+ ------------------------------------------------------------
24
+ orders damaged 392/400 8
25
+ breaks after row 182: database disk image is malformed
26
+ audit_log intact 400/400 0
27
+ customers intact 400/400 0
28
+
29
+ 1192 rows read, 8 unreadable.
30
+ Tables marked intact above are safe to export. Restore the damaged
31
+ ones from backup rather than trusting a partial read.
32
+ ```
33
+
34
+ ## Install
35
+
36
+ ```bash
37
+ pip install sqliteproof
38
+ ```
39
+
40
+ Python 3.9+. **No dependencies** — standard library only.
41
+
42
+ ## Usage
43
+
44
+ ```bash
45
+ sqliteproof app.db # full report
46
+ sqliteproof app.db --json # machine-readable
47
+ sqliteproof app.db --quiet # verdict line only
48
+ ```
49
+
50
+ Exit codes: **0** intact · **1** damaged · **2** unreadable or undetermined. Drops straight
51
+ into a backup script:
52
+
53
+ ```bash
54
+ sqliteproof app.db --quiet || echo "corruption detected" | mail -s alert me@example.com
55
+ ```
56
+
57
+ ## It opens the database read-only
58
+
59
+ A tool asked to inspect a damaged file must never be able to damage it further. The
60
+ connection is opened with `mode=ro` and nothing is written to the database, ever.
61
+
62
+ Runs entirely on your machine. No network, nothing uploaded.
63
+
64
+ ## It will not bluff
65
+
66
+ | verdict | meaning |
67
+ |---|---|
68
+ | `INTACT` | every table read completely and row counts match |
69
+ | `DAMAGED` | some rows are unreadable — **with the table named and the break point located** |
70
+ | `UNKNOWN` | the damage prevents a determination |
71
+
72
+ `UNKNOWN` is never dressed up as clean. A tool that reports a corrupt database as healthy
73
+ is worse than no tool.
74
+
75
+ ## Limitations — read these first
76
+
77
+ **Structural, not semantic.** It verifies rows can be *read*. It cannot detect corruption
78
+ that produces valid-looking values — a flipped bit inside an integer that still parses is
79
+ invisible to it.
80
+
81
+ **Row counts come from the same damaged btree.** When `COUNT(*)` itself fails, the expected
82
+ count is unknown and the verdict degrades to `UNKNOWN` rather than guessing.
83
+
84
+ **It does not repair anything.** It tells you what survived so you can export the good
85
+ tables and restore the rest. Recovery is a different tool.
86
+
87
+ **v0.1.0.** Tested against databases built by SQLite and damaged at known page offsets:
88
+ 3/3 corrupted databases localised to the correct table, 0 false alarms, 0 false-clean.
89
+ That corpus is deliberate byte corruption, which is one failure mode among several — real
90
+ corruption also arrives via truncated files, interrupted writes, and failing disks.
91
+
92
+ ## Tests
93
+
94
+ ```bash
95
+ python sqliteproof/tests/corrupt.py # build the corpus
96
+ python sqliteproof/tests/score.py # score localisation vs ground truth
97
+ ```
98
+
99
+ Ground truth comes from SQLite's own page allocation: tables are built one at a time and
100
+ the pages added between "before" and "after" belong to that table. The file format isn't
101
+ my invention and the damage lands where SQLite chose to put the data.
102
+
103
+ ## If you got here from an error message
104
+
105
+ These are the messages SQLite prints when a database has gone bad. If you pasted one into a
106
+ search engine and landed here, this is what each one means and what this tool does about it.
107
+
108
+ | what SQLite told you | what it means | what sqliteproof adds |
109
+ |---|---|---|
110
+ | `database disk image is malformed` | The B-tree structure is damaged somewhere. SQLite will not say where. | Which tables still read completely, which are damaged, and how many rows are gone |
111
+ | `Tree N page M cell K: Offset … out of range` | `PRAGMA integrity_check` found a specific broken page | Translates the page into the table that owns it |
112
+ | `malformed database schema` | The schema table itself is damaged | Reports UNREADABLE rather than guessing |
113
+ | `no such table` after a crash | Possibly a lost schema page | Reads what is still there, read-only, without writing |
114
+
115
+ Two things worth knowing before you do anything else:
116
+
117
+ 1. **Copy the file first.** Work on the copy. Several recovery approaches write to the
118
+ database, and a failed repair on your only copy is unrecoverable. sqliteproof itself opens
119
+ the file read-only and never writes, which is why it is safe to run first.
120
+ 2. **A clean report is not a guarantee of correct data.** It means every row was structurally
121
+ readable. Corruption that produces valid-looking values cannot be detected this way, and
122
+ this tool says so rather than implying otherwise.
123
+
124
+ If you need to *repair* rather than assess, SQLite's own `.recover` command in the `sqlite3`
125
+ shell is the right next step. This tool tells you whether that is worth doing and what you
126
+ stand to lose — it does not replace it.
127
+
128
+
129
+ ## Related
130
+
131
+ **[statementproof](https://github.com/OrbitalKeyAi/statementproof)** — the same idea for
132
+ bank statement PDFs. Most converters hand you a CSV and leave you to trust it;
133
+ statementproof checks the extracted rows against the statement's own opening and closing
134
+ balances and names the row where the running total stops following.
135
+
136
+ ## License
137
+
138
+ MIT.
@@ -1,34 +1,38 @@
1
- [build-system]
2
- requires = ["setuptools>=68", "wheel"]
3
- build-backend = "setuptools.build_meta"
4
-
5
- [project]
6
- name = "sqliteproof"
7
- version = "0.1.0"
8
- description = "Structural validation for SQLite databases -- tells you which tables survived, not which pages broke."
9
- readme = "README.md"
10
- requires-python = ">=3.9"
11
- license = { text = "MIT" }
12
- keywords = ["sqlite", "corruption", "integrity", "database", "validation", "backup", "recovery", "cli", "devops"]
13
- classifiers = [
14
- "Development Status :: 3 - Alpha",
15
- "Intended Audience :: System Administrators",
16
- "Intended Audience :: Developers",
17
- "Topic :: Database",
18
- "Topic :: System :: Recovery Tools",
19
- "License :: OSI Approved :: MIT License",
20
- "Programming Language :: Python :: 3",
21
- "Operating System :: OS Independent",
22
- ]
23
- dependencies = []
24
-
25
- [project.urls]
26
- Homepage = "https://github.com/OrbitalKeyAi/sqliteproof"
27
- Source = "https://github.com/OrbitalKeyAi/sqliteproof"
28
- Issues = "https://github.com/OrbitalKeyAi/sqliteproof/issues"
29
-
30
- [project.scripts]
31
- sqliteproof = "sqliteproof.cli:main"
32
-
33
- [tool.setuptools]
34
- packages = ["sqliteproof"]
1
+ [build-system]
2
+ requires = ["setuptools>=68", "wheel"]
3
+ build-backend = "setuptools.build_meta"
4
+
5
+ [project]
6
+ name = "sqliteproof"
7
+ version = "0.1.1"
8
+ description = "Recover what survived a corrupt SQLite database: which tables are intact, how many rows are lost, what is safe to export. For 'database disk image is malformed'."
9
+ readme = "README.md"
10
+ requires-python = ">=3.9"
11
+ license = { text = "MIT" }
12
+ keywords = [
13
+ "sqlite", "corruption", "corrupt", "malformed", "database-disk-image-is-malformed",
14
+ "integrity-check", "recover", "recovery", "repair", "salvage", "forensics",
15
+ "database", "validation", "backup", "restore", "dba", "sysadmin", "devops", "cli",
16
+ ]
17
+ classifiers = [
18
+ "Development Status :: 3 - Alpha",
19
+ "Intended Audience :: System Administrators",
20
+ "Intended Audience :: Developers",
21
+ "Topic :: Database",
22
+ "Topic :: System :: Recovery Tools",
23
+ "License :: OSI Approved :: MIT License",
24
+ "Programming Language :: Python :: 3",
25
+ "Operating System :: OS Independent",
26
+ ]
27
+ dependencies = []
28
+
29
+ [project.urls]
30
+ Homepage = "https://github.com/OrbitalKeyAi/sqliteproof"
31
+ Source = "https://github.com/OrbitalKeyAi/sqliteproof"
32
+ Issues = "https://github.com/OrbitalKeyAi/sqliteproof/issues"
33
+
34
+ [project.scripts]
35
+ sqliteproof = "sqliteproof.cli:main"
36
+
37
+ [tool.setuptools]
38
+ packages = ["sqliteproof"]
@@ -1,2 +1,2 @@
1
- """sqliteproof - structural validation for SQLite databases."""
2
- __version__ = "0.1.0"
1
+ """sqliteproof - structural validation for SQLite databases."""
2
+ __version__ = "0.1.1"
@@ -1,12 +1,12 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: sqliteproof
3
- Version: 0.1.0
4
- Summary: Structural validation for SQLite databases -- tells you which tables survived, not which pages broke.
3
+ Version: 0.1.1
4
+ Summary: Recover what survived a corrupt SQLite database: which tables are intact, how many rows are lost, what is safe to export. For 'database disk image is malformed'.
5
5
  License: MIT
6
6
  Project-URL: Homepage, https://github.com/OrbitalKeyAi/sqliteproof
7
7
  Project-URL: Source, https://github.com/OrbitalKeyAi/sqliteproof
8
8
  Project-URL: Issues, https://github.com/OrbitalKeyAi/sqliteproof/issues
9
- Keywords: sqlite,corruption,integrity,database,validation,backup,recovery,cli,devops
9
+ Keywords: sqlite,corruption,corrupt,malformed,database-disk-image-is-malformed,integrity-check,recover,recovery,repair,salvage,forensics,database,validation,backup,restore,dba,sysadmin,devops,cli
10
10
  Classifier: Development Status :: 3 - Alpha
11
11
  Classifier: Intended Audience :: System Administrators
12
12
  Classifier: Intended Audience :: Developers
@@ -122,6 +122,39 @@ Ground truth comes from SQLite's own page allocation: tables are built one at a
122
122
  the pages added between "before" and "after" belong to that table. The file format isn't
123
123
  my invention and the damage lands where SQLite chose to put the data.
124
124
 
125
+ ## If you got here from an error message
126
+
127
+ These are the messages SQLite prints when a database has gone bad. If you pasted one into a
128
+ search engine and landed here, this is what each one means and what this tool does about it.
129
+
130
+ | what SQLite told you | what it means | what sqliteproof adds |
131
+ |---|---|---|
132
+ | `database disk image is malformed` | The B-tree structure is damaged somewhere. SQLite will not say where. | Which tables still read completely, which are damaged, and how many rows are gone |
133
+ | `Tree N page M cell K: Offset … out of range` | `PRAGMA integrity_check` found a specific broken page | Translates the page into the table that owns it |
134
+ | `malformed database schema` | The schema table itself is damaged | Reports UNREADABLE rather than guessing |
135
+ | `no such table` after a crash | Possibly a lost schema page | Reads what is still there, read-only, without writing |
136
+
137
+ Two things worth knowing before you do anything else:
138
+
139
+ 1. **Copy the file first.** Work on the copy. Several recovery approaches write to the
140
+ database, and a failed repair on your only copy is unrecoverable. sqliteproof itself opens
141
+ the file read-only and never writes, which is why it is safe to run first.
142
+ 2. **A clean report is not a guarantee of correct data.** It means every row was structurally
143
+ readable. Corruption that produces valid-looking values cannot be detected this way, and
144
+ this tool says so rather than implying otherwise.
145
+
146
+ If you need to *repair* rather than assess, SQLite's own `.recover` command in the `sqlite3`
147
+ shell is the right next step. This tool tells you whether that is worth doing and what you
148
+ stand to lose — it does not replace it.
149
+
150
+
151
+ ## Related
152
+
153
+ **[statementproof](https://github.com/OrbitalKeyAi/statementproof)** — the same idea for
154
+ bank statement PDFs. Most converters hand you a CSV and leave you to trust it;
155
+ statementproof checks the extracted rows against the statement's own opening and closing
156
+ balances and names the row where the running total stops following.
157
+
125
158
  ## License
126
159
 
127
160
  MIT.
File without changes
File without changes