doltlite 0.11.2__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,201 @@
1
+ Apache License
2
+ Version 2.0, January 2004
3
+ http://www.apache.org/licenses/
4
+
5
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
6
+
7
+ 1. Definitions.
8
+
9
+ "License" shall mean the terms and conditions for use, reproduction,
10
+ and distribution as defined by Sections 1 through 9 of this document.
11
+
12
+ "Licensor" shall mean the copyright owner or entity authorized by
13
+ the copyright owner that is granting the License.
14
+
15
+ "Legal Entity" shall mean the union of the acting entity and all
16
+ other entities that control, are controlled by, or are under common
17
+ control with that entity. For the purposes of this definition,
18
+ "control" means (i) the power, direct or indirect, to cause the
19
+ direction or management of such entity, whether by contract or
20
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
21
+ outstanding shares, or (iii) beneficial ownership of such entity.
22
+
23
+ "You" (or "Your") shall mean an individual or Legal Entity
24
+ exercising permissions granted by this License.
25
+
26
+ "Source" form shall mean the preferred form for making modifications,
27
+ including but not limited to software source code, documentation
28
+ source, and configuration files.
29
+
30
+ "Object" form shall mean any form resulting from mechanical
31
+ transformation or translation of a Source form, including but
32
+ not limited to compiled object code, generated documentation,
33
+ and conversions to other media types.
34
+
35
+ "Work" shall mean the work of authorship, whether in Source or
36
+ Object form, made available under the License, as indicated by a
37
+ copyright notice that is included in or attached to the work
38
+ (an example is provided in the Appendix below).
39
+
40
+ "Derivative Works" shall mean any work, whether in Source or Object
41
+ form, that is based on (or derived from) the Work and for which the
42
+ editorial revisions, annotations, elaborations, or other modifications
43
+ represent, as a whole, an original work of authorship. For the purposes
44
+ of this License, Derivative Works shall not include works that remain
45
+ separable from, or merely link (or bind by name) to the interfaces of,
46
+ the Work and Derivative Works thereof.
47
+
48
+ "Contribution" shall mean any work of authorship, including
49
+ the original version of the Work and any modifications or additions
50
+ to that Work or Derivative Works thereof, that is intentionally
51
+ submitted to Licensor for inclusion in the Work by the copyright owner
52
+ or by an individual or Legal Entity authorized to submit on behalf of
53
+ the copyright owner. For the purposes of this definition, "submitted"
54
+ means any form of electronic, verbal, or written communication sent
55
+ to the Licensor or its representatives, including but not limited to
56
+ communication on electronic mailing lists, source code control systems,
57
+ and issue tracking systems that are managed by, or on behalf of, the
58
+ Licensor for the purpose of discussing and improving the Work, but
59
+ excluding communication that is conspicuously marked or otherwise
60
+ designated in writing by the copyright owner as "Not a Contribution."
61
+
62
+ "Contributor" shall mean Licensor and any individual or Legal Entity
63
+ on behalf of whom a Contribution has been received by Licensor and
64
+ subsequently incorporated within the Work.
65
+
66
+ 2. Grant of Copyright License. Subject to the terms and conditions of
67
+ this License, each Contributor hereby grants to You a perpetual,
68
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
69
+ copyright license to reproduce, prepare Derivative Works of,
70
+ publicly display, publicly perform, sublicense, and distribute the
71
+ Work and such Derivative Works in Source or Object form.
72
+
73
+ 3. Grant of Patent License. Subject to the terms and conditions of
74
+ this License, each Contributor hereby grants to You a perpetual,
75
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
76
+ (except as stated in this section) patent license to make, have made,
77
+ use, offer to sell, sell, import, and otherwise transfer the Work,
78
+ where such license applies only to those patent claims licensable
79
+ by such Contributor that are necessarily infringed by their
80
+ Contribution(s) alone or by combination of their Contribution(s)
81
+ with the Work to which such Contribution(s) was submitted. If You
82
+ institute patent litigation against any entity (including a
83
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
84
+ or a Contribution incorporated within the Work constitutes direct
85
+ or contributory patent infringement, then any patent licenses
86
+ granted to You under this License for that Work shall terminate
87
+ as of the date such litigation is filed.
88
+
89
+ 4. Redistribution. You may reproduce and distribute copies of the
90
+ Work or Derivative Works thereof in any medium, with or without
91
+ modifications, and in Source or Object form, provided that You
92
+ meet the following conditions:
93
+
94
+ (a) You must give any other recipients of the Work or
95
+ Derivative Works a copy of this License; and
96
+
97
+ (b) You must cause any modified files to carry prominent notices
98
+ stating that You changed the files; and
99
+
100
+ (c) You must retain, in the Source form of any Derivative Works
101
+ that You distribute, all copyright, patent, trademark, and
102
+ attribution notices from the Source form of the Work,
103
+ excluding those notices that do not pertain to any part of
104
+ the Derivative Works; and
105
+
106
+ (d) If the Work includes a "NOTICE" text file as part of its
107
+ distribution, then any Derivative Works that You distribute must
108
+ include a readable copy of the attribution notices contained
109
+ within such NOTICE file, excluding those notices that do not
110
+ pertain to any part of the Derivative Works, in at least one
111
+ of the following places: within a NOTICE text file distributed
112
+ as part of the Derivative Works; within the Source form or
113
+ documentation, if provided along with the Derivative Works; or,
114
+ within a display generated by the Derivative Works, if and
115
+ wherever such third-party notices normally appear. The contents
116
+ of the NOTICE file are for informational purposes only and
117
+ do not modify the License. You may add Your own attribution
118
+ notices within Derivative Works that You distribute, alongside
119
+ or as an addendum to the NOTICE text from the Work, provided
120
+ that such additional attribution notices cannot be construed
121
+ as modifying the License.
122
+
123
+ You may add Your own copyright statement to Your modifications and
124
+ may provide additional or different license terms and conditions
125
+ for use, reproduction, or distribution of Your modifications, or
126
+ for any such Derivative Works as a whole, provided Your use,
127
+ reproduction, and distribution of the Work otherwise complies with
128
+ the conditions stated in this License.
129
+
130
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
131
+ any Contribution intentionally submitted for inclusion in the Work
132
+ by You to the Licensor shall be under the terms and conditions of
133
+ this License, without any additional terms or conditions.
134
+ Notwithstanding the above, nothing herein shall supersede or modify
135
+ the terms of any separate license agreement you may have executed
136
+ with Licensor regarding such Contributions.
137
+
138
+ 6. Trademarks. This License does not grant permission to use the trade
139
+ names, trademarks, service marks, or product names of the Licensor,
140
+ except as required for reasonable and customary use in describing the
141
+ origin of the Work and reproducing the content of the NOTICE file.
142
+
143
+ 7. Disclaimer of Warranty. Unless required by applicable law or
144
+ agreed to in writing, Licensor provides the Work (and each
145
+ Contributor provides its Contributions) on an "AS IS" BASIS,
146
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
147
+ implied, including, without limitation, any warranties or conditions
148
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
149
+ PARTICULAR PURPOSE. You are solely responsible for determining the
150
+ appropriateness of using or redistributing the Work and assume any
151
+ risks associated with Your exercise of permissions under this License.
152
+
153
+ 8. Limitation of Liability. In no event and under no legal theory,
154
+ whether in tort (including negligence), contract, or otherwise,
155
+ unless required by applicable law (such as deliberate and grossly
156
+ negligent acts) or agreed to in writing, shall any Contributor be
157
+ liable to You for damages, including any direct, indirect, special,
158
+ incidental, or consequential damages of any character arising as a
159
+ result of this License or out of the use or inability to use the
160
+ Work (including but not limited to damages for loss of goodwill,
161
+ work stoppage, computer failure or malfunction, or any and all
162
+ other commercial damages or losses), even if such Contributor
163
+ has been advised of the possibility of such damages.
164
+
165
+ 9. Accepting Warranty or Additional Liability. While redistributing
166
+ the Work or Derivative Works thereof, You may choose to offer,
167
+ and charge a fee for, acceptance of support, warranty, indemnity,
168
+ or other liability obligations and/or rights consistent with this
169
+ License. However, in accepting such obligations, You may act only
170
+ on Your own behalf and on Your sole responsibility, not on behalf
171
+ of any other Contributor, and only if You agree to indemnify,
172
+ defend, and hold each Contributor harmless for any liability
173
+ incurred by, or claims asserted against, such Contributor by reason
174
+ of your accepting any such warranty or additional liability.
175
+
176
+ END OF TERMS AND CONDITIONS
177
+
178
+ APPENDIX: How to apply the Apache License to your work.
179
+
180
+ To apply the Apache License to your work, attach the following
181
+ boilerplate notice, with the fields enclosed by brackets "[]"
182
+ replaced with your own identifying information. (Don't include
183
+ the brackets!) The text should be enclosed in the appropriate
184
+ comment syntax for the file format. We also recommend that a
185
+ file or class name and description of purpose be included on the
186
+ same "printed page" as the copyright notice for easier
187
+ identification within third-party archives.
188
+
189
+ Copyright [yyyy] [name of copyright owner]
190
+
191
+ Licensed under the Apache License, Version 2.0 (the "License");
192
+ you may not use this file except in compliance with the License.
193
+ You may obtain a copy of the License at
194
+
195
+ http://www.apache.org/licenses/LICENSE-2.0
196
+
197
+ Unless required by applicable law or agreed to in writing, software
198
+ distributed under the License is distributed on an "AS IS" BASIS,
199
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
200
+ See the License for the specific language governing permissions and
201
+ limitations under the License.
@@ -0,0 +1,174 @@
1
+ Metadata-Version: 2.4
2
+ Name: doltlite
3
+ Version: 0.11.2
4
+ Summary: Python loader for doltlite — Dolt version control through SQLite's drop-in API.
5
+ Author-email: DoltHub <tim@dolthub.com>
6
+ License: Apache-2.0
7
+ Project-URL: Homepage, https://github.com/dolthub/doltlite
8
+ Project-URL: Source, https://github.com/dolthub/doltlite-python
9
+ Keywords: dolt,doltlite,sqlite,version-control,database
10
+ Classifier: Development Status :: 3 - Alpha
11
+ Classifier: Intended Audience :: Developers
12
+ Classifier: License :: OSI Approved :: Apache Software License
13
+ Classifier: Operating System :: MacOS
14
+ Classifier: Operating System :: POSIX :: Linux
15
+ Classifier: Programming Language :: Python :: 3
16
+ Classifier: Programming Language :: Python :: 3 :: Only
17
+ Classifier: Topic :: Database
18
+ Requires-Python: >=3.9
19
+ Description-Content-Type: text/markdown
20
+ License-File: LICENSE
21
+ Dynamic: license-file
22
+
23
+ # doltlite (Python)
24
+
25
+ A Python loader for [doltlite](https://github.com/dolthub/doltlite) —
26
+ Dolt's version control through SQLite's drop-in API.
27
+
28
+ ```python
29
+ import doltlite # one-time bootstrap
30
+ import sqlite3
31
+
32
+ conn = sqlite3.connect("repo.db")
33
+ conn.execute("CREATE TABLE t(x INT)")
34
+ conn.execute("INSERT INTO t VALUES (1)")
35
+ conn.execute("SELECT dolt_commit('-A', '-m', 'init')")
36
+ ```
37
+
38
+ `import doltlite` makes libdoltlite the active SQLite engine for the
39
+ current interpreter, so anything that uses the standard `sqlite3` module
40
+ — **including SQLAlchemy with `sqlite:///...` URLs** — transparently
41
+ gains `dolt_commit`, `dolt_branch`, `dolt_merge`, `dolt_log`,
42
+ `dolt_diff_<table>`, and the other Dolt SQL functions and virtual tables.
43
+
44
+ ## Install
45
+
46
+ ```bash
47
+ pip install doltlite
48
+ ```
49
+
50
+ Wheels bundle a precompiled `libdoltlite` for macOS (arm64) and Linux
51
+ (x86_64, aarch64). No system-level setup required.
52
+
53
+ Intel Mac wheels aren't shipped in v0.11.x — GitHub's free-tier
54
+ macos-13 runners queue for hours and block releases. Intel Mac users
55
+ should build libdoltlite locally and use the `DOLTLITE_LIB` path below.
56
+
57
+ For development against a local checkout of doltlite, point
58
+ `DOLTLITE_LIB` at your built library instead:
59
+
60
+ ```bash
61
+ DOLTLITE_LIB=/path/to/libdoltlite.dylib python3 your_script.py
62
+ ```
63
+
64
+ ## Requirements
65
+
66
+ The package piggybacks on Python's stdlib `sqlite3`, which must load
67
+ SQLite as a shared extension at runtime. The following Pythons work:
68
+
69
+ - Distro / system Python (Linux)
70
+ - Homebrew Python (macOS, Linux)
71
+ - pyenv-built Python
72
+ - Conda Python
73
+
74
+ These do **not** work because their `_sqlite3` is statically linked into
75
+ the interpreter:
76
+
77
+ - **python-build-standalone** interpreters — the default for `uv python
78
+ install`, `mise`, and Rye
79
+
80
+ If you use `uv`, target one of the supported Pythons explicitly:
81
+
82
+ ```bash
83
+ uv venv --python /opt/homebrew/bin/python3 # or /usr/bin/python3
84
+ ```
85
+
86
+ ## How the bootstrap works
87
+
88
+ Doltlite is a SQLite drop-in: it implements the same `sqlite3_*` C API
89
+ and adds Dolt-specific functions and virtual tables on top. To use it
90
+ from Python, libdoltlite has to be loaded ahead of the system libsqlite3
91
+ so its symbols win during the dynamic-link symbol-resolution pass that
92
+ Python's `_sqlite3` module triggers when it's first imported.
93
+
94
+ The mechanics differ by platform:
95
+
96
+ ### Linux
97
+
98
+ ELF flat-namespace symbol resolution: if libdoltlite is loaded into the
99
+ process with `RTLD_GLOBAL` **before** `_sqlite3` is loaded, its
100
+ `sqlite3_*` symbols enter the global namespace and the later
101
+ `libsqlite3.so` lookup finds them first.
102
+
103
+ `import doltlite` does `ctypes.CDLL(libdoltlite_path,
104
+ mode=ctypes.RTLD_GLOBAL)` in this case — no re-exec required.
105
+
106
+ If `sqlite3` was already imported before `doltlite`, that boat has
107
+ sailed: we fall back to re-execing the interpreter with `LD_PRELOAD`
108
+ set, so libdoltlite is loaded at process start.
109
+
110
+ ### macOS
111
+
112
+ macOS uses a two-level namespace: `_sqlite3.so` has an `LC_LOAD_DYLIB`
113
+ command bound to a specific `libsqlite3.dylib` path (e.g.
114
+ `/opt/homebrew/opt/sqlite/lib/libsqlite3.dylib`). Plain `dlopen` /
115
+ `ctypes.CDLL` does **not** redirect that resolution, and
116
+ `DYLD_INSERT_LIBRARIES` alone doesn't either — the inserted library has
117
+ to have an `install_name` (`LC_ID_DYLIB`) that matches the path
118
+ `_sqlite3.so` was linked against.
119
+
120
+ `import doltlite` does:
121
+
122
+ 1. Detect that path via `otool -L $(python3 -c 'import _sqlite3;
123
+ print(_sqlite3.__file__)')`.
124
+ 2. Copy libdoltlite to `$TMPDIR/.../libsqlite3.dylib` (cached per
125
+ (lib, mtime, install_name)).
126
+ 3. Rewrite the shim's install_name with `install_name_tool -id <path>`.
127
+ 4. Re-exec the interpreter with `DYLD_INSERT_LIBRARIES=<shim>`.
128
+
129
+ The two-level lookup then accepts the shim because the install_name
130
+ matches.
131
+
132
+ This requires `otool` and `install_name_tool` — install Xcode Command
133
+ Line Tools (`xcode-select --install`) if missing.
134
+
135
+ ## Re-exec caveats
136
+
137
+ When a re-exec is required (macOS, or Linux-after-sqlite3-loaded), the
138
+ bootstrap calls `os.execvpe` with `sys.argv`. That means the
139
+ invocation must name a script file we can replay:
140
+
141
+ - ✅ `python3 my_script.py`
142
+ - ✅ `python3 -m my_package`
143
+ - ❌ `python3 -c "import doltlite; ..."` — code string isn't in argv
144
+ - ❌ Interactive REPL / Jupyter — there's no script to re-exec
145
+
146
+ In the unsupported cases the bootstrap raises `DoltliteLoadError` with
147
+ a clear workaround: set `DYLD_INSERT_LIBRARIES` (macOS) or `LD_PRELOAD`
148
+ (Linux) yourself before starting Python.
149
+
150
+ ## API
151
+
152
+ ```python
153
+ import doltlite
154
+
155
+ # Side-effect import does the bootstrap automatically. Subsequent
156
+ # imports are no-ops thanks to a process-env marker.
157
+
158
+ # If you want to bootstrap explicitly (e.g. inside a function):
159
+ doltlite.bootstrap()
160
+
161
+ # Find where the loaded libdoltlite came from:
162
+ doltlite.libdoltlite_path() # absolute path
163
+ ```
164
+
165
+ ## See also
166
+
167
+ - [doltlite](https://github.com/dolthub/doltlite) — the underlying
168
+ SQLite + Dolt engine
169
+ - [doltlite-sqlalchemy-getting-started](https://github.com/timsehn/doltlite-sqlalchemy-getting-started)
170
+ — end-to-end SQLAlchemy demo (commits, branches, schema change, merge)
171
+
172
+ ## License
173
+
174
+ Apache License 2.0. See [LICENSE](LICENSE).
@@ -0,0 +1,152 @@
1
+ # doltlite (Python)
2
+
3
+ A Python loader for [doltlite](https://github.com/dolthub/doltlite) —
4
+ Dolt's version control through SQLite's drop-in API.
5
+
6
+ ```python
7
+ import doltlite # one-time bootstrap
8
+ import sqlite3
9
+
10
+ conn = sqlite3.connect("repo.db")
11
+ conn.execute("CREATE TABLE t(x INT)")
12
+ conn.execute("INSERT INTO t VALUES (1)")
13
+ conn.execute("SELECT dolt_commit('-A', '-m', 'init')")
14
+ ```
15
+
16
+ `import doltlite` makes libdoltlite the active SQLite engine for the
17
+ current interpreter, so anything that uses the standard `sqlite3` module
18
+ — **including SQLAlchemy with `sqlite:///...` URLs** — transparently
19
+ gains `dolt_commit`, `dolt_branch`, `dolt_merge`, `dolt_log`,
20
+ `dolt_diff_<table>`, and the other Dolt SQL functions and virtual tables.
21
+
22
+ ## Install
23
+
24
+ ```bash
25
+ pip install doltlite
26
+ ```
27
+
28
+ Wheels bundle a precompiled `libdoltlite` for macOS (arm64) and Linux
29
+ (x86_64, aarch64). No system-level setup required.
30
+
31
+ Intel Mac wheels aren't shipped in v0.11.x — GitHub's free-tier
32
+ macos-13 runners queue for hours and block releases. Intel Mac users
33
+ should build libdoltlite locally and use the `DOLTLITE_LIB` path below.
34
+
35
+ For development against a local checkout of doltlite, point
36
+ `DOLTLITE_LIB` at your built library instead:
37
+
38
+ ```bash
39
+ DOLTLITE_LIB=/path/to/libdoltlite.dylib python3 your_script.py
40
+ ```
41
+
42
+ ## Requirements
43
+
44
+ The package piggybacks on Python's stdlib `sqlite3`, which must load
45
+ SQLite as a shared extension at runtime. The following Pythons work:
46
+
47
+ - Distro / system Python (Linux)
48
+ - Homebrew Python (macOS, Linux)
49
+ - pyenv-built Python
50
+ - Conda Python
51
+
52
+ These do **not** work because their `_sqlite3` is statically linked into
53
+ the interpreter:
54
+
55
+ - **python-build-standalone** interpreters — the default for `uv python
56
+ install`, `mise`, and Rye
57
+
58
+ If you use `uv`, target one of the supported Pythons explicitly:
59
+
60
+ ```bash
61
+ uv venv --python /opt/homebrew/bin/python3 # or /usr/bin/python3
62
+ ```
63
+
64
+ ## How the bootstrap works
65
+
66
+ Doltlite is a SQLite drop-in: it implements the same `sqlite3_*` C API
67
+ and adds Dolt-specific functions and virtual tables on top. To use it
68
+ from Python, libdoltlite has to be loaded ahead of the system libsqlite3
69
+ so its symbols win during the dynamic-link symbol-resolution pass that
70
+ Python's `_sqlite3` module triggers when it's first imported.
71
+
72
+ The mechanics differ by platform:
73
+
74
+ ### Linux
75
+
76
+ ELF flat-namespace symbol resolution: if libdoltlite is loaded into the
77
+ process with `RTLD_GLOBAL` **before** `_sqlite3` is loaded, its
78
+ `sqlite3_*` symbols enter the global namespace and the later
79
+ `libsqlite3.so` lookup finds them first.
80
+
81
+ `import doltlite` does `ctypes.CDLL(libdoltlite_path,
82
+ mode=ctypes.RTLD_GLOBAL)` in this case — no re-exec required.
83
+
84
+ If `sqlite3` was already imported before `doltlite`, that boat has
85
+ sailed: we fall back to re-execing the interpreter with `LD_PRELOAD`
86
+ set, so libdoltlite is loaded at process start.
87
+
88
+ ### macOS
89
+
90
+ macOS uses a two-level namespace: `_sqlite3.so` has an `LC_LOAD_DYLIB`
91
+ command bound to a specific `libsqlite3.dylib` path (e.g.
92
+ `/opt/homebrew/opt/sqlite/lib/libsqlite3.dylib`). Plain `dlopen` /
93
+ `ctypes.CDLL` does **not** redirect that resolution, and
94
+ `DYLD_INSERT_LIBRARIES` alone doesn't either — the inserted library has
95
+ to have an `install_name` (`LC_ID_DYLIB`) that matches the path
96
+ `_sqlite3.so` was linked against.
97
+
98
+ `import doltlite` does:
99
+
100
+ 1. Detect that path via `otool -L $(python3 -c 'import _sqlite3;
101
+ print(_sqlite3.__file__)')`.
102
+ 2. Copy libdoltlite to `$TMPDIR/.../libsqlite3.dylib` (cached per
103
+ (lib, mtime, install_name)).
104
+ 3. Rewrite the shim's install_name with `install_name_tool -id <path>`.
105
+ 4. Re-exec the interpreter with `DYLD_INSERT_LIBRARIES=<shim>`.
106
+
107
+ The two-level lookup then accepts the shim because the install_name
108
+ matches.
109
+
110
+ This requires `otool` and `install_name_tool` — install Xcode Command
111
+ Line Tools (`xcode-select --install`) if missing.
112
+
113
+ ## Re-exec caveats
114
+
115
+ When a re-exec is required (macOS, or Linux-after-sqlite3-loaded), the
116
+ bootstrap calls `os.execvpe` with `sys.argv`. That means the
117
+ invocation must name a script file we can replay:
118
+
119
+ - ✅ `python3 my_script.py`
120
+ - ✅ `python3 -m my_package`
121
+ - ❌ `python3 -c "import doltlite; ..."` — code string isn't in argv
122
+ - ❌ Interactive REPL / Jupyter — there's no script to re-exec
123
+
124
+ In the unsupported cases the bootstrap raises `DoltliteLoadError` with
125
+ a clear workaround: set `DYLD_INSERT_LIBRARIES` (macOS) or `LD_PRELOAD`
126
+ (Linux) yourself before starting Python.
127
+
128
+ ## API
129
+
130
+ ```python
131
+ import doltlite
132
+
133
+ # Side-effect import does the bootstrap automatically. Subsequent
134
+ # imports are no-ops thanks to a process-env marker.
135
+
136
+ # If you want to bootstrap explicitly (e.g. inside a function):
137
+ doltlite.bootstrap()
138
+
139
+ # Find where the loaded libdoltlite came from:
140
+ doltlite.libdoltlite_path() # absolute path
141
+ ```
142
+
143
+ ## See also
144
+
145
+ - [doltlite](https://github.com/dolthub/doltlite) — the underlying
146
+ SQLite + Dolt engine
147
+ - [doltlite-sqlalchemy-getting-started](https://github.com/timsehn/doltlite-sqlalchemy-getting-started)
148
+ — end-to-end SQLAlchemy demo (commits, branches, schema change, merge)
149
+
150
+ ## License
151
+
152
+ Apache License 2.0. See [LICENSE](LICENSE).
@@ -0,0 +1,61 @@
1
+ [build-system]
2
+ requires = ["setuptools>=68", "wheel"]
3
+ build-backend = "setuptools.build_meta"
4
+
5
+ [project]
6
+ name = "doltlite"
7
+ # Tracks the bundled libdoltlite release. Bump in lockstep with
8
+ # DOLTLITE_REF in scripts/build-libdoltlite.sh. Loader-only fixes between
9
+ # libdoltlite releases use post-releases: 0.11.2.post1, 0.11.2.post2, etc.
10
+ version = "0.11.2"
11
+ description = "Python loader for doltlite — Dolt version control through SQLite's drop-in API."
12
+ readme = "README.md"
13
+ requires-python = ">=3.9"
14
+ license = { text = "Apache-2.0" }
15
+ authors = [
16
+ { name = "DoltHub", email = "tim@dolthub.com" },
17
+ ]
18
+ keywords = ["dolt", "doltlite", "sqlite", "version-control", "database"]
19
+ classifiers = [
20
+ "Development Status :: 3 - Alpha",
21
+ "Intended Audience :: Developers",
22
+ "License :: OSI Approved :: Apache Software License",
23
+ "Operating System :: MacOS",
24
+ "Operating System :: POSIX :: Linux",
25
+ "Programming Language :: Python :: 3",
26
+ "Programming Language :: Python :: 3 :: Only",
27
+ "Topic :: Database",
28
+ ]
29
+
30
+ [project.urls]
31
+ Homepage = "https://github.com/dolthub/doltlite"
32
+ Source = "https://github.com/dolthub/doltlite-python"
33
+
34
+ [tool.setuptools]
35
+ package-dir = { "" = "src" }
36
+
37
+ [tool.setuptools.packages.find]
38
+ where = ["src"]
39
+
40
+ [tool.setuptools.package-data]
41
+ doltlite = ["_lib/libdoltlite.dylib", "_lib/libdoltlite.so"]
42
+
43
+ # Forces wheels to be tagged as platform-specific so we don't accidentally
44
+ # upload a wheel that bundles a macOS dylib but claims to be pure-python.
45
+ [tool.setuptools.exclude-package-data]
46
+ doltlite = ["_lib/.gitkeep"]
47
+
48
+ # --- cibuildwheel ---
49
+ # Each platform builds libdoltlite from source via before-build, then bundles
50
+ # the resulting library as package data. See .github/workflows/wheels.yml.
51
+ [tool.cibuildwheel]
52
+ build = "cp39-* cp310-* cp311-* cp312-* cp313-*"
53
+ skip = "*-musllinux_* *-win_* *-manylinux_i686"
54
+ build-frontend = "build"
55
+ before-build = "bash scripts/build-libdoltlite.sh"
56
+
57
+ [tool.cibuildwheel.macos]
58
+ archs = ["arm64", "x86_64"]
59
+
60
+ [tool.cibuildwheel.linux]
61
+ archs = ["x86_64", "aarch64"]
@@ -0,0 +1,4 @@
1
+ [egg_info]
2
+ tag_build =
3
+ tag_date = 0
4
+
@@ -0,0 +1,28 @@
1
+ """Shim that produces a `py3-none-<platform>` wheel.
2
+
3
+ The package is pure Python by file count but ships a precompiled
4
+ libdoltlite shared library — so its wheel must be tagged for a specific
5
+ OS + arch, yet libdoltlite is the same .dylib/.so for every CPython
6
+ version. So we want `py3-none-macosx_15_0_arm64` rather than either
7
+ `py3-none-any` (wrong: not portable across OS/arch) or
8
+ `cp314-cp314-macosx_15_0_arm64` (wrong: one wheel per Python version).
9
+
10
+ All other metadata lives in pyproject.toml.
11
+ """
12
+ from setuptools import setup
13
+ from setuptools.dist import Distribution
14
+ from wheel.bdist_wheel import bdist_wheel
15
+
16
+
17
+ class BinaryDistribution(Distribution):
18
+ def has_ext_modules(self):
19
+ return True
20
+
21
+
22
+ class PlatformWheel(bdist_wheel):
23
+ def get_tag(self):
24
+ _impl, _abi, plat = super().get_tag()
25
+ return "py3", "none", plat
26
+
27
+
28
+ setup(distclass=BinaryDistribution, cmdclass={"bdist_wheel": PlatformWheel})
@@ -0,0 +1,23 @@
1
+ """Side-effect import: bootstraps libdoltlite so that the standard
2
+ `sqlite3` module (and anything that uses it, including SQLAlchemy) sees
3
+ Dolt's version control functions and virtual tables.
4
+
5
+ Usage:
6
+
7
+ import doltlite # one-time bootstrap
8
+ import sqlite3
9
+ conn = sqlite3.connect("repo.db")
10
+ conn.execute("SELECT dolt_commit('-A', '-m', 'init')")
11
+
12
+ On macOS, and on Linux when `sqlite3` was already imported, the
13
+ interpreter re-execs itself once with the right loader environment
14
+ (`DYLD_INSERT_LIBRARIES` / `LD_PRELOAD`). On a clean Linux process the
15
+ preload happens in-process via `ctypes.CDLL(..., RTLD_GLOBAL)` and no
16
+ re-exec occurs.
17
+ """
18
+ from ._loader import bootstrap, libdoltlite_path
19
+
20
+ bootstrap()
21
+
22
+ __all__ = ["bootstrap", "libdoltlite_path"]
23
+ __version__ = "0.11.2"
@@ -0,0 +1,207 @@
1
+ """Cross-platform loader for libdoltlite.
2
+
3
+ The goal: make `import doltlite` enough for any subsequent `sqlite3` /
4
+ SQLAlchemy code to talk to doltlite. The challenge is that Python's
5
+ `sqlite3` module is a thin wrapper over libsqlite3; we have to make
6
+ libdoltlite's `sqlite3_*` symbols win over the system libsqlite3's.
7
+
8
+ Strategy by platform:
9
+
10
+ - **Linux** uses flat-namespace symbol resolution. If we `dlopen(lib,
11
+ RTLD_GLOBAL)` *before* `_sqlite3` is loaded, libdoltlite's symbols
12
+ end up in the global namespace and the later dynamic resolution from
13
+ libsqlite3 picks them up. If `sqlite3` was already imported, we have
14
+ to re-exec the interpreter with `LD_PRELOAD` set.
15
+
16
+ - **macOS** uses a two-level namespace: `_sqlite3.so` is linked with an
17
+ absolute `LC_LOAD_DYLIB` reference to a specific `libsqlite3.dylib`
18
+ (e.g. `/opt/homebrew/opt/sqlite/lib/libsqlite3.dylib`). Plain
19
+ `ctypes.CDLL` does not redirect that lookup. The only mechanism that
20
+ does is a `DYLD_INSERT_LIBRARIES` library whose own `install_name`
21
+ (`LC_ID_DYLIB`) matches the path `_sqlite3.so` was linked against.
22
+ We build that shim from libdoltlite via `install_name_tool -id`,
23
+ then re-exec with `DYLD_INSERT_LIBRARIES` pointing at the shim.
24
+ """
25
+ from __future__ import annotations
26
+
27
+ import ctypes
28
+ import os
29
+ import shutil
30
+ import subprocess
31
+ import sys
32
+ import tempfile
33
+ from pathlib import Path
34
+ from typing import Optional
35
+
36
+
37
+ _BOOTSTRAP_MARKER = "_DOLTLITE_BOOTSTRAPPED"
38
+ _PKG_LIB_DIR = Path(__file__).resolve().parent / "_lib"
39
+
40
+
41
+ class DoltliteLoadError(RuntimeError):
42
+ """Raised when libdoltlite cannot be located or loaded."""
43
+
44
+
45
+ def libdoltlite_path() -> str:
46
+ """Return the absolute path of the libdoltlite that will be loaded.
47
+
48
+ Resolution order:
49
+ 1. `DOLTLITE_LIB` environment variable (absolute path)
50
+ 2. Library bundled inside the installed `doltlite` package
51
+ (`<pkg>/_lib/libdoltlite.{dylib,so}`)
52
+
53
+ Raises `DoltliteLoadError` if neither is available.
54
+ """
55
+ env = os.environ.get("DOLTLITE_LIB")
56
+ if env:
57
+ if not os.path.exists(env):
58
+ raise DoltliteLoadError(
59
+ f"DOLTLITE_LIB points to a missing file: {env}"
60
+ )
61
+ return env
62
+
63
+ name = "libdoltlite.dylib" if sys.platform == "darwin" else "libdoltlite.so"
64
+ bundled = _PKG_LIB_DIR / name
65
+ if bundled.is_file():
66
+ return str(bundled)
67
+
68
+ raise DoltliteLoadError(
69
+ f"libdoltlite not found.\n"
70
+ f" Looked for bundled: {bundled}\n"
71
+ f" DOLTLITE_LIB env var: <unset>\n"
72
+ f"Either install a wheel that bundles libdoltlite, or set "
73
+ f"DOLTLITE_LIB to the absolute path of {name}."
74
+ )
75
+
76
+
77
+ def _is_bootstrapped() -> bool:
78
+ return os.environ.get(_BOOTSTRAP_MARKER) == "1"
79
+
80
+
81
+ def _sqlite3_already_imported() -> bool:
82
+ return "sqlite3" in sys.modules or "_sqlite3" in sys.modules
83
+
84
+
85
+ def bootstrap() -> None:
86
+ """Make libdoltlite the active SQLite engine for this interpreter.
87
+
88
+ Idempotent: subsequent calls (and re-entrant calls after a re-exec)
89
+ are no-ops. Raises `DoltliteLoadError` if the library can't be found
90
+ or if a re-exec is needed but `sys.argv` does not name a script we
91
+ can replay (interactive shell, `python -c …`, REPL, notebook).
92
+ """
93
+ if _is_bootstrapped():
94
+ return
95
+
96
+ lib = libdoltlite_path()
97
+
98
+ if sys.platform != "darwin" and not _sqlite3_already_imported():
99
+ ctypes.CDLL(lib, mode=ctypes.RTLD_GLOBAL)
100
+ os.environ[_BOOTSTRAP_MARKER] = "1"
101
+ return
102
+
103
+ _require_replayable_argv()
104
+
105
+ env = dict(os.environ)
106
+ env[_BOOTSTRAP_MARKER] = "1"
107
+
108
+ if sys.platform == "darwin":
109
+ env["DYLD_INSERT_LIBRARIES"] = _build_macos_shim(lib)
110
+ else:
111
+ existing = env.get("LD_PRELOAD", "").strip()
112
+ env["LD_PRELOAD"] = f"{lib} {existing}".strip()
113
+
114
+ os.execvpe(sys.executable, [sys.executable, *sys.argv], env)
115
+
116
+
117
+ def _require_replayable_argv() -> None:
118
+ """Bail out clearly if we can't safely re-exec the current process.
119
+
120
+ `python -c "..."` and an interactive REPL have no script path in
121
+ `sys.argv` that we can re-exec — the user's code is consumed before
122
+ `sys.argv` is built. Jupyter / IPython kernels are similar.
123
+ """
124
+ arg0 = sys.argv[0] if sys.argv else ""
125
+ if arg0 and arg0 != "-c" and os.path.exists(arg0):
126
+ return
127
+
128
+ if sys.platform == "darwin":
129
+ envvar, name = "DYLD_INSERT_LIBRARIES", "libdoltlite.dylib"
130
+ shim_hint = (
131
+ " (On macOS the inserted library must be named libsqlite3.dylib "
132
+ "with its install_name set to the path your Python's _sqlite3 "
133
+ "was linked against. See doltlite._loader for the recipe.)"
134
+ )
135
+ else:
136
+ envvar, name = "LD_PRELOAD", "libdoltlite.so"
137
+ shim_hint = ""
138
+
139
+ raise DoltliteLoadError(
140
+ "doltlite bootstrap requires re-execing the interpreter, but the "
141
+ f"current invocation (sys.argv[0]={arg0!r}) does not name a "
142
+ "script that can be replayed (interactive shell, `python -c …`, "
143
+ "Jupyter, etc.).\n"
144
+ f"Workaround: set {envvar} yourself before starting Python so "
145
+ f"{name} is loaded at process start.\n"
146
+ + shim_hint
147
+ )
148
+
149
+
150
+ def _detect_sqlite3_install_name() -> str:
151
+ """On macOS, return the libsqlite3 path that the active interpreter's
152
+ `_sqlite3` extension was linked against."""
153
+ import _sqlite3
154
+
155
+ try:
156
+ out = subprocess.run(
157
+ ["otool", "-L", _sqlite3.__file__],
158
+ check=True, capture_output=True, text=True,
159
+ ).stdout
160
+ except FileNotFoundError as e:
161
+ raise DoltliteLoadError(
162
+ "otool is required on macOS to detect the active "
163
+ "libsqlite3 path. Install Xcode Command Line Tools."
164
+ ) from e
165
+
166
+ for line in out.splitlines():
167
+ token = line.strip().split(" ", 1)[0]
168
+ if "/libsqlite3" in token and token.endswith(".dylib"):
169
+ return token
170
+
171
+ raise DoltliteLoadError(
172
+ "Could not find a libsqlite3 dependency in "
173
+ f"{_sqlite3.__file__}. Output of otool -L:\n{out}"
174
+ )
175
+
176
+
177
+ def _build_macos_shim(lib: str) -> str:
178
+ """Produce a libsqlite3.dylib shim whose install_name matches the path
179
+ `_sqlite3.so` was linked against. The shim is cached under $TMPDIR
180
+ keyed by (lib path, mtime, target install_name)."""
181
+ install_name = _detect_sqlite3_install_name()
182
+ src_stat = os.stat(lib)
183
+ key = f"{abs(hash((lib, src_stat.st_mtime_ns, install_name))):016x}"
184
+ cache_dir = Path(tempfile.gettempdir()) / f"doltlite-shim-{key}"
185
+ shim = cache_dir / "libsqlite3.dylib"
186
+
187
+ if shim.exists():
188
+ return str(shim)
189
+
190
+ cache_dir.mkdir(parents=True, exist_ok=True)
191
+ shutil.copyfile(lib, shim)
192
+ try:
193
+ subprocess.run(
194
+ ["install_name_tool", "-id", install_name, str(shim)],
195
+ check=True, capture_output=True, text=True,
196
+ )
197
+ except FileNotFoundError as e:
198
+ raise DoltliteLoadError(
199
+ "install_name_tool is required on macOS. Install Xcode "
200
+ "Command Line Tools (`xcode-select --install`)."
201
+ ) from e
202
+ except subprocess.CalledProcessError as e:
203
+ raise DoltliteLoadError(
204
+ f"install_name_tool failed for {shim}: {e.stderr}"
205
+ ) from e
206
+
207
+ return str(shim)
@@ -0,0 +1,174 @@
1
+ Metadata-Version: 2.4
2
+ Name: doltlite
3
+ Version: 0.11.2
4
+ Summary: Python loader for doltlite — Dolt version control through SQLite's drop-in API.
5
+ Author-email: DoltHub <tim@dolthub.com>
6
+ License: Apache-2.0
7
+ Project-URL: Homepage, https://github.com/dolthub/doltlite
8
+ Project-URL: Source, https://github.com/dolthub/doltlite-python
9
+ Keywords: dolt,doltlite,sqlite,version-control,database
10
+ Classifier: Development Status :: 3 - Alpha
11
+ Classifier: Intended Audience :: Developers
12
+ Classifier: License :: OSI Approved :: Apache Software License
13
+ Classifier: Operating System :: MacOS
14
+ Classifier: Operating System :: POSIX :: Linux
15
+ Classifier: Programming Language :: Python :: 3
16
+ Classifier: Programming Language :: Python :: 3 :: Only
17
+ Classifier: Topic :: Database
18
+ Requires-Python: >=3.9
19
+ Description-Content-Type: text/markdown
20
+ License-File: LICENSE
21
+ Dynamic: license-file
22
+
23
+ # doltlite (Python)
24
+
25
+ A Python loader for [doltlite](https://github.com/dolthub/doltlite) —
26
+ Dolt's version control through SQLite's drop-in API.
27
+
28
+ ```python
29
+ import doltlite # one-time bootstrap
30
+ import sqlite3
31
+
32
+ conn = sqlite3.connect("repo.db")
33
+ conn.execute("CREATE TABLE t(x INT)")
34
+ conn.execute("INSERT INTO t VALUES (1)")
35
+ conn.execute("SELECT dolt_commit('-A', '-m', 'init')")
36
+ ```
37
+
38
+ `import doltlite` makes libdoltlite the active SQLite engine for the
39
+ current interpreter, so anything that uses the standard `sqlite3` module
40
+ — **including SQLAlchemy with `sqlite:///...` URLs** — transparently
41
+ gains `dolt_commit`, `dolt_branch`, `dolt_merge`, `dolt_log`,
42
+ `dolt_diff_<table>`, and the other Dolt SQL functions and virtual tables.
43
+
44
+ ## Install
45
+
46
+ ```bash
47
+ pip install doltlite
48
+ ```
49
+
50
+ Wheels bundle a precompiled `libdoltlite` for macOS (arm64) and Linux
51
+ (x86_64, aarch64). No system-level setup required.
52
+
53
+ Intel Mac wheels aren't shipped in v0.11.x — GitHub's free-tier
54
+ macos-13 runners queue for hours and block releases. Intel Mac users
55
+ should build libdoltlite locally and use the `DOLTLITE_LIB` path below.
56
+
57
+ For development against a local checkout of doltlite, point
58
+ `DOLTLITE_LIB` at your built library instead:
59
+
60
+ ```bash
61
+ DOLTLITE_LIB=/path/to/libdoltlite.dylib python3 your_script.py
62
+ ```
63
+
64
+ ## Requirements
65
+
66
+ The package piggybacks on Python's stdlib `sqlite3`, which must load
67
+ SQLite as a shared extension at runtime. The following Pythons work:
68
+
69
+ - Distro / system Python (Linux)
70
+ - Homebrew Python (macOS, Linux)
71
+ - pyenv-built Python
72
+ - Conda Python
73
+
74
+ These do **not** work because their `_sqlite3` is statically linked into
75
+ the interpreter:
76
+
77
+ - **python-build-standalone** interpreters — the default for `uv python
78
+ install`, `mise`, and Rye
79
+
80
+ If you use `uv`, target one of the supported Pythons explicitly:
81
+
82
+ ```bash
83
+ uv venv --python /opt/homebrew/bin/python3 # or /usr/bin/python3
84
+ ```
85
+
86
+ ## How the bootstrap works
87
+
88
+ Doltlite is a SQLite drop-in: it implements the same `sqlite3_*` C API
89
+ and adds Dolt-specific functions and virtual tables on top. To use it
90
+ from Python, libdoltlite has to be loaded ahead of the system libsqlite3
91
+ so its symbols win during the dynamic-link symbol-resolution pass that
92
+ Python's `_sqlite3` module triggers when it's first imported.
93
+
94
+ The mechanics differ by platform:
95
+
96
+ ### Linux
97
+
98
+ ELF flat-namespace symbol resolution: if libdoltlite is loaded into the
99
+ process with `RTLD_GLOBAL` **before** `_sqlite3` is loaded, its
100
+ `sqlite3_*` symbols enter the global namespace and the later
101
+ `libsqlite3.so` lookup finds them first.
102
+
103
+ `import doltlite` does `ctypes.CDLL(libdoltlite_path,
104
+ mode=ctypes.RTLD_GLOBAL)` in this case — no re-exec required.
105
+
106
+ If `sqlite3` was already imported before `doltlite`, that boat has
107
+ sailed: we fall back to re-execing the interpreter with `LD_PRELOAD`
108
+ set, so libdoltlite is loaded at process start.
109
+
110
+ ### macOS
111
+
112
+ macOS uses a two-level namespace: `_sqlite3.so` has an `LC_LOAD_DYLIB`
113
+ command bound to a specific `libsqlite3.dylib` path (e.g.
114
+ `/opt/homebrew/opt/sqlite/lib/libsqlite3.dylib`). Plain `dlopen` /
115
+ `ctypes.CDLL` does **not** redirect that resolution, and
116
+ `DYLD_INSERT_LIBRARIES` alone doesn't either — the inserted library has
117
+ to have an `install_name` (`LC_ID_DYLIB`) that matches the path
118
+ `_sqlite3.so` was linked against.
119
+
120
+ `import doltlite` does:
121
+
122
+ 1. Detect that path via `otool -L $(python3 -c 'import _sqlite3;
123
+ print(_sqlite3.__file__)')`.
124
+ 2. Copy libdoltlite to `$TMPDIR/.../libsqlite3.dylib` (cached per
125
+ (lib, mtime, install_name)).
126
+ 3. Rewrite the shim's install_name with `install_name_tool -id <path>`.
127
+ 4. Re-exec the interpreter with `DYLD_INSERT_LIBRARIES=<shim>`.
128
+
129
+ The two-level lookup then accepts the shim because the install_name
130
+ matches.
131
+
132
+ This requires `otool` and `install_name_tool` — install Xcode Command
133
+ Line Tools (`xcode-select --install`) if missing.
134
+
135
+ ## Re-exec caveats
136
+
137
+ When a re-exec is required (macOS, or Linux-after-sqlite3-loaded), the
138
+ bootstrap calls `os.execvpe` with `sys.argv`. That means the
139
+ invocation must name a script file we can replay:
140
+
141
+ - ✅ `python3 my_script.py`
142
+ - ✅ `python3 -m my_package`
143
+ - ❌ `python3 -c "import doltlite; ..."` — code string isn't in argv
144
+ - ❌ Interactive REPL / Jupyter — there's no script to re-exec
145
+
146
+ In the unsupported cases the bootstrap raises `DoltliteLoadError` with
147
+ a clear workaround: set `DYLD_INSERT_LIBRARIES` (macOS) or `LD_PRELOAD`
148
+ (Linux) yourself before starting Python.
149
+
150
+ ## API
151
+
152
+ ```python
153
+ import doltlite
154
+
155
+ # Side-effect import does the bootstrap automatically. Subsequent
156
+ # imports are no-ops thanks to a process-env marker.
157
+
158
+ # If you want to bootstrap explicitly (e.g. inside a function):
159
+ doltlite.bootstrap()
160
+
161
+ # Find where the loaded libdoltlite came from:
162
+ doltlite.libdoltlite_path() # absolute path
163
+ ```
164
+
165
+ ## See also
166
+
167
+ - [doltlite](https://github.com/dolthub/doltlite) — the underlying
168
+ SQLite + Dolt engine
169
+ - [doltlite-sqlalchemy-getting-started](https://github.com/timsehn/doltlite-sqlalchemy-getting-started)
170
+ — end-to-end SQLAlchemy demo (commits, branches, schema change, merge)
171
+
172
+ ## License
173
+
174
+ Apache License 2.0. See [LICENSE](LICENSE).
@@ -0,0 +1,10 @@
1
+ LICENSE
2
+ README.md
3
+ pyproject.toml
4
+ setup.py
5
+ src/doltlite/__init__.py
6
+ src/doltlite/_loader.py
7
+ src/doltlite.egg-info/PKG-INFO
8
+ src/doltlite.egg-info/SOURCES.txt
9
+ src/doltlite.egg-info/dependency_links.txt
10
+ src/doltlite.egg-info/top_level.txt
@@ -0,0 +1 @@
1
+ doltlite