@photostructure/sqlite 0.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +43 -0
- package/LICENSE +21 -0
- package/README.md +522 -0
- package/SECURITY.md +114 -0
- package/binding.gyp +94 -0
- package/dist/index.cjs +134 -0
- package/dist/index.cjs.map +1 -0
- package/dist/index.d.cts +408 -0
- package/dist/index.d.mts +408 -0
- package/dist/index.d.ts +408 -0
- package/dist/index.mjs +103 -0
- package/dist/index.mjs.map +1 -0
- package/package.json +144 -0
- package/prebuilds/darwin-arm64/@photostructure+sqlite.glibc.node +0 -0
- package/prebuilds/linux-arm64/@photostructure+sqlite.glibc.node +0 -0
- package/prebuilds/linux-arm64/@photostructure+sqlite.musl.node +0 -0
- package/prebuilds/linux-x64/@photostructure+sqlite.glibc.node +0 -0
- package/prebuilds/linux-x64/@photostructure+sqlite.musl.node +0 -0
- package/prebuilds/win32-x64/@photostructure+sqlite.glibc.node +0 -0
- package/scripts/post-build.mjs +21 -0
- package/scripts/prebuild-linux-glibc.sh +108 -0
- package/src/aggregate_function.cpp +417 -0
- package/src/aggregate_function.h +116 -0
- package/src/binding.cpp +160 -0
- package/src/dirname.ts +13 -0
- package/src/index.ts +465 -0
- package/src/shims/base_object-inl.h +8 -0
- package/src/shims/base_object.h +50 -0
- package/src/shims/debug_utils-inl.h +23 -0
- package/src/shims/env-inl.h +19 -0
- package/src/shims/memory_tracker-inl.h +17 -0
- package/src/shims/napi_extensions.h +73 -0
- package/src/shims/node.h +16 -0
- package/src/shims/node_errors.h +66 -0
- package/src/shims/node_mem-inl.h +8 -0
- package/src/shims/node_mem.h +31 -0
- package/src/shims/node_url.h +23 -0
- package/src/shims/promise_resolver.h +31 -0
- package/src/shims/util-inl.h +18 -0
- package/src/shims/util.h +101 -0
- package/src/sqlite_impl.cpp +2440 -0
- package/src/sqlite_impl.h +314 -0
- package/src/stack_path.ts +64 -0
- package/src/types/node-gyp-build.d.ts +4 -0
- package/src/upstream/node_sqlite.cc +2706 -0
- package/src/upstream/node_sqlite.h +234 -0
- package/src/upstream/sqlite.gyp +38 -0
- package/src/upstream/sqlite.js +19 -0
- package/src/upstream/sqlite3.c +262809 -0
- package/src/upstream/sqlite3.h +13773 -0
- package/src/upstream/sqlite3ext.h +723 -0
- package/src/user_function.cpp +225 -0
- package/src/user_function.h +40 -0
package/CHANGELOG.md
ADDED
|
@@ -0,0 +1,43 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to this project will be documented in this file.
|
|
4
|
+
|
|
5
|
+
## [0.1.0] - 2025-01-06
|
|
6
|
+
|
|
7
|
+
### Added
|
|
8
|
+
|
|
9
|
+
- Initial release of `@photostructure/sqlite` - standalone SQLite for Node.js 20+
|
|
10
|
+
- Full compatibility with Node.js built-in SQLite module API
|
|
11
|
+
- Core SQLite operations with `DatabaseSync` and `StatementSync` classes
|
|
12
|
+
- User-defined scalar and aggregate functions with full window function support
|
|
13
|
+
- Database backup and restoration capabilities
|
|
14
|
+
- SQLite sessions and changesets for change tracking
|
|
15
|
+
- Extension loading support with automatic platform-specific file resolution
|
|
16
|
+
- TypeScript definitions with complete type coverage
|
|
17
|
+
- Cross-platform prebuilt binaries for Windows, macOS, and Linux (x64, ARM64)
|
|
18
|
+
- Comprehensive test suite with 89+ tests covering all functionality
|
|
19
|
+
- Memory safety validation with Valgrind and sanitizers
|
|
20
|
+
- Performance benchmarking suite comparing to better-sqlite3
|
|
21
|
+
- Automated synchronization from Node.js upstream SQLite implementation
|
|
22
|
+
- CI/CD pipeline with security scanning and multi-platform builds
|
|
23
|
+
|
|
24
|
+
### Features
|
|
25
|
+
|
|
26
|
+
- **Synchronous API**: Fast, blocking database operations ideal for scripts and tools
|
|
27
|
+
- **Parameter binding**: Support for all SQLite data types including BigInt
|
|
28
|
+
- **Error handling**: Detailed error messages with SQLite error codes
|
|
29
|
+
- **Resource limits**: Control memory usage and query complexity
|
|
30
|
+
- **Safe integer handling**: JavaScript-safe integer conversion with overflow detection
|
|
31
|
+
- **Multi-process support**: Safe concurrent access from multiple Node.js processes
|
|
32
|
+
- **Worker thread support**: Full functionality in worker threads
|
|
33
|
+
- **Strict tables**: Support for SQLite's strict table mode
|
|
34
|
+
- **Double-quoted strings**: Configurable SQL syntax compatibility
|
|
35
|
+
|
|
36
|
+
### Platform Support
|
|
37
|
+
|
|
38
|
+
- Node.js 20.0.0 and later
|
|
39
|
+
- Windows (x64, ARM64)
|
|
40
|
+
- macOS (x64, ARM64)
|
|
41
|
+
- Linux (x64, ARM64), (glibc 2.28+, musl)
|
|
42
|
+
|
|
43
|
+
[0.1.0]: https://github.com/PhotoStructure/node-sqlite/releases/tag/v0.1.0
|
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2025 PhotoStructure Inc.
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
ADDED
|
@@ -0,0 +1,522 @@
|
|
|
1
|
+

|
|
2
|
+
|
|
3
|
+
## 🚀 Drop-in Replacement for node:sqlite
|
|
4
|
+
|
|
5
|
+
This package provides **100% API compatibility** with Node.js v24 built-in SQLite module (`node:sqlite`). You can seamlessly switch between this package and the built-in module without changing any code.
|
|
6
|
+
|
|
7
|
+
```javascript
|
|
8
|
+
// Using Node.js built-in SQLite (requires Node.js 22.5.0+ and --experimental-sqlite flag)
|
|
9
|
+
const { DatabaseSync } = require("node:sqlite");
|
|
10
|
+
|
|
11
|
+
// Using @photostructure/sqlite (works on Node.js 20+ without any flags)
|
|
12
|
+
const { DatabaseSync } = require("@photostructure/sqlite");
|
|
13
|
+
|
|
14
|
+
// The API is identical - no code changes needed!
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
## Overview
|
|
18
|
+
|
|
19
|
+
Node.js has an [experimental built-in SQLite module](https://nodejs.org/docs/latest/api/sqlite.html) that provides synchronous database operations with excellent performance. However, it's only available in the newest Node.js versions, and requires the `--experimental-sqlite` flag.
|
|
20
|
+
|
|
21
|
+
This package extracts that implementation into a standalone library that:
|
|
22
|
+
|
|
23
|
+
- **Works everywhere**: Compatible with Node.js 20+ without experimental flags
|
|
24
|
+
- **Drop-in replacement**: 100% API compatible with `node:sqlite` - no code changes needed
|
|
25
|
+
- **Full-featured**: Includes all SQLite extensions (FTS, JSON, math functions, etc.)
|
|
26
|
+
- **High performance**: Direct SQLite C library integration with minimal overhead
|
|
27
|
+
- **Type-safe**: Complete TypeScript definitions matching Node.js exactly
|
|
28
|
+
- **Worker thread support**: Full support for Node.js worker threads with proper isolation
|
|
29
|
+
- **Future-proof**: When `node:sqlite` becomes stable, switching back requires zero code changes
|
|
30
|
+
|
|
31
|
+
## Installation
|
|
32
|
+
|
|
33
|
+
```bash
|
|
34
|
+
npm install @photostructure/sqlite
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
## Quick Start
|
|
38
|
+
|
|
39
|
+
### As a Drop-in Replacement
|
|
40
|
+
|
|
41
|
+
```javascript
|
|
42
|
+
// If you have code using node:sqlite:
|
|
43
|
+
const { DatabaseSync } = require("node:sqlite");
|
|
44
|
+
|
|
45
|
+
// Simply replace with:
|
|
46
|
+
const { DatabaseSync } = require("@photostructure/sqlite");
|
|
47
|
+
// That's it! No other changes needed.
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
### Basic Example
|
|
51
|
+
|
|
52
|
+
```typescript
|
|
53
|
+
import { DatabaseSync } from "@photostructure/sqlite";
|
|
54
|
+
|
|
55
|
+
// Create an in-memory database
|
|
56
|
+
const db = new DatabaseSync(":memory:");
|
|
57
|
+
|
|
58
|
+
// Create a table
|
|
59
|
+
db.exec(`
|
|
60
|
+
CREATE TABLE users (
|
|
61
|
+
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
62
|
+
name TEXT NOT NULL,
|
|
63
|
+
email TEXT UNIQUE
|
|
64
|
+
)
|
|
65
|
+
`);
|
|
66
|
+
|
|
67
|
+
// Insert data
|
|
68
|
+
const insertStmt = db.prepare("INSERT INTO users (name, email) VALUES (?, ?)");
|
|
69
|
+
const result = insertStmt.run("Alice Johnson", "alice@example.com");
|
|
70
|
+
console.log("Inserted user with ID:", result.lastInsertRowid);
|
|
71
|
+
|
|
72
|
+
// Query data
|
|
73
|
+
const selectStmt = db.prepare("SELECT * FROM users WHERE id = ?");
|
|
74
|
+
const user = selectStmt.get(result.lastInsertRowid);
|
|
75
|
+
console.log("User:", user);
|
|
76
|
+
|
|
77
|
+
// Clean up
|
|
78
|
+
db.close();
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
## API Reference
|
|
82
|
+
|
|
83
|
+
### Database Configuration Options
|
|
84
|
+
|
|
85
|
+
### Custom Functions
|
|
86
|
+
|
|
87
|
+
```typescript
|
|
88
|
+
// Register a simple custom SQL function
|
|
89
|
+
db.function("multiply", (a, b) => a * b);
|
|
90
|
+
|
|
91
|
+
// With options
|
|
92
|
+
db.function(
|
|
93
|
+
"hash",
|
|
94
|
+
{
|
|
95
|
+
deterministic: true, // Same inputs always produce same output
|
|
96
|
+
directOnly: true, // Cannot be called from triggers/views
|
|
97
|
+
},
|
|
98
|
+
(value) => {
|
|
99
|
+
return crypto.createHash("sha256").update(String(value)).digest("hex");
|
|
100
|
+
},
|
|
101
|
+
);
|
|
102
|
+
|
|
103
|
+
// Aggregate function
|
|
104
|
+
db.aggregate("custom_sum", {
|
|
105
|
+
start: 0,
|
|
106
|
+
step: (sum, value) => sum + value,
|
|
107
|
+
result: (sum) => sum,
|
|
108
|
+
});
|
|
109
|
+
|
|
110
|
+
// Use in SQL
|
|
111
|
+
const result = db
|
|
112
|
+
.prepare("SELECT custom_sum(price) as total FROM products")
|
|
113
|
+
.get();
|
|
114
|
+
console.log(result.total);
|
|
115
|
+
```
|
|
116
|
+
|
|
117
|
+
### Database Backup
|
|
118
|
+
|
|
119
|
+
```typescript
|
|
120
|
+
// Simple backup
|
|
121
|
+
await db.backup("./backup.db");
|
|
122
|
+
|
|
123
|
+
// Backup with progress monitoring
|
|
124
|
+
await db.backup("./backup.db", {
|
|
125
|
+
rate: 10, // Copy 10 pages per iteration
|
|
126
|
+
progress: ({ totalPages, remainingPages }) => {
|
|
127
|
+
const percent = (
|
|
128
|
+
((totalPages - remainingPages) / totalPages) *
|
|
129
|
+
100
|
|
130
|
+
).toFixed(1);
|
|
131
|
+
console.log(`Backup progress: ${percent}%`);
|
|
132
|
+
},
|
|
133
|
+
});
|
|
134
|
+
|
|
135
|
+
// Backup specific attached database
|
|
136
|
+
db.exec("ATTACH DATABASE 'other.db' AS other");
|
|
137
|
+
await db.backup("./other-backup.db", {
|
|
138
|
+
source: "other", // Backup the attached database instead of main
|
|
139
|
+
});
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
### Database Restoration
|
|
143
|
+
|
|
144
|
+
```typescript
|
|
145
|
+
// Restore from a backup file
|
|
146
|
+
import * as fs from "fs";
|
|
147
|
+
|
|
148
|
+
// Close the current database
|
|
149
|
+
db.close();
|
|
150
|
+
|
|
151
|
+
// Copy the backup file over the current database file
|
|
152
|
+
fs.copyFileSync("./backup.db", "./mydata.db");
|
|
153
|
+
|
|
154
|
+
// Reopen the database with the restored data
|
|
155
|
+
const restoredDb = new DatabaseSync("./mydata.db");
|
|
156
|
+
|
|
157
|
+
// Verify restoration
|
|
158
|
+
const count = restoredDb.prepare("SELECT COUNT(*) as count FROM users").get();
|
|
159
|
+
console.log(`Restored database has ${count.count} users`);
|
|
160
|
+
```
|
|
161
|
+
|
|
162
|
+
### Worker Thread Support
|
|
163
|
+
|
|
164
|
+
This package has full support for Node.js worker threads. Each worker thread gets its own isolated SQLite environment.
|
|
165
|
+
|
|
166
|
+
```javascript
|
|
167
|
+
// main.js
|
|
168
|
+
const { Worker } = require("worker_threads");
|
|
169
|
+
|
|
170
|
+
// Spawn multiple workers to handle database operations
|
|
171
|
+
const worker1 = new Worker("./db-worker.js");
|
|
172
|
+
const worker2 = new Worker("./db-worker.js");
|
|
173
|
+
|
|
174
|
+
// Send queries to workers
|
|
175
|
+
worker1.postMessage({
|
|
176
|
+
sql: "SELECT * FROM users WHERE active = ?",
|
|
177
|
+
params: [true],
|
|
178
|
+
});
|
|
179
|
+
|
|
180
|
+
// db-worker.js
|
|
181
|
+
const { parentPort } = require("worker_threads");
|
|
182
|
+
const { DatabaseSync } = require("@photostructure/sqlite");
|
|
183
|
+
|
|
184
|
+
// Each worker creates its own database connection
|
|
185
|
+
const db = new DatabaseSync("./app.db");
|
|
186
|
+
|
|
187
|
+
parentPort.on("message", ({ sql, params }) => {
|
|
188
|
+
try {
|
|
189
|
+
const stmt = db.prepare(sql);
|
|
190
|
+
const results = stmt.all(...params);
|
|
191
|
+
stmt.finalize();
|
|
192
|
+
parentPort.postMessage({ success: true, results });
|
|
193
|
+
} catch (error) {
|
|
194
|
+
parentPort.postMessage({ success: false, error: error.message });
|
|
195
|
+
}
|
|
196
|
+
});
|
|
197
|
+
```
|
|
198
|
+
|
|
199
|
+
Key points:
|
|
200
|
+
|
|
201
|
+
- Each worker thread must create its own `DatabaseSync` instance
|
|
202
|
+
- Database connections cannot be shared between threads
|
|
203
|
+
- SQLite's built-in thread safety (multi-thread mode) ensures data integrity
|
|
204
|
+
- No special initialization required - just use normally in each worker
|
|
205
|
+
|
|
206
|
+
### Extension Loading
|
|
207
|
+
|
|
208
|
+
SQLite extensions can be loaded to add custom functionality. Extension loading requires explicit permission for security.
|
|
209
|
+
|
|
210
|
+
```javascript
|
|
211
|
+
// Enable extension loading at database creation
|
|
212
|
+
const db = new DatabaseSync("./mydb.sqlite", {
|
|
213
|
+
allowExtension: true,
|
|
214
|
+
});
|
|
215
|
+
|
|
216
|
+
// Enable extension loading (required before loading)
|
|
217
|
+
db.enableLoadExtension(true);
|
|
218
|
+
|
|
219
|
+
// Load an extension
|
|
220
|
+
db.loadExtension("./extensions/vector.so");
|
|
221
|
+
|
|
222
|
+
// Optionally specify an entry point
|
|
223
|
+
db.loadExtension("./extensions/custom.so", "sqlite3_custom_init");
|
|
224
|
+
|
|
225
|
+
// Disable extension loading when done for security
|
|
226
|
+
db.enableLoadExtension(false);
|
|
227
|
+
```
|
|
228
|
+
|
|
229
|
+
### Session-based Change Tracking
|
|
230
|
+
|
|
231
|
+
SQLite's session extension allows you to record changes and apply them to other databases - perfect for synchronization, replication, or undo/redo functionality. This feature is available in both `node:sqlite` and `@photostructure/sqlite`, but not in better-sqlite3.
|
|
232
|
+
|
|
233
|
+
```typescript
|
|
234
|
+
// Create a session to track changes
|
|
235
|
+
const session = db.createSession({ table: "users" });
|
|
236
|
+
|
|
237
|
+
// Make some changes
|
|
238
|
+
db.prepare("UPDATE users SET name = ? WHERE id = ?").run("Alice Smith", 1);
|
|
239
|
+
db.prepare("INSERT INTO users (name, email) VALUES (?, ?)").run(
|
|
240
|
+
"Bob",
|
|
241
|
+
"bob@example.com",
|
|
242
|
+
);
|
|
243
|
+
|
|
244
|
+
// Get the changes as a changeset
|
|
245
|
+
const changeset = session.changeset();
|
|
246
|
+
session.close();
|
|
247
|
+
|
|
248
|
+
// Apply changes to another database
|
|
249
|
+
const otherDb = new DatabaseSync("./replica.db");
|
|
250
|
+
const applied = otherDb.applyChangeset(changeset, {
|
|
251
|
+
onConflict: (conflict) => {
|
|
252
|
+
console.log(`Conflict on table ${conflict.table}`);
|
|
253
|
+
return constants.SQLITE_CHANGESET_REPLACE; // Resolve by replacing
|
|
254
|
+
},
|
|
255
|
+
});
|
|
256
|
+
```
|
|
257
|
+
|
|
258
|
+
### Parameter Binding
|
|
259
|
+
|
|
260
|
+
```typescript
|
|
261
|
+
const stmt = db.prepare("SELECT * FROM users WHERE name = ? AND age > ?");
|
|
262
|
+
|
|
263
|
+
// Positional parameters
|
|
264
|
+
const users1 = stmt.all("Alice", 25);
|
|
265
|
+
|
|
266
|
+
// Named parameters (with object)
|
|
267
|
+
const stmt2 = db.prepare(
|
|
268
|
+
"SELECT * FROM users WHERE name = $name AND age > $age",
|
|
269
|
+
);
|
|
270
|
+
const users2 = stmt2.all({ name: "Alice", age: 25 });
|
|
271
|
+
```
|
|
272
|
+
|
|
273
|
+
### Using with TypeScript
|
|
274
|
+
|
|
275
|
+
```typescript
|
|
276
|
+
interface User {
|
|
277
|
+
id: number;
|
|
278
|
+
name: string;
|
|
279
|
+
email: string;
|
|
280
|
+
created_at: string;
|
|
281
|
+
}
|
|
282
|
+
|
|
283
|
+
const db = new DatabaseSync("users.db");
|
|
284
|
+
const stmt = db.prepare("SELECT * FROM users WHERE id = ?");
|
|
285
|
+
|
|
286
|
+
const user = stmt.get(1) as User;
|
|
287
|
+
console.log(`User: ${user.name} <${user.email}>`);
|
|
288
|
+
```
|
|
289
|
+
|
|
290
|
+
## Performance
|
|
291
|
+
|
|
292
|
+
This package provides performance comparable to Node.js's built-in SQLite and better-sqlite3, with:
|
|
293
|
+
|
|
294
|
+
- **Synchronous operations** - No async/await overhead
|
|
295
|
+
- **Direct C library access** - Minimal JavaScript ↔ native boundary crossings
|
|
296
|
+
- **Full SQLite features** - FTS5, JSON functions, R\*Tree indexes, math functions, session extension (including changesets)
|
|
297
|
+
|
|
298
|
+
Performance is quite similar to node:sqlite and better-sqlite3, while significantly faster than async sqlite3 due to synchronous operations.
|
|
299
|
+
|
|
300
|
+
## Platform Support
|
|
301
|
+
|
|
302
|
+
| Platform | x64 | ARM64 | Notes |
|
|
303
|
+
| -------- | --- | ----- | ------------------------------------------------ |
|
|
304
|
+
| Linux | ✅ | ✅ | GLIBC 2.31+ (Ubuntu 20.04+, Debian 11+, RHEL 8+) |
|
|
305
|
+
| Alpine | ✅ | ✅ | Alpine 3.21+ (musl libc) |
|
|
306
|
+
| macOS | ✅ | ✅ | macOS 10.15+ |
|
|
307
|
+
| Windows | ✅ | ✅ | Windows 10+ |
|
|
308
|
+
|
|
309
|
+
### Linux Distribution Requirements
|
|
310
|
+
|
|
311
|
+
**Supported distributions** (with prebuilt binaries):
|
|
312
|
+
|
|
313
|
+
- Ubuntu 20.04 LTS and newer
|
|
314
|
+
- Debian 11 (Bullseye) and newer
|
|
315
|
+
- RHEL/CentOS/Rocky/Alma Linux 8 and newer
|
|
316
|
+
- Fedora 32 and newer
|
|
317
|
+
- Alpine Linux 3.21 and newer (musl libc)
|
|
318
|
+
- Any distribution with GLIBC 2.31 or newer
|
|
319
|
+
|
|
320
|
+
**Not supported** (GLIBC too old):
|
|
321
|
+
|
|
322
|
+
- Debian 10 (Buster) - GLIBC 2.28
|
|
323
|
+
- Ubuntu 18.04 LTS - GLIBC 2.27
|
|
324
|
+
- CentOS 7 - GLIBC 2.17
|
|
325
|
+
- Amazon Linux 2 - GLIBC 2.26
|
|
326
|
+
|
|
327
|
+
> **Note**: While Node.js 20 itself supports these older distributions, our prebuilt binaries require GLIBC 2.31+ due to toolchain requirements. Users on older distributions can still compile from source if they have a compatible compiler (GCC 10+ with C++20 support).
|
|
328
|
+
|
|
329
|
+
Prebuilt binaries are provided for all supported platforms. If a prebuilt binary isn't available, the package will compile from source using node-gyp.
|
|
330
|
+
|
|
331
|
+
## Development Requirements
|
|
332
|
+
|
|
333
|
+
- **Node.js**: v20 or higher
|
|
334
|
+
- **Build tools** (if compiling from source):
|
|
335
|
+
- Linux: `build-essential`, `python3` (3.8+), GCC 10+ or Clang 10+
|
|
336
|
+
- macOS: Xcode command line tools
|
|
337
|
+
- Windows: Visual Studio Build Tools 2019 or newer
|
|
338
|
+
- **Python**: 3.8 or higher (required by node-gyp v11)
|
|
339
|
+
|
|
340
|
+
## Alternatives
|
|
341
|
+
|
|
342
|
+
When choosing a SQLite library for Node.js, you have several excellent options. Here's how **`@photostructure/sqlite`** compares to the alternatives:
|
|
343
|
+
|
|
344
|
+
### 🏷️ [`node:sqlite`](https://nodejs.org/docs/latest/api/sqlite.html) — Node.js Built-in Module
|
|
345
|
+
|
|
346
|
+
_The official SQLite module included with Node.js 22.5.0+ (experimental)_
|
|
347
|
+
|
|
348
|
+
**✨ Pros:**
|
|
349
|
+
|
|
350
|
+
- **Zero dependencies** — Built directly into Node.js
|
|
351
|
+
- **Official support** — Maintained by the Node.js core team
|
|
352
|
+
- **Clean synchronous API** — Simple, predictable blocking operations
|
|
353
|
+
- **Full SQLite power** — FTS5, JSON functions, R\*Tree, sessions/changesets, and more
|
|
354
|
+
|
|
355
|
+
**⚠️ Cons:**
|
|
356
|
+
|
|
357
|
+
- **Experimental status** — Not yet stable for production use
|
|
358
|
+
- **Requires Node.js 22.5.0+** — Won't work on older versions
|
|
359
|
+
- **Flag required** — Must use `--experimental-sqlite` to enable
|
|
360
|
+
- **API may change** — Breaking changes possible before stable release
|
|
361
|
+
- **Limited real-world usage** — Few production deployments to learn from
|
|
362
|
+
|
|
363
|
+
**🎯 Best for:** Experimental projects, early adopters, and preparing for the future when it becomes stable.
|
|
364
|
+
|
|
365
|
+
---
|
|
366
|
+
|
|
367
|
+
### 🚀 [`better-sqlite3`](https://github.com/WiseLibs/better-sqlite3) — The Performance Champion
|
|
368
|
+
|
|
369
|
+
_The most popular high-performance synchronous SQLite library_
|
|
370
|
+
|
|
371
|
+
**✨ Pros:**
|
|
372
|
+
|
|
373
|
+
- **Blazing fast** — 2-15x faster than async alternatives
|
|
374
|
+
- **Rock-solid stability** — Battle-tested in thousands of production apps
|
|
375
|
+
- **Rich feature set** — User functions, aggregates, virtual tables, extensions
|
|
376
|
+
- **Extensive community** — Large ecosystem with many resources
|
|
377
|
+
|
|
378
|
+
**⚠️ Cons:**
|
|
379
|
+
|
|
380
|
+
- **Different API** — Not compatible with Node.js built-in SQLite
|
|
381
|
+
- **V8-specific** — Requires separate builds for each Node.js version
|
|
382
|
+
- **Synchronous only** — No async operations (usually a feature, not a bug)
|
|
383
|
+
- **Migration effort** — Switching from other libraries requires code changes
|
|
384
|
+
- **No session support** — Doesn't expose SQLite's session/changeset functionality
|
|
385
|
+
|
|
386
|
+
**🎯 Best for:** High-performance applications where you want maximum speed and control over the API.
|
|
387
|
+
|
|
388
|
+
---
|
|
389
|
+
|
|
390
|
+
### 📦 [`sqlite3`](https://github.com/TryGhost/node-sqlite3) — The Async Classic
|
|
391
|
+
|
|
392
|
+
_The original asynchronous SQLite binding for Node.js_
|
|
393
|
+
|
|
394
|
+
**✨ Pros:**
|
|
395
|
+
|
|
396
|
+
- **Battle-tested legacy** — 10+ years of production use
|
|
397
|
+
- **Massive ecosystem** — 4000+ dependent packages
|
|
398
|
+
- **Truly asynchronous** — Non-blocking operations won't freeze your app
|
|
399
|
+
- **Extensive resources** — Countless tutorials and Stack Overflow answers
|
|
400
|
+
- **Extension support** — Works with SQLCipher for encryption
|
|
401
|
+
- **Node-API stable** — One build works across Node.js versions
|
|
402
|
+
|
|
403
|
+
**⚠️ Cons:**
|
|
404
|
+
|
|
405
|
+
- **Significantly slower** — 2-15x performance penalty vs synchronous libs
|
|
406
|
+
- **Callback complexity** — Prone to callback hell without careful design
|
|
407
|
+
- **Unnecessary overhead** — SQLite is inherently synchronous anyway
|
|
408
|
+
- **Memory management quirks** — Exposes low-level C concerns to JavaScript
|
|
409
|
+
- **Concurrency issues** — Mutex contention under heavy load
|
|
410
|
+
|
|
411
|
+
**🎯 Best for:** Legacy codebases, apps requiring true async operations, or when you need SQLCipher encryption.
|
|
412
|
+
|
|
413
|
+
---
|
|
414
|
+
|
|
415
|
+
## 🎯 Quick Decision Guide
|
|
416
|
+
|
|
417
|
+
### Choose **`@photostructure/sqlite`** when you want:
|
|
418
|
+
|
|
419
|
+
- ✅ **Future-proof code** that works with both this package AND `node:sqlite`
|
|
420
|
+
- ✅ **Node.js API compatibility** without waiting for stable release
|
|
421
|
+
- ✅ **Broad Node.js support** (v20+) without experimental flags
|
|
422
|
+
- ✅ **Synchronous performance** with a clean, official API
|
|
423
|
+
- ✅ **Node-API stability** — one build works across Node.js versions
|
|
424
|
+
- ✅ **Zero migration path** when `node:sqlite` becomes stable
|
|
425
|
+
- ✅ **Session/changeset support** for replication and synchronization
|
|
426
|
+
|
|
427
|
+
### Choose **`better-sqlite3`** when you want:
|
|
428
|
+
|
|
429
|
+
- ✅ The most mature and feature-rich synchronous SQLite library
|
|
430
|
+
- ✅ Maximum performance above all else
|
|
431
|
+
- ✅ A specific API design that differs from Node.js
|
|
432
|
+
|
|
433
|
+
### Choose **`sqlite3`** when you have:
|
|
434
|
+
|
|
435
|
+
- ✅ Legacy code using async/callback patterns
|
|
436
|
+
- ✅ Hard requirement for non-blocking operations
|
|
437
|
+
- ✅ Need for SQLCipher encryption
|
|
438
|
+
|
|
439
|
+
### Choose **`node:sqlite`** when you're:
|
|
440
|
+
|
|
441
|
+
- ✅ Experimenting with bleeding-edge Node.js features
|
|
442
|
+
- ✅ Building proof-of-concepts for future migration
|
|
443
|
+
- ✅ Working in environments where you control the Node.js version
|
|
444
|
+
|
|
445
|
+
## Contributing
|
|
446
|
+
|
|
447
|
+
Contributions are welcome! This project maintains 100% API compatibility with Node.js's built-in SQLite module. Please run tests with `npm test` and ensure code passes linting with `npm run lint` before submitting changes. When adding new features, include corresponding tests and ensure they match Node.js SQLite behavior exactly.
|
|
448
|
+
|
|
449
|
+
The project includes automated sync scripts to keep up-to-date with:
|
|
450
|
+
|
|
451
|
+
- **Node.js SQLite implementation** via `npm run sync:node`
|
|
452
|
+
- **SQLite library updates** via `npm run sync:sqlite`
|
|
453
|
+
|
|
454
|
+
## Security
|
|
455
|
+
|
|
456
|
+
This project takes security seriously and employs multiple layers of protection:
|
|
457
|
+
|
|
458
|
+
- **Automated scanning**: npm audit, Snyk, OSV Scanner, CodeQL (JS/TS and C++), and TruffleHog
|
|
459
|
+
- **Weekly security scans**: Automated checks for new vulnerabilities
|
|
460
|
+
- **Rapid patching**: Security fixes are prioritized and released quickly
|
|
461
|
+
- **Memory safety**: Validated through ASAN, valgrind, and comprehensive testing
|
|
462
|
+
|
|
463
|
+
### Running Security Scans Locally
|
|
464
|
+
|
|
465
|
+
```bash
|
|
466
|
+
# Install security tools (OSV Scanner, etc.)
|
|
467
|
+
./scripts/setup-security-tools.sh
|
|
468
|
+
|
|
469
|
+
# Run all security scans
|
|
470
|
+
npm run security
|
|
471
|
+
```
|
|
472
|
+
|
|
473
|
+
For details, see our [Security Policy](./SECURITY.md). To report vulnerabilities, please email security@photostructure.com.
|
|
474
|
+
|
|
475
|
+
## License
|
|
476
|
+
|
|
477
|
+
MIT License - see [LICENSE](./LICENSE) for details.
|
|
478
|
+
|
|
479
|
+
This package includes SQLite, which is in the public domain, as well as code from the Node.js project, which is MIT licensed.
|
|
480
|
+
|
|
481
|
+
## Support
|
|
482
|
+
|
|
483
|
+
- 🐛 **Bug reports**: [GitHub Issues](https://github.com/photostructure/node-sqlite/issues)
|
|
484
|
+
- 💬 **Questions**: [GitHub Discussions](https://github.com/photostructure/node-sqlite/discussions)
|
|
485
|
+
- 📧 **Security issues**: see [SECURITY.md](./SECURITY.md)
|
|
486
|
+
|
|
487
|
+
---
|
|
488
|
+
|
|
489
|
+
## Development
|
|
490
|
+
|
|
491
|
+
This project was built with substantial assistance from [Claude Code](https://claude.ai/referral/gM3vgw7pfA), an AI coding assistant.
|
|
492
|
+
|
|
493
|
+
Note that all changes are human-reviewed before merging.
|
|
494
|
+
|
|
495
|
+
### Project Timeline
|
|
496
|
+
|
|
497
|
+
- <details>
|
|
498
|
+
<summary>900+ lines of C++</summary>
|
|
499
|
+
`find . -name "*.cpp" -o -name "*.h" -not -path "./node_modules/*" -not -path "./vendored/*" -not -path "*/upstream/*" -exec wc -l {} +`
|
|
500
|
+
</details>
|
|
501
|
+
- <details>
|
|
502
|
+
<summary>17,000 lines of comprehensive TypeScript tests
|
|
503
|
+
</summary>
|
|
504
|
+
`find . -name "*.ts" -not -path "./node_modules/*" -not -path "./vendored/*" -not -path "*/upstream/*" -exec wc -l {} +`
|
|
505
|
+
</details>
|
|
506
|
+
- **400+ tests** with full API compliance running in both ESM and CJS modes
|
|
507
|
+
- **Multi-platform CI/CD** with automated builds
|
|
508
|
+
- **Security scanning** and memory leak detection
|
|
509
|
+
- **Automated sync** from Node.js and SQLite upstream
|
|
510
|
+
- **Robust [benchmarking suite](./benchmark/README.md)** including all popular Node.js SQLite libraries
|
|
511
|
+
|
|
512
|
+
### Development Cost
|
|
513
|
+
|
|
514
|
+
- **API usage**: ~$1400 in Claude API tokens
|
|
515
|
+
- **Actual cost**: $200/month MAX 20x plan subscription
|
|
516
|
+
- **Time saved**: At least a month of setup, analysis, porting and testing
|
|
517
|
+
|
|
518
|
+
This project demonstrates how AI-assisted development can accelerate complex system programming while maintaining high code quality through comprehensive testing and human oversight.
|
|
519
|
+
|
|
520
|
+
---
|
|
521
|
+
|
|
522
|
+
**Note**: This package is not affiliated with the Node.js project. It extracts and redistributes Node.js's SQLite implementation under the MIT license.
|
package/SECURITY.md
ADDED
|
@@ -0,0 +1,114 @@
|
|
|
1
|
+
# Security Policy
|
|
2
|
+
|
|
3
|
+
## Supported Versions
|
|
4
|
+
|
|
5
|
+
Currently, we support security updates for the following versions:
|
|
6
|
+
|
|
7
|
+
| Version | Supported |
|
|
8
|
+
| ------- | ------------------ |
|
|
9
|
+
| 0.1.x | :white_check_mark: |
|
|
10
|
+
| < 0.1 | :x: |
|
|
11
|
+
|
|
12
|
+
## Reporting a Vulnerability
|
|
13
|
+
|
|
14
|
+
We take the security of @photostructure/sqlite seriously. If you believe you have found a security vulnerability, please report it to us as described below.
|
|
15
|
+
|
|
16
|
+
### How to Report
|
|
17
|
+
|
|
18
|
+
**Please do not report security vulnerabilities through public GitHub issues.**
|
|
19
|
+
|
|
20
|
+
Instead, please report them via one of the following methods:
|
|
21
|
+
|
|
22
|
+
1. Email us at security@photostructure.com
|
|
23
|
+
2. Use GitHub's private vulnerability reporting feature (if available)
|
|
24
|
+
|
|
25
|
+
### What to Include
|
|
26
|
+
|
|
27
|
+
Please include the following information in your report:
|
|
28
|
+
|
|
29
|
+
- Type of issue (e.g., buffer overflow, SQL injection, cross-site scripting, etc.)
|
|
30
|
+
- Full paths of source file(s) related to the manifestation of the issue
|
|
31
|
+
- The location of the affected source code (tag/branch/commit or direct URL)
|
|
32
|
+
- Any special configuration required to reproduce the issue
|
|
33
|
+
- Step-by-step instructions to reproduce the issue
|
|
34
|
+
- Proof-of-concept or exploit code (if possible)
|
|
35
|
+
- Impact of the issue, including how an attacker might exploit it
|
|
36
|
+
|
|
37
|
+
### Response Timeline
|
|
38
|
+
|
|
39
|
+
- We will acknowledge receipt of your vulnerability report within 48 hours
|
|
40
|
+
- We will provide a more detailed response within 7 days
|
|
41
|
+
- We will work on fixes and coordinate disclosure timeline with you
|
|
42
|
+
|
|
43
|
+
## Security Measures
|
|
44
|
+
|
|
45
|
+
### Automated Security Scanning
|
|
46
|
+
|
|
47
|
+
This project employs multiple layers of automated security scanning:
|
|
48
|
+
|
|
49
|
+
1. **npm audit** - Scans for known vulnerabilities in dependencies
|
|
50
|
+
2. **Snyk** - Advanced vulnerability detection and remediation
|
|
51
|
+
3. **OSV Scanner** - Google's Open Source Vulnerabilities scanner
|
|
52
|
+
4. **CodeQL** - GitHub's semantic code analysis for both JavaScript/TypeScript and C++
|
|
53
|
+
5. **TruffleHog** - Secrets detection in code
|
|
54
|
+
|
|
55
|
+
These scans run automatically on:
|
|
56
|
+
|
|
57
|
+
- Every push to the main branch
|
|
58
|
+
- Every pull request
|
|
59
|
+
- Weekly scheduled scans
|
|
60
|
+
- Manual workflow dispatch
|
|
61
|
+
|
|
62
|
+
### Development Practices
|
|
63
|
+
|
|
64
|
+
- All dependencies are regularly updated via Dependabot
|
|
65
|
+
- Security patches are prioritized and released quickly
|
|
66
|
+
- Native C++ code is analyzed with clang-tidy and ASAN
|
|
67
|
+
- Memory safety is validated through comprehensive testing
|
|
68
|
+
|
|
69
|
+
### Native Code Security
|
|
70
|
+
|
|
71
|
+
Since this package includes native C++ bindings to SQLite:
|
|
72
|
+
|
|
73
|
+
- We use the official SQLite amalgamation source
|
|
74
|
+
- SQLite is compiled with recommended security flags
|
|
75
|
+
- Buffer overflows are prevented through careful memory management
|
|
76
|
+
- All user inputs are properly validated before passing to SQLite
|
|
77
|
+
|
|
78
|
+
## Security Configuration
|
|
79
|
+
|
|
80
|
+
### SQLite Security Features
|
|
81
|
+
|
|
82
|
+
The following SQLite security features are available:
|
|
83
|
+
|
|
84
|
+
```javascript
|
|
85
|
+
// Restrict file access to read-only
|
|
86
|
+
const db = new DatabaseSync("database.db", {
|
|
87
|
+
readonly: true,
|
|
88
|
+
});
|
|
89
|
+
|
|
90
|
+
// Disable extension loading by default
|
|
91
|
+
// Extensions must be explicitly enabled
|
|
92
|
+
db.allowExtension(); // Required first
|
|
93
|
+
db.enableLoadExtension(true); // Then enable
|
|
94
|
+
db.loadExtension("path/to/extension");
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
### Best Practices
|
|
98
|
+
|
|
99
|
+
1. **Always validate and sanitize user input** before using in SQL queries
|
|
100
|
+
2. **Use parameterized queries** to prevent SQL injection
|
|
101
|
+
3. **Run with minimal permissions** when possible
|
|
102
|
+
4. **Keep dependencies updated** regularly
|
|
103
|
+
5. **Monitor security advisories** for SQLite and Node.js
|
|
104
|
+
|
|
105
|
+
## Disclosure Policy
|
|
106
|
+
|
|
107
|
+
When we receive a security report, we will:
|
|
108
|
+
|
|
109
|
+
1. Confirm the problem and determine affected versions
|
|
110
|
+
2. Audit code to find similar problems
|
|
111
|
+
3. Prepare fixes for all supported versions
|
|
112
|
+
4. Coordinate disclosure with the reporter
|
|
113
|
+
|
|
114
|
+
We aim to disclose vulnerabilities responsibly, balancing the need for users to be informed with giving them time to update.
|