rust-kgdb 0.1.8 → 0.1.10
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +385 -230
- package/package.json +8 -8
- package/rust-kgdb-napi.darwin-arm64.node +0 -0
- package/rust-kgdb-napi.node +0 -0
package/README.md
CHANGED
|
@@ -1,186 +1,415 @@
|
|
|
1
|
-
# rust-kgdb
|
|
1
|
+
# rust-kgdb
|
|
2
2
|
|
|
3
3
|
[](https://www.npmjs.com/package/rust-kgdb)
|
|
4
4
|
[](https://opensource.org/licenses/Apache-2.0)
|
|
5
5
|
|
|
6
|
-
**Production-ready
|
|
6
|
+
**Production-ready RDF/hypergraph database with 100% W3C SPARQL 1.1 + RDF 1.2 compliance, worst-case optimal joins (WCOJ), and pluggable storage backends.**
|
|
7
7
|
|
|
8
|
-
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
## Why rust-kgdb?
|
|
11
|
+
|
|
12
|
+
| Feature | rust-kgdb | Apache Jena | RDFox |
|
|
13
|
+
|---------|-----------|-------------|-------|
|
|
14
|
+
| **Lookup Speed** | 2.78 µs | ~50 µs | 50-100 µs |
|
|
15
|
+
| **Memory/Triple** | 24 bytes | 50-60 bytes | 32 bytes |
|
|
16
|
+
| **SPARQL 1.1** | 100% | 100% | 95% |
|
|
17
|
+
| **RDF 1.2** | 100% | Partial | No |
|
|
18
|
+
| **WCOJ** | ✅ LeapFrog | ❌ | ❌ |
|
|
19
|
+
| **Mobile-Ready** | ✅ iOS/Android | ❌ | ❌ |
|
|
20
|
+
|
|
21
|
+
---
|
|
22
|
+
|
|
23
|
+
## Core Technical Innovations
|
|
24
|
+
|
|
25
|
+
### 1. Worst-Case Optimal Joins (WCOJ)
|
|
26
|
+
|
|
27
|
+
Traditional databases use **nested-loop joins** with O(n²) to O(n⁴) complexity. rust-kgdb implements the **LeapFrog TrieJoin** algorithm—a worst-case optimal join that achieves O(n log n) for multi-way joins.
|
|
28
|
+
|
|
29
|
+
**How it works:**
|
|
30
|
+
- **Trie Data Structure**: Triples indexed hierarchically (S→P→O) using BTreeMap for sorted access
|
|
31
|
+
- **Variable Ordering**: Frequency-based analysis orders variables for optimal intersection
|
|
32
|
+
- **LeapFrog Iterator**: Binary search across sorted iterators finds intersections without materializing intermediate results
|
|
33
|
+
|
|
34
|
+
```
|
|
35
|
+
Query: SELECT ?x ?y ?z WHERE { ?x :p ?y . ?y :q ?z . ?x :r ?z }
|
|
36
|
+
|
|
37
|
+
Nested Loop: O(n³) - examines every combination
|
|
38
|
+
WCOJ: O(n log n) - iterates in sorted order, seeks forward on mismatch
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
| Query Pattern | Before (Nested Loop) | After (WCOJ) | Speedup |
|
|
42
|
+
|---------------|---------------------|--------------|---------|
|
|
43
|
+
| 3-way star | O(n³) | O(n log n) | **50-100x** |
|
|
44
|
+
| 4+ way complex | O(n⁴) | O(n log n) | **100-1000x** |
|
|
45
|
+
| Chain queries | O(n²) | O(n log n) | **10-20x** |
|
|
46
|
+
|
|
47
|
+
### 2. Sparse Matrix Engine (CSR Format)
|
|
48
|
+
|
|
49
|
+
Binary relations (e.g., `foaf:knows`, `rdfs:subClassOf`) are converted to **Compressed Sparse Row (CSR)** matrices for cache-efficient join evaluation:
|
|
50
|
+
|
|
51
|
+
- **Memory**: O(nnz) where nnz = number of edges (not O(n²))
|
|
52
|
+
- **Matrix Multiplication**: Replaces nested-loop joins
|
|
53
|
+
- **Transitive Closure**: Semi-naive Δ-matrix evaluation (not iterated powers)
|
|
54
|
+
|
|
55
|
+
```rust
|
|
56
|
+
// Traditional: O(n²) nested loops
|
|
57
|
+
for (s, p, o) in triples { ... }
|
|
58
|
+
|
|
59
|
+
// CSR Matrix: O(nnz) cache-friendly iteration
|
|
60
|
+
row_ptr[i] → col_indices[j] → values[j]
|
|
61
|
+
```
|
|
9
62
|
|
|
10
|
-
|
|
11
|
-
- **100% W3C RDF 1.2 Compliance** - Full standard implementation
|
|
12
|
-
- **WCOJ Execution** (v0.1.8) - LeapFrog TrieJoin for optimal multi-way joins
|
|
13
|
-
- **Zero-Copy Semantics** - Minimal allocations, maximum performance
|
|
14
|
-
- **Blazing Fast** - 2.78 µs triple lookups, 146K triples/sec bulk insert
|
|
15
|
-
- **Memory Efficient** - 24 bytes/triple (25% better than RDFox)
|
|
16
|
-
- **Native Rust** - Safe, reliable, production-ready
|
|
63
|
+
**Used for**: RDFS/OWL reasoning, transitive closure, Datalog evaluation.
|
|
17
64
|
|
|
18
|
-
|
|
65
|
+
### 3. SIMD + PGO Compiler Optimizations
|
|
19
66
|
|
|
20
|
-
|
|
67
|
+
**Zero code changes—pure compiler-level performance gains.**
|
|
21
68
|
|
|
22
|
-
|
|
|
23
|
-
|
|
24
|
-
| **
|
|
25
|
-
| **
|
|
26
|
-
| **
|
|
69
|
+
| Optimization | Technology | Effect |
|
|
70
|
+
|--------------|------------|--------|
|
|
71
|
+
| **SIMD Vectorization** | AVX2/BMI2 (Intel), NEON (ARM) | 8-wide parallel operations |
|
|
72
|
+
| **Profile-Guided Optimization** | LLVM PGO | Hot path optimization, branch prediction |
|
|
73
|
+
| **Link-Time Optimization** | LTO (fat) | Cross-crate inlining, dead code elimination |
|
|
27
74
|
|
|
28
|
-
|
|
75
|
+
**Benchmark Results (LUBM, Intel Skylake):**
|
|
29
76
|
|
|
30
|
-
|
|
|
31
|
-
|
|
32
|
-
|
|
|
33
|
-
|
|
|
34
|
-
|
|
|
77
|
+
| Query | Before | After (SIMD+PGO) | Improvement |
|
|
78
|
+
|-------|--------|------------------|-------------|
|
|
79
|
+
| Q5: 2-hop chain | 230ms | 53ms | **77% faster** |
|
|
80
|
+
| Q3: 3-way star | 177ms | 62ms | **65% faster** |
|
|
81
|
+
| Q4: 3-hop chain | 254ms | 101ms | **60% faster** |
|
|
82
|
+
| Q8: Triangle | 410ms | 193ms | **53% faster** |
|
|
83
|
+
| Q7: Hierarchy | 343ms | 198ms | **42% faster** |
|
|
84
|
+
| Q6: 6-way complex | 641ms | 464ms | **28% faster** |
|
|
85
|
+
| Q2: 5-way star | 234ms | 183ms | **22% faster** |
|
|
86
|
+
| Q1: 4-way star | 283ms | 258ms | **9% faster** |
|
|
35
87
|
|
|
36
|
-
|
|
88
|
+
**Average speedup: 44.5%** across all queries.
|
|
89
|
+
|
|
90
|
+
### 4. Quad Indexing (SPOC)
|
|
91
|
+
|
|
92
|
+
Four complementary indexes enable O(1) pattern matching regardless of query shape:
|
|
93
|
+
|
|
94
|
+
| Index | Pattern | Use Case |
|
|
95
|
+
|-------|---------|----------|
|
|
96
|
+
| **SPOC** | `(?s, ?p, ?o, ?g)` | Subject-centric queries |
|
|
97
|
+
| **POCS** | `(?p, ?o, ?c, ?s)` | Property enumeration |
|
|
98
|
+
| **OCSP** | `(?o, ?c, ?s, ?p)` | Object lookups (reverse links) |
|
|
99
|
+
| **CSPO** | `(?c, ?s, ?p, ?o)` | Named graph iteration |
|
|
100
|
+
|
|
101
|
+
---
|
|
102
|
+
|
|
103
|
+
## Storage Backends
|
|
104
|
+
|
|
105
|
+
rust-kgdb uses a pluggable storage architecture. **Default is in-memory** (zero configuration). For persistence, enable RocksDB.
|
|
106
|
+
|
|
107
|
+
| Backend | Feature Flag | Use Case | Status |
|
|
108
|
+
|---------|--------------|----------|--------|
|
|
109
|
+
| **InMemory** | `default` | Development, testing, embedded | ✅ **Production Ready** |
|
|
110
|
+
| **RocksDB** | `rocksdb-backend` | Production, large datasets | ✅ **61 tests passing** |
|
|
111
|
+
| **LMDB** | `lmdb-backend` | Read-heavy workloads | ⏳ Planned v0.2.0 |
|
|
112
|
+
|
|
113
|
+
### InMemory (Default)
|
|
114
|
+
|
|
115
|
+
Zero configuration, maximum performance. Data is volatile (lost on process exit).
|
|
116
|
+
|
|
117
|
+
**High-Performance Data Structures:**
|
|
118
|
+
|
|
119
|
+
| Component | Structure | Why |
|
|
120
|
+
|-----------|-----------|-----|
|
|
121
|
+
| **Triple Store** | `DashMap` | Lock-free concurrent hash map, 100K pre-allocation |
|
|
122
|
+
| **WCOJ Trie** | `BTreeMap` | Sorted iteration for LeapFrog intersection |
|
|
123
|
+
| **Dictionary** | `FxHashSet` | String interning with rustc-optimized hashing |
|
|
124
|
+
| **Hypergraph** | `FxHashMap` | Fast node→edge adjacency lists |
|
|
125
|
+
| **Reasoning** | `AHashMap` | RDFS/OWL inference with DoS-resistant hashing |
|
|
126
|
+
| **Datalog** | `FxHashMap` | Semi-naive evaluation with delta propagation |
|
|
127
|
+
|
|
128
|
+
**Why these structures enable sub-microsecond performance:**
|
|
129
|
+
- **DashMap**: Sharded locks (16 shards default) → near-linear scaling on multi-core
|
|
130
|
+
- **FxHashMap**: Rust compiler's hash function → 30% faster than std HashMap
|
|
131
|
+
- **BTreeMap**: O(log n) ordered iteration → enables binary search in LeapFrog
|
|
132
|
+
- **Pre-allocation**: 100K capacity avoids rehashing during bulk inserts
|
|
133
|
+
|
|
134
|
+
```rust
|
|
135
|
+
use storage::{QuadStore, InMemoryBackend};
|
|
136
|
+
|
|
137
|
+
let store = QuadStore::new(InMemoryBackend::new());
|
|
138
|
+
// Ultra-fast: 2.78 µs lookups, zero disk I/O
|
|
139
|
+
```
|
|
140
|
+
|
|
141
|
+
### RocksDB (Persistent)
|
|
142
|
+
|
|
143
|
+
LSM-tree based storage with ACID transactions. Tested with **61 comprehensive tests**.
|
|
144
|
+
|
|
145
|
+
```toml
|
|
146
|
+
# Cargo.toml - Enable RocksDB backend
|
|
147
|
+
[dependencies]
|
|
148
|
+
storage = { version = "0.1.10", features = ["rocksdb-backend"] }
|
|
149
|
+
```
|
|
150
|
+
|
|
151
|
+
```rust
|
|
152
|
+
use storage::{QuadStore, RocksDbBackend};
|
|
153
|
+
|
|
154
|
+
// Create persistent database
|
|
155
|
+
let backend = RocksDbBackend::new("/path/to/data")?;
|
|
156
|
+
let store = QuadStore::new(backend);
|
|
157
|
+
|
|
158
|
+
// Features:
|
|
159
|
+
// - ACID transactions
|
|
160
|
+
// - Snappy compression (automatic)
|
|
161
|
+
// - Crash recovery
|
|
162
|
+
// - Range & prefix scanning
|
|
163
|
+
// - 1MB+ value support
|
|
164
|
+
|
|
165
|
+
// Force sync to disk
|
|
166
|
+
store.flush()?;
|
|
167
|
+
```
|
|
168
|
+
|
|
169
|
+
**RocksDB Test Coverage:**
|
|
170
|
+
- Basic CRUD operations (14 tests)
|
|
171
|
+
- Range scanning (8 tests)
|
|
172
|
+
- Prefix scanning (6 tests)
|
|
173
|
+
- Batch operations (8 tests)
|
|
174
|
+
- Transactions (8 tests)
|
|
175
|
+
- Concurrent access (5 tests)
|
|
176
|
+
- Unicode & binary data (4 tests)
|
|
177
|
+
- Large key/value handling (8 tests)
|
|
178
|
+
|
|
179
|
+
### TypeScript SDK
|
|
180
|
+
|
|
181
|
+
The npm package uses the in-memory backend—ideal for:
|
|
182
|
+
- Knowledge graph queries
|
|
183
|
+
- SPARQL execution
|
|
184
|
+
- Data transformation pipelines
|
|
185
|
+
- Embedded applications
|
|
186
|
+
|
|
187
|
+
```typescript
|
|
188
|
+
import { GraphDB } from 'rust-kgdb'
|
|
189
|
+
|
|
190
|
+
// In-memory database (default, no configuration needed)
|
|
191
|
+
const db = new GraphDB('http://example.org/app')
|
|
192
|
+
|
|
193
|
+
// For persistence, export via CONSTRUCT:
|
|
194
|
+
const ntriples = db.queryConstruct('CONSTRUCT { ?s ?p ?o } WHERE { ?s ?p ?o }')
|
|
195
|
+
fs.writeFileSync('backup.nt', ntriples)
|
|
196
|
+
```
|
|
197
|
+
|
|
198
|
+
---
|
|
199
|
+
|
|
200
|
+
## Installation
|
|
37
201
|
|
|
38
202
|
```bash
|
|
39
203
|
npm install rust-kgdb
|
|
40
204
|
```
|
|
41
205
|
|
|
42
|
-
###
|
|
206
|
+
### Platform Support
|
|
207
|
+
|
|
208
|
+
| Platform | Architecture | Status | SIMD |
|
|
209
|
+
|----------|-------------|--------|------|
|
|
210
|
+
| **macOS** | Intel (x64) | ✅ | AVX2, BMI2, POPCNT |
|
|
211
|
+
| **macOS** | Apple Silicon (arm64) | ✅ | NEON |
|
|
212
|
+
| **Linux** | x64 | ✅ | AVX2, BMI2, POPCNT |
|
|
213
|
+
| **Linux** | arm64 | ✅ | NEON |
|
|
214
|
+
| **Windows** | x64 | ✅ | AVX2, BMI2, POPCNT |
|
|
215
|
+
| **Windows** | arm64 | ⏳ v0.2.0 | — |
|
|
216
|
+
|
|
217
|
+
**No compilation required**—pre-built native binaries included.
|
|
218
|
+
|
|
219
|
+
---
|
|
43
220
|
|
|
44
|
-
|
|
45
|
-
- No additional dependencies required (native bindings included)
|
|
221
|
+
## Quick Start
|
|
46
222
|
|
|
47
|
-
|
|
223
|
+
### Complete Working Example
|
|
48
224
|
|
|
49
225
|
```typescript
|
|
50
|
-
import { GraphDB
|
|
226
|
+
import { GraphDB } from 'rust-kgdb'
|
|
51
227
|
|
|
52
|
-
// Create
|
|
53
|
-
const db = new GraphDB('http://example.org/
|
|
228
|
+
// 1. Create database
|
|
229
|
+
const db = new GraphDB('http://example.org/myapp')
|
|
54
230
|
|
|
55
|
-
//
|
|
231
|
+
// 2. Load data (Turtle format)
|
|
56
232
|
db.loadTtl(`
|
|
57
233
|
@prefix foaf: <http://xmlns.com/foaf/0.1/> .
|
|
234
|
+
@prefix ex: <http://example.org/> .
|
|
58
235
|
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
236
|
+
ex:alice a foaf:Person ;
|
|
237
|
+
foaf:name "Alice" ;
|
|
238
|
+
foaf:age 30 ;
|
|
239
|
+
foaf:knows ex:bob, ex:charlie .
|
|
62
240
|
|
|
63
|
-
|
|
64
|
-
|
|
241
|
+
ex:bob a foaf:Person ;
|
|
242
|
+
foaf:name "Bob" ;
|
|
243
|
+
foaf:age 25 ;
|
|
244
|
+
foaf:knows ex:charlie .
|
|
245
|
+
|
|
246
|
+
ex:charlie a foaf:Person ;
|
|
247
|
+
foaf:name "Charlie" ;
|
|
248
|
+
foaf:age 35 .
|
|
65
249
|
`, null)
|
|
66
250
|
|
|
67
|
-
//
|
|
68
|
-
const
|
|
251
|
+
// 3. Query: Find friends-of-friends (WCOJ optimized!)
|
|
252
|
+
const fof = db.querySelect(`
|
|
69
253
|
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
|
|
254
|
+
PREFIX ex: <http://example.org/>
|
|
70
255
|
|
|
71
|
-
SELECT ?person ?
|
|
72
|
-
?person foaf:
|
|
73
|
-
|
|
256
|
+
SELECT ?person ?friend ?fof WHERE {
|
|
257
|
+
?person foaf:knows ?friend .
|
|
258
|
+
?friend foaf:knows ?fof .
|
|
259
|
+
FILTER(?person != ?fof)
|
|
74
260
|
}
|
|
75
|
-
ORDER BY DESC(?age)
|
|
76
261
|
`)
|
|
262
|
+
console.log('Friends of Friends:', fof)
|
|
263
|
+
// [{ person: 'ex:alice', friend: 'ex:bob', fof: 'ex:charlie' }]
|
|
264
|
+
|
|
265
|
+
// 4. Aggregation: Average age
|
|
266
|
+
const stats = db.querySelect(`
|
|
267
|
+
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
|
|
77
268
|
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
269
|
+
SELECT (COUNT(?p) AS ?count) (AVG(?age) AS ?avgAge) WHERE {
|
|
270
|
+
?p a foaf:Person ; foaf:age ?age .
|
|
271
|
+
}
|
|
272
|
+
`)
|
|
273
|
+
console.log('Stats:', stats)
|
|
274
|
+
// [{ count: '3', avgAge: '30.0' }]
|
|
83
275
|
|
|
84
|
-
//
|
|
276
|
+
// 5. ASK query
|
|
85
277
|
const hasAlice = db.queryAsk(`
|
|
86
|
-
|
|
278
|
+
PREFIX ex: <http://example.org/>
|
|
279
|
+
ASK { ex:alice a <http://xmlns.com/foaf/0.1/Person> }
|
|
87
280
|
`)
|
|
88
|
-
console.log(hasAlice)
|
|
281
|
+
console.log('Has Alice?', hasAlice) // true
|
|
89
282
|
|
|
90
|
-
//
|
|
91
|
-
|
|
283
|
+
// 6. CONSTRUCT query
|
|
284
|
+
const graph = db.queryConstruct(`
|
|
285
|
+
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
|
|
286
|
+
PREFIX ex: <http://example.org/>
|
|
287
|
+
|
|
288
|
+
CONSTRUCT { ?p foaf:knows ?f }
|
|
289
|
+
WHERE { ?p foaf:knows ?f }
|
|
290
|
+
`)
|
|
291
|
+
console.log('Extracted graph:', graph)
|
|
292
|
+
|
|
293
|
+
// 7. Count and cleanup
|
|
294
|
+
console.log('Triple count:', db.count()) // 11
|
|
295
|
+
db.clear()
|
|
92
296
|
```
|
|
93
297
|
|
|
94
|
-
|
|
298
|
+
### Save to File
|
|
95
299
|
|
|
96
|
-
|
|
300
|
+
```typescript
|
|
301
|
+
import { writeFileSync } from 'fs'
|
|
302
|
+
|
|
303
|
+
// Save as N-Triples
|
|
304
|
+
const db = new GraphDB('http://example.org/export')
|
|
305
|
+
db.loadTtl(`<http://example.org/s> <http://example.org/p> "value" .`, null)
|
|
306
|
+
|
|
307
|
+
const ntriples = db.queryConstruct(`CONSTRUCT { ?s ?p ?o } WHERE { ?s ?p ?o }`)
|
|
308
|
+
writeFileSync('output.nt', ntriples)
|
|
309
|
+
```
|
|
310
|
+
|
|
311
|
+
---
|
|
312
|
+
|
|
313
|
+
## SPARQL 1.1 Features (100% W3C Compliant)
|
|
314
|
+
|
|
315
|
+
### Query Forms
|
|
97
316
|
|
|
98
317
|
```typescript
|
|
99
|
-
//
|
|
100
|
-
|
|
101
|
-
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
|
|
318
|
+
// SELECT - return bindings
|
|
319
|
+
db.querySelect('SELECT ?s ?p ?o WHERE { ?s ?p ?o } LIMIT 10')
|
|
102
320
|
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
?person foaf:age ?age .
|
|
106
|
-
?person foaf:email ?email .
|
|
107
|
-
}
|
|
108
|
-
`)
|
|
321
|
+
// ASK - boolean existence check
|
|
322
|
+
db.queryAsk('ASK { <http://example.org/x> ?p ?o }')
|
|
109
323
|
|
|
110
|
-
//
|
|
111
|
-
|
|
324
|
+
// CONSTRUCT - build new graph
|
|
325
|
+
db.queryConstruct('CONSTRUCT { ?s <http://new/prop> ?o } WHERE { ?s ?p ?o }')
|
|
112
326
|
```
|
|
113
327
|
|
|
114
|
-
###
|
|
328
|
+
### Aggregates
|
|
115
329
|
|
|
116
330
|
```typescript
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
?person2 org:worksAt ?company .
|
|
124
|
-
?person1 org:name ?name1 .
|
|
125
|
-
?person2 org:name ?name2 .
|
|
126
|
-
FILTER(?person1 != ?person2)
|
|
127
|
-
}
|
|
331
|
+
db.querySelect(`
|
|
332
|
+
SELECT ?type (COUNT(*) AS ?count) (AVG(?value) AS ?avg)
|
|
333
|
+
WHERE { ?s a ?type ; <http://ex/value> ?value }
|
|
334
|
+
GROUP BY ?type
|
|
335
|
+
HAVING (COUNT(*) > 5)
|
|
336
|
+
ORDER BY DESC(?count)
|
|
128
337
|
`)
|
|
338
|
+
```
|
|
339
|
+
|
|
340
|
+
### Property Paths
|
|
341
|
+
|
|
342
|
+
```typescript
|
|
343
|
+
// Transitive closure (rdfs:subClassOf*)
|
|
344
|
+
db.querySelect('SELECT ?class WHERE { ?class rdfs:subClassOf* <http://top/Class> }')
|
|
345
|
+
|
|
346
|
+
// Alternative paths
|
|
347
|
+
db.querySelect('SELECT ?name WHERE { ?x (foaf:name|rdfs:label) ?name }')
|
|
129
348
|
|
|
130
|
-
//
|
|
131
|
-
|
|
349
|
+
// Sequence paths
|
|
350
|
+
db.querySelect('SELECT ?grandparent WHERE { ?x foaf:parent/foaf:parent ?grandparent }')
|
|
132
351
|
```
|
|
133
352
|
|
|
134
|
-
###
|
|
353
|
+
### Named Graphs
|
|
135
354
|
|
|
136
355
|
```typescript
|
|
137
|
-
//
|
|
138
|
-
|
|
139
|
-
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
|
|
356
|
+
// Load into named graph
|
|
357
|
+
db.loadTtl('<http://s> <http://p> "o" .', 'http://example.org/graph1')
|
|
140
358
|
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
359
|
+
// Query specific graph
|
|
360
|
+
db.querySelect(`
|
|
361
|
+
SELECT ?s ?p ?o WHERE {
|
|
362
|
+
GRAPH <http://example.org/graph1> { ?s ?p ?o }
|
|
144
363
|
}
|
|
145
364
|
`)
|
|
365
|
+
```
|
|
366
|
+
|
|
367
|
+
### UPDATE Operations
|
|
368
|
+
|
|
369
|
+
```typescript
|
|
370
|
+
// INSERT DATA
|
|
371
|
+
db.updateInsert(`
|
|
372
|
+
INSERT DATA { <http://ex/new> <http://ex/prop> "value" }
|
|
373
|
+
`)
|
|
146
374
|
|
|
147
|
-
//
|
|
148
|
-
|
|
375
|
+
// DELETE WHERE
|
|
376
|
+
db.updateDelete(`
|
|
377
|
+
DELETE WHERE { ?s <http://ex/deprecated> ?o }
|
|
378
|
+
`)
|
|
149
379
|
```
|
|
150
380
|
|
|
151
|
-
|
|
381
|
+
---
|
|
382
|
+
|
|
383
|
+
## API Reference
|
|
152
384
|
|
|
153
385
|
### GraphDB Class
|
|
154
386
|
|
|
155
387
|
```typescript
|
|
156
388
|
class GraphDB {
|
|
157
|
-
// Create
|
|
158
|
-
static inMemory(): GraphDB
|
|
159
|
-
constructor(baseUri: string)
|
|
389
|
+
constructor(baseUri: string) // Create with base URI
|
|
390
|
+
static inMemory(): GraphDB // Create anonymous in-memory DB
|
|
160
391
|
|
|
161
|
-
// Data
|
|
162
|
-
loadTtl(data: string,
|
|
163
|
-
loadNTriples(data: string,
|
|
392
|
+
// Data Loading
|
|
393
|
+
loadTtl(data: string, graph: string | null): void
|
|
394
|
+
loadNTriples(data: string, graph: string | null): void
|
|
164
395
|
|
|
165
|
-
// SPARQL
|
|
396
|
+
// SPARQL Queries (WCOJ-optimized)
|
|
166
397
|
querySelect(sparql: string): Array<Record<string, string>>
|
|
167
398
|
queryAsk(sparql: string): boolean
|
|
168
|
-
queryConstruct(sparql: string): string
|
|
399
|
+
queryConstruct(sparql: string): string // Returns N-Triples
|
|
169
400
|
|
|
170
|
-
// SPARQL
|
|
401
|
+
// SPARQL Updates
|
|
171
402
|
updateInsert(sparql: string): void
|
|
172
403
|
updateDelete(sparql: string): void
|
|
173
404
|
|
|
174
|
-
// Database
|
|
405
|
+
// Database Operations
|
|
175
406
|
count(): number
|
|
176
407
|
clear(): void
|
|
177
|
-
|
|
178
|
-
// Metadata
|
|
179
408
|
getVersion(): string
|
|
180
409
|
}
|
|
181
410
|
```
|
|
182
411
|
|
|
183
|
-
### Node Class
|
|
412
|
+
### Node Class
|
|
184
413
|
|
|
185
414
|
```typescript
|
|
186
415
|
class Node {
|
|
@@ -194,165 +423,91 @@ class Node {
|
|
|
194
423
|
}
|
|
195
424
|
```
|
|
196
425
|
|
|
197
|
-
|
|
198
|
-
|
|
199
|
-
### SPARQL UPDATE Operations
|
|
200
|
-
|
|
201
|
-
```typescript
|
|
202
|
-
// INSERT DATA
|
|
203
|
-
db.updateInsert(`
|
|
204
|
-
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
|
|
426
|
+
---
|
|
205
427
|
|
|
206
|
-
|
|
207
|
-
<http://example.org/charlie> foaf:name "Charlie" ;
|
|
208
|
-
foaf:age 35 .
|
|
209
|
-
}
|
|
210
|
-
`)
|
|
428
|
+
## Performance Characteristics
|
|
211
429
|
|
|
212
|
-
|
|
213
|
-
db.updateDelete(`
|
|
214
|
-
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
|
|
215
|
-
|
|
216
|
-
DELETE WHERE {
|
|
217
|
-
?person foaf:age ?age .
|
|
218
|
-
FILTER(?age < 18)
|
|
219
|
-
}
|
|
220
|
-
`)
|
|
221
|
-
```
|
|
430
|
+
### Complexity Analysis
|
|
222
431
|
|
|
223
|
-
|
|
432
|
+
| Operation | Complexity | Notes |
|
|
433
|
+
|-----------|------------|-------|
|
|
434
|
+
| Triple lookup | O(1) | Hash-based SPOC index |
|
|
435
|
+
| Pattern scan | O(k) | k = matching triples |
|
|
436
|
+
| Star join (WCOJ) | O(n log n) | LeapFrog intersection |
|
|
437
|
+
| Complex join (WCOJ) | O(n log n) | Trie-based |
|
|
438
|
+
| Transitive closure | O(n²) worst | CSR matrix optimization |
|
|
439
|
+
| Bulk insert | O(n) | Batch indexing |
|
|
224
440
|
|
|
225
|
-
|
|
226
|
-
// Load into named graph
|
|
227
|
-
db.loadTtl(`
|
|
228
|
-
<http://example.org/resource> <http://purl.org/dc/terms/title> "Title" .
|
|
229
|
-
`, 'http://example.org/graph1')
|
|
441
|
+
### Memory Layout
|
|
230
442
|
|
|
231
|
-
// Query specific graph
|
|
232
|
-
const results = db.querySelect(`
|
|
233
|
-
SELECT ?s ?p ?o WHERE {
|
|
234
|
-
GRAPH <http://example.org/graph1> {
|
|
235
|
-
?s ?p ?o .
|
|
236
|
-
}
|
|
237
|
-
}
|
|
238
|
-
`)
|
|
239
443
|
```
|
|
240
|
-
|
|
241
|
-
|
|
242
|
-
|
|
243
|
-
|
|
244
|
-
|
|
245
|
-
|
|
246
|
-
|
|
247
|
-
|
|
248
|
-
SELECT
|
|
249
|
-
(COUNT(?person) AS ?count)
|
|
250
|
-
(AVG(?age) AS ?avgAge)
|
|
251
|
-
(MIN(?age) AS ?minAge)
|
|
252
|
-
(MAX(?age) AS ?maxAge)
|
|
253
|
-
WHERE {
|
|
254
|
-
?person foaf:age ?age .
|
|
255
|
-
}
|
|
256
|
-
`)
|
|
444
|
+
Triple: 24 bytes
|
|
445
|
+
├── Subject: 8 bytes (dictionary ID)
|
|
446
|
+
├── Predicate: 8 bytes (dictionary ID)
|
|
447
|
+
└── Object: 8 bytes (dictionary ID)
|
|
448
|
+
|
|
449
|
+
String Interning: All URIs/literals stored once in Dictionary
|
|
450
|
+
Index Overhead: ~4x base triple size (4 indexes)
|
|
451
|
+
Total: ~120 bytes/triple including indexes
|
|
257
452
|
```
|
|
258
453
|
|
|
259
|
-
|
|
260
|
-
|
|
261
|
-
```typescript
|
|
262
|
-
// Transitive closure with *
|
|
263
|
-
const transitiveKnows = db.querySelect(`
|
|
264
|
-
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
|
|
265
|
-
|
|
266
|
-
SELECT ?person ?connected WHERE {
|
|
267
|
-
<http://example.org/alice> foaf:knows* ?connected .
|
|
268
|
-
}
|
|
269
|
-
`)
|
|
270
|
-
|
|
271
|
-
// Alternative paths with |
|
|
272
|
-
const nameOrLabel = db.querySelect(`
|
|
273
|
-
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
|
|
274
|
-
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
|
|
275
|
-
|
|
276
|
-
SELECT ?resource ?name WHERE {
|
|
277
|
-
?resource (foaf:name|rdfs:label) ?name .
|
|
278
|
-
}
|
|
279
|
-
`)
|
|
280
|
-
```
|
|
454
|
+
---
|
|
281
455
|
|
|
282
|
-
##
|
|
456
|
+
## Version History
|
|
283
457
|
|
|
284
|
-
|
|
285
|
-
- **Bindings**: NAPI-RS for native Node.js addon
|
|
286
|
-
- **Storage**: Pluggable backends (InMemory, RocksDB, LMDB)
|
|
287
|
-
- **Indexing**: SPOC, POCS, OCSP, CSPO quad indexes
|
|
288
|
-
- **Query Optimizer**: Automatic WCOJ detection and execution
|
|
289
|
-
- **WCOJ Engine**: LeapFrog TrieJoin with variable ordering analysis
|
|
458
|
+
### v0.1.9 (2025-12-01) - SIMD + PGO Release
|
|
290
459
|
|
|
291
|
-
|
|
460
|
+
- **44.5% average speedup** via SIMD + PGO compiler optimizations
|
|
461
|
+
- WCOJ execution with LeapFrog TrieJoin
|
|
462
|
+
- Release automation infrastructure
|
|
463
|
+
- All packages updated to gonnect-uk namespace
|
|
292
464
|
|
|
293
|
-
### v0.1.8 (2025-12-01) - WCOJ Execution
|
|
465
|
+
### v0.1.8 (2025-12-01) - WCOJ Execution
|
|
294
466
|
|
|
295
|
-
-
|
|
296
|
-
-
|
|
297
|
-
-
|
|
298
|
-
- ✅ **100-1000x Speedup** for complex joins (4+ patterns)
|
|
299
|
-
- ✅ **577 Tests Passing** - Comprehensive end-to-end verification
|
|
300
|
-
- ✅ **Zero Regressions** - All existing queries work unchanged
|
|
467
|
+
- WCOJ execution path activated
|
|
468
|
+
- Variable ordering analysis for optimal joins
|
|
469
|
+
- 577 tests passing
|
|
301
470
|
|
|
302
471
|
### v0.1.7 (2025-11-30)
|
|
303
472
|
|
|
304
473
|
- Query optimizer with automatic strategy selection
|
|
305
474
|
- WCOJ algorithm integration (planning phase)
|
|
306
|
-
- Query plan visualization API
|
|
307
475
|
|
|
308
476
|
### v0.1.3 (2025-11-18)
|
|
309
477
|
|
|
310
|
-
- Initial TypeScript SDK
|
|
478
|
+
- Initial TypeScript SDK
|
|
311
479
|
- 100% W3C SPARQL 1.1 compliance
|
|
312
480
|
- 100% W3C RDF 1.2 compliance
|
|
313
481
|
|
|
314
|
-
|
|
315
|
-
|
|
316
|
-
```bash
|
|
317
|
-
# Run test suite
|
|
318
|
-
npm test
|
|
319
|
-
|
|
320
|
-
# Run specific tests
|
|
321
|
-
npm test -- --testNamePattern="star query"
|
|
322
|
-
```
|
|
323
|
-
|
|
324
|
-
## 🤝 Contributing
|
|
325
|
-
|
|
326
|
-
Contributions are welcome! Please see [CONTRIBUTING.md](https://github.com/zenya/rust-kgdb/blob/main/CONTRIBUTING.md)
|
|
482
|
+
---
|
|
327
483
|
|
|
328
|
-
##
|
|
484
|
+
## Use Cases
|
|
329
485
|
|
|
330
|
-
|
|
486
|
+
| Domain | Application |
|
|
487
|
+
|--------|-------------|
|
|
488
|
+
| **Knowledge Graphs** | Enterprise ontologies, taxonomies |
|
|
489
|
+
| **Semantic Search** | Structured queries over unstructured data |
|
|
490
|
+
| **Data Integration** | ETL with SPARQL CONSTRUCT |
|
|
491
|
+
| **Compliance** | SHACL validation, provenance tracking |
|
|
492
|
+
| **Graph Analytics** | Pattern detection, community analysis |
|
|
493
|
+
| **Mobile Apps** | Embedded RDF on iOS/Android |
|
|
331
494
|
|
|
332
|
-
|
|
495
|
+
---
|
|
333
496
|
|
|
334
|
-
|
|
335
|
-
- [Documentation](https://github.com/zenya/rust-kgdb/tree/main/docs)
|
|
336
|
-
- [CHANGELOG](https://github.com/zenya/rust-kgdb/blob/main/CHANGELOG.md)
|
|
337
|
-
- [W3C SPARQL 1.1 Spec](https://www.w3.org/TR/sparql11-query/)
|
|
338
|
-
- [W3C RDF 1.2 Spec](https://www.w3.org/TR/rdf12-concepts/)
|
|
497
|
+
## Links
|
|
339
498
|
|
|
340
|
-
|
|
499
|
+
- [GitHub Repository](https://github.com/gonnect-uk/rust-kgdb)
|
|
500
|
+
- [Documentation](https://github.com/gonnect-uk/rust-kgdb/tree/main/docs)
|
|
501
|
+
- [CHANGELOG](https://github.com/gonnect-uk/rust-kgdb/blob/main/CHANGELOG.md)
|
|
502
|
+
- [W3C SPARQL 1.1](https://www.w3.org/TR/sparql11-query/)
|
|
503
|
+
- [W3C RDF 1.2](https://www.w3.org/TR/rdf12-concepts/)
|
|
341
504
|
|
|
342
|
-
|
|
343
|
-
- **Semantic Search** - Query structured data with SPARQL
|
|
344
|
-
- **Data Integration** - Combine data from multiple sources
|
|
345
|
-
- **Ontology Reasoning** - RDFS and OWL inference
|
|
346
|
-
- **Graph Analytics** - Complex pattern matching with WCOJ
|
|
347
|
-
- **Mobile Apps** - Embedded RDF database for iOS/Android
|
|
505
|
+
---
|
|
348
506
|
|
|
349
|
-
##
|
|
507
|
+
## License
|
|
350
508
|
|
|
351
|
-
|
|
352
|
-
- [ ] v0.1.9: SIMD optimizations for 2-4x additional speedup
|
|
353
|
-
- [ ] v0.2.0: Profile-guided optimization (PGO)
|
|
354
|
-
- [ ] v0.3.0: Distributed query execution
|
|
509
|
+
Apache License 2.0
|
|
355
510
|
|
|
356
511
|
---
|
|
357
512
|
|
|
358
|
-
**Built with
|
|
513
|
+
**Built with Rust + NAPI-RS**
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "rust-kgdb",
|
|
3
|
-
"version": "0.1.
|
|
3
|
+
"version": "0.1.10",
|
|
4
4
|
"description": "High-performance RDF/SPARQL database with 100% W3C compliance and WCOJ execution",
|
|
5
5
|
"main": "index.js",
|
|
6
6
|
"types": "index.d.ts",
|
|
@@ -21,7 +21,7 @@
|
|
|
21
21
|
"build:debug": "napi build --platform native/rust-kgdb-napi",
|
|
22
22
|
"prepublishOnly": "napi prepublish -t npm",
|
|
23
23
|
"test": "jest",
|
|
24
|
-
"version": "
|
|
24
|
+
"version": "0.1.10"
|
|
25
25
|
},
|
|
26
26
|
"keywords": [
|
|
27
27
|
"rdf",
|
|
@@ -37,7 +37,7 @@
|
|
|
37
37
|
"license": "Apache-2.0",
|
|
38
38
|
"repository": {
|
|
39
39
|
"type": "git",
|
|
40
|
-
"url": "https://github.com/
|
|
40
|
+
"url": "https://github.com/gonnect-uk/rust-kgdb"
|
|
41
41
|
},
|
|
42
42
|
"devDependencies": {
|
|
43
43
|
"@napi-rs/cli": "^2.18.0",
|
|
@@ -56,10 +56,10 @@
|
|
|
56
56
|
"*.node"
|
|
57
57
|
],
|
|
58
58
|
"optionalDependencies": {
|
|
59
|
-
"rust-kgdb-win32-x64-msvc": "0.1.
|
|
60
|
-
"rust-kgdb-darwin-x64": "0.1.
|
|
61
|
-
"rust-kgdb-linux-x64-gnu": "0.1.
|
|
62
|
-
"rust-kgdb-darwin-arm64": "0.1.
|
|
63
|
-
"rust-kgdb-linux-arm64-gnu": "0.1.
|
|
59
|
+
"rust-kgdb-win32-x64-msvc": "0.1.10",
|
|
60
|
+
"rust-kgdb-darwin-x64": "0.1.10",
|
|
61
|
+
"rust-kgdb-linux-x64-gnu": "0.1.10",
|
|
62
|
+
"rust-kgdb-darwin-arm64": "0.1.10",
|
|
63
|
+
"rust-kgdb-linux-arm64-gnu": "0.1.10"
|
|
64
64
|
}
|
|
65
65
|
}
|
|
Binary file
|
package/rust-kgdb-napi.node
DELETED
|
Binary file
|