@plures/pluresdb 1.6.10 → 2.9.6

package/README.md

@@ -5,329 +5,137 @@
 [![Deno version](https://img.shields.io/badge/deno-v2.x-blue)](https://deno.land)
 [![License: AGPL-3.0](https://img.shields.io/badge/License-AGPL--3.0-blue.svg)](https://opensource.org/licenses/AGPL-3.0)
 
-**Local-First Graph Database with SQLite Compatibility**
+**Local-First P2P Graph Database — v2.0**
 
-PluresDB is a CRDT-based graph database that speaks SQLite. Built with Rust for performance and TypeScript for accessibility, it provides SQLite API compatibility while adding graph relationships, vector search, and P2P synchronization. Perfect for desktop applications, VSCode extensions, and personal knowledge management.
+PluresDB is a CRDT-based graph database built with Rust and TypeScript. It
+provides a native sled-backed storage layer with CRDT conflict resolution,
+vector search, and P2P synchronisation — ideal for desktop apps, VSCode
+extensions, and personal knowledge management.
 
-> 💡 **Ideal for Windows Desktop Apps**: Drop-in SQLite replacement with graph capabilities, vector search, and a comprehensive web UI. [Get Started on Windows →](docs/WINDOWS_GETTING_STARTED.md)
+> **v2.0 breaking change:** The `rusqlite` dependency has been removed from
+> `pluresdb-core` by default.  `CrdtStore::with_persistence()` now accepts
+> `Arc<dyn StorageEngine>` instead of `Arc<Database>`.  If you need the legacy
+> SQL layer, enable the `sqlite-compat` Cargo feature.  See
+> [MIGRATION.md](MIGRATION.md) for upgrade instructions.
 
-## 🚀 Quick Start
-
-### Install
+## Install
 
 ```bash
-# Node.js / npm
-npm install pluresdb
+# Node.js
+npm install @plures/pluresdb
 
 # Deno
 deno add @plures/pluresdb
 
 # Rust
-cargo add pluresdb-core pluresdb-storage
+cargo add pluresdb-core
+
+# CLI
+cargo install pluresdb-cli
 
-# Windows (Winget)
+# Windows
 winget install pluresdb.pluresdb
 
 # Docker
 docker pull pluresdb/pluresdb:latest
 ```
 
-### Use It
-
-```typescript
-import { PluresNode, SQLiteCompatibleAPI } from "pluresdb";
-
-// Start the database
-const db = new PluresNode({
-  config: { port: 34567, dataDir: "./data" },
-  autoStart: true
-});
-
-// Use SQLite-compatible API
-const sqlite = new SQLiteCompatibleAPI();
-
-// Create tables
-await sqlite.exec(`
-  CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT, email TEXT)
-`);
-
-// Insert data
-await sqlite.run(
-  "INSERT INTO users (name, email) VALUES (?, ?)", 
-  ["Alice", "alice@example.com"]
-);
-
-// Query data
-const users = await sqlite.all("SELECT * FROM users");
-
-// Vector search
-const results = await sqlite.vectorSearch("machine learning", 10);
-```
-
-## 🎯 What PluresDB Does
-
-### Core Capabilities
-
-- **SQLite Compatibility**: Drop in for SQLite with 95% API compatibility
-- **Graph Relationships**: Store and query connected data with CRDT conflict resolution
-- **Vector Search**: Semantic similarity search with HNSW indexing
-- **Local-First**: Runs entirely on your machine, syncs when you want
-- **P2P Sync**: Encrypted data sharing across devices without servers
-- **Web UI**: 24-tab management interface for data exploration
-
-### Built For
-
-- **Desktop Applications**: Embedded database with graph and vector capabilities
-- **VSCode Extensions**: SQLite-compatible storage with enhanced features
-- **Knowledge Management**: Personal wikis, note-taking, research databases
-- **Offline-First Apps**: Full functionality without network connectivity
-- **Prototyping**: Quick database setup with TypeScript/Rust support
-
-## 📖 How to Use PluresDB
-
-### For Node.js Applications
-
-```typescript
-import { SQLiteCompatibleAPI } from "pluresdb";
-
-const db = new SQLiteCompatibleAPI({
-  config: { dataDir: "./data" }
-});
-
-// Use familiar SQLite methods
-await db.exec("CREATE TABLE ...");
-await db.run("INSERT INTO ...", params);
-const rows = await db.all("SELECT ...");
-```
-
-### For Deno Applications
-
-```typescript
-import { GunDB, startApiServer } from "jsr:@plures/pluresdb";
-
-const db = new GunDB();
-await db.ready();
-
-db.serve({ port: 34567 });
-const api = startApiServer({ port: 8080, db });
-
-await db.put("user:alice", { name: "Alice" });
-const user = await db.get("user:alice");
-```
-
-### For Rust Applications
+## Quick Example
 
 ```rust
-use pluresdb_core::{Database, DatabaseOptions};
-
-let db = Database::open(
-    DatabaseOptions::with_file("./data/plures.db")
-        .create_if_missing(true)
-)?;
-
-db.put("user:1", json!({"name": "Alice"}))?;
-let user = db.get("user:1")?;
+// Rust
+use pluresdb_core::CrdtStore;
+use pluresdb_storage::{MemoryStorage, StorageEngine};
+use serde_json::json;
+use std::sync::Arc;
+
+let storage = Arc::new(MemoryStorage::default());
+let store = CrdtStore::default()
+    .with_persistence(storage as Arc<dyn StorageEngine>);
+
+store.put("user:1", "actor-a", json!({ "name": "Alice" }));
+let record = store.get("user:1");
 ```
 
-### For VSCode Extensions
-
-Replace SQLite in your VSCode extension with PluresDB:
-
-```typescript
-import { SQLiteCompatibleAPI } from "pluresdb";
-
-export function activate(context: vscode.ExtensionContext) {
-  const db = new SQLiteCompatibleAPI({
-    config: {
-      dataDir: path.join(context.globalStorageUri.fsPath, "pluresdb")
-    }
-  });
-
-  // Same SQLite API, enhanced capabilities
-  await db.exec("CREATE TABLE settings (key TEXT, value TEXT)");
-  await db.run("INSERT INTO settings VALUES (?, ?)", ["theme", "dark"]);
-}
-```
-
-## 🌐 Web Interface
-
-PluresDB includes a comprehensive Svelte-based web UI at `http://localhost:34568`:
-
-- **Data Explorer**: Browse, edit, and manage your data with JSON editing
-- **Graph Visualization**: Interactive Cytoscape.js graph views
-- **Vector Search UI**: Semantic search with similarity scoring
-- **Type Management**: Define schemas and validate data
-- **P2P Controls**: Manage peers, encryption, and cross-device sync
-- **Performance Monitoring**: Real-time metrics and profiling
-- **History & Time Travel**: Version history with diff and restore
-
-## 🔌 API Options
-
-### SQLite-Compatible API
-
-```typescript
-// Database operations
-await sqlite.exec(sql);                    // Execute SQL
-await sqlite.run(sql, params);             // Run with parameters
-await sqlite.get(sql, params);             // Get single row
-await sqlite.all(sql, params);             // Get all rows
-
-// Key-value operations
-await sqlite.put(key, value);              // Store data
-await sqlite.getValue(key);                // Retrieve data
-await sqlite.delete(key);                  // Remove data
-
-// Vector search
-await sqlite.vectorSearch(query, limit);   // Semantic search
-```
-
-### better-sqlite3-Compatible API
-
-For synchronous-style ergonomics:
-
-```typescript
-import Database from "pluresdb/better-sqlite3";
-
-const db = await new Database("./data.db", { autoStart: true }).open();
-
-const insert = db.prepare("INSERT INTO users (name) VALUES (?)");
-await insert.run("Ada Lovelace");
-
-const select = db.prepare("SELECT * FROM users");
-const users = await select.all();
-```
-
-### REST API
+## Features
+
+- **CRDT Store** — conflict-free replicated data with vector clocks
+- **Native Storage** — sled-backed persistence (WAL, encryption, replay)
+- **Vector Search** — approximate nearest-neighbour via HNSW (cosine similarity)
+- **Auto-Embedding** — pluggable `EmbedText` trait; `FastEmbedder` for local ONNX models
+- **P2P Sync** — Hyperswarm DHT or WebSocket relay, end-to-end encrypted
+- **Local-First** — full functionality offline; sync is opt-in
+- **Multi-Platform** — Node.js (N-API), Deno (JSR), Rust, CLI, Docker, WASM
+- **SQLite optional** — legacy SQL layer via `sqlite-compat` feature flag
+
+## Documentation
+
+| Document | Description |
+|---|---|
+| [docs/GETTING_STARTED.md](docs/GETTING_STARTED.md) | Quick start for Node.js, Deno, Rust, CLI, Docker, Windows |
+| [docs/API.md](docs/API.md) | Complete API reference (Rust, Node.js, Deno, REST, CLI) |
+| [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) | Internals: CRDTs, storage, HNSW, P2P protocol |
+| [docs/WINDOWS_GETTING_STARTED.md](docs/WINDOWS_GETTING_STARTED.md) | Windows-specific setup guide |
+| [docs/HYPERSWARM_SYNC.md](docs/HYPERSWARM_SYNC.md) | P2P sync deep-dive |
+| [docs/SYNC_TRANSPORT.md](docs/SYNC_TRANSPORT.md) | Relay transport for corporate networks |
+| [docs/LOCAL_FIRST_INTEGRATION.md](docs/LOCAL_FIRST_INTEGRATION.md) | WASM, Tauri, IPC integration guides |
+| [docs/TESTING.md](docs/TESTING.md) | Test suite and CI notes |
+| [MIGRATION.md](MIGRATION.md) | Upgrade guide: v1.x → v2.0 |
+| [CONTRIBUTING.md](CONTRIBUTING.md) | Contribution guide |
+| [CHANGELOG.md](CHANGELOG.md) | Release history |
+| [SECURITY.md](SECURITY.md) | Security policy |
+
+## Architecture
+
+PluresDB is a Rust-first monorepo:
+
+| Crate | Responsibility |
+|---|---|
+| `pluresdb-core` | CRDT store, HNSW vector index, `EmbedText` trait; SQLite optional via `sqlite-compat` |
+| `pluresdb-storage` | Pluggable backends: Sled (WAL, encryption), in-memory |
+| `pluresdb-sync` | `SyncBroadcaster`, `Transport` trait, Hyperswarm / relay / disabled |
+| `pluresdb-cli` | `pluresdb` binary |
+| `pluresdb-node` | N-API bindings for Node.js |
+| `pluresdb-deno` | Deno FFI bindings |
+| `pluresdb-wasm` | `wasm-bindgen` bindings for browsers |
+| `pluresdb-ipc` | Shared-memory IPC server/client |
+| `legacy/` | TypeScript layer (being replaced by Rust crates) |
+
+See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for a full description with
+data-flow diagrams.
+
+## Testing
 
 ```bash
-# Create/update node
-curl -X POST http://localhost:34567/api/put \
-  -H "Content-Type: application/json" \
-  -d '{"id": "user:1", "data": {"name": "Alice"}}'
-
-# Retrieve node
-curl http://localhost:34567/api/get?id=user:1
-
-# Delete node
-curl -X DELETE http://localhost:34567/api/delete?id=user:1
-
-# List all nodes
-curl http://localhost:34567/api/list
-
-# Vector search
-curl -X POST http://localhost:34567/api/search \
-  -H "Content-Type: application/json" \
-  -d '{"query": "machine learning", "limit": 10}'
-```
-
-### Local-First APIs
-
-PluresDB provides production-ready Rust implementations for local-first integration:
-
-**WASM (Browser)** - Rust implementation complete, use directly:
-```javascript
-// Via wasm-bindgen (compile from source)
-import { PluresDBBrowser } from "./pluresdb-wasm/pkg";
-const db = new PluresDBBrowser("my-app");
-await db.init_persistence();
-await db.put("user:1", { name: "Alice" });
-```
-
-**Tauri (Desktop Apps)**
-```rust
-#[tauri::command]
-async fn db_put(state: State<'_, AppState>, id: String, data: Value) -> Result<String> {
-    state.db.lock().put(id, data)
-}
-```
-
-**IPC (Native Apps)**
-```rust
-let mut server = IPCServer::new("my-app", store)?;
-server.start()?;
-// Client connects via shared memory
+npm run verify        # TypeScript build + all Deno tests
+cargo test --workspace  # Rust tests
 ```
 
-See [Local-First Integration](docs/LOCAL_FIRST_INTEGRATION.md) for complete guides.
-
-## 🗂️ Architecture
-
-PluresDB is built as a Rust-first monorepo:
-
-- **`crates/pluresdb-core`**: CRDT storage engine
-- **`crates/pluresdb-storage`**: Storage backends (Sled, SQLite, RocksDB)
-- **`crates/pluresdb-sync`**: P2P synchronization
-- **`crates/pluresdb-cli`**: Command-line interface
-- **`crates/pluresdb-wasm`**: WebAssembly bindings
-- **`crates/pluresdb-ipc`**: IPC shared memory
-- **`legacy/`**: TypeScript/Node.js/Deno implementations
-- **`web/svelte/`**: Web UI components
-
-## 📦 Distribution
-
-PluresDB is available through multiple channels:
-
-- **npm**: [`pluresdb`](https://www.npmjs.com/package/pluresdb) - Node.js package
-- **JSR**: [`@plures/pluresdb`](https://jsr.io/@plures/pluresdb) - Deno module
-- **crates.io**: Rust crates ([pluresdb-core](https://crates.io/crates/pluresdb-core), [pluresdb-storage](https://crates.io/crates/pluresdb-storage), [pluresdb-sync](https://crates.io/crates/pluresdb-sync))
-- **Winget**: `pluresdb.pluresdb` - Windows package manager
-- **Docker Hub**: [`pluresdb/pluresdb`](https://hub.docker.com/r/pluresdb/pluresdb) - Container images
-- **GitHub Releases**: Pre-built binaries for Windows, macOS, Linux
-
-## 🔒 Security
-
-PluresDB implements comprehensive security measures:
-
-- **Input Validation**: All user inputs are validated and sanitized
-- **Prototype Pollution Protection**: Safe object handling
-- **Audit Logging**: Complete operation logs
-- **Local Storage**: Data stays on your machine by default
-- **End-to-End Encryption**: Secure P2P synchronization
-- **AGPL v3 License**: Ensures modifications remain open source
+Network-dependent Hyperswarm tests are skipped automatically in CI.
+See [docs/TESTING.md](docs/TESTING.md) for details.
 
-Report security issues privately via our [Security Policy](SECURITY.md).
+## Distribution
 
-## 📊 Performance
+- **npm**: [`pluresdb`](https://www.npmjs.com/package/pluresdb)
+- **JSR**: [`@plures/pluresdb`](https://jsr.io/@plures/pluresdb)
+- **crates.io**: [`pluresdb-core`](https://crates.io/crates/pluresdb-core), [`pluresdb-sync`](https://crates.io/crates/pluresdb-sync)
+- **Winget**: `pluresdb.pluresdb`
+- **Docker**: [`pluresdb/pluresdb`](https://hub.docker.com/r/pluresdb/pluresdb)
+- **GitHub Releases**: pre-built binaries for Windows, macOS, Linux
 
-- **CRDT Operations**: Efficient conflict-free data structures in Rust
-- **Vector Search**: HNSW-based similarity search
-- **Storage Backends**: Sled, SQLite, or RocksDB
-- **Local Operations**: ~5-10ms REST API latency
-- **Zero Network**: Full functionality without internet
+## Security
 
-## 🧪 Testing
+All inputs are validated and sanitised. P2P communications are end-to-end
+encrypted. Report vulnerabilities privately — see [SECURITY.md](SECURITY.md).
 
-Run the comprehensive test suite:
-
-```bash
-npm run verify
-```
-
-This executes TypeScript compilation and all Deno test suites (unit, integration, performance, security).
-
-## 📚 Documentation
-
-- [Windows Getting Started Guide](docs/WINDOWS_GETTING_STARTED.md)
-- [Local-First Integration](docs/LOCAL_FIRST_INTEGRATION.md)
-- [VSCode Extension Example](examples/vscode-extension-integration.ts)
-- [Contributing Guide](CONTRIBUTING.md)
-- [CHANGELOG](CHANGELOG.md)
-
-## 🤝 Contributing
-
-Contributions are welcome! See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
+## Contributing
 
+Contributions are welcome. See [CONTRIBUTING.md](CONTRIBUTING.md).
 All contributions are licensed under AGPL v3.
 
-## 📄 License
-
-GNU Affero General Public License v3.0 (AGPL v3). See [LICENSE](LICENSE) for details.
-
-## 🆘 Support
+## License
 
-- **Issues**: [GitHub Issues](https://github.com/plures/pluresdb/issues)
-- **Discussions**: [GitHub Discussions](https://github.com/plures/pluresdb/discussions)
-- **Security**: [Security Policy](SECURITY.md)
+GNU Affero General Public License v3.0. See [LICENSE](LICENSE).
 
 ---
 

package/crates/README.md

@@ -0,0 +1,99 @@
+# PluresDB Rust Crates
+
+This directory contains all Rust crates for the PluresDB project.
+
+## Crates Overview
+
+### Main Library Crate (Recommended)
+
+- **pluresdb** - Unified main crate that re-exports all core functionality
+
+### Core Crates
+
+- **pluresdb-core** - Core CRDTs, data structures, and query primitives
+- **pluresdb-sync** - Sync orchestration primitives for PluresDB peers
+- **pluresdb-storage** - Storage abstraction layer with multiple backends
+
+### Application Crates
+
+- **pluresdb-cli** - Command-line interface for managing PluresDB nodes
+
+### Bindings
+
+- **pluresdb-node** - Node.js bindings using N-API
+- **pluresdb-deno** - Deno bindings using deno_bindgen FFI
+
+## Status
+
+| Crate | Status | Registry |
+|-------|--------|----------|
+| pluresdb | ✅ Ready | crates.io |
+| pluresdb-core | ✅ Published | crates.io |
+| pluresdb-sync | ✅ Published | crates.io |
+| pluresdb-storage | ✅ Ready | crates.io |
+| pluresdb-cli | ✅ Ready | crates.io |
+| pluresdb-node | ✅ Ready | npm |
+| pluresdb-deno | ✅ Ready | JSR |
+
+## Quick Start
+
+### Installing Rust Crates
+
+```bash
+# Install CLI
+cargo install pluresdb-cli
+
+# Add main crate to your project (recommended)
+cargo add pluresdb
+
+# Or add individual crates if you prefer
+cargo add pluresdb-core
+cargo add pluresdb-storage
+cargo add pluresdb-sync
+```
+
+### Installing Node.js Bindings
+
+```bash
+npm install @plures/pluresdb-native
+```
+
+### Installing Deno Bindings
+
+```bash
+deno add jsr:@plures/pluresdb
+```
+
+## Building from Source
+
+```bash
+# Build all crates
+cargo build --workspace --release
+
+# Build specific crate
+cargo build -p pluresdb-cli --release
+
+# Run tests
+cargo test --workspace
+```
+
+## Publishing
+
+See [PUBLISHING_GUIDE.md](./PUBLISHING_GUIDE.md) for detailed publishing instructions.
+
+## Documentation
+
+- [Crate Organization](./CRATE_ORGANIZATION.md) - Guide to choosing the right crate
+- [Implementation Status](./IMPLEMENTATION_STATUS.md)
+- [Completion Summary](./COMPLETION_SUMMARY.md)
+- [Publishing Guide](./PUBLISHING_GUIDE.md)
+- [Next Steps](./NEXT_STEPS.md)
+
+## Version
+
+All crates use the workspace version: **1.4.2**
+
+## License
+
+AGPL-3.0
+