@dikolab/kbdb 0.4.3 → 0.6.0

package/docs/goals/sync.md

@@ -1,217 +0,0 @@
-# Sync -- Cross-Machine Synchronization
-
-*Export the database as a git-friendly directory, move
-it with any transport, import on the other side.*
-
-## Goal
-
-kbdb is currently single-machine -- "persists across
-sessions" means across sessions on the same box. For
-teams, multi-device use, or backup, the database needs
-a sync story.
-
-The design uses export/import as the primitive, with
-git as the recommended transport. Sync is manual and
-explicit. There is no background replication.
-
-This goal addresses gap #5 from
-[Opus 4.8 Second Brain Review](../issues/2026-06-23-opus-48-second-brain-review.md):
-"No sync/remote story unless you build one on top,
-which limits 'persists across sessions' to 'across
-sessions on this box.'"
-
-## Sync Primitives
-
-Two operations form the sync foundation.
-
-### Export
-
-Serialize the database into a portable, git-friendly
-directory:
-
-```
-kbdb export --format git --db .kbdb --out ./kbdb-sync/
-```
-
-The exported directory mirrors `.kbdb/` minus indexes:
-
-- `catalog.toml` -- database table of contents
-- `sections/**/*.section` -- all section files
-- `documents/**/*.toml` -- all document manifests
-
-Excluded:
-
-- `indexes/` -- derived data, rebuildable
-- Vector indexes (`vectors.idx`, `vectors-meta.toml`)
-- PID files and lock files
-
-The exported directory is a valid git subtree. It
-contains no binary blobs except section content
-(image sections, which are stored as base64 text).
-Every file is human-readable and git-diffable.
-
-### Import
-
-Restore a database from an exported directory:
-
-```
-kbdb import --from ./kbdb-sync/ --db .kbdb
-```
-
-Import reads the exported directory and writes sections
-and documents into the target database. After import,
-run `rebuild` to regenerate all indexes (lexical and
-vector, if configured).
-
-Import is **additive** -- existing sections with
-different kbids are preserved. Only the conflict
-resolution rules below can cause existing sections to
-be superseded.
-
-## Conflict Resolution
-
-When importing sections that conflict with existing
-ones, the following rules apply in priority order:
-
-| Condition | Action |
-|-----------|--------|
-| Same kbid | Skip -- identical content already exists |
-| Same `source_key`, imported `created_at` is newer | Supersede -- imported section replaces local one |
-| Same `source_key`, local `created_at` is newer or equal | Keep -- local version is retained |
-| No `source_key` match | Add as new section |
-
-This leverages the existing supersession mechanism
-documented in [Database](database.md). The content
-hash (kbid) remains the primary identity. The source
-key is the logical identity used for supersession
-matching across machines.
-
-## Git-Based Workflow
-
-Recommended workflow for teams:
-
-```
-Machine A                    Git repo                    Machine B
-─────────                    ────────                    ─────────
-kbdb export --format git ──→ git push ──→ git pull ──→ kbdb import
-                                                         kbdb rebuild
-```
-
-Steps:
-
-1. Machine A exports to a directory within a git repo.
-2. Machine A commits and pushes.
-3. Machine B pulls and imports.
-4. Machine B rebuilds indexes.
-
-This is a manual, explicit sync -- not automatic
-real-time replication. Any transport that moves the
-exported directory works: git, rsync, scp, a shared
-drive, an archive file. Git is recommended because the
-export format is designed to be git-diffable.
-
-## Vector Index Handling
-
-Vector indexes (`vectors.idx`, `vectors-meta.toml`)
-are excluded from export -- they are derived data, the
-same as lexical indexes.
-
-After import:
-
-- If an embedding provider is configured, `rebuild`
-  regenerates vectors using the local model.
-- If the importing machine uses a different embedding
-  model than the exporting machine, vectors are
-  computed fresh with the local model. No action is
-  required to handle this case -- vectors are always
-  rebuilt from content, never transferred.
-
-## Export Formats
-
-| Format | Description | Use Case |
-|--------|-------------|----------|
-| `git` | Flat directory, text-friendly | Git-based sync |
-| `archive` | Single tarball (`.tar.gz`) | Backup, transfer |
-
-The default format is `git` (the directory format).
-
-## CLI Surface
-
-```
-kbdb export [--format git|archive] [--out <path>]
-kbdb import --from <path> [--dry-run]
-```
-
-- `kbdb export` -- exports the database to the
-  specified output path. Defaults to `--format git`.
-  The `--out` path is created if it does not exist.
-- `kbdb import` -- imports from an exported directory
-  or archive. Runs conflict resolution for each
-  incoming section and document.
-- `--dry-run` on import -- shows what would change
-  (sections added, superseded, skipped) without
-  writing anything.
-
-## Import Report
-
-Import returns a structured report:
-
-```json
-{
-  "sections_added": 42,
-  "sections_skipped": 3,
-  "sections_superseded": 5,
-  "documents_added": 2,
-  "documents_skipped": 1,
-  "rebuild_required": true
-}
-```
-
-`rebuild_required` is always `true` after a successful
-import that adds or supersedes any sections, because
-the lexical index is out of date. The caller is
-responsible for running `rebuild` after import.
-
-## Why Not CRDTs
-
-CRDTs (Conflict-free Replicated Data Types) would
-enable automatic, real-time sync without conflicts.
-The design chooses export/import instead for three
-reasons:
-
-1. **Identity conflict** -- The kbid is a content
-   hash. Content changes create new identities rather
-   than updating existing ones. This conflicts with
-   CRDT merge semantics, which assume stable object
-   identity across mutations.
-
-2. **Complexity** -- CRDT support at the current user
-   scale is speculative complexity (YAGNI). The
-   export/import model covers the primary use cases
-   (backup, team sharing, multi-device) with familiar
-   tooling and no new dependencies.
-
-3. **Existing supersession** -- The source key
-   mechanism already provides a logical identity layer
-   above content addressing. Conflict resolution at
-   import time uses the same supersession logic that
-   `kbdb learn --replace` uses. No new primitives are
-   required.
-
-CRDT support may be revisited in a future version if
-demand warrants it.
-
-## Design Constraints
-
-- Sync is manual and explicit -- no background
-  replication.
-- Export excludes all derived data (indexes, vectors).
-- Import respects the existing supersession model
-  from [Database](database.md).
-- No new dependencies -- uses existing file I/O.
-- Git is recommended but not required -- any transport
-  that moves the exported directory works.
-- The single-machine write lock documented in
-  [Concurrent Write Safety](concurrency.md) is not
-  relevant to sync. Import is a local write operation
-  and acquires the lock normally.