@fireproof/core 0.0.8 → 0.0.10

package/README.md

@@ -0,0 +1,148 @@
+# 🔥 Fireproof
+
+Fireproof is a realtime database for today's interactive applications. It uses immutable data and distributed protocols 
+to offer a new kind of database that:
+- can be embedded in any page or app, with a flexible data ownership model
+- scales without incurring developer costs, thanks to Filecoin
+- uses cryptographically verifiable protocols (what plants crave)
+
+Learn more about the [concepts and architecture behind Fireproof](https://fireproof.storage/documentation/how-the-database-engine-works/), or jump to the [quick start](#quick-start) for React and server-side examples.
+
+### Status
+
+Fireproof is alpha software, you should only use it if you are planning to contribute. For now, [check out our React TodoMVC implementation running in browser-local mode.](https://main--lucky-naiad-5aa507.netlify.app/) It demonstrates document persistence, index queries, and event subscriptions, and uses the [`useFireproof()` React hook.](https://github.com/fireproof-storage/fireproof/blob/main/packages/fireproof/hooks/use-fireproof.tsx)
+
+[![Test](https://github.com/jchris/fireproof/actions/workflows/test.yml/badge.svg)](https://github.com/jchris/fireproof/actions/workflows/test.yml)
+[![JavaScript Style Guide](https://img.shields.io/badge/code_style-standard-brightgreen.svg)](https://standardjs.com)
+
+## Usage
+
+```js
+import Fireproof from 'fireproof';
+
+async function main() {
+  const database = Fireproof.storage('my-db');
+  const ok = await database.put({
+    name: 'alice',
+    age: 42
+  });
+  
+  const doc = await database.get(ok.id);
+  console.log(doc.name); // 'alice'
+}
+
+main();
+```
+
+## Features
+
+### Document Store
+
+A simple put, get, and delete interface for keeping track of all your JSON documents. Once your data is in Fireproof you can access it from any app or website. Fireproof document store uses MVCC versioning and Merkle clocks so you can always recover the version you are looking for.
+
+```js
+const { id, ref } = await database.put({
+    _id: 'three-thousand'
+    name: 'André',
+    age: 47
+});
+const doc = await database.get('three-thousand', {mvcc : true}) // mvcc is optional
+// {
+//    _id  : 'three-thousand'
+//    _clock : CID(bafy84...agfw7)
+//    name : 'André',
+//    age  : 47
+// }
+```
+
+The `_clock` allows you to query a stable snapshot of that version of the database. Fireproof uses immutable data structures under the hood, so you can always rollback to old data. Files can be embedded anywhere in your document using IPFS links like `{"/":"bafybeih3e3zdiehbqfpxzpppxrb6kaaw4xkbqzyr2f5pwr5refq2te2ape"}`, with API sugar coming soon.
+
+### Flexible Indexes
+
+Fireproof indexes are defined by custom JavaScript functions that you write, allowing you to easily index and search your data in the way that works best for your application. Easily handle data variety and schema drift by normalizing any data to the desired index.
+
+```js
+const index = new Index(database, function (doc, map) {
+  map(doc.age, doc.name)
+})
+const { rows, ref } = await index.query({ range: [40, 52] })
+// [ { key: 42, value: 'alice', id: 'a1s3b32a-3c3a-4b5e-9c1c-8c5c0c5c0c5c' },
+//   { key: 47, value: 'André', id: 'three-thousand' } ]
+```
+
+### Realtime Updates
+
+Subscribe to query changes in your application, so your UI updates automatically. Use the supplied React hooks, our Redux connector, or simple function calls to be notified of relevant changes.
+
+```js
+const listener = new Listener(database, function(doc, emit) {
+  if (doc.type == 'member') {
+    emit('member')
+  }
+})
+listener.on('member', (id) => {
+  const doc = await db.get(id)
+  alert(`Member update ${doc.name}`)
+})
+```
+
+### Self-sovereign Identity
+
+Fireproof is so easy to integrate with any site or app because you can get started right away, and set up an account later. By default users write to their own database copy, so you can get pretty far before you even have to think about API keys. [Authorization is via non-extractable keypair](https://ucan.xyz), like TouchID / FaceID.
+
+### Automatic Replication
+
+Documents changes are persisted to [Filecoin](https://filecoin.io) via [web3.storage](https://web3.storage), and made available over [IPFS] and on a global content delivery network. All you need to do to sync state is send a link to the latest database head, and Fireproof will take care of the rest. [Learn how to enable replication.](#status)
+
+### Cryptographic Proofs
+
+The [UCAN protocol](https://ucan.xyz) verifably links Fireproof updates to authorized agents via cryptographic proof chains. These proofs are portable like bearer tokens, but because invocations are signed by end-user device keys, UCAN proofs don't need to be hidden to be secure, allowing for delegation of service capabilities across devices and parties. Additionally, Fireproof's Merkle clocks and hash trees are immutable and self-validating, making merging changes safe and efficient. Fireproof makes cryptographic proofs available for all of it's operations, making it an ideal verfiable document database for smart contracts and other applications running in trustless environments. [Proof chains provide performance benefits as well](https://purrfect-tracker-45c.notion.site/Data-Routing-23c37b269b4c4c3dacb60d0077113bcb), by allowing recipients to skip costly I/O operations and instead cryptographically verify that changes contain all of the required context.
+
+## Limitations 💣
+
+### Security
+
+Until encryption support is enabled, all data written to Fireproof is public. There are no big hurdles for this feature but it's not ready yet.
+
+### Replication
+
+Currently Fireproof writes transactions and proofs to [CAR files](https://ipld.io/specs/transport/car/carv2/) which are well suited for peer and cloud replication. They are stored in IndexedDB locally, with cloud replication coming very soon.
+
+### Pre-beta Software
+
+While the underlying data structures and libraries Fireproof uses are trusted with billions of dollars worth of data, Fireproof started in February of 2023. Results may vary.
+
+## Thanks 🙏
+
+Fireproof is a synthesis of work done by people in the web community over the years. I couldn't even begin to name all the folks who made pivotal contributions. Without npm, React, and VS Code all this would have taken so much longer. Thanks to everyone who supported me getting into database development via Apache CouchDB, one of the original document databases. The distinguishing work on immutable datastructures comes from the years of consideration [IPFS](https://ipfs.tech), [IPLD](https://ipld.io), and the [Filecoin APIs](https://docs.filecoin.io) have enjoyed.
+
+Thanks to Alan Shaw and Mikeal Rogers without whom this project would have never got started. The core Merkle hash-tree clock is based on [Alan's Pail](https://github.com/alanshaw/pail), and you can see the repository history goes all the way back to work begun as a branch of that repo. Mikeal wrote [the prolly trees implementation](https://github.com/mikeal/prolly-trees).
+
+## Quick Start
+
+Look in the `examples/` directory for projects using the database. It's not picky how you use it, but we want to provide convenient jumping off places. Think of the examples as great to fork when starting your next project.
+
+If are adding Fireproof to an existing page, just install it and try some operations.
+
+```sh
+npm install @fireproof/core
+```
+
+In your `app.js` or `app.tsx` file:
+
+```js
+import { Fireproof } from '@fireproof/core'
+const fireproof = Fireproof.storage()
+const ok = await fireproof.put({ hello: 'world' })
+const doc = await fireproof.get(ok.id)
+```
+
+🤫 I like to drop a `window.fireproof = fireproof` in there as a development aid.
+
+# Contributing
+
+Feel free to join in. All welcome. [Open an issue](https://github.com/jchris/fireproof/issues)!
+
+# License
+
+Dual-licensed under [MIT or Apache 2.0](https://github.com/jchris/fireproof/blob/main/LICENSE.md)

package/hooks/use-fireproof.ts

@@ -2,7 +2,8 @@
 // eslint-disable-next-line @typescript-eslint/ban-ts-comment
 // @ts-ignore
 import { useEffect, useState, createContext } from 'react'
-import { Fireproof, Listener } from '../index'
+import { Fireproof, Listener, Hydrator } from '../index'
+
 
 export interface FireproofCtxValue {
   addSubscriber: (label: String, fn: Function) => void
@@ -18,6 +19,7 @@ export const FireproofCtx = createContext<FireproofCtxValue>({
 const inboundSubscriberQueue = new Map()
 const database = Fireproof.storage()
 const listener = new Listener(database)
+let startedSetup = false;
 
 /**
  * @function useFireproof
@@ -30,16 +32,15 @@ export function useFireproof(defineDatabaseFn: Function, setupDatabaseFn: Functi
   const [ready, setReady] = useState(false)
   defineDatabaseFn = defineDatabaseFn || (() => {})
   setupDatabaseFn = setupDatabaseFn || (() => {})
+  // console.log('useFireproof', database, ready)
 
-  if (!ready) {
-    defineDatabaseFn(database)
-  }
 
   const addSubscriber = (label: String, fn: Function) => {
     inboundSubscriberQueue.set(label, fn)
   }
 
   const listenerCallback = async () => {
+    // console.log ('listenerCallback', JSON.stringify(database))
     localSet('fireproof', JSON.stringify(database))
     for (const [, fn] of inboundSubscriberQueue) fn()
   }
@@ -47,20 +48,25 @@ export function useFireproof(defineDatabaseFn: Function, setupDatabaseFn: Functi
   useEffect(() => {
     const doSetup = async () => {
       if (ready) return
+      if (startedSetup) return
+      startedSetup = true
+      defineDatabaseFn(database) // define indexes before querying them
       const fp = localGet('fireproof')
       if (fp) {
-        const { clock } = JSON.parse(fp)
+        const serialized = JSON.parse(fp)
+        // console.log('serialized', JSON.stringify(serialized.indexes.map(c => c.clock)))
         console.log("Loading previous database clock. (localStorage.removeItem('fireproof') to reset)")
-        await database.setClock(clock)
+        Hydrator.fromJSON(serialized, database)
+        // await database.setClock(clock)
         try {
           const changes = await database.changesSince()
           if (changes.rows.length < 2) {
-            console.log('Resetting database')
+            // console.log('Resetting database')
             throw new Error('Resetting database')
           }
         } catch (e) {
           console.error(`Error loading previous database clock. ${fp} Resetting.`, e)
-          await database.setClock([])
+          await database.setClock([]) // todo this should be resetClock and also reset the indexes
           await setupDatabaseFn(database)
           localSet('fireproof', JSON.stringify(database))
         }

package/index.js

@@ -1,5 +1,6 @@
 import Fireproof from './src/fireproof'
 import Index from './src/db-index'
 import Listener from './src/listener'
+import Hydrator from './src/hydrator'
 
-export { Fireproof, Index, Listener }
+export { Fireproof, Index, Listener, Hydrator }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@fireproof/core",
-  "version": "0.0.8",
+  "version": "0.0.10",
   "description": "Realtime database for IPFS",
   "main": "index.js",
   "type": "module",
@@ -9,6 +9,7 @@
     "test:mocha": "mocha test/*.test.js",
     "test:watch": "npm run test:mocha -- -w --parallel test/*.test.js",
     "coverage": "c8 -r html -r text npm test",
+    "prepublishOnly" : "cp ../../README.md .",
     "lint": "standard",
     "lint:fix": "standard --fix"
   },

package/src/db-index.js

@@ -1,12 +1,11 @@
-// import { create, load } from 'prolly-trees/db-index'
-import { create, load } from '../../../../prolly-trees/src/db-index.js'
+import { create, load } from 'prolly-trees/db-index'
+// import { create, load } from '../../../../prolly-trees/src/db-index.js'
 
 import { sha256 as hasher } from 'multiformats/hashes/sha2'
 import { nocache as cache } from 'prolly-trees/cache'
 import { bf, simpleCompare } from 'prolly-trees/utils'
 import { makeGetBlock } from './prolly.js'
 import { cidsToProof } from './fireproof.js'
-import { CID } from 'multiformats'
 
 import * as codec from '@ipld/dag-cbor'
 // import { create as createBlock } from 'multiformats/block'
@@ -73,16 +72,16 @@ const makeDoc = ({ key, value }) => ({ _id: key, ...value })
  * Transforms a set of changes to DbIndex entries using a map function.
  *
  * @param {ChangeEvent[]} changes
- * @param {Function} mapFun
+ * @param {Function} mapFn
  * @returns {DbIndexEntry[]} The DbIndex entries generated by the map function.
  * @private
  * @memberof DbIndex
  */
-const indexEntriesForChanges = (changes, mapFun) => {
+const indexEntriesForChanges = (changes, mapFn) => {
   const indexEntries = []
   changes.forEach(({ key, value, del }) => {
     if (del || !value) return
-    mapFun(makeDoc({ key, value }), (k, v) => {
+    mapFn(makeDoc({ key, value }), (k, v) => {
       indexEntries.push({
         key: [charwise.encode(k), key],
         value: v
@@ -99,11 +98,12 @@ const indexEntriesForChanges = (changes, mapFun) => {
  * @classdesc An DbIndex can be used to order and filter the documents in a Fireproof database.
  *
  * @param {Fireproof} database - The Fireproof database instance to DbIndex.
- * @param {Function} mapFun - The map function to apply to each entry in the database.
+ * @param {Function} mapFn - The map function to apply to each entry in the database.
  *
  */
 export default class DbIndex {
-  constructor (database, mapFun) {
+  constructor (database, mapFn, clock) {
+    // console.log('DbIndex constructor', database.constructor.name, typeof mapFn, clock)
     /**
      * The database instance to DbIndex.
      * @type {Fireproof}
@@ -113,33 +113,62 @@ export default class DbIndex {
      * The map function to apply to each entry in the database.
      * @type {Function}
      */
-    this.mapFun = mapFun
-
-    this.database.indexes.set(mapFun.toString(), this)
 
+    if (typeof mapFn === 'string') {
+      this.mapFnString = mapFn
+    } else {
+      this.mapFn = mapFn
+      this.mapFnString = mapFn.toString()
+    }
     this.indexById = { root: null, cid: null }
     this.indexByKey = { root: null, cid: null }
-
     this.dbHead = null
-
+    if (clock) {
+      this.indexById.cid = clock.byId
+      this.indexByKey.cid = clock.byKey
+      this.dbHead = clock.db
+    }
     this.instanceId = this.database.instanceId + `.DbIndex.${Math.random().toString(36).substring(2, 7)}`
-
     this.updateIndexPromise = null
+    DbIndex.registerWithDatabase(this, this.database)
+  }
+
+  static registerWithDatabase (inIndex, database) {
+    // console.log('.reg > in Index', inIndex.instanceId, { live: !!inIndex.mapFn }, inIndex.indexByKey, inIndex.mapFnString)
+    if (database.indexes.has(inIndex.mapFnString)) {
+      // merge our inIndex code with the inIndex clock or vice versa
+      // keep the code instance, discard the clock instance
+      const existingIndex = database.indexes.get(inIndex.mapFnString)
+      // console.log('.reg - existingIndex', existingIndex.instanceId, { live: !!inIndex.mapFn }, existingIndex.indexByKey)
+      if (existingIndex.mapFn) { // this one also has other config
+        existingIndex.dbHead = inIndex.dbHead
+        existingIndex.indexById.cid = inIndex.indexById.cid
+        existingIndex.indexByKey.cid = inIndex.indexByKey.cid
+      } else {
+        // console.log('.reg use inIndex with existingIndex clock')
+        inIndex.dbHead = existingIndex.dbHead
+        inIndex.indexById.cid = existingIndex.indexById.cid
+        inIndex.indexByKey.cid = existingIndex.indexByKey.cid
+        database.indexes.set(inIndex.mapFnString, inIndex)
+      }
+    } else {
+      // console.log('.reg - fresh')
+      database.indexes.set(inIndex.mapFnString, inIndex)
+    }
+    // console.log('.reg after', JSON.stringify([...database.indexes.values()].map(i => [i.instanceId, typeof i.mapFn, i.indexByKey, i.indexById])))
   }
 
   toJSON () {
-    return { code: this.mapFun?.toString(), clock: { db: this.dbHead?.map(cid => cid.toString()), byId: this.indexById.cid?.toString(), byKey: this.indexByKey.cid?.toString() } }
+    const indexJson = { code: this.mapFn?.toString(), clock: { db: null, byId: null, byKey: null } }
+    indexJson.clock.db = this.dbHead?.map(cid => cid.toString())
+    indexJson.clock.byId = this.indexById.cid?.toString()
+    indexJson.clock.byKey = this.indexByKey.cid?.toString()
+    return indexJson
   }
 
-  static fromJSON (database, { code, clock: { byId, byKey, db } }) {
-    let mapFun
-    // eslint-disable-next-line
-    eval("mapFun = "+ code)
-    const index = new DbIndex(database, mapFun)
-    index.indexById.cid = CID.parse(byId)
-    index.indexByKey.cid = CID.parse(byKey)
-    index.dbHead = db.map(cid => CID.parse(cid))
-    return index
+  static fromJSON (database, { code, clock }) {
+    // console.log('DbIndex.fromJSON', database.constructor.name, code, clock)
+    return new DbIndex(database, code, clock)
   }
 
   /**
@@ -166,6 +195,7 @@ export default class DbIndex {
 
     // }
     // console.time(callId + '.doIndexQuery')
+    // console.log('query', query)
     const response = await doIndexQuery(this.database.blocks, this.indexByKey, query)
     // console.timeEnd(callId + '.doIndexQuery')
 
@@ -196,12 +226,12 @@ export default class DbIndex {
 
   async #innerUpdateIndex (inBlocks) {
     // const callTag = Math.random().toString(36).substring(4)
-    // console.log(`#updateIndex ${callTag} >`, this.instanceId, this.dbHead?.toString(), this.dbIndexRoot?.cid.toString(), this.indexByIdRoot?.cid.toString())
+    // console.log(`#updateIndex ${callTag} >`, this.instanceId, this.dbHead?.toString(), this.indexByKey.cid?.toString(), this.indexById.cid?.toString())
     // todo remove this hack
     if (ALWAYS_REBUILD) {
-      this.dbHead = null // hack
-      this.indexByKey = null // hack
-      this.dbIndexRoot = null
+      this.indexById = { root: null, cid: null }
+      this.indexByKey = { root: null, cid: null }
+      this.dbHead = null
     }
     // console.log('dbHead', this.dbHead)
     // console.time(callTag + '.changesSince')
@@ -210,28 +240,34 @@ export default class DbIndex {
     // console.log('result.rows.length', result.rows.length)
 
     // console.time(callTag + '.doTransaction#updateIndex')
+    // console.log('#updateIndex changes length', result.rows.length)
 
     if (result.rows.length === 0) {
-      // console.log('#updateIndex < no changes')
+      // console.log('#updateIndex < no changes', result.clock)
       this.dbHead = result.clock
       return
     }
     await doTransaction('#updateIndex', inBlocks, async (blocks) => {
       let oldIndexEntries = []
       let removeByIdIndexEntries = []
-      if (this.dbHead) { // need a maybe load
+      await loadIndex(blocks, this.indexById, idIndexOpts)
+      await loadIndex(blocks, this.indexByKey, dbIndexOpts)
+      if (this.dbHead) {
         const oldChangeEntries = await this.indexById.root.getMany(result.rows.map(({ key }) => key))
         oldIndexEntries = oldChangeEntries.result.map((key) => ({ key, del: true }))
         removeByIdIndexEntries = oldIndexEntries.map(({ key }) => ({ key: key[1], del: true }))
       }
-      const indexEntries = indexEntriesForChanges(result.rows, this.mapFun)
+      if (!this.mapFn) {
+        throw new Error('No live map function installed for index, cannot update. Make sure your index definition runs before any queries.' + (this.mapFnString ? ' Your code should match the stored map function source:\n' + this.mapFnString : ''))
+      }
+      const indexEntries = indexEntriesForChanges(result.rows, this.mapFn)
       const byIdIndexEntries = indexEntries.map(({ key }) => ({ key: key[1], value: key }))
       this.indexById = await bulkIndex(blocks, this.indexById, removeByIdIndexEntries.concat(byIdIndexEntries), idIndexOpts)
       this.indexByKey = await bulkIndex(blocks, this.indexByKey, oldIndexEntries.concat(indexEntries), dbIndexOpts)
       this.dbHead = result.clock
     })
     // console.timeEnd(callTag + '.doTransaction#updateIndex')
-    // console.log(`#updateIndex ${callTag} <`, this.instanceId, this.dbHead?.toString(), this.dbIndexRoot?.cid.toString(), this.indexByIdRoot?.cid.toString())
+    // console.log(`#updateIndex ${callTag} <`, this.instanceId, this.dbHead?.toString(), this.indexByKey.cid?.toString(), this.indexById.cid?.toString())
   }
 }
 
@@ -271,13 +307,18 @@ async function bulkIndex (blocks, inIndex, indexEntries, opts) {
   return { root: returnNode, cid: returnRootBlock.cid }
 }
 
-async function doIndexQuery (blocks, indexByKey, query) {
-  if (!indexByKey.root) {
-    const cid = indexByKey.cid
-    if (!cid) return { result: [] }
+async function loadIndex (blocks, index, indexOpts) {
+  if (!index.root) {
+    const cid = index.cid
+    if (!cid) return
     const { getBlock } = makeGetBlock(blocks)
-    indexByKey.root = await load({ cid, get: getBlock, ...dbIndexOpts })
+    index.root = await load({ cid, get: getBlock, ...indexOpts })
   }
+  return index.root
+}
+
+async function doIndexQuery (blocks, indexByKey, query) {
+  await loadIndex(blocks, indexByKey, dbIndexOpts)
   if (query.range) {
     const encodedRange = query.range.map((key) => charwise.encode(key))
     return indexByKey.root.range(...encodedRange)

package/src/fireproof.js

@@ -42,22 +42,27 @@ export default class Fireproof {
   }
 
   /**
-   * Returns a snapshot of the current Fireproof instance as a new instance.
-   * @function snapshot
-   * @param {CID[]} clock - The Merkle clock head to use for the snapshot.
-   * @returns {Fireproof}
-   *    A new Fireproof instance representing the snapshot.
+   * Renders the Fireproof instance as a JSON object.
+   * @returns {Object} - The JSON representation of the Fireproof instance. Includes clock heads for the database and its indexes.
    * @memberof Fireproof
    * @instance
    */
-  snapshot (clock) {
-    // how to handle listeners, views, and config?
-    // todo needs a test for listeners, views, and config
-    return new Fireproof(this.blocks, clock || this.clock)
+  toJSON () {
+    // todo this also needs to return the index roots...
+    return {
+      clock: this.clock.map(cid => cid.toString()),
+      name: this.name,
+      indexes: [...this.indexes.values()].map(index => index.toJSON())
+    }
+  }
+
+  hydrate ({ clock, name }) {
+    this.name = name
+    this.clock = clock
   }
 
   /**
-   * Move the current instance to a new point in time. This triggers a notification to all listeners
+   * Triggers a notification to all listeners
    * of the Fireproof instance so they can repaint UI, etc.
    * @param {CID[] } clock
    *    Clock to use for the snapshot.
@@ -65,25 +70,8 @@ export default class Fireproof {
    * @memberof Fireproof
    * @instance
    */
-  async setClock (clock) {
-    // console.log('setClock', this.instanceId, clock)
-    this.clock = clock.map((item) => (item['/'] ? item['/'] : item))
-    await this.#notifyListeners({ reset: true, clock })
-  }
-
-  /**
-   * Renders the Fireproof instance as a JSON object.
-   * @returns {Object} - The JSON representation of the Fireproof instance. Includes clock heads for the database and its indexes.
-   * @memberof Fireproof
-   * @instance
-   */
-  toJSON () {
-    // todo this also needs to return the index roots...
-    return {
-      clock: this.clock.map(cid => cid.toString()),
-      name: this.name,
-      indexes: [...this.indexes.values()].map((index) => index.toJSON())
-    }
+  async notifyReset () {
+    await this.#notifyListeners({ reset: true, clock: this.clock })
   }
 
   /**
@@ -295,7 +283,7 @@ export default class Fireproof {
   }
 
   setCarUploader (carUploaderFn) {
-    console.log('registering car uploader')
+    // console.log('registering car uploader')
     // https://en.wikipedia.org/wiki/Law_of_Demeter - this is a violation of the law of demeter
     this.blocks.valet.uploadFunction = carUploaderFn
   }

package/src/hydrator.js

@@ -1,10 +1,51 @@
-import Fireproof from './fireproof.js'
 import DbIndex from './db-index.js'
+import Fireproof from './fireproof.js'
+import { CID } from 'multiformats'
+
+const parseCID = cid => typeof cid === 'string' ? CID.parse(cid) : cid
+
+export default class Hydrator {
+  static fromJSON (json, database) {
+    database.hydrate({ clock: json.clock.map(c => parseCID(c)), name: json.name })
+    for (const { code, clock: { byId, byKey, db } } of json.indexes) {
+      DbIndex.fromJSON(database, {
+        clock: {
+          byId: byId ? parseCID(byId) : null,
+          byKey: byKey ? parseCID(byKey) : null,
+          db: db ? db.map(c => parseCID(c)) : null
+        },
+        code
+      })
+    }
+    return database
+  }
+
+  static snapshot (database, clock) {
+    const definition = database.toJSON()
+    const withBlocks = new Fireproof(database.blocks)
+    if (clock) {
+      definition.clock = clock.map(c => parseCID(c))
+      definition.indexes.forEach(index => {
+        index.clock.byId = null
+        index.clock.byKey = null
+        index.clock.db = null
+      })
+    }
+    const snappedDb = this.fromJSON(definition, withBlocks)
+    ;([...database.indexes.values()]).forEach(index => {
+      snappedDb.indexes.get(index.mapFnString).mapFn = index.mapFn
+    })
+    return snappedDb
+  }
 
-export function fromJSON (json, blocks) {
-  const fp = new Fireproof(blocks, json.clock, { name: json.name })
-  for (const index of json.indexes) {
-    DbIndex.fromJSON(fp, index)
+  static async zoom (database, clock) {
+    ;([...database.indexes.values()]).forEach(index => {
+      index.indexById = { root: null, cid: null }
+      index.indexByKey = { root: null, cid: null }
+      index.dbHead = null
+    })
+    database.clock = clock.map(c => parseCID(c))
+    await database.notifyReset()
+    return database
   }
-  return fp
 }

package/test/db-index.test.js

@@ -3,6 +3,7 @@ import assert from 'node:assert'
 import Blockstore from '../src/blockstore.js'
 import Fireproof from '../src/fireproof.js'
 import DbIndex from '../src/db-index.js'
+import Hydrator from '../src/hydrator.js'
 console.x = function () {}
 
 describe('DbIndex query', () => {
@@ -77,7 +78,7 @@ describe('DbIndex query', () => {
     // console.x('bresult.rows', bresult.rows)
     assert.equal(bresult.rows.length, 6, 'all row matched')
 
-    const oldHead = database.clock
+    const snapClock = database.clock
 
     const notYet = await database.get('xxxx-3c3a-4b5e-9c1c-8c5c0c5c0c5c').catch((e) => e)
     assert.equal(notYet.message, 'Not found', 'not yet there')
@@ -91,7 +92,7 @@ describe('DbIndex query', () => {
     assert(gotX.name === 'Xander', 'got Xander')
     console.x('got X')
 
-    const snap = database.snapshot(oldHead)
+    const snap = Hydrator.snapshot(database, snapClock)
 
     const aliceOld = await snap.get('a1s3b32a-3c3a-4b5e-9c1c-8c5c0c5c0c5c')// .catch((e) => e)
     console.x('aliceOld', aliceOld)
@@ -124,7 +125,7 @@ describe('DbIndex query', () => {
     assert.equal(result.rows.length, 1, '1 row matched')
     assert(result.rows[0].key === 53, 'correct key')
 
-    const snap = database.snapshot(database.clock)
+    const snap = Hydrator.snapshot(database)
 
     console.x('--- make Xander 63')
     const response = await database.put({ _id: DOCID, name: 'Xander', age: 63 })
@@ -172,7 +173,7 @@ describe('DbIndex query', () => {
     assert.equal(result.rows.length, 1, '1 row matched')
     assert(result.rows[0].key === 53, 'correct key')
 
-    const snap = database.snapshot(database.clock)
+    const snap = Hydrator.snapshot(database)
 
     console.x('--- delete Xander 53')
     const response = await database.del(DOCID)

package/test/fireproof.test.js

@@ -2,6 +2,7 @@ import { describe, it, beforeEach } from 'mocha'
 import assert from 'node:assert'
 import Blockstore from '../src/blockstore.js'
 import Fireproof from '../src/fireproof.js'
+import Hydrator from '../src/hydrator.js'
 // import * as codec from '@ipld/dag-cbor'
 
 let database, resp0
@@ -130,7 +131,7 @@ describe('Fireproof', () => {
     assert(response.id, 'should have id')
     assert.equal(response.id, dogKey)
     assert.equal(value._id, dogKey)
-    const oldClock = database.clock
+    const snapshot = Hydrator.snapshot(database)
 
     const avalue = await database.get(dogKey)
     assert.equal(avalue.name, value.name)
@@ -147,7 +148,6 @@ describe('Fireproof', () => {
     assert.equal(bvalue.age, 3)
     assert.equal(bvalue._id, dogKey)
 
-    const snapshot = database.snapshot(oldClock)
     const snapdoc = await snapshot.get(dogKey)
     // console.log('snapdoc', snapdoc)
     // assert(snapdoc.id, 'should have id')