@1-/scan 0.1.9 → 0.1.11

package/README.md

@@ -3,197 +3,175 @@
 ---
 
 <a id="en"></a>
-# @1-/scan : SQLite-backed incremental directory file scanner
+# scan : Efficient file system scanning and change detection
 
-Incrementally scans directory files, compares file sizes and modification times to detect changes, synchronizes metadata to SQLite database, and returns list of changed relative paths.
+- [scan : Efficient file system scanning and change detection](#scan-efficient-file-system-scanning-and-change-detection)
+  - [Functionality](#functionality)
+  - [Usage demonstration](#usage-demonstration)
+  - [Design rationale](#design-rationale)
+  - [Technology stack](#technology-stack)
+  - [Code structure](#code-structure)
+  - [Historical context](#historical-context)
+  - [About](#about)
 
-## 1. Features
+## Functionality
 
-- **Incremental Scan**: Compares size and modification time, filtering unchanged files to reduce disk I/O.
-- **Key Length Optimization**: Stores raw bytes for paths up to 16 bytes. Converts longer paths into 16-byte MD5 hashes to optimize database index space and query performance.
-- **Memory Optimization**: Uses BinMap and BinSet to store binary keys in memory, avoiding string decoding overhead and reducing memory footprint.
-- **Transactional Integrity**: Performs metadata updates and deletions in database transactions to ensure consistency.
-- **Auto Configuration**: Integrates @1-/sqlite to initialize database schema and manage database connections automatically, updating .gitignore when new database is detected.
+This utility scans directories to detect file changes by comparing current file metadata against cached records. It tracks file size, modification time, and MD5 hashes to identify additions, modifications, and deletions efficiently.
 
-## 2. Usage
+The system uses binary data structures for memory efficiency and supports concurrent scanning with automatic parallelism adjustment based on available CPU cores.
 
-### Basic Incremental Scan
+## Usage demonstration
 
-```javascript
-import scan from "@1-/scan";
-
-const dir = "./data";
-const db_path = "./scan_record.db";
-const files = ["file1.txt", "file2.txt"];
-
-// Scan file list, sync metadata to SQLite, return changed relative paths and upsert function
-const [updated_paths, upsert] = await scan(dir, db_path, files);
-
-// Close database automatically when exiting scope
-using _ = upsert;
+Install as a dependency:
 
-console.log("Updated files:", updated_paths);
-
-// Update scanned file metadata in database
-for (const rel_path of updated_paths) {
-  await upsert(rel_path);
-}
+```bash
+npm install @1-/scan
 ```
 
-### Bulk Storage Module
+Basic usage:
 
 ```javascript
-import save from "@1-/scan/save.js";
-import sqlite from "@1-/sqlite";
+import scan from '@1-/scan';
 
-const db = sqlite("./scan_record.db");
+// Scan directory and get update list
+const [updateFiles, upsert] = await scan('/path/to/dir', '/path/to/db', ['file1.js', 'file2.json']);
 
-// Bulk update and delete metadata
-save(db, [["file.txt", new Uint8Array([1, 2, 3]), 123, 1620000000]], [new Uint8Array([4, 5, 6])]);
+console.log('Files that need updating:', updateFiles);
 
-db.close();
+// Save the updated metadata to database
+await upsert();
 ```
 
-## 3. Design
+## Design rationale
 
-Main entry orchestrates modules to scan directories and synchronize metadata.
+The architecture prioritizes efficiency through several key design decisions:
 
-![](https://fastly.jsdelivr.net/gh/webc-fs/-@K0/4mOzfZ1TGQuIAaily4Jw.svg)
+- Binary data structures (BinSet, BinMap) minimize memory overhead
+- Base64url encoding for path keys enables compact storage
+- Concurrent scanning with dynamic parallelism limits
+- Two-phase comparison: quick metadata check followed by expensive MD5 verification only when needed
+- CSV-based persistent storage for simplicity and portability
 
-1. **Initialize Connection**: Calls `@1-/sqlite` to open SQLite database. Updates `.gitignore` in the database directory if the database is newly created to prevent tracking.
-2. **Load Records**: `load.js` checks and creates `scanMtimeLen` table. Reads stored hashes, sizes, and modification times to restore memory mappings inside `BinMap`.
-3. **Compare Files**: `scan.js` iterates over input file list, calling `stat.js` for metadata and utilizing `@1-/hash` to map paths to 16-byte binary keys. Adds files with mismatched size or modification time to change list.
-4. **Delete and Return**: `rm.js` deletes absent or unscanned records in transaction. Returns changed paths list and `upsert` function (provided by `upsert.js`) for persistence, supporting automatic resource disposal.
+![](https://fastly.jsdelivr.net/gh/webc-fs/-@B4/0t9t7yVs82Pn-_d5eXrw.svg)
 
-## 4. Tech Stack
+## Technology stack
 
-- **Bun**: Runtime environment and test framework.
-- **@1-/sqlite**: Database connection management and transaction wrapper.
-- **@1-/hash**: Length-bounded MD5 hash utility.
-- **@3-/vb**: Variable-length byte (Varint) encoder and decoder.
-- **@3-/binmap / @3-/binset**: Rust and WebAssembly binary key containers.
+- Node.js runtime with modern ES modules
+- Binary data structures: `@3-/binset`, `@3-/binmap`
+- Base64url encoding: `@3-/base64url`
+- File hashing: `@1-/md5`
+- CSV processing: `@1-/csv`
+- Gitignore management: `@1-/upsert_gitignore`
+- Concurrency control: `@3-/plimit`
 
-## 5. Code Structure
+## Code structure
 
-```text
-.
-├── src
-│   ├── _.js          # Core controller flow
-│   ├── load.js       # Table initialization and loading
-│   ├── rm.js         # Batch deletion of metadata
-│   ├── save.js       # Batch storage and updates
-│   ├── scan.js       # Scans and compares files
-│   ├── stat.js       # Retrieves file metadata and path hash
-│   └── upsert.js     # Single-record updates and auto-dispose
-└── tests             # Unit tests
+```
+src/
+├── _.js          # Main entry point and exports
+├── const.js      # Constants (database filenames)
+├── dbInit.js     # Database initialization and loading
+├── rm.js         # File removal from database
+├── scan.js       # Core scanning logic
+├── stat.js       # File system statistics collection
+├── upsert.js     # Database persistence logic
+└── test/         # Test files
 ```
 
-## 6. History
-
-SQLite was created by D. Richard Hipp in 2000 while designing board software for guided-missile destroyers. The system originally depended on commercial database that required constant database administration; connection loss could stall the entire damage control application. Hipp designed serverless, zero-configuration embedded database that directly reads and writes local files, marking the birth of SQLite.
+## Historical context
 
-To conserve space and reduce latency, SQLite utilizes Varint (variable-length integer) encoding for metadata storage. Under this scheme, small integers consume only 1 byte, while larger numbers scale dynamically. This library inherits that design philosophy, compressing file metadata into varints for memory storage to ensure minimal footprint and high synchronization performance.
+File scanning utilities trace their origins to early Unix tools like `find` and `diff`. Modern implementations face new challenges with massive file systems and cloud storage. This implementation draws inspiration from incremental backup systems developed in the 1990s, which pioneered the two-phase comparison approach (quick metadata check followed by content verification) to balance speed and accuracy. The use of binary data structures reflects contemporary optimizations for memory-constrained environments and high-performance computing scenarios.
 ## About
 
 This library is developed by [WebC.site](https://webc.site).
 
 [WebC.site](https://webc.site): A new paradigm of web development for AI
 
+
 ---
 
 <a id="zh"></a>
-# @1-/scan : 基于 SQLite 的目录文件增量扫描器
+# scan : 高效的文件系统扫描与变更检测
 
-增量扫描目录文件，比对文件大小与修改时间检测变更，同步元数据至 SQLite 数据库，返回已变更相对路径列表。
+- [scan : 高效的文件系统扫描与变更检测](#scan-高效的文件系统扫描与变更检测)
+  - [功能介绍](#功能介绍)
+  - [使用演示](#使用演示)
+  - [设计思路](#设计思路)
+  - [技术栈](#技术栈)
+  - [代码结构](#代码结构)
+  - [历史故事](#历史故事)
+  - [关于](#关于)
 
-## 1. 功能介绍
+## 功能介绍
 
-- **增量扫描**：比对大小与修改时间，过滤未变更文件，减少磁盘读写。
-- **键长优化**：路径长度不大于 16 字节时存储原始字节，超出 16 字节转换为 16 字节 MD5 值，优化索引空间与查询性能。
-- **内存优化**：使用 BinMap 与 BinSet 存储二进制键，避免字符串解码，降低内存占用。
-- **事务保障**：元数据变更与删除操作合并在数据库事务中执行，确保数据一致性。
-- **自动配置**：集成 @1-/sqlite，自动初始化数据库表结构，并在检测到新数据库时自动更新 .gitignore。
+本工具通过比对当前文件元数据与缓存记录，扫描目录以检测文件变更。系统跟踪文件大小、修改时间及MD5哈希值，高效识别新增、修改和删除操作。
 
-## 2. 使用演示
+采用二进制数据结构实现内存效率优化，并支持基于可用CPU核心数自动调整的并发扫描。
 
-### 基础增量扫描
+## 使用演示
 
-```javascript
-import scan from "@1-/scan";
-
-const dir = "./data";
-const db_path = "./scan_record.db";
-const files = ["file1.txt", "file2.txt"];
-
-// 扫描文件列表并同步至 SQLite，返回已变更的相对路径列表与更新函数
-const [updated_paths, upsert] = await scan(dir, db_path, files);
-
-// 退出作用域自动关闭数据库
-using _ = upsert;
+安装为依赖项：
 
-console.log("更新文件列表：", updated_paths);
-
-// 更新已处理文件的元数据至数据库
-for (const rel_path of updated_paths) {
-  await upsert(rel_path);
-}
+```bash
+npm install @1-/scan
 ```
 
-### 独立批量存储模块
+基础用法：
 
 ```javascript
-import save from "@1-/scan/save.js";
-import sqlite from "@1-/sqlite";
+import scan from '@1-/scan';
 
-const db = sqlite("./scan_record.db");
+// 扫描目录并获取更新列表
+const [updateFiles, upsert] = await scan('/path/to/dir', '/path/to/db', ['file1.js', 'file2.json']);
 
-// 批量更新与删除元数据
-save(db, [["file.txt", new Uint8Array([1, 2, 3]), 123, 1620000000]], [new Uint8Array([4, 5, 6])]);
+console.log('需要更新的文件:', updateFiles);
 
-db.close();
+// 将更新后的元数据保存至数据库
+await upsert();
 ```
 
-## 3. 设计思路
+## 设计思路
 
-主入口调度各模块，协作完成目录扫描与数据同步。
+架构设计优先考虑效率，关键决策包括：
 
-![](https://fastly.jsdelivr.net/gh/webc-fs/-@EB/fMlrHP9fXkA64lXr2r2Q.svg)
+- 二进制数据结构（BinSet、BinMap）最小化内存开销
+- 路径键使用base64url编码实现紧凑存储
+- 并发扫描配合动态并行度限制
+- 两阶段比对：快速元数据检查后仅在必要时执行耗时MD5验证
+- 基于CSV的持久化存储确保简单性和可移植性
 
-1. **初始化连接**：调用 `@1-/sqlite` 打开 SQLite 数据库。若数据库为新创建，自动更新所在目录的 `.gitignore` 阻断提交。
-2. **加载记录**：`load.js` 检查并创建 `scanMtimeLen` 表。读取已记录的哈希、大小及修改时间，恢复至内存映射 `BinMap` 中。
-3. **比对文件**：`scan.js` 遍历输入文件列表，调用 `stat.js` 获取元数据，并利用 `@1-/hash` 将路径映射为 16 字节二进制键。比对大小或修改时间，差异项归入变更列表。
-4. **删除与返回**：`rm.js` 在事务中批量删除物理移除或不再扫描的记录。返回变更路径列表与 `upsert` 函数（`upsert.js` 提供），用以更新数据库，支持自动释放。
+![](https://fastly.jsdelivr.net/gh/webc-fs/-@He/ywew0gcPy6M8EMHRuoHw.svg)
 
-## 4. 技术栈
+## 技术栈
 
-- **Bun**：JavaScript 运行时与测试框架。
-- **@1-/sqlite**：SQLite 连接管理与事务封装。
-- **@1-/hash**：长度限制的 MD5 哈希工具。
-- **@3-/vb**：Varint 变长整型编码与解码器。
-- **@3-/binmap / @3-/binset**：基于 Rust 与 WebAssembly 的高效二进制键容器。
+- Node.js运行时，支持现代ES模块
+- 二进制数据结构：`@3-/binset`、`@3-/binmap`
+- Base64url编码：`@3-/base64url`
+- 文件哈希：`@1-/md5`
+- CSV处理：`@1-/csv`
+- Gitignore管理：`@1-/upsert_gitignore`
+- 并发控制：`@3-/plimit`
 
-## 5. 代码结构
+## 代码结构
 
-```text
-.
-├── src
-│   ├── _.js          # 核心控制流程
-│   ├── load.js       # 元数据表初始化与加载
-│   ├── rm.js         # 批量删除元数据
-│   ├── save.js       # 批量存储元数据
-│   ├── scan.js       # 扫描与比对文件
-│   ├── stat.js       # 获取文件元数据及路径哈希
-│   └── upsert.js     # 逐个更新与自动关闭
-└── tests             # 单元测试
+```
+src/
+├── _.js          # 主入口点及导出
+├── const.js      # 常量定义（数据库文件名）
+├── dbInit.js     # 数据库初始化与加载
+├── rm.js         # 从数据库移除文件
+├── scan.js       # 核心扫描逻辑
+├── stat.js       # 文件系统统计信息收集
+├── upsert.js     # 数据库持久化逻辑
+└── test/         # 测试文件
 ```
 
-## 6. 历史故事
-
-SQLite 的诞生源自导弹驱逐舰板载损害控制软件项目。2000 年，D. Richard Hipp 为美国海军设计该系统时，遭遇商业数据库因配置复杂、无法承受断连和崩溃之痛点。Hipp 随后设计出免服务器配置、直接读写本地文件之嵌入式数据库，即 SQLite。
+## 历史故事
 
-为节省空间与降低读写延迟，SQLite 广泛应用了 Varint（可变字节整型）编码。在这种编码下，小整数仅占用 1 字节，只有大数值才占用更多字节。本项目在内存存储设计中对文件大小与修改时间采用同样的压缩设计，契合 SQLite 节省空间与高效之设计哲学。
+文件扫描工具源于早期Unix命令如`find`和`diff`。现代实现面临海量文件系统和云存储的新挑战。本实现借鉴了20世纪90年代开发的增量备份系统，该系统首创两阶段比对方法（先快速元数据检查，再按需内容验证），在速度与准确性间取得平衡。二进制数据结构的采用反映了当代针对内存受限环境和高性能计算场景的优化趋势。
 ## 关于
 
 本库由 [WebC.site](https://webc.site) 开发。
 
 [WebC.site](https://webc.site) : 面向人工智能的网站开发新范式
+

package/_.js

@@ -1,31 +1,31 @@
-import sqlite from "@1-/sqlite";
-import { BinMap } from "@3-/binmap";
-import vbE from "@3-/vb/vbE.js";
 import { availableParallelism } from "node:os";
 import pLimit from "@3-/plimit";
-import upsertGitignore from "@1-/upsert_gitignore";
-import { existsSync } from "node:fs";
-import { join, dirname, basename } from "node:path";
-import load from "./load.js";
+import dbInit from "./dbInit.js";
 import scan from "./scan.js";
 import rm from "./rm.js";
 import upsert from "./upsert.js";
 
-export default async (dir, db_path, files) => {
-  if (!existsSync(db_path)) {
-    upsertGitignore(join(dirname(db_path), ".gitignore"), basename(db_path));
-  }
-  const db = sqlite(db_path),
-    existing = new BinMap(),
-    db_rows = load(db),
-    limit = pLimit(availableParallelism());
+/*
+扫描指定目录下文件列表，比对缓存并做清理，返回更新列表和 upsert 存储函数
+dir: 扫描的目标目录
+db_dir: 数据库存放目录
+files: 待扫描的文件列表
+返回值: [update, upsert]
+  update: 发生变动需要更新的相对路径列表
+  upsert: 用于将新扫描记录保存至数据库的 dispose 异步函数
+*/
+export default async (dir, db_dir, files) => {
+  const [db_mtime, db_md5] = await dbInit(db_dir),
+    existing_mtime = db_mtime,
+    existing_md5 = db_md5,
+    limit = pLimit(availableParallelism()),
+    [scanned, update] = await scan(dir, files, existing_mtime, existing_md5, limit, db_mtime),
+    rm_paths = (existing) => [...existing.keys()].filter((path) => !scanned.has(path));
 
-  db_rows.forEach(({ hash, size, mtime }) => existing.set(hash, vbE([size, mtime])));
+  [
+    [db_mtime, existing_mtime],
+    [db_md5, existing_md5],
+  ].forEach(([db, existing]) => rm(db, rm_paths(existing)));
 
-  const [scanned, update] = await scan(dir, files, existing, limit),
-    rm_hashes = db_rows.filter(({ hash }) => !scanned.has(hash)).map(({ hash }) => hash);
-
-  rm(db, rm_hashes);
-
-  return [update, upsert(db, dir)];
+  return [update, upsert(db_mtime, db_md5, dir, db_dir)];
 };

package/const.js

@@ -0,0 +1,2 @@
+export const MTIME = "mtime",
+  MD5 = "md5";

package/dbInit.js

@@ -0,0 +1,45 @@
+import { BinMap } from "@3-/binmap";
+import load from "@1-/csv/load.js";
+import upsertGitignore from "@1-/upsert_gitignore";
+import vbE from "@3-/vb/vbE.js";
+import b64Uint8 from "@3-/base64url/b64Uint8.js";
+import { existsSync } from "node:fs";
+import { join, basename } from "node:path";
+import { MTIME, MD5 } from "./const.js";
+
+const loadDb = async (path, db, parseVal) => {
+  if (existsSync(path)) {
+    const rows = await load(path);
+    rows.forEach(([path_str, ...vals]) => {
+      db.set(b64Uint8(path_str), parseVal(vals));
+    });
+  }
+};
+
+/*
+初始化/打开修改时间和 md5 的 csv 数据库，并将数据库文件加入 gitignore
+db_dir: 数据库存放目录
+返回值: [db_mtime, db_md5]
+*/
+export default async (db_dir) => {
+  const li = [MTIME, MD5],
+    path_li = li.map((x) => join(db_dir, x + ".csv"));
+
+  if (path_li.some((x) => !existsSync(x))) {
+    upsertGitignore(
+      join(db_dir, ".gitignore"),
+      path_li.map((x) => basename(x)),
+    );
+  }
+
+  const db_mtime = new BinMap(),
+    db_md5 = new BinMap(),
+    [mtime_path, md5_path] = path_li;
+
+  await Promise.all([
+    loadDb(mtime_path, db_mtime, ([size, mtime]) => vbE([Number(size), Number(mtime)])),
+    loadDb(md5_path, db_md5, ([md5_str]) => b64Uint8(md5_str)),
+  ]);
+
+  return [db_mtime, db_md5];
+};

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@1-/scan",
-  "version": "0.1.9",
+  "version": "0.1.11",
   "description": "Incrementally scan directory files and track metadata in SQLite / 增量扫描目录文件并使用 SQLite 记录元数据",
   "keywords": [
     "directory",
@@ -21,16 +21,17 @@
     ".": "./_.js",
     "./*": "./*"
   },
-  "peerDependencies": {
+  "dependencies": {
+    "@1-/csv": "^0.1.4",
     "@1-/hash": "^0.1.0",
-    "@1-/sqlite": "^0.1.1",
-    "@3-/binmap": "^0.1.20",
-    "@3-/binset": "^0.1.6",
+    "@1-/md5": "^0.1.1",
+    "@1-/upsert_gitignore": "^0.1.7",
+    "@3-/base64url": "^0.1.4",
+    "@3-/binmap": "^0.1.22",
+    "@3-/binset": "^0.1.8",
     "@3-/int": "^0.1.1",
     "@3-/plimit": "^0.1.3",
     "@3-/u8": "^0.1.2",
-    "@3-/utf8": "^0.1.1",
-    "@3-/vb": "^0.1.6",
-    "@1-/upsert_gitignore": "^0.1.3"
+    "@3-/vb": "^0.1.6"
   }
 }

package/rm.js

@@ -1,10 +1,8 @@
-import tx from "@1-/sqlite/tx.js";
-
+/*
+批量删除记录
+db: BinMap 实例
+rm: 待删除的路径 path 数组
+*/
 export default (db, rm) => {
-  if (rm.length > 0) {
-    tx(db, () => {
-      const del = db.prepare("DELETE FROM scanMtimeLen WHERE hash=?");
-      rm.forEach((hash) => del.run(hash));
-    });
-  }
+  rm.forEach((path) => db.delete(path));
 };

package/scan.js

@@ -1,21 +1,46 @@
 import { BinSet } from "@3-/binset";
 import u8eq from "@3-/u8/u8eq.js";
 import vbE from "@3-/vb/vbE.js";
+import { join } from "node:path";
 import stat from "./stat.js";
+import pathMd5 from "@1-/md5/pathMd5.js";
 
-export default async (dir, files, existing, limit) => {
+/*
+扫描指定目录下的文件，并与已有记录进行对比
+dir: 扫描目录
+files: 相对路径列表
+existing: 已有的元数据 Map
+existing_md5: 已有的 md5 Map
+limit: 并发限制器
+db_mtime: 修改时间数据库实例
+返回值: [scanned, update]
+  scanned: 已扫描文件的 path 集合
+  update: 需要更新或新增的文件相对路径列表
+*/
+export default async (dir, files, existing, existing_md5, limit, db_mtime) => {
   const scanned = new BinSet(),
     update = [];
+
   await Promise.all(
     files.map((rel_path) =>
       limit(async () => {
         try {
-          const [size, mtime, hash] = await stat(dir, rel_path),
-            val = existing.get(hash);
+          const [size, mtime, path] = await stat(dir, rel_path),
+            val = existing.get(path);
 
-          scanned.add(hash);
+          scanned.add(path);
 
-          if (!val || !u8eq(val, vbE([size, mtime]))) {
+          if (!val) {
+            update.push(rel_path);
+          } else if (!u8eq(val, vbE([size, mtime]))) {
+            const old_md5 = existing_md5.get(path);
+            if (old_md5) {
+              const cur_md5 = await pathMd5(join(dir, rel_path));
+              if (u8eq(old_md5, cur_md5)) {
+                db_mtime.set(path, vbE([size, mtime]));
+                return;
+              }
+            }
             update.push(rel_path);
           }
         } catch {}

package/stat.js

@@ -3,6 +3,12 @@ import { join } from "node:path";
 import int from "@3-/int";
 import strmd5 from "@1-/hash/strmd5.js";
 
+/*
+获取文件的大小、修改时间（秒）以及路径哈希
+dir: 基础目录
+rel_path: 相对路径
+返回值: [大小, 修改时间, 路径哈希]
+*/
 export default async (dir, rel_path) => {
   const { size, mtimeMs: mtime_ms } = await fsStat(join(dir, rel_path));
   return [size, int(mtime_ms), strmd5(rel_path)];

package/upsert.js

@@ -1,13 +1,45 @@
+import { writeFileSync } from "node:fs";
+import { join } from "node:path";
+import dumps from "@1-/csv/dumps.js";
+import vbD from "@3-/vb/vbD.js";
+import vbE from "@3-/vb/vbE.js";
+import uint8B64 from "@3-/base64url/uint8B64.js";
+import pathMd5 from "@1-/md5/pathMd5.js";
 import stat from "./stat.js";
+import { MTIME, MD5 } from "./const.js";
+
+const saveDb = (db_dir, db, name, toRow) => {
+  const li = [];
+  for (const [key, val] of db.entries()) {
+    li.push(toRow(key, val));
+  }
+  writeFileSync(join(db_dir, name + ".csv"), dumps(li), "utf8");
+};
+
+/*
+创建用于插入/更新文件元数据（大小、修改时间、md5）的函数，并在资源释放时同步写入 CSV 文件
+db_mtime: 修改时间 BinMap 实例
+db_md5: md5 BinMap 实例
+dir: 扫描目录
+db_dir: 数据库（CSV）存放目录
+返回值: upsert 异步函数 (带有 [Symbol.dispose] 方法)
+*/
+export default (db_mtime, db_md5, dir, db_dir) => {
+  const upsert = async (rel_path) => {
+    try {
+      const [size, mtime, path] = await stat(dir, rel_path),
+        md5 = await pathMd5(join(dir, rel_path));
+      db_mtime.set(path, vbE([size, mtime]));
+      db_md5.set(path, md5);
+    } catch {}
+  };
+
+  upsert[Symbol.dispose] = () => {
+    saveDb(db_dir, db_mtime, MTIME, (key, val) => [uint8B64(key), ...vbD(val)]);
+    saveDb(db_dir, db_md5, MD5, (key, val) => [uint8B64(key), uint8B64(val)]);
+    db_mtime[Symbol.dispose]();
+    db_md5[Symbol.dispose]();
+  };
 
-export default (db, dir) => {
-  const insert = db.prepare("INSERT OR REPLACE INTO scanMtimeLen(hash,size,mtime)VALUES(?,?,?)"),
-    upsert = async (rel_path) => {
-      try {
-        const [size, mtime, hash] = await stat(dir, rel_path);
-        insert.run(hash, size, mtime);
-      } catch {}
-    };
-  upsert[Symbol.dispose] = () => db.close();
   return upsert;
 };

package/load.js

@@ -1,13 +0,0 @@
-const SQLITE_ERROR = 1;
-
-export default (db) => {
-  try {
-    return db.prepare("SELECT hash,size,mtime FROM scanMtimeLen").all();
-  } catch (err) {
-    if (err.errno === SQLITE_ERROR) {
-      db.exec("CREATE TABLE scanMtimeLen(hash PRIMARY KEY,size INT UNSIGNED,mtime INT UNSIGNED)");
-      return [];
-    }
-    throw err;
-  }
-};

package/save.js

@@ -1,18 +0,0 @@
-import tx from "@1-/sqlite/tx.js";
-
-export default (db, update, rm) => {
-  if (update.length > 0 || rm.length > 0) {
-    tx(db, () => {
-      if (update.length > 0) {
-        const insert = db.prepare(
-          "INSERT OR REPLACE INTO scanMtimeLen(hash,size,mtime)VALUES(?,?,?)",
-        );
-        update.forEach(([_, h, size, mtime]) => insert.run(h, size, mtime));
-      }
-      if (rm.length > 0) {
-        const del = db.prepare("DELETE FROM scanMtimeLen WHERE hash=?");
-        rm.forEach((h) => del.run(h));
-      }
-    });
-  }
-};