@1-/scan 0.1.0 → 0.1.2

package/README.md

@@ -5,26 +5,28 @@
 <a id="en"></a>
 # @1-/scan : Incrementally scan directory files and track metadata in SQLite
 
-Incrementally scans directory files, tracks file sizes and modification times, and synchronizes status into a SQLite database.
+Incrementally scans directory files, tracks file sizes and modification times, and synchronizes status into an SQLite database using Bun's native SQLite driver (`bun:sqlite`), returning an array of updated relative paths.
 
 ## Features
 
-- **Incremental Scanning**: Detects and updates only new, modified, or deleted files, avoiding redundant file system operations
-- **Space-Efficient Storage**: Employs Varint compression (`@3-/vb`) to serialize and compare file sizes and modification times
-- **Smart Path Key**: Stores short relative paths (≤ 16 bytes) as raw binary to preserve readability, while hashing longer paths to 16-byte MD5 digests to optimize index performance
-- **Database Synchronization**: Synchronizes updates and deletions in a single atomic transaction
-- **Ignore Pattern Support**: Integrates ignore rules dynamically during traversal
+- Incremental Scanning: Detects and updates only new, modified, or deleted files, avoiding redundant file system operations.
+- Space-Efficient Storage: Employs Varint compression to serialize and compare file sizes and modification times.
+- Smart Path Key: Stores relative paths not exceeding 16 bytes as raw binary to preserve readability, while hashing longer paths to 16-byte MD5 digests to optimize index performance.
+- Database Synchronization: Synchronizes updates and deletions in a single atomic transaction.
+- Ignore Pattern Support: Integrates ignore rules dynamically during traversal.
+- Native SQLite: Leverages Bun's native, high-performance `bun:sqlite` engine, eliminating external build dependencies.
 
 ## Usage
 
 ```javascript
 import scan from "@1-/scan";
 
-const directoryPath = "./src";
-const sqliteDbPath = "./files.db";
+const dir = "./src";
+const dbPath = "./files.db";
 
-// Scan directory and sync records into SQLite
-await scan(directoryPath, sqliteDbPath);
+// Scan directory and sync records into SQLite, returning an array of updated relative paths
+const updatedPaths = await scan(dir, dbPath);
+console.log(updatedPaths);
 ```
 
 ## Design Ideas
@@ -34,21 +36,21 @@ Execution flow of modules:
 ```mermaid
 graph TD
     Entry["_.js (Main)"] -->|Open database| Sqlite[sqlite.js]
-    Entry -->|Load existing records| FileAll[fileAll.js]
+    Entry -->|Load existing records| Load[load.js]
     Entry -->|Walk and compare| DirWalk[dirWalk.js]
     DirWalk -->|Traverse files| Walk["@1-/walk/walkRelIgnore"]
     DirWalk -->|Optimize keys| Hash[hash.js]
-    Entry -->|Apply modifications| FileWrite[fileWrite.js]
-    FileWrite -->|Wrap transaction| Trans[trans.js]
+    Entry -->|Apply modifications| Save[save.js]
+    Save -->|Wrap transaction| Trans[trans.js]
 ```
 
 ## Tech Stack
 
-- **Bun**: Runtime and test runner
-- **SQLite**: Local relational database engine
-- **@1-/walk**: Directory walker with ignore support
-- **@3-/vb**: Variable-length byte encoder
-- **@3-/binmap** / **@3-/binset**: Efficient binary collection structures
+- Bun: Runtime and test runner
+- Bun SQLite: Bun's built-in high-performance SQLite engine
+- `@1-/walk`: Directory walker with ignore support
+- `@3-/vb`: Variable-length byte encoder
+- `@3-/binmap` / `@3-/binset`: Efficient binary collection structures
 
 ## Directory Structure
 
@@ -57,8 +59,8 @@ graph TD
 ├── src
 │   ├── _.js          # Entry point orchestrating the scanning and sync process
 │   ├── dirWalk.js    # Recursively scans files and filters modified ones
-│   ├── fileAll.js    # Retrieves database records and initializes schema
-│   ├── fileWrite.js  # Performs bulk database inserts and deletes
+│   ├── load.js       # Retrieves database records and initializes schema
+│   ├── save.js       # Performs bulk database inserts and deletes
 │   ├── hash.js       # Processes path keys into raw bytes or MD5 digests
 │   ├── sqlite.js     # Manages SQLite database connection and disposal
 │   └── trans.js      # Wraps operations inside an SQL transaction
@@ -77,50 +79,52 @@ To optimize space inside the database file, SQLite internally uses variable-leng
 <a id="zh"></a>
 # @1-/scan : 增量扫描目录文件并使用 SQLite 记录元数据
 
-增量扫描指定目录，比对并同步文件大小与修改时间，将记录存入 SQLite 数据库。
+基于 Bun 原生的高性能内置 SQLite 数据库（`bun:sqlite`）增量扫描目录，比对并同步文件大小与修改时间，并返回有变更的相对路径数组。
 
 ## 功能介绍
 
-- **增量扫描**：仅处理新增、修改或删除的文件，避免冗余文件操作
-- **紧凑存储**：使用可变字节码（Varint）压缩技术（`@3-/vb`）比对并保存文件大小和修改时间
-- **智能哈希**：短相对路径（≤ 16 字节）保留原始字节，长相对路径哈希为 16 字节 MD5，优化数据库索引效率
-- **事务同步**：更新与删除操作合并至单次数据库事务，确保一致性
-- **规则过滤**：基于 `@1-/walk` 的忽略规则过滤特定文件与目录
+- 增量扫描：仅处理新增、修改或删除的文件，避免冗余文件操作。
+- 紧凑存储：使用可变字节码（Varint）压缩技术比对并保存文件大小和修改时间。
+- 路径映射：相对路径长度不大于 16 字节时保留原始字节，大于 16 字节时计算为 16 字节 MD5，优化数据库索引。
+- 事务同步：更新与删除操作合并至单次数据库事务，确保一致性。
+- 规则过滤：基于 `@1-/walk` 的忽略规则过滤特定文件与目录。
+- 原生数据库：使用 Bun 内置的 `bun:sqlite`，性能优异且无需安装或编译外部依赖。
 
 ## 使用演示
 
 ```javascript
 import scan from "@1-/scan";
 
-const directoryPath = "./src";
-const sqliteDbPath = "./files.db";
+const dir = "./src";
+const dbPath = "./files.db";
 
-// 扫描目录并同步至 SQLite 数据库
-await scan(directoryPath, sqliteDbPath);
+// 扫描目录并同步至 SQLite 数据库，返回有变更的相对路径数组
+const updatedPaths = await scan(dir, dbPath);
+console.log(updatedPaths);
 ```
 
 ## 设计思路
 
-模块调用流程如下：
+模块调用流程：
 
 ```mermaid
 graph TD
     Entry["_.js (主入口)"] -->|打开数据库| Sqlite[sqlite.js]
-    Entry -->|载入已有记录| FileAll[fileAll.js]
+    Entry -->|载入已有记录| Load[load.js]
     Entry -->|遍历并比对| DirWalk[dirWalk.js]
     DirWalk -->|扫描文件系统| Walk["@1-/walk/walkRelIgnore"]
     DirWalk -->|计算路径哈希| Hash[hash.js]
-    Entry -->|写入变更数据| FileWrite[fileWrite.js]
-    FileWrite -->|执行事务控制| Trans[trans.js]
+    Entry -->|写入变更数据| Save[save.js]
+    Save -->|执行事务控制| Trans[trans.js]
 ```
 
 ## 技术栈
 
-- **Bun**: 运行环境与测试工具
-- **SQLite**: 本地关系型数据库
-- **@1-/walk**: 支持忽略规则的目录遍历工具
-- **@3-/vb**: 可变长度整型编码器
-- **@3-/binmap** / **@3-/binset**: 二进制哈希键容器
+- Bun：运行环境与测试工具
+- Bun SQLite：Bun 内置的高性能 SQLite 模块
+- `@1-/walk`：支持忽略规则的目录遍历工具
+- `@3-/vb`：可变长度整型编码器
+- `@3-/binmap` / `@3-/binset`：二进制哈希键容器
 
 ## 目录结构
 
@@ -129,8 +133,8 @@ graph TD
 ├── src
 │   ├── _.js          # 主入口，统筹扫描与同步逻辑
 │   ├── dirWalk.js    # 递归遍历目录，比对筛选出变更文件
-│   ├── fileAll.js    # 读取数据库中全部记录，初始化数据表
-│   ├── fileWrite.js  # 事务内执行批量插入与删除
+│   ├── load.js       # 读取数据库中全部记录，初始化数据表
+│   ├── save.js       # 事务内执行批量插入与删除
 │   ├── hash.js       # 计算相对路径哈希值或保留原始字节
 │   ├── sqlite.js     # 管理 SQLite 数据库连接及资源释放
 │   └── trans.js      # 封装数据库事务控制
@@ -139,7 +143,7 @@ graph TD
 
 ## 历史故事
 
-SQLite 由 D. Richard Hipp 于 2000 年为美国海军驱逐舰的控制系统编写。当时系统采用的商业数据库需繁琐的系统管理，一旦数据库故障系统便无法运行。Hipp 因而设计出无服务器、零配置且直接读写单文件的 SQLite。
+SQLite 由 D. Richard Hipp 于 2000 年为驱逐舰控制系统编写。当时系统采用的商业数据库需要繁琐的管理，且一旦故障系统便无法运行。Hipp 设计出无服务器、零配置且直接读写单文件的 SQLite。
 
-为极限节约存储空间，SQLite 内部大量采用可变长度整数（Varint）编码。本项目同样引入 Varint 压缩算法，对文件大小与修改时间做高效编码后再作比对存储，延续了 SQLite 追求极致性能与紧凑空间的优良传统。
+为节约存储空间，SQLite 内部采用可变长度整数（Varint）编码。本项目同样引入 Varint 压缩算法，对文件大小与修改时间做编码后再比对存储，延续了 SQLite 追求性能与紧凑空间的优良传统。
 ../doc/zh/about.md

package/_.js

@@ -1,14 +1,14 @@
 import { BinMap } from "@3-/binmap";
 import vbE from "@3-/vb/vbE.js";
 import sqlite from "./sqlite.js";
-import fileAll from "./fileAll.js";
+import load from "./load.js";
 import dirWalk from "./dirWalk.js";
-import fileWrite from "./fileWrite.js";
+import save from "./save.js";
 
 export default async (dir, db_path) => {
   using db = sqlite(db_path);
   const existing = new BinMap(),
-    db_rows = fileAll(db);
+    db_rows = load(db);
 
   for (const row of db_rows) {
     existing.set(row.hash, vbE([row.size, row.mtime]));
@@ -22,5 +22,6 @@ export default async (dir, db_path) => {
     }
   }
 
-  fileWrite(db, to_update, to_delete);
+  save(db, to_update, to_delete);
+  return to_update.map(([rel_path]) => rel_path);
 };

package/dirWalk.js

@@ -24,7 +24,7 @@ export default async (dir, existing) => {
       if (val && u8eq(val, vbE([size, mtime]))) {
         return;
       }
-      to_update.push([h, size, mtime]);
+      to_update.push([rel_path, h, size, mtime]);
     }
   });
 

package/{fileAll.js → load.js}

@@ -4,7 +4,7 @@ export default (db) => {
   try {
     return db.prepare("SELECT hash,size,mtime FROM file").all();
   } catch (err) {
-    if (err.errcode === SQLITE_ERROR) {
+    if (err.errno === SQLITE_ERROR) {
       db.exec("CREATE TABLE file(hash PRIMARY KEY,size INT UNSIGNED,mtime INT UNSIGNED)");
       return [];
     }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@1-/scan",
-  "version": "0.1.0",
+  "version": "0.1.2",
   "description": "Incrementally scan directory files and track metadata in SQLite / 增量扫描目录文件并使用 SQLite 记录元数据",
   "keywords": [
     "scan",

package/{fileWrite.js → save.js}

@@ -5,8 +5,8 @@ export default (db, to_update, to_delete) => {
     trans(db, () => {
       if (to_update.length > 0) {
         const insert = db.prepare("INSERT OR REPLACE INTO file(hash,size,mtime)VALUES(?,?,?)");
-        for (const record of to_update) {
-          insert.run(...record);
+        for (const [_, h, size, mtime] of to_update) {
+          insert.run(h, size, mtime);
         }
       }
       if (to_delete.length > 0) {

package/sqlite.js

@@ -1,7 +1,7 @@
-const { DatabaseSync } = await import("node:sqlite");
+import { Database } from "bun:sqlite";
 
 export default (db_path) => {
-  const db = new DatabaseSync(db_path);
+  const db = new Database(db_path);
   db[Symbol.dispose] = () => db.close();
   return db;
 };