@1-/scan 0.1.0 → 0.1.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -9,22 +9,23 @@ Incrementally scans directory files, tracks file sizes and modification times, a
9
9
 
10
10
  ## Features
11
11
 
12
- - **Incremental Scanning**: Detects and updates only new, modified, or deleted files, avoiding redundant file system operations
13
- - **Space-Efficient Storage**: Employs Varint compression (`@3-/vb`) to serialize and compare file sizes and modification times
14
- - **Smart Path Key**: Stores short relative paths (≤ 16 bytes) as raw binary to preserve readability, while hashing longer paths to 16-byte MD5 digests to optimize index performance
15
- - **Database Synchronization**: Synchronizes updates and deletions in a single atomic transaction
16
- - **Ignore Pattern Support**: Integrates ignore rules dynamically during traversal
12
+ - Incremental Scanning: Detects and updates only new, modified, or deleted files, avoiding redundant file system operations.
13
+ - Space-Efficient Storage: Employs Varint compression to serialize and compare file sizes and modification times.
14
+ - Smart Path Key: Stores relative paths not exceeding 16 bytes as raw binary to preserve readability, while hashing longer paths to 16-byte MD5 digests to optimize index performance.
15
+ - Database Synchronization: Synchronizes updates and deletions in a single atomic transaction.
16
+ - Ignore Pattern Support: Integrates ignore rules dynamically during traversal.
17
17
 
18
18
  ## Usage
19
19
 
20
20
  ```javascript
21
21
  import scan from "@1-/scan";
22
22
 
23
- const directoryPath = "./src";
24
- const sqliteDbPath = "./files.db";
23
+ const dir = "./src";
24
+ const dbPath = "./files.db";
25
25
 
26
- // Scan directory and sync records into SQLite
27
- await scan(directoryPath, sqliteDbPath);
26
+ // Scan directory and sync records into SQLite, returning an array of updated relative paths
27
+ const updatedPaths = await scan(dir, dbPath);
28
+ console.log(updatedPaths);
28
29
  ```
29
30
 
30
31
  ## Design Ideas
@@ -44,11 +45,11 @@ graph TD
44
45
 
45
46
  ## Tech Stack
46
47
 
47
- - **Bun**: Runtime and test runner
48
- - **SQLite**: Local relational database engine
49
- - **@1-/walk**: Directory walker with ignore support
50
- - **@3-/vb**: Variable-length byte encoder
51
- - **@3-/binmap** / **@3-/binset**: Efficient binary collection structures
48
+ - Bun: Runtime and test runner
49
+ - SQLite: Local relational database engine
50
+ - `@1-/walk`: Directory walker with ignore support
51
+ - `@3-/vb`: Variable-length byte encoder
52
+ - `@3-/binmap` / `@3-/binset`: Efficient binary collection structures
52
53
 
53
54
  ## Directory Structure
54
55
 
@@ -77,31 +78,32 @@ To optimize space inside the database file, SQLite internally uses variable-leng
77
78
  <a id="zh"></a>
78
79
  # @1-/scan : 增量扫描目录文件并使用 SQLite 记录元数据
79
80
 
80
- 增量扫描指定目录,比对并同步文件大小与修改时间,将记录存入 SQLite 数据库。
81
+ 增量扫描目录,比对并同步文件大小与修改时间,记录存入 SQLite 数据库。
81
82
 
82
83
  ## 功能介绍
83
84
 
84
- - **增量扫描**:仅处理新增、修改或删除的文件,避免冗余文件操作
85
- - **紧凑存储**:使用可变字节码(Varint)压缩技术(`@3-/vb`)比对并保存文件大小和修改时间
86
- - **智能哈希**:短相对路径(≤ 16 字节)保留原始字节,长相对路径哈希为 16 字节 MD5,优化数据库索引效率
87
- - **事务同步**:更新与删除操作合并至单次数据库事务,确保一致性
88
- - **规则过滤**:基于 `@1-/walk` 的忽略规则过滤特定文件与目录
85
+ - 增量扫描:仅处理新增、修改或删除的文件,避免冗余文件操作。
86
+ - 紧凑存储:使用可变字节码(Varint)压缩技术比对并保存文件大小和修改时间。
87
+ - 路径映射:相对路径长度不大于 16 字节时保留原始字节,大于 16 字节时计算为 16 字节 MD5,优化数据库索引。
88
+ - 事务同步:更新与删除操作合并至单次数据库事务,确保一致性。
89
+ - 规则过滤:基于 `@1-/walk` 的忽略规则过滤特定文件与目录。
89
90
 
90
91
  ## 使用演示
91
92
 
92
93
  ```javascript
93
94
  import scan from "@1-/scan";
94
95
 
95
- const directoryPath = "./src";
96
- const sqliteDbPath = "./files.db";
96
+ const dir = "./src";
97
+ const dbPath = "./files.db";
97
98
 
98
- // 扫描目录并同步至 SQLite 数据库
99
- await scan(directoryPath, sqliteDbPath);
99
+ // 扫描目录并同步至 SQLite 数据库,返回有变更的相对路径数组
100
+ const updatedPaths = await scan(dir, dbPath);
101
+ console.log(updatedPaths);
100
102
  ```
101
103
 
102
104
  ## 设计思路
103
105
 
104
- 模块调用流程如下:
106
+ 模块调用流程:
105
107
 
106
108
  ```mermaid
107
109
  graph TD
@@ -116,11 +118,11 @@ graph TD
116
118
 
117
119
  ## 技术栈
118
120
 
119
- - **Bun**: 运行环境与测试工具
120
- - **SQLite**: 本地关系型数据库
121
- - **@1-/walk**: 支持忽略规则的目录遍历工具
122
- - **@3-/vb**: 可变长度整型编码器
123
- - **@3-/binmap** / **@3-/binset**: 二进制哈希键容器
121
+ - Bun:运行环境与测试工具
122
+ - SQLite:本地关系型数据库
123
+ - `@1-/walk`:支持忽略规则的目录遍历工具
124
+ - `@3-/vb`:可变长度整型编码器
125
+ - `@3-/binmap` / `@3-/binset`:二进制哈希键容器
124
126
 
125
127
  ## 目录结构
126
128
 
@@ -139,7 +141,7 @@ graph TD
139
141
 
140
142
  ## 历史故事
141
143
 
142
- SQLite 由 D. Richard Hipp 于 2000 年为美国海军驱逐舰的控制系统编写。当时系统采用的商业数据库需繁琐的系统管理,一旦数据库故障系统便无法运行。Hipp 因而设计出无服务器、零配置且直接读写单文件的 SQLite。
144
+ SQLite 由 D. Richard Hipp 于 2000 年为驱逐舰控制系统编写。当时系统采用的商业数据库需要繁琐的管理,且一旦故障系统便无法运行。Hipp 设计出无服务器、零配置且直接读写单文件的 SQLite。
143
145
 
144
- 为极限节约存储空间,SQLite 内部大量采用可变长度整数(Varint)编码。本项目同样引入 Varint 压缩算法,对文件大小与修改时间做高效编码后再作比对存储,延续了 SQLite 追求极致性能与紧凑空间的优良传统。
146
+ 为节约存储空间,SQLite 内部采用可变长度整数(Varint)编码。本项目同样引入 Varint 压缩算法,对文件大小与修改时间做编码后再比对存储,延续了 SQLite 追求性能与紧凑空间的优良传统。
145
147
  ../doc/zh/about.md
package/_.js CHANGED
@@ -23,4 +23,5 @@ export default async (dir, db_path) => {
23
23
  }
24
24
 
25
25
  fileWrite(db, to_update, to_delete);
26
+ return to_update.map(([rel_path]) => rel_path);
26
27
  };
package/dirWalk.js CHANGED
@@ -24,7 +24,7 @@ export default async (dir, existing) => {
24
24
  if (val && u8eq(val, vbE([size, mtime]))) {
25
25
  return;
26
26
  }
27
- to_update.push([h, size, mtime]);
27
+ to_update.push([rel_path, h, size, mtime]);
28
28
  }
29
29
  });
30
30
 
package/fileWrite.js CHANGED
@@ -5,8 +5,8 @@ export default (db, to_update, to_delete) => {
5
5
  trans(db, () => {
6
6
  if (to_update.length > 0) {
7
7
  const insert = db.prepare("INSERT OR REPLACE INTO file(hash,size,mtime)VALUES(?,?,?)");
8
- for (const record of to_update) {
9
- insert.run(...record);
8
+ for (const [_, h, size, mtime] of to_update) {
9
+ insert.run(h, size, mtime);
10
10
  }
11
11
  }
12
12
  if (to_delete.length > 0) {
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@1-/scan",
3
- "version": "0.1.0",
3
+ "version": "0.1.1",
4
4
  "description": "Incrementally scan directory files and track metadata in SQLite / 增量扫描目录文件并使用 SQLite 记录元数据",
5
5
  "keywords": [
6
6
  "scan",