endef 2.0.2 → 2.0.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -2,7 +2,7 @@
2
2
 
3
3
  `endef` 是一个纯 Node.js 的文件旁路转换工具。
4
4
 
5
- 它适合这样的场景:某些常见源码或文本后缀的文件在系统读取、复制、粘贴、编辑器打开时出现乱码或异常,但通过 Node.js 脚本读取仍然正常。`endef` 会先把目标文件内容写到同目录的 `.endef` 副本里,再删除原文件,并用 `.endef` 副本内容新建原文件。
5
+ 它适合这样的场景:某些常见源码或文本后缀的文件在系统读取、复制、粘贴、编辑器打开时出现乱码或异常,但通过 Node.js 脚本读取仍然正常。`endef` 会先把目标文件内容写到同目录的 `.endef` 副本里,再用脚本批量删除原文件并把 `.endef` 改回原文件名。
6
6
 
7
7
  示例:
8
8
 
@@ -16,11 +16,10 @@ README.md -> README.md.endef -> README.md
16
16
 
17
17
  ## 特性
18
18
 
19
- - Node.js 实现,不依赖 Bash、find、xargs、grep shell 工具
19
+ - Node.js CLI,恢复阶段可自动调用 Bash PowerShell 脚本
20
20
  - 默认旁路后缀为 `.endef`
21
21
  - `en` 默认转换所有匹配后缀的文件
22
- - `en --marker` 可以只转换命中 marker 的文件,marker 支持数组配置,也支持 CLI 手动指定多个
23
- - `de` 会删除原文件,用 `.endef` 临时副本内容新建原文件,再删除 `.endef`
22
+ - `de` 会用脚本批量删除原文件,并把 `.endef` 临时副本改回原文件名
24
23
  - `pack` 会把恢复后的目录打包成 `.tar.gz`
25
24
  - 不引入生产依赖
26
25
 
@@ -42,31 +41,7 @@ npm install -g endef
42
41
  endef en /path/to/project
43
42
  ```
44
43
 
45
- 只转换命中默认 marker 的文件:
46
-
47
- ```bash
48
- endef en /path/to/project --marker
49
- ```
50
-
51
- 手动指定 marker:
52
-
53
- ```bash
54
- endef en /path/to/project --marker E-SafeNet
55
- ```
56
-
57
- 指定多个 marker:
58
-
59
- ```bash
60
- endef en /path/to/project --marker E-SafeNet --marker BadText
61
- ```
62
-
63
- 也可以用英文逗号分隔:
64
-
65
- ```bash
66
- endef en /path/to/project --marker E-SafeNet,BadText
67
- ```
68
-
69
- 删除原文件,用 `.endef` 副本内容新建原文件,再删除 `.endef`:
44
+ 用脚本批量删除原文件,并把 `.endef` 改回原文件名:
70
45
 
71
46
  ```bash
72
47
  endef de /path/to/project
@@ -107,11 +82,11 @@ endef all [directory] [options]
107
82
 
108
83
  `en` 会扫描目标目录,把匹配后缀的文件读取出来,并写成同目录的 `.endef` 副本。
109
84
 
110
- `de` 会扫描目标目录中的 `.endef` 文件,删除对应的原文件,用 `.endef` 副本内容新建原文件,然后删除 `.endef` 副本。这个命令会直接修改文件,请在执行前确认目录和备份策略。
85
+ `de` 会扫描目标目录中的 `.endef` 文件,并用外部脚本批量删除原文件、把 `.endef` 改回原文件名。默认模式是 `auto`:优先使用 Bash 的 `find + xargs -P`,没有 Bash 时在 Windows 上使用 PowerShell。这个命令会直接修改文件,请在执行前确认目录和备份策略。
111
86
 
112
87
  `pack` 会按当前配置扫描当前目录,把没有被 `excludeDirs` 排除的文件打包成 `.tar.gz`。如果输出文件已存在,会直接覆盖。
113
88
 
114
- `all` 会按顺序执行 `en`、`de`、`pack`,适合确认配置后一次性完成旁路转换、新建恢复和打包。
89
+ `all` 会按顺序执行 `en`、`de`、`pack`,适合确认配置后一次性完成旁路转换、脚本恢复和打包。
115
90
 
116
91
  ## 参数
117
92
 
@@ -121,18 +96,17 @@ endef all [directory] [options]
121
96
  --ext <list> 目标文件后缀,多个值用英文逗号分隔
122
97
  --exclude <list> 排除目录名,多个值用英文逗号分隔
123
98
  --concurrency <n> 并发文件操作数量
124
- --marker [text] 只为命中任意 marker 的文件生成副本
125
99
  --out <file> 压缩包输出名称
100
+ --mode <mode> de/all 恢复模式:auto、bash、powershell、node
126
101
  ```
127
102
 
128
103
  示例:
129
104
 
130
105
  ```bash
131
106
  endef en . --ext .js,.ts,.vue,.md --exclude node_modules,.git,dist
132
- endef en . --marker E-SafeNet --marker BadText
133
107
  endef de .
134
108
  endef pack recovered
135
- endef all . --marker --out recovered
109
+ endef all . --out recovered
136
110
  ```
137
111
 
138
112
  ## 配置文件
@@ -150,7 +124,6 @@ endef all . --marker --out recovered
150
124
  ```json
151
125
  {
152
126
  "suffix": ".endef",
153
- "markers": ["E-SafeNet"],
154
127
  "extensions": [
155
128
  ".js",
156
129
  ".jsx",
@@ -186,49 +159,33 @@ endef all . --marker --out recovered
186
159
  }
187
160
  ```
188
161
 
189
- ## marker 模式
190
-
191
- 默认情况下:
192
-
193
- ```bash
194
- endef en .
195
- ```
196
-
197
- 会为所有匹配后缀的文件生成 `.endef` 副本。
198
-
199
- 启用 marker 模式后:
200
-
201
- ```bash
202
- endef en . --marker
203
- ```
162
+ ## 恢复策略
204
163
 
205
- 只会为命中任意 marker 的文件生成 `.endef` 副本。`--marker` 不带值时使用配置中的 `markers`,默认是:
164
+ 恢复时,`endef` 会把:
206
165
 
207
- ```json
208
- ["E-SafeNet"]
166
+ ```text
167
+ file.ext.endef
209
168
  ```
210
169
 
211
- 如果 CLI 指定了 marker,则使用 CLI 的值:
170
+ 恢复为:
212
171
 
213
- ```bash
214
- endef en . --marker E-SafeNet --marker BadText
172
+ ```text
173
+ file.ext
215
174
  ```
216
175
 
217
- ## 恢复策略
218
-
219
- 恢复时,`endef` 会把:
176
+ `de` 默认使用脚本恢复。`auto` 模式会优先尝试 Bash,因为 `find + xargs -P` 可以并行处理大量文件;如果没有 Bash,则在 Windows 上使用单个 PowerShell 进程批量处理。恢复时会把:
220
177
 
221
178
  ```text
222
179
  file.ext.endef
223
180
  ```
224
181
 
225
- 删除原文件后,用 `.endef` 内容新建:
182
+ 改回:
226
183
 
227
184
  ```text
228
185
  file.ext
229
186
  ```
230
187
 
231
- 如果 `file.ext` 已经存在,会先被删除,然后用 `.endef` 副本内容创建新的 `file.ext`。创建成功后,`file.ext.endef` 会被删除。这个行为符合工具的核心假设:原文件可能已经被异常内容污染,而 `.endef` 是通过旁路读取保存出来的可恢复内容。
188
+ 如果 `file.ext` 已经存在,会先被删除。这个行为符合工具的核心假设:原文件可能已经被异常内容污染,而 `.endef` 是通过旁路读取保存出来的可恢复内容。
232
189
 
233
190
  ## 打包策略
234
191
 
@@ -286,10 +243,10 @@ endef de .
286
243
  endef pack
287
244
  ```
288
245
 
289
- 同样支持 marker、排除目录和输出名称:
246
+ 同样支持排除目录和输出名称:
290
247
 
291
248
  ```bash
292
- endef all . --marker E-SafeNet,BadText --exclude node_modules,.git,dist --out recovered
249
+ endef all . --exclude node_modules,.git,dist --out recovered
293
250
  ```
294
251
 
295
252
  ## 安全建议
@@ -297,7 +254,6 @@ endef all . --marker E-SafeNet,BadText --exclude node_modules,.git,dist --out re
297
254
  - 操作前尽量手动备份目录或磁盘
298
255
  - 优先在干净系统、离线环境或挂载出来的磁盘上运行
299
256
  - 先在小目录中测试 `endef en` 和 `endef de`
300
- - 不确定 marker 是否完整时,优先使用默认的全量 `endef en`
301
257
  - 不要直接运行恢复出来的未知脚本或可执行文件
302
258
  - 恢复后建议做杀毒、哈希校验和人工抽查
303
259
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "endef",
3
- "version": "2.0.2",
3
+ "version": "2.0.4",
4
4
  "description": "Recover text and source files through safe .endef sidecar copies.",
5
5
  "bin": {
6
6
  "endef": "bin/main.js"
package/src/config.js CHANGED
@@ -4,7 +4,6 @@ const path = require('path')
4
4
 
5
5
  const defaultConfig = {
6
6
  suffix: '.endef',
7
- markers: ['E-SafeNet'],
8
7
  extensions: [
9
8
  '.js',
10
9
  '.jsx',
@@ -36,7 +35,8 @@ const defaultConfig = {
36
35
  ],
37
36
  excludeDirs: ['node_modules', '.git', 'dist', '.next', '.nuxt', '.cache'],
38
37
  excludeFiles: ['.endef.json'],
39
- concurrency: 32
38
+ concurrency: 32,
39
+ restoreMode: 'auto'
40
40
  }
41
41
 
42
42
  function loadConfig(cwd, explicitConfigFile, targetDirectory) {
@@ -73,35 +73,13 @@ function stripBom(value) {
73
73
  }
74
74
 
75
75
  function mergeConfig(base, next) {
76
- const markers = normalizeMarkers(next.markers)
77
-
78
76
  return {
79
77
  ...base,
80
78
  ...next,
81
79
  extensions: Array.isArray(next.extensions) ? next.extensions : base.extensions,
82
80
  excludeDirs: Array.isArray(next.excludeDirs) ? next.excludeDirs : base.excludeDirs,
83
- excludeFiles: Array.isArray(next.excludeFiles) ? next.excludeFiles : base.excludeFiles,
84
- markers: markers.length > 0 ? markers : base.markers
81
+ excludeFiles: Array.isArray(next.excludeFiles) ? next.excludeFiles : base.excludeFiles
85
82
  }
86
83
  }
87
84
 
88
- function normalizeMarkers(value) {
89
- if (Array.isArray(value)) {
90
- return value.map(item => String(item).trim()).filter(Boolean)
91
- }
92
-
93
- return []
94
- }
95
-
96
- function splitMarkers(value) {
97
- if (!value) {
98
- return []
99
- }
100
-
101
- return String(value)
102
- .split(',')
103
- .map(item => item.trim())
104
- .filter(Boolean)
105
- }
106
-
107
- module.exports = { defaultConfig, loadConfig, normalizeMarkers, splitMarkers }
85
+ module.exports = { defaultConfig, loadConfig }
package/src/de.js CHANGED
@@ -1,95 +1,233 @@
1
1
  const fs = require('fs/promises')
2
2
  const path = require('path')
3
+ const { execFile } = require('child_process')
4
+ const { promisify } = require('util')
3
5
  const { defaultConfig } = require('./config')
4
- const { createLimiter, walk } = require('./utils')
6
+
7
+ const execFileAsync = promisify(execFile)
5
8
 
6
9
  async function restoreFiles(directory, options = {}) {
7
10
  const config = {
8
11
  ...defaultConfig,
9
12
  ...options,
10
- directory: path.resolve(directory || process.cwd())
13
+ directory: path.resolve(directory || process.cwd()),
14
+ restoreMode: options.restoreMode || 'auto'
11
15
  }
12
- const stats = createStats()
13
- const limit = createLimiter(config.concurrency)
14
- const tasks = []
16
+ const mode = await resolveRestoreMode(config.restoreMode)
15
17
 
16
- await walk(config.directory, config, async filePath => {
17
- if (!filePath.endsWith(config.suffix)) {
18
- return
19
- }
18
+ if (mode === 'bash') {
19
+ return restoreWithBash(config)
20
+ }
20
21
 
21
- tasks.push(
22
- limit(async () => {
23
- await restoreOne(filePath, config, stats)
24
- }).catch(error => {
25
- stats.failed += 1
26
- console.error(`Failed: ${filePath}`)
27
- console.error(error && error.message ? error.message : error)
28
- })
29
- )
30
- })
22
+ if (mode === 'powershell') {
23
+ return restoreWithPowerShell(config)
24
+ }
31
25
 
32
- await Promise.all(tasks)
33
- console.log(
34
- `Done. restored=${stats.restored}, removed=${stats.removed}, created=${stats.created}, deleted=${stats.deleted}, failed=${stats.failed}`
35
- )
36
- return stats
26
+ return restoreWithNode(config)
37
27
  }
38
28
 
39
- async function restoreOne(filePath, config, stats) {
40
- const targetPath = filePath.slice(0, -config.suffix.length)
41
- const targetExists = await exists(targetPath)
42
- const data = await fs.readFile(filePath)
29
+ async function resolveRestoreMode(mode) {
30
+ if (mode !== 'auto') {
31
+ return mode
32
+ }
43
33
 
44
- await fs.rm(targetPath, { force: true })
45
- await waitForMissing(targetPath)
46
- await fs.writeFile(targetPath, data, { flag: 'wx' })
47
- await fs.rm(filePath, { force: true })
34
+ if (await hasCommand('bash', ['--version'])) {
35
+ return 'bash'
36
+ }
48
37
 
49
- stats.restored += 1
50
- if (targetExists) {
51
- stats.removed += 1
38
+ if (process.platform === 'win32') {
39
+ return 'powershell'
52
40
  }
53
- stats.created += 1
54
- stats.deleted += 1
55
- console.log(`Restored: ${filePath} -> ${targetPath}`)
41
+
42
+ return 'node'
56
43
  }
57
44
 
58
- async function exists(filePath) {
45
+ async function hasCommand(command, args) {
59
46
  try {
60
- await fs.access(filePath)
47
+ await execFileAsync(command, args, { windowsHide: true })
61
48
  return true
62
49
  } catch (error) {
63
50
  return false
64
51
  }
65
52
  }
66
53
 
67
- async function waitForMissing(filePath) {
68
- for (let index = 0; index < 20; index += 1) {
69
- if (!(await exists(filePath))) {
70
- return
71
- }
54
+ async function restoreWithBash(config) {
55
+ const root = await toBashPath(config.directory)
56
+ const excludeDirs = config.excludeDirs.join('\n')
57
+ const jobs = String(config.concurrency || 32)
58
+ const script = `
59
+ set -e
60
+ root=$1
61
+ suffix=$2
62
+ jobs=$3
63
+ cd "$root"
64
+ find_args=(.)
65
+ while IFS= read -r dir; do
66
+ if [ -n "$dir" ]; then
67
+ find_args+=( -path "*/$dir" -prune -o )
68
+ fi
69
+ done <<< "$ENDEF_EXCLUDE_DIRS"
70
+ find_args+=( -type f -name "*$suffix" -print0 )
71
+ find "\${find_args[@]}" | xargs -0 -r -P "$jobs" -I{} bash -c '
72
+ suffix="$1"
73
+ file="$2"
74
+ target="\${file%$suffix}"
75
+ rm -f -- "$target"
76
+ mv -f -- "$file" "$target"
77
+ printf "%s\\0" "$target"
78
+ ' _ "$suffix" {}
79
+ `
72
80
 
73
- await delay(25)
81
+ const { stdout } = await execFileAsync('bash', ['-lc', script, '_', root, config.suffix, jobs], {
82
+ env: {
83
+ ...process.env,
84
+ ENDEF_EXCLUDE_DIRS: excludeDirs
85
+ },
86
+ maxBuffer: 1024 * 1024 * 64,
87
+ windowsHide: true
88
+ })
89
+ const restored = countNullSeparated(stdout)
90
+
91
+ console.log(`Done. mode=bash, restored=${restored}, failed=0`)
92
+ return {
93
+ mode: 'bash',
94
+ restored,
95
+ failed: 0
96
+ }
97
+ }
98
+
99
+ async function toBashPath(filePath) {
100
+ if (process.platform !== 'win32') {
101
+ return filePath
74
102
  }
75
103
 
76
- throw new Error(`Target still exists after remove: ${filePath}`)
104
+ try {
105
+ const { stdout } = await execFileAsync('bash', ['-lc', 'cygpath -u "$1"', '_', filePath], {
106
+ windowsHide: true
107
+ })
108
+
109
+ return stdout.trim() || filePath
110
+ } catch (error) {
111
+ return filePath
112
+ }
77
113
  }
78
114
 
79
- function delay(ms) {
80
- return new Promise(resolve => {
81
- setTimeout(resolve, ms)
82
- })
115
+ async function restoreWithPowerShell(config) {
116
+ const script = `
117
+ $root = $env:ENDEF_ROOT
118
+ $suffix = $env:ENDEF_SUFFIX
119
+ $excludeText = $env:ENDEF_EXCLUDE_DIRS
120
+ $excludeDirs = @()
121
+ if ($excludeText) {
122
+ $excludeDirs = $excludeText -split "\\n" | Where-Object { $_ }
123
+ }
124
+ $restored = 0
125
+ Get-ChildItem -LiteralPath $root -Recurse -File -Filter "*$suffix" | Where-Object {
126
+ $cleanRoot = $root.TrimEnd('\\', '/')
127
+ $relative = $_.FullName.Substring($cleanRoot.Length).TrimStart('\\', '/')
128
+ $parts = $relative -split '[\\\\/]'
129
+ $skip = $false
130
+ foreach ($dir in $excludeDirs) {
131
+ if ($parts -contains $dir) {
132
+ $skip = $true
133
+ break
134
+ }
135
+ }
136
+ -not $skip
137
+ } | ForEach-Object {
138
+ $target = $_.FullName.Substring(0, $_.FullName.Length - $suffix.Length)
139
+ Remove-Item -LiteralPath $target -Force -ErrorAction SilentlyContinue
140
+ Move-Item -LiteralPath $_.FullName -Destination $target -Force
141
+ $restored += 1
83
142
  }
143
+ Write-Output $restored
144
+ `
145
+ const { stdout } = await execFileAsync(
146
+ 'powershell.exe',
147
+ [
148
+ '-NoProfile',
149
+ '-NonInteractive',
150
+ '-ExecutionPolicy',
151
+ 'Bypass',
152
+ '-Command',
153
+ script
154
+ ],
155
+ {
156
+ env: {
157
+ ...process.env,
158
+ ENDEF_ROOT: config.directory,
159
+ ENDEF_SUFFIX: config.suffix,
160
+ ENDEF_EXCLUDE_DIRS: config.excludeDirs.join('\n')
161
+ },
162
+ maxBuffer: 1024 * 1024 * 64,
163
+ windowsHide: true
164
+ }
165
+ )
166
+ const restored = Number(stdout.trim()) || 0
84
167
 
85
- function createStats() {
168
+ console.log(`Done. mode=powershell, restored=${restored}, failed=0`)
86
169
  return {
170
+ mode: 'powershell',
171
+ restored,
172
+ failed: 0
173
+ }
174
+ }
175
+
176
+ async function restoreWithNode(config) {
177
+ const stats = {
87
178
  restored: 0,
88
- removed: 0,
89
- created: 0,
90
- deleted: 0,
91
179
  failed: 0
92
180
  }
181
+
182
+ await restoreNodeDirectory(config.directory, config, stats)
183
+ console.log(`Done. mode=node, restored=${stats.restored}, failed=${stats.failed}`)
184
+ return {
185
+ mode: 'node',
186
+ restored: stats.restored,
187
+ failed: stats.failed
188
+ }
189
+ }
190
+
191
+ async function restoreNodeDirectory(directory, config, stats) {
192
+ const entries = await fs.readdir(directory, { withFileTypes: true })
193
+
194
+ await Promise.all(
195
+ entries.map(async entry => {
196
+ const filePath = path.join(directory, entry.name)
197
+
198
+ if (entry.isDirectory()) {
199
+ if (config.excludeDirs.includes(entry.name)) {
200
+ return
201
+ }
202
+
203
+ await restoreNodeDirectory(filePath, config, stats)
204
+ return
205
+ }
206
+
207
+ if (!entry.isFile() || !filePath.endsWith(config.suffix)) {
208
+ return
209
+ }
210
+
211
+ try {
212
+ const targetPath = filePath.slice(0, -config.suffix.length)
213
+ await fs.rm(targetPath, { force: true })
214
+ await fs.rename(filePath, targetPath)
215
+ stats.restored += 1
216
+ } catch (error) {
217
+ stats.failed += 1
218
+ console.error(`Failed: ${filePath}`)
219
+ console.error(error && error.message ? error.message : error)
220
+ }
221
+ })
222
+ )
223
+ }
224
+
225
+ function countNullSeparated(value) {
226
+ if (!value) {
227
+ return 0
228
+ }
229
+
230
+ return value.split('\0').filter(Boolean).length
93
231
  }
94
232
 
95
233
  module.exports = restoreFiles
package/src/en.js CHANGED
@@ -29,11 +29,6 @@ async function processFiles(directory, options = {}) {
29
29
  const outputPath = `${filePath}${config.suffix}`
30
30
  const data = await fs.readFile(filePath)
31
31
 
32
- if (config.markerFilter && !matchesAnyMarker(data, config.markers)) {
33
- stats.skipped += 1
34
- return
35
- }
36
-
37
32
  await fs.writeFile(outputPath, data)
38
33
  console.log(`Written: ${outputPath}`)
39
34
  stats.written += 1
@@ -58,12 +53,4 @@ function createStats() {
58
53
  }
59
54
  }
60
55
 
61
- function matchesAnyMarker(data, markers) {
62
- if (!Array.isArray(markers) || markers.length === 0) {
63
- return false
64
- }
65
-
66
- return markers.some(marker => data.includes(marker))
67
- }
68
-
69
56
  module.exports = processFiles
package/src/pack.js CHANGED
@@ -2,36 +2,93 @@ const fs = require('fs')
2
2
  const fsp = require('fs/promises')
3
3
  const path = require('path')
4
4
  const zlib = require('zlib')
5
+ const { promisify } = require('util')
5
6
  const { once } = require('events')
6
7
  const { pipeline } = require('stream/promises')
7
8
  const { defaultConfig } = require('./config')
8
9
  const { walk } = require('./utils')
9
10
 
11
+ const gunzip = promisify(zlib.gunzip)
12
+
10
13
  async function packFiles(directory, options = {}) {
11
- const config = {
14
+ const config = createPackConfig(directory, options)
15
+ const outputPath = resolveOutputPath(config.directory, config.out)
16
+ const entries = await collectPackEntries(config, outputPath)
17
+
18
+ await fsp.rm(outputPath, { force: true })
19
+ await writeTarGz(config.directory, outputPath, entries)
20
+ console.log(`Packed: ${outputPath}`)
21
+ console.log(`Done. files=${entries.length}`)
22
+
23
+ return {
24
+ outputPath,
25
+ files: entries.length
26
+ }
27
+ }
28
+
29
+ function createPackConfig(directory, options) {
30
+ return {
12
31
  ...defaultConfig,
13
32
  ...options,
14
33
  directory: path.resolve(directory || process.cwd())
15
34
  }
16
- const outputPath = resolveOutputPath(config.directory, config.out)
17
- const files = []
35
+ }
36
+
37
+ async function collectPackEntries(config, outputPath) {
38
+ const entries = []
18
39
 
19
40
  await walk(config.directory, config, async filePath => {
20
- if (path.resolve(filePath) === outputPath || filePath.endsWith(config.suffix)) {
41
+ if (shouldSkipPackFile(filePath, config, outputPath)) {
21
42
  return
22
43
  }
23
44
 
24
- files.push(filePath)
45
+ entries.push(createEntry(config.directory, filePath, filePath))
25
46
  })
26
47
 
27
- await fsp.rm(outputPath, { force: true })
28
- await writeTarGz(config.directory, outputPath, files)
29
- console.log(`Packed: ${outputPath}`)
30
- console.log(`Done. files=${files.length}`)
48
+ return entries
49
+ }
50
+
51
+ async function collectRecoveryEntries(config, options = {}) {
52
+ const entries = []
53
+ const sidecarTargets = new Set()
54
+
55
+ await walk(config.directory, config, async filePath => {
56
+ if (filePath.endsWith(config.suffix)) {
57
+ sidecarTargets.add(path.resolve(filePath.slice(0, -config.suffix.length)))
58
+ }
59
+ })
60
+
61
+ await walk(config.directory, config, async filePath => {
62
+ const resolvedPath = path.resolve(filePath)
63
+
64
+ if (options.outputPath && resolvedPath === path.resolve(options.outputPath)) {
65
+ return
66
+ }
67
+
68
+ if (filePath.endsWith(config.suffix)) {
69
+ const targetPath = filePath.slice(0, -config.suffix.length)
70
+ entries.push(createEntry(config.directory, filePath, targetPath))
71
+ return
72
+ }
73
+
74
+ if (options.onlySidecars || sidecarTargets.has(resolvedPath)) {
75
+ return
76
+ }
77
+
78
+ entries.push(createEntry(config.directory, filePath, filePath))
79
+ })
80
+
81
+ return entries
82
+ }
83
+
84
+ function shouldSkipPackFile(filePath, config, outputPath) {
85
+ return path.resolve(filePath) === outputPath || filePath.endsWith(config.suffix)
86
+ }
31
87
 
88
+ function createEntry(rootDirectory, sourcePath, archiveTargetPath) {
32
89
  return {
33
- outputPath,
34
- files: files.length
90
+ sourcePath,
91
+ archivePath: toArchivePath(path.relative(rootDirectory, archiveTargetPath))
35
92
  }
36
93
  }
37
94
 
@@ -51,30 +108,77 @@ function normalizeOutputName(out) {
51
108
  return `${out}.tar.gz`
52
109
  }
53
110
 
54
- async function writeTarGz(rootDirectory, outputPath, files) {
111
+ async function writeTarGz(rootDirectory, outputPath, entries) {
55
112
  await fsp.mkdir(path.dirname(outputPath), { recursive: true })
56
113
 
57
114
  const gzip = zlib.createGzip()
58
115
  const output = fs.createWriteStream(outputPath)
59
- const writer = writeTarEntries(rootDirectory, files, gzip)
116
+ const writer = writeTarEntries(entries, gzip)
60
117
 
61
118
  await Promise.all([writer, pipeline(gzip, output)])
62
119
  }
63
120
 
64
- async function writeTarEntries(rootDirectory, files, stream) {
65
- for (const filePath of files) {
66
- const stat = await fsp.stat(filePath)
67
- const archivePath = toArchivePath(path.relative(rootDirectory, filePath))
68
-
69
- await writeToStream(stream, createHeader(archivePath, stat))
121
+ async function writeTarEntries(entries, stream) {
122
+ for (const entry of entries) {
123
+ const stat = await fsp.stat(entry.sourcePath)
70
124
 
71
- await writeFileToStream(filePath, stream)
125
+ await writeToStream(stream, createHeader(entry.archivePath, stat))
126
+ await writeFileToStream(entry.sourcePath, stream)
72
127
  await writeToStream(stream, Buffer.alloc(paddingSize(stat.size)))
73
128
  }
74
129
 
75
130
  stream.end(Buffer.alloc(1024))
76
131
  }
77
132
 
133
+ async function extractTarGz(archivePath, directory) {
134
+ const archive = await fsp.readFile(archivePath)
135
+ const data = await gunzip(archive)
136
+ let offset = 0
137
+ let extracted = 0
138
+
139
+ while (offset + 512 <= data.length) {
140
+ const header = data.subarray(offset, offset + 512)
141
+
142
+ if (header.every(value => value === 0)) {
143
+ break
144
+ }
145
+
146
+ const entry = readHeader(header)
147
+ offset += 512
148
+
149
+ const content = data.subarray(offset, offset + entry.size)
150
+ const outputPath = path.resolve(directory, entry.name)
151
+
152
+ if (!outputPath.startsWith(path.resolve(directory) + path.sep)) {
153
+ throw new Error(`Unsafe archive path: ${entry.name}`)
154
+ }
155
+
156
+ await fsp.mkdir(path.dirname(outputPath), { recursive: true })
157
+ await fsp.rm(outputPath, { force: true })
158
+ await fsp.writeFile(outputPath, content, { flag: 'wx' })
159
+
160
+ offset += entry.size + paddingSize(entry.size)
161
+ extracted += 1
162
+ }
163
+
164
+ return extracted
165
+ }
166
+
167
+ async function removeSidecars(directory, config) {
168
+ let deleted = 0
169
+
170
+ await walk(directory, config, async filePath => {
171
+ if (!filePath.endsWith(config.suffix)) {
172
+ return
173
+ }
174
+
175
+ await fsp.rm(filePath, { force: true })
176
+ deleted += 1
177
+ })
178
+
179
+ return deleted
180
+ }
181
+
78
182
  async function writeFileToStream(filePath, stream) {
79
183
  for await (const chunk of fs.createReadStream(filePath)) {
80
184
  await writeToStream(stream, chunk)
@@ -113,6 +217,20 @@ function createHeader(filePath, stat) {
113
217
  return header
114
218
  }
115
219
 
220
+ function readHeader(header) {
221
+ const name = readString(header, 0, 100)
222
+ const prefix = readString(header, 345, 155)
223
+
224
+ return {
225
+ name: prefix ? `${prefix}/${name}` : name,
226
+ size: parseInt(readString(header, 124, 12).trim() || '0', 8)
227
+ }
228
+ }
229
+
230
+ function readString(buffer, offset, length) {
231
+ return buffer.subarray(offset, offset + length).toString('utf8').replace(/\0.*$/, '')
232
+ }
233
+
116
234
  function splitTarPath(filePath) {
117
235
  if (Buffer.byteLength(filePath) <= 100) {
118
236
  return {
@@ -156,3 +274,9 @@ function paddingSize(size) {
156
274
  }
157
275
 
158
276
  module.exports = packFiles
277
+ module.exports.createPackConfig = createPackConfig
278
+ module.exports.collectRecoveryEntries = collectRecoveryEntries
279
+ module.exports.extractTarGz = extractTarGz
280
+ module.exports.removeSidecars = removeSidecars
281
+ module.exports.resolveOutputPath = resolveOutputPath
282
+ module.exports.writeTarGz = writeTarGz
package/src/run.js CHANGED
@@ -1,7 +1,7 @@
1
1
  const enF = require('./en')
2
2
  const deF = require('./de')
3
3
  const packF = require('./pack')
4
- const { loadConfig, splitMarkers } = require('./config')
4
+ const { loadConfig } = require('./config')
5
5
 
6
6
  function printHelp() {
7
7
  console.log(`Usage:
@@ -16,16 +16,14 @@ Options:
16
16
  --ext <list> Comma separated target extensions
17
17
  --exclude <list> Comma separated excluded directory names
18
18
  --concurrency <n> Concurrent file operation limit
19
- --marker [text] Only create sidecar files when any marker is found
20
19
  --out <file> Archive output name
20
+ --mode <mode> Restore mode for de/all: auto, bash, powershell, node
21
21
  `)
22
22
  }
23
23
 
24
24
  function parseArgs(command, args) {
25
25
  const result = {
26
26
  directory: process.cwd(),
27
- markerFilter: undefined,
28
- markers: [],
29
27
  positionals: []
30
28
  }
31
29
 
@@ -98,20 +96,14 @@ function parseArgs(command, args) {
98
96
  continue
99
97
  }
100
98
 
101
- if (arg.startsWith('--marker=')) {
102
- result.markerFilter = true
103
- result.markers.push(...splitMarkers(arg.slice('--marker='.length)))
99
+ if (arg.startsWith('--mode=')) {
100
+ result.restoreMode = arg.slice('--mode='.length)
104
101
  continue
105
102
  }
106
103
 
107
- if (arg === '--marker') {
108
- result.markerFilter = true
109
-
110
- if (args[index + 1] && !args[index + 1].startsWith('-')) {
111
- result.markers.push(...splitMarkers(args[index + 1]))
112
- index += 1
113
- }
114
-
104
+ if (arg === '--mode') {
105
+ result.restoreMode = args[index + 1]
106
+ index += 1
115
107
  continue
116
108
  }
117
109
 
@@ -160,8 +152,7 @@ async function run(args) {
160
152
  const options = {
161
153
  ...config,
162
154
  ...cliOptions,
163
- directory: cliOptions.directory || config.directory || process.cwd(),
164
- markers: cliOptions.markers && cliOptions.markers.length > 0 ? cliOptions.markers : config.markers
155
+ directory: cliOptions.directory || config.directory || process.cwd()
165
156
  }
166
157
 
167
158
  if (command === 'en') {