endef 2.0.2 → 2.0.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +20 -64
- package/package.json +1 -1
- package/src/config.js +4 -26
- package/src/de.js +193 -55
- package/src/en.js +0 -13
- package/src/pack.js +144 -20
- package/src/run.js +8 -17
package/README.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
`endef` 是一个纯 Node.js 的文件旁路转换工具。
|
|
4
4
|
|
|
5
|
-
它适合这样的场景:某些常见源码或文本后缀的文件在系统读取、复制、粘贴、编辑器打开时出现乱码或异常,但通过 Node.js 脚本读取仍然正常。`endef` 会先把目标文件内容写到同目录的 `.endef`
|
|
5
|
+
它适合这样的场景:某些常见源码或文本后缀的文件在系统读取、复制、粘贴、编辑器打开时出现乱码或异常,但通过 Node.js 脚本读取仍然正常。`endef` 会先把目标文件内容写到同目录的 `.endef` 副本里,再用脚本批量删除原文件并把 `.endef` 改回原文件名。
|
|
6
6
|
|
|
7
7
|
示例:
|
|
8
8
|
|
|
@@ -16,11 +16,10 @@ README.md -> README.md.endef -> README.md
|
|
|
16
16
|
|
|
17
17
|
## 特性
|
|
18
18
|
|
|
19
|
-
-
|
|
19
|
+
- Node.js CLI,恢复阶段可自动调用 Bash 或 PowerShell 脚本
|
|
20
20
|
- 默认旁路后缀为 `.endef`
|
|
21
21
|
- `en` 默认转换所有匹配后缀的文件
|
|
22
|
-
- `
|
|
23
|
-
- `de` 会删除原文件,用 `.endef` 临时副本内容新建原文件,再删除 `.endef`
|
|
22
|
+
- `de` 会用脚本批量删除原文件,并把 `.endef` 临时副本改回原文件名
|
|
24
23
|
- `pack` 会把恢复后的目录打包成 `.tar.gz`
|
|
25
24
|
- 不引入生产依赖
|
|
26
25
|
|
|
@@ -42,31 +41,7 @@ npm install -g endef
|
|
|
42
41
|
endef en /path/to/project
|
|
43
42
|
```
|
|
44
43
|
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
```bash
|
|
48
|
-
endef en /path/to/project --marker
|
|
49
|
-
```
|
|
50
|
-
|
|
51
|
-
手动指定 marker:
|
|
52
|
-
|
|
53
|
-
```bash
|
|
54
|
-
endef en /path/to/project --marker E-SafeNet
|
|
55
|
-
```
|
|
56
|
-
|
|
57
|
-
指定多个 marker:
|
|
58
|
-
|
|
59
|
-
```bash
|
|
60
|
-
endef en /path/to/project --marker E-SafeNet --marker BadText
|
|
61
|
-
```
|
|
62
|
-
|
|
63
|
-
也可以用英文逗号分隔:
|
|
64
|
-
|
|
65
|
-
```bash
|
|
66
|
-
endef en /path/to/project --marker E-SafeNet,BadText
|
|
67
|
-
```
|
|
68
|
-
|
|
69
|
-
删除原文件,用 `.endef` 副本内容新建原文件,再删除 `.endef`:
|
|
44
|
+
用脚本批量删除原文件,并把 `.endef` 改回原文件名:
|
|
70
45
|
|
|
71
46
|
```bash
|
|
72
47
|
endef de /path/to/project
|
|
@@ -107,11 +82,11 @@ endef all [directory] [options]
|
|
|
107
82
|
|
|
108
83
|
`en` 会扫描目标目录,把匹配后缀的文件读取出来,并写成同目录的 `.endef` 副本。
|
|
109
84
|
|
|
110
|
-
`de` 会扫描目标目录中的 `.endef`
|
|
85
|
+
`de` 会扫描目标目录中的 `.endef` 文件,并用外部脚本批量删除原文件、把 `.endef` 改回原文件名。默认模式是 `auto`:优先使用 Bash 的 `find + xargs -P`,没有 Bash 时在 Windows 上使用 PowerShell。这个命令会直接修改文件,请在执行前确认目录和备份策略。
|
|
111
86
|
|
|
112
87
|
`pack` 会按当前配置扫描当前目录,把没有被 `excludeDirs` 排除的文件打包成 `.tar.gz`。如果输出文件已存在,会直接覆盖。
|
|
113
88
|
|
|
114
|
-
`all` 会按顺序执行 `en`、`de`、`pack
|
|
89
|
+
`all` 会按顺序执行 `en`、`de`、`pack`,适合确认配置后一次性完成旁路转换、脚本恢复和打包。
|
|
115
90
|
|
|
116
91
|
## 参数
|
|
117
92
|
|
|
@@ -121,18 +96,17 @@ endef all [directory] [options]
|
|
|
121
96
|
--ext <list> 目标文件后缀,多个值用英文逗号分隔
|
|
122
97
|
--exclude <list> 排除目录名,多个值用英文逗号分隔
|
|
123
98
|
--concurrency <n> 并发文件操作数量
|
|
124
|
-
--marker [text] 只为命中任意 marker 的文件生成副本
|
|
125
99
|
--out <file> 压缩包输出名称
|
|
100
|
+
--mode <mode> de/all 恢复模式:auto、bash、powershell、node
|
|
126
101
|
```
|
|
127
102
|
|
|
128
103
|
示例:
|
|
129
104
|
|
|
130
105
|
```bash
|
|
131
106
|
endef en . --ext .js,.ts,.vue,.md --exclude node_modules,.git,dist
|
|
132
|
-
endef en . --marker E-SafeNet --marker BadText
|
|
133
107
|
endef de .
|
|
134
108
|
endef pack recovered
|
|
135
|
-
endef all . --
|
|
109
|
+
endef all . --out recovered
|
|
136
110
|
```
|
|
137
111
|
|
|
138
112
|
## 配置文件
|
|
@@ -150,7 +124,6 @@ endef all . --marker --out recovered
|
|
|
150
124
|
```json
|
|
151
125
|
{
|
|
152
126
|
"suffix": ".endef",
|
|
153
|
-
"markers": ["E-SafeNet"],
|
|
154
127
|
"extensions": [
|
|
155
128
|
".js",
|
|
156
129
|
".jsx",
|
|
@@ -186,49 +159,33 @@ endef all . --marker --out recovered
|
|
|
186
159
|
}
|
|
187
160
|
```
|
|
188
161
|
|
|
189
|
-
##
|
|
190
|
-
|
|
191
|
-
默认情况下:
|
|
192
|
-
|
|
193
|
-
```bash
|
|
194
|
-
endef en .
|
|
195
|
-
```
|
|
196
|
-
|
|
197
|
-
会为所有匹配后缀的文件生成 `.endef` 副本。
|
|
198
|
-
|
|
199
|
-
启用 marker 模式后:
|
|
200
|
-
|
|
201
|
-
```bash
|
|
202
|
-
endef en . --marker
|
|
203
|
-
```
|
|
162
|
+
## 恢复策略
|
|
204
163
|
|
|
205
|
-
|
|
164
|
+
恢复时,`endef` 会把:
|
|
206
165
|
|
|
207
|
-
```
|
|
208
|
-
|
|
166
|
+
```text
|
|
167
|
+
file.ext.endef
|
|
209
168
|
```
|
|
210
169
|
|
|
211
|
-
|
|
170
|
+
恢复为:
|
|
212
171
|
|
|
213
|
-
```
|
|
214
|
-
|
|
172
|
+
```text
|
|
173
|
+
file.ext
|
|
215
174
|
```
|
|
216
175
|
|
|
217
|
-
|
|
218
|
-
|
|
219
|
-
恢复时,`endef` 会把:
|
|
176
|
+
`de` 默认使用脚本恢复。`auto` 模式会优先尝试 Bash,因为 `find + xargs -P` 可以并行处理大量文件;如果没有 Bash,则在 Windows 上使用单个 PowerShell 进程批量处理。恢复时会把:
|
|
220
177
|
|
|
221
178
|
```text
|
|
222
179
|
file.ext.endef
|
|
223
180
|
```
|
|
224
181
|
|
|
225
|
-
|
|
182
|
+
改回:
|
|
226
183
|
|
|
227
184
|
```text
|
|
228
185
|
file.ext
|
|
229
186
|
```
|
|
230
187
|
|
|
231
|
-
如果 `file.ext`
|
|
188
|
+
如果 `file.ext` 已经存在,会先被删除。这个行为符合工具的核心假设:原文件可能已经被异常内容污染,而 `.endef` 是通过旁路读取保存出来的可恢复内容。
|
|
232
189
|
|
|
233
190
|
## 打包策略
|
|
234
191
|
|
|
@@ -286,10 +243,10 @@ endef de .
|
|
|
286
243
|
endef pack
|
|
287
244
|
```
|
|
288
245
|
|
|
289
|
-
|
|
246
|
+
同样支持排除目录和输出名称:
|
|
290
247
|
|
|
291
248
|
```bash
|
|
292
|
-
endef all . --
|
|
249
|
+
endef all . --exclude node_modules,.git,dist --out recovered
|
|
293
250
|
```
|
|
294
251
|
|
|
295
252
|
## 安全建议
|
|
@@ -297,7 +254,6 @@ endef all . --marker E-SafeNet,BadText --exclude node_modules,.git,dist --out re
|
|
|
297
254
|
- 操作前尽量手动备份目录或磁盘
|
|
298
255
|
- 优先在干净系统、离线环境或挂载出来的磁盘上运行
|
|
299
256
|
- 先在小目录中测试 `endef en` 和 `endef de`
|
|
300
|
-
- 不确定 marker 是否完整时,优先使用默认的全量 `endef en`
|
|
301
257
|
- 不要直接运行恢复出来的未知脚本或可执行文件
|
|
302
258
|
- 恢复后建议做杀毒、哈希校验和人工抽查
|
|
303
259
|
|
package/package.json
CHANGED
package/src/config.js
CHANGED
|
@@ -4,7 +4,6 @@ const path = require('path')
|
|
|
4
4
|
|
|
5
5
|
const defaultConfig = {
|
|
6
6
|
suffix: '.endef',
|
|
7
|
-
markers: ['E-SafeNet'],
|
|
8
7
|
extensions: [
|
|
9
8
|
'.js',
|
|
10
9
|
'.jsx',
|
|
@@ -36,7 +35,8 @@ const defaultConfig = {
|
|
|
36
35
|
],
|
|
37
36
|
excludeDirs: ['node_modules', '.git', 'dist', '.next', '.nuxt', '.cache'],
|
|
38
37
|
excludeFiles: ['.endef.json'],
|
|
39
|
-
concurrency: 32
|
|
38
|
+
concurrency: 32,
|
|
39
|
+
restoreMode: 'auto'
|
|
40
40
|
}
|
|
41
41
|
|
|
42
42
|
function loadConfig(cwd, explicitConfigFile, targetDirectory) {
|
|
@@ -73,35 +73,13 @@ function stripBom(value) {
|
|
|
73
73
|
}
|
|
74
74
|
|
|
75
75
|
function mergeConfig(base, next) {
|
|
76
|
-
const markers = normalizeMarkers(next.markers)
|
|
77
|
-
|
|
78
76
|
return {
|
|
79
77
|
...base,
|
|
80
78
|
...next,
|
|
81
79
|
extensions: Array.isArray(next.extensions) ? next.extensions : base.extensions,
|
|
82
80
|
excludeDirs: Array.isArray(next.excludeDirs) ? next.excludeDirs : base.excludeDirs,
|
|
83
|
-
excludeFiles: Array.isArray(next.excludeFiles) ? next.excludeFiles : base.excludeFiles
|
|
84
|
-
markers: markers.length > 0 ? markers : base.markers
|
|
81
|
+
excludeFiles: Array.isArray(next.excludeFiles) ? next.excludeFiles : base.excludeFiles
|
|
85
82
|
}
|
|
86
83
|
}
|
|
87
84
|
|
|
88
|
-
|
|
89
|
-
if (Array.isArray(value)) {
|
|
90
|
-
return value.map(item => String(item).trim()).filter(Boolean)
|
|
91
|
-
}
|
|
92
|
-
|
|
93
|
-
return []
|
|
94
|
-
}
|
|
95
|
-
|
|
96
|
-
function splitMarkers(value) {
|
|
97
|
-
if (!value) {
|
|
98
|
-
return []
|
|
99
|
-
}
|
|
100
|
-
|
|
101
|
-
return String(value)
|
|
102
|
-
.split(',')
|
|
103
|
-
.map(item => item.trim())
|
|
104
|
-
.filter(Boolean)
|
|
105
|
-
}
|
|
106
|
-
|
|
107
|
-
module.exports = { defaultConfig, loadConfig, normalizeMarkers, splitMarkers }
|
|
85
|
+
module.exports = { defaultConfig, loadConfig }
|
package/src/de.js
CHANGED
|
@@ -1,95 +1,233 @@
|
|
|
1
1
|
const fs = require('fs/promises')
|
|
2
2
|
const path = require('path')
|
|
3
|
+
const { execFile } = require('child_process')
|
|
4
|
+
const { promisify } = require('util')
|
|
3
5
|
const { defaultConfig } = require('./config')
|
|
4
|
-
|
|
6
|
+
|
|
7
|
+
const execFileAsync = promisify(execFile)
|
|
5
8
|
|
|
6
9
|
async function restoreFiles(directory, options = {}) {
|
|
7
10
|
const config = {
|
|
8
11
|
...defaultConfig,
|
|
9
12
|
...options,
|
|
10
|
-
directory: path.resolve(directory || process.cwd())
|
|
13
|
+
directory: path.resolve(directory || process.cwd()),
|
|
14
|
+
restoreMode: options.restoreMode || 'auto'
|
|
11
15
|
}
|
|
12
|
-
const
|
|
13
|
-
const limit = createLimiter(config.concurrency)
|
|
14
|
-
const tasks = []
|
|
16
|
+
const mode = await resolveRestoreMode(config.restoreMode)
|
|
15
17
|
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
}
|
|
18
|
+
if (mode === 'bash') {
|
|
19
|
+
return restoreWithBash(config)
|
|
20
|
+
}
|
|
20
21
|
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
}).catch(error => {
|
|
25
|
-
stats.failed += 1
|
|
26
|
-
console.error(`Failed: ${filePath}`)
|
|
27
|
-
console.error(error && error.message ? error.message : error)
|
|
28
|
-
})
|
|
29
|
-
)
|
|
30
|
-
})
|
|
22
|
+
if (mode === 'powershell') {
|
|
23
|
+
return restoreWithPowerShell(config)
|
|
24
|
+
}
|
|
31
25
|
|
|
32
|
-
|
|
33
|
-
console.log(
|
|
34
|
-
`Done. restored=${stats.restored}, removed=${stats.removed}, created=${stats.created}, deleted=${stats.deleted}, failed=${stats.failed}`
|
|
35
|
-
)
|
|
36
|
-
return stats
|
|
26
|
+
return restoreWithNode(config)
|
|
37
27
|
}
|
|
38
28
|
|
|
39
|
-
async function
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
29
|
+
async function resolveRestoreMode(mode) {
|
|
30
|
+
if (mode !== 'auto') {
|
|
31
|
+
return mode
|
|
32
|
+
}
|
|
43
33
|
|
|
44
|
-
await
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
await fs.rm(filePath, { force: true })
|
|
34
|
+
if (await hasCommand('bash', ['--version'])) {
|
|
35
|
+
return 'bash'
|
|
36
|
+
}
|
|
48
37
|
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
stats.removed += 1
|
|
38
|
+
if (process.platform === 'win32') {
|
|
39
|
+
return 'powershell'
|
|
52
40
|
}
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
console.log(`Restored: ${filePath} -> ${targetPath}`)
|
|
41
|
+
|
|
42
|
+
return 'node'
|
|
56
43
|
}
|
|
57
44
|
|
|
58
|
-
async function
|
|
45
|
+
async function hasCommand(command, args) {
|
|
59
46
|
try {
|
|
60
|
-
await
|
|
47
|
+
await execFileAsync(command, args, { windowsHide: true })
|
|
61
48
|
return true
|
|
62
49
|
} catch (error) {
|
|
63
50
|
return false
|
|
64
51
|
}
|
|
65
52
|
}
|
|
66
53
|
|
|
67
|
-
async function
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
54
|
+
async function restoreWithBash(config) {
|
|
55
|
+
const root = await toBashPath(config.directory)
|
|
56
|
+
const excludeDirs = config.excludeDirs.join('\n')
|
|
57
|
+
const jobs = String(config.concurrency || 32)
|
|
58
|
+
const script = `
|
|
59
|
+
set -e
|
|
60
|
+
root=$1
|
|
61
|
+
suffix=$2
|
|
62
|
+
jobs=$3
|
|
63
|
+
cd "$root"
|
|
64
|
+
find_args=(.)
|
|
65
|
+
while IFS= read -r dir; do
|
|
66
|
+
if [ -n "$dir" ]; then
|
|
67
|
+
find_args+=( -path "*/$dir" -prune -o )
|
|
68
|
+
fi
|
|
69
|
+
done <<< "$ENDEF_EXCLUDE_DIRS"
|
|
70
|
+
find_args+=( -type f -name "*$suffix" -print0 )
|
|
71
|
+
find "\${find_args[@]}" | xargs -0 -r -P "$jobs" -I{} bash -c '
|
|
72
|
+
suffix="$1"
|
|
73
|
+
file="$2"
|
|
74
|
+
target="\${file%$suffix}"
|
|
75
|
+
rm -f -- "$target"
|
|
76
|
+
mv -f -- "$file" "$target"
|
|
77
|
+
printf "%s\\0" "$target"
|
|
78
|
+
' _ "$suffix" {}
|
|
79
|
+
`
|
|
72
80
|
|
|
73
|
-
|
|
81
|
+
const { stdout } = await execFileAsync('bash', ['-lc', script, '_', root, config.suffix, jobs], {
|
|
82
|
+
env: {
|
|
83
|
+
...process.env,
|
|
84
|
+
ENDEF_EXCLUDE_DIRS: excludeDirs
|
|
85
|
+
},
|
|
86
|
+
maxBuffer: 1024 * 1024 * 64,
|
|
87
|
+
windowsHide: true
|
|
88
|
+
})
|
|
89
|
+
const restored = countNullSeparated(stdout)
|
|
90
|
+
|
|
91
|
+
console.log(`Done. mode=bash, restored=${restored}, failed=0`)
|
|
92
|
+
return {
|
|
93
|
+
mode: 'bash',
|
|
94
|
+
restored,
|
|
95
|
+
failed: 0
|
|
96
|
+
}
|
|
97
|
+
}
|
|
98
|
+
|
|
99
|
+
async function toBashPath(filePath) {
|
|
100
|
+
if (process.platform !== 'win32') {
|
|
101
|
+
return filePath
|
|
74
102
|
}
|
|
75
103
|
|
|
76
|
-
|
|
104
|
+
try {
|
|
105
|
+
const { stdout } = await execFileAsync('bash', ['-lc', 'cygpath -u "$1"', '_', filePath], {
|
|
106
|
+
windowsHide: true
|
|
107
|
+
})
|
|
108
|
+
|
|
109
|
+
return stdout.trim() || filePath
|
|
110
|
+
} catch (error) {
|
|
111
|
+
return filePath
|
|
112
|
+
}
|
|
77
113
|
}
|
|
78
114
|
|
|
79
|
-
function
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
115
|
+
async function restoreWithPowerShell(config) {
|
|
116
|
+
const script = `
|
|
117
|
+
$root = $env:ENDEF_ROOT
|
|
118
|
+
$suffix = $env:ENDEF_SUFFIX
|
|
119
|
+
$excludeText = $env:ENDEF_EXCLUDE_DIRS
|
|
120
|
+
$excludeDirs = @()
|
|
121
|
+
if ($excludeText) {
|
|
122
|
+
$excludeDirs = $excludeText -split "\\n" | Where-Object { $_ }
|
|
123
|
+
}
|
|
124
|
+
$restored = 0
|
|
125
|
+
Get-ChildItem -LiteralPath $root -Recurse -File -Filter "*$suffix" | Where-Object {
|
|
126
|
+
$cleanRoot = $root.TrimEnd('\\', '/')
|
|
127
|
+
$relative = $_.FullName.Substring($cleanRoot.Length).TrimStart('\\', '/')
|
|
128
|
+
$parts = $relative -split '[\\\\/]'
|
|
129
|
+
$skip = $false
|
|
130
|
+
foreach ($dir in $excludeDirs) {
|
|
131
|
+
if ($parts -contains $dir) {
|
|
132
|
+
$skip = $true
|
|
133
|
+
break
|
|
134
|
+
}
|
|
135
|
+
}
|
|
136
|
+
-not $skip
|
|
137
|
+
} | ForEach-Object {
|
|
138
|
+
$target = $_.FullName.Substring(0, $_.FullName.Length - $suffix.Length)
|
|
139
|
+
Remove-Item -LiteralPath $target -Force -ErrorAction SilentlyContinue
|
|
140
|
+
Move-Item -LiteralPath $_.FullName -Destination $target -Force
|
|
141
|
+
$restored += 1
|
|
83
142
|
}
|
|
143
|
+
Write-Output $restored
|
|
144
|
+
`
|
|
145
|
+
const { stdout } = await execFileAsync(
|
|
146
|
+
'powershell.exe',
|
|
147
|
+
[
|
|
148
|
+
'-NoProfile',
|
|
149
|
+
'-NonInteractive',
|
|
150
|
+
'-ExecutionPolicy',
|
|
151
|
+
'Bypass',
|
|
152
|
+
'-Command',
|
|
153
|
+
script
|
|
154
|
+
],
|
|
155
|
+
{
|
|
156
|
+
env: {
|
|
157
|
+
...process.env,
|
|
158
|
+
ENDEF_ROOT: config.directory,
|
|
159
|
+
ENDEF_SUFFIX: config.suffix,
|
|
160
|
+
ENDEF_EXCLUDE_DIRS: config.excludeDirs.join('\n')
|
|
161
|
+
},
|
|
162
|
+
maxBuffer: 1024 * 1024 * 64,
|
|
163
|
+
windowsHide: true
|
|
164
|
+
}
|
|
165
|
+
)
|
|
166
|
+
const restored = Number(stdout.trim()) || 0
|
|
84
167
|
|
|
85
|
-
|
|
168
|
+
console.log(`Done. mode=powershell, restored=${restored}, failed=0`)
|
|
86
169
|
return {
|
|
170
|
+
mode: 'powershell',
|
|
171
|
+
restored,
|
|
172
|
+
failed: 0
|
|
173
|
+
}
|
|
174
|
+
}
|
|
175
|
+
|
|
176
|
+
async function restoreWithNode(config) {
|
|
177
|
+
const stats = {
|
|
87
178
|
restored: 0,
|
|
88
|
-
removed: 0,
|
|
89
|
-
created: 0,
|
|
90
|
-
deleted: 0,
|
|
91
179
|
failed: 0
|
|
92
180
|
}
|
|
181
|
+
|
|
182
|
+
await restoreNodeDirectory(config.directory, config, stats)
|
|
183
|
+
console.log(`Done. mode=node, restored=${stats.restored}, failed=${stats.failed}`)
|
|
184
|
+
return {
|
|
185
|
+
mode: 'node',
|
|
186
|
+
restored: stats.restored,
|
|
187
|
+
failed: stats.failed
|
|
188
|
+
}
|
|
189
|
+
}
|
|
190
|
+
|
|
191
|
+
async function restoreNodeDirectory(directory, config, stats) {
|
|
192
|
+
const entries = await fs.readdir(directory, { withFileTypes: true })
|
|
193
|
+
|
|
194
|
+
await Promise.all(
|
|
195
|
+
entries.map(async entry => {
|
|
196
|
+
const filePath = path.join(directory, entry.name)
|
|
197
|
+
|
|
198
|
+
if (entry.isDirectory()) {
|
|
199
|
+
if (config.excludeDirs.includes(entry.name)) {
|
|
200
|
+
return
|
|
201
|
+
}
|
|
202
|
+
|
|
203
|
+
await restoreNodeDirectory(filePath, config, stats)
|
|
204
|
+
return
|
|
205
|
+
}
|
|
206
|
+
|
|
207
|
+
if (!entry.isFile() || !filePath.endsWith(config.suffix)) {
|
|
208
|
+
return
|
|
209
|
+
}
|
|
210
|
+
|
|
211
|
+
try {
|
|
212
|
+
const targetPath = filePath.slice(0, -config.suffix.length)
|
|
213
|
+
await fs.rm(targetPath, { force: true })
|
|
214
|
+
await fs.rename(filePath, targetPath)
|
|
215
|
+
stats.restored += 1
|
|
216
|
+
} catch (error) {
|
|
217
|
+
stats.failed += 1
|
|
218
|
+
console.error(`Failed: ${filePath}`)
|
|
219
|
+
console.error(error && error.message ? error.message : error)
|
|
220
|
+
}
|
|
221
|
+
})
|
|
222
|
+
)
|
|
223
|
+
}
|
|
224
|
+
|
|
225
|
+
function countNullSeparated(value) {
|
|
226
|
+
if (!value) {
|
|
227
|
+
return 0
|
|
228
|
+
}
|
|
229
|
+
|
|
230
|
+
return value.split('\0').filter(Boolean).length
|
|
93
231
|
}
|
|
94
232
|
|
|
95
233
|
module.exports = restoreFiles
|
package/src/en.js
CHANGED
|
@@ -29,11 +29,6 @@ async function processFiles(directory, options = {}) {
|
|
|
29
29
|
const outputPath = `${filePath}${config.suffix}`
|
|
30
30
|
const data = await fs.readFile(filePath)
|
|
31
31
|
|
|
32
|
-
if (config.markerFilter && !matchesAnyMarker(data, config.markers)) {
|
|
33
|
-
stats.skipped += 1
|
|
34
|
-
return
|
|
35
|
-
}
|
|
36
|
-
|
|
37
32
|
await fs.writeFile(outputPath, data)
|
|
38
33
|
console.log(`Written: ${outputPath}`)
|
|
39
34
|
stats.written += 1
|
|
@@ -58,12 +53,4 @@ function createStats() {
|
|
|
58
53
|
}
|
|
59
54
|
}
|
|
60
55
|
|
|
61
|
-
function matchesAnyMarker(data, markers) {
|
|
62
|
-
if (!Array.isArray(markers) || markers.length === 0) {
|
|
63
|
-
return false
|
|
64
|
-
}
|
|
65
|
-
|
|
66
|
-
return markers.some(marker => data.includes(marker))
|
|
67
|
-
}
|
|
68
|
-
|
|
69
56
|
module.exports = processFiles
|
package/src/pack.js
CHANGED
|
@@ -2,36 +2,93 @@ const fs = require('fs')
|
|
|
2
2
|
const fsp = require('fs/promises')
|
|
3
3
|
const path = require('path')
|
|
4
4
|
const zlib = require('zlib')
|
|
5
|
+
const { promisify } = require('util')
|
|
5
6
|
const { once } = require('events')
|
|
6
7
|
const { pipeline } = require('stream/promises')
|
|
7
8
|
const { defaultConfig } = require('./config')
|
|
8
9
|
const { walk } = require('./utils')
|
|
9
10
|
|
|
11
|
+
const gunzip = promisify(zlib.gunzip)
|
|
12
|
+
|
|
10
13
|
async function packFiles(directory, options = {}) {
|
|
11
|
-
const config =
|
|
14
|
+
const config = createPackConfig(directory, options)
|
|
15
|
+
const outputPath = resolveOutputPath(config.directory, config.out)
|
|
16
|
+
const entries = await collectPackEntries(config, outputPath)
|
|
17
|
+
|
|
18
|
+
await fsp.rm(outputPath, { force: true })
|
|
19
|
+
await writeTarGz(config.directory, outputPath, entries)
|
|
20
|
+
console.log(`Packed: ${outputPath}`)
|
|
21
|
+
console.log(`Done. files=${entries.length}`)
|
|
22
|
+
|
|
23
|
+
return {
|
|
24
|
+
outputPath,
|
|
25
|
+
files: entries.length
|
|
26
|
+
}
|
|
27
|
+
}
|
|
28
|
+
|
|
29
|
+
function createPackConfig(directory, options) {
|
|
30
|
+
return {
|
|
12
31
|
...defaultConfig,
|
|
13
32
|
...options,
|
|
14
33
|
directory: path.resolve(directory || process.cwd())
|
|
15
34
|
}
|
|
16
|
-
|
|
17
|
-
|
|
35
|
+
}
|
|
36
|
+
|
|
37
|
+
async function collectPackEntries(config, outputPath) {
|
|
38
|
+
const entries = []
|
|
18
39
|
|
|
19
40
|
await walk(config.directory, config, async filePath => {
|
|
20
|
-
if (
|
|
41
|
+
if (shouldSkipPackFile(filePath, config, outputPath)) {
|
|
21
42
|
return
|
|
22
43
|
}
|
|
23
44
|
|
|
24
|
-
|
|
45
|
+
entries.push(createEntry(config.directory, filePath, filePath))
|
|
25
46
|
})
|
|
26
47
|
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
48
|
+
return entries
|
|
49
|
+
}
|
|
50
|
+
|
|
51
|
+
async function collectRecoveryEntries(config, options = {}) {
|
|
52
|
+
const entries = []
|
|
53
|
+
const sidecarTargets = new Set()
|
|
54
|
+
|
|
55
|
+
await walk(config.directory, config, async filePath => {
|
|
56
|
+
if (filePath.endsWith(config.suffix)) {
|
|
57
|
+
sidecarTargets.add(path.resolve(filePath.slice(0, -config.suffix.length)))
|
|
58
|
+
}
|
|
59
|
+
})
|
|
60
|
+
|
|
61
|
+
await walk(config.directory, config, async filePath => {
|
|
62
|
+
const resolvedPath = path.resolve(filePath)
|
|
63
|
+
|
|
64
|
+
if (options.outputPath && resolvedPath === path.resolve(options.outputPath)) {
|
|
65
|
+
return
|
|
66
|
+
}
|
|
67
|
+
|
|
68
|
+
if (filePath.endsWith(config.suffix)) {
|
|
69
|
+
const targetPath = filePath.slice(0, -config.suffix.length)
|
|
70
|
+
entries.push(createEntry(config.directory, filePath, targetPath))
|
|
71
|
+
return
|
|
72
|
+
}
|
|
73
|
+
|
|
74
|
+
if (options.onlySidecars || sidecarTargets.has(resolvedPath)) {
|
|
75
|
+
return
|
|
76
|
+
}
|
|
77
|
+
|
|
78
|
+
entries.push(createEntry(config.directory, filePath, filePath))
|
|
79
|
+
})
|
|
80
|
+
|
|
81
|
+
return entries
|
|
82
|
+
}
|
|
83
|
+
|
|
84
|
+
function shouldSkipPackFile(filePath, config, outputPath) {
|
|
85
|
+
return path.resolve(filePath) === outputPath || filePath.endsWith(config.suffix)
|
|
86
|
+
}
|
|
31
87
|
|
|
88
|
+
function createEntry(rootDirectory, sourcePath, archiveTargetPath) {
|
|
32
89
|
return {
|
|
33
|
-
|
|
34
|
-
|
|
90
|
+
sourcePath,
|
|
91
|
+
archivePath: toArchivePath(path.relative(rootDirectory, archiveTargetPath))
|
|
35
92
|
}
|
|
36
93
|
}
|
|
37
94
|
|
|
@@ -51,30 +108,77 @@ function normalizeOutputName(out) {
|
|
|
51
108
|
return `${out}.tar.gz`
|
|
52
109
|
}
|
|
53
110
|
|
|
54
|
-
async function writeTarGz(rootDirectory, outputPath,
|
|
111
|
+
async function writeTarGz(rootDirectory, outputPath, entries) {
|
|
55
112
|
await fsp.mkdir(path.dirname(outputPath), { recursive: true })
|
|
56
113
|
|
|
57
114
|
const gzip = zlib.createGzip()
|
|
58
115
|
const output = fs.createWriteStream(outputPath)
|
|
59
|
-
const writer = writeTarEntries(
|
|
116
|
+
const writer = writeTarEntries(entries, gzip)
|
|
60
117
|
|
|
61
118
|
await Promise.all([writer, pipeline(gzip, output)])
|
|
62
119
|
}
|
|
63
120
|
|
|
64
|
-
async function writeTarEntries(
|
|
65
|
-
for (const
|
|
66
|
-
const stat = await fsp.stat(
|
|
67
|
-
const archivePath = toArchivePath(path.relative(rootDirectory, filePath))
|
|
68
|
-
|
|
69
|
-
await writeToStream(stream, createHeader(archivePath, stat))
|
|
121
|
+
async function writeTarEntries(entries, stream) {
|
|
122
|
+
for (const entry of entries) {
|
|
123
|
+
const stat = await fsp.stat(entry.sourcePath)
|
|
70
124
|
|
|
71
|
-
await
|
|
125
|
+
await writeToStream(stream, createHeader(entry.archivePath, stat))
|
|
126
|
+
await writeFileToStream(entry.sourcePath, stream)
|
|
72
127
|
await writeToStream(stream, Buffer.alloc(paddingSize(stat.size)))
|
|
73
128
|
}
|
|
74
129
|
|
|
75
130
|
stream.end(Buffer.alloc(1024))
|
|
76
131
|
}
|
|
77
132
|
|
|
133
|
+
async function extractTarGz(archivePath, directory) {
|
|
134
|
+
const archive = await fsp.readFile(archivePath)
|
|
135
|
+
const data = await gunzip(archive)
|
|
136
|
+
let offset = 0
|
|
137
|
+
let extracted = 0
|
|
138
|
+
|
|
139
|
+
while (offset + 512 <= data.length) {
|
|
140
|
+
const header = data.subarray(offset, offset + 512)
|
|
141
|
+
|
|
142
|
+
if (header.every(value => value === 0)) {
|
|
143
|
+
break
|
|
144
|
+
}
|
|
145
|
+
|
|
146
|
+
const entry = readHeader(header)
|
|
147
|
+
offset += 512
|
|
148
|
+
|
|
149
|
+
const content = data.subarray(offset, offset + entry.size)
|
|
150
|
+
const outputPath = path.resolve(directory, entry.name)
|
|
151
|
+
|
|
152
|
+
if (!outputPath.startsWith(path.resolve(directory) + path.sep)) {
|
|
153
|
+
throw new Error(`Unsafe archive path: ${entry.name}`)
|
|
154
|
+
}
|
|
155
|
+
|
|
156
|
+
await fsp.mkdir(path.dirname(outputPath), { recursive: true })
|
|
157
|
+
await fsp.rm(outputPath, { force: true })
|
|
158
|
+
await fsp.writeFile(outputPath, content, { flag: 'wx' })
|
|
159
|
+
|
|
160
|
+
offset += entry.size + paddingSize(entry.size)
|
|
161
|
+
extracted += 1
|
|
162
|
+
}
|
|
163
|
+
|
|
164
|
+
return extracted
|
|
165
|
+
}
|
|
166
|
+
|
|
167
|
+
async function removeSidecars(directory, config) {
|
|
168
|
+
let deleted = 0
|
|
169
|
+
|
|
170
|
+
await walk(directory, config, async filePath => {
|
|
171
|
+
if (!filePath.endsWith(config.suffix)) {
|
|
172
|
+
return
|
|
173
|
+
}
|
|
174
|
+
|
|
175
|
+
await fsp.rm(filePath, { force: true })
|
|
176
|
+
deleted += 1
|
|
177
|
+
})
|
|
178
|
+
|
|
179
|
+
return deleted
|
|
180
|
+
}
|
|
181
|
+
|
|
78
182
|
async function writeFileToStream(filePath, stream) {
|
|
79
183
|
for await (const chunk of fs.createReadStream(filePath)) {
|
|
80
184
|
await writeToStream(stream, chunk)
|
|
@@ -113,6 +217,20 @@ function createHeader(filePath, stat) {
|
|
|
113
217
|
return header
|
|
114
218
|
}
|
|
115
219
|
|
|
220
|
+
function readHeader(header) {
|
|
221
|
+
const name = readString(header, 0, 100)
|
|
222
|
+
const prefix = readString(header, 345, 155)
|
|
223
|
+
|
|
224
|
+
return {
|
|
225
|
+
name: prefix ? `${prefix}/${name}` : name,
|
|
226
|
+
size: parseInt(readString(header, 124, 12).trim() || '0', 8)
|
|
227
|
+
}
|
|
228
|
+
}
|
|
229
|
+
|
|
230
|
+
function readString(buffer, offset, length) {
|
|
231
|
+
return buffer.subarray(offset, offset + length).toString('utf8').replace(/\0.*$/, '')
|
|
232
|
+
}
|
|
233
|
+
|
|
116
234
|
function splitTarPath(filePath) {
|
|
117
235
|
if (Buffer.byteLength(filePath) <= 100) {
|
|
118
236
|
return {
|
|
@@ -156,3 +274,9 @@ function paddingSize(size) {
|
|
|
156
274
|
}
|
|
157
275
|
|
|
158
276
|
module.exports = packFiles
|
|
277
|
+
module.exports.createPackConfig = createPackConfig
|
|
278
|
+
module.exports.collectRecoveryEntries = collectRecoveryEntries
|
|
279
|
+
module.exports.extractTarGz = extractTarGz
|
|
280
|
+
module.exports.removeSidecars = removeSidecars
|
|
281
|
+
module.exports.resolveOutputPath = resolveOutputPath
|
|
282
|
+
module.exports.writeTarGz = writeTarGz
|
package/src/run.js
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
const enF = require('./en')
|
|
2
2
|
const deF = require('./de')
|
|
3
3
|
const packF = require('./pack')
|
|
4
|
-
const { loadConfig
|
|
4
|
+
const { loadConfig } = require('./config')
|
|
5
5
|
|
|
6
6
|
function printHelp() {
|
|
7
7
|
console.log(`Usage:
|
|
@@ -16,16 +16,14 @@ Options:
|
|
|
16
16
|
--ext <list> Comma separated target extensions
|
|
17
17
|
--exclude <list> Comma separated excluded directory names
|
|
18
18
|
--concurrency <n> Concurrent file operation limit
|
|
19
|
-
--marker [text] Only create sidecar files when any marker is found
|
|
20
19
|
--out <file> Archive output name
|
|
20
|
+
--mode <mode> Restore mode for de/all: auto, bash, powershell, node
|
|
21
21
|
`)
|
|
22
22
|
}
|
|
23
23
|
|
|
24
24
|
function parseArgs(command, args) {
|
|
25
25
|
const result = {
|
|
26
26
|
directory: process.cwd(),
|
|
27
|
-
markerFilter: undefined,
|
|
28
|
-
markers: [],
|
|
29
27
|
positionals: []
|
|
30
28
|
}
|
|
31
29
|
|
|
@@ -98,20 +96,14 @@ function parseArgs(command, args) {
|
|
|
98
96
|
continue
|
|
99
97
|
}
|
|
100
98
|
|
|
101
|
-
if (arg.startsWith('--
|
|
102
|
-
result.
|
|
103
|
-
result.markers.push(...splitMarkers(arg.slice('--marker='.length)))
|
|
99
|
+
if (arg.startsWith('--mode=')) {
|
|
100
|
+
result.restoreMode = arg.slice('--mode='.length)
|
|
104
101
|
continue
|
|
105
102
|
}
|
|
106
103
|
|
|
107
|
-
if (arg === '--
|
|
108
|
-
result.
|
|
109
|
-
|
|
110
|
-
if (args[index + 1] && !args[index + 1].startsWith('-')) {
|
|
111
|
-
result.markers.push(...splitMarkers(args[index + 1]))
|
|
112
|
-
index += 1
|
|
113
|
-
}
|
|
114
|
-
|
|
104
|
+
if (arg === '--mode') {
|
|
105
|
+
result.restoreMode = args[index + 1]
|
|
106
|
+
index += 1
|
|
115
107
|
continue
|
|
116
108
|
}
|
|
117
109
|
|
|
@@ -160,8 +152,7 @@ async function run(args) {
|
|
|
160
152
|
const options = {
|
|
161
153
|
...config,
|
|
162
154
|
...cliOptions,
|
|
163
|
-
directory: cliOptions.directory || config.directory || process.cwd()
|
|
164
|
-
markers: cliOptions.markers && cliOptions.markers.length > 0 ? cliOptions.markers : config.markers
|
|
155
|
+
directory: cliOptions.directory || config.directory || process.cwd()
|
|
165
156
|
}
|
|
166
157
|
|
|
167
158
|
if (command === 'en') {
|