endef 2.0.1 → 2.0.3

package/README.md

@@ -2,7 +2,7 @@
 
 `endef` 是一个纯 Node.js 的文件旁路转换工具。
 
-它适合这样的场景：某些常见源码或文本后缀的文件在系统读取、复制、粘贴、编辑器打开时出现乱码或异常，但通过 Node.js 脚本读取仍然正常。`endef` 会先把目标文件内容写到同目录的 `.endef` 副本里，再删除原文件，并把 `.endef` 副本重命名回原文件名。
+它适合这样的场景：某些常见源码或文本后缀的文件在系统读取、复制、粘贴、编辑器打开时出现乱码或异常，但通过 Node.js 脚本读取仍然正常。`endef` 会先把目标文件内容写到同目录的 `.endef` 副本里，再通过临时 `.tar.gz` 恢复包解压回原文件名。
 
 示例：
 
@@ -19,8 +19,7 @@ README.md -> README.md.endef -> README.md
 - 纯 Node.js 实现，不依赖 Bash、find、xargs、grep 等 shell 工具
 - 默认旁路后缀为 `.endef`
 - `en` 默认转换所有匹配后缀的文件
-- `en --marker` 可以只转换命中 marker 的文件，marker 支持数组配置，也支持 CLI 手动指定多个
-- `de` 会删除原文件，并把 `.endef` 临时副本重命名回原文件名
+- `de` 会把 `.endef` 临时副本做成恢复包，解压回原文件名，再删除 `.endef`
 - `pack` 会把恢复后的目录打包成 `.tar.gz`
 - 不引入生产依赖
 
@@ -42,31 +41,7 @@ npm install -g endef
 endef en /path/to/project
 ```
 
-只转换命中默认 marker 的文件：
-
-```bash
-endef en /path/to/project --marker
-```
-
-手动指定 marker：
-
-```bash
-endef en /path/to/project --marker E-SafeNet
-```
-
-指定多个 marker：
-
-```bash
-endef en /path/to/project --marker E-SafeNet --marker BadText
-```
-
-也可以用英文逗号分隔：
-
-```bash
-endef en /path/to/project --marker E-SafeNet,BadText
-```
-
-删除原文件，并把 `.endef` 副本重命名回原文件名：
+通过临时恢复包解压回原文件名，并删除 `.endef`：
 
 ```bash
 endef de /path/to/project
@@ -107,11 +82,11 @@ endef all [directory] [options]
 
 `en` 会扫描目标目录，把匹配后缀的文件读取出来，并写成同目录的 `.endef` 副本。
 
-`de` 会扫描目标目录中的 `.endef` 文件，删除对应的原文件，然后把 `.endef` 副本重命名回原文件名。这个命令会直接修改文件，请在执行前确认目录和备份策略。
+`de` 会扫描目标目录中的 `.endef` 文件，生成一个临时恢复包，把 `.endef` 内容映射回原文件名并解压，然后删除 `.endef` 副本。这个命令会直接修改文件，请在执行前确认目录和备份策略。
 
 `pack` 会按当前配置扫描当前目录，把没有被 `excludeDirs` 排除的文件打包成 `.tar.gz`。如果输出文件已存在，会直接覆盖。
 
-`all` 会按顺序执行 `en`、`de`、`pack`，适合确认配置后一次性完成旁路转换、重命名恢复和打包。
+`all` 会先执行 `en`，然后生成一次最终恢复包；这个包既会被解压回目录，也会作为最终 `.tar.gz` 保留下来，避免 `de` 和 `pack` 重复打包。
 
 ## 参数
 
@@ -121,7 +96,6 @@ endef all [directory] [options]
 --ext <list>          目标文件后缀，多个值用英文逗号分隔
 --exclude <list>      排除目录名，多个值用英文逗号分隔
 --concurrency <n>     并发文件操作数量
---marker [text]       只为命中任意 marker 的文件生成副本
 --out <file>          压缩包输出名称
 ```
 
@@ -129,10 +103,9 @@ endef all [directory] [options]
 
 ```bash
 endef en . --ext .js,.ts,.vue,.md --exclude node_modules,.git,dist
-endef en . --marker E-SafeNet --marker BadText
 endef de .
 endef pack recovered
-endef all . --marker --out recovered
+endef all . --out recovered
 ```
 
 ## 配置文件
@@ -150,7 +123,6 @@ endef all . --marker --out recovered
 ```json
 {
   "suffix": ".endef",
-  "markers": ["E-SafeNet"],
   "extensions": [
     ".js",
     ".jsx",
@@ -186,34 +158,6 @@ endef all . --marker --out recovered
 }
 ```
 
-## marker 模式
-
-默认情况下：
-
-```bash
-endef en .
-```
-
-会为所有匹配后缀的文件生成 `.endef` 副本。
-
-启用 marker 模式后：
-
-```bash
-endef en . --marker
-```
-
-只会为命中任意 marker 的文件生成 `.endef` 副本。`--marker` 不带值时使用配置中的 `markers`，默认是：
-
-```json
-["E-SafeNet"]
-```
-
-如果 CLI 指定了 marker，则使用 CLI 的值：
-
-```bash
-endef en . --marker E-SafeNet --marker BadText
-```
-
 ## 恢复策略
 
 恢复时，`endef` 会把：
@@ -222,13 +166,13 @@ endef en . --marker E-SafeNet --marker BadText
 file.ext.endef
 ```
 
-删除原文件后重命名为：
+通过临时恢复包解压为：
 
 ```text
 file.ext
 ```
 
-如果 `file.ext` 已经存在，会先被删除，然后由 `.endef` 副本重命名得到新的 `file.ext`。这个行为符合工具的核心假设：原文件可能已经被异常内容污染，而 `.endef` 是通过旁路读取保存出来的可恢复内容。
+`de` 会先生成临时 `.tar.gz` 恢复包，包内路径使用原文件名；随后解压这个恢复包覆盖原文件，解压成功后删除 `file.ext.endef` 和临时恢复包。这个行为符合工具的核心假设：原文件可能已经被异常内容污染，而 `.endef` 是通过旁路读取保存出来的可恢复内容。
 
 ## 打包策略
 
@@ -286,10 +230,10 @@ endef de .
 endef pack
 ```
 
-同样支持 marker、排除目录和输出名称：
+同样支持排除目录和输出名称：
 
 ```bash
-endef all . --marker E-SafeNet,BadText --exclude node_modules,.git,dist --out recovered
+endef all . --exclude node_modules,.git,dist --out recovered
 ```
 
 ## 安全建议
@@ -297,7 +241,6 @@ endef all . --marker E-SafeNet,BadText --exclude node_modules,.git,dist --out re
 - 操作前尽量手动备份目录或磁盘
 - 优先在干净系统、离线环境或挂载出来的磁盘上运行
 - 先在小目录中测试 `endef en` 和 `endef de`
-- 不确定 marker 是否完整时，优先使用默认的全量 `endef en`
 - 不要直接运行恢复出来的未知脚本或可执行文件
 - 恢复后建议做杀毒、哈希校验和人工抽查
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "endef",
-  "version": "2.0.1",
+  "version": "2.0.3",
   "description": "Recover text and source files through safe .endef sidecar copies.",
   "bin": {
     "endef": "bin/main.js"

package/src/config.js

@@ -4,7 +4,6 @@ const path = require('path')
 
 const defaultConfig = {
   suffix: '.endef',
-  markers: ['E-SafeNet'],
   extensions: [
     '.js',
     '.jsx',
@@ -73,35 +72,13 @@ function stripBom(value) {
 }
 
 function mergeConfig(base, next) {
-  const markers = normalizeMarkers(next.markers)
-
   return {
     ...base,
     ...next,
     extensions: Array.isArray(next.extensions) ? next.extensions : base.extensions,
     excludeDirs: Array.isArray(next.excludeDirs) ? next.excludeDirs : base.excludeDirs,
-    excludeFiles: Array.isArray(next.excludeFiles) ? next.excludeFiles : base.excludeFiles,
-    markers: markers.length > 0 ? markers : base.markers
+    excludeFiles: Array.isArray(next.excludeFiles) ? next.excludeFiles : base.excludeFiles
   }
 }
 
-function normalizeMarkers(value) {
-  if (Array.isArray(value)) {
-    return value.map(item => String(item).trim()).filter(Boolean)
-  }
-
-  return []
-}
-
-function splitMarkers(value) {
-  if (!value) {
-    return []
-  }
-
-  return String(value)
-    .split(',')
-    .map(item => item.trim())
-    .filter(Boolean)
-}
-
-module.exports = { defaultConfig, loadConfig, normalizeMarkers, splitMarkers }
+module.exports = { defaultConfig, loadConfig }

package/src/de.js

@@ -1,69 +1,47 @@
 const fs = require('fs/promises')
 const path = require('path')
 const { defaultConfig } = require('./config')
-const { createLimiter, walk } = require('./utils')
+const {
+  collectRecoveryEntries,
+  createPackConfig,
+  extractTarGz,
+  removeSidecars,
+  writeTarGz
+} = require('./pack')
 
 async function restoreFiles(directory, options = {}) {
-  const config = {
+  const config = createPackConfig(directory, {
     ...defaultConfig,
-    ...options,
-    directory: path.resolve(directory || process.cwd())
-  }
-  const stats = createStats()
-  const limit = createLimiter(config.concurrency)
-  const tasks = []
-
-  await walk(config.directory, config, async filePath => {
-    if (!filePath.endsWith(config.suffix)) {
-      return
-    }
-
-    tasks.push(
-      limit(async () => {
-        await restoreOne(filePath, config, stats)
-      }).catch(error => {
-        stats.failed += 1
-        console.error(`Failed: ${filePath}`)
-        console.error(error && error.message ? error.message : error)
-      })
-    )
+    ...options
+  })
+  const archivePath = path.join(config.directory, `.endef-restore-${Date.now()}.tar.gz`)
+  const entries = await collectRecoveryEntries(config, {
+    onlySidecars: true,
+    outputPath: archivePath
   })
 
-  await Promise.all(tasks)
-  console.log(`Done. restored=${stats.restored}, removed=${stats.removed}, renamed=${stats.renamed}, failed=${stats.failed}`)
-  return stats
-}
-
-async function restoreOne(filePath, config, stats) {
-  const targetPath = filePath.slice(0, -config.suffix.length)
-  const targetExists = await exists(targetPath)
-
-  await fs.rm(targetPath, { force: true })
-  await fs.rename(filePath, targetPath)
-
-  stats.restored += 1
-  if (targetExists) {
-    stats.removed += 1
+  if (entries.length === 0) {
+    console.log('Done. restored=0, deleted=0, failed=0')
+    return {
+      restored: 0,
+      deleted: 0,
+      failed: 0
+    }
   }
-  stats.renamed += 1
-  console.log(`Restored: ${filePath} -> ${targetPath}`)
-}
 
-async function exists(filePath) {
   try {
-    await fs.access(filePath)
-    return true
-  } catch (error) {
-    return false
-  }
-}
-
-function createStats() {
-  return {
-    restored: 0,
-    removed: 0,
-    renamed: 0,
-    failed: 0
+    await writeTarGz(config.directory, archivePath, entries)
+    const restored = await extractTarGz(archivePath, config.directory)
+    const deleted = await removeSidecars(config.directory, config)
+
+    console.log(`Done. restored=${restored}, deleted=${deleted}, failed=0`)
+    return {
+      restored,
+      deleted,
+      failed: 0
+    }
+  } finally {
+    await fs.rm(archivePath, { force: true })
   }
 }
 

package/src/en.js

@@ -27,14 +27,9 @@ async function processFiles(directory, options = {}) {
     tasks.push(
       limit(async () => {
         const outputPath = `${filePath}${config.suffix}`
-        const data = await fs.readFile(filePath, 'utf8')
+        const data = await fs.readFile(filePath)
 
-        if (config.markerFilter && !matchesAnyMarker(data, config.markers)) {
-          stats.skipped += 1
-          return
-        }
-
-        await fs.writeFile(outputPath, data, 'utf8')
+        await fs.writeFile(outputPath, data)
         console.log(`Written: ${outputPath}`)
         stats.written += 1
       }).catch(error => {
@@ -58,12 +53,4 @@ function createStats() {
   }
 }
 
-function matchesAnyMarker(data, markers) {
-  if (!Array.isArray(markers) || markers.length === 0) {
-    return false
-  }
-
-  return markers.some(marker => data.includes(marker))
-}
-
 module.exports = processFiles

package/src/pack.js

@@ -2,36 +2,93 @@ const fs = require('fs')
 const fsp = require('fs/promises')
 const path = require('path')
 const zlib = require('zlib')
+const { promisify } = require('util')
 const { once } = require('events')
 const { pipeline } = require('stream/promises')
 const { defaultConfig } = require('./config')
 const { walk } = require('./utils')
 
+const gunzip = promisify(zlib.gunzip)
+
 async function packFiles(directory, options = {}) {
-  const config = {
+  const config = createPackConfig(directory, options)
+  const outputPath = resolveOutputPath(config.directory, config.out)
+  const entries = await collectPackEntries(config, outputPath)
+
+  await fsp.rm(outputPath, { force: true })
+  await writeTarGz(config.directory, outputPath, entries)
+  console.log(`Packed: ${outputPath}`)
+  console.log(`Done. files=${entries.length}`)
+
+  return {
+    outputPath,
+    files: entries.length
+  }
+}
+
+function createPackConfig(directory, options) {
+  return {
     ...defaultConfig,
     ...options,
     directory: path.resolve(directory || process.cwd())
   }
-  const outputPath = resolveOutputPath(config.directory, config.out)
-  const files = []
+}
+
+async function collectPackEntries(config, outputPath) {
+  const entries = []
 
   await walk(config.directory, config, async filePath => {
-    if (path.resolve(filePath) === outputPath || filePath.endsWith(config.suffix)) {
+    if (shouldSkipPackFile(filePath, config, outputPath)) {
       return
     }
 
-    files.push(filePath)
+    entries.push(createEntry(config.directory, filePath, filePath))
   })
 
-  await fsp.rm(outputPath, { force: true })
-  await writeTarGz(config.directory, outputPath, files)
-  console.log(`Packed: ${outputPath}`)
-  console.log(`Done. files=${files.length}`)
+  return entries
+}
+
+async function collectRecoveryEntries(config, options = {}) {
+  const entries = []
+  const sidecarTargets = new Set()
+
+  await walk(config.directory, config, async filePath => {
+    if (filePath.endsWith(config.suffix)) {
+      sidecarTargets.add(path.resolve(filePath.slice(0, -config.suffix.length)))
+    }
+  })
+
+  await walk(config.directory, config, async filePath => {
+    const resolvedPath = path.resolve(filePath)
+
+    if (options.outputPath && resolvedPath === path.resolve(options.outputPath)) {
+      return
+    }
+
+    if (filePath.endsWith(config.suffix)) {
+      const targetPath = filePath.slice(0, -config.suffix.length)
+      entries.push(createEntry(config.directory, filePath, targetPath))
+      return
+    }
+
+    if (options.onlySidecars || sidecarTargets.has(resolvedPath)) {
+      return
+    }
+
+    entries.push(createEntry(config.directory, filePath, filePath))
+  })
+
+  return entries
+}
+
+function shouldSkipPackFile(filePath, config, outputPath) {
+  return path.resolve(filePath) === outputPath || filePath.endsWith(config.suffix)
+}
 
+function createEntry(rootDirectory, sourcePath, archiveTargetPath) {
   return {
-    outputPath,
-    files: files.length
+    sourcePath,
+    archivePath: toArchivePath(path.relative(rootDirectory, archiveTargetPath))
   }
 }
 
@@ -51,30 +108,77 @@ function normalizeOutputName(out) {
   return `${out}.tar.gz`
 }
 
-async function writeTarGz(rootDirectory, outputPath, files) {
+async function writeTarGz(rootDirectory, outputPath, entries) {
   await fsp.mkdir(path.dirname(outputPath), { recursive: true })
 
   const gzip = zlib.createGzip()
   const output = fs.createWriteStream(outputPath)
-  const writer = writeTarEntries(rootDirectory, files, gzip)
+  const writer = writeTarEntries(entries, gzip)
 
   await Promise.all([writer, pipeline(gzip, output)])
 }
 
-async function writeTarEntries(rootDirectory, files, stream) {
-  for (const filePath of files) {
-    const stat = await fsp.stat(filePath)
-    const archivePath = toArchivePath(path.relative(rootDirectory, filePath))
-
-    await writeToStream(stream, createHeader(archivePath, stat))
+async function writeTarEntries(entries, stream) {
+  for (const entry of entries) {
+    const stat = await fsp.stat(entry.sourcePath)
 
-    await writeFileToStream(filePath, stream)
+    await writeToStream(stream, createHeader(entry.archivePath, stat))
+    await writeFileToStream(entry.sourcePath, stream)
     await writeToStream(stream, Buffer.alloc(paddingSize(stat.size)))
   }
 
   stream.end(Buffer.alloc(1024))
 }
 
+async function extractTarGz(archivePath, directory) {
+  const archive = await fsp.readFile(archivePath)
+  const data = await gunzip(archive)
+  let offset = 0
+  let extracted = 0
+
+  while (offset + 512 <= data.length) {
+    const header = data.subarray(offset, offset + 512)
+
+    if (header.every(value => value === 0)) {
+      break
+    }
+
+    const entry = readHeader(header)
+    offset += 512
+
+    const content = data.subarray(offset, offset + entry.size)
+    const outputPath = path.resolve(directory, entry.name)
+
+    if (!outputPath.startsWith(path.resolve(directory) + path.sep)) {
+      throw new Error(`Unsafe archive path: ${entry.name}`)
+    }
+
+    await fsp.mkdir(path.dirname(outputPath), { recursive: true })
+    await fsp.rm(outputPath, { force: true })
+    await fsp.writeFile(outputPath, content, { flag: 'wx' })
+
+    offset += entry.size + paddingSize(entry.size)
+    extracted += 1
+  }
+
+  return extracted
+}
+
+async function removeSidecars(directory, config) {
+  let deleted = 0
+
+  await walk(directory, config, async filePath => {
+    if (!filePath.endsWith(config.suffix)) {
+      return
+    }
+
+    await fsp.rm(filePath, { force: true })
+    deleted += 1
+  })
+
+  return deleted
+}
+
 async function writeFileToStream(filePath, stream) {
   for await (const chunk of fs.createReadStream(filePath)) {
     await writeToStream(stream, chunk)
@@ -113,6 +217,20 @@ function createHeader(filePath, stat) {
   return header
 }
 
+function readHeader(header) {
+  const name = readString(header, 0, 100)
+  const prefix = readString(header, 345, 155)
+
+  return {
+    name: prefix ? `${prefix}/${name}` : name,
+    size: parseInt(readString(header, 124, 12).trim() || '0', 8)
+  }
+}
+
+function readString(buffer, offset, length) {
+  return buffer.subarray(offset, offset + length).toString('utf8').replace(/\0.*$/, '')
+}
+
 function splitTarPath(filePath) {
   if (Buffer.byteLength(filePath) <= 100) {
     return {
@@ -156,3 +274,9 @@ function paddingSize(size) {
 }
 
 module.exports = packFiles
+module.exports.createPackConfig = createPackConfig
+module.exports.collectRecoveryEntries = collectRecoveryEntries
+module.exports.extractTarGz = extractTarGz
+module.exports.removeSidecars = removeSidecars
+module.exports.resolveOutputPath = resolveOutputPath
+module.exports.writeTarGz = writeTarGz

package/src/run.js

@@ -1,7 +1,15 @@
 const enF = require('./en')
 const deF = require('./de')
 const packF = require('./pack')
-const { loadConfig, splitMarkers } = require('./config')
+const { loadConfig } = require('./config')
+const {
+  collectRecoveryEntries,
+  createPackConfig,
+  extractTarGz,
+  removeSidecars,
+  resolveOutputPath,
+  writeTarGz
+} = require('./pack')
 
 function printHelp() {
   console.log(`Usage:
@@ -16,7 +24,6 @@ Options:
   --ext <list>          Comma separated target extensions
   --exclude <list>      Comma separated excluded directory names
   --concurrency <n>     Concurrent file operation limit
-  --marker [text]       Only create sidecar files when any marker is found
   --out <file>          Archive output name
 `)
 }
@@ -24,8 +31,6 @@ Options:
 function parseArgs(command, args) {
   const result = {
     directory: process.cwd(),
-    markerFilter: undefined,
-    markers: [],
     positionals: []
   }
 
@@ -98,23 +103,6 @@ function parseArgs(command, args) {
       continue
     }
 
-    if (arg.startsWith('--marker=')) {
-      result.markerFilter = true
-      result.markers.push(...splitMarkers(arg.slice('--marker='.length)))
-      continue
-    }
-
-    if (arg === '--marker') {
-      result.markerFilter = true
-
-      if (args[index + 1] && !args[index + 1].startsWith('-')) {
-        result.markers.push(...splitMarkers(args[index + 1]))
-        index += 1
-      }
-
-      continue
-    }
-
     if (!arg.startsWith('-')) {
       result.positionals.push(arg)
     }
@@ -160,8 +148,7 @@ async function run(args) {
   const options = {
     ...config,
     ...cliOptions,
-    directory: cliOptions.directory || config.directory || process.cwd(),
-    markers: cliOptions.markers && cliOptions.markers.length > 0 ? cliOptions.markers : config.markers
+    directory: cliOptions.directory || config.directory || process.cwd()
   }
 
   if (command === 'en') {
@@ -181,14 +168,28 @@ async function run(args) {
 
   if (command === 'all') {
     await enF(options.directory, options)
-    await deF(options.directory, options)
-    await packF(options.directory, options)
+    await runAll(options.directory, options)
     return
   }
 
   printHelp()
 }
 
+async function runAll(directory, options) {
+  const config = createPackConfig(directory, options)
+  const outputPath = resolveOutputPath(config.directory, config.out)
+  const entries = await collectRecoveryEntries(config, {
+    outputPath
+  })
+
+  await writeTarGz(config.directory, outputPath, entries)
+  const restored = await extractTarGz(outputPath, config.directory)
+  const deleted = await removeSidecars(config.directory, config)
+
+  console.log(`Packed: ${outputPath}`)
+  console.log(`Done. files=${entries.length}, restored=${restored}, deleted=${deleted}`)
+}
+
 function omitUndefined(value) {
   return Object.keys(value).reduce((result, key) => {
     if (value[key] !== undefined) {