claude-token-saver 3.28.0 → 3.28.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.en.md +25 -2
- package/README.md +25 -2
- package/package.json +1 -1
- package/presets/doc2md/convert.py +61 -0
- package/src/doc2md.cjs +250 -36
- package/src/fig2md.cjs +11 -2
package/README.en.md
CHANGED
|
@@ -397,12 +397,18 @@ Three situations, three different interception points:
|
|
|
397
397
|
That second row is measured, not assumed: a `.pdf` Read fires the hook, and a `.pptx` Read in the same session leaves no hook log entry at all.
|
|
398
398
|
|
|
399
399
|
```bash
|
|
400
|
-
claude-token-saver doc2md
|
|
401
|
-
claude-token-saver doc2md on # register the Read hook
|
|
400
|
+
claude-token-saver doc2md on # register the hooks (the converter installs itself)
|
|
402
401
|
claude-token-saver doc2md # check converter + hook registration
|
|
403
402
|
claude-token-saver doc2md report.pptx # convert by hand and see the result
|
|
403
|
+
claude-token-saver doc2md install-converter # only to get the install out of the way early
|
|
404
404
|
```
|
|
405
405
|
|
|
406
|
+
**The converter installs itself.** Any rollout step a person has to be told about is a step some of them skip, so the converter installs in the background the moment a document first shows up, and converts as soon as it is ready. Measured: about 30s for the first document (15s install plus markitdown's first import), then 3.7s for a new document and 0.1s on a cache hit. The `.fig` parser installs in half a second on the first Figma file.
|
|
407
|
+
|
|
408
|
+
It installs on first use rather than at `install` time: the venv is 47MB, and someone who never opens a document should not pay for it. Set `CTS_DOC2MD_NO_AUTOINSTALL=1` to turn the automatic install off.
|
|
409
|
+
|
|
410
|
+
**Python 3.10+ is required** — markitdown's own floor, and macOS still ships 3.9 as `/usr/bin/python3`. The venv is built on an interpreter chosen by version rather than by PATH order. Built on 3.9, pip resolves markitdown to a 2019 placeholder release (0.0.1a1): the install looks like it worked and every conversion then dies at import. This was found by walking into it. When nothing on the machine is new enough, the message points at `brew install python` instead of at an install command that cannot succeed.
|
|
411
|
+
|
|
406
412
|
The converter goes into a venv this tool owns (`<state dir>/doc2md-venv`): no system interpreter is touched, and uninstalling the CLI takes it along. An existing markitdown on `uv tool` or `PATH` is preferred over building a new one.
|
|
407
413
|
|
|
408
414
|
Conversion is [markitdown](https://github.com/microsoft/markitdown). Slide numbers, heading levels, tables, speaker notes and per-sheet headings all survive, and non-Latin text comes through intact.
|
|
@@ -455,6 +461,23 @@ Conversion is one-way — editing the cached `.md` changes nothing in the source
|
|
|
455
461
|
|
|
456
462
|
All four formats were exercised end to end on 2026-09-06: 10 docx run replacements plus three consecutive re-saves, a pptx bar-to-line chart swap with an added data point, xlsx value edits and a new row, and a fig text edit with re-encode and re-parse. In every case the original was byte-identical afterwards and the re-converted copy showed the change. One caveat: removing a chart shape from a pptx leaves the old chart XML part orphaned — PowerPoint ignores it, but delete the part and its rels for a clean file. Charts and images never appear in a conversion, so visual edits must be confirmed in the application itself.
|
|
457
463
|
|
|
464
|
+
### Locked documents, and Windows
|
|
465
|
+
|
|
466
|
+
**A password-protected document is a state, not an error.** Office encrypts by wrapping the package in an OLE compound file rather than a zip, so opening one as a zip used to report "not a zip file" — which reads as a broken download and sends the user after the wrong problem. It is now identified before conversion:
|
|
467
|
+
|
|
468
|
+
```
|
|
469
|
+
✗ encrypted: password-protected Office file (OLE-wrapped)
|
|
470
|
+
✗ encrypted: password-protected PDF
|
|
471
|
+
```
|
|
472
|
+
|
|
473
|
+
The model is told to ask for an unlocked copy. This tool never asks for or stores a password, and never blocks the original `Read`, so work continues either way. A PDF that merely restricts printing still opens and still converts — checked against a false positive — and a legacy `.xls`, which is an OLE file by design, is not mistaken for an encrypted one.
|
|
474
|
+
|
|
475
|
+
**Windows is supported.** For teams with Windows machines:
|
|
476
|
+
|
|
477
|
+
- The Python search uses the `py -3` launcher. `python3` is rarely on PATH there, and a bare `python` may be the Store alias stub that opens a web page instead of running anything. Venv interpreters are looked for at `Scripts\python.exe`.
|
|
478
|
+
- The `.fig` parser installs through `npm.cmd` via the shell, and the package spec dropped its caret (`openfig-core@0.4.x`): in cmd.exe `^` is the escape character and never reaches npm.
|
|
479
|
+
- The background install and every child process set `windowsHide`, so no console window appears in the middle of someone's prompt.
|
|
480
|
+
|
|
458
481
|
`claude-token-saver doc2md --clean` empties the conversion cache; `doc2md off` removes the hook. Removal filters for this tool's own entry, so anything else you registered under `PreToolUse` stays.
|
|
459
482
|
|
|
460
483
|
## 🌐 Behind a gateway (Bedrock / Vertex)
|
package/README.md
CHANGED
|
@@ -389,12 +389,18 @@ doc2md 는 그 파일을 한 번 변환해 두고 원본 대신 변환본을 읽
|
|
|
389
389
|
두 번째 줄의 제약은 실측으로 확인한 것입니다. `.pdf` 를 Read 하면 훅이 실행되고, 같은 세션에서 `.pptx` 를 Read 하면 훅 로그에 아무 기록도 남지 않습니다.
|
|
390
390
|
|
|
391
391
|
```bash
|
|
392
|
-
claude-token-saver doc2md
|
|
393
|
-
claude-token-saver doc2md on # Read 훅 등록
|
|
392
|
+
claude-token-saver doc2md on # 훅 등록 (변환기는 첫 문서에서 자동 설치)
|
|
394
393
|
claude-token-saver doc2md # 변환기·훅 등록 상태 확인
|
|
395
394
|
claude-token-saver doc2md 보고서.pptx # 직접 변환해 결과 확인
|
|
395
|
+
claude-token-saver doc2md install-converter # 설치를 미리 끝내 두고 싶을 때만
|
|
396
396
|
```
|
|
397
397
|
|
|
398
|
+
**변환기는 알아서 깔립니다.** 팀에 배포할 때 각자 설치 명령을 실행하게 만들면 그 단계에서 빠지는 사람이 생깁니다. 그래서 문서가 처음 등장하는 시점에 변환기가 백그라운드로 설치되고, 설치가 끝나는 대로 곧바로 변환합니다. 실측으로 첫 문서는 약 30초(설치 15초 + markitdown 최초 임포트), 이후로는 새 문서 3.7초, 캐시 적중 0.1초입니다. `.fig` 파서는 첫 Figma 파일에서 0.5초 만에 깔립니다.
|
|
399
|
+
|
|
400
|
+
설치는 `install` 단계가 아니라 첫 사용 시점에 합니다. venv 가 47MB 라서, 문서를 다루지 않는 사람은 낼 이유가 없는 비용입니다. 자동 설치를 끄려면 `CTS_DOC2MD_NO_AUTOINSTALL=1` 을 설정하십시오.
|
|
401
|
+
|
|
402
|
+
**파이썬 3.10 이상이 필요합니다.** markitdown 의 요구 사항이고, macOS 기본 `/usr/bin/python3` 는 3.9 입니다. 이 도구는 PATH 순서를 따르지 않고 3.10 이상인 인터프리터를 골라 venv 를 만듭니다. 3.9 로 만들면 pip 가 markitdown 을 2019 년 자리표시자 릴리스(0.0.1a1)로 해석해서, 설치는 성공한 것처럼 보이지만 모든 변환이 임포트 단계에서 죽습니다. 실제로 이 함정을 밟고 잡았습니다. 3.10 이상이 아예 없으면 설치 명령을 안내하는 대신 `brew install python` 을 안내합니다.
|
|
403
|
+
|
|
398
404
|
변환기는 도구 전용 venv(`<상태 디렉터리>/doc2md-venv`)에 설치합니다. 시스템 파이썬을 건드리지 않고, CLI를 지우면 함께 사라집니다. 이미 `uv tool` 이나 다른 경로에 markitdown 이 있으면 그쪽을 먼저 씁니다.
|
|
399
405
|
|
|
400
406
|
변환은 [markitdown](https://github.com/microsoft/markitdown)이 담당하며, 슬라이드 번호와 제목 계층, 표, 발표자 노트, 시트 구분이 모두 남습니다. 한글도 깨지지 않습니다.
|
|
@@ -451,6 +457,23 @@ PDF 는 첨부하면 모델이 내용을 그대로 읽습니다. 대신 페이
|
|
|
451
457
|
|
|
452
458
|
네 형식 모두 실제로 몇 바퀴 돌려 검증했습니다(2026-09-06): docx 텍스트 치환 10건과 3회 연속 재저장, pptx 막대→꺾은선 차트 교체와 데이터 행 추가, xlsx 값 정정·행 추가, fig 텍스트 수정·재인코드·재파싱. 전 케이스에서 원본은 바이트 그대로였고, 수정본 재변환에 변경 내용이 반영됐습니다. 한 가지 주의: pptx 에서 차트 도형을 제거하면 옛 차트 XML 파트가 고아로 남습니다. PowerPoint 는 무시하지만, 깔끔히 하려면 파트와 rels 도 지우십시오. 차트·이미지 같은 시각 요소는 변환본에 잡히지 않으므로, 시각 편집의 최종 확인은 해당 프로그램에서 해야 합니다.
|
|
453
459
|
|
|
460
|
+
### 암호가 걸린 문서와 Windows
|
|
461
|
+
|
|
462
|
+
**암호 문서는 오류가 아니라 상태로 다룹니다.** 사내에서 받는 문서 중 일부는 암호가 걸려 있습니다. 이전에는 암호 걸린 docx 를 "손상된 zip"이라고 알려서 사용자가 원인을 엉뚱한 곳에서 찾게 만들었습니다. 암호가 걸린 Office 문서는 zip 이 아니라 OLE 복합 문서로 저장되기 때문입니다. 지금은 변환 전에 판별해서 이렇게 알립니다.
|
|
463
|
+
|
|
464
|
+
```
|
|
465
|
+
✗ encrypted: password-protected Office file (OLE-wrapped)
|
|
466
|
+
✗ encrypted: password-protected PDF
|
|
467
|
+
```
|
|
468
|
+
|
|
469
|
+
모델에게는 암호를 푼 사본을 사용자에게 요청하라고 안내합니다. 이 도구는 암호를 묻지도 저장하지도 않습니다. 어느 경우에도 원본 `Read` 를 막지 않으므로 작업이 중단되지 않습니다. 열람은 자유롭고 인쇄만 제한된 PDF 는 암호 문서가 아니므로 그대로 변환합니다(오탐 확인 완료). 구형 `.xls` 도 원래 OLE 형식이라 암호로 오인하지 않습니다.
|
|
470
|
+
|
|
471
|
+
**Windows 를 지원합니다.** 사내에 Windows 사용자가 있어 다음을 맞췄습니다.
|
|
472
|
+
|
|
473
|
+
- 파이썬 탐색이 `py -3` 런처를 씁니다. Windows 에서는 `python3` 가 PATH 에 없는 경우가 많고, 맨 `python` 은 실행 대신 마이크로소프트 스토어를 여는 별칭 스텁일 수 있습니다. venv 기반 인터프리터도 `Scripts\python.exe` 경로로 찾습니다.
|
|
474
|
+
- `.fig` 파서 설치가 `npm.cmd` 를 셸로 호출합니다. 그리고 패키지 지정자에서 캐럿을 뺐습니다(`openfig-core@0.4.x`). cmd.exe 에서 `^` 는 이스케이프 문자라 npm 에 닿기 전에 먹힙니다.
|
|
475
|
+
- 백그라운드 자동 설치와 모든 하위 프로세스에 `windowsHide` 를 걸어, 프롬프트 도중에 콘솔 창이 튀어나오지 않게 했습니다.
|
|
476
|
+
|
|
454
477
|
`claude-token-saver doc2md --clean` 으로 변환 캐시를 비우고, `doc2md off` 로 훅을 제거합니다. 훅 해제는 자기 항목만 골라 지우므로 `PreToolUse` 에 등록해 둔 다른 훅은 그대로 남습니다.
|
|
455
478
|
|
|
456
479
|
## 🌐 Bedrock·Vertex 경유 환경
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "claude-token-saver",
|
|
3
|
-
"version": "3.28.
|
|
3
|
+
"version": "3.28.2",
|
|
4
4
|
"description": "Route the easy work your expensive Claude model keeps repeating down to haiku/sonnet — post-hoc session analysis, no realtime router, no extra LLM calls.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|
|
@@ -43,6 +43,55 @@ def fail(reason, detail=""):
|
|
|
43
43
|
sys.exit(0)
|
|
44
44
|
|
|
45
45
|
|
|
46
|
+
# A password-protected OOXML file is not a zip at all: Office wraps the whole
|
|
47
|
+
# package in an OLE compound file whose streams hold the ciphertext. Opening it
|
|
48
|
+
# as a zip therefore reports "not a zip file", which reads as a broken download
|
|
49
|
+
# and sends the user looking for the wrong problem.
|
|
50
|
+
OLE_MAGIC = b"\xd0\xcf\x11\xe0\xa1\xb1\x1a\xe1"
|
|
51
|
+
|
|
52
|
+
|
|
53
|
+
def encryption_problem(path, ext):
|
|
54
|
+
"""('encrypted', detail) when the file is password-protected, else None."""
|
|
55
|
+
try:
|
|
56
|
+
with open(path, "rb") as fh:
|
|
57
|
+
head = fh.read(8)
|
|
58
|
+
except OSError:
|
|
59
|
+
return None
|
|
60
|
+
|
|
61
|
+
# Legacy .xls is an OLE file by design, so the magic alone proves nothing
|
|
62
|
+
# there. For the modern formats it can only mean encryption.
|
|
63
|
+
if ext in ZIP_EXTS and head == OLE_MAGIC:
|
|
64
|
+
return ("encrypted", "password-protected Office file (OLE-wrapped)")
|
|
65
|
+
|
|
66
|
+
# Some producers keep the zip container and put the ciphertext inside it.
|
|
67
|
+
if ext in ZIP_EXTS:
|
|
68
|
+
try:
|
|
69
|
+
import zipfile
|
|
70
|
+
with zipfile.ZipFile(path) as z:
|
|
71
|
+
names = z.namelist()
|
|
72
|
+
if any(n.startswith("EncryptedPackage") for n in names):
|
|
73
|
+
return ("encrypted", "password-protected Office file")
|
|
74
|
+
except Exception:
|
|
75
|
+
return None
|
|
76
|
+
|
|
77
|
+
if ext == ".pdf":
|
|
78
|
+
try:
|
|
79
|
+
from pdfminer.pdfparser import PDFParser
|
|
80
|
+
from pdfminer.pdfdocument import PDFDocument
|
|
81
|
+
with open(path, "rb") as fh:
|
|
82
|
+
doc = PDFDocument(PDFParser(fh))
|
|
83
|
+
# An empty owner password is the ordinary "printing restricted"
|
|
84
|
+
# case, which extracts fine. Only a document that refuses to open
|
|
85
|
+
# counts as encrypted here.
|
|
86
|
+
if doc.encryption is not None and not doc.is_extractable:
|
|
87
|
+
return ("encrypted", "password-protected PDF")
|
|
88
|
+
except Exception as e:
|
|
89
|
+
if "password" in str(type(e).__name__).lower() or "password" in str(e).lower():
|
|
90
|
+
return ("encrypted", "password-protected PDF")
|
|
91
|
+
return None
|
|
92
|
+
return None
|
|
93
|
+
|
|
94
|
+
|
|
46
95
|
def check_zip(path):
|
|
47
96
|
"""Classify an archive before opening it as a document.
|
|
48
97
|
|
|
@@ -189,6 +238,13 @@ def main():
|
|
|
189
238
|
fail("missing", path)
|
|
190
239
|
|
|
191
240
|
ext = os.path.splitext(path)[1].lower()
|
|
241
|
+
|
|
242
|
+
# Checked before anything else opens the file: an encrypted document is a
|
|
243
|
+
# normal thing to receive, not a failure to report as corruption.
|
|
244
|
+
locked = encryption_problem(path, ext)
|
|
245
|
+
if locked:
|
|
246
|
+
fail(locked[0], locked[1])
|
|
247
|
+
|
|
192
248
|
if ext in ZIP_EXTS:
|
|
193
249
|
problem = check_zip(path)
|
|
194
250
|
if problem:
|
|
@@ -224,6 +280,11 @@ def main():
|
|
|
224
280
|
result = MarkItDown().convert(path)
|
|
225
281
|
text = (result.text_content or "").strip()
|
|
226
282
|
except Exception as e:
|
|
283
|
+
# markitdown surfaces the password failure from whichever backend hit
|
|
284
|
+
# it, so the type name is the reliable part.
|
|
285
|
+
blob = (type(e).__name__ + " " + str(e)).lower()
|
|
286
|
+
if "password" in blob or "encrypted" in blob:
|
|
287
|
+
fail("encrypted", "the file is password-protected")
|
|
227
288
|
fail("convert-failed", e)
|
|
228
289
|
|
|
229
290
|
if not text:
|
package/src/doc2md.cjs
CHANGED
|
@@ -26,7 +26,7 @@ const fs = require('node:fs');
|
|
|
26
26
|
const os = require('node:os');
|
|
27
27
|
const path = require('node:path');
|
|
28
28
|
const crypto = require('node:crypto');
|
|
29
|
-
const { spawnSync } = require('node:child_process');
|
|
29
|
+
const { spawn, spawnSync } = require('node:child_process');
|
|
30
30
|
const ledger = require('./doc2md-ledger.cjs');
|
|
31
31
|
|
|
32
32
|
// Formats where the original is of no use to the model. Images are absent
|
|
@@ -226,28 +226,62 @@ function writeCache(filePath, markdown, extra) {
|
|
|
226
226
|
* memoized for the life of this process, and the caller memoizes across
|
|
227
227
|
* processes through the notice file.
|
|
228
228
|
*/
|
|
229
|
+
// The readiness test for an interpreter. Importing the *class* matters: a pip
|
|
230
|
+
// install creates the package directory long before it finishes writing into
|
|
231
|
+
// it, so a bare `import markitdown` reports success mid-install and the
|
|
232
|
+
// conversion then fails with "cannot import name 'MarkItDown'". Measured
|
|
233
|
+
// during the first-use auto-install, where the window is about a second wide.
|
|
234
|
+
const PROBE = 'from markitdown import MarkItDown';
|
|
235
|
+
|
|
236
|
+
const IS_WINDOWS = process.platform === 'win32';
|
|
237
|
+
|
|
238
|
+
/**
|
|
239
|
+
* Interpreters to try, as `{ bin, args }` so the Windows launcher can carry
|
|
240
|
+
* its version flag. `py -3` is how a Windows box usually reaches Python at
|
|
241
|
+
* all: `python3` is rarely on PATH there, and a bare `python` may be the App
|
|
242
|
+
* Execution Alias stub that opens the Microsoft Store instead of running
|
|
243
|
+
* anything.
|
|
244
|
+
*/
|
|
245
|
+
function interpreterCandidates() {
|
|
246
|
+
const out = [];
|
|
247
|
+
if (process.env.CTS_DOC2MD_PYTHON) out.push({ bin: process.env.CTS_DOC2MD_PYTHON, args: [] });
|
|
248
|
+
// The tool's own venv, created by `doc2md install-converter`. First because
|
|
249
|
+
// it is the only one this tool controls: telling people to `pip install`
|
|
250
|
+
// into the system interpreter is how a token-saving CLI ends up owning a
|
|
251
|
+
// break in someone else's project.
|
|
252
|
+
out.push({ bin: managedPython(), args: [] });
|
|
253
|
+
if (IS_WINDOWS) {
|
|
254
|
+
out.push(
|
|
255
|
+
{ bin: path.join(os.homedir(), '.local', 'share', 'uv', 'tools', 'markitdown', 'Scripts', 'python.exe'), args: [] },
|
|
256
|
+
{ bin: 'py', args: ['-3'] },
|
|
257
|
+
{ bin: 'python', args: [] },
|
|
258
|
+
);
|
|
259
|
+
} else {
|
|
260
|
+
out.push(
|
|
261
|
+
{ bin: path.join(os.homedir(), '.local', 'share', 'uv', 'tools', 'markitdown', 'bin', 'python'), args: [] },
|
|
262
|
+
{ bin: path.join(os.homedir(), '.local', 'bin', 'markitdown-python'), args: [] },
|
|
263
|
+
{ bin: 'python3', args: [] },
|
|
264
|
+
{ bin: 'python', args: [] },
|
|
265
|
+
);
|
|
266
|
+
}
|
|
267
|
+
return out;
|
|
268
|
+
}
|
|
269
|
+
|
|
229
270
|
let interpreterCache;
|
|
230
271
|
function findInterpreter() {
|
|
231
272
|
if (interpreterCache !== undefined) return interpreterCache;
|
|
232
|
-
const
|
|
233
|
-
if (process.env.CTS_DOC2MD_PYTHON) candidates.push(process.env.CTS_DOC2MD_PYTHON);
|
|
234
|
-
candidates.push(
|
|
235
|
-
// The tool's own venv, created by `doc2md install-converter`. First
|
|
236
|
-
// because it is the only one this tool controls: telling people to
|
|
237
|
-
// `pip install` into the system interpreter is how a token-saving CLI
|
|
238
|
-
// ends up owning a break in someone else's project.
|
|
239
|
-
managedPython(),
|
|
240
|
-
path.join(os.homedir(), '.local', 'share', 'uv', 'tools', 'markitdown', 'bin', 'python'),
|
|
241
|
-
path.join(os.homedir(), '.local', 'bin', 'markitdown-python'),
|
|
242
|
-
'python3',
|
|
243
|
-
'python',
|
|
244
|
-
);
|
|
245
|
-
for (const bin of candidates) {
|
|
273
|
+
for (const { bin, args } of interpreterCandidates()) {
|
|
246
274
|
try {
|
|
247
|
-
const probe = spawnSync(bin, ['-c',
|
|
275
|
+
const probe = spawnSync(bin, [...args, '-c', PROBE], {
|
|
276
|
+
timeout: 20_000,
|
|
277
|
+
stdio: 'ignore',
|
|
278
|
+
windowsHide: true,
|
|
279
|
+
});
|
|
248
280
|
if (probe.status === 0) {
|
|
249
|
-
|
|
250
|
-
|
|
281
|
+
// Only a bare interpreter is cached as a string; the launcher form
|
|
282
|
+
// keeps its flag, since dropping it would run the wrong Python.
|
|
283
|
+
interpreterCache = args.length ? { bin, args } : bin;
|
|
284
|
+
return interpreterCache;
|
|
251
285
|
}
|
|
252
286
|
} catch { /* candidate unusable, try the next */ }
|
|
253
287
|
}
|
|
@@ -255,6 +289,11 @@ function findInterpreter() {
|
|
|
255
289
|
return null;
|
|
256
290
|
}
|
|
257
291
|
|
|
292
|
+
/** Split a `findInterpreter()` result into the pair spawnSync wants. */
|
|
293
|
+
function interpreterParts(found) {
|
|
294
|
+
return typeof found === 'string' ? { bin: found, args: [] } : found;
|
|
295
|
+
}
|
|
296
|
+
|
|
258
297
|
const CONVERTER = path.join(__dirname, '..', 'presets', 'doc2md', 'convert.py');
|
|
259
298
|
|
|
260
299
|
/** Path to the interpreter inside the venv this tool manages. */
|
|
@@ -281,19 +320,72 @@ const EDIT_LIBS = ['python-pptx', 'python-docx', 'openpyxl'];
|
|
|
281
320
|
* for, it goes somewhere this tool owns, so uninstalling the CLI takes the
|
|
282
321
|
* whole thing with it and no system interpreter is touched.
|
|
283
322
|
*/
|
|
323
|
+
/**
|
|
324
|
+
* A Python new enough to run markitdown (3.10+), or null.
|
|
325
|
+
*
|
|
326
|
+
* Explicit version names are tried before the bare `python3` so a modern
|
|
327
|
+
* Homebrew interpreter wins over the system one regardless of PATH order.
|
|
328
|
+
*/
|
|
329
|
+
function findVenvBase() {
|
|
330
|
+
const candidates = IS_WINDOWS
|
|
331
|
+
? [
|
|
332
|
+
// The launcher first, asked for each version in turn: it knows about
|
|
333
|
+
// installs that never touched PATH, which is the normal case on
|
|
334
|
+
// Windows.
|
|
335
|
+
{ bin: 'py', args: ['-3.14'] }, { bin: 'py', args: ['-3.13'] },
|
|
336
|
+
{ bin: 'py', args: ['-3.12'] }, { bin: 'py', args: ['-3.11'] },
|
|
337
|
+
{ bin: 'py', args: ['-3.10'] }, { bin: 'py', args: ['-3'] },
|
|
338
|
+
{ bin: 'python', args: [] },
|
|
339
|
+
]
|
|
340
|
+
: [
|
|
341
|
+
{ bin: 'python3.14', args: [] }, { bin: 'python3.13', args: [] },
|
|
342
|
+
{ bin: 'python3.12', args: [] }, { bin: 'python3.11', args: [] },
|
|
343
|
+
{ bin: 'python3.10', args: [] },
|
|
344
|
+
{ bin: 'python3', args: [] }, { bin: 'python', args: [] },
|
|
345
|
+
];
|
|
346
|
+
for (const { bin, args } of candidates) {
|
|
347
|
+
const r = spawnSync(bin, [...args, '-c', 'import sys; print("%d.%d" % sys.version_info[:2])'], {
|
|
348
|
+
encoding: 'utf8',
|
|
349
|
+
timeout: 20_000,
|
|
350
|
+
windowsHide: true,
|
|
351
|
+
});
|
|
352
|
+
if (r.status !== 0) continue;
|
|
353
|
+
const [major, minor] = String(r.stdout).trim().split('.').map(Number);
|
|
354
|
+
if (major > 3 || (major === 3 && minor >= 10)) {
|
|
355
|
+
return { bin, args, version: `python ${major}.${minor}` };
|
|
356
|
+
}
|
|
357
|
+
}
|
|
358
|
+
return null;
|
|
359
|
+
}
|
|
360
|
+
|
|
284
361
|
function installConverter({ onProgress = () => {} } = {}) {
|
|
285
362
|
const venv = path.join(userDataDir(), 'doc2md-venv');
|
|
286
363
|
const target = managedPython();
|
|
287
364
|
|
|
288
365
|
if (!fs.existsSync(target)) {
|
|
289
|
-
|
|
290
|
-
|
|
291
|
-
|
|
292
|
-
|
|
293
|
-
|
|
366
|
+
// The base interpreter is chosen by version, not by whichever `python3`
|
|
367
|
+
// comes first on PATH. markitdown needs 3.10+, and macOS still ships 3.9
|
|
368
|
+
// as /usr/bin/python3: building the venv on that one installs a
|
|
369
|
+
// seven-year-old placeholder release (0.0.1a1) that has no MarkItDown
|
|
370
|
+
// class in it, and every conversion then fails at import time. Measured
|
|
371
|
+
// on this machine, where /usr/bin/python3 precedes Homebrew's 3.14.
|
|
372
|
+
const base = findVenvBase();
|
|
373
|
+
if (!base) {
|
|
374
|
+
return {
|
|
375
|
+
ok: false,
|
|
376
|
+
reason: 'no-python',
|
|
377
|
+
detail: 'markitdown needs Python 3.10 or newer; none was found on PATH '
|
|
378
|
+
+ '(macOS /usr/bin/python3 is 3.9 — install a newer one, e.g. `brew install python`)',
|
|
379
|
+
};
|
|
294
380
|
}
|
|
295
|
-
|
|
296
|
-
|
|
381
|
+
onProgress(`creating ${venv} (${base.version})`);
|
|
382
|
+
const r = spawnSync(base.bin, [...(base.args || []), '-m', 'venv', venv], {
|
|
383
|
+
encoding: 'utf8',
|
|
384
|
+
timeout: 180_000,
|
|
385
|
+
windowsHide: true,
|
|
386
|
+
});
|
|
387
|
+
if (r.status !== 0) {
|
|
388
|
+
return { ok: false, reason: 'no-python', detail: (r.stderr || 'venv creation failed').slice(0, 300) };
|
|
297
389
|
}
|
|
298
390
|
}
|
|
299
391
|
|
|
@@ -301,6 +393,7 @@ function installConverter({ onProgress = () => {} } = {}) {
|
|
|
301
393
|
const install = spawnSync(target, ['-m', 'pip', 'install', '--quiet', MARKITDOWN_SPEC, ...EDIT_LIBS], {
|
|
302
394
|
encoding: 'utf8',
|
|
303
395
|
timeout: 900_000,
|
|
396
|
+
windowsHide: true,
|
|
304
397
|
});
|
|
305
398
|
if (install.status !== 0) {
|
|
306
399
|
return { ok: false, reason: 'pip-failed', detail: (install.stderr || '').slice(0, 400) };
|
|
@@ -308,7 +401,7 @@ function installConverter({ onProgress = () => {} } = {}) {
|
|
|
308
401
|
|
|
309
402
|
// The probe is the actual acceptance test: pip can exit 0 and still leave an
|
|
310
403
|
// interpreter that cannot import what was asked for.
|
|
311
|
-
const probe = spawnSync(target, ['-c',
|
|
404
|
+
const probe = spawnSync(target, ['-c', PROBE], { timeout: 60_000, stdio: 'ignore', windowsHide: true });
|
|
312
405
|
if (probe.status !== 0) {
|
|
313
406
|
return { ok: false, reason: 'import-failed', detail: 'installed, but markitdown does not import' };
|
|
314
407
|
}
|
|
@@ -320,8 +413,8 @@ function installConverter({ onProgress = () => {} } = {}) {
|
|
|
320
413
|
/**
|
|
321
414
|
* Convert one file. Returns `{ ok: true, cacheFile, meta }`, or
|
|
322
415
|
* `{ ok: false, reason, detail }` where reason is one of:
|
|
323
|
-
* no-markitdown | too-
|
|
324
|
-
* convert-failed | timeout
|
|
416
|
+
* no-markitdown | python-too-old | no-figparser | encrypted | too-large |
|
|
417
|
+
* sensitive | unsafe-archive | no-text | convert-failed | timeout
|
|
325
418
|
*
|
|
326
419
|
* Every failure is a reason to leave the original Read alone, never to break
|
|
327
420
|
* it. That is the whole contract with the hook.
|
|
@@ -370,7 +463,14 @@ function convert(filePath, { converter = CONVERTER, python: pythonOverride = nul
|
|
|
370
463
|
// Figma files take the Node converter; everything else goes to markitdown.
|
|
371
464
|
if (path.extname(filePath).toLowerCase() === '.fig') {
|
|
372
465
|
const fig2md = require('./fig2md.cjs');
|
|
373
|
-
|
|
466
|
+
let result = spawnFigConvert(fig2md, filePath);
|
|
467
|
+
// The .fig parser is an npm install of a few hundred KB, fast enough to
|
|
468
|
+
// wait for inline the first time a Figma file turns up.
|
|
469
|
+
if (!result.ok && result.reason === 'no-figparser'
|
|
470
|
+
&& process.env.CTS_DOC2MD_NO_AUTOINSTALL !== '1') {
|
|
471
|
+
fig2md.installFigParser(userDataDir());
|
|
472
|
+
result = spawnFigConvert(fig2md, filePath);
|
|
473
|
+
}
|
|
374
474
|
if (!result.ok) return result;
|
|
375
475
|
const written = writeCache(filePath, result.markdown, {
|
|
376
476
|
note: result.note, truncated: false, rows: 0, pages: 0, markupBytes: 0,
|
|
@@ -381,13 +481,29 @@ function convert(filePath, { converter = CONVERTER, python: pythonOverride = nul
|
|
|
381
481
|
// The override exists so tests can drive a stub converter with any Python at
|
|
382
482
|
// all: the normal search insists the interpreter can import markitdown,
|
|
383
483
|
// which would make the whole path untestable without the real package.
|
|
384
|
-
|
|
484
|
+
let python = pythonOverride || findInterpreter();
|
|
485
|
+
if (!python && !pythonOverride) {
|
|
486
|
+
// Telling someone to run the install command is wrong when the install
|
|
487
|
+
// cannot succeed on this machine. A too-old interpreter is a different
|
|
488
|
+
// problem with a different fix, so it gets its own reason rather than
|
|
489
|
+
// hiding behind "no converter".
|
|
490
|
+
if (!findVenvBase()) {
|
|
491
|
+
return {
|
|
492
|
+
ok: false,
|
|
493
|
+
reason: 'python-too-old',
|
|
494
|
+
detail: 'markitdown needs Python 3.10 or newer; none was found on PATH',
|
|
495
|
+
};
|
|
496
|
+
}
|
|
497
|
+
if (ensureConverterInstalled()) python = findInterpreter();
|
|
498
|
+
}
|
|
385
499
|
if (!python) return { ok: false, reason: 'no-markitdown' };
|
|
386
500
|
|
|
387
|
-
const
|
|
501
|
+
const py = interpreterParts(python);
|
|
502
|
+
const run = spawnSync(py.bin, [...py.args, converter, filePath], {
|
|
388
503
|
encoding: 'utf8',
|
|
389
504
|
timeout: CONVERT_TIMEOUT_MS,
|
|
390
505
|
maxBuffer: MAX_MARKDOWN_BYTES * 4,
|
|
506
|
+
windowsHide: true,
|
|
391
507
|
});
|
|
392
508
|
if (run.error && run.error.code === 'ETIMEDOUT') return { ok: false, reason: 'timeout' };
|
|
393
509
|
if (run.status !== 0) {
|
|
@@ -421,6 +537,75 @@ function convert(filePath, { converter = CONVERTER, python: pythonOverride = nul
|
|
|
421
537
|
return { ok: true, cached: false, cacheFile: written.cacheFile, meta: written.meta };
|
|
422
538
|
}
|
|
423
539
|
|
|
540
|
+
/**
|
|
541
|
+
* First-use install, so nobody has to be told to run a setup command.
|
|
542
|
+
*
|
|
543
|
+
* A team rollout dies on any step a person has to be told about, so the
|
|
544
|
+
* converter installs itself the first time a document actually shows up. It
|
|
545
|
+
* is still lazy rather than part of `install`: the venv is 47MB, and someone
|
|
546
|
+
* who never opens a document should never pay for it.
|
|
547
|
+
*
|
|
548
|
+
* The install runs detached and the caller waits only briefly. A cold install
|
|
549
|
+
* measured 6 seconds on a fast connection, but a corporate network can be far
|
|
550
|
+
* slower, and a hook that blocks a prompt for a minute is worse than a
|
|
551
|
+
* document that converts on the next turn. So: start it, wait up to
|
|
552
|
+
* `waitMs`, and if it is still going, say so and let this turn proceed
|
|
553
|
+
* without the conversion.
|
|
554
|
+
*
|
|
555
|
+
* Returns true when a converter is ready to use right now.
|
|
556
|
+
*/
|
|
557
|
+
function ensureConverterInstalled({ waitMs = 15_000 } = {}) {
|
|
558
|
+
if (process.env.CTS_DOC2MD_NO_AUTOINSTALL === '1') return false;
|
|
559
|
+
if (findInterpreter()) return true;
|
|
560
|
+
|
|
561
|
+
const lock = path.join(userDataDir(), 'doc2md-install.lock');
|
|
562
|
+
let running = false;
|
|
563
|
+
try {
|
|
564
|
+
const started = JSON.parse(fs.readFileSync(lock, 'utf8')).startedAt;
|
|
565
|
+
// A lock older than the pip timeout is a crashed run, not a live one.
|
|
566
|
+
running = Number.isFinite(started) && Date.now() - started < 900_000;
|
|
567
|
+
} catch { /* no lock, or an unreadable one: treat as not running */ }
|
|
568
|
+
|
|
569
|
+
if (!running) {
|
|
570
|
+
try {
|
|
571
|
+
fs.mkdirSync(userDataDir(), { recursive: true });
|
|
572
|
+
fs.writeFileSync(lock, JSON.stringify({ startedAt: Date.now() }));
|
|
573
|
+
const cli = path.join(__dirname, '..', 'bin', 'cli.js');
|
|
574
|
+
// Detached, so a session that ends mid-install does not take the
|
|
575
|
+
// install with it — the next session finds it finished.
|
|
576
|
+
const child = spawn(process.execPath, [cli, 'doc2md', 'install-converter'], {
|
|
577
|
+
detached: true,
|
|
578
|
+
stdio: 'ignore',
|
|
579
|
+
// Without this Windows pops a console window for the install, in the
|
|
580
|
+
// middle of someone's prompt.
|
|
581
|
+
windowsHide: true,
|
|
582
|
+
});
|
|
583
|
+
child.unref();
|
|
584
|
+
} catch {
|
|
585
|
+
try { fs.rmSync(lock, { force: true }); } catch { /* best effort */ }
|
|
586
|
+
return false;
|
|
587
|
+
}
|
|
588
|
+
}
|
|
589
|
+
|
|
590
|
+
// Poll cheaply: the interpreter file appearing is the first sign, and the
|
|
591
|
+
// import probe is the acceptance test. interpreterCache has to be cleared
|
|
592
|
+
// or the memoized null from the top of this function would stick.
|
|
593
|
+
const deadline = Date.now() + waitMs;
|
|
594
|
+
while (Date.now() < deadline) {
|
|
595
|
+
if (fs.existsSync(managedPython())) {
|
|
596
|
+
interpreterCache = undefined;
|
|
597
|
+
if (findInterpreter()) {
|
|
598
|
+
try { fs.rmSync(lock, { force: true }); } catch { /* best effort */ }
|
|
599
|
+
return true;
|
|
600
|
+
}
|
|
601
|
+
}
|
|
602
|
+
// A synchronous sleep, because every caller here is synchronous. 400ms
|
|
603
|
+
// keeps the poll count low over a 15s wait.
|
|
604
|
+
Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, 400);
|
|
605
|
+
}
|
|
606
|
+
return false;
|
|
607
|
+
}
|
|
608
|
+
|
|
424
609
|
/** Where the "markitdown is not installed" notice records that it was shown. */
|
|
425
610
|
function noticePath() {
|
|
426
611
|
return path.join(userDataDir(), 'doc2md-notice.json');
|
|
@@ -490,9 +675,26 @@ function decideForRead(context, opts = {}) {
|
|
|
490
675
|
markNoticeShown();
|
|
491
676
|
return {
|
|
492
677
|
deny: false,
|
|
493
|
-
reason: `[doc2md] ${name} 를
|
|
494
|
-
+
|
|
495
|
-
+
|
|
678
|
+
reason: `[doc2md] ${name} 를 변환할 변환기를 지금 설치하고 있습니다(첫 실행에만 걸립니다).\n`
|
|
679
|
+
+ ' 설치가 끝나면 다음 요청부터 자동으로 변환합니다. 이번 turn 은 원본을 그대로 읽습니다.\n'
|
|
680
|
+
+ ` 진행 상황: ${INSTALL_HINT} 를 직접 실행하면 설치 로그를 볼 수 있습니다.`,
|
|
681
|
+
};
|
|
682
|
+
}
|
|
683
|
+
if (result.reason === 'encrypted') {
|
|
684
|
+
return {
|
|
685
|
+
deny: false,
|
|
686
|
+
reason: `[doc2md] ${name} 는 암호가 걸린 문서라 변환하지 못했습니다. `
|
|
687
|
+
+ '사용자에게 암호를 푼 사본을 요청하십시오. 이 도구는 암호를 묻거나 저장하지 않습니다.',
|
|
688
|
+
};
|
|
689
|
+
}
|
|
690
|
+
if (result.reason === 'python-too-old') {
|
|
691
|
+
if (noticeAlreadyShown()) return null;
|
|
692
|
+
markNoticeShown();
|
|
693
|
+
return {
|
|
694
|
+
deny: false,
|
|
695
|
+
reason: `[doc2md] ${name} 를 변환하지 못했습니다. 변환기(markitdown)는 Python 3.10 이상이 필요한데 PATH 에서 찾지 못했습니다.\n`
|
|
696
|
+
+ ' macOS 기본 /usr/bin/python3 는 3.9 입니다. `brew install python` 으로 새 버전을 설치한 뒤 다시 시도하십시오.\n'
|
|
697
|
+
+ ' 그때까지는 원본을 그대로 읽습니다. 이 안내는 한 번만 표시됩니다.',
|
|
496
698
|
};
|
|
497
699
|
}
|
|
498
700
|
if (result.reason === 'no-text') {
|
|
@@ -613,8 +815,16 @@ function contextForPrompt(payload, opts = {}) {
|
|
|
613
815
|
}
|
|
614
816
|
} else if (result.reason === 'no-markitdown') {
|
|
615
817
|
lines.push(lang === 'ko'
|
|
616
|
-
? ` ${name}:
|
|
617
|
-
: ` ${name}:
|
|
818
|
+
? ` ${name}: 변환기를 설치하는 중입니다(첫 실행에만 걸립니다). 설치가 끝나면 다음 요청부터 자동 변환됩니다.`
|
|
819
|
+
: ` ${name}: the converter is installing now (first run only). It will convert automatically from the next request.`);
|
|
820
|
+
} else if (result.reason === 'encrypted') {
|
|
821
|
+
lines.push(lang === 'ko'
|
|
822
|
+
? ` ${name}: 암호가 걸린 문서입니다. 암호를 푼 사본을 달라고 사용자에게 요청하십시오.`
|
|
823
|
+
: ` ${name}: the document is password-protected. Ask the user for an unlocked copy.`);
|
|
824
|
+
} else if (result.reason === 'python-too-old') {
|
|
825
|
+
lines.push(lang === 'ko'
|
|
826
|
+
? ` ${name}: 변환기가 Python 3.10 이상을 요구하는데 PATH 에 없습니다(macOS 기본은 3.9). \`brew install python\` 후 다시 시도하도록 사용자에게 안내하십시오.`
|
|
827
|
+
: ` ${name}: the converter needs Python 3.10+, and none is on PATH (macOS ships 3.9). Tell the user to install a newer Python, e.g. \`brew install python\`.`);
|
|
618
828
|
} else if (result.reason === 'no-figparser') {
|
|
619
829
|
lines.push(lang === 'ko'
|
|
620
830
|
? ` ${name}: .fig 파서가 없어 변환하지 못했습니다. 설치: claude-token-saver doc2md install-converter`
|
|
@@ -744,6 +954,10 @@ module.exports = {
|
|
|
744
954
|
MARKITDOWN_SPEC,
|
|
745
955
|
managedPython,
|
|
746
956
|
installConverter,
|
|
957
|
+
ensureConverterInstalled,
|
|
958
|
+
findVenvBase,
|
|
959
|
+
interpreterCandidates,
|
|
960
|
+
interpreterParts,
|
|
747
961
|
clearNotice,
|
|
748
962
|
cacheDir,
|
|
749
963
|
cachePathFor,
|
package/src/fig2md.cjs
CHANGED
|
@@ -28,7 +28,10 @@ const fs = require('node:fs');
|
|
|
28
28
|
const path = require('node:path');
|
|
29
29
|
const { spawnSync } = require('node:child_process');
|
|
30
30
|
|
|
31
|
-
|
|
31
|
+
// `0.4.x` rather than `^0.4.1` on purpose: the Windows install goes through
|
|
32
|
+
// cmd.exe, where `^` is the escape character and would be eaten before npm
|
|
33
|
+
// ever saw it. The range is the same one caret means for a 0.x package.
|
|
34
|
+
const FIG_PARSER_SPEC = 'openfig-core@0.4.x';
|
|
32
35
|
|
|
33
36
|
/** Where the on-demand parser install lives, under the tool's state dir. */
|
|
34
37
|
function managedFigDir(userDataDir) {
|
|
@@ -56,10 +59,16 @@ function installFigParser(userDataDir, { onProgress = () => {} } = {}) {
|
|
|
56
59
|
fs.writeFileSync(pkgJson, JSON.stringify({ name: 'doc2md-fig', private: true }) + '\n');
|
|
57
60
|
}
|
|
58
61
|
onProgress(`installing ${FIG_PARSER_SPEC}`);
|
|
59
|
-
|
|
62
|
+
// npm on Windows is npm.cmd, which spawnSync cannot execute directly — it
|
|
63
|
+
// needs the shell. Elsewhere the shell is avoided, since the package spec
|
|
64
|
+
// would then go through shell quoting for no benefit.
|
|
65
|
+
const isWindows = process.platform === 'win32';
|
|
66
|
+
const r = spawnSync(isWindows ? 'npm.cmd' : 'npm', ['install', '--no-audit', '--no-fund', '--silent', FIG_PARSER_SPEC], {
|
|
60
67
|
cwd: dir,
|
|
61
68
|
encoding: 'utf8',
|
|
62
69
|
timeout: 300_000,
|
|
70
|
+
windowsHide: true,
|
|
71
|
+
shell: isWindows,
|
|
63
72
|
});
|
|
64
73
|
if (r.status !== 0) {
|
|
65
74
|
return { ok: false, reason: 'npm-failed', detail: (r.stderr || '').slice(0, 400) };
|