sift-light 1.1.0 → 1.1.1-2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +11 -0
- package/README.md +2 -0
- package/README.zh-CN.md +2 -0
- package/package.json +1 -1
- package/plugins/sift-light/.claude-plugin/plugin.json +1 -1
- package/plugins/sift-light/.codex-plugin/plugin.json +1 -1
- package/plugins/sift-light/.mcp.json +1 -1
- package/plugins/sift-light/kimi.plugin.json +2 -2
- package/plugins/sift-light/omp-extension.mjs +126 -106
- package/plugins/sift-light/package.json +1 -1
- package/src/evidence-service.ts +1 -1
- package/src/evidence-validation.ts +16 -77
- package/src/inspect.ts +43 -14
- package/src/mcp-server.mjs +122 -102
- package/src/rg.ts +11 -1
- package/src/scan-revisions.ts +78 -1
- package/src/service.ts +1 -1
- package/src/snapshot-store.ts +1 -0
- package/src/source-document.ts +2 -5
- package/src/source-inspection.ts +1 -0
- package/src/types.ts +2 -0
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,16 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 1.1.1-2 — 2026-10-13
|
|
4
|
+
|
|
5
|
+
- Clarify the shared 32 MiB source-read budget for `inspect`, including why smaller output windows and source continuations do not admit oversized files.
|
|
6
|
+
- Keep search behavior, public interfaces and default configuration unchanged; this is a documentation and maintainer-verification release.
|
|
7
|
+
|
|
8
|
+
## 1.1.1-1 — 2026-10-12
|
|
9
|
+
|
|
10
|
+
- Bind retained ordinary search results to verified worktree content identities instead of relying on file metadata alone.
|
|
11
|
+
- Recheck the stored content identity before source inspection, navigation and saved-evidence validation; changed, missing or incomplete sources remain explicit.
|
|
12
|
+
- Preserve exact locations, independent result lanes, source verification and existing result budgets while adding regression coverage for content changes that metadata checks can miss.
|
|
13
|
+
|
|
3
14
|
## 1.1.0 — 2026-10-12
|
|
4
15
|
|
|
5
16
|
- Add opt-in BYO HTTP JSON providers for Jev-compatible semantic judging and multimodal retrieval.
|
package/README.md
CHANGED
|
@@ -53,6 +53,8 @@ Long results arrive in pages with a way to continue. When the original material
|
|
|
53
53
|
A complete snapshot describes match retention, not complete source text. Truncated matching-line excerpts show their limit and an executable `inspectRequest`, which remains usable after the final match page. Follow pagination cursors instead of repeating the query with a different limit.
|
|
54
54
|
In Pi and OMP, the passive session status includes the loaded package version, counts returned new queries, distinguishes complete, partial and unfinished results, and reports non-cancelled failed calls. Cursor and operation continuations do not inflate the new-query count.
|
|
55
55
|
|
|
56
|
+
`inspect` loads and verifies the source document before selecting a bounded output window. Its request shares a 32 MiB source-read budget; a single file larger than that budget is rejected even when only line 1 is requested. Changing the line, reducing displayed context or retrying does not bypass that limit. For a large log, search for the relevant text first, or use the host's bounded read capability on the known exact file. Source continuation pages apply to documents already admitted within the budget; they do not provide streaming inspection of oversized files.
|
|
57
|
+
|
|
56
58
|
Ordinary searches keep repository ignore rules, but now report policy-filtered filesystem coverage instead of calling an absence exhaustive when ignored files exist. Filename discovery (`mode: "files"`) does the same: it says how many files were ignored and which of them match the query, and `ignorePolicy: "include"` lists them. Each file candidate shows its score and whether it is an exact, substring or fuzzy match. Use `mode: "audit"` with named literal `patterns` for one bounded receipt covering declared scope, enumerated/searched/skipped files, ignored-file policy, per-pattern `present`/`absent_with_complete_coverage`/`unknown` findings, and start/end source stability. Set `ignorePolicy: "include"` when an audit must include ignored configuration and generated files; `.git` internals and protected paths remain excluded.
|
|
57
59
|
|
|
58
60
|
### Discover language capabilities before loading a provider
|
package/README.zh-CN.md
CHANGED
|
@@ -53,6 +53,8 @@ Concept 或 hybrid 较慢时,会在默认五秒等待窗口内返回 `status:
|
|
|
53
53
|
完整快照表示匹配保留完整,不代表源码正文没有截断。长行摘录会明确标记限制,并给出最后一页之后仍可执行的 `inspectRequest`。需要更多结果时沿游标继续,不要仅为翻页而修改 limit 重搜。
|
|
54
54
|
Pi 和 OMP 的被动 session 状态会显示当前加载的包版本,统计已返回的新查询,区分完整、部分和未完成结果,并报告未取消的失败调用。cursor 与 operation 续接不会重复计入新查询。
|
|
55
55
|
|
|
56
|
+
`inspect` 会先加载并校验源文档,再选择有界的输出窗口;同一请求共享 32 MiB 源内容读取预算。单个文件超过预算时,即使只请求第 1 行也会被拒绝,换行号、减少显示上下文或重试都不能绕过该限制。对于大日志,可以先检索相关文本,或使用宿主的有界读取能力分段读取已知精确文件。源码续读页只适用于已经在预算内接纳的文档,不提供超大文件的流式 inspect。
|
|
57
|
+
|
|
56
58
|
普通搜索继续遵循仓库 ignore 规则,但只要存在被忽略文件,就会明确说明文件系统覆盖受策略过滤,不再把“接纳文件里没找到”说成“整个目录绝对不存在”。文件名发现(`mode: "files"`)同样会说明有多少文件被忽略、其中哪些匹配查询,`ignorePolicy: "include"` 可以把它们列出来;每个候选都显示分数以及是精确、子串还是模糊匹配。需要发布前收口时,可以用 `mode: "audit"` 配合带名字的字面量 `patterns`,一次拿到声明范围、枚举/搜索/跳过文件、ignore 策略、每个模式的 `present`、`absent_with_complete_coverage` 或 `unknown` 结论,以及搜索前后的来源稳定性。审计必须包含被忽略的配置或生成文件时,设置 `ignorePolicy: "include"`;`.git` 内部和受保护路径仍然不会开放。
|
|
57
59
|
|
|
58
60
|
### 先发现语言能力,再按需加载提供方
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "sift-light",
|
|
3
|
-
"version": "1.1.
|
|
3
|
+
"version": "1.1.1-2",
|
|
4
4
|
"description": "Require sift-light for conventional local searches while keeping development tools available.",
|
|
5
5
|
"author": {
|
|
6
6
|
"name": "baoer"
|
|
@@ -13,7 +13,7 @@
|
|
|
13
13
|
"args": [
|
|
14
14
|
"--yes",
|
|
15
15
|
"--package",
|
|
16
|
-
"sift-light-runtime@npm:sift-light@1.1.
|
|
16
|
+
"sift-light-runtime@npm:sift-light@1.1.1-2",
|
|
17
17
|
"sift-light-mcp",
|
|
18
18
|
"--stdio"
|
|
19
19
|
],
|
|
@@ -1406,7 +1406,10 @@ function sourceRangeFromBytes(content, startLine, endLine, targetLine = startLin
|
|
|
1406
1406
|
}
|
|
1407
1407
|
|
|
1408
1408
|
// src/scan-revisions.ts
|
|
1409
|
+
import { createHash } from "crypto";
|
|
1410
|
+
import { open } from "fs/promises";
|
|
1409
1411
|
import { resolve as resolve4 } from "path";
|
|
1412
|
+
var SOURCE_HASH_CONCURRENCY = 4;
|
|
1410
1413
|
async function captureRevision(path, revisions, onRevisionError) {
|
|
1411
1414
|
let failed = false;
|
|
1412
1415
|
let failure;
|
|
@@ -1502,6 +1505,67 @@ async function retainStableSourceRevisions(paths, before, signal) {
|
|
|
1502
1505
|
return initial !== undefined && sameSourceRevision(initial, revision);
|
|
1503
1506
|
}));
|
|
1504
1507
|
}
|
|
1508
|
+
async function captureStableSourceReference(path, expected, signal) {
|
|
1509
|
+
if (signal?.aborted)
|
|
1510
|
+
throw abortError();
|
|
1511
|
+
const before = await getSourceRevision(path);
|
|
1512
|
+
if (!before || !sameSourceRevision(expected, before))
|
|
1513
|
+
return;
|
|
1514
|
+
const handle2 = await open(path, "r");
|
|
1515
|
+
try {
|
|
1516
|
+
const metadata2 = await handle2.stat();
|
|
1517
|
+
if (!metadata2.isFile() || !sameSourceRevision(before, sourceRevisionFromStats(metadata2))) {
|
|
1518
|
+
return;
|
|
1519
|
+
}
|
|
1520
|
+
const hash = createHash("sha256");
|
|
1521
|
+
const buffer = Buffer.allocUnsafe(64 * 1024);
|
|
1522
|
+
let offset = 0;
|
|
1523
|
+
while (offset < before.size) {
|
|
1524
|
+
if (signal?.aborted)
|
|
1525
|
+
throw abortError();
|
|
1526
|
+
const read = await handle2.read(buffer, 0, Math.min(buffer.length, before.size - offset), offset);
|
|
1527
|
+
if (read.bytesRead === 0)
|
|
1528
|
+
return;
|
|
1529
|
+
hash.update(buffer.subarray(0, read.bytesRead));
|
|
1530
|
+
offset += read.bytesRead;
|
|
1531
|
+
}
|
|
1532
|
+
const after = await getSourceRevision(path);
|
|
1533
|
+
if (!after || !sameSourceRevision(before, after))
|
|
1534
|
+
return;
|
|
1535
|
+
return {
|
|
1536
|
+
path,
|
|
1537
|
+
origin: { kind: "worktree", revision: after, contentHash: hash.digest("hex") }
|
|
1538
|
+
};
|
|
1539
|
+
} finally {
|
|
1540
|
+
await handle2.close();
|
|
1541
|
+
}
|
|
1542
|
+
}
|
|
1543
|
+
async function captureStableSourceReferences(paths, before, signal) {
|
|
1544
|
+
const candidates = [...paths].filter((path) => before.has(path));
|
|
1545
|
+
const references = new Map;
|
|
1546
|
+
for (let offset = 0;offset < candidates.length; offset += SOURCE_HASH_CONCURRENCY) {
|
|
1547
|
+
if (signal?.aborted)
|
|
1548
|
+
throw abortError();
|
|
1549
|
+
const batch = candidates.slice(offset, offset + SOURCE_HASH_CONCURRENCY);
|
|
1550
|
+
const captured = await Promise.all(batch.map(async (path) => {
|
|
1551
|
+
const expected = before.get(path);
|
|
1552
|
+
if (!expected)
|
|
1553
|
+
return;
|
|
1554
|
+
try {
|
|
1555
|
+
return await captureStableSourceReference(path, expected, signal);
|
|
1556
|
+
} catch (error) {
|
|
1557
|
+
if (signal?.aborted)
|
|
1558
|
+
throw error;
|
|
1559
|
+
return;
|
|
1560
|
+
}
|
|
1561
|
+
}));
|
|
1562
|
+
for (const reference of captured) {
|
|
1563
|
+
if (reference)
|
|
1564
|
+
references.set(reference.path, reference);
|
|
1565
|
+
}
|
|
1566
|
+
}
|
|
1567
|
+
return references;
|
|
1568
|
+
}
|
|
1505
1569
|
|
|
1506
1570
|
// src/rg.ts
|
|
1507
1571
|
function isRecord(value) {
|
|
@@ -1903,6 +1967,7 @@ function createRipgrepRunner(options = {}) {
|
|
|
1903
1967
|
const retainedPaths = new Set(matches.map((match) => match.absolutePath).filter((path) => !lossyPaths.has(path)));
|
|
1904
1968
|
await assertRetainedPathsAllowed(policy, [...retainedPaths], signal);
|
|
1905
1969
|
const sourceRevisions = await retainStableSourceRevisions(retainedPaths, before.revisions, signal);
|
|
1970
|
+
const sourceReferences = await captureStableSourceReferences(retainedPaths, sourceRevisions, signal);
|
|
1906
1971
|
if (signal?.aborted)
|
|
1907
1972
|
throw abortError();
|
|
1908
1973
|
return {
|
|
@@ -1911,6 +1976,7 @@ function createRipgrepRunner(options = {}) {
|
|
|
1911
1976
|
totalMatches,
|
|
1912
1977
|
fileCounts,
|
|
1913
1978
|
sourceRevisions,
|
|
1979
|
+
sourceReferences,
|
|
1914
1980
|
snapshotComplete: matches.length === totalMatches && !modificationTimeFilterIncomplete && retention.details.reasons.length === 0,
|
|
1915
1981
|
filesystemCoverage,
|
|
1916
1982
|
filesystemErrorSamples: [...filesystemErrorSamples],
|
|
@@ -2117,7 +2183,7 @@ function createCtagsStructureProvider(options = {}) {
|
|
|
2117
2183
|
// package.json
|
|
2118
2184
|
var package_default = {
|
|
2119
2185
|
name: "sift-light",
|
|
2120
|
-
version: "1.1.
|
|
2186
|
+
version: "1.1.1-2",
|
|
2121
2187
|
description: "Context-efficient local search for files, documents, notes and logs across Pi, OMP and MCP clients",
|
|
2122
2188
|
keywords: [
|
|
2123
2189
|
"ai-agent",
|
|
@@ -2733,7 +2799,7 @@ class OperationLifecycle {
|
|
|
2733
2799
|
}
|
|
2734
2800
|
|
|
2735
2801
|
// src/media-search.ts
|
|
2736
|
-
import { createHash, randomUUID as randomUUID4 } from "crypto";
|
|
2802
|
+
import { createHash as createHash2, randomUUID as randomUUID4 } from "crypto";
|
|
2737
2803
|
import { mkdtemp, mkdir, readFile as readFile4, realpath as realpath3, rm, stat as stat5, writeFile } from "fs/promises";
|
|
2738
2804
|
import { tmpdir } from "os";
|
|
2739
2805
|
import { basename as basename2, dirname as dirname2, extname as extname2, join as join3, resolve as resolve8 } from "path";
|
|
@@ -5643,7 +5709,7 @@ function bucketFor(state, inst) {
|
|
|
5643
5709
|
return bucket;
|
|
5644
5710
|
}
|
|
5645
5711
|
var handoff;
|
|
5646
|
-
var
|
|
5712
|
+
var open2 = [];
|
|
5647
5713
|
var memo = {
|
|
5648
5714
|
alloc(_inst, payload, empty) {
|
|
5649
5715
|
const bucket = handoff;
|
|
@@ -5652,7 +5718,7 @@ var memo = {
|
|
|
5652
5718
|
handoff = undefined;
|
|
5653
5719
|
const entry = { value: empty, issues: null };
|
|
5654
5720
|
bucket.set(payload.value, entry);
|
|
5655
|
-
|
|
5721
|
+
open2.push(entry);
|
|
5656
5722
|
return empty;
|
|
5657
5723
|
},
|
|
5658
5724
|
guard(inst) {
|
|
@@ -5718,10 +5784,10 @@ var memo = {
|
|
|
5718
5784
|
return payload;
|
|
5719
5785
|
}
|
|
5720
5786
|
handoff = bucket;
|
|
5721
|
-
const depth =
|
|
5787
|
+
const depth = open2.length;
|
|
5722
5788
|
const result = base(payload, ctx);
|
|
5723
5789
|
handoff = undefined;
|
|
5724
|
-
const entry =
|
|
5790
|
+
const entry = open2.length > depth ? open2.pop() : undefined;
|
|
5725
5791
|
if (result instanceof Promise) {
|
|
5726
5792
|
return result.then((r) => {
|
|
5727
5793
|
if (entry)
|
|
@@ -8775,7 +8841,7 @@ var extensions = new Set([
|
|
|
8775
8841
|
var maxFileBytes = 20 * 1024 * 1024;
|
|
8776
8842
|
var maxCorpusBytes = 256 * 1024 * 1024;
|
|
8777
8843
|
function hash(bytes) {
|
|
8778
|
-
return
|
|
8844
|
+
return createHash2("sha256").update(bytes).digest("hex");
|
|
8779
8845
|
}
|
|
8780
8846
|
async function sourceBytes(cwd, path, signal) {
|
|
8781
8847
|
const policy = new SearchPathPolicy(cwd);
|
|
@@ -8920,7 +8986,7 @@ class MediaSearch {
|
|
|
8920
8986
|
} else if (this.#remoteConfig) {
|
|
8921
8987
|
this.#remote ??= new RemoteMediaClient(this.#remoteConfig);
|
|
8922
8988
|
const remoteFiles = [...admitted.values()].map((entry) => ({
|
|
8923
|
-
id:
|
|
8989
|
+
id: createHash2("sha256").update(entry.staged).digest("hex").slice(0, 32),
|
|
8924
8990
|
path: entry.staged
|
|
8925
8991
|
}));
|
|
8926
8992
|
response = await this.#remote.search({
|
|
@@ -9086,7 +9152,7 @@ ${JSON.stringify(details)}`,
|
|
|
9086
9152
|
}
|
|
9087
9153
|
|
|
9088
9154
|
// src/data-source-http.ts
|
|
9089
|
-
import { open as
|
|
9155
|
+
import { open as open3 } from "fs/promises";
|
|
9090
9156
|
class DataSourceError extends Error {
|
|
9091
9157
|
code;
|
|
9092
9158
|
constructor(code) {
|
|
@@ -9101,7 +9167,7 @@ async function credential(config) {
|
|
|
9101
9167
|
value = process.env[config.tokenEnv];
|
|
9102
9168
|
if (config.tokenFile) {
|
|
9103
9169
|
try {
|
|
9104
|
-
const handle2 = await
|
|
9170
|
+
const handle2 = await open3(config.tokenFile, "r");
|
|
9105
9171
|
try {
|
|
9106
9172
|
const info2 = await handle2.stat();
|
|
9107
9173
|
if (!info2.isFile() || info2.size > 16384)
|
|
@@ -9396,7 +9462,7 @@ class DataSources {
|
|
|
9396
9462
|
}
|
|
9397
9463
|
|
|
9398
9464
|
// src/service.ts
|
|
9399
|
-
import { createHash as
|
|
9465
|
+
import { createHash as createHash8 } from "crypto";
|
|
9400
9466
|
|
|
9401
9467
|
// src/concept-worker-client.ts
|
|
9402
9468
|
import { randomUUID as randomUUID5 } from "crypto";
|
|
@@ -9940,7 +10006,7 @@ function rangeEvidence(document2, range) {
|
|
|
9940
10006
|
|
|
9941
10007
|
// src/concept-search.ts
|
|
9942
10008
|
import { resolve as resolve14 } from "path";
|
|
9943
|
-
import { createHash as
|
|
10009
|
+
import { createHash as createHash6 } from "crypto";
|
|
9944
10010
|
|
|
9945
10011
|
// src/request.ts
|
|
9946
10012
|
function list(value) {
|
|
@@ -10802,13 +10868,13 @@ async function parseSyntax(path, text, signal, pattern) {
|
|
|
10802
10868
|
}
|
|
10803
10869
|
|
|
10804
10870
|
// src/concept-source-generation.ts
|
|
10805
|
-
import { createHash as
|
|
10871
|
+
import { createHash as createHash5 } from "crypto";
|
|
10806
10872
|
|
|
10807
10873
|
// src/source-access.ts
|
|
10808
10874
|
import { extname as extname4, resolve as resolve13 } from "path";
|
|
10809
10875
|
|
|
10810
10876
|
// src/historical-paths.ts
|
|
10811
|
-
import { lstat, mkdir as mkdir3, mkdtemp as mkdtemp2, open as
|
|
10877
|
+
import { lstat, mkdir as mkdir3, mkdtemp as mkdtemp2, open as open4, rm as rm3, writeFile as writeFile2 } from "fs/promises";
|
|
10812
10878
|
import { constants as constants2 } from "fs";
|
|
10813
10879
|
import { tmpdir as tmpdir2 } from "os";
|
|
10814
10880
|
import { dirname as dirname5, join as join6, parse as parse4, relative as relative4, resolve as resolve10 } from "path";
|
|
@@ -10870,7 +10936,7 @@ async function filterHistoricalPaths(cwd, paths, request, signal) {
|
|
|
10870
10936
|
throw new SiftLightError("Current ignore rules are not regular files; historical path filtering is unavailable");
|
|
10871
10937
|
if (before.size > MAX_SOURCE_FILE_BYTES || ignoreBytesRead + before.size > MAX_STRUCTURE_BYTES)
|
|
10872
10938
|
throw new SiftLightError("Current ignore rules exceed the source read budget");
|
|
10873
|
-
const handle2 = await
|
|
10939
|
+
const handle2 = await open4(path, constants2.O_RDONLY | (constants2.O_NOFOLLOW ?? 0));
|
|
10874
10940
|
let bytes;
|
|
10875
10941
|
try {
|
|
10876
10942
|
if (!sameSourceRevision(sourceRevisionFromStats(before), sourceRevisionFromStats(await handle2.stat())))
|
|
@@ -11107,9 +11173,9 @@ async function sourceSimilarity(oldContent, newContent, budget) {
|
|
|
11107
11173
|
}
|
|
11108
11174
|
|
|
11109
11175
|
// src/git-repository.ts
|
|
11110
|
-
import { createHash as
|
|
11176
|
+
import { createHash as createHash3 } from "crypto";
|
|
11111
11177
|
import { constants as constants3 } from "fs";
|
|
11112
|
-
import { lstat as lstat2, open as
|
|
11178
|
+
import { lstat as lstat2, open as open5 } from "fs/promises";
|
|
11113
11179
|
import { isAbsolute as isAbsolute7, relative as relative5, resolve as resolve11, sep as sep3 } from "path";
|
|
11114
11180
|
|
|
11115
11181
|
// src/git-process.ts
|
|
@@ -11377,7 +11443,7 @@ async function readGitBlob(cwd, commit, entry, budget, signal) {
|
|
|
11377
11443
|
budget.bytes += output.length;
|
|
11378
11444
|
if (output.length !== size)
|
|
11379
11445
|
throw new SiftLightError("Git blob size does not match its immutable tree entry");
|
|
11380
|
-
const verifiedBlob =
|
|
11446
|
+
const verifiedBlob = createHash3(blob.length === 40 ? "sha1" : "sha256").update(`blob ${String(output.length)}\x00`).update(output).digest("hex");
|
|
11381
11447
|
if (verifiedBlob !== blob)
|
|
11382
11448
|
throw new SiftLightError("Git blob bytes do not match their immutable object identity");
|
|
11383
11449
|
return {
|
|
@@ -11386,7 +11452,7 @@ async function readGitBlob(cwd, commit, entry, budget, signal) {
|
|
|
11386
11452
|
sourceStatus: output.includes(0) ? "binary" : "available",
|
|
11387
11453
|
...output.includes(0) ? { reason: "Binary source contains NUL bytes" } : { content: output },
|
|
11388
11454
|
origin: { kind: "git", commit, blob },
|
|
11389
|
-
contentHash:
|
|
11455
|
+
contentHash: createHash3("sha256").update(output).digest("hex")
|
|
11390
11456
|
};
|
|
11391
11457
|
}
|
|
11392
11458
|
async function readWorktreeSource(cwd, path, budget, signal) {
|
|
@@ -11417,7 +11483,7 @@ async function readWorktreeSource(cwd, path, budget, signal) {
|
|
|
11417
11483
|
if (budget.bytes + before.size > budget.maxBytes)
|
|
11418
11484
|
return limitedSource(path, mode, `Source reads exceed the ${String(budget.maxBytes)} byte request limit`);
|
|
11419
11485
|
await assertExistingPathInsideCwd(absolute, cwd);
|
|
11420
|
-
const handle2 = await
|
|
11486
|
+
const handle2 = await open5(absolute, constants3.O_RDONLY | (constants3.O_NOFOLLOW ?? 0));
|
|
11421
11487
|
try {
|
|
11422
11488
|
if (!sameSourceRevision(sourceRevisionFromStats(before), sourceRevisionFromStats(await handle2.stat())))
|
|
11423
11489
|
throw new SiftLightError("Working source changed before reading");
|
|
@@ -11446,9 +11512,9 @@ async function readWorktreeSource(cwd, path, budget, signal) {
|
|
|
11446
11512
|
origin: {
|
|
11447
11513
|
kind: "worktree",
|
|
11448
11514
|
revision: sourceRevisionFromStats(after),
|
|
11449
|
-
contentHash:
|
|
11515
|
+
contentHash: createHash3("sha256").update(content).digest("hex")
|
|
11450
11516
|
},
|
|
11451
|
-
contentHash:
|
|
11517
|
+
contentHash: createHash3("sha256").update(content).digest("hex")
|
|
11452
11518
|
};
|
|
11453
11519
|
} finally {
|
|
11454
11520
|
await handle2.close();
|
|
@@ -11693,8 +11759,8 @@ async function readGitSource(cwd, identity, signal, options = {}) {
|
|
|
11693
11759
|
|
|
11694
11760
|
// src/source-document.ts
|
|
11695
11761
|
import { isUtf8 } from "buffer";
|
|
11696
|
-
import { createHash as
|
|
11697
|
-
import { open as
|
|
11762
|
+
import { createHash as createHash4 } from "crypto";
|
|
11763
|
+
import { open as open6, realpath as realpath4 } from "fs/promises";
|
|
11698
11764
|
import { relative as relative6, resolve as resolve12 } from "path";
|
|
11699
11765
|
class SourceDocumentError extends SiftLightError {
|
|
11700
11766
|
reason;
|
|
@@ -11705,7 +11771,7 @@ class SourceDocumentError extends SiftLightError {
|
|
|
11705
11771
|
}
|
|
11706
11772
|
}
|
|
11707
11773
|
function contentHash(bytes) {
|
|
11708
|
-
return
|
|
11774
|
+
return createHash4("sha256").update(bytes).digest("hex");
|
|
11709
11775
|
}
|
|
11710
11776
|
|
|
11711
11777
|
class SourceDocument {
|
|
@@ -11860,14 +11926,14 @@ async function readWorkspaceDocument(path, cwd, signal, expected, readBudget = M
|
|
|
11860
11926
|
if (!before)
|
|
11861
11927
|
throw new SourceDocumentError("source-unavailable", "Source is unavailable");
|
|
11862
11928
|
if (expected && !sameSourceRevision(before, expected.revision)) {
|
|
11863
|
-
throw new SourceDocumentError("source-changed", "Source changed;
|
|
11929
|
+
throw new SourceDocumentError("source-changed", "Source changed; refresh the search");
|
|
11864
11930
|
}
|
|
11865
11931
|
if (before.size > Math.min(MAX_SOURCE_FILE_BYTES, readBudget)) {
|
|
11866
11932
|
throw new SourceDocumentError("file-too-large", "Source exceeds the 5 MiB file limit");
|
|
11867
11933
|
}
|
|
11868
11934
|
if (signal?.aborted)
|
|
11869
11935
|
throw abortError();
|
|
11870
|
-
const handle2 = await
|
|
11936
|
+
const handle2 = await open6(canonical, "r");
|
|
11871
11937
|
let bytes;
|
|
11872
11938
|
try {
|
|
11873
11939
|
const metadata2 = await handle2.stat();
|
|
@@ -11902,7 +11968,7 @@ async function readWorkspaceDocument(path, cwd, signal, expected, readBudget = M
|
|
|
11902
11968
|
throw abortError();
|
|
11903
11969
|
const hash = contentHash(bytes);
|
|
11904
11970
|
if (expected && expected.contentHash !== hash) {
|
|
11905
|
-
throw new SourceDocumentError("source-changed", "Source content changed;
|
|
11971
|
+
throw new SourceDocumentError("source-changed", "Source content changed; refresh the search");
|
|
11906
11972
|
}
|
|
11907
11973
|
return new SourceDocument({
|
|
11908
11974
|
path: isPathInsideCwd(canonical, canonicalCwd) ? relative6(canonicalCwd, canonical).replaceAll("\\", "/") : canonical.replaceAll("\\", "/"),
|
|
@@ -12197,7 +12263,7 @@ async function createConceptSourceGeneration(access, filters, providedFiles) {
|
|
|
12197
12263
|
reasons.push(`${path}: ${error.message}`);
|
|
12198
12264
|
}
|
|
12199
12265
|
}
|
|
12200
|
-
const inventoryHash =
|
|
12266
|
+
const inventoryHash = createHash5("sha256").update(JSON.stringify(inventory)).digest("hex").slice(0, 32);
|
|
12201
12267
|
return {
|
|
12202
12268
|
cwd: access.cwd,
|
|
12203
12269
|
filters: {
|
|
@@ -12536,7 +12602,7 @@ async function runConceptSearch(input, access, infer, onProgress, options = {})
|
|
|
12536
12602
|
batchFileLimit: MAX_STRUCTURE_FILES,
|
|
12537
12603
|
fileLimit: access.maxFiles,
|
|
12538
12604
|
startedAt: generationStartedAt,
|
|
12539
|
-
inventoryHash:
|
|
12605
|
+
inventoryHash: createHash6("sha256").update(JSON.stringify(inventory)).digest("hex").slice(0, 32)
|
|
12540
12606
|
};
|
|
12541
12607
|
result.counts = {
|
|
12542
12608
|
...lexical ? { relevanceRanking: 1 } : {},
|
|
@@ -12716,7 +12782,7 @@ async function structuralSearch(input, access) {
|
|
|
12716
12782
|
}
|
|
12717
12783
|
|
|
12718
12784
|
// src/native-text-search.ts
|
|
12719
|
-
import { createHash as
|
|
12785
|
+
import { createHash as createHash7 } from "crypto";
|
|
12720
12786
|
import { mkdtemp as mkdtemp3, mkdir as mkdir4, readFile as readFile5, realpath as realpath5, rm as rm4, writeFile as writeFile3 } from "fs/promises";
|
|
12721
12787
|
import { tmpdir as tmpdir3 } from "os";
|
|
12722
12788
|
import { basename as basename3, dirname as dirname6, isAbsolute as isAbsolute8, join as join7, resolve as resolve15 } from "path";
|
|
@@ -12792,11 +12858,11 @@ function createNativeTextSearch(config, search) {
|
|
|
12792
12858
|
if (access.signal?.aborted)
|
|
12793
12859
|
throw abortError();
|
|
12794
12860
|
const path = document2.path;
|
|
12795
|
-
const relative = isAbsolute8(path) ? join7(
|
|
12861
|
+
const relative = isAbsolute8(path) ? join7(createHash7("sha256").update(dirname6(path)).digest("hex"), basename3(path)) : path;
|
|
12796
12862
|
const staged = resolve15(corpus, relative);
|
|
12797
12863
|
if (!staged.startsWith(corpus + "/"))
|
|
12798
12864
|
throw new SiftLightError("Native staging path escaped its corpus");
|
|
12799
|
-
const hash =
|
|
12865
|
+
const hash = createHash7("sha256").update(document2.bytes).digest("hex");
|
|
12800
12866
|
const previous = known.get(path);
|
|
12801
12867
|
if (!previous || previous.hash !== hash) {
|
|
12802
12868
|
await mkdir4(dirname6(staged), { recursive: true });
|
|
@@ -12842,7 +12908,7 @@ function createNativeTextSearch(config, search) {
|
|
|
12842
12908
|
throw new SiftLightError("Native text returned an unknown or duplicate source");
|
|
12843
12909
|
seen.add(entry.staged);
|
|
12844
12910
|
const content = await readFile5(entry.staged);
|
|
12845
|
-
if (
|
|
12911
|
+
if (createHash7("sha256").update(content).digest("hex") !== entry.hash)
|
|
12846
12912
|
throw new SiftLightError("Native staged source changed");
|
|
12847
12913
|
const document2 = new SourceDocument(entry.source, content);
|
|
12848
12914
|
const range = document2.lineRange(hit.line);
|
|
@@ -13656,6 +13722,7 @@ function resolveInspectionTarget(input, cwd, snapshots) {
|
|
|
13656
13722
|
const absolutePath = retainedMatch?.absolutePath ?? resolve16(cwd, path);
|
|
13657
13723
|
new SearchPathPolicy(cwd).assertPath(absolutePath);
|
|
13658
13724
|
let expectedRevision;
|
|
13725
|
+
let expectedReference;
|
|
13659
13726
|
if (input.cursor) {
|
|
13660
13727
|
const { snapshot } = snapshots.resolve(input.cursor);
|
|
13661
13728
|
retainedMatch ??= snapshot.matches.find((match) => match.absolutePath === absolutePath && match.lineNumber === line);
|
|
@@ -13663,13 +13730,15 @@ function resolveInspectionTarget(input, cwd, snapshots) {
|
|
|
13663
13730
|
throw new CursorError("The requested line is not a retained match in this snapshot.");
|
|
13664
13731
|
}
|
|
13665
13732
|
expectedRevision = snapshot.sourceRevisions.get(absolutePath);
|
|
13733
|
+
expectedReference = snapshot.sourceReferences?.get(absolutePath);
|
|
13666
13734
|
}
|
|
13667
13735
|
return {
|
|
13668
13736
|
path,
|
|
13669
13737
|
absolutePath,
|
|
13670
13738
|
line,
|
|
13671
|
-
unverified: input.cursor !== undefined && expectedRevision === undefined,
|
|
13739
|
+
unverified: input.cursor !== undefined && (expectedReference === undefined || expectedRevision === undefined),
|
|
13672
13740
|
...retainedMatch ? { retainedMatch } : {},
|
|
13741
|
+
...expectedReference ? { expectedReference } : {},
|
|
13673
13742
|
...expectedRevision ? { expectedRevision } : {}
|
|
13674
13743
|
};
|
|
13675
13744
|
}
|
|
@@ -15767,6 +15836,7 @@ function matchInspectionTarget(target) {
|
|
|
15767
15836
|
path: target.path,
|
|
15768
15837
|
line: target.line,
|
|
15769
15838
|
unverified: target.unverified,
|
|
15839
|
+
...target.expectedReference ? { reference: target.expectedReference } : {},
|
|
15770
15840
|
...target.expectedRevision ? { expectedRevision: target.expectedRevision } : {},
|
|
15771
15841
|
...target.retainedMatch?.occurrences[0] ? { focus: target.retainedMatch.occurrences[0].byteStart } : {}
|
|
15772
15842
|
};
|
|
@@ -16352,9 +16422,6 @@ async function failure(error, expected, path, cwd, signal) {
|
|
|
16352
16422
|
function sourceKey(reference) {
|
|
16353
16423
|
return JSON.stringify([reference.path, reference.origin]);
|
|
16354
16424
|
}
|
|
16355
|
-
function revisionKey(path, revision) {
|
|
16356
|
-
return JSON.stringify([path, revision]);
|
|
16357
|
-
}
|
|
16358
16425
|
function selectedIndex(index, count, label) {
|
|
16359
16426
|
if (index === undefined)
|
|
16360
16427
|
return;
|
|
@@ -16405,8 +16472,8 @@ function snapshotTargets(snapshot, matchIndex) {
|
|
|
16405
16472
|
const missing = [];
|
|
16406
16473
|
const seen = new Set;
|
|
16407
16474
|
for (const path of selectedPaths) {
|
|
16408
|
-
const
|
|
16409
|
-
if (!
|
|
16475
|
+
const reference = snapshot.sourceReferences?.get(path);
|
|
16476
|
+
if (!reference) {
|
|
16410
16477
|
const key = JSON.stringify(["missing", path]);
|
|
16411
16478
|
if (!seen.has(key)) {
|
|
16412
16479
|
seen.add(key);
|
|
@@ -16414,74 +16481,19 @@ function snapshotTargets(snapshot, matchIndex) {
|
|
|
16414
16481
|
path,
|
|
16415
16482
|
role: "source",
|
|
16416
16483
|
status: "unknown",
|
|
16417
|
-
reason: "Saved search has no source
|
|
16484
|
+
reason: "Saved search has no verified source content identity"
|
|
16418
16485
|
});
|
|
16419
16486
|
}
|
|
16420
16487
|
continue;
|
|
16421
16488
|
}
|
|
16422
16489
|
addUniqueTarget(targets, seen, {
|
|
16423
|
-
key:
|
|
16490
|
+
key: sourceKey(reference),
|
|
16424
16491
|
path,
|
|
16425
|
-
|
|
16492
|
+
expected: reference
|
|
16426
16493
|
});
|
|
16427
16494
|
}
|
|
16428
16495
|
return { targets, missing };
|
|
16429
16496
|
}
|
|
16430
|
-
async function validateSnapshotTarget(target, cwd, policy, signal) {
|
|
16431
|
-
if (!target.revision)
|
|
16432
|
-
throw new Error("Snapshot validation target omitted its revision");
|
|
16433
|
-
try {
|
|
16434
|
-
const absolute = resolve22(cwd, target.path);
|
|
16435
|
-
const canonical = await policy.resolveExistingPath(absolute);
|
|
16436
|
-
if (!canonical) {
|
|
16437
|
-
const result = await confirmWorktreeState(target.path, cwd, signal);
|
|
16438
|
-
return { path: target.path, role: "source", ...result };
|
|
16439
|
-
}
|
|
16440
|
-
const before = await stat7(absolute);
|
|
16441
|
-
const beforeCanonical = canonical;
|
|
16442
|
-
const after = await stat7(absolute);
|
|
16443
|
-
const afterCanonical = await realpath7(absolute);
|
|
16444
|
-
if (!before.isFile() || !after.isFile()) {
|
|
16445
|
-
return {
|
|
16446
|
-
path: target.path,
|
|
16447
|
-
role: "source",
|
|
16448
|
-
status: "stale",
|
|
16449
|
-
reason: "Source is no longer a regular file"
|
|
16450
|
-
};
|
|
16451
|
-
}
|
|
16452
|
-
if (beforeCanonical !== afterCanonical || !sameSourceRevision(sourceRevisionFromStats(before), sourceRevisionFromStats(after))) {
|
|
16453
|
-
return {
|
|
16454
|
-
path: target.path,
|
|
16455
|
-
role: "source",
|
|
16456
|
-
status: "stale",
|
|
16457
|
-
reason: "Source changed while checking its revision"
|
|
16458
|
-
};
|
|
16459
|
-
}
|
|
16460
|
-
const current = sourceRevisionFromStats(after);
|
|
16461
|
-
const unchanged = sameSourceRevision(target.revision, current);
|
|
16462
|
-
return {
|
|
16463
|
-
path: target.path,
|
|
16464
|
-
role: "source",
|
|
16465
|
-
status: unchanged ? "current" : "stale",
|
|
16466
|
-
...unchanged ? {} : { reason: "Source revision changed" }
|
|
16467
|
-
};
|
|
16468
|
-
} catch (error) {
|
|
16469
|
-
if (isAbort(error, signal))
|
|
16470
|
-
throw error;
|
|
16471
|
-
const code = systemErrorCode(error);
|
|
16472
|
-
if (code === "ENOENT") {
|
|
16473
|
-
const classified = await confirmWorktreeState(target.path, cwd, signal);
|
|
16474
|
-
return { path: target.path, role: "source", ...classified };
|
|
16475
|
-
}
|
|
16476
|
-
const classified = await failure(error, undefined, target.path, cwd, signal);
|
|
16477
|
-
return {
|
|
16478
|
-
path: target.path,
|
|
16479
|
-
role: "source",
|
|
16480
|
-
status: classified.status,
|
|
16481
|
-
reason: classified.reason
|
|
16482
|
-
};
|
|
16483
|
-
}
|
|
16484
|
-
}
|
|
16485
16497
|
async function validateAnalysisTarget(target, access, cwd, signal) {
|
|
16486
16498
|
if (!target.expected)
|
|
16487
16499
|
throw new Error("Analysis validation target omitted its source reference");
|
|
@@ -16579,14 +16591,17 @@ async function validateSavedEvidence(options) {
|
|
|
16579
16591
|
const selected = snapshotTargets(snapshot, options.matchIndex);
|
|
16580
16592
|
targets = selected.targets;
|
|
16581
16593
|
missing.push(...selected.missing);
|
|
16582
|
-
const policy = new SearchPathPolicy(options.cwd);
|
|
16583
16594
|
sources = [...missing];
|
|
16584
|
-
let checked = 0;
|
|
16585
16595
|
const maxFiles = options.maxFiles ?? MAX_STRUCTURE_FILES;
|
|
16596
|
+
const access = new SourceAccess(options.cwd, options.queue, options.signal, {
|
|
16597
|
+
maxFiles
|
|
16598
|
+
});
|
|
16599
|
+
let checked = 0;
|
|
16600
|
+
let budgetExhausted = false;
|
|
16586
16601
|
for (const target of targets) {
|
|
16587
16602
|
if (options.signal?.aborted)
|
|
16588
16603
|
throw abortError();
|
|
16589
|
-
if (checked >= maxFiles) {
|
|
16604
|
+
if (budgetExhausted || checked >= maxFiles) {
|
|
16590
16605
|
sources.push({
|
|
16591
16606
|
path: target.path,
|
|
16592
16607
|
role: "source",
|
|
@@ -16596,7 +16611,11 @@ async function validateSavedEvidence(options) {
|
|
|
16596
16611
|
continue;
|
|
16597
16612
|
}
|
|
16598
16613
|
checked += 1;
|
|
16599
|
-
|
|
16614
|
+
const status = await validateAnalysisTarget(target, access, options.cwd, options.signal);
|
|
16615
|
+
const { budgetReached, ...sourceStatus } = status;
|
|
16616
|
+
sources.push(sourceStatus);
|
|
16617
|
+
if (budgetReached)
|
|
16618
|
+
budgetExhausted = true;
|
|
16600
16619
|
}
|
|
16601
16620
|
}
|
|
16602
16621
|
if (sources.length === 0)
|
|
@@ -19490,7 +19509,7 @@ class EvidenceService {
|
|
|
19490
19509
|
if (selected.unverified)
|
|
19491
19510
|
throw new SiftLightError("Snapshot source revision is unverified; refresh the search");
|
|
19492
19511
|
if (selected.expectedRevision) {
|
|
19493
|
-
const doc = await access.load(path);
|
|
19512
|
+
const doc = await access.load(path, selected.reference);
|
|
19494
19513
|
if (doc.reference.origin.kind !== "worktree" || !sameSourceRevision(selected.expectedRevision, doc.reference.origin.revision))
|
|
19495
19514
|
throw new SiftLightError("Source changed; refresh the search");
|
|
19496
19515
|
reference = doc.reference;
|
|
@@ -20275,7 +20294,8 @@ class SnapshotStore {
|
|
|
20275
20294
|
return scan.retention?.accountedBytes ?? Buffer.byteLength(JSON.stringify({
|
|
20276
20295
|
matches: scan.matches,
|
|
20277
20296
|
fileCounts: [...scan.fileCounts],
|
|
20278
|
-
sourceRevisions: [...scan.sourceRevisions]
|
|
20297
|
+
sourceRevisions: [...scan.sourceRevisions],
|
|
20298
|
+
sourceReferences: scan.sourceReferences ? [...scan.sourceReferences] : []
|
|
20279
20299
|
}));
|
|
20280
20300
|
}
|
|
20281
20301
|
#scanOccurrences(scan) {
|
|
@@ -20804,11 +20824,11 @@ function cursorPathSelection(input, cwd) {
|
|
|
20804
20824
|
absolutePaths.add(absolutePath);
|
|
20805
20825
|
labels.push(label);
|
|
20806
20826
|
}
|
|
20807
|
-
const key =
|
|
20827
|
+
const key = createHash8("sha256").update([...absolutePaths].toSorted((left, right) => left.localeCompare(right)).join("\x00")).digest("hex").slice(0, 16);
|
|
20808
20828
|
return { labels, absolutePaths, key };
|
|
20809
20829
|
}
|
|
20810
20830
|
function baseDetails2(snapshot, mode) {
|
|
20811
|
-
const sourceUnverifiedFileCount = new Set(snapshot.matches.filter((match) => !snapshot.
|
|
20831
|
+
const sourceUnverifiedFileCount = new Set(snapshot.matches.filter((match) => !snapshot.sourceReferences?.has(match.absolutePath)).map((match) => match.absolutePath)).size;
|
|
20812
20832
|
return {
|
|
20813
20833
|
version: 1,
|
|
20814
20834
|
mode,
|