@dailephd/my-dev-kit-lab 0.9.3 → 0.9.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +26 -0
- package/README.md +7 -3
- package/dist/src/evaluation/affectedNeighborhood.js +140 -32
- package/dist/src/evaluation/benchmarkSandbox/createBenchmarkSandbox.js +9 -1
- package/dist/src/evaluation/benchmarkSandbox/gitExecutor.js +91 -1
- package/dist/src/evaluation/runMyDevKitRetrieval.js +137 -4
- package/dist/src/evaluation/syntheticRepository/materialize.js +103 -7
- package/dist/src/experiments/plugins/incrementalChangeStaleness/comparison.js +10 -0
- package/dist/src/experiments/plugins/incrementalChangeStaleness/comparisonV2.js +9 -1
- package/dist/src/experiments/plugins/incrementalChangeStaleness/execution.js +41 -5
- package/dist/src/experiments/plugins/incrementalChangeStaleness/executionV2.js +4 -4
- package/dist/src/report/experiments/buildIncrementalChangeStalenessReportV2.js +4 -0
- package/dist/src/report/experiments/incrementalChangeStalenessReportRowsV2.js +1 -1
- package/dist/src/securityValidation/attackScenarios/boundedSourceScan.js +112 -25
- package/dist/src/securityValidation/attackScenarios/scenarios/networkAssumptionScenario.js +58 -6
- package/dist/src/securityValidation/cliAdversarial/pathBoundaryChecks.js +108 -70
- package/dist/src/securityValidation/cliAdversarial/tempWorkspace.js +57 -8
- package/dist/src/securityValidation/cliAdversarial/traversalContainment.js +161 -0
- package/dist/src/securityValidation/commandRunner.js +52 -26
- package/dist/src/securityValidation/config.js +10 -0
- package/dist/src/securityValidation/packageChecks/parseNpmPackDryRun.js +46 -21
- package/dist/src/securityValidation/validate/runCliSecuritySuiteCheck.js +21 -5
- package/dist/src/securityValidation/validate/runSecurityValidation.js +2 -2
- package/dist/src/tutorial/tutorialAssertions.js +7 -0
- package/docs/METRICS.md +11 -1
- package/package.json +3 -3
package/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,32 @@
|
|
|
2
2
|
|
|
3
3
|
All notable changes to my-dev-kit-lab are documented here.
|
|
4
4
|
|
|
5
|
+
## [0.9.5] - 2026-10-27
|
|
6
|
+
|
|
7
|
+
### Staleness experiment evidence correction
|
|
8
|
+
|
|
9
|
+
- The `incremental-change-staleness` experiment now retrieves through an opt-in bounded multi-source protocol in the shared retrieval owner (`runMyDevKitBoundedRetrievalFromIndex`) so multi-file required evidence can be observed; default top-one retrieval and the meaning of its `filesRead` are unchanged for every other caller. Only files whose own `source` command succeeded count as read.
|
|
10
|
+
- Required-file evidence reports `unknown` instead of `missing` when the bounded protocol cannot observe the requirement, and a full-refresh reference that did not itself observe the required files can no longer make equal results look like `same`/no observed regression: the relation is `unknown` and the classification `inconclusive`.
|
|
11
|
+
- The V2 report labels the fake-agent score as synthetic, distinguishes read from merely surfaced files, and states that runtime command-log paths are not retained after cleanup. Persisted V1/V2 schemas, treatment ordering, scenarios, answer keys and the fake agent and scorer are unchanged.
|
|
12
|
+
- The validation harness now gives disposable provider shims an explicit CommonJS package boundary; pins benchmark Git operations to their own initialized repository and rejects ancestor discovery and dangerous per-call redirects; and contains adversarial `--out`/`--index` traversal tests inside disposable enclosures before launching the tested CLI.
|
|
13
|
+
- Tutorial and non-Git fixtures now establish their package and repository boundaries explicitly. On Windows, synthetic repository publication retries only transient `EPERM` rename failures, with a finite three-attempt limit; if staging cleanup also fails, its diagnostic is appended while the primary materialization error remains the reported error.
|
|
14
|
+
- Packed-consumer staleness acceptance reuses the installed producer's reference-aware comparisons for all six scenarios and all three comparisons per scenario, including ordered reason codes; it retains the production artifact validator and does not change the scientific standard.
|
|
15
|
+
- Replaced drive-root adversarial test probes with enclosed safe/unsafe witnesses. Network scanning recognizes two narrowly reviewed local HTTP sites only with strict source identities (CRLF-to-LF normalization only), expected fetch counts, caller evidence, and physical provenance; unreviewed calls and reportable provenance refusals remain major findings.
|
|
16
|
+
- Corrected shared bounded source collection so ordinary files named like generated directories remain eligible, while actual generated directories stay excluded and case aliases cannot inherit reviewed trust.
|
|
17
|
+
- Security command timeouts now use bounded process-tree termination; required nested security-suite failures retain actual timeout, exit, signal, start-error and cleanup evidence without weakening major-finding severity.
|
|
18
|
+
- No Lab CLI flags, experiment schemas, or dependencies were added. Default retrieval behavior outside this experiment remains unchanged; persisted V1/V2 schemas, treatment ordering, scenarios, answer keys, fake agent, and scorer remain unchanged.
|
|
19
|
+
|
|
20
|
+
## [0.9.4] - 2026-10-23
|
|
21
|
+
|
|
22
|
+
### Kit schema-3 and npm 12 compatibility
|
|
23
|
+
|
|
24
|
+
- Accepted my-dev-kit symbol-index schema `3` in addition to schema `2` for the `incremental-change-staleness` affected-neighborhood evidence. Schema-3 symbols use their persisted qualified `nodeId`; identities are not reconstructed from paths and simple names. Same-name symbols in one file stay distinct, ambiguous names are reported as ambiguous, and missing or mismatched identities stay unresolved. Other schema values are rejected, and schema-2 results are unchanged.
|
|
25
|
+
- Package content checks and `npm run verify:packed-package` accept the npm 11 and npm 12 `npm pack --json` formats through one validator that requires exactly one package, a nonempty file list, and safe relative paths.
|
|
26
|
+
- Corrected the packed-package verifier so a symlink or junction invocation runs, early failures print `PACKED_PACKAGE_VERDICT: FAIL`, and verified failures exit nonzero.
|
|
27
|
+
- Corrected the npm 12 output-prefix assumption in the target `test:security` test without weakening its assertions.
|
|
28
|
+
- Patched vulnerable development-only dependencies by pinning Vite to 7.3.7 and Vitest to 4.1.12 (with `@vitest/mocker` 4.1.12), which clears the dependency-audit advisories GHSA-587h-7wmj-h5pm, GHSA-qxg6-89mg-rqv8, and GHSA-xp26-3454-mjgp. The published runtime dependencies and package contents are unchanged.
|
|
29
|
+
- Security-verdict policy, report and execution schemas, commands, and package contents are unchanged.
|
|
30
|
+
|
|
5
31
|
## [0.9.3] - 2026-10-16
|
|
6
32
|
|
|
7
33
|
- Improved security-validator runtime evidence and secret-scan accuracy while retaining bounded proof and candidate-versus-confirmed finding distinctions.
|
package/README.md
CHANGED
|
@@ -6,21 +6,25 @@ my-dev-kit provides local repository indexing and graph-guided retrieval. my-dev
|
|
|
6
6
|
|
|
7
7
|
Whole-ecosystem workflows are centralized in [my-dev-kit/docs/ECOSYSTEM_DEVELOPMENT_WORKFLOWS.md](https://github.com/dailephd/my-dev-kit/blob/main/docs/ECOSYSTEM_DEVELOPMENT_WORKFLOWS.md). Its [command-surface compatibility map](https://github.com/dailephd/my-dev-kit/blob/main/docs/ECOSYSTEM_DEVELOPMENT_WORKFLOWS.md#915-command-surface-compatibility-map) documents which Lab outputs can safely seed my-dev-kit investigation, Observer reference work, or Orchestrator evidence, and which look-alike artifacts are not direct inputs.
|
|
8
8
|
|
|
9
|
-
|
|
9
|
+
The v0.9.3 release added hardened prompt contracts and output validation, extensible reports with metric evidence, broader gallery inputs, and prompt-template comparison. The v0.4.5 context-integrity evaluation remains intentionally frozen against the published `@dailephd/my-dev-kit@1.10.4` and `@dailephd/my-dev-kit-orchestrator@1.2.3` contracts; those versions are historical validation baselines, not a statement that they are the ecosystem's current releases. See [docs/CURRENT_STATE.md](docs/CURRENT_STATE.md) for current state.
|
|
10
10
|
|
|
11
11
|
v0.9.0 adds the `agent-success-rate` plugin, the eighth registered experiment plugin, which evaluates implementation patches under the `raw-full-file` and `context-pack` treatments.
|
|
12
12
|
|
|
13
|
-
v0.9.
|
|
13
|
+
v0.9.4 is the previous package release.
|
|
14
14
|
|
|
15
15
|
The v0.9.2 release corrected external npm package entrypoint validation, preserved strict Lab self-package validation, eliminated nine inherited false positives, corrected Semgrep failed/incomplete-scan evidence, and included campaign filesystem identity, path-safety, and cross-platform corrections. Security verdict strictness is unchanged. The v0.9.2 release-document state is final; actual npm publication is established separately through registry evidence.
|
|
16
16
|
|
|
17
17
|
The v0.9.3 release adds configurable Codex/Claude prompt contracts, strict structured-diff output handling, generic report sections, HTML metric evidence and caveats, source-based gallery items including tutorial manifests, and prompt-template comparison. See [docs/CURRENT_STATE.md](docs/CURRENT_STATE.md) for release and validation details.
|
|
18
18
|
|
|
19
|
+
The v0.9.4 release supports my-dev-kit symbol-index schema 3 alongside schema 2, preserves qualified symbol identities, and handles ambiguous names conservatively. Package inventory validation supports npm 11 and npm 12 formats, and the packed-package verifier reports early failures reliably. Development-only Vite and Vitest dependencies are patched; security-verdict policy is unchanged. See [docs/CURRENT_STATE.md](docs/CURRENT_STATE.md), [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md), [docs/COMMANDS.md](docs/COMMANDS.md), and [docs/WORKFLOWS.md](docs/WORKFLOWS.md).
|
|
20
|
+
|
|
21
|
+
The v0.9.5 release corrects incremental-change-staleness evidence interpretation and strengthens disposable validation boundaries. Its bounded multi-source evidence path is opt-in; default top-one retrieval is unchanged. Insufficient evidence remains unknown or inconclusive, and synthetic fake-agent scores stay distinct from observed retrieval. No Lab CLI flags or experiment schemas were added.
|
|
22
|
+
|
|
19
23
|
v0.4.6 adds a supported `my-dev-kit-lab` installed CLI router (`--help`, `--version`, `security validate`, `audit`, the `experiment` family, `report render`, `plots generate`, `gallery build`, `demo final`, and the historical direct final-demo invocation form), a writable lab workspace model kept separate from the installed package and the inspected target, and a permanent packed-tarball installation/execution acceptance gate (`npm run verify:packed-package`). See [docs/ROADMAP.md](docs/ROADMAP.md) and [docs/CURRENT_STATE.md](docs/CURRENT_STATE.md) for status detail and [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for the runtime path model. The "Installed CLI" section below documents the shipped command surface; the source-checkout `npm run` workflow in Quickstart remains available for contributors.
|
|
20
24
|
|
|
21
25
|
## Current capabilities
|
|
22
26
|
|
|
23
|
-
Package metadata is `0.9.
|
|
27
|
+
Package metadata is `0.9.5`. The v0.9.5 release corrects bounded evidence collection and uncertainty handling in the incremental-change-staleness experiment, preserves default retrieval behavior, and strengthens disposable test and validation boundaries. It adds no command, flag, experiment schema, or dependency. The v0.9.4 release supports my-dev-kit symbol-index schema 3 alongside schema 2, preserves qualified identities, handles ambiguity conservatively, supports npm 11 and npm 12 package inventories, corrects packed-package verifier behavior, and patches development-only security dependencies. Security-verdict policy is unchanged. The v0.9.3 release added configurable Codex/Claude prompt contracts, strict structured-diff output handling, generic report sections, HTML metric evidence and caveats, source-based gallery items including tutorial manifests, and prompt-template comparison. Registered metrics have evidence-based glossary definitions and links, and prompt comparison validates persisted identity and checkpoint evidence. Real-provider improvement remains NOT_VERIFIED because no paid smoke was authorized; its measurement is a separate opt-in empirical objective, not a v0.9.3 release gate. Bounded tool-use remains prompt guidance, not provider enforcement. See [Current State](docs/CURRENT_STATE.md) and [the roadmap](docs/ROADMAP.md).
|
|
24
28
|
|
|
25
29
|
- **Run context-strategy experiments:** compare `raw-full-file` with `my-dev-kit-guided` using deterministic fixtures or locally configured Codex and Claude CLIs.
|
|
26
30
|
- **Measure warm-index reuse (v0.5.0–v0.5.1, released):** the `warm-index-reuse` experiment plugin builds one my-dev-kit index per benchmark project, reuses it across that project's tasks, and compares every task with a matched `raw-full-file` baseline. Reports separate one-time index-build cost from per-task retrieval cost, show amortized index cost and cumulative measurements, and include deterministic fake-agent correctness and token evidence; `plots generate` renders four warm-index SVG charts. The v0.5.1 release adds a dedicated 12-task benchmark corpus (`benchmarks/contracts/warm-index-benchmark-cases.json`: six medium and six large/mixed tasks tagged `localized`, `cross-module`, or `broad-change`) for comparing warm-index behavior as task count grows. Estimated context tokens (a character-based context-size estimate) and fake-agent total tokens (simulated harness telemetry) are reported separately, and neither is provider billing telemetry.
|
|
@@ -4,7 +4,13 @@ import { compareCodeUnits, interpretIndexManifest, readJson } from "./indexSnaps
|
|
|
4
4
|
export const AFFECTED_NEIGHBORHOOD_GRAPH_EVIDENCE_SCHEMA_VERSION = "my-dev-kit-lab-affected-neighborhood-graph-evidence-v1";
|
|
5
5
|
export const AFFECTED_NEIGHBORHOOD_SEED_MAPPING_SCHEMA_VERSION = "my-dev-kit-lab-affected-neighborhood-seed-mapping-v1";
|
|
6
6
|
const MANIFEST_FILE = "manifest.json";
|
|
7
|
-
|
|
7
|
+
/**
|
|
8
|
+
* Schema 2 (my-dev-kit 1.13.x) identifies a symbol by file path + name. Schema 3 (my-dev-kit 1.14.x)
|
|
9
|
+
* additionally persists each symbol's actual graph `nodeId`, which may be a distinct `symbolq:` identity
|
|
10
|
+
* for same-named members. Both are understood; anything else is rejected.
|
|
11
|
+
*/
|
|
12
|
+
const SUPPORTED_SYMBOL_INDEX_SCHEMA_VERSIONS = ["2", "3"];
|
|
13
|
+
const SCHEMA_WITH_PERSISTED_SYMBOL_NODE_IDS = "3";
|
|
8
14
|
const SUPPORTED_CODE_GRAPH_ARTIFACT_KIND = "code-graph";
|
|
9
15
|
const SUPPORTED_CODE_GRAPH_VERSION_PATTERN = /^1\./;
|
|
10
16
|
// Same bounds as the v0.6.0 freshness/snapshot evidence lists.
|
|
@@ -57,18 +63,23 @@ function sortWarnings(warnings) {
|
|
|
57
63
|
return [...warnings].sort((left, right) => compareCodeUnits(left.code, right.code) || compareCodeUnits(left.subject ?? "", right.subject ?? "") || compareCodeUnits(left.message, right.message));
|
|
58
64
|
}
|
|
59
65
|
/**
|
|
60
|
-
* Policy: reads indexed-file membership and baseline
|
|
61
|
-
*
|
|
62
|
-
*
|
|
63
|
-
*
|
|
66
|
+
* Policy: reads indexed-file membership and baseline symbol identities from a symbol-index artifact.
|
|
67
|
+
* Schema versions 2 and 3 are understood; anything else is unsupported. Under schema 2 a symbol's
|
|
68
|
+
* identity is its upstream file+name node ID. Under schema 3 it is the `nodeId` the index persisted,
|
|
69
|
+
* which is never reconstructed from the simple name: same-named members keep distinct identities,
|
|
70
|
+
* and a record without a usable `nodeId` stays visible as an unmappable identity with a warning
|
|
71
|
+
* instead of receiving a guessed one. A file entry whose symbol list is malformed keeps its
|
|
72
|
+
* membership but yields no symbol record; a malformed symbol entry is dropped with a warning. Pure.
|
|
64
73
|
*/
|
|
65
74
|
export function interpretSymbolIndexForNeighborhood(symbolIndex) {
|
|
66
75
|
if (!isRecord(symbolIndex)) {
|
|
67
76
|
return { ok: false, message: "Symbol index is not a JSON object." };
|
|
68
77
|
}
|
|
69
|
-
|
|
70
|
-
|
|
78
|
+
const schemaVersion = symbolIndex.schemaVersion;
|
|
79
|
+
if (typeof schemaVersion !== "string" || !SUPPORTED_SYMBOL_INDEX_SCHEMA_VERSIONS.includes(schemaVersion)) {
|
|
80
|
+
return { ok: false, message: `Symbol index schemaVersion is not a supported version (${SUPPORTED_SYMBOL_INDEX_SCHEMA_VERSIONS.join(", ")}).` };
|
|
71
81
|
}
|
|
82
|
+
const persistsNodeIds = schemaVersion === SCHEMA_WITH_PERSISTED_SYMBOL_NODE_IDS;
|
|
72
83
|
if (!Array.isArray(symbolIndex.files)) {
|
|
73
84
|
return { ok: false, message: "Symbol index does not list indexed files." };
|
|
74
85
|
}
|
|
@@ -92,29 +103,57 @@ export function interpretSymbolIndexForNeighborhood(symbolIndex) {
|
|
|
92
103
|
});
|
|
93
104
|
continue;
|
|
94
105
|
}
|
|
95
|
-
const
|
|
106
|
+
const identities = new Map();
|
|
96
107
|
const duplicates = new Set();
|
|
108
|
+
const missingNodeIdNames = new Set();
|
|
97
109
|
for (const symbol of entry.symbols) {
|
|
98
110
|
if (!isRecord(symbol) || !nonEmptyString(symbol.name)) {
|
|
99
111
|
warnings.push({ code: "invalid-symbol-entry", subject: boundedText(filePath), message: "Symbol index entry lacks a symbol name and was ignored." });
|
|
100
112
|
continue;
|
|
101
113
|
}
|
|
102
|
-
|
|
103
|
-
|
|
114
|
+
const name = symbol.name;
|
|
115
|
+
let identity;
|
|
116
|
+
let identityKey;
|
|
117
|
+
if (!persistsNodeIds) {
|
|
118
|
+
identity = { name, nodeId: symbolNodeId(filePath, name), persisted: false };
|
|
119
|
+
identityKey = identity.nodeId;
|
|
120
|
+
}
|
|
121
|
+
else if (nonEmptyString(symbol.nodeId)) {
|
|
122
|
+
identity = { name, nodeId: symbol.nodeId, persisted: true };
|
|
123
|
+
identityKey = `id:${symbol.nodeId}`;
|
|
104
124
|
}
|
|
105
|
-
|
|
125
|
+
else {
|
|
126
|
+
// No persisted identity: one unmappable identity per name, never a reconstructed one.
|
|
127
|
+
identity = { name, nodeId: null, persisted: true };
|
|
128
|
+
identityKey = `missing:${name}`;
|
|
129
|
+
missingNodeIdNames.add(name);
|
|
130
|
+
}
|
|
131
|
+
if (identities.has(identityKey)) {
|
|
132
|
+
duplicates.add(name);
|
|
133
|
+
continue;
|
|
134
|
+
}
|
|
135
|
+
identities.set(identityKey, identity);
|
|
106
136
|
}
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
137
|
+
for (const name of [...missingNodeIdNames].sort(compareCodeUnits)) {
|
|
138
|
+
warnings.push({
|
|
139
|
+
code: "symbol-node-id-missing",
|
|
140
|
+
subject: boundedText(`${filePath}#${name}`),
|
|
141
|
+
message: "Schema-3 symbol record has no usable persisted nodeId; its identity is not guessed and it cannot be mapped to a graph node.",
|
|
142
|
+
});
|
|
143
|
+
}
|
|
144
|
+
const sortedRecords = [...identities.values()].sort((left, right) => compareCodeUnits(left.nodeId ?? "", right.nodeId ?? "") || compareCodeUnits(left.name, right.name));
|
|
145
|
+
const names = [...new Set(sortedRecords.map((record) => record.name))].sort(compareCodeUnits);
|
|
146
|
+
const sameName = names.filter((name) => sortedRecords.filter((record) => record.name === name).length > 1);
|
|
147
|
+
// Schema-2 evidence keeps its original shape: each name is its only identity, so no per-record data is added.
|
|
148
|
+
const fileRecord = { symbolNames: names, duplicateSymbolNames: [...duplicates].sort(compareCodeUnits) };
|
|
149
|
+
records.set(filePath, persistsNodeIds ? { ...fileRecord, records: sortedRecords, sameNameSymbolNames: sameName } : fileRecord);
|
|
111
150
|
}
|
|
112
151
|
const indexedFilePaths = [...paths].sort(compareCodeUnits);
|
|
113
152
|
const symbolFiles = new Map();
|
|
114
153
|
for (const key of [...records.keys()].sort(compareCodeUnits)) {
|
|
115
154
|
symbolFiles.set(key, records.get(key));
|
|
116
155
|
}
|
|
117
|
-
return { ok: true, value: { indexedFilePaths, symbolFiles, warnings } };
|
|
156
|
+
return { ok: true, value: { schemaVersion, indexedFilePaths, symbolFiles, warnings } };
|
|
118
157
|
}
|
|
119
158
|
/**
|
|
120
159
|
* Policy: validates the structure of a code-graph artifact and preserves its nodes and edges by
|
|
@@ -234,7 +273,7 @@ export async function loadAffectedNeighborhoodGraphEvidence(options) {
|
|
|
234
273
|
}
|
|
235
274
|
const symbolIndexSummary = {
|
|
236
275
|
path: normalizePath(path.relative(path.resolve(indexDir), symbolIndexFile)),
|
|
237
|
-
schemaVersion:
|
|
276
|
+
schemaVersion: symbols.value.schemaVersion,
|
|
238
277
|
};
|
|
239
278
|
const warnings = [...symbols.value.warnings];
|
|
240
279
|
let codeGraphSummary = null;
|
|
@@ -310,10 +349,35 @@ function lookupNode(evidence, expectedNodeId, kind) {
|
|
|
310
349
|
export function lookupFileNode(evidence, filePath) {
|
|
311
350
|
return lookupNode(evidence, fileNodeId(filePath), "file");
|
|
312
351
|
}
|
|
313
|
-
/**
|
|
352
|
+
/**
|
|
353
|
+
* Same-file symbol lookup by the upstream file+name identity. Never searches for a same-named symbol
|
|
354
|
+
* in another file. This is the schema-2 identity only; it does not find a qualified (`symbolq:`) member.
|
|
355
|
+
*/
|
|
314
356
|
export function lookupSymbolNode(evidence, filePath, symbolName) {
|
|
315
357
|
return lookupNode(evidence, symbolNodeId(filePath, symbolName), "symbol");
|
|
316
358
|
}
|
|
359
|
+
/**
|
|
360
|
+
* Exact lookup of one baseline symbol identity. The node must exist under the identity the index
|
|
361
|
+
* actually recorded, be a `symbol` node, and (when the graph names a path) belong to the same
|
|
362
|
+
* file; otherwise it is unresolved. A record without a persisted identity is never mapped, and no
|
|
363
|
+
* identity is derived from its name.
|
|
364
|
+
*/
|
|
365
|
+
export function lookupSymbolRecordNode(evidence, filePath, record) {
|
|
366
|
+
if (record.nodeId === null)
|
|
367
|
+
return { ok: false, expectedNodeId: "", reason: "symbol-node-id-missing" };
|
|
368
|
+
const lookup = lookupNode(evidence, record.nodeId, "symbol");
|
|
369
|
+
if (!lookup.ok)
|
|
370
|
+
return lookup;
|
|
371
|
+
const node = evidence.nodes.get(lookup.nodeId);
|
|
372
|
+
if (node && node.path !== null && normalizePath(node.path) !== filePath) {
|
|
373
|
+
return { ok: false, expectedNodeId: record.nodeId, reason: "graph-node-path-mismatch" };
|
|
374
|
+
}
|
|
375
|
+
return lookup;
|
|
376
|
+
}
|
|
377
|
+
/** The baseline identities of one indexed file; evidence without per-record data keeps the schema-2 file+name identity. */
|
|
378
|
+
function symbolRecordsOf(filePath, file) {
|
|
379
|
+
return file.records ?? file.symbolNames.map((name) => ({ name, nodeId: symbolNodeId(filePath, name), persisted: false }));
|
|
380
|
+
}
|
|
317
381
|
/**
|
|
318
382
|
* Maps confirmed changed indexed files, and the baseline symbols belonging to them, onto baseline
|
|
319
383
|
* graph nodes. Pure and deterministic: consumes only the v0.6.0 snapshot and freshness assessment
|
|
@@ -404,18 +468,21 @@ export function mapAffectedNeighborhoodSeeds(input) {
|
|
|
404
468
|
continue;
|
|
405
469
|
}
|
|
406
470
|
if (record.duplicateSymbolNames.length > 0) {
|
|
407
|
-
warn("duplicate-symbol-records", `Baseline symbol index
|
|
471
|
+
warn("duplicate-symbol-records", `Baseline symbol index repeats ${record.duplicateSymbolNames.length} identical symbol record(s); each counts as one identity.`, filePath);
|
|
408
472
|
}
|
|
409
|
-
|
|
473
|
+
// Each distinct baseline identity is a seed in its own right: same-named members that the index
|
|
474
|
+
// persisted with different node IDs are never merged, and none is reconstructed from the name.
|
|
475
|
+
for (const identity of symbolRecordsOf(filePath, record)) {
|
|
476
|
+
const name = identity.name;
|
|
410
477
|
symbolIdentityCount += 1;
|
|
411
|
-
const symbolExpectedId =
|
|
478
|
+
const symbolExpectedId = identity.nodeId ?? "";
|
|
412
479
|
let symbolResolvedId = null;
|
|
413
480
|
let symbolReason = null;
|
|
414
481
|
if (!codeGraphLoaded) {
|
|
415
482
|
symbolReason = "graph-unavailable";
|
|
416
483
|
}
|
|
417
484
|
else {
|
|
418
|
-
const lookup =
|
|
485
|
+
const lookup = lookupSymbolRecordNode(graph, filePath, identity);
|
|
419
486
|
if (lookup.ok)
|
|
420
487
|
symbolResolvedId = lookup.nodeId;
|
|
421
488
|
else
|
|
@@ -426,7 +493,12 @@ export function mapAffectedNeighborhoodSeeds(input) {
|
|
|
426
493
|
seeds.add(symbolResolvedId);
|
|
427
494
|
}
|
|
428
495
|
else {
|
|
429
|
-
unresolved.push({
|
|
496
|
+
unresolved.push({
|
|
497
|
+
subject: "changed-symbol",
|
|
498
|
+
path: boundedText(filePath),
|
|
499
|
+
name: boundedText(name),
|
|
500
|
+
reason: symbolReason,
|
|
501
|
+
});
|
|
430
502
|
}
|
|
431
503
|
changedSymbols.push({
|
|
432
504
|
path: filePath,
|
|
@@ -490,11 +562,41 @@ function uniqueSorted(values) {
|
|
|
490
562
|
function normalizeExpectedPath(value) {
|
|
491
563
|
return normalizePath(value).replace(/^(\.\/)+/, "");
|
|
492
564
|
}
|
|
565
|
+
/**
|
|
566
|
+
* Candidate graph nodes for one expected symbol in one expected file. A request matches a baseline
|
|
567
|
+
* identity by its simple name or by that identity's exact persisted node ID; every matching identity
|
|
568
|
+
* is a distinct candidate (same-named members are never collapsed). A file the symbol index has no
|
|
569
|
+
* record for keeps the schema-2 file+name lookup. Nothing is derived from the name beyond that.
|
|
570
|
+
*/
|
|
571
|
+
function resolveExpectedSymbolInFile(graph, filePath, request) {
|
|
572
|
+
const file = graph.symbolFiles.get(filePath);
|
|
573
|
+
if (!file) {
|
|
574
|
+
const lookup = lookupSymbolNode(graph, filePath, request);
|
|
575
|
+
return { nodeIds: lookup.ok ? [lookup.nodeId] : [], identityMissing: false, pathMismatch: !lookup.ok && lookup.reason === "graph-node-path-mismatch" };
|
|
576
|
+
}
|
|
577
|
+
const nodeIds = [];
|
|
578
|
+
let identityMissing = false;
|
|
579
|
+
let pathMismatch = false;
|
|
580
|
+
for (const identity of symbolRecordsOf(filePath, file)) {
|
|
581
|
+
if (identity.name !== request && identity.nodeId !== request)
|
|
582
|
+
continue;
|
|
583
|
+
const lookup = lookupSymbolRecordNode(graph, filePath, identity);
|
|
584
|
+
if (lookup.ok)
|
|
585
|
+
nodeIds.push(lookup.nodeId);
|
|
586
|
+
else if (lookup.reason === "symbol-node-id-missing")
|
|
587
|
+
identityMissing = true;
|
|
588
|
+
else if (lookup.reason === "graph-node-path-mismatch")
|
|
589
|
+
pathMismatch = true;
|
|
590
|
+
}
|
|
591
|
+
return { nodeIds, identityMissing, pathMismatch };
|
|
592
|
+
}
|
|
493
593
|
/**
|
|
494
594
|
* Policy: maps a task's expected files and symbols to baseline graph nodes. Files resolve only to
|
|
495
595
|
* their exact `file:` node. A symbol resolves only within the task's own expected files and only
|
|
496
|
-
* when exactly one
|
|
497
|
-
*
|
|
596
|
+
* when exactly one matching identity exists there; zero matches is unresolved and several (in one
|
|
597
|
+
* file or across files, including same-named members with distinct persisted node IDs) is
|
|
598
|
+
* ambiguous (no winner is picked, no node is added). A request that is itself a persisted node ID
|
|
599
|
+
* resolves to exactly that identity. Never searches the rest of the graph. Pure.
|
|
498
600
|
*/
|
|
499
601
|
export function mapAffectedNeighborhoodTask(input) {
|
|
500
602
|
const { graph } = input;
|
|
@@ -561,15 +663,21 @@ export function mapAffectedNeighborhoodTask(input) {
|
|
|
561
663
|
const resolvedSymbols = [];
|
|
562
664
|
for (const name of expectedSymbols) {
|
|
563
665
|
// Scope is the task's expected files, whether or not their own file node resolved.
|
|
564
|
-
const
|
|
565
|
-
|
|
566
|
-
|
|
567
|
-
|
|
568
|
-
|
|
666
|
+
const found = expectedFiles.map((filePath) => resolveExpectedSymbolInFile(graph, filePath, name));
|
|
667
|
+
const candidates = uniqueSorted(found.flatMap((entry) => entry.nodeIds));
|
|
668
|
+
// A same-name identity that persisted no node ID cannot be excluded, so a lone mappable candidate
|
|
669
|
+
// is not trusted as "the" owner when such a sibling exists.
|
|
670
|
+
const identityMissing = found.some((entry) => entry.identityMissing);
|
|
671
|
+
if (candidates.length === 1 && !identityMissing) {
|
|
569
672
|
resolvedSymbols.push({ name, nodeId: candidates[0] });
|
|
570
673
|
}
|
|
571
|
-
else if (candidates.length
|
|
572
|
-
|
|
674
|
+
else if (candidates.length <= 1) {
|
|
675
|
+
const reason = identityMissing
|
|
676
|
+
? "symbol-node-id-missing"
|
|
677
|
+
: found.some((entry) => entry.pathMismatch)
|
|
678
|
+
? "graph-node-path-mismatch"
|
|
679
|
+
: "graph-node-missing";
|
|
680
|
+
unresolved.push({ subject: "expected-symbol", name: boundedText(name), reason });
|
|
573
681
|
}
|
|
574
682
|
else {
|
|
575
683
|
ambiguous.push({ name: boundedText(name), candidateNodeIds: candidates });
|
|
@@ -3,7 +3,7 @@ import { copyFile, lstat, mkdir, realpath, rm, stat, writeFile } from "node:fs/p
|
|
|
3
3
|
import path from "node:path";
|
|
4
4
|
import { resolveWithinRoot } from "../../core/pathSafety.js";
|
|
5
5
|
import { BenchmarkSandboxError } from "./errors.js";
|
|
6
|
-
import { excerptGitFailure, runSandboxGit } from "./gitExecutor.js";
|
|
6
|
+
import { excerptGitFailure, runSandboxGit, verifySandboxRepositoryBinding } from "./gitExecutor.js";
|
|
7
7
|
import { BENCHMARK_SANDBOX_EXCLUDED_NAMES, isSameOrInside, resolvePhysicalPath, SANDBOX_ID_PATTERN } from "./pathPolicy.js";
|
|
8
8
|
import { digestTreeSnapshot, scanProjectTree, snapshotProjectTree } from "./treeSnapshot.js";
|
|
9
9
|
import { BENCHMARK_SANDBOX_BASELINE_BRANCH, BENCHMARK_SANDBOX_OWNER_MARKER_FILE, BENCHMARK_SANDBOX_SCHEMA_VERSION } from "./types.js";
|
|
@@ -93,6 +93,14 @@ export async function createBenchmarkSandbox(options) {
|
|
|
93
93
|
}
|
|
94
94
|
const git = { projectRoot, evidenceRoot };
|
|
95
95
|
await mustGit(git, ["init", "-q", `--initial-branch=${BENCHMARK_SANDBOX_BASELINE_BRANCH}`], "init");
|
|
96
|
+
// A zero exit from init is not proof that later commands reach this repository: on some hosts Git discovery
|
|
97
|
+
// skips the new .git and selects an ancestor. Nothing may be configured, staged or committed before this passes.
|
|
98
|
+
const binding = await verifySandboxRepositoryBinding(git);
|
|
99
|
+
if (!binding.ok) {
|
|
100
|
+
throw binding.unavailable
|
|
101
|
+
? new BenchmarkSandboxError("GIT_UNAVAILABLE", "the git executable is not available.")
|
|
102
|
+
: new BenchmarkSandboxError("GIT_REPOSITORY_MISMATCH", binding.reason);
|
|
103
|
+
}
|
|
96
104
|
await mustGit(git, ["config", "user.name", "my-dev-kit-lab sandbox"], "config-name");
|
|
97
105
|
await mustGit(git, ["config", "user.email", "sandbox@my-dev-kit-lab.invalid"], "config-email");
|
|
98
106
|
await mustGit(git, ["config", "commit.gpgsign", "false"], "config-gpg");
|
|
@@ -1,8 +1,13 @@
|
|
|
1
|
+
import { realpath as realpathCallback } from "node:fs";
|
|
2
|
+
import { lstat } from "node:fs/promises";
|
|
1
3
|
import path from "node:path";
|
|
4
|
+
import { promisify } from "node:util";
|
|
2
5
|
import { runMeasuredCommand } from "../../core/runMeasuredCommand.js";
|
|
3
6
|
import { buildMinimalHostEnv } from "./minimalHostEnv.js";
|
|
4
7
|
// Git (including Git for Windows) opens this as an empty config file; os.devNull (\\.\nul) is rejected by Git on Windows.
|
|
5
8
|
const NULL_CONFIG_PATH = process.platform === "win32" ? "NUL" : "/dev/null";
|
|
9
|
+
/** Variables that can redirect which repository, index or object store a Git command operates on. */
|
|
10
|
+
const REPOSITORY_REDIRECT_ENV = /^GIT_(DIR|WORK_TREE|COMMON_DIR|INDEX_FILE|OBJECT_DIRECTORY|ALTERNATE_OBJECT_DIRECTORIES|NAMESPACE|CEILING_DIRECTORIES|PREFIX|DISCOVERY_ACROSS_FILESYSTEM)$/i;
|
|
6
11
|
const DEFAULT_GIT_TIMEOUT_MS = 60_000;
|
|
7
12
|
const GIT_OUTPUT_LIMIT_BYTES = 8 * 1024 * 1024;
|
|
8
13
|
const sequenceByEvidenceRoot = new Map();
|
|
@@ -17,6 +22,18 @@ function nextSequence(evidenceRoot) {
|
|
|
17
22
|
* and the system's Git configuration so hashes and diffs do not depend on the developer machine.
|
|
18
23
|
*/
|
|
19
24
|
export async function runSandboxGit(target, args, options = {}) {
|
|
25
|
+
const redirecting = Object.keys(options.env ?? {}).find((key) => REPOSITORY_REDIRECT_ENV.test(key));
|
|
26
|
+
if (redirecting !== undefined) {
|
|
27
|
+
return {
|
|
28
|
+
ok: false,
|
|
29
|
+
exitCode: null,
|
|
30
|
+
stdout: "",
|
|
31
|
+
stderr: "",
|
|
32
|
+
timedOut: false,
|
|
33
|
+
unavailable: false,
|
|
34
|
+
error: `sandbox git refuses an environment override that can redirect the repository (${redirecting}).`
|
|
35
|
+
};
|
|
36
|
+
}
|
|
20
37
|
const label = (options.label ?? args[0] ?? "git").replace(/[^A-Za-z0-9_-]/g, "-").slice(0, 32);
|
|
21
38
|
const commandId = `git-${String(nextSequence(target.evidenceRoot)).padStart(4, "0")}-${label}`;
|
|
22
39
|
const env = buildMinimalHostEnv(process.env, {
|
|
@@ -26,7 +43,15 @@ export async function runSandboxGit(target, args, options = {}) {
|
|
|
26
43
|
GIT_OPTIONAL_LOCKS: "0",
|
|
27
44
|
HOME: target.evidenceRoot,
|
|
28
45
|
USERPROFILE: target.evidenceRoot,
|
|
29
|
-
...options.env
|
|
46
|
+
...options.env,
|
|
47
|
+
// The sandbox project sits below other directories that may be Git repositories. Never let discovery walk
|
|
48
|
+
// above it, and (unless verifying) never rely on discovery at all. The pin is relative to the project working
|
|
49
|
+
// directory on purpose: Git for Windows rejects an absolute GIT_DIR of roughly 245 characters or more
|
|
50
|
+
// ("'$GIT_DIR' too big") although the repository itself works at those lengths.
|
|
51
|
+
GIT_CEILING_DIRECTORIES: path.dirname(target.projectRoot),
|
|
52
|
+
...(options.repositoryBinding === "discovered"
|
|
53
|
+
? {}
|
|
54
|
+
: { GIT_DIR: ".git", GIT_WORK_TREE: "." })
|
|
30
55
|
});
|
|
31
56
|
const result = await runMeasuredCommand({
|
|
32
57
|
commandId,
|
|
@@ -60,3 +85,68 @@ export function excerptGitFailure(result) {
|
|
|
60
85
|
const text = (result.stderr || result.stdout || result.error || "no output").trim().replace(/\s+/g, " ");
|
|
61
86
|
return text.length > 300 ? `${text.slice(0, 300)}...` : text;
|
|
62
87
|
}
|
|
88
|
+
function samePhysicalPath(left, right) {
|
|
89
|
+
const caseFold = process.platform === "win32" || process.platform === "darwin";
|
|
90
|
+
const a = path.normalize(left);
|
|
91
|
+
const b = path.normalize(right);
|
|
92
|
+
return caseFold ? a.toLowerCase() === b.toLowerCase() : a === b;
|
|
93
|
+
}
|
|
94
|
+
// The native variant resolves Windows 8.3 short names and reports the on-disk case, matching what Git prints.
|
|
95
|
+
const realpathNative = promisify(realpathCallback.native);
|
|
96
|
+
async function physical(candidate) {
|
|
97
|
+
return realpathNative(path.resolve(candidate));
|
|
98
|
+
}
|
|
99
|
+
/**
|
|
100
|
+
* Proves that Git, discovering from the sandbox project (bounded by a ceiling at its parent), resolves to the
|
|
101
|
+
* project's own physically contained `.git` directory and to the project as work tree. Exit status of
|
|
102
|
+
* `git init` is not evidence of this; on some hosts `init` succeeds while discovery selects an ancestor.
|
|
103
|
+
*/
|
|
104
|
+
export async function verifySandboxRepositoryBinding(target) {
|
|
105
|
+
const fail = (reason, unavailable = false) => ({ ok: false, unavailable, reason });
|
|
106
|
+
const gitEntry = path.join(target.projectRoot, ".git");
|
|
107
|
+
let stats;
|
|
108
|
+
try {
|
|
109
|
+
stats = await lstat(gitEntry);
|
|
110
|
+
}
|
|
111
|
+
catch {
|
|
112
|
+
return fail("the sandbox project has no .git entry after initialization.");
|
|
113
|
+
}
|
|
114
|
+
if (stats.isSymbolicLink() || !stats.isDirectory()) {
|
|
115
|
+
return fail("the sandbox .git entry is not a real local directory.");
|
|
116
|
+
}
|
|
117
|
+
let expectedRoot;
|
|
118
|
+
try {
|
|
119
|
+
expectedRoot = await physical(target.projectRoot);
|
|
120
|
+
if (!samePhysicalPath(await physical(gitEntry), path.join(expectedRoot, ".git"))) {
|
|
121
|
+
return fail("the sandbox .git directory is not physically contained in the sandbox project.");
|
|
122
|
+
}
|
|
123
|
+
}
|
|
124
|
+
catch (error) {
|
|
125
|
+
return fail(`the sandbox repository paths could not be resolved: ${error instanceof Error ? error.message : String(error)}`);
|
|
126
|
+
}
|
|
127
|
+
const result = await runSandboxGit(target, ["rev-parse", "--absolute-git-dir", "--show-toplevel"], {
|
|
128
|
+
label: "verify-root",
|
|
129
|
+
repositoryBinding: "discovered"
|
|
130
|
+
});
|
|
131
|
+
if (result.unavailable)
|
|
132
|
+
return fail("the git executable is not available.", true);
|
|
133
|
+
if (!result.ok) {
|
|
134
|
+
return fail(`git does not recognize the new sandbox repository from its project directory: ${excerptGitFailure(result)}`);
|
|
135
|
+
}
|
|
136
|
+
const lines = result.stdout.split(/\r?\n/).filter((line) => line.trim() !== "");
|
|
137
|
+
if (lines.length !== 2)
|
|
138
|
+
return fail("git repository discovery returned an unexpected shape.");
|
|
139
|
+
try {
|
|
140
|
+
const [discoveredGitDir, discoveredTopLevel] = await Promise.all([physical(lines[0]), physical(lines[1])]);
|
|
141
|
+
if (!samePhysicalPath(discoveredGitDir, path.join(expectedRoot, ".git"))) {
|
|
142
|
+
return fail("git discovered a repository administration directory other than the sandbox project's own .git.");
|
|
143
|
+
}
|
|
144
|
+
if (!samePhysicalPath(discoveredTopLevel, expectedRoot)) {
|
|
145
|
+
return fail("git discovered a work tree other than the sandbox project.");
|
|
146
|
+
}
|
|
147
|
+
}
|
|
148
|
+
catch (error) {
|
|
149
|
+
return fail(`the discovered repository paths could not be resolved: ${error instanceof Error ? error.message : String(error)}`);
|
|
150
|
+
}
|
|
151
|
+
return { ok: true };
|
|
152
|
+
}
|