akm-cli 0.9.8-beta.1 → 0.9.8-beta.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +92 -0
- package/dist/commands/health/data-dir-usage.js +165 -0
- package/dist/commands/health.js +19 -4
- package/dist/commands/workflow-cli.js +11 -61
- package/dist/core/config/config-io.js +21 -8
- package/dist/core/state/migrations.js +22 -0
- package/dist/core/warn.js +15 -0
- package/dist/indexer/indexer.js +42 -21
- package/dist/indexer/passes/dir-staleness.js +30 -9
- package/dist/scripts/akm-migrate-node.js +37 -4
- package/dist/scripts/akm-migrate.js +37 -4
- package/dist/storage/repositories/index-entries-repository.js +24 -11
- package/dist/storage/repositories/index-meta-repository.js +6 -4
- package/dist/storage/repositories/index-schema.js +16 -1
- package/dist/storage/repositories/proposals-repository.js +4 -1
- package/dist/tasks/resolve-akm-bin.js +15 -0
- package/dist/tasks/source/task-to-v3.js +10 -2
- package/docs/migration/v0.9.1-to-v0.9.2.md +18 -4
- package/docs/reference/cli.md +1 -1
- package/docs/reference/data-and-telemetry.md +1 -0
- package/docs/reference/tasks.md +11 -0
- package/package.json +1 -5
package/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,63 @@ All notable changes to this project will be documented in this file.
|
|
|
4
4
|
|
|
5
5
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
|
6
6
|
|
|
7
|
+
## [0.9.8-beta.2] - 2026-09-02
|
|
8
|
+
|
|
9
|
+
> **Adds state migration `026-proposals-strip-legacy-fragment-refs`.** The
|
|
10
|
+
> one-way caveat below applies to it as well: once this build opens
|
|
11
|
+
> `state.db`, 0.9.8-beta.1 and earlier refuse it with `unknown migration ID
|
|
12
|
+
> 026-proposals-strip-legacy-fragment-refs`.
|
|
13
|
+
|
|
14
|
+
### Added
|
|
15
|
+
|
|
16
|
+
- **`akm health` reports data-dir disk usage** (#896). A `data-dir-usage`
|
|
17
|
+
advisory sums the data directory with a stat-only walk and warns when it is
|
|
18
|
+
more than 3× the three live databases (state.db, index.db, logs.db) or when
|
|
19
|
+
one top-level subdirectory holds more than half of it, naming that
|
|
20
|
+
subdirectory with its size and share (for example `backups/ is 70G (94% of
|
|
21
|
+
data dir)`). The walk stops after 100,000 entries and says so. Silent when
|
|
22
|
+
nothing looks wrong.
|
|
23
|
+
|
|
24
|
+
### Fixed
|
|
25
|
+
|
|
26
|
+
- **`akm task sync` no longer spawns `npm root --global` on every call** (#901).
|
|
27
|
+
The npm-global-root probe behind `resolveAkmInvocation` is memoized for the
|
|
28
|
+
process, so a `task sync --rebind` cycle spawns npm at most once instead of
|
|
29
|
+
twice, and an installation that loops it every minute stops accumulating an
|
|
30
|
+
npm debug log per spawn.
|
|
31
|
+
- **A blocked v2 task now says how to convert it** (#902, #899). The
|
|
32
|
+
`argv-array-has-no-portable-shell-string` blocker printed by `akm migrate`
|
|
33
|
+
and the `TASK_SCHEMA_VERSION_UNSUPPORTED` read error now state that manual
|
|
34
|
+
conversion is required and name the rewrite (`command:` argv array →
|
|
35
|
+
`run:` string plus `shell:`). The full v2 → v4 field mapping is documented in
|
|
36
|
+
`docs/migration/v0.9.1-to-v0.9.2.md`.
|
|
37
|
+
- **Legacy `#fragment` proposal rows are repaired instead of warned about
|
|
38
|
+
forever** (#898). State migration 026 strips the retired export-fragment
|
|
39
|
+
selector from `proposals.ref` in place so the rows parse again, and an
|
|
40
|
+
unparseable proposal row now warns once per process instead of once per
|
|
41
|
+
read (`akm health --report` read the table seven times).
|
|
42
|
+
|
|
43
|
+
- **A no-op incremental `akm index` no longer costs minutes of CPU** (#900).
|
|
44
|
+
Two causes: the per-directory freshness check ran two full scans of the
|
|
45
|
+
`entries` table for every directory (O(directories × entries)), and every
|
|
46
|
+
file was read, hashed, and parsed before the freshness check decided the
|
|
47
|
+
directory was unchanged. The directory lookup now uses the existing
|
|
48
|
+
`file_path` index, and a stat-based gate over each directory's walked file
|
|
49
|
+
set skips unchanged directories before any file is read. On a synthetic
|
|
50
|
+
800-directory, 4,000-entry corpus a no-op pass fell from ~37 s to under 1 s
|
|
51
|
+
of CPU with identical entries and search results. The persisted directory
|
|
52
|
+
fingerprint now covers every walked file and `index_dir_state` gains a
|
|
53
|
+
`row_count` column; an existing index.db drains each directory once more
|
|
54
|
+
after upgrading, then takes the fast path.
|
|
55
|
+
|
|
56
|
+
- **Task-migration snapshots are capped at the five most recent** (#897).
|
|
57
|
+
`akm migrate apply` writes one snapshot directory per run under
|
|
58
|
+
`backups/task-v3` and `backups/task-v4` and never pruned them; each apply
|
|
59
|
+
now keeps the five newest and removes the rest, the same policy config
|
|
60
|
+
backups already use. Nothing in the current code writes the legacy
|
|
61
|
+
`backups/migrations`, `manual`, `releases`, or `operations` directories,
|
|
62
|
+
so they are left alone; the new health advisory is what surfaces them.
|
|
63
|
+
|
|
7
64
|
## [0.9.8-beta.1] - 2026-09-01
|
|
8
65
|
|
|
9
66
|
A cleanup and stabilization release: deletion of machinery that policed the
|
|
@@ -11,10 +68,45 @@ codebase's shape rather than its behaviour, and — because auditing for that
|
|
|
11
68
|
machinery meant reading the code closely — a run of real defects it had been
|
|
12
69
|
sitting on top of.
|
|
13
70
|
|
|
71
|
+
> **Upgrading is one-way for `state.db`.** This release adds migration
|
|
72
|
+
> `025-task-history-vocabulary-backfill`. Once any 0.9.8 command opens
|
|
73
|
+
> `state.db`, the ledger contains an ID that 0.9.7 does not know, and 0.9.7
|
|
74
|
+
> refuses to open it: `Refusing to open a database with a newer migration
|
|
75
|
+
> ledger: unknown migration ID 025-task-history-vocabulary-backfill`.
|
|
76
|
+
>
|
|
77
|
+
> The refusal is deliberate — an older binary must not write a database whose
|
|
78
|
+
> schema it cannot reason about — but the practical effect is that
|
|
79
|
+
> **downgrading to 0.9.7 requires restoring a `state.db` backup.** Commands
|
|
80
|
+
> that only read the derived index (`akm info`, `akm search`) keep working on
|
|
81
|
+
> 0.9.7; everything that touches `state.db` (`akm health`, `akm task`,
|
|
82
|
+
> `akm improve`, proposals) does not.
|
|
83
|
+
>
|
|
84
|
+
> Snapshot `state.db` before upgrading if you may need to go back:
|
|
85
|
+
>
|
|
86
|
+
> ```sh
|
|
87
|
+
> akm info --format json # confirm your data dir
|
|
88
|
+
> sqlite3 "$DATA_DIR/state.db" "VACUUM INTO '''state.db.pre-0.9.8.bak'''"
|
|
89
|
+
> ```
|
|
90
|
+
|
|
14
91
|
Two security holes, two search-correctness bugs, a locale-dependent hash, a
|
|
15
92
|
deletion shield that failed open, and sixteen places that answered a failure
|
|
16
93
|
with a confident wrong answer instead of an error.
|
|
17
94
|
|
|
95
|
+
### Changed
|
|
96
|
+
|
|
97
|
+
- **`akm workflow plan` returns JSON by default**, like every other command
|
|
98
|
+
(#903). It was the one verb whose unmarked default was a human summary. That
|
|
99
|
+
exception cost a bespoke branch which could not reliably distinguish "no
|
|
100
|
+
format named" from "`--format json` named globally before the subcommand" —
|
|
101
|
+
citty parses each command level against its own argv, so the leaf read
|
|
102
|
+
`undefined` in both cases. Working around that meant reading the invocation
|
|
103
|
+
singleton, folding in a persisted `output.format`, and finally leaving a
|
|
104
|
+
resolved `"json"` on the text branch because it was indistinguishable from
|
|
105
|
+
"nothing configured" — which meant an explicit `--format json` silently did
|
|
106
|
+
nothing for anyone whose config already resolved to json. Deleted, along with
|
|
107
|
+
~60 lines of comment justifying it. `--format text` still renders the same
|
|
108
|
+
summary through the same formatter; it is simply no longer the default.
|
|
109
|
+
|
|
18
110
|
### Fixed
|
|
19
111
|
|
|
20
112
|
- **Historical state migrations are reachable where akm cannot reinstall
|
|
@@ -0,0 +1,165 @@
|
|
|
1
|
+
// This Source Code Form is subject to the terms of the Mozilla Public
|
|
2
|
+
// License, v. 2.0. If a copy of the MPL was not distributed with this
|
|
3
|
+
// file, You can obtain one at https://mozilla.org/MPL/2.0/.
|
|
4
|
+
/**
|
|
5
|
+
* `data-dir-usage` advisory for `akm health` (#896).
|
|
6
|
+
*
|
|
7
|
+
* A real environment's `$XDG_DATA_HOME/akm` grew to 74 GB with none of the
|
|
8
|
+
* ~20 health checks saying a word about disk. 70 of the 74 GB turned out to
|
|
9
|
+
* be `backups/` (unpruned migration snapshots, see #897); the live working
|
|
10
|
+
* set (state.db + index.db + logs.db) was ~4.2 GB. Naming the largest
|
|
11
|
+
* top-level contributor and its share of the total is most of the value —
|
|
12
|
+
* `backups/ is 70G (94% of data dir)` is self-diagnosing where "akm health
|
|
13
|
+
* says nothing" is not.
|
|
14
|
+
*
|
|
15
|
+
* Best-effort and read-only: a plain recursive `fs` stat walk over the data
|
|
16
|
+
* dir, no `du` shell-out. Silent (returns `undefined`) whenever nothing
|
|
17
|
+
* looks wrong, matching the stash-exposure/type-directory-check house
|
|
18
|
+
* pattern — this is not a "always show a pass line" check.
|
|
19
|
+
*/
|
|
20
|
+
import fs from "node:fs";
|
|
21
|
+
import path from "node:path";
|
|
22
|
+
/**
|
|
23
|
+
* Warn when the data dir's total size is more than this many times the
|
|
24
|
+
* combined size of the three live databases (state.db + index.db +
|
|
25
|
+
* logs.db). Chosen so a healthy install (backups roughly comparable to the
|
|
26
|
+
* live working set) stays quiet, while an order-of-magnitude blowup like
|
|
27
|
+
* the 74 GB/4.2 GB (~17x) incident trips it.
|
|
28
|
+
*/
|
|
29
|
+
const DATA_DIR_BLOAT_RATIO_THRESHOLD = 3;
|
|
30
|
+
/**
|
|
31
|
+
* Warn when a single top-level subdirectory accounts for more than this
|
|
32
|
+
* percentage of the data dir's total size — the "one thing ate the disk"
|
|
33
|
+
* signal (94% for `backups/` in the incident).
|
|
34
|
+
*/
|
|
35
|
+
const DOMINANT_SUBDIR_PERCENT_THRESHOLD = 50;
|
|
36
|
+
/**
|
|
37
|
+
* Cap on the number of filesystem entries the recursive size walk will
|
|
38
|
+
* `stat`. A 70 GB tree of a few thousand backup copies is cheap to walk
|
|
39
|
+
* (stat-only), but a data dir polluted with hundreds of thousands of small
|
|
40
|
+
* files (task logs, npm logs) must not make `akm health` slow. Past this
|
|
41
|
+
* cap the walk stops descending further and the advisory says its size
|
|
42
|
+
* figures are a lower bound.
|
|
43
|
+
*/
|
|
44
|
+
const MAX_WALK_ENTRIES = 100_000;
|
|
45
|
+
const LIVE_DB_FILES = ["state.db", "index.db", "logs.db"];
|
|
46
|
+
/**
|
|
47
|
+
* Recursively sum file sizes under `root` (stat-only, symlinks not
|
|
48
|
+
* followed so a cyclic or huge-target symlink can't blow up the walk).
|
|
49
|
+
* `budget` is a shared mutable counter across the whole tree so the
|
|
50
|
+
* `MAX_WALK_ENTRIES` cap applies to the walk as a whole, not per-branch.
|
|
51
|
+
*/
|
|
52
|
+
function sizeOfPath(root, budget) {
|
|
53
|
+
let stat;
|
|
54
|
+
try {
|
|
55
|
+
stat = fs.lstatSync(root);
|
|
56
|
+
}
|
|
57
|
+
catch {
|
|
58
|
+
return { bytes: 0, truncated: false };
|
|
59
|
+
}
|
|
60
|
+
if (stat.isSymbolicLink())
|
|
61
|
+
return { bytes: 0, truncated: false };
|
|
62
|
+
if (!stat.isDirectory())
|
|
63
|
+
return { bytes: stat.size, truncated: false };
|
|
64
|
+
let entries;
|
|
65
|
+
try {
|
|
66
|
+
entries = fs.readdirSync(root, { withFileTypes: true });
|
|
67
|
+
}
|
|
68
|
+
catch {
|
|
69
|
+
return { bytes: 0, truncated: false };
|
|
70
|
+
}
|
|
71
|
+
let bytes = 0;
|
|
72
|
+
let truncated = false;
|
|
73
|
+
for (const entry of entries) {
|
|
74
|
+
if (budget.remaining <= 0) {
|
|
75
|
+
truncated = true;
|
|
76
|
+
break;
|
|
77
|
+
}
|
|
78
|
+
budget.remaining--;
|
|
79
|
+
const sub = sizeOfPath(path.join(root, entry.name), budget);
|
|
80
|
+
bytes += sub.bytes;
|
|
81
|
+
if (sub.truncated)
|
|
82
|
+
truncated = true;
|
|
83
|
+
}
|
|
84
|
+
return { bytes, truncated };
|
|
85
|
+
}
|
|
86
|
+
/** `1610612736` -> `"1.5G"`. Values under 10 in a unit keep one decimal; 10+ round to an integer. */
|
|
87
|
+
function formatBytes(bytes) {
|
|
88
|
+
const units = ["B", "K", "M", "G", "T"];
|
|
89
|
+
let value = bytes;
|
|
90
|
+
let unit = 0;
|
|
91
|
+
while (value >= 1024 && unit < units.length - 1) {
|
|
92
|
+
value /= 1024;
|
|
93
|
+
unit++;
|
|
94
|
+
}
|
|
95
|
+
const rendered = unit === 0 ? String(Math.round(value)) : value < 10 ? value.toFixed(1) : String(Math.round(value));
|
|
96
|
+
return `${rendered}${units[unit]}`;
|
|
97
|
+
}
|
|
98
|
+
/**
|
|
99
|
+
* Build the `data-dir-usage` advisory, or `undefined` when the data dir is
|
|
100
|
+
* missing/empty/unreadable or its size looks unremarkable (neither
|
|
101
|
+
* threshold trips). `dataDir` is the caller-resolved `getDataDir()` path —
|
|
102
|
+
* this module never resolves paths or reads env itself.
|
|
103
|
+
*/
|
|
104
|
+
export function collectDataDirUsageAdvisory(dataDir) {
|
|
105
|
+
let topEntries;
|
|
106
|
+
try {
|
|
107
|
+
topEntries = fs.readdirSync(dataDir, { withFileTypes: true });
|
|
108
|
+
}
|
|
109
|
+
catch {
|
|
110
|
+
return undefined; // no data dir yet — nothing to report.
|
|
111
|
+
}
|
|
112
|
+
const budget = { remaining: MAX_WALK_ENTRIES };
|
|
113
|
+
let totalBytes = 0;
|
|
114
|
+
let truncated = false;
|
|
115
|
+
const sizes = new Map();
|
|
116
|
+
for (const entry of topEntries) {
|
|
117
|
+
const size = sizeOfPath(path.join(dataDir, entry.name), budget);
|
|
118
|
+
totalBytes += size.bytes;
|
|
119
|
+
if (size.truncated)
|
|
120
|
+
truncated = true;
|
|
121
|
+
sizes.set(entry.name, { bytes: size.bytes, isDirectory: entry.isDirectory() });
|
|
122
|
+
}
|
|
123
|
+
if (totalBytes === 0)
|
|
124
|
+
return undefined;
|
|
125
|
+
const subdirs = [...sizes]
|
|
126
|
+
.filter(([, size]) => size.isDirectory)
|
|
127
|
+
.map(([name, size]) => ({ name, bytes: size.bytes, percent: (size.bytes / totalBytes) * 100 }))
|
|
128
|
+
.sort((a, b) => b.bytes - a.bytes);
|
|
129
|
+
const largest = subdirs[0];
|
|
130
|
+
const liveDbBreakdown = Object.fromEntries(LIVE_DB_FILES.map((f) => [f, sizes.get(f)?.bytes ?? 0]));
|
|
131
|
+
const liveDbBytes = Object.values(liveDbBreakdown).reduce((a, b) => a + b, 0);
|
|
132
|
+
const ratio = liveDbBytes > 0 ? totalBytes / liveDbBytes : undefined;
|
|
133
|
+
const bloatWarn = ratio !== undefined && ratio > DATA_DIR_BLOAT_RATIO_THRESHOLD;
|
|
134
|
+
const dominantWarn = largest !== undefined && largest.percent > DOMINANT_SUBDIR_PERCENT_THRESHOLD;
|
|
135
|
+
if (!bloatWarn && !dominantWarn)
|
|
136
|
+
return undefined;
|
|
137
|
+
const parts = [`data dir is ${formatBytes(totalBytes)} at ${dataDir}`];
|
|
138
|
+
if (largest) {
|
|
139
|
+
parts.push(`${largest.name}/ is ${formatBytes(largest.bytes)} (${Math.round(largest.percent)}% of data dir)`);
|
|
140
|
+
}
|
|
141
|
+
if (ratio !== undefined) {
|
|
142
|
+
parts.push(`live databases (${LIVE_DB_FILES.join("+")}) total ${formatBytes(liveDbBytes)}, ~${ratio.toFixed(1)}x smaller`);
|
|
143
|
+
}
|
|
144
|
+
if (truncated) {
|
|
145
|
+
parts.push(`size figures are a lower bound — the walk stopped after ${MAX_WALK_ENTRIES} entries`);
|
|
146
|
+
}
|
|
147
|
+
const message = `${parts.join("; ")}.`;
|
|
148
|
+
return {
|
|
149
|
+
name: "data-dir-usage",
|
|
150
|
+
kind: "deterministic",
|
|
151
|
+
status: "warn",
|
|
152
|
+
confidence: "medium",
|
|
153
|
+
message,
|
|
154
|
+
evidence: {
|
|
155
|
+
dataDir,
|
|
156
|
+
totalBytes,
|
|
157
|
+
liveDbBytes,
|
|
158
|
+
liveDbBreakdown,
|
|
159
|
+
largestSubdir: largest,
|
|
160
|
+
ratio,
|
|
161
|
+
walkBounded: truncated,
|
|
162
|
+
maxWalkEntries: MAX_WALK_ENTRIES,
|
|
163
|
+
},
|
|
164
|
+
};
|
|
165
|
+
}
|
package/dist/commands/health.js
CHANGED
|
@@ -9,7 +9,7 @@ import { ConfigError, UsageError } from "../core/errors.js";
|
|
|
9
9
|
import { readEvents } from "../core/events.js";
|
|
10
10
|
import { openLogsDatabase } from "../core/logs-db.js";
|
|
11
11
|
import { classifyPathAccess, describeInaccessiblePath } from "../core/path-access.js";
|
|
12
|
-
import { getConfigPath, getDbPath, getStateDbPathInDataDir } from "../core/paths.js";
|
|
12
|
+
import { getConfigPath, getDataDir, getDbPath, getStateDbPathInDataDir } from "../core/paths.js";
|
|
13
13
|
import { listExistingTableNames, openStateDatabase } from "../core/state-db.js";
|
|
14
14
|
import { DURATION_UNITS, parseDuration, parseSinceToIso } from "../core/time.js";
|
|
15
15
|
import { closeDatabase, openReadonlyExistingDatabase } from "../storage/repositories/index-connection.js";
|
|
@@ -18,6 +18,7 @@ import { queryTaskHistory } from "../storage/repositories/task-history-repositor
|
|
|
18
18
|
import { pkgVersion } from "../version.js";
|
|
19
19
|
import { collectImproveAdvisories } from "./health/advisories.js";
|
|
20
20
|
import { HEALTH_CHECKS, runHealthEngineProbes } from "./health/checks.js";
|
|
21
|
+
import { collectDataDirUsageAdvisory } from "./health/data-dir-usage.js";
|
|
21
22
|
import { buildImproveSkipSummary, computeWallTimeStats, isAgentTaskHistoryRow, roundRate, summarizeImproveCompleted, summarizeImproveRuns, taskFailureDetail, } from "./health/improve-metrics.js";
|
|
22
23
|
import { emptyLlmUsageAggregate, readLlmUsageAggregate } from "./health/llm-usage.js";
|
|
23
24
|
import { computeDegradationMetrics, computeDenominatorFixedCoverage, computeEnrichmentMintingRollup, probeStateDbRoundTrip, } from "./health/metrics.js";
|
|
@@ -197,9 +198,10 @@ function gatherImproveSummaryPhase(db, stateDbPath, since, now) {
|
|
|
197
198
|
return { improveSummary, perRunSummaries };
|
|
198
199
|
}
|
|
199
200
|
/**
|
|
200
|
-
* The
|
|
201
|
-
*
|
|
202
|
-
* (binary-config-skew, egress-endpoints),
|
|
201
|
+
* The best-effort advisory groups beyond the health-check registry: improve
|
|
202
|
+
* advisories, the `stash-git-exposure` probe, the 08 surfaces group
|
|
203
|
+
* (binary-config-skew, egress-endpoints), `type-directory-disagreement`
|
|
204
|
+
* (#831), `data-dir-usage` (#896), and `plugin-version` (itlackey/akm#832).
|
|
203
205
|
* Order matches emission order in the returned array. A probe/filesystem
|
|
204
206
|
* failure in any try/catch must not abort the health report — each group
|
|
205
207
|
* degrades to "no advisory" independently.
|
|
@@ -251,6 +253,19 @@ function gatherAncillaryAdvisories(db, stateDbPath, since, improveSummary, optio
|
|
|
251
253
|
catch {
|
|
252
254
|
// Non-fatal.
|
|
253
255
|
}
|
|
256
|
+
// #896: report the data dir's total size and its largest top-level
|
|
257
|
+
// subdirectory, so a disk-usage blowup (e.g. unpruned migration snapshot
|
|
258
|
+
// backups, #897) is self-diagnosing instead of requiring `du` archaeology.
|
|
259
|
+
// Best-effort — an unreadable/missing data dir must not abort the health
|
|
260
|
+
// report.
|
|
261
|
+
try {
|
|
262
|
+
const dataDirUsage = collectDataDirUsageAdvisory(getDataDir());
|
|
263
|
+
if (dataDirUsage)
|
|
264
|
+
advisories.push(dataDirUsage);
|
|
265
|
+
}
|
|
266
|
+
catch {
|
|
267
|
+
// Non-fatal.
|
|
268
|
+
}
|
|
254
269
|
// itlackey/akm#832: report installed Claude Code harness plugin version(s)
|
|
255
270
|
// and warn when stale or when the plugin's own akm-cli version range no
|
|
256
271
|
// longer admits this CLI. Best-effort — no plugin installed, an unreadable
|
|
@@ -7,18 +7,12 @@
|
|
|
7
7
|
* `create --print` emits Markdown; execution accepts peer `.md` and
|
|
8
8
|
* GitHub-shaped `.yml` workflow sources. Validate with `akm lint --type workflows`.
|
|
9
9
|
*/
|
|
10
|
-
import { getParsedInvocation } from "../cli/invocation.js";
|
|
11
10
|
import { getStringArg } from "../cli/parse-args.js";
|
|
12
11
|
import { defineGroupCommand, defineJsonCommand, EXIT_CODES, output } from "../cli/shared.js";
|
|
13
12
|
import { armAbortDeadline } from "../core/abort-deadline.js";
|
|
14
13
|
import { assertFlatAssetName, combineCreatePath, normalizeCreateSubPath } from "../core/asset/asset-create.js";
|
|
15
14
|
import { NotFoundError, UsageError } from "../core/errors.js";
|
|
16
15
|
import { akmIndex } from "../indexer/indexer.js";
|
|
17
|
-
import { getOutputMode } from "../output/context.js";
|
|
18
|
-
import { renderGenericText } from "../output/generic-render.js";
|
|
19
|
-
import { deliverRendered } from "../output/html-render.js";
|
|
20
|
-
import { shapeForCommand } from "../output/shapes.js";
|
|
21
|
-
import { formatPlain } from "../output/text.js";
|
|
22
16
|
import { assertWorkflowMarkdownName, createWorkflowAsset, getWorkflowTemplate } from "../workflows/authoring/authoring.js";
|
|
23
17
|
import { WORKFLOW_MAX_TIMEOUT_MS } from "../workflows/ir/schema.js";
|
|
24
18
|
import { abandonWorkflowRun, getWorkflowStatus, hasWorkflowRun, listWorkflowRuns, resumeWorkflowRun, } from "../workflows/runtime/runs.js";
|
|
@@ -319,61 +313,17 @@ const workflowPlanCommand = defineJsonCommand({
|
|
|
319
313
|
},
|
|
320
314
|
async run({ args }) {
|
|
321
315
|
const result = await akmWorkflowPlan(args.ref);
|
|
322
|
-
//
|
|
323
|
-
//
|
|
324
|
-
//
|
|
325
|
-
//
|
|
326
|
-
//
|
|
327
|
-
//
|
|
328
|
-
//
|
|
329
|
-
//
|
|
330
|
-
//
|
|
331
|
-
//
|
|
332
|
-
//
|
|
333
|
-
// (e.g. `akm --format json workflow plan <ref>`) is consumed by the ROOT
|
|
334
|
-
// command's own declared `format` arg before the `workflow`/`plan`
|
|
335
|
-
// subcommand tokens are even resolved — this LEAF's `args.format` reads
|
|
336
|
-
// `undefined` in exactly that case too, indistinguishable from "no
|
|
337
|
-
// format was named anywhere". Reproduced live: that invocation printed
|
|
338
|
-
// the human TEXT summary at exit 0 even though `getOutputMode().format`
|
|
339
|
-
// was already `"json"` (the control, `akm --format json workflow list`,
|
|
340
|
-
// correctly emitted JSON — only this leaf's own arg-read was wrong).
|
|
341
|
-
// Detect it instead off the process-wide invocation singleton
|
|
342
|
-
// (`getParsedInvocation`, src/cli/invocation.ts) — the same canonical,
|
|
343
|
-
// position-independent argv parse `src/cli.ts` mints ONCE at startup
|
|
344
|
-
// (`setParsedInvocation`, immediately before `initOutputMode` builds the
|
|
345
|
-
// `getOutputMode()` singleton from that identical argv), so this agrees
|
|
346
|
-
// with `getOutputMode()` regardless of where `--format` appeared. A bare
|
|
347
|
-
// `process.argv` read is reserved for `src/cli.ts`/`cli/invocation.ts`
|
|
348
|
-
// themselves (`lint-process-argv.ts`); every other module reads through
|
|
349
|
-
// this singleton instead. When explicit, this defers to the normal
|
|
350
|
-
// `output()` path (json/yaml/text/md/html/jsonl, `--output <path>`)
|
|
351
|
-
// unchanged; when absent, it reproduces `output()`'s OWN "text" branch
|
|
352
|
-
// verbatim (same shape/detail projection, same registered-formatter-or-
|
|
353
|
-
// generic-fallback, same `--output <path>` handling) without touching
|
|
354
|
-
// the shared dispatcher other commands rely on.
|
|
355
|
-
//
|
|
356
|
-
// "No format named anywhere" also has to check the RESOLVED mode, not
|
|
357
|
-
// just argv: `getOutputMode().format` already folds a persisted
|
|
358
|
-
// `output.format` config default in ahead of the hardcoded "json"
|
|
359
|
-
// fallback (`resolveOutputMode`, src/output/context.ts — argv ?? config
|
|
360
|
-
// default ?? "json"). A user who has configured e.g. `output.format:
|
|
361
|
-
// "yaml"` gets yaml from every other command with no `--format` on the
|
|
362
|
-
// line; `workflow plan` must honor that too instead of forcing its
|
|
363
|
-
// human-text branch over a real persisted default. A resolved format of
|
|
364
|
-
// exactly "json" is deliberately left on the text branch below: it's
|
|
365
|
-
// indistinguishable from "nothing configured" (DEFAULT_CONFIG.output.format
|
|
366
|
-
// is also "json" — OutputConfigSchema's `format` carries no independent
|
|
367
|
-
// zod default, so the merge is the only source), and collapsing that case
|
|
368
|
-
// onto the JSON envelope would erase the documented unmarked-default text
|
|
369
|
-
// summary for the overwhelmingly common "user configured nothing" case.
|
|
370
|
-
const mode = getOutputMode();
|
|
371
|
-
if (getParsedInvocation().getFlagValue("--format") === undefined && mode.format === "json") {
|
|
372
|
-
const shaped = shapeForCommand("workflow-plan", result, mode.detail, mode.shape);
|
|
373
|
-
const plain = formatPlain("workflow-plan", shaped, mode.detail);
|
|
374
|
-
deliverRendered(plain ?? renderGenericText("workflow-plan", shaped), mode.outputPath);
|
|
375
|
-
return;
|
|
376
|
-
}
|
|
316
|
+
// json-by-default, like every other verb (#903). This used to default to
|
|
317
|
+
// the human summary, which cost ~60 lines of branch: `args.format` cannot
|
|
318
|
+
// detect "no format named" (citty parses per level, so a global
|
|
319
|
+
// pre-subcommand `--format json` is eaten by the ROOT command and the leaf
|
|
320
|
+
// reads undefined), so it had to route through `getParsedInvocation()`,
|
|
321
|
+
// then fold in a persisted `output.format`, and then still leave a
|
|
322
|
+
// resolved "json" on the text branch because that is indistinguishable
|
|
323
|
+
// from "nothing configured". The last compromise meant an explicit
|
|
324
|
+
// `--format json` silently did nothing for anyone whose config already
|
|
325
|
+
// resolved to json. `--format text` still renders the summary through the
|
|
326
|
+
// registered formatter; it is just no longer the unmarked default.
|
|
377
327
|
output("workflow-plan", result);
|
|
378
328
|
},
|
|
379
329
|
});
|
|
@@ -115,17 +115,30 @@ export function backupExistingConfig(configPath, now = new Date()) {
|
|
|
115
115
|
return { timestamped, latest };
|
|
116
116
|
}
|
|
117
117
|
function pruneOldBackups(backupDir) {
|
|
118
|
+
pruneToNewest(backupDir, MAX_CONFIG_BACKUPS, (entry) => entry.isFile() && isTimestampedConfigBackup(entry.name));
|
|
119
|
+
}
|
|
120
|
+
function isTimestampedConfigBackup(name) {
|
|
121
|
+
return name.startsWith("config-") && name.endsWith(".json") && name !== "config.latest.json";
|
|
122
|
+
}
|
|
123
|
+
/**
|
|
124
|
+
* Keep the `keep` most-recently-modified entries of `dir` that `select`
|
|
125
|
+
* admits and remove the rest. Best-effort: an unreadable dir is a no-op, an
|
|
126
|
+
* unreadable entry sorts oldest (pruned first), and a failed removal is
|
|
127
|
+
* retried by the next call. Shared by the config backups above and the task
|
|
128
|
+
* migration snapshots (#897).
|
|
129
|
+
*/
|
|
130
|
+
export function pruneToNewest(dir, keep, select) {
|
|
118
131
|
let entries;
|
|
119
132
|
try {
|
|
120
|
-
entries = fs.readdirSync(
|
|
133
|
+
entries = fs.readdirSync(dir, { withFileTypes: true });
|
|
121
134
|
}
|
|
122
135
|
catch {
|
|
123
136
|
return;
|
|
124
137
|
}
|
|
125
|
-
const
|
|
126
|
-
.filter(
|
|
127
|
-
.map((
|
|
128
|
-
const full = path.join(
|
|
138
|
+
const candidates = entries
|
|
139
|
+
.filter(select)
|
|
140
|
+
.map((entry) => {
|
|
141
|
+
const full = path.join(dir, entry.name);
|
|
129
142
|
let mtime = 0;
|
|
130
143
|
try {
|
|
131
144
|
mtime = fs.statSync(full).mtimeMs;
|
|
@@ -136,12 +149,12 @@ function pruneOldBackups(backupDir) {
|
|
|
136
149
|
return { path: full, mtime };
|
|
137
150
|
})
|
|
138
151
|
.sort((a, b) => b.mtime - a.mtime);
|
|
139
|
-
for (const stale of
|
|
152
|
+
for (const stale of candidates.slice(keep)) {
|
|
140
153
|
try {
|
|
141
|
-
fs.
|
|
154
|
+
fs.rmSync(stale.path, { recursive: true, force: true });
|
|
142
155
|
}
|
|
143
156
|
catch {
|
|
144
|
-
// Best-effort prune; next
|
|
157
|
+
// Best-effort prune; the next call will retry.
|
|
145
158
|
}
|
|
146
159
|
}
|
|
147
160
|
}
|
|
@@ -36,6 +36,7 @@ export const STATE_MIGRATION_SAFETY_BY_ID = Object.freeze({
|
|
|
36
36
|
"023-child-workflow-runs": "additive",
|
|
37
37
|
"024-workflow-run-outputs": "additive",
|
|
38
38
|
"025-task-history-vocabulary-backfill": "data-preserving-rebuild",
|
|
39
|
+
"026-proposals-strip-legacy-fragment-refs": "data-preserving-rebuild",
|
|
39
40
|
});
|
|
40
41
|
export const STATE_MIGRATIONS = [
|
|
41
42
|
// ── Migration 001 — initial schema ──────────────────────────────────────────
|
|
@@ -1127,6 +1128,27 @@ export const STATE_MIGRATIONS = [
|
|
|
1127
1128
|
AND json_extract(metadata_json, '$.targetVocab') IS NULL;
|
|
1128
1129
|
`,
|
|
1129
1130
|
},
|
|
1131
|
+
// ── Migration 026 — strip the retired proposal-ref export fragment (#898) ──
|
|
1132
|
+
//
|
|
1133
|
+
// Older releases could write a proposal `ref` carrying the export-fragment
|
|
1134
|
+
// selector (`[bundle//]conceptId#fragment`, see src/core/asset/asset-ref.ts).
|
|
1135
|
+
// `currentProposalRef` (proposals-repository.ts) now rejects any fragment, so
|
|
1136
|
+
// those rows failed to parse on every read and could not be listed, repaired,
|
|
1137
|
+
// or deleted through the CLI. Every surface that still reads an archived row
|
|
1138
|
+
// is concept-scoped, so the fragment carries nothing they use.
|
|
1139
|
+
//
|
|
1140
|
+
// `#` is legal in a ref only as the fragment separator, so truncating at the
|
|
1141
|
+
// first `#` is exactly the split `parseBundleRef` performs. `ref` has no
|
|
1142
|
+
// UNIQUE constraint, and no writer ever produced an empty concept id, so
|
|
1143
|
+
// every legacy row normalizes without a drop path.
|
|
1144
|
+
{
|
|
1145
|
+
id: "026-proposals-strip-legacy-fragment-refs",
|
|
1146
|
+
up: `
|
|
1147
|
+
UPDATE proposals
|
|
1148
|
+
SET ref = substr(ref, 1, instr(ref, '#') - 1)
|
|
1149
|
+
WHERE ref LIKE '%#%';
|
|
1150
|
+
`,
|
|
1151
|
+
},
|
|
1130
1152
|
];
|
|
1131
1153
|
assertMigrationRegistry(STATE_MIGRATIONS);
|
|
1132
1154
|
function assertStateMigrationSafetyRegistry() {
|
package/dist/core/warn.js
CHANGED
|
@@ -122,6 +122,21 @@ export function warn(...args) {
|
|
|
122
122
|
console.warn(...args);
|
|
123
123
|
}
|
|
124
124
|
}
|
|
125
|
+
const warnedOnceKeys = new Set();
|
|
126
|
+
/**
|
|
127
|
+
* `warn()` at most once per process for a given `key`. For diagnostics that
|
|
128
|
+
* would otherwise repeat on every read of the same bad row or value.
|
|
129
|
+
*/
|
|
130
|
+
export function warnOnce(key, ...args) {
|
|
131
|
+
if (warnedOnceKeys.has(key))
|
|
132
|
+
return;
|
|
133
|
+
warnedOnceKeys.add(key);
|
|
134
|
+
warn(...args);
|
|
135
|
+
}
|
|
136
|
+
/** TEST-ONLY. Forget every `warnOnce` key so a test can re-trigger a warning. */
|
|
137
|
+
export function _resetWarnOnceForTests() {
|
|
138
|
+
warnedOnceKeys.clear();
|
|
139
|
+
}
|
|
125
140
|
/**
|
|
126
141
|
* Emit an error to stderr unless --quiet is active.
|
|
127
142
|
* Always written to the log file if one is active.
|
package/dist/indexer/indexer.js
CHANGED
|
@@ -30,7 +30,7 @@ import { deleteStoredGraph } from "./db/graph-db.js";
|
|
|
30
30
|
import { deriveEntryProvenance, deriveInstallations } from "./installations.js";
|
|
31
31
|
import { indexedPathMatchesOwner, resolveAdapterConceptOwner, } from "./lookup/adapter-concept-owner.js";
|
|
32
32
|
import { generateEmbeddingsForDb } from "./materialize-embeddings.js";
|
|
33
|
-
import { canUseIncrementalSkip, computeDirFingerprint,
|
|
33
|
+
import { canUseIncrementalSkip, computeDirFingerprint, getCachedDirState, getDirIndexState, inferZeroRowReason, } from "./passes/dir-staleness.js";
|
|
34
34
|
import { isEnrichmentComplete, isWorkflowSkipWarning } from "./passes/metadata.js";
|
|
35
35
|
import { drainDirDocuments } from "./scan/drain-dir.js";
|
|
36
36
|
import { buildSearchText } from "./search/search-fields.js";
|
|
@@ -360,6 +360,19 @@ export function _setIndexTransactionHookForTests(hook) {
|
|
|
360
360
|
function indexTransactionHook(point) {
|
|
361
361
|
indexTransactionHookForTests?.(point);
|
|
362
362
|
}
|
|
363
|
+
let drainObserverForTests;
|
|
364
|
+
/**
|
|
365
|
+
* TEST-ONLY. Observe every directory that actually reaches
|
|
366
|
+
* `drainDirDocuments` — the per-file read/sha256-hash/frontmatter-parse step
|
|
367
|
+
* (#900) — with the directory path and its walked file count. `undefined`
|
|
368
|
+
* restores. A directory the pre-drain gate (`getCachedDirState`) skips never
|
|
369
|
+
* fires this observer, so it is the
|
|
370
|
+
* seam #900's own tests use to assert an unchanged directory's files are
|
|
371
|
+
* never read on a no-op incremental run.
|
|
372
|
+
*/
|
|
373
|
+
export function _setDrainObserverForTests(observer) {
|
|
374
|
+
drainObserverForTests = observer;
|
|
375
|
+
}
|
|
363
376
|
/**
|
|
364
377
|
* Detect an adapter for every resolvable source that does not declare one, and
|
|
365
378
|
* persist each detection into `config.json`.
|
|
@@ -883,17 +896,19 @@ async function scanSourceDirs(db, allSourceEntries, isIncremental, builtAtMs, ha
|
|
|
883
896
|
// Incremental freshness gate shared by both branches: consult the persisted
|
|
884
897
|
// dir state and record either a skip (unchanged + eligible for incremental
|
|
885
898
|
// skip) or a scan record carrying the candidate stash.
|
|
886
|
-
const recordFreshnessDecision = (dirPath, currentStashDir, stateFiles, stash, hashByFile, conceptIdByFile, indexVariant, forceScan, pruneMissing) => {
|
|
887
|
-
const previousState = getDirIndexState(db, dirPath, stateFiles, builtAtMs, indexVariant);
|
|
899
|
+
const recordFreshnessDecision = (dirPath, currentStashDir, stateFiles, fingerprint, stash, hashByFile, conceptIdByFile, indexVariant, forceScan, pruneMissing) => {
|
|
900
|
+
const previousState = getDirIndexState(db, dirPath, stateFiles, builtAtMs, indexVariant, fingerprint);
|
|
888
901
|
if (isIncremental && !forceScan && !previousState.stale && canUseIncrementalSkip(previousState, priorDirsChanged)) {
|
|
889
902
|
skippedDirs++;
|
|
890
903
|
dirRecords.push({
|
|
891
904
|
dirPath,
|
|
892
905
|
currentStashDir,
|
|
893
906
|
files: stateFiles,
|
|
907
|
+
fingerprint,
|
|
894
908
|
stash: null,
|
|
895
909
|
skip: true,
|
|
896
910
|
reason: previousState.reason,
|
|
911
|
+
persistedRowCount: previousState.persistedRowCount,
|
|
897
912
|
indexVariant,
|
|
898
913
|
});
|
|
899
914
|
reportDirDecision("skip", dirPath, currentStashDir, previousState.reason, previousState.persistedRowCount);
|
|
@@ -906,6 +921,7 @@ async function scanSourceDirs(db, allSourceEntries, isIncremental, builtAtMs, ha
|
|
|
906
921
|
dirPath,
|
|
907
922
|
currentStashDir,
|
|
908
923
|
files: stateFiles,
|
|
924
|
+
fingerprint,
|
|
909
925
|
stash,
|
|
910
926
|
skip: false,
|
|
911
927
|
reason,
|
|
@@ -959,10 +975,13 @@ async function scanSourceDirs(db, allSourceEntries, isIncremental, builtAtMs, ha
|
|
|
959
975
|
reportDirDecision("skip", dirPath, currentStashDir, reason);
|
|
960
976
|
continue;
|
|
961
977
|
}
|
|
962
|
-
|
|
978
|
+
// #900: decide from stat data alone whether the directory can be skipped,
|
|
979
|
+
// before drainDirDocuments reads, hashes, and parses every file.
|
|
980
|
+
const fingerprint = computeDirFingerprint(dirPath, indexableFiles, indexVariant);
|
|
981
|
+
const cachedState = isIncremental &&
|
|
963
982
|
!forceScan &&
|
|
964
|
-
|
|
965
|
-
if (
|
|
983
|
+
getCachedDirState(db, dirPath, indexableFiles, builtAtMs, priorDirsChanged, indexVariant, fingerprint);
|
|
984
|
+
if (cachedState) {
|
|
966
985
|
skippedDirs++;
|
|
967
986
|
dirRecords.push({
|
|
968
987
|
dirPath,
|
|
@@ -970,15 +989,16 @@ async function scanSourceDirs(db, allSourceEntries, isIncremental, builtAtMs, ha
|
|
|
970
989
|
files: indexableFiles,
|
|
971
990
|
stash: null,
|
|
972
991
|
skip: true,
|
|
973
|
-
reason:
|
|
992
|
+
reason: cachedState.reason,
|
|
974
993
|
indexVariant,
|
|
975
994
|
});
|
|
976
|
-
reportDirDecision("skip", dirPath, currentStashDir,
|
|
995
|
+
reportDirDecision("skip", dirPath, currentStashDir, cachedState.reason, cachedState.persistedRowCount);
|
|
977
996
|
continue;
|
|
978
997
|
}
|
|
979
998
|
// F4a M-core-2 (the flip): drain the dir's `IndexDocument` stream via the
|
|
980
999
|
// component's dispatched `adapter.recognize` (broken workflows dropped-with-
|
|
981
1000
|
// warning at the drain layer) and reconstruct the durable `IndexDocument`s.
|
|
1001
|
+
drainObserverForTests?.(dirPath, ctxs.length);
|
|
982
1002
|
const drained = drainDirDocuments(adapter, component, ctxs);
|
|
983
1003
|
if (drained.warnings.length)
|
|
984
1004
|
warnings.push(...drained.warnings);
|
|
@@ -992,7 +1012,7 @@ async function scanSourceDirs(db, allSourceEntries, isIncremental, builtAtMs, ha
|
|
|
992
1012
|
if (generated.entries.length > 0) {
|
|
993
1013
|
generatedCount += generated.entries.length;
|
|
994
1014
|
}
|
|
995
|
-
recordFreshnessDecision(dirPath, currentStashDir, staleFiles, stash, drained.hashByFile, drained.conceptIdByFile, indexVariant, forceScan, walkComplete);
|
|
1015
|
+
recordFreshnessDecision(dirPath, currentStashDir, staleFiles, fingerprint, stash, drained.hashByFile, drained.conceptIdByFile, indexVariant, forceScan, walkComplete);
|
|
996
1016
|
}
|
|
997
1017
|
}
|
|
998
1018
|
return {
|
|
@@ -1089,7 +1109,7 @@ function persistDirRecords(db, dirRecords, doFullDelete, warnings, sourceRoots,
|
|
|
1089
1109
|
// tests/integration/indexer/reindex-generation-atomicity.test.ts.
|
|
1090
1110
|
indexTransactionHook("full-delete-applied");
|
|
1091
1111
|
}
|
|
1092
|
-
for (const { dirPath, currentStashDir, files, stash, skip, reason, hashByFile, conceptIdByFile, indexVariant, remove, pruneMissing, } of dirRecords) {
|
|
1112
|
+
for (const { dirPath, currentStashDir, files, fingerprint, stash, skip, reason, persistedRowCount, hashByFile, conceptIdByFile, indexVariant, remove, pruneMissing, } of dirRecords) {
|
|
1093
1113
|
const bundle = bundleByRoot.get(path.resolve(currentStashDir));
|
|
1094
1114
|
if (!bundle)
|
|
1095
1115
|
throw new Error(`Missing bundle provenance for indexed source ${currentStashDir}`);
|
|
@@ -1102,14 +1122,11 @@ function persistDirRecords(db, dirRecords, doFullDelete, warnings, sourceRoots,
|
|
|
1102
1122
|
continue;
|
|
1103
1123
|
}
|
|
1104
1124
|
if (skip) {
|
|
1105
|
-
|
|
1106
|
-
|
|
1107
|
-
|
|
1108
|
-
|
|
1109
|
-
|
|
1110
|
-
fileMtimeMaxMs: fingerprint.fileMtimeMaxMs,
|
|
1111
|
-
reason: reason.kind,
|
|
1112
|
-
});
|
|
1125
|
+
// "unchanged" is the post-drain verdict: re-persist so the row carries
|
|
1126
|
+
// row_count and the gate skips this directory before draining next
|
|
1127
|
+
// time. "unchanged-precheck" already matched the stored row.
|
|
1128
|
+
if (reason?.kind === "unchanged" && fingerprint) {
|
|
1129
|
+
upsertIndexDirState(db, { dirPath, ...fingerprint, reason: reason.kind, rowCount: persistedRowCount });
|
|
1113
1130
|
}
|
|
1114
1131
|
continue;
|
|
1115
1132
|
}
|
|
@@ -1165,7 +1182,7 @@ function persistDirRecords(db, dirRecords, doFullDelete, warnings, sourceRoots,
|
|
|
1165
1182
|
if (pruneMissing !== false) {
|
|
1166
1183
|
addEntryIds(deletedUsageEntryIds, deleteEntriesByDirExceptRefs(db, dirPath, bundle.bundleId, keptItemRefs, { cleanupUsageEvents: false }));
|
|
1167
1184
|
}
|
|
1168
|
-
const
|
|
1185
|
+
const persistedFingerprint = fingerprint ?? computeDirFingerprint(dirPath, files, indexVariant);
|
|
1169
1186
|
const persistedReason = persistedRows === 0
|
|
1170
1187
|
? inferZeroRowReason(stash, reason, warnings, dirPath, dedupedRows)
|
|
1171
1188
|
: reason?.kind === "full-rebuild"
|
|
@@ -1173,9 +1190,13 @@ function persistDirRecords(db, dirRecords, doFullDelete, warnings, sourceRoots,
|
|
|
1173
1190
|
: (reason?.kind ?? "updated");
|
|
1174
1191
|
upsertIndexDirState(db, {
|
|
1175
1192
|
dirPath,
|
|
1176
|
-
|
|
1177
|
-
fileMtimeMaxMs: fingerprint.fileMtimeMaxMs,
|
|
1193
|
+
...persistedFingerprint,
|
|
1178
1194
|
reason: persistedReason,
|
|
1195
|
+
// A directory that lost rows to per-source dedup depends on the
|
|
1196
|
+
// directories persisted before it, not only on its own files, so it
|
|
1197
|
+
// must keep draining every run (as it did before the gate) until a
|
|
1198
|
+
// drain persists it without dedup. NULL keeps the gate closed.
|
|
1199
|
+
rowCount: dedupedRows === 0 ? persistedRows : undefined,
|
|
1179
1200
|
});
|
|
1180
1201
|
if (persistedRows === 0) {
|
|
1181
1202
|
// Warn only when the dir had files that *could* produce entries (.md or
|
|
@@ -10,20 +10,25 @@
|
|
|
10
10
|
*
|
|
11
11
|
* Two persisted signals back the decision:
|
|
12
12
|
* 1. The `entries` rows already indexed for the directory (`getEntriesByDir`).
|
|
13
|
-
* 2. The `index_dir_state`
|
|
14
|
-
*
|
|
15
|
-
*
|
|
13
|
+
* 2. The `index_dir_state` row (`getIndexDirState`): the fingerprint of the
|
|
14
|
+
* directory's walked file set (basename set + max mtime, `computeDirFingerprint`)
|
|
15
|
+
* as of its last drain, plus the row count that drain persisted.
|
|
16
16
|
*
|
|
17
|
-
* `
|
|
18
|
-
*
|
|
17
|
+
* `getCachedDirState` is the pre-drain gate (#900): a directory whose walked
|
|
18
|
+
* fingerprint still matches its row is skipped before any file is read.
|
|
19
19
|
*/
|
|
20
20
|
import fs from "node:fs";
|
|
21
21
|
import path from "node:path";
|
|
22
22
|
import { getEntriesByDir } from "../../storage/repositories/index-entries-repository.js";
|
|
23
23
|
import { getIndexDirState } from "../../storage/repositories/index-meta-repository.js";
|
|
24
|
-
|
|
24
|
+
/**
|
|
25
|
+
* Post-drain freshness verdict. `files` is the recognized file set the drain
|
|
26
|
+
* produced (compared against the persisted entries); `fingerprint` is the
|
|
27
|
+
* walked-set fingerprint compared against the persisted row and defaults to
|
|
28
|
+
* one computed over `files`.
|
|
29
|
+
*/
|
|
30
|
+
export function getDirIndexState(db, dirPath, files, builtAtMs, indexVariant = "", fingerprint = computeDirFingerprint(dirPath, files, indexVariant)) {
|
|
25
31
|
const prevEntries = getEntriesByDir(db, dirPath);
|
|
26
|
-
const fingerprint = computeDirFingerprint(dirPath, files, indexVariant);
|
|
27
32
|
if (prevEntries.length > 0) {
|
|
28
33
|
const staleReason = getDirStaleReason(dirPath, files, prevEntries, builtAtMs);
|
|
29
34
|
if (staleReason)
|
|
@@ -54,8 +59,24 @@ export function getDirIndexState(db, dirPath, files, builtAtMs, indexVariant = "
|
|
|
54
59
|
persistedRowCount: 0,
|
|
55
60
|
};
|
|
56
61
|
}
|
|
57
|
-
|
|
58
|
-
|
|
62
|
+
/**
|
|
63
|
+
* Pre-drain gate (#900). A directory whose walked-set fingerprint matches its
|
|
64
|
+
* persisted row cannot recognize differently than last time, so it is skipped
|
|
65
|
+
* before `drainDirDocuments` reads a single file. A row that recorded a real
|
|
66
|
+
* generation (`rowCount > 0`) is skipped outright; a zero-row or pre-#900 row
|
|
67
|
+
* goes through the entries-aware check so the dedup-order guard still applies.
|
|
68
|
+
*/
|
|
69
|
+
export function getCachedDirState(db, dirPath, files, builtAtMs, priorDirsChanged, indexVariant, fingerprint) {
|
|
70
|
+
const cached = getIndexDirState(db, dirPath);
|
|
71
|
+
if (!cached ||
|
|
72
|
+
cached.fileSetHash !== fingerprint.fileSetHash ||
|
|
73
|
+
cached.fileMtimeMaxMs !== fingerprint.fileMtimeMaxMs) {
|
|
74
|
+
return undefined;
|
|
75
|
+
}
|
|
76
|
+
if (cached.rowCount !== undefined && cached.rowCount > 0) {
|
|
77
|
+
return { stale: false, reason: { kind: "unchanged-precheck" }, persistedRowCount: cached.rowCount };
|
|
78
|
+
}
|
|
79
|
+
const state = getDirIndexState(db, dirPath, files, builtAtMs, indexVariant, fingerprint);
|
|
59
80
|
if (state.stale || state.reason.kind !== "cached-zero-row-state")
|
|
60
81
|
return undefined;
|
|
61
82
|
if (!canUseIncrementalSkip(state, priorDirsChanged))
|
|
@@ -13104,6 +13104,7 @@ function warn(...args) {
|
|
|
13104
13104
|
console.warn(...args);
|
|
13105
13105
|
}
|
|
13106
13106
|
}
|
|
13107
|
+
var warnedOnceKeys = new Set;
|
|
13107
13108
|
function warnVerbose(...args) {
|
|
13108
13109
|
if (sinkOverride) {
|
|
13109
13110
|
sinkOverride("warnVerbose", args);
|
|
@@ -16855,6 +16856,7 @@ function base(input) {
|
|
|
16855
16856
|
...input.containmentRoot ? { containmentRoot: input.containmentRoot } : {}
|
|
16856
16857
|
};
|
|
16857
16858
|
}
|
|
16859
|
+
var ARGV_ARRAY_BLOCK_DETAIL = "Manual conversion required: an array `command:` has no safe v3 `run:` string. Rewrite it by hand as " + "`run:` (string) plus `shell:` — see docs/migration/v0.9.1-to-v0.9.2.md for the full v2 to v4 field mapping.";
|
|
16858
16860
|
function blocked(input, reason, detail) {
|
|
16859
16861
|
return Object.freeze({ status: "blocked", ...base(input), reason, ...detail ? { detail } : {} });
|
|
16860
16862
|
}
|
|
@@ -17117,8 +17119,10 @@ function planLegacyTaskDataToV3(input, data) {
|
|
|
17117
17119
|
} catch (cause) {
|
|
17118
17120
|
return blocked(input, "invalid-v2-task", cause instanceof Error ? cause.message : String(cause));
|
|
17119
17121
|
}
|
|
17120
|
-
if (isReason(migrated))
|
|
17121
|
-
|
|
17122
|
+
if (isReason(migrated)) {
|
|
17123
|
+
const detail = migrated === "argv-array-has-no-portable-shell-string" ? ARGV_ARRAY_BLOCK_DETAIL : undefined;
|
|
17124
|
+
return blocked(input, migrated, detail);
|
|
17125
|
+
}
|
|
17122
17126
|
const after = Buffer.from($stringify(migrated), "utf8");
|
|
17123
17127
|
try {
|
|
17124
17128
|
parseTaskV3Yaml({
|
|
@@ -23871,6 +23875,27 @@ function describeJsonRoot(value) {
|
|
|
23871
23875
|
return "a boolean";
|
|
23872
23876
|
return typeof value;
|
|
23873
23877
|
}
|
|
23878
|
+
function pruneToNewest(dir, keep, select) {
|
|
23879
|
+
let entries;
|
|
23880
|
+
try {
|
|
23881
|
+
entries = fs18.readdirSync(dir, { withFileTypes: true });
|
|
23882
|
+
} catch {
|
|
23883
|
+
return;
|
|
23884
|
+
}
|
|
23885
|
+
const candidates = entries.filter(select).map((entry) => {
|
|
23886
|
+
const full = path27.join(dir, entry.name);
|
|
23887
|
+
let mtime = 0;
|
|
23888
|
+
try {
|
|
23889
|
+
mtime = fs18.statSync(full).mtimeMs;
|
|
23890
|
+
} catch {}
|
|
23891
|
+
return { path: full, mtime };
|
|
23892
|
+
}).sort((a, b) => b.mtime - a.mtime);
|
|
23893
|
+
for (const stale of candidates.slice(keep)) {
|
|
23894
|
+
try {
|
|
23895
|
+
fs18.rmSync(stale.path, { recursive: true, force: true });
|
|
23896
|
+
} catch {}
|
|
23897
|
+
}
|
|
23898
|
+
}
|
|
23874
23899
|
function getConfigLockPath() {
|
|
23875
23900
|
return path27.join(getConfigDir(), "config.json.lck");
|
|
23876
23901
|
}
|
|
@@ -29844,6 +29869,10 @@ function inspectCurrentTaskPlan() {
|
|
|
29844
29869
|
function inspectMigrationPlan() {
|
|
29845
29870
|
return inspectCurrentTaskPlan().result;
|
|
29846
29871
|
}
|
|
29872
|
+
var MAX_TASK_MIGRATION_BACKUPS = 5;
|
|
29873
|
+
function pruneTaskMigrationBackups(generationBackupDir) {
|
|
29874
|
+
pruneToNewest(generationBackupDir, MAX_TASK_MIGRATION_BACKUPS, (entry) => entry.isDirectory());
|
|
29875
|
+
}
|
|
29847
29876
|
function printPlan(plan) {
|
|
29848
29877
|
console.log(JSON.stringify(plan));
|
|
29849
29878
|
if (plan.status === "blocked")
|
|
@@ -29861,12 +29890,14 @@ async function runMigrationApply(options = {}) {
|
|
|
29861
29890
|
const before = inspectCurrentTaskPlan();
|
|
29862
29891
|
if (before.result.taskV3Migration.changed === 0)
|
|
29863
29892
|
return before.result;
|
|
29864
|
-
const
|
|
29893
|
+
const backupRoot = path33.join(getDataDir(), "backups", "task-v3");
|
|
29894
|
+
const backupPath = path33.join(backupRoot, `${Date.now()}-${randomUUID2()}`);
|
|
29865
29895
|
const applied = applyTaskToV3MigrationPlan(before.plan, { backupRoot: backupPath });
|
|
29866
29896
|
const after = inspectCurrentTaskPlan().result;
|
|
29867
29897
|
if (after.taskV3Migration.changed > 0) {
|
|
29868
29898
|
throw new ConfigError("Task migration did not converge to task v3.", "INVALID_CONFIG_FILE");
|
|
29869
29899
|
}
|
|
29900
|
+
pruneTaskMigrationBackups(backupRoot);
|
|
29870
29901
|
return { ...after, backupPath, applied: applied.changed.length };
|
|
29871
29902
|
}));
|
|
29872
29903
|
printPlan(result);
|
|
@@ -29924,12 +29955,14 @@ async function runTaskV4MigrationApply(options = {}) {
|
|
|
29924
29955
|
const before = inspectCurrentTaskV4Plan();
|
|
29925
29956
|
if (before.result.taskV4Migration.changed === 0)
|
|
29926
29957
|
return before.result;
|
|
29927
|
-
const
|
|
29958
|
+
const backupRoot = path33.join(getDataDir(), "backups", "task-v4");
|
|
29959
|
+
const backupPath = path33.join(backupRoot, `${Date.now()}-${randomUUID2()}`);
|
|
29928
29960
|
const applied = applyTaskToV4MigrationPlan(before.plan, { backupRoot: backupPath });
|
|
29929
29961
|
const after = inspectCurrentTaskV4Plan().result;
|
|
29930
29962
|
if (after.taskV4Migration.changed > 0) {
|
|
29931
29963
|
throw new ConfigError("Task migration did not converge to task source v4.", "INVALID_CONFIG_FILE");
|
|
29932
29964
|
}
|
|
29965
|
+
pruneTaskMigrationBackups(backupRoot);
|
|
29933
29966
|
return { ...after, backupPath, applied: applied.changed.length };
|
|
29934
29967
|
}));
|
|
29935
29968
|
printPlan(result);
|
|
@@ -13026,6 +13026,7 @@ function warn(...args) {
|
|
|
13026
13026
|
console.warn(...args);
|
|
13027
13027
|
}
|
|
13028
13028
|
}
|
|
13029
|
+
var warnedOnceKeys = new Set;
|
|
13029
13030
|
function warnVerbose(...args) {
|
|
13030
13031
|
if (sinkOverride) {
|
|
13031
13032
|
sinkOverride("warnVerbose", args);
|
|
@@ -16777,6 +16778,7 @@ function base(input) {
|
|
|
16777
16778
|
...input.containmentRoot ? { containmentRoot: input.containmentRoot } : {}
|
|
16778
16779
|
};
|
|
16779
16780
|
}
|
|
16781
|
+
var ARGV_ARRAY_BLOCK_DETAIL = "Manual conversion required: an array `command:` has no safe v3 `run:` string. Rewrite it by hand as " + "`run:` (string) plus `shell:` \u2014 see docs/migration/v0.9.1-to-v0.9.2.md for the full v2 to v4 field mapping.";
|
|
16780
16782
|
function blocked(input, reason, detail) {
|
|
16781
16783
|
return Object.freeze({ status: "blocked", ...base(input), reason, ...detail ? { detail } : {} });
|
|
16782
16784
|
}
|
|
@@ -17039,8 +17041,10 @@ function planLegacyTaskDataToV3(input, data) {
|
|
|
17039
17041
|
} catch (cause) {
|
|
17040
17042
|
return blocked(input, "invalid-v2-task", cause instanceof Error ? cause.message : String(cause));
|
|
17041
17043
|
}
|
|
17042
|
-
if (isReason(migrated))
|
|
17043
|
-
|
|
17044
|
+
if (isReason(migrated)) {
|
|
17045
|
+
const detail = migrated === "argv-array-has-no-portable-shell-string" ? ARGV_ARRAY_BLOCK_DETAIL : undefined;
|
|
17046
|
+
return blocked(input, migrated, detail);
|
|
17047
|
+
}
|
|
17044
17048
|
const after = Buffer.from($stringify(migrated), "utf8");
|
|
17045
17049
|
try {
|
|
17046
17050
|
parseTaskV3Yaml({
|
|
@@ -23793,6 +23797,27 @@ function describeJsonRoot(value) {
|
|
|
23793
23797
|
return "a boolean";
|
|
23794
23798
|
return typeof value;
|
|
23795
23799
|
}
|
|
23800
|
+
function pruneToNewest(dir, keep, select) {
|
|
23801
|
+
let entries;
|
|
23802
|
+
try {
|
|
23803
|
+
entries = fs18.readdirSync(dir, { withFileTypes: true });
|
|
23804
|
+
} catch {
|
|
23805
|
+
return;
|
|
23806
|
+
}
|
|
23807
|
+
const candidates = entries.filter(select).map((entry) => {
|
|
23808
|
+
const full = path27.join(dir, entry.name);
|
|
23809
|
+
let mtime = 0;
|
|
23810
|
+
try {
|
|
23811
|
+
mtime = fs18.statSync(full).mtimeMs;
|
|
23812
|
+
} catch {}
|
|
23813
|
+
return { path: full, mtime };
|
|
23814
|
+
}).sort((a, b) => b.mtime - a.mtime);
|
|
23815
|
+
for (const stale of candidates.slice(keep)) {
|
|
23816
|
+
try {
|
|
23817
|
+
fs18.rmSync(stale.path, { recursive: true, force: true });
|
|
23818
|
+
} catch {}
|
|
23819
|
+
}
|
|
23820
|
+
}
|
|
23796
23821
|
function getConfigLockPath() {
|
|
23797
23822
|
return path27.join(getConfigDir(), "config.json.lck");
|
|
23798
23823
|
}
|
|
@@ -29766,6 +29791,10 @@ function inspectCurrentTaskPlan() {
|
|
|
29766
29791
|
function inspectMigrationPlan() {
|
|
29767
29792
|
return inspectCurrentTaskPlan().result;
|
|
29768
29793
|
}
|
|
29794
|
+
var MAX_TASK_MIGRATION_BACKUPS = 5;
|
|
29795
|
+
function pruneTaskMigrationBackups(generationBackupDir) {
|
|
29796
|
+
pruneToNewest(generationBackupDir, MAX_TASK_MIGRATION_BACKUPS, (entry) => entry.isDirectory());
|
|
29797
|
+
}
|
|
29769
29798
|
function printPlan(plan) {
|
|
29770
29799
|
console.log(JSON.stringify(plan));
|
|
29771
29800
|
if (plan.status === "blocked")
|
|
@@ -29783,12 +29812,14 @@ async function runMigrationApply(options = {}) {
|
|
|
29783
29812
|
const before = inspectCurrentTaskPlan();
|
|
29784
29813
|
if (before.result.taskV3Migration.changed === 0)
|
|
29785
29814
|
return before.result;
|
|
29786
|
-
const
|
|
29815
|
+
const backupRoot = path33.join(getDataDir(), "backups", "task-v3");
|
|
29816
|
+
const backupPath = path33.join(backupRoot, `${Date.now()}-${randomUUID2()}`);
|
|
29787
29817
|
const applied = applyTaskToV3MigrationPlan(before.plan, { backupRoot: backupPath });
|
|
29788
29818
|
const after = inspectCurrentTaskPlan().result;
|
|
29789
29819
|
if (after.taskV3Migration.changed > 0) {
|
|
29790
29820
|
throw new ConfigError("Task migration did not converge to task v3.", "INVALID_CONFIG_FILE");
|
|
29791
29821
|
}
|
|
29822
|
+
pruneTaskMigrationBackups(backupRoot);
|
|
29792
29823
|
return { ...after, backupPath, applied: applied.changed.length };
|
|
29793
29824
|
}));
|
|
29794
29825
|
printPlan(result);
|
|
@@ -29846,12 +29877,14 @@ async function runTaskV4MigrationApply(options = {}) {
|
|
|
29846
29877
|
const before = inspectCurrentTaskV4Plan();
|
|
29847
29878
|
if (before.result.taskV4Migration.changed === 0)
|
|
29848
29879
|
return before.result;
|
|
29849
|
-
const
|
|
29880
|
+
const backupRoot = path33.join(getDataDir(), "backups", "task-v4");
|
|
29881
|
+
const backupPath = path33.join(backupRoot, `${Date.now()}-${randomUUID2()}`);
|
|
29850
29882
|
const applied = applyTaskToV4MigrationPlan(before.plan, { backupRoot: backupPath });
|
|
29851
29883
|
const after = inspectCurrentTaskV4Plan().result;
|
|
29852
29884
|
if (after.taskV4Migration.changed > 0) {
|
|
29853
29885
|
throw new ConfigError("Task migration did not converge to task source v4.", "INVALID_CONFIG_FILE");
|
|
29854
29886
|
}
|
|
29887
|
+
pruneTaskMigrationBackups(backupRoot);
|
|
29855
29888
|
return { ...after, backupPath, applied: applied.changed.length };
|
|
29856
29889
|
}));
|
|
29857
29890
|
printPlan(result);
|
|
@@ -336,12 +336,28 @@ export function getPositiveFeedbackCountsByIds(ids) {
|
|
|
336
336
|
}, "positive feedback counts are best-effort");
|
|
337
337
|
return result;
|
|
338
338
|
}
|
|
339
|
-
|
|
340
|
-
|
|
341
|
-
|
|
342
|
-
|
|
339
|
+
/**
|
|
340
|
+
* Rows whose `file_path` sits directly in `dirPath`. A half-open byte range
|
|
341
|
+
* over `idx_entries_file_path` (`[dir + sep, dir + sep + 1)`) turns the lookup
|
|
342
|
+
* into an index seek; the range is exact for "starts with `dir/`" but also
|
|
343
|
+
* admits nested subdirectories (`/a/b/c/x.md` for `/a/b`), so the dirname
|
|
344
|
+
* post-filter stays.
|
|
345
|
+
*/
|
|
346
|
+
function selectRowsInDirectory(db, dirPath, columns, bundleId) {
|
|
343
347
|
const resolvedDir = path.resolve(dirPath);
|
|
344
|
-
|
|
348
|
+
const prefix = resolvedDir + path.sep;
|
|
349
|
+
const upperBound = resolvedDir + String.fromCharCode(path.sep.charCodeAt(0) + 1);
|
|
350
|
+
const params = [prefix, upperBound];
|
|
351
|
+
let sql = `SELECT ${columns} FROM entries WHERE file_path >= ? AND file_path < ?`;
|
|
352
|
+
if (bundleId) {
|
|
353
|
+
sql += " AND bundle_id = ?";
|
|
354
|
+
params.push(bundleId);
|
|
355
|
+
}
|
|
356
|
+
const rows = db.prepare(sql).all(...params);
|
|
357
|
+
return rows.filter((row) => path.dirname(row.file_path) === resolvedDir);
|
|
358
|
+
}
|
|
359
|
+
function rowsInDirectory(db, dirPath, bundleId) {
|
|
360
|
+
return selectRowsInDirectory(db, dirPath, "id, item_ref, file_path", bundleId);
|
|
345
361
|
}
|
|
346
362
|
function deleteEntryRows(db, rows, options = {}) {
|
|
347
363
|
if (rows.length === 0)
|
|
@@ -629,9 +645,7 @@ export function getEntryById(db, id) {
|
|
|
629
645
|
};
|
|
630
646
|
}
|
|
631
647
|
export function getEntriesByDir(db, dirPath) {
|
|
632
|
-
|
|
633
|
-
const rows = db.prepare(`SELECT ${ENTRY_COLUMNS} FROM entries`).all().filter((row) => ids.has(row.id));
|
|
634
|
-
return parseEntryRows(rows, "getEntriesByDir");
|
|
648
|
+
return parseEntryRows(selectRowsInDirectory(db, dirPath, ENTRY_COLUMNS), "getEntriesByDir");
|
|
635
649
|
}
|
|
636
650
|
/** Return every directory previously indexed for one canonical bundle. */
|
|
637
651
|
export function getIndexedDirPathsByBundleId(db, bundleId) {
|
|
@@ -640,9 +654,8 @@ export function getIndexedDirPathsByBundleId(db, bundleId) {
|
|
|
640
654
|
}
|
|
641
655
|
/** Return every persisted bundle owner for one physical directory. */
|
|
642
656
|
export function getIndexedBundleIdsByDir(db, dirPath) {
|
|
643
|
-
const
|
|
644
|
-
|
|
645
|
-
return [...new Set(rows.filter((row) => ids.has(row.id)).map((row) => row.bundle_id))];
|
|
657
|
+
const rows = selectRowsInDirectory(db, dirPath, "bundle_id, file_path");
|
|
658
|
+
return [...new Set(rows.map((row) => row.bundle_id))];
|
|
646
659
|
}
|
|
647
660
|
/**
|
|
648
661
|
* Resolve a single `entries.id` by exact `file_path` (the canonical on-disk
|
|
@@ -23,7 +23,7 @@ export function deleteMeta(db, key) {
|
|
|
23
23
|
// ── Per-directory index state ───────────────────────────────────────────────
|
|
24
24
|
export function getIndexDirState(db, dirPath) {
|
|
25
25
|
const row = db
|
|
26
|
-
.prepare("SELECT dir_path, file_set_hash, file_mtime_max_ms, reason, updated_at FROM index_dir_state WHERE dir_path = ?")
|
|
26
|
+
.prepare("SELECT dir_path, file_set_hash, file_mtime_max_ms, reason, updated_at, row_count FROM index_dir_state WHERE dir_path = ?")
|
|
27
27
|
.get(dirPath);
|
|
28
28
|
if (!row)
|
|
29
29
|
return undefined;
|
|
@@ -33,16 +33,18 @@ export function getIndexDirState(db, dirPath) {
|
|
|
33
33
|
fileMtimeMaxMs: row.file_mtime_max_ms,
|
|
34
34
|
reason: row.reason,
|
|
35
35
|
updatedAt: row.updated_at,
|
|
36
|
+
rowCount: row.row_count ?? undefined,
|
|
36
37
|
};
|
|
37
38
|
}
|
|
38
39
|
export function upsertIndexDirState(db, state) {
|
|
39
|
-
db.prepare(`INSERT INTO index_dir_state (dir_path, file_set_hash, file_mtime_max_ms, reason, updated_at)
|
|
40
|
-
VALUES (?, ?, ?, ?, ?)
|
|
40
|
+
db.prepare(`INSERT INTO index_dir_state (dir_path, file_set_hash, file_mtime_max_ms, reason, updated_at, row_count)
|
|
41
|
+
VALUES (?, ?, ?, ?, ?, ?)
|
|
41
42
|
ON CONFLICT(dir_path) DO UPDATE SET
|
|
42
43
|
file_set_hash = excluded.file_set_hash,
|
|
43
44
|
file_mtime_max_ms = excluded.file_mtime_max_ms,
|
|
44
45
|
reason = excluded.reason,
|
|
45
|
-
updated_at = excluded.updated_at
|
|
46
|
+
updated_at = excluded.updated_at,
|
|
47
|
+
row_count = excluded.row_count`).run(state.dirPath, state.fileSetHash, state.fileMtimeMaxMs, state.reason, new Date().toISOString(), state.rowCount ?? null);
|
|
46
48
|
}
|
|
47
49
|
export function deleteIndexDirState(db, dirPath) {
|
|
48
50
|
db.prepare("DELETE FROM index_dir_state WHERE dir_path = ?").run(dirPath);
|
|
@@ -268,9 +268,11 @@ export function ensureSchema(db, embeddingDim) {
|
|
|
268
268
|
file_set_hash TEXT NOT NULL,
|
|
269
269
|
file_mtime_max_ms REAL NOT NULL,
|
|
270
270
|
reason TEXT NOT NULL,
|
|
271
|
-
updated_at TEXT NOT NULL
|
|
271
|
+
updated_at TEXT NOT NULL,
|
|
272
|
+
row_count INTEGER
|
|
272
273
|
);
|
|
273
274
|
`);
|
|
275
|
+
ensureIndexDirStateRowCountColumn(db);
|
|
274
276
|
// LLM enrichment result cache. Stores a SHA-256 body hash and the JSON
|
|
275
277
|
// result for each asset so that subsequent `akm index --enrich` runs can
|
|
276
278
|
// skip the LLM call when the body hasn't changed. The cache is keyed by
|
|
@@ -379,3 +381,16 @@ function tableExists(db, name) {
|
|
|
379
381
|
const row = db.prepare("SELECT 1 FROM sqlite_master WHERE type='table' AND name=? LIMIT 1").get(name);
|
|
380
382
|
return row !== undefined && row !== null;
|
|
381
383
|
}
|
|
384
|
+
/**
|
|
385
|
+
* #900: `row_count` was added after the table's first release, so a database
|
|
386
|
+
* created before it needs an `ALTER TABLE` (`CREATE TABLE IF NOT EXISTS` only
|
|
387
|
+
* shapes a fresh table). Idempotent. Pre-existing rows keep NULL until their
|
|
388
|
+
* directory is next drained; index.db is a regenerable cache, so nothing is
|
|
389
|
+
* backfilled.
|
|
390
|
+
*/
|
|
391
|
+
function ensureIndexDirStateRowCountColumn(db) {
|
|
392
|
+
const columns = db.prepare("PRAGMA table_info(index_dir_state)").all();
|
|
393
|
+
if (!columns.some((column) => column.name === "row_count")) {
|
|
394
|
+
db.exec("ALTER TABLE index_dir_state ADD COLUMN row_count INTEGER");
|
|
395
|
+
}
|
|
396
|
+
}
|
|
@@ -12,6 +12,7 @@
|
|
|
12
12
|
import path from "node:path";
|
|
13
13
|
import { stashDirFor } from "../../core/asset/asset-placement.js";
|
|
14
14
|
import { bundleRefToString, isBundleSlug, parseBundleRef } from "../../core/asset/asset-ref.js";
|
|
15
|
+
import { warnOnce } from "../../core/warn.js";
|
|
15
16
|
/** Serialize `Proposal.changes` for `metadata_json` (see {@link StoredFileChange}). */
|
|
16
17
|
function changesToStored(changes) {
|
|
17
18
|
return changes.map((c, i) => ({
|
|
@@ -361,8 +362,10 @@ export function listStateProposals(db, options = {}) {
|
|
|
361
362
|
proposals.push(proposalRowToProposal(row));
|
|
362
363
|
}
|
|
363
364
|
catch (error) {
|
|
365
|
+
// Once per row per process (#898): health alone reads this table several
|
|
366
|
+
// times per invocation.
|
|
364
367
|
const message = error instanceof Error ? error.message : String(error);
|
|
365
|
-
|
|
368
|
+
warnOnce(`unparseable-proposal-row:${row.id}`, `[akm] Skipping unparseable proposal row (id=${row.id}, ref=${row.ref}): ${message}`);
|
|
366
369
|
}
|
|
367
370
|
}
|
|
368
371
|
return proposals;
|
|
@@ -152,7 +152,22 @@ function samePath(left, right) {
|
|
|
152
152
|
};
|
|
153
153
|
return normalize(left) === normalize(right);
|
|
154
154
|
}
|
|
155
|
+
// #901: the global root can't change within a process lifetime, and every
|
|
156
|
+
// `resolveAkmInvocation()` call in a scheduler-sync process re-derives the
|
|
157
|
+
// same `nodePath` — so this spawns `npm root --global` at most once per
|
|
158
|
+
// process instead of once per call. Keyed by nodePath (rather than a bare
|
|
159
|
+
// once-only flag) so a differently-invoked probe later in the same process
|
|
160
|
+
// still gets its own answer instead of a stale one.
|
|
161
|
+
let cachedNpmGlobalRoot;
|
|
155
162
|
function resolveNpmGlobalRoot(nodePath, env) {
|
|
163
|
+
if (cachedNpmGlobalRoot && cachedNpmGlobalRoot.nodePath === nodePath) {
|
|
164
|
+
return cachedNpmGlobalRoot.value;
|
|
165
|
+
}
|
|
166
|
+
const value = resolveNpmGlobalRootUncached(nodePath, env);
|
|
167
|
+
cachedNpmGlobalRoot = { nodePath, value };
|
|
168
|
+
return value;
|
|
169
|
+
}
|
|
170
|
+
function resolveNpmGlobalRootUncached(nodePath, env) {
|
|
156
171
|
const npmCli = resolveAssociatedNpmCli(nodePath);
|
|
157
172
|
if (!npmCli)
|
|
158
173
|
return undefined;
|
|
@@ -112,6 +112,12 @@ function base(input) {
|
|
|
112
112
|
...(input.containmentRoot ? { containmentRoot: input.containmentRoot } : {}),
|
|
113
113
|
};
|
|
114
114
|
}
|
|
115
|
+
/**
|
|
116
|
+
* #902: the one blocker with an unambiguous remedy. The sibling shell-safety
|
|
117
|
+
* reasons need a case-by-case judgement and stay reason-only.
|
|
118
|
+
*/
|
|
119
|
+
const ARGV_ARRAY_BLOCK_DETAIL = "Manual conversion required: an array `command:` has no safe v3 `run:` string. Rewrite it by hand as " +
|
|
120
|
+
"`run:` (string) plus `shell:` — see docs/migration/v0.9.1-to-v0.9.2.md for the full v2 to v4 field mapping.";
|
|
115
121
|
function blocked(input, reason, detail) {
|
|
116
122
|
return Object.freeze({ status: "blocked", ...base(input), reason, ...(detail ? { detail } : {}) });
|
|
117
123
|
}
|
|
@@ -389,8 +395,10 @@ export function planLegacyTaskDataToV3(input, data) {
|
|
|
389
395
|
catch (cause) {
|
|
390
396
|
return blocked(input, "invalid-v2-task", cause instanceof Error ? cause.message : String(cause));
|
|
391
397
|
}
|
|
392
|
-
if (isReason(migrated))
|
|
393
|
-
|
|
398
|
+
if (isReason(migrated)) {
|
|
399
|
+
const detail = migrated === "argv-array-has-no-portable-shell-string" ? ARGV_ARRAY_BLOCK_DETAIL : undefined;
|
|
400
|
+
return blocked(input, migrated, detail);
|
|
401
|
+
}
|
|
394
402
|
const after = Buffer.from(stringifyYaml(migrated), "utf8");
|
|
395
403
|
try {
|
|
396
404
|
parseTaskV3Yaml({
|
|
@@ -212,10 +212,24 @@ command: [node, scripts/release.js, "--exact value"]
|
|
|
212
212
|
|
|
213
213
|
Shell-sensitive strings — assignments, shell builtins, reserved words, and
|
|
214
214
|
similar constructs — are also blocked when translating them would invent
|
|
215
|
-
shell semantics. The original bytes remain untouched
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
|
|
215
|
+
shell semantics. The original bytes remain untouched, and `akm migrate
|
|
216
|
+
status`/`apply` names the block as `argv-array-has-no-portable-shell-string`
|
|
217
|
+
(or a sibling shell-safety reason) rather than guessing: this is manual
|
|
218
|
+
conversion, not a case the migrator can be re-run to fix. Rewrite the file
|
|
219
|
+
by hand using this field mapping:
|
|
220
|
+
|
|
221
|
+
| v2 | v4 |
|
|
222
|
+
|---|---|
|
|
223
|
+
| `command:` (array, argv style) | `run:` (string) plus `shell:` |
|
|
224
|
+
| `timeoutMs:` | `timeout:` |
|
|
225
|
+
| `enabled:` (document level) | removed — use `schedule:` as a list of `{cron, enabled}` entries |
|
|
226
|
+
| `schedule:` (cron string) | still accepted as a bare string, or as the list form above |
|
|
227
|
+
|
|
228
|
+
validate it, and rerun the preview. You can either write the replacement
|
|
229
|
+
directly as `version: 4` (generation 1 then reports it `already-v4` and
|
|
230
|
+
leaves it alone) or as a `version: 3` `run:` string, letting the second
|
|
231
|
+
generation carry it the rest of the way to v4 in the same `akm migrate
|
|
232
|
+
apply` run.
|
|
219
233
|
|
|
220
234
|
### Migrating task v3 to task source v4
|
|
221
235
|
|
package/docs/reference/cli.md
CHANGED
|
@@ -582,7 +582,7 @@ Subcommands:
|
|
|
582
582
|
| `list` | List workflow runs (optionally filtered by `--ref`; `--active` shows only `status=active` runs, excluding `blocked`/`failed`/`completed`). Child workflow runs are excluded unless `--children` is passed |
|
|
583
583
|
| `resume <run-id>` | Flip a `blocked` or `failed` run back to `active`. Completed runs cannot be resumed |
|
|
584
584
|
| `abandon <run-id>` | Mark a run failed so it stops counting as active (`resume` can reopen it) |
|
|
585
|
-
| `plan <ref>` | **Evolving.** Compile and freeze a workflow WITHOUT publishing a run: the canonical step graph, per-step frozen target kinds, task/child expansion, input bindings, source read set, and lowering notices — zero durable writes.
|
|
585
|
+
| `plan <ref>` | **Evolving.** Compile and freeze a workflow WITHOUT publishing a run: the canonical step graph, per-step frozen target kinds, task/child expansion, input bindings, source read set, and lowering notices — zero durable writes. Returns the full JSON envelope by default, like every other command; pass `--format text` for a human-readable summary |
|
|
586
586
|
|
|
587
587
|
The public `workflow start`, `next`, and `complete` lifecycle was removed in
|
|
588
588
|
0.9, along with the experimental `brief`/`report` external-driver protocol.
|
|
@@ -60,6 +60,7 @@ Override: set `AKM_CONFIG_DIR` or `XDG_CONFIG_HOME`.
|
|
|
60
60
|
| `state.db` | Events, local usage telemetry, proposals, task history, improve run results, and workflow run state/history (the former `workflow.db` was folded in during the 0.9.0 cutover) | **No** — deletes event/usage logs, proposal queue, improve history, and workflow run history |
|
|
61
61
|
| `logs.db` | Structured, high-volume task/run log lines (`{ts, task_id, run_id, stream, level, line}`), joined to `state.db`'s `task_history` rows by `task_id@started_at`. Kept separate from `state.db` because log lines are append-only and freely purgeable, unlike durable state | Yes — log lines are regenerable per run; deleting loses historical run output only |
|
|
62
62
|
| `akm.lock` | Inter-process write lock | Yes — recreated automatically |
|
|
63
|
+
| `backups/task-v3/`, `backups/task-v4/` | Copies of task files taken by `akm migrate apply` before it rewrites them, one timestamped directory per run; the five most recent per generation are kept (#897) | Yes — once the migrated tasks are verified |
|
|
63
64
|
| `akm.lock.lck` | Lock write sentinel | Yes — recreated automatically |
|
|
64
65
|
|
|
65
66
|
Override: set `AKM_DATA_DIR` or `XDG_DATA_HOME`.
|
package/docs/reference/tasks.md
CHANGED
|
@@ -325,6 +325,17 @@ for both generations combined. Resolve every blocked file manually, then
|
|
|
325
325
|
preview again. Apply validates a complete replacement before writing and
|
|
326
326
|
backs up each original immediately before replacement.
|
|
327
327
|
|
|
328
|
+
Common v2 → v3 blocked reasons and what to do about each — these need a
|
|
329
|
+
hand-authored replacement, not a re-run; see [the 0.9.1 to 0.9.2 migration
|
|
330
|
+
guide](../migration/v0.9.1-to-v0.9.2.md#v2--v3-blocked-cases) for the full v2
|
|
331
|
+
to v4 field mapping (`command:` array → `run:` + `shell:`, `timeoutMs:` →
|
|
332
|
+
`timeout:`, document-level `enabled:` → per-`schedule:`-entry `enabled`):
|
|
333
|
+
|
|
334
|
+
| Reason | Meaning | Fix |
|
|
335
|
+
|---|---|---|
|
|
336
|
+
| `argv-array-has-no-portable-shell-string` | The task's `command:` is an argv array; no single shell string is provably equivalent. | Rewrite the file by hand — a `run:` string plus `shell:` — using the field mapping above. |
|
|
337
|
+
| `shell-quoting-changes-v2-whitespace-split-semantics`, `shell-operators-change-v2-literal-argv-semantics`, `shell-command-resolution-changes-v2-literal-argv-semantics` | The `command:` string contains quoting, shell operators, or a bare executable name whose v2 argv-exec behavior a v3 `run:` (host-shell) invocation cannot reproduce unambiguously. | Review the command's intended shell semantics and author the v3/v4 `run:`/`shell:` fields by hand. |
|
|
338
|
+
|
|
328
339
|
Common v3 → v4 blocked reasons and what to do about each:
|
|
329
340
|
|
|
330
341
|
| Reason | Meaning | Fix |
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "akm-cli",
|
|
3
|
-
"version": "0.9.8-beta.
|
|
3
|
+
"version": "0.9.8-beta.2",
|
|
4
4
|
"type": "module",
|
|
5
5
|
"description": "akm (Agent Knowledge Manager) — a portable, local-first capability library for AI agents. Discover, load, share, and improve reusable skills, scripts, workflows, and knowledge across any shell-capable coding agent, including Claude Code, OpenCode, and Cursor.",
|
|
6
6
|
"keywords": [
|
|
@@ -77,14 +77,10 @@
|
|
|
77
77
|
"test:integration": "bash scripts/test-integration.sh",
|
|
78
78
|
"test:node-smoke": "bun scripts/node-smoke.ts",
|
|
79
79
|
"test:node-compat": "AKM_NODE_COMPAT_TESTS=1 bun test --timeout=120000 tests/integration/node-compat.test.ts",
|
|
80
|
-
"lint:isolation": "bun scripts/lint-tests-isolation.ts",
|
|
81
80
|
"lint:doc-examples": "bun scripts/lint-doc-examples.ts",
|
|
82
|
-
"lint:active-docs-terminology": "bun scripts/lint-active-docs-terminology.ts",
|
|
83
81
|
"publish:devto": "npx -y @sinedied/devto-cli push \"docs/posts/**/*.md\" --token \"$DEVTO_TOKEN\" --repo \"$GITHUB_REPOSITORY\" --branch \"${GITHUB_REF_NAME:-main}\" --reconcile",
|
|
84
82
|
"release:check": "./tests/release-check.sh",
|
|
85
83
|
"lint": "bunx biome check src/ tests/ scripts/ && bun scripts/lint-secret-resolver-boundary.ts && bun scripts/lint-shipped-assets.ts && bun scripts/lint-doc-examples.ts",
|
|
86
|
-
"lint:runtime-boundary": "bun scripts/lint-runtime-boundary.ts",
|
|
87
|
-
"lint:tests-isolation": "bun scripts/lint-tests-isolation.ts",
|
|
88
84
|
"lint:fix": "bunx biome check --write src/ tests/ scripts/",
|
|
89
85
|
"format": "bunx biome format --write src/ tests/ scripts/",
|
|
90
86
|
"prepublishOnly": "cp .github/README.npm.md README.md && bun run build",
|