@titan-design/session-graph 0.10.0 → 0.12.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +54 -3
- package/dist/index.d.ts +32 -2
- package/dist/index.js +169 -6
- package/dist/index.js.map +1 -1
- package/package.json +3 -3
package/README.md
CHANGED
|
@@ -21,8 +21,12 @@ const summary = await refreshCorpus(graph, await discoverTranscripts());
|
|
|
21
21
|
`openSessionGraph` takes an optional `schemaVersion`: the highest migration version the
|
|
22
22
|
*caller* owns, checked before any migration runs. A product layering its own tables on this
|
|
23
23
|
graph passes its own top version, since this package's `MIGRATIONS` are one band of a shared
|
|
24
|
-
database rather than the top of it. `products/session-miner` numbers its own from
|
|
25
|
-
passes
|
|
24
|
+
database rather than the top of it. `products/session-miner` numbers its own from 2000 and
|
|
25
|
+
passes 2002, clear of active-work's band at 1001.
|
|
26
|
+
|
|
27
|
+
`{ readonly: true }` opens a graph another process owns without writing to it. No
|
|
28
|
+
migrations run, and the open throws `SessionGraphNotMigratedError` when the graph lacks any
|
|
29
|
+
migration this package declares, so a reader never meets a table it does not expect.
|
|
26
30
|
|
|
27
31
|
## What a refresh does
|
|
28
32
|
|
|
@@ -174,6 +178,44 @@ Everything except `transcript` is derivable, which is what makes the schema safe
|
|
|
174
178
|
by drop-and-rederive.
|
|
175
179
|
|
|
176
180
|
|
|
181
|
+
## Injected context
|
|
182
|
+
|
|
183
|
+
A `user` record carries more than the human typed. The harness, hooks and agent-chat put
|
|
184
|
+
their own blocks there, and indexing them would make every session match its own
|
|
185
|
+
reminders and briefs. `stripInjected` removes them before a prompt reaches `search_fts`,
|
|
186
|
+
and `readIndexedText` applies it again on readback, so a miner excerpt shows the same
|
|
187
|
+
text. Only prompt spans change. `fact`, `normalized_event` and the audit tables keep
|
|
188
|
+
every line, and a turn with nothing left indexes no prompt span.
|
|
189
|
+
An index built before this rule keeps its old prompt spans until `resetIndex` rebuilds it.
|
|
190
|
+
|
|
191
|
+
- **Whole line.** A line whose session-read inbound cause is anything but `human_typed` or
|
|
192
|
+
`tool_result` indexes no prompt. That covers `isMeta` lines (hook and SessionStart
|
|
193
|
+
bootstrap output, skill loads, harness resumes), peer channel messages, task
|
|
194
|
+
notifications, compaction summaries, scheduled wakeups, image notes and local commands.
|
|
195
|
+
- **Headless prompts.** A line whose `promptSource` is `sdk` indexes no prompt. The host
|
|
196
|
+
writes `sdk` for every headless turn, which is how an agent-chat spawn brief with no
|
|
197
|
+
framing is told apart from a typed prompt. The cost: a person running `claude -p` also
|
|
198
|
+
writes `sdk` turns, and those are excluded too. Pass `indexSdkPrompts: true` to
|
|
199
|
+
`refreshCorpus`, `indexTranscript` or `applyDelta` to keep them. `isUntypedPrompt` exposes
|
|
200
|
+
the rule.
|
|
201
|
+
- **Blocks cut on their own lines.** session-read's closed `INJECTED_MARKERS`
|
|
202
|
+
(`<system-reminder>`, `<channel source="...">`, `<task-notification>`), plus
|
|
203
|
+
`<user-prompt-submit-hook>`, `<command-name>`, `<command-message>`,
|
|
204
|
+
`<local-command-caveat>`, `<local-command-stdout>`, `<local-command-stderr>`,
|
|
205
|
+
`<bash-stdout>` and `<bash-stderr>`. A block is cut only when its opening tag starts a
|
|
206
|
+
line and its closing tag ends one (another cut block may follow on the same line). Nested
|
|
207
|
+
tags of the same name are balanced. A tag quoted inside a sentence stays.
|
|
208
|
+
`[Request interrupted by user…]` notes are cut wherever they sit.
|
|
209
|
+
- **Kept words.** `<command-args>` and `<bash-input>` lose their tags and keep their
|
|
210
|
+
contents, because the human typed them.
|
|
211
|
+
- **Whole turn by text.** session-read's unclosed markers heading the text (local-command
|
|
212
|
+
output, compaction summary, image note, loop wakeup, the agent-chat orientation header).
|
|
213
|
+
agent-chat puts no tag around a spawn brief, so a brief typed into a live session is known
|
|
214
|
+
by the framing agent-chat writes itself: a `# Predecessor:` handover, or the isolated or
|
|
215
|
+
shared worktree note that ends the brief.
|
|
216
|
+
- **Tool-result echoes.** `tool_result` blocks never enter prompt text; a user line's prompt
|
|
217
|
+
span holds its text blocks only. Background results arrive as `<task-notification>`.
|
|
218
|
+
|
|
177
219
|
## Mixed harnesses
|
|
178
220
|
|
|
179
221
|
`indexCodexSource(graph, source)` consumes descriptors returned by session-read's
|
|
@@ -222,7 +264,16 @@ empty tables, so existing rows are untouched.
|
|
|
222
264
|
session-analytics. A rewritten transcript purges its sessions' episodes.
|
|
223
265
|
- `price` holds USD per million tokens by model prefix and effective date.
|
|
224
266
|
`syncPrices(graph, rows, { tableVersion, source })` replaces the whole table in one
|
|
225
|
-
transaction.
|
|
267
|
+
transaction. `reconcilePrices(graph, rows, { tableVersion, source })` upserts instead: it adds
|
|
268
|
+
missing models, updates changed rates, stamps the table version, and keeps rows the table
|
|
269
|
+
does not name unless it has a `source`, in which case rows of that source the table dropped
|
|
270
|
+
are pruned. A stored row from a newer `tableVersion` is never overwritten or pruned, so an
|
|
271
|
+
older caller cannot downgrade rates. It writes nothing when the graph already matches. Pass session-analytics'
|
|
272
|
+
`PRICE_TABLE` and `PRICE_TABLE_VERSION`; session-graph cannot import that package, so the
|
|
273
|
+
product does. `titan-miner` reconciles every time it opens the graph, so `titan-miner refresh`
|
|
274
|
+
brings a live graph's prices current. Only that entry point reconciles: a graph opened any
|
|
275
|
+
other way (a read-only connection, another product's bundled graph) keeps whatever rows it
|
|
276
|
+
stored, and its cost reports price from them. Such a caller must call `reconcilePrices` itself.
|
|
226
277
|
- `request_dedup` collapses fan-out copies of a request to the earliest one. Every cost
|
|
227
278
|
query reads it, never `request`. `request_cost` prices each row by longest model prefix
|
|
228
279
|
and latest `effective_from`; an unmatched model reads `priced = 0` and costs 0.
|
package/dist/index.d.ts
CHANGED
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
import { Db, WatermarkTable, EdgeTable, SpanFtsTables, Migration } from '@titan-design/store-sqlite';
|
|
2
|
-
import { TranscriptDelta, DiscoveredTranscript, SessionSourceDescriptor, SessionUsageSummary } from '@titan-design/session-read';
|
|
2
|
+
import { TranscriptDelta, DiscoveredTranscript, SessionSourceDescriptor, WakeCause, Inbound, SessionUsageSummary } from '@titan-design/session-read';
|
|
3
3
|
|
|
4
4
|
/** One open session graph: the connection plus the kit helpers bound to its tables. */
|
|
5
5
|
interface SessionGraph {
|
|
@@ -15,6 +15,15 @@ interface OpenSessionGraphOptions {
|
|
|
15
15
|
* package's: `MIGRATIONS` here is one band of a shared database, never the top of it.
|
|
16
16
|
*/
|
|
17
17
|
schemaVersion?: number;
|
|
18
|
+
/**
|
|
19
|
+
* Open a graph another process owns without writing to it: no migrations run, and
|
|
20
|
+
* the graph must already carry every migration this package declares.
|
|
21
|
+
*/
|
|
22
|
+
readonly?: boolean;
|
|
23
|
+
}
|
|
24
|
+
declare class SessionGraphNotMigratedError extends Error {
|
|
25
|
+
readonly missing: readonly string[];
|
|
26
|
+
constructor(missing: readonly string[]);
|
|
18
27
|
}
|
|
19
28
|
declare function openSessionGraph(dbPath: string, options?: OpenSessionGraphOptions): SessionGraph;
|
|
20
29
|
/**
|
|
@@ -111,6 +120,8 @@ declare function applyDelta(graph: SessionGraph, transcriptId: number, delta: Tr
|
|
|
111
120
|
interface DeltaSource {
|
|
112
121
|
/** The Claude config dir the transcript was found under; `null` when discovery did not say. */
|
|
113
122
|
account?: string | null;
|
|
123
|
+
/** Index prompts the host labels `promptSource: "sdk"` (spawned agents and `claude -p`). Off by default; see README "Injected context". */
|
|
124
|
+
indexSdkPrompts?: boolean;
|
|
114
125
|
}
|
|
115
126
|
|
|
116
127
|
/**
|
|
@@ -365,6 +376,8 @@ interface IndexOptions {
|
|
|
365
376
|
resolveOrigins?: OriginResolver;
|
|
366
377
|
/** Fill PR state, merge, close and review rounds from the caller's forge. Runs once per `refreshCorpus` pass. Absent means transcripts alone. */
|
|
367
378
|
resolvePrs?: PrResolver;
|
|
379
|
+
/** Index prompts the host labels `promptSource: "sdk"`: spawned agents' briefs, and `claude -p` runs a person typed. Default false. */
|
|
380
|
+
indexSdkPrompts?: boolean;
|
|
368
381
|
}
|
|
369
382
|
interface RefreshOptions extends IndexOptions {
|
|
370
383
|
/** Which sender profiles' chat verdicts count toward review rounds. Defaults to `reviewer` and `*-reviewer`. */
|
|
@@ -475,6 +488,17 @@ interface SyncPricesOptions {
|
|
|
475
488
|
}
|
|
476
489
|
/** Replace every `price` row with `rows` in one transaction, so `request_cost` never reads a half-written table. */
|
|
477
490
|
declare function syncPrices(graph: SessionGraph, rows: readonly PriceInput[], options: SyncPricesOptions): number;
|
|
491
|
+
interface ReconcileResult {
|
|
492
|
+
added: number;
|
|
493
|
+
updated: number;
|
|
494
|
+
pruned: number;
|
|
495
|
+
}
|
|
496
|
+
/**
|
|
497
|
+
* Upsert `rows` into `price`, stamping `tableVersion`; a current graph is not written.
|
|
498
|
+
* A stored row from a newer table version is left alone, so an older caller never downgrades rates.
|
|
499
|
+
* With a `source`, rows of that source the table no longer names are pruned (a longer stale prefix would win in `request_cost`), unless a newer version wrote any of them.
|
|
500
|
+
*/
|
|
501
|
+
declare function reconcilePrices(graph: SessionGraph, rows: readonly PriceInput[], options: SyncPricesOptions): ReconcileResult;
|
|
478
502
|
|
|
479
503
|
interface NormalizedIndexResult {
|
|
480
504
|
status: "indexed" | "unchanged" | "missing" | "quarantined";
|
|
@@ -485,6 +509,12 @@ interface NormalizedIndexResult {
|
|
|
485
509
|
/** Replay changed sources into temporary staging; swap rows and watermark atomically. */
|
|
486
510
|
declare function indexCodexSource(graph: SessionGraph, source: SessionSourceDescriptor): Promise<NormalizedIndexResult>;
|
|
487
511
|
|
|
512
|
+
declare function isInjectedCause(cause: WakeCause): boolean;
|
|
513
|
+
/** A line nobody typed: an injected cause, or a headless `sdk` prompt unless the caller keeps those. */
|
|
514
|
+
declare function isUntypedPrompt(inbound: Pick<Inbound, "cause" | "promptSource">, indexSdkPrompts?: boolean): boolean;
|
|
515
|
+
/** Prompt text with every injected block removed; empty when nothing the human wrote is left. */
|
|
516
|
+
declare function stripInjected(text: string): string;
|
|
517
|
+
|
|
488
518
|
interface IndexedSpan {
|
|
489
519
|
sourceId: number;
|
|
490
520
|
byteOffset: number;
|
|
@@ -520,4 +550,4 @@ declare function normalizedUsage(graph: SessionGraph, ref: string): NormalizedUs
|
|
|
520
550
|
/** Ambiguity is explicit; workspace session bodies remain in their original namespace. */
|
|
521
551
|
declare function resolveConversationAlias(db: Db, legacyRef: string): string | null;
|
|
522
552
|
|
|
523
|
-
export { AUDIT_DDL, AUDIT_FACET, AUDIT_MIGRATION_NAME, AUDIT_TABLES, type BackfillOptions, type BackfillSummary, type ConversationSummary, DEFAULT_FACET_LIMIT, DERIVED_TABLES, DOMAIN_DDL, type DeltaSource, EPISODE_TABLE, type EpisodeRow, type ExternalEvent, FACET_TABLE, type IndexOptions, type IndexedSpan, KIT, MIGRATIONS, NO_ENRICHMENT, NO_ORIGINS, NO_PR_OUTCOMES, NO_TASK_LINK, type NormalizedIndexResult, type NormalizedUsageSummary, ORIGIN_DDL, ORIGIN_MIGRATION_NAME, ORIGIN_TABLES, ORIGIN_TASK_LINK_MIGRATION_NAME, ORIGIN_VIEWS, type OpenSessionGraphOptions, type OriginEnrichment, type OriginResolution, type OriginResolver, type PrEnrichment, type PrKey, type PrResolution, type PrResolver, type PriceInput, REVIEW_DDL, REVIEW_TABLE, REVIEW_VERDICT_MIGRATION_NAME, type ReconcileCounts, type RefreshOptions, type RefreshSummary, type ResolvedOrigin, type ResolvedPr, type ResolvedReview, type ResolvedTask, type ReviewProjection, type ReviewRoundOptions, type ReviewerProfilePredicate, type SessionGraph, type SyncPricesOptions, type TaskEnrichment, type TaskLinkSource, type TaskResolution, type TaskResolver, type TranscriptOutcome, allSessionIds, allTaskIds, applyAudit, applyDelta, backfillFacets, countRounds, enrichPrs, enrichTasks, indexCodexSource, indexTranscript, isReviewerProfile, normalizedSessions, normalizedUsage, openSessionGraph, projectReviewRounds, prsNeedingOutcome, purgeTranscript, readIndexedText, reconcile, refreshCorpus, replaceEpisodes, resetIndex, resolveConversationAlias, resolveOrigins, rollupSessions, sessionsNeedingOrigin, syncPrices };
|
|
553
|
+
export { AUDIT_DDL, AUDIT_FACET, AUDIT_MIGRATION_NAME, AUDIT_TABLES, type BackfillOptions, type BackfillSummary, type ConversationSummary, DEFAULT_FACET_LIMIT, DERIVED_TABLES, DOMAIN_DDL, type DeltaSource, EPISODE_TABLE, type EpisodeRow, type ExternalEvent, FACET_TABLE, type IndexOptions, type IndexedSpan, KIT, MIGRATIONS, NO_ENRICHMENT, NO_ORIGINS, NO_PR_OUTCOMES, NO_TASK_LINK, type NormalizedIndexResult, type NormalizedUsageSummary, ORIGIN_DDL, ORIGIN_MIGRATION_NAME, ORIGIN_TABLES, ORIGIN_TASK_LINK_MIGRATION_NAME, ORIGIN_VIEWS, type OpenSessionGraphOptions, type OriginEnrichment, type OriginResolution, type OriginResolver, type PrEnrichment, type PrKey, type PrResolution, type PrResolver, type PriceInput, REVIEW_DDL, REVIEW_TABLE, REVIEW_VERDICT_MIGRATION_NAME, type ReconcileCounts, type ReconcileResult, type RefreshOptions, type RefreshSummary, type ResolvedOrigin, type ResolvedPr, type ResolvedReview, type ResolvedTask, type ReviewProjection, type ReviewRoundOptions, type ReviewerProfilePredicate, type SessionGraph, SessionGraphNotMigratedError, type SyncPricesOptions, type TaskEnrichment, type TaskLinkSource, type TaskResolution, type TaskResolver, type TranscriptOutcome, allSessionIds, allTaskIds, applyAudit, applyDelta, backfillFacets, countRounds, enrichPrs, enrichTasks, indexCodexSource, indexTranscript, isInjectedCause, isReviewerProfile, isUntypedPrompt, normalizedSessions, normalizedUsage, openSessionGraph, projectReviewRounds, prsNeedingOutcome, purgeTranscript, readIndexedText, reconcile, reconcilePrices, refreshCorpus, replaceEpisodes, resetIndex, resolveConversationAlias, resolveOrigins, rollupSessions, sessionsNeedingOrigin, stripInjected, syncPrices };
|
package/dist/index.js
CHANGED
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
// src/graph.ts
|
|
2
|
-
import { EdgeTable, SpanFtsTables, WatermarkTable, openDatabase, runMigrations } from "@titan-design/store-sqlite";
|
|
2
|
+
import { EdgeTable, MIGRATION_TABLE_NAME, SpanFtsTables, WatermarkTable, hasTable, openDatabase, runMigrations } from "@titan-design/store-sqlite";
|
|
3
3
|
|
|
4
4
|
// src/audit-schema.ts
|
|
5
5
|
var AUDIT_MIGRATION_NAME = "audit tables";
|
|
@@ -511,9 +511,18 @@ var MIGRATIONS = [
|
|
|
511
511
|
];
|
|
512
512
|
|
|
513
513
|
// src/graph.ts
|
|
514
|
+
var SessionGraphNotMigratedError = class extends Error {
|
|
515
|
+
constructor(missing) {
|
|
516
|
+
super(`the graph does not carry session-graph migrations ${missing.join(", ")}; let its owner migrate it before opening it read-only`);
|
|
517
|
+
this.missing = missing;
|
|
518
|
+
this.name = "SessionGraphNotMigratedError";
|
|
519
|
+
}
|
|
520
|
+
missing;
|
|
521
|
+
};
|
|
514
522
|
function openSessionGraph(dbPath, options = {}) {
|
|
515
|
-
const db = openDatabase(dbPath, { schemaVersion: options.schemaVersion });
|
|
516
|
-
|
|
523
|
+
const db = openDatabase(dbPath, { schemaVersion: options.schemaVersion, readonly: options.readonly });
|
|
524
|
+
if (options.readonly) assertMigratedOrClose(db);
|
|
525
|
+
else runMigrations(db, MIGRATIONS);
|
|
517
526
|
return {
|
|
518
527
|
db,
|
|
519
528
|
transcripts: new WatermarkTable(db, { name: KIT.watermark }),
|
|
@@ -521,6 +530,18 @@ function openSessionGraph(dbPath, options = {}) {
|
|
|
521
530
|
spans: new SpanFtsTables(db, { name: KIT.spanFts })
|
|
522
531
|
};
|
|
523
532
|
}
|
|
533
|
+
function assertMigratedOrClose(db) {
|
|
534
|
+
const applied = appliedNames(db);
|
|
535
|
+
const missing = MIGRATIONS.filter((m) => applied.get(m.version) !== m.name).map((m) => `${m.version} (${m.name})`);
|
|
536
|
+
if (missing.length === 0) return;
|
|
537
|
+
db.close();
|
|
538
|
+
throw new SessionGraphNotMigratedError(missing);
|
|
539
|
+
}
|
|
540
|
+
function appliedNames(db) {
|
|
541
|
+
if (!hasTable(db, MIGRATION_TABLE_NAME)) return /* @__PURE__ */ new Map();
|
|
542
|
+
const rows = db.prepare(`SELECT version, name FROM ${MIGRATION_TABLE_NAME}`).all();
|
|
543
|
+
return new Map(rows.map((r) => [r.version, r.name]));
|
|
544
|
+
}
|
|
524
545
|
function resetIndex(graph) {
|
|
525
546
|
graph.db.transaction(() => {
|
|
526
547
|
for (const table of DERIVED_TABLES) graph.db.exec(`DELETE FROM "${table}"`);
|
|
@@ -667,6 +688,89 @@ function advanceFacet(db, transcriptId, fromByteOffset, indexedTo, version = EXT
|
|
|
667
688
|
db.prepare(`UPDATE ${FACET_TABLE} SET indexed_to = ? WHERE transcript_id = ? AND facet = ? AND version = ?`).run(indexedTo, transcriptId, AUDIT_FACET, version);
|
|
668
689
|
}
|
|
669
690
|
|
|
691
|
+
// src/injected-text.ts
|
|
692
|
+
import { hasMarker, INJECTED_MARKERS } from "@titan-design/session-read";
|
|
693
|
+
var INJECTED_CAUSES = /* @__PURE__ */ new Set([
|
|
694
|
+
"compaction",
|
|
695
|
+
"channel_message",
|
|
696
|
+
"channel_system",
|
|
697
|
+
"task_notification",
|
|
698
|
+
"scheduled_wakeup",
|
|
699
|
+
"usage_limit_resume",
|
|
700
|
+
"harness_resume",
|
|
701
|
+
"local_command",
|
|
702
|
+
"image_meta",
|
|
703
|
+
"hook_or_reminder"
|
|
704
|
+
]);
|
|
705
|
+
function isInjectedCause(cause) {
|
|
706
|
+
return INJECTED_CAUSES.has(cause);
|
|
707
|
+
}
|
|
708
|
+
function isUntypedPrompt(inbound, indexSdkPrompts = false) {
|
|
709
|
+
return isInjectedCause(inbound.cause) || !indexSdkPrompts && inbound.promptSource === "sdk";
|
|
710
|
+
}
|
|
711
|
+
var CUT_TAGS = [
|
|
712
|
+
...Object.values(INJECTED_MARKERS).flatMap((m) => m.close ? [m.close.slice(2, -1)] : []),
|
|
713
|
+
"user-prompt-submit-hook",
|
|
714
|
+
"command-name",
|
|
715
|
+
"command-message",
|
|
716
|
+
"local-command-caveat",
|
|
717
|
+
"local-command-stdout",
|
|
718
|
+
"local-command-stderr",
|
|
719
|
+
"bash-stdout",
|
|
720
|
+
"bash-stderr"
|
|
721
|
+
].join("|");
|
|
722
|
+
var CUT_OPEN = new RegExp(`<(${CUT_TAGS})(?=[\\s>])[^>]*>`, "g");
|
|
723
|
+
var LINE_END_OR_BLOCK = new RegExp(`^[ \\t]*(?:\\r?\\n|$|<(?:${CUT_TAGS})(?=[\\s>]))`);
|
|
724
|
+
function opensBlock(text, at, keptFrom) {
|
|
725
|
+
const lead = text.slice(Math.max(text.lastIndexOf("\n", at - 1) + 1, keptFrom), at);
|
|
726
|
+
return lead.trim() === "";
|
|
727
|
+
}
|
|
728
|
+
function balancedClose(text, tag, from) {
|
|
729
|
+
const tags = new RegExp(`<(/?)${tag}(?=[\\s>])[^>]*>`, "g");
|
|
730
|
+
tags.lastIndex = from;
|
|
731
|
+
let depth = 1;
|
|
732
|
+
for (let m = tags.exec(text); m; m = tags.exec(text)) {
|
|
733
|
+
depth += m[1] ? -1 : 1;
|
|
734
|
+
if (depth === 0) return tags.lastIndex;
|
|
735
|
+
}
|
|
736
|
+
return null;
|
|
737
|
+
}
|
|
738
|
+
function closesLine(text, at) {
|
|
739
|
+
return LINE_END_OR_BLOCK.test(text.slice(at));
|
|
740
|
+
}
|
|
741
|
+
function cutBlocks(text) {
|
|
742
|
+
let kept = "";
|
|
743
|
+
let keptFrom = 0;
|
|
744
|
+
CUT_OPEN.lastIndex = 0;
|
|
745
|
+
for (let open = CUT_OPEN.exec(text); open; open = CUT_OPEN.exec(text)) {
|
|
746
|
+
if (!opensBlock(text, open.index, keptFrom)) continue;
|
|
747
|
+
const end = balancedClose(text, open[1], CUT_OPEN.lastIndex);
|
|
748
|
+
if (end === null || !closesLine(text, end)) continue;
|
|
749
|
+
kept += text.slice(keptFrom, open.index);
|
|
750
|
+
keptFrom = end;
|
|
751
|
+
CUT_OPEN.lastIndex = end;
|
|
752
|
+
}
|
|
753
|
+
return kept + text.slice(keptFrom);
|
|
754
|
+
}
|
|
755
|
+
var INTERRUPTION = /\[Request interrupted by user[^\]\n]*\]/g;
|
|
756
|
+
var HUMAN_WRAPPERS = ["command-args", "bash-input"].map((tag) => new RegExp(`</?${tag}>`, "g"));
|
|
757
|
+
var WHOLE_TURN_MARKERS = Object.keys(INJECTED_MARKERS).filter((name) => INJECTED_MARKERS[name].close === null);
|
|
758
|
+
var SPAWN_BRIEF_FRAMING = [
|
|
759
|
+
/^# Predecessor: .*\n\nYou are taking over from /,
|
|
760
|
+
/Commit your work there; nothing outside it is yours to change\.\s*$/,
|
|
761
|
+
/You may be sharing it with other agents, so stay inside the paths you were given\.\s*$/
|
|
762
|
+
];
|
|
763
|
+
function isWholeTurnInjected(text) {
|
|
764
|
+
const trimmed = text.trim();
|
|
765
|
+
return WHOLE_TURN_MARKERS.some((name) => hasMarker(trimmed, name)) || SPAWN_BRIEF_FRAMING.some((re) => re.test(trimmed));
|
|
766
|
+
}
|
|
767
|
+
function stripInjected(text) {
|
|
768
|
+
let kept = cutBlocks(text);
|
|
769
|
+
for (const re of [INTERRUPTION, ...HUMAN_WRAPPERS]) kept = kept.replace(re, "");
|
|
770
|
+
if (isWholeTurnInjected(kept)) return "";
|
|
771
|
+
return kept.replace(/[ \t]+\n/g, "\n").replace(/\n{3,}/g, "\n\n").trim();
|
|
772
|
+
}
|
|
773
|
+
|
|
670
774
|
// src/apply.ts
|
|
671
775
|
import { RELATIONS, sessionRef } from "@titan-design/session-read";
|
|
672
776
|
function applyDelta(graph, transcriptId, delta, source = {}) {
|
|
@@ -677,6 +781,7 @@ function applyDelta(graph, transcriptId, delta, source = {}) {
|
|
|
677
781
|
applyAssets(graph.db, delta);
|
|
678
782
|
applyPhases(graph.db, transcriptId, delta);
|
|
679
783
|
applyLinkedRows(graph, transcriptId, delta);
|
|
784
|
+
applySpans(graph, transcriptId, delta, source.indexSdkPrompts ?? false);
|
|
680
785
|
applyReviewVerdicts(graph.db, transcriptId, delta);
|
|
681
786
|
applyAudit(graph.db, transcriptId, delta);
|
|
682
787
|
})();
|
|
@@ -815,8 +920,14 @@ function applyLinkedRows(graph, transcriptId, delta) {
|
|
|
815
920
|
const linkTranscript = db.prepare("INSERT INTO subagent (agent_ref, child_session_id) VALUES (?, ?) ON CONFLICT (agent_ref) DO UPDATE SET child_session_id = excluded.child_session_id");
|
|
816
921
|
for (const l of delta.subagentTranscripts) linkTranscript.run(l.agentRef, l.childSessionId);
|
|
817
922
|
for (const e of delta.edges) assertTranscriptEdge(graph, { sourceRef: e.sourceRef, relation: e.relation, targetRef: e.targetRef, tValid: e.ts, factId: factId(e.byteOffset) });
|
|
923
|
+
}
|
|
924
|
+
function applySpans(graph, transcriptId, delta, indexSdkPrompts) {
|
|
925
|
+
const injectedLines = new Set(delta.inbound.filter((i) => isUntypedPrompt(i, indexSdkPrompts)).map((i) => i.byteOffset));
|
|
818
926
|
for (const span of delta.spans) {
|
|
819
|
-
|
|
927
|
+
const isPrompt = span.field === "prompt";
|
|
928
|
+
const text = !isPrompt ? span.text : injectedLines.has(span.byteOffset) ? "" : stripInjected(span.text);
|
|
929
|
+
if (isPrompt && !text) continue;
|
|
930
|
+
graph.spans.index({ ownerRef: sessionRef(span.sessionId), field: span.field, sourceId: transcriptId, byteOffset: span.byteOffset, byteLength: span.byteLength }, text);
|
|
820
931
|
}
|
|
821
932
|
}
|
|
822
933
|
function assertTranscriptEdge(graph, edge) {
|
|
@@ -1403,7 +1514,7 @@ async function indexTranscript(graph, transcript, options = {}) {
|
|
|
1403
1514
|
});
|
|
1404
1515
|
const rewound = point.state === "rewritten" || delta.restartedFromZero;
|
|
1405
1516
|
if (rewound) purgeTranscript(graph, row.sourceId);
|
|
1406
|
-
applyDelta(graph, row.sourceId, delta, { account: transcript.account });
|
|
1517
|
+
applyDelta(graph, row.sourceId, delta, { account: transcript.account, indexSdkPrompts: options.indexSdkPrompts });
|
|
1407
1518
|
advanceFacet(graph.db, row.sourceId, rewound ? 0 : point.start, delta.lastByteOffset);
|
|
1408
1519
|
const stat2 = await fs.stat(transcript.absolutePath);
|
|
1409
1520
|
graph.transcripts.advance(transcript.displayPath, {
|
|
@@ -1523,6 +1634,48 @@ function syncPrices(graph, rows, options) {
|
|
|
1523
1634
|
})();
|
|
1524
1635
|
return rows.length;
|
|
1525
1636
|
}
|
|
1637
|
+
function matchesRow(stored, r, version, source) {
|
|
1638
|
+
return stored.table_version === version && stored.source === source && [stored.input_usd_mtok, stored.cache_read_usd_mtok, stored.cache_write_5m_usd_mtok, stored.cache_write_1h_usd_mtok, stored.output_usd_mtok].every((v, i) => v === [r.input, r.cacheRead, r.cacheWrite5m, r.cacheWrite1h, r.output][i]);
|
|
1639
|
+
}
|
|
1640
|
+
function reconcilePrices(graph, rows, options) {
|
|
1641
|
+
const find = graph.db.prepare("SELECT * FROM price WHERE model = ? AND effective_from = ?");
|
|
1642
|
+
const upsert = graph.db.prepare(
|
|
1643
|
+
`INSERT INTO price (model, effective_from, table_version, input_usd_mtok, cache_read_usd_mtok,
|
|
1644
|
+
cache_write_5m_usd_mtok, cache_write_1h_usd_mtok, output_usd_mtok, source)
|
|
1645
|
+
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
|
|
1646
|
+
ON CONFLICT (model, effective_from) DO UPDATE SET table_version = excluded.table_version,
|
|
1647
|
+
input_usd_mtok = excluded.input_usd_mtok, cache_read_usd_mtok = excluded.cache_read_usd_mtok,
|
|
1648
|
+
cache_write_5m_usd_mtok = excluded.cache_write_5m_usd_mtok, cache_write_1h_usd_mtok = excluded.cache_write_1h_usd_mtok,
|
|
1649
|
+
output_usd_mtok = excluded.output_usd_mtok, source = excluded.source`
|
|
1650
|
+
);
|
|
1651
|
+
const source = options.source ?? null;
|
|
1652
|
+
const result = { added: 0, updated: 0, pruned: 0 };
|
|
1653
|
+
graph.db.transaction(() => {
|
|
1654
|
+
for (const r of rows) {
|
|
1655
|
+
const stored = find.get(r.modelPrefix, r.effectiveFrom);
|
|
1656
|
+
if (stored && stored.table_version > options.tableVersion) continue;
|
|
1657
|
+
if (stored && matchesRow(stored, r, options.tableVersion, source)) continue;
|
|
1658
|
+
upsert.run(r.modelPrefix, r.effectiveFrom, options.tableVersion, r.input, r.cacheRead, r.cacheWrite5m, r.cacheWrite1h, r.output, source);
|
|
1659
|
+
if (stored) result.updated++;
|
|
1660
|
+
else result.added++;
|
|
1661
|
+
}
|
|
1662
|
+
if (source !== null) result.pruned = pruneAbsent(graph, rows, source, options.tableVersion);
|
|
1663
|
+
})();
|
|
1664
|
+
return result;
|
|
1665
|
+
}
|
|
1666
|
+
function pruneAbsent(graph, rows, source, tableVersion) {
|
|
1667
|
+
const owned = graph.db.prepare("SELECT model, effective_from, table_version FROM price WHERE source = ?").all(source);
|
|
1668
|
+
if (owned.some((o) => o.table_version > tableVersion)) return 0;
|
|
1669
|
+
const keep = new Set(rows.map((r) => `${r.modelPrefix}\0${r.effectiveFrom}`));
|
|
1670
|
+
const drop = graph.db.prepare("DELETE FROM price WHERE model = ? AND effective_from = ?");
|
|
1671
|
+
let pruned = 0;
|
|
1672
|
+
for (const o of owned) {
|
|
1673
|
+
if (keep.has(`${o.model}\0${o.effective_from}`)) continue;
|
|
1674
|
+
drop.run(o.model, o.effective_from);
|
|
1675
|
+
pruned++;
|
|
1676
|
+
}
|
|
1677
|
+
return pruned;
|
|
1678
|
+
}
|
|
1526
1679
|
|
|
1527
1680
|
// src/normalized-index.ts
|
|
1528
1681
|
import { randomUUID } from "crypto";
|
|
@@ -1556,7 +1709,8 @@ function insertObservation(graph, transcriptId, o) {
|
|
|
1556
1709
|
);
|
|
1557
1710
|
}
|
|
1558
1711
|
function observationText(o) {
|
|
1559
|
-
if (o.kind === "message") return o.content.map((p) => ({ field:
|
|
1712
|
+
if (o.kind === "message" && o.role === "user") return o.content.map((p) => ({ field: "prompt", text: stripInjected(p.text).slice(0, SPAN_TEXT_CAP), locator: p.locator })).filter((p) => p.text);
|
|
1713
|
+
if (o.kind === "message") return o.content.map((p) => ({ field: "assistant_response", text: p.text.slice(0, SPAN_TEXT_CAP), locator: p.locator }));
|
|
1560
1714
|
if (o.kind === "tool_call" && o.inputLocator) return [{ field: "tool_input", text: normalizedSearchText(o.input), locator: o.inputLocator }];
|
|
1561
1715
|
if (o.kind === "tool_result" && o.outputLocator) return [{ field: "tool_result", text: normalizedSearchText(o.output), locator: o.outputLocator }];
|
|
1562
1716
|
return [];
|
|
@@ -1659,6 +1813,10 @@ function* stagedRows(graph, table) {
|
|
|
1659
1813
|
import { contentHash as contentHash3, prefixHash } from "@titan-design/locator";
|
|
1660
1814
|
import { readSessionSourceText, readSessionText, SessionUsageAccumulator } from "@titan-design/session-read";
|
|
1661
1815
|
async function readIndexedText(graph, span) {
|
|
1816
|
+
const text = await readSourceText(graph, span);
|
|
1817
|
+
return text !== null && span.field === "prompt" ? stripInjected(text) : text;
|
|
1818
|
+
}
|
|
1819
|
+
async function readSourceText(graph, span) {
|
|
1662
1820
|
const row = graph.db.prepare(`SELECT n.locators FROM normalized_span n JOIN search_span s USING(span_id)
|
|
1663
1821
|
WHERE s.source_id = ? AND s.byte_offset = ? AND s.field = ?`).get(span.sourceId, span.byteOffset, span.field);
|
|
1664
1822
|
if (row) {
|
|
@@ -1756,6 +1914,7 @@ export {
|
|
|
1756
1914
|
REVIEW_DDL,
|
|
1757
1915
|
REVIEW_TABLE,
|
|
1758
1916
|
REVIEW_VERDICT_MIGRATION_NAME,
|
|
1917
|
+
SessionGraphNotMigratedError,
|
|
1759
1918
|
allSessionIds,
|
|
1760
1919
|
allTaskIds,
|
|
1761
1920
|
applyAudit,
|
|
@@ -1766,7 +1925,9 @@ export {
|
|
|
1766
1925
|
enrichTasks,
|
|
1767
1926
|
indexCodexSource,
|
|
1768
1927
|
indexTranscript,
|
|
1928
|
+
isInjectedCause,
|
|
1769
1929
|
isReviewerProfile,
|
|
1930
|
+
isUntypedPrompt,
|
|
1770
1931
|
normalizedSessions,
|
|
1771
1932
|
normalizedUsage,
|
|
1772
1933
|
openSessionGraph,
|
|
@@ -1775,6 +1936,7 @@ export {
|
|
|
1775
1936
|
purgeTranscript,
|
|
1776
1937
|
readIndexedText,
|
|
1777
1938
|
reconcile,
|
|
1939
|
+
reconcilePrices,
|
|
1778
1940
|
refreshCorpus,
|
|
1779
1941
|
replaceEpisodes,
|
|
1780
1942
|
resetIndex,
|
|
@@ -1782,6 +1944,7 @@ export {
|
|
|
1782
1944
|
resolveOrigins,
|
|
1783
1945
|
rollupSessions,
|
|
1784
1946
|
sessionsNeedingOrigin,
|
|
1947
|
+
stripInjected,
|
|
1785
1948
|
syncPrices
|
|
1786
1949
|
};
|
|
1787
1950
|
//# sourceMappingURL=index.js.map
|