@kaelio/ktx 0.7.0 → 0.9.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (142) hide show
  1. package/assets/python/{kaelio_ktx-0.7.0-py3-none-any.whl → kaelio_ktx-0.9.0-py3-none-any.whl} +0 -0
  2. package/assets/python/manifest.json +4 -4
  3. package/dist/.tsbuildinfo +1 -1
  4. package/dist/cli-program.js +7 -0
  5. package/dist/cli-runtime.js +50 -3
  6. package/dist/command-schemas.d.ts +1 -1
  7. package/dist/command-tree.js +5 -1
  8. package/dist/commands/completion-commands.d.ts +3 -0
  9. package/dist/commands/completion-commands.js +38 -0
  10. package/dist/commands/ingest-commands.js +0 -4
  11. package/dist/commands/knowledge-commands.js +15 -2
  12. package/dist/commands/setup-commands.js +3 -3
  13. package/dist/commands/sl-commands.js +19 -7
  14. package/dist/completion/complete-engine.d.ts +19 -0
  15. package/dist/completion/complete-engine.js +128 -0
  16. package/dist/completion/completion-scripts.d.ts +1 -0
  17. package/dist/completion/completion-scripts.js +36 -0
  18. package/dist/completion/dynamic-candidates.d.ts +6 -0
  19. package/dist/completion/dynamic-candidates.js +98 -0
  20. package/dist/connection-drivers.d.ts +3 -0
  21. package/dist/connection-drivers.js +17 -0
  22. package/dist/connection-recovery.d.ts +34 -0
  23. package/dist/connection-recovery.js +82 -0
  24. package/dist/connection.js +3 -1
  25. package/dist/context/ingest/adapters/historic-sql/bigquery-query-history-reader.js +71 -20
  26. package/dist/context/ingest/adapters/historic-sql/chunk-unified.js +2 -1
  27. package/dist/context/ingest/adapters/historic-sql/connection-dialect.d.ts +9 -0
  28. package/dist/context/ingest/adapters/historic-sql/connection-dialect.js +15 -4
  29. package/dist/context/ingest/adapters/historic-sql/pattern-inputs.js +8 -2
  30. package/dist/context/ingest/adapters/historic-sql/query-history-filter-picker.d.ts +29 -0
  31. package/dist/context/ingest/adapters/historic-sql/query-history-filter-picker.js +190 -0
  32. package/dist/context/ingest/adapters/historic-sql/scope-floor.d.ts +18 -0
  33. package/dist/context/ingest/adapters/historic-sql/scope-floor.js +229 -0
  34. package/dist/context/ingest/adapters/historic-sql/scope-membership.d.ts +8 -0
  35. package/dist/context/ingest/adapters/historic-sql/scope-membership.js +29 -0
  36. package/dist/context/ingest/adapters/historic-sql/snowflake-query-history-reader.js +68 -19
  37. package/dist/context/ingest/adapters/historic-sql/stage-unified.js +57 -50
  38. package/dist/context/ingest/adapters/historic-sql/types.d.ts +36 -3
  39. package/dist/context/ingest/adapters/historic-sql/types.js +14 -2
  40. package/dist/context/ingest/context-evidence/sqlite-context-evidence-store.d.ts +1 -1
  41. package/dist/context/ingest/ingest-bundle.runner.d.ts +8 -0
  42. package/dist/context/ingest/ingest-bundle.runner.js +72 -15
  43. package/dist/context/ingest/ingest-profile.d.ts +102 -0
  44. package/dist/context/ingest/ingest-profile.js +306 -0
  45. package/dist/context/ingest/isolated-diff/patch-integrator.js +75 -5
  46. package/dist/context/ingest/isolated-diff/work-unit-executor.js +25 -2
  47. package/dist/context/ingest/local-adapters.js +21 -4
  48. package/dist/context/ingest/local-bundle-runtime.js +4 -2
  49. package/dist/context/ingest/local-ingest.d.ts +1 -1
  50. package/dist/context/ingest/local-ingest.js +6 -4
  51. package/dist/context/ingest/memory-flow/events.js +2 -1
  52. package/dist/context/ingest/ports.d.ts +2 -0
  53. package/dist/context/ingest/reports.d.ts +3 -0
  54. package/dist/context/ingest/reports.js +10 -0
  55. package/dist/context/ingest/stages/stage-3-work-units.d.ts +3 -1
  56. package/dist/context/ingest/stages/stage-3-work-units.js +2 -0
  57. package/dist/context/ingest/stages/stage-4-reconciliation.d.ts +2 -1
  58. package/dist/context/ingest/stages/stage-4-reconciliation.js +1 -1
  59. package/dist/context/ingest/tools/tool-call-logger.d.ts +6 -0
  60. package/dist/context/ingest/tools/tool-call-logger.js +36 -1
  61. package/dist/context/llm/ai-sdk-runtime.js +32 -3
  62. package/dist/context/llm/claude-code-runtime.js +35 -2
  63. package/dist/context/llm/codex-exec-events.d.ts +20 -0
  64. package/dist/context/llm/codex-exec-events.js +155 -0
  65. package/dist/context/llm/codex-isolation.d.ts +3 -0
  66. package/dist/context/llm/codex-isolation.js +5 -0
  67. package/dist/context/llm/codex-mcp-runtime-server.d.ts +24 -0
  68. package/dist/context/llm/codex-mcp-runtime-server.js +51 -0
  69. package/dist/context/llm/codex-models.d.ts +2 -0
  70. package/dist/context/llm/codex-models.js +17 -0
  71. package/dist/context/llm/codex-runtime-config.d.ts +16 -0
  72. package/dist/context/llm/codex-runtime-config.js +19 -0
  73. package/dist/context/llm/codex-runtime.d.ts +37 -0
  74. package/dist/context/llm/codex-runtime.js +304 -0
  75. package/dist/context/llm/codex-sdk-runner.d.ts +21 -0
  76. package/dist/context/llm/codex-sdk-runner.js +63 -0
  77. package/dist/context/llm/local-config.d.ts +2 -0
  78. package/dist/context/llm/local-config.js +12 -1
  79. package/dist/context/llm/runtime-port.d.ts +25 -0
  80. package/dist/context/mcp/context-tools.d.ts +2 -1
  81. package/dist/context/mcp/context-tools.js +82 -15
  82. package/dist/context/mcp/server.js +4 -0
  83. package/dist/context/mcp/types.d.ts +15 -1
  84. package/dist/context/project/config.d.ts +3 -0
  85. package/dist/context/project/config.js +6 -2
  86. package/dist/context/project/driver-schemas.js +1 -1
  87. package/dist/context/search/discover.js +4 -3
  88. package/dist/context/sl/local-sl.d.ts +15 -0
  89. package/dist/context/sl/local-sl.js +30 -0
  90. package/dist/context/sql-analysis/http-sql-analysis-port.js +32 -2
  91. package/dist/context/sql-analysis/ports.d.ts +12 -2
  92. package/dist/context/tools/context-candidate-mark.tool.d.ts +2 -2
  93. package/dist/context/wiki/local-knowledge.d.ts +10 -0
  94. package/dist/context/wiki/local-knowledge.js +22 -0
  95. package/dist/context-build-view.d.ts +0 -3
  96. package/dist/context-build-view.js +5 -39
  97. package/dist/ingest.js +7 -10
  98. package/dist/io/buffered-command-io.d.ts +11 -0
  99. package/dist/io/buffered-command-io.js +28 -0
  100. package/dist/knowledge.d.ts +5 -0
  101. package/dist/knowledge.js +10 -1
  102. package/dist/llm/types.d.ts +1 -1
  103. package/dist/local-adapters.d.ts +10 -2
  104. package/dist/local-adapters.js +19 -3
  105. package/dist/next-steps.js +1 -2
  106. package/dist/progress-port-adapter.d.ts +6 -0
  107. package/dist/progress-port-adapter.js +18 -0
  108. package/dist/public-ingest-copy.js +1 -1
  109. package/dist/public-ingest.d.ts +20 -8
  110. package/dist/public-ingest.js +198 -61
  111. package/dist/scan.js +3 -1
  112. package/dist/setup-context.d.ts +2 -0
  113. package/dist/setup-context.js +138 -64
  114. package/dist/setup-databases.d.ts +17 -1
  115. package/dist/setup-databases.js +366 -326
  116. package/dist/setup-models.d.ts +10 -1
  117. package/dist/setup-models.js +90 -2
  118. package/dist/setup-ready-menu.d.ts +16 -2
  119. package/dist/setup-ready-menu.js +37 -5
  120. package/dist/setup-sources.js +141 -33
  121. package/dist/setup.js +24 -12
  122. package/dist/skills/analytics/SKILL.md +6 -1
  123. package/dist/sl.d.ts +6 -1
  124. package/dist/sl.js +32 -8
  125. package/dist/status-project.d.ts +11 -0
  126. package/dist/status-project.js +50 -1
  127. package/dist/telemetry/command-hook.d.ts +1 -0
  128. package/dist/telemetry/command-hook.js +3 -1
  129. package/dist/telemetry/emitter.js +1 -1
  130. package/dist/telemetry/events.d.ts +15 -9
  131. package/dist/telemetry/events.js +17 -5
  132. package/dist/telemetry/identity.d.ts +1 -2
  133. package/dist/telemetry/identity.js +13 -10
  134. package/dist/telemetry/index.d.ts +13 -1
  135. package/dist/telemetry/index.js +18 -3
  136. package/dist/telemetry/scrubber.d.ts +10 -0
  137. package/dist/telemetry/scrubber.js +20 -0
  138. package/package.json +20 -19
  139. package/dist/ingest-depth.d.ts +0 -8
  140. package/dist/ingest-depth.js +0 -56
  141. package/dist/setup-database-context-depth.d.ts +0 -23
  142. package/dist/setup-database-context-depth.js +0 -84
@@ -39,7 +39,7 @@ export interface AnthropicModelChoice {
39
39
  label: string;
40
40
  recommended: boolean;
41
41
  }
42
- export type KtxSetupLlmBackend = 'anthropic' | 'vertex' | 'claude-code';
42
+ export type KtxSetupLlmBackend = 'anthropic' | 'vertex' | 'claude-code' | 'codex';
43
43
  /** @internal */
44
44
  export interface KtxSetupModelPromptAdapter {
45
45
  select(options: {
@@ -76,6 +76,15 @@ export interface KtxSetupModelDeps {
76
76
  ok: false;
77
77
  message: string;
78
78
  }>;
79
+ codexAuthProbe?: (input: {
80
+ projectDir: string;
81
+ model: string;
82
+ }) => Promise<{
83
+ ok: true;
84
+ } | {
85
+ ok: false;
86
+ message: string;
87
+ }>;
79
88
  readGcloudProject?: () => Promise<string | undefined>;
80
89
  listGcloudProjects?: () => Promise<GcloudProjectChoice[]>;
81
90
  spinner?: () => KtxCliSpinner;
@@ -3,6 +3,9 @@ import { writeFile } from 'node:fs/promises';
3
3
  import { promisify } from 'node:util';
4
4
  import { resolveLocalKtxLlmConfig } from './context/llm/local-config.js';
5
5
  import { runClaudeCodeAuthProbe } from './context/llm/claude-code-runtime.js';
6
+ import { formatCodexIsolationWarning } from './context/llm/codex-isolation.js';
7
+ import { runCodexAuthProbe } from './context/llm/codex-runtime.js';
8
+ import { DEFAULT_CODEX_MODEL } from './context/llm/codex-models.js';
6
9
  import { resolveKtxConfigReference } from './context/core/config-reference.js';
7
10
  import { serializeKtxProjectConfig } from './context/project/config.js';
8
11
  import { loadKtxProject } from './context/project/project.js';
@@ -37,6 +40,19 @@ const CLAUDE_CODE_MODELS = [
37
40
  { id: 'opus', label: 'Claude Opus', recommended: false },
38
41
  { id: 'haiku', label: 'Claude Haiku', recommended: false },
39
42
  ];
43
+ // Curated Codex models from OpenAI's current lineup that work under both
44
+ // ChatGPT-account (subscription) and API-key auth. Intentionally omitted:
45
+ // the `*-codex` ids (e.g. gpt-5.3-codex, gpt-5.2-codex) are API-key-only and
46
+ // fail on ChatGPT-account auth, and gpt-5.3-codex-spark is a ChatGPT-Pro-only
47
+ // research preview. Codex resolves real availability per account at runtime
48
+ // (its binary remote-fetches the model list), so this is a convenience
49
+ // shortlist only — the manual-entry option accepts any id your account's
50
+ // `codex` picker exposes, and the auth probe reports an unsupported choice.
51
+ const CODEX_MODELS = [
52
+ { id: 'gpt-5.5', label: 'GPT-5.5', recommended: true },
53
+ { id: 'gpt-5.4', label: 'GPT-5.4', recommended: false },
54
+ { id: 'gpt-5.4-mini', label: 'GPT-5.4 mini', recommended: false },
55
+ ];
40
56
  const HIDDEN_ANTHROPIC_MODEL_PATTERNS = [
41
57
  /^claude-sonnet-4$/i,
42
58
  /^claude-opus-4$/i,
@@ -149,7 +165,10 @@ export function isKtxSetupLlmConfigReady(config) {
149
165
  if (resolved.backend === 'vertex') {
150
166
  return typeof resolved.vertex?.location === 'string' && resolved.vertex.location.trim().length > 0;
151
167
  }
152
- return resolved.backend === 'anthropic' || resolved.backend === 'gateway' || resolved.backend === 'claude-code';
168
+ return (resolved.backend === 'anthropic' ||
169
+ resolved.backend === 'gateway' ||
170
+ resolved.backend === 'claude-code' ||
171
+ resolved.backend === 'codex');
153
172
  }
154
173
  function hasUsableConfiguredLlm(config) {
155
174
  return isKtxSetupLlmConfigReady(config.llm);
@@ -162,6 +181,13 @@ function buildProjectLlmConfig(existing, provider, model) {
162
181
  promptCaching: existing.promptCaching,
163
182
  };
164
183
  }
184
+ if (provider.backend === 'codex') {
185
+ return {
186
+ provider: { backend: 'codex' },
187
+ models: { ...existing.models, default: model },
188
+ promptCaching: existing.promptCaching,
189
+ };
190
+ }
165
191
  if (provider.backend === 'vertex') {
166
192
  return {
167
193
  provider: {
@@ -340,6 +366,7 @@ async function chooseBackend(args, io, deps) {
340
366
  message: 'Which LLM provider should KTX use?',
341
367
  options: [
342
368
  { value: 'claude-code', label: 'Claude subscription (Pro/Max)' },
369
+ { value: 'codex', label: 'Codex subscription' },
343
370
  { value: 'anthropic', label: 'Anthropic API key' },
344
371
  { value: 'vertex', label: 'Google Vertex AI for Anthropic Claude' },
345
372
  { value: 'back', label: 'Back' },
@@ -350,7 +377,7 @@ async function chooseBackend(args, io, deps) {
350
377
  }
351
378
  return {
352
379
  status: 'ready',
353
- backend: choice === 'vertex' || choice === 'claude-code' ? choice : 'anthropic',
380
+ backend: choice === 'vertex' || choice === 'claude-code' || choice === 'codex' ? choice : 'anthropic',
354
381
  prompted: true,
355
382
  };
356
383
  }
@@ -671,6 +698,42 @@ async function chooseClaudeCodeModel(args, deps) {
671
698
  }
672
699
  return { status: 'ready', model: choice };
673
700
  }
701
+ async function chooseCodexModel(args, deps) {
702
+ const providedModel = requestedModel(args);
703
+ if (providedModel) {
704
+ return { status: 'ready', model: providedModel };
705
+ }
706
+ if (args.inputMode === 'disabled') {
707
+ return { status: 'ready', model: DEFAULT_CODEX_MODEL };
708
+ }
709
+ const prompts = deps.prompts ?? createPromptAdapter();
710
+ const choice = await prompts.select({
711
+ message: `Which Codex model should KTX use?\n\n${ANTHROPIC_MODEL_PROMPT_CONTEXT}`,
712
+ options: [
713
+ ...CODEX_MODELS.map((model) => ({
714
+ value: model.id,
715
+ label: model.label,
716
+ ...(model.recommended ? { hint: 'recommended' } : {}),
717
+ })),
718
+ { value: 'manual', label: 'Enter a Codex model ID manually' },
719
+ { value: 'back', label: 'Back' },
720
+ ],
721
+ });
722
+ if (choice === 'back') {
723
+ return { status: 'back' };
724
+ }
725
+ if (choice === 'manual') {
726
+ const manual = await prompts.text({
727
+ message: withTextInputNavigation('Codex model ID'),
728
+ placeholder: CODEX_MODELS.find((model) => model.recommended)?.id ?? CODEX_MODELS[0]?.id,
729
+ });
730
+ if (manual === undefined) {
731
+ return { status: 'back' };
732
+ }
733
+ return manual.trim() ? { status: 'ready', model: manual.trim() } : { status: 'missing-input' };
734
+ }
735
+ return { status: 'ready', model: choice };
736
+ }
674
737
  async function persistLlmConfig(projectDir, provider, model) {
675
738
  const project = await loadKtxProject({ projectDir });
676
739
  const config = {
@@ -783,6 +846,31 @@ export async function runKtxSetupAnthropicModelStep(args, io, deps = {}) {
783
846
  io.stdout.write(`│ LLM ready: yes (${model.model})\n`);
784
847
  return { status: 'ready', projectDir: args.projectDir };
785
848
  }
849
+ if (backendChoice.backend === 'codex') {
850
+ const model = await chooseCodexModel(backendArgs, deps);
851
+ if (model.status === 'back' && backendChoice.prompted) {
852
+ attemptArgs = buildInteractiveRetryArgs(args);
853
+ continue;
854
+ }
855
+ if (model.status === 'invalid-credential') {
856
+ return { status: 'failed', projectDir: args.projectDir };
857
+ }
858
+ if (model.status !== 'ready') {
859
+ return { status: model.status, projectDir: args.projectDir };
860
+ }
861
+ const probe = deps.codexAuthProbe ?? runCodexAuthProbe;
862
+ const health = await probe({ projectDir: args.projectDir, model: model.model });
863
+ if (!health.ok) {
864
+ io.stderr.write(`${health.message}\n`);
865
+ return { status: 'failed', projectDir: args.projectDir };
866
+ }
867
+ // Prefix the clack gutter so the warning sits inside the setup frame
868
+ // instead of breaking out of it; kept on stderr for scripted runs.
869
+ io.stderr.write(`│ ${formatCodexIsolationWarning()}\n`);
870
+ await persistLlmConfig(args.projectDir, { backend: 'codex' }, model.model);
871
+ io.stdout.write(`│ LLM ready: yes (codex, ${model.model})\n`);
872
+ return { status: 'ready', projectDir: args.projectDir };
873
+ }
786
874
  const credential = await chooseCredentialRef(backendArgs, io, deps);
787
875
  if (credential.status === 'back' && backendChoice.prompted) {
788
876
  attemptArgs = buildInteractiveRetryArgs(args);
@@ -1,6 +1,11 @@
1
1
  import { type KtxSetupPromptOption } from './setup-prompts.js';
2
2
  import type { KtxSetupStatus } from './setup.js';
3
3
  export type KtxSetupReadyAction = 'models' | 'embeddings' | 'databases' | 'sources' | 'runtime' | 'context' | 'agents' | 'exit';
4
+ /**
5
+ * Where a project stands once its `ktx.yaml` exists. Single source of truth for the
6
+ * end-of-setup interception: each state maps to exactly one obvious next action.
7
+ */
8
+ export type KtxSetupCompletion = 'incomplete' | 'needs-context' | 'needs-agents' | 'ready';
4
9
  interface KtxSetupReadyMenuPromptAdapter {
5
10
  select(options: {
6
11
  message: string;
@@ -11,8 +16,17 @@ interface KtxSetupReadyMenuPromptAdapter {
11
16
  export interface KtxSetupReadyMenuDeps {
12
17
  prompts?: KtxSetupReadyMenuPromptAdapter;
13
18
  }
14
- export declare function isKtxPreAgentSetupReady(status: KtxSetupStatus): boolean;
15
- export declare function isKtxSetupReady(status: KtxSetupStatus): boolean;
19
+ export declare function setupHasContextTargets(status: KtxSetupStatus): boolean;
20
+ export declare function classifyKtxSetupCompletion(status: KtxSetupStatus): KtxSetupCompletion;
21
+ /**
22
+ * Shown when a returning user re-runs `ktx setup` on a fully-ready project. Leads with
23
+ * "you're done" (the readiness note is printed by the caller first) and keeps the
24
+ * section editor one explicit step away rather than defaulting into it.
25
+ */
26
+ export declare function runKtxSetupReadyMenu(status: KtxSetupStatus, deps?: KtxSetupReadyMenuDeps): Promise<{
27
+ action: KtxSetupReadyAction;
28
+ }>;
29
+ /** @internal Reached only through {@link runKtxSetupReadyMenu}; exported for unit tests. */
16
30
  export declare function runKtxSetupReadyChangeMenu(status: KtxSetupStatus, deps?: KtxSetupReadyMenuDeps): Promise<{
17
31
  action: KtxSetupReadyAction;
18
32
  }>;
@@ -1,23 +1,55 @@
1
1
  import { createKtxSetupPromptAdapter, } from './setup-prompts.js';
2
- export function isKtxPreAgentSetupReady(status) {
2
+ export function setupHasContextTargets(status) {
3
+ return status.databases.length > 0 || status.sources.length > 0;
4
+ }
5
+ function setupConfigReady(status) {
3
6
  return (status.project.ready &&
4
7
  status.llm.ready &&
5
8
  status.embeddings.ready &&
6
9
  status.databases.every((database) => database.ready) &&
7
10
  status.sources.every((source) => source.ready) &&
8
11
  status.runtime.ready &&
9
- status.context.ready);
12
+ setupHasContextTargets(status));
10
13
  }
11
- export function isKtxSetupReady(status) {
12
- return isKtxPreAgentSetupReady(status) && status.agents.some((agent) => agent.ready);
14
+ export function classifyKtxSetupCompletion(status) {
15
+ if (!setupConfigReady(status)) {
16
+ return 'incomplete';
17
+ }
18
+ if (!status.context.ready) {
19
+ return 'needs-context';
20
+ }
21
+ if (!status.agents.some((agent) => agent.ready)) {
22
+ return 'needs-agents';
23
+ }
24
+ return 'ready';
13
25
  }
14
26
  function createPromptAdapter() {
15
27
  return createKtxSetupPromptAdapter({ selectCancelValue: 'exit' });
16
28
  }
29
+ /**
30
+ * Shown when a returning user re-runs `ktx setup` on a fully-ready project. Leads with
31
+ * "you're done" (the readiness note is printed by the caller first) and keeps the
32
+ * section editor one explicit step away rather than defaulting into it.
33
+ */
34
+ export async function runKtxSetupReadyMenu(status, deps = {}) {
35
+ const prompts = deps.prompts ?? createPromptAdapter();
36
+ const choice = await prompts.select({
37
+ message: 'Anything else?',
38
+ options: [
39
+ { value: 'done', label: "Done — I'll start using ktx" },
40
+ { value: 'change', label: 'Change a setting' },
41
+ ],
42
+ });
43
+ if (choice !== 'change') {
44
+ return { action: 'exit' };
45
+ }
46
+ return runKtxSetupReadyChangeMenu(status, { prompts });
47
+ }
48
+ /** @internal Reached only through {@link runKtxSetupReadyMenu}; exported for unit tests. */
17
49
  export async function runKtxSetupReadyChangeMenu(status, deps = {}) {
18
50
  const prompts = deps.prompts ?? createPromptAdapter();
19
51
  const action = (await prompts.select({
20
- message: `KTX is already set up for ${status.project.name ?? status.project.path}. What would you like to change?`,
52
+ message: 'What would you like to change?',
21
53
  options: [
22
54
  { value: 'models', label: 'Models' },
23
55
  { value: 'embeddings', label: 'Embeddings' },
@@ -19,6 +19,7 @@ import { markKtxSetupStateStepComplete } from './context/project/setup-config.js
19
19
  import { errorMessage, writePrefixedLines } from './clack.js';
20
20
  import { pickNotionRootPages } from './notion-page-picker.js';
21
21
  import { runKtxSourceMapping } from './source-mapping.js';
22
+ import { runConnectionSetupWithRecovery, } from './connection-recovery.js';
22
23
  import { withMultiselectNavigation, withTextInputNavigation } from './prompt-navigation.js';
23
24
  import { runKtxPublicIngest } from './public-ingest.js';
24
25
  import { writeProjectLocalSecretReference } from './setup-secrets.js';
@@ -114,6 +115,31 @@ function credentialRef(value, label) {
114
115
  }
115
116
  return ref;
116
117
  }
118
+ // Each connector reads exactly one credential ref; the flag name mirrors the
119
+ // ktx.yaml field it writes (auth_token_ref / api_key_ref / client_secret_ref).
120
+ const SOURCE_CREDENTIAL_FLAG = {
121
+ dbt: { field: 'sourceAuthTokenRef', flag: '--source-auth-token-ref' },
122
+ metricflow: { field: 'sourceAuthTokenRef', flag: '--source-auth-token-ref' },
123
+ lookml: { field: 'sourceAuthTokenRef', flag: '--source-auth-token-ref' },
124
+ notion: { field: 'sourceAuthTokenRef', flag: '--source-auth-token-ref' },
125
+ metabase: { field: 'sourceApiKeyRef', flag: '--source-api-key-ref' },
126
+ looker: { field: 'sourceClientSecretRef', flag: '--source-client-secret-ref' },
127
+ };
128
+ const ALL_SOURCE_CREDENTIAL_FLAGS = [
129
+ { field: 'sourceAuthTokenRef', flag: '--source-auth-token-ref' },
130
+ { field: 'sourceApiKeyRef', flag: '--source-api-key-ref' },
131
+ { field: 'sourceClientSecretRef', flag: '--source-client-secret-ref' },
132
+ ];
133
+ // Reject a credential ref flag the chosen source does not read, so a wrong flag
134
+ // fails loudly instead of being silently dropped (KLO-724).
135
+ function assertSourceCredentialFlags(source, args) {
136
+ const allowed = SOURCE_CREDENTIAL_FLAG[source];
137
+ for (const { field, flag } of ALL_SOURCE_CREDENTIAL_FLAGS) {
138
+ if (args[field] && field !== allowed.field) {
139
+ throw new Error(`${flag} does not apply to --source ${source}; use ${allowed.flag}.`);
140
+ }
141
+ }
142
+ }
117
143
  async function chooseSourceCredentialRef(input) {
118
144
  while (true) {
119
145
  const choice = await input.prompts.select({
@@ -385,7 +411,7 @@ function buildNotionConnection(args) {
385
411
  }
386
412
  return {
387
413
  driver: 'notion',
388
- auth_token_ref: credentialRef(args.sourceApiKeyRef, 'Notion token ref'),
414
+ auth_token_ref: credentialRef(args.sourceAuthTokenRef, 'Notion token ref'),
389
415
  crawl_mode: crawlMode,
390
416
  ...(rootPageIds.length > 0 ? { root_page_ids: rootPageIds } : {}),
391
417
  root_database_ids: [],
@@ -1064,11 +1090,11 @@ async function promptForInteractiveSource(args, source, prompts, io, deps, defau
1064
1090
  label: 'Notion integration token',
1065
1091
  envName: 'NOTION_TOKEN',
1066
1092
  secretFileName: `${currentState.sourceConnectionId ?? 'notion-main'}-token`,
1067
- existingRef: currentState.sourceApiKeyRef,
1093
+ existingRef: currentState.sourceAuthTokenRef,
1068
1094
  });
1069
1095
  if (ref === 'back')
1070
1096
  return 'back';
1071
- currentState.sourceApiKeyRef = ref;
1097
+ currentState.sourceAuthTokenRef = ref;
1072
1098
  return 'next';
1073
1099
  },
1074
1100
  async (currentState) => {
@@ -1096,7 +1122,7 @@ async function promptForInteractiveSource(args, source, prompts, io, deps, defau
1096
1122
  connectionId,
1097
1123
  connection: {
1098
1124
  driver: 'notion',
1099
- auth_token_ref: credentialRef(currentState.sourceApiKeyRef, 'Notion token ref'),
1125
+ auth_token_ref: credentialRef(currentState.sourceAuthTokenRef, 'Notion token ref'),
1100
1126
  crawl_mode: 'selected_roots',
1101
1127
  root_page_ids: currentState.notionRootPageIds ?? [],
1102
1128
  root_database_ids: [],
@@ -1260,7 +1286,7 @@ function sourceArgsFromExistingConnection(input) {
1260
1286
  }
1261
1287
  return sourceArgs;
1262
1288
  }
1263
- sourceArgs.sourceApiKeyRef = stringField(input.connection.auth_token_ref);
1289
+ sourceArgs.sourceAuthTokenRef = stringField(input.connection.auth_token_ref);
1264
1290
  sourceArgs.notionCrawlMode =
1265
1291
  input.connection.crawl_mode === 'all_accessible' ? 'all_accessible' : 'selected_roots';
1266
1292
  if (Array.isArray(input.connection.root_page_ids)) {
@@ -1409,56 +1435,125 @@ async function validateSource(source, args, deps) {
1409
1435
  }
1410
1436
  return await (deps.validateNotion ?? defaultValidateNotion)(args.connection);
1411
1437
  }
1412
- async function saveValidateAndMaybeBuildSource(input) {
1413
- const connectionId = input.sourceChoice.kind === 'existing'
1438
+ async function createSourceSetupRollback(projectDir) {
1439
+ const project = await loadKtxProject({ projectDir });
1440
+ const previousConfig = project.config;
1441
+ const configPath = project.configPath;
1442
+ return async () => {
1443
+ await writeFile(configPath, serializeKtxProjectConfig(previousConfig), 'utf-8');
1444
+ };
1445
+ }
1446
+ function sourceConnectionId(input) {
1447
+ return input.sourceChoice.kind === 'existing' || input.sourceChoice.kind === 'edited'
1414
1448
  ? input.sourceChoice.connectionId
1415
- : input.sourceChoice.kind === 'edited'
1416
- ? input.sourceChoice.connectionId
1417
- : (input.sourceChoice.args.sourceConnectionId ?? `${input.source}-main`);
1418
- const connection = input.sourceChoice.kind === 'existing'
1419
- ? input.sourceChoice.connection
1420
- : buildConnection(input.source, input.sourceChoice.args);
1421
- const rollback = input.sourceChoice.kind === 'existing'
1422
- ? undefined
1423
- : await writeSourceConnection(input.args.projectDir, connectionId, connection, sourceAdapter(input.source), input.io);
1424
- if (input.sourceChoice.kind === 'existing') {
1425
- await ensureSourceAdapterEnabled(input.args.projectDir, input.source);
1426
- }
1427
- const validation = await validateSource(input.source, { projectDir: input.args.projectDir, connectionId, connection }, input.deps);
1449
+ : (input.sourceChoice.args.sourceConnectionId ?? `${input.source}-main`);
1450
+ }
1451
+ async function validateSourceConnectionAndMapping(input) {
1452
+ const validation = await validateSource(input.source, { projectDir: input.args.projectDir, connectionId: input.connectionId, connection: input.connection }, input.deps);
1428
1453
  if (!validation.ok) {
1429
- await rollback?.();
1430
1454
  input.io.stderr.write(`${validation.message}\n`);
1431
1455
  return { status: 'failed' };
1432
1456
  }
1433
1457
  if (input.source === 'metabase' || input.source === 'looker') {
1434
- input.prompts.log?.(`Validating ${sourceLabel(input.source)} mapping…`);
1435
- const mappingCode = await (input.deps.runMapping ?? defaultRunMapping)(input.args.projectDir, connectionId, createSetupPrefixedIo(input.io));
1458
+ input.prompts.log?.(`Validating ${sourceLabel(input.source)} mapping...`);
1459
+ const mappingCode = await (input.deps.runMapping ?? defaultRunMapping)(input.args.projectDir, input.connectionId, createSetupPrefixedIo(input.io));
1436
1460
  if (mappingCode !== 0) {
1437
- await rollback?.();
1438
1461
  return { status: 'failed' };
1439
1462
  }
1440
1463
  }
1464
+ return { status: 'ok' };
1465
+ }
1466
+ async function saveValidateAndMaybeBuildSource(input) {
1467
+ let latestChoice = input.sourceChoice;
1468
+ let latestConnectionId = sourceConnectionId({ source: input.source, sourceChoice: latestChoice });
1469
+ let latestConnection = latestChoice.kind === 'existing'
1470
+ ? latestChoice.connection
1471
+ : buildConnection(input.source, latestChoice.args);
1472
+ let configureCount = 0;
1473
+ let rollbackAfterConfigure;
1474
+ const outcome = await runConnectionSetupWithRecovery({
1475
+ label: latestConnectionId,
1476
+ interactive: input.args.inputMode !== 'disabled',
1477
+ allowSkip: true,
1478
+ io: input.io,
1479
+ prompts: input.prompts,
1480
+ snapshot: async () => {
1481
+ rollbackAfterConfigure = await createSourceSetupRollback(input.args.projectDir);
1482
+ return rollbackAfterConfigure;
1483
+ },
1484
+ configure: async () => {
1485
+ configureCount += 1;
1486
+ if (latestChoice.kind === 'existing' && configureCount === 1) {
1487
+ await ensureSourceAdapterEnabled(input.args.projectDir, input.source);
1488
+ return 'configured';
1489
+ }
1490
+ const project = await loadKtxProject({ projectDir: input.args.projectDir });
1491
+ const currentConnection = project.config.connections[latestConnectionId] ?? latestConnection;
1492
+ const useAlreadyPromptedArgs = configureCount === 1 && latestChoice.kind !== 'existing';
1493
+ const sourceArgs = useAlreadyPromptedArgs && latestChoice.kind !== 'existing'
1494
+ ? latestChoice.args
1495
+ : input.args.inputMode === 'disabled'
1496
+ ? sourceArgsFromExistingConnection({
1497
+ args: input.args,
1498
+ source: input.source,
1499
+ connectionId: latestConnectionId,
1500
+ connection: currentConnection,
1501
+ })
1502
+ : await promptForInteractiveSource(sourceArgsFromExistingConnection({
1503
+ args: input.args,
1504
+ source: input.source,
1505
+ connectionId: latestConnectionId,
1506
+ connection: currentConnection,
1507
+ }), input.source, input.prompts, input.io, {
1508
+ pickNotionRootPages: input.deps.pickNotionRootPages,
1509
+ discoverMetabaseDatabases: input.deps.discoverMetabaseDatabases,
1510
+ }, latestConnectionId, input.deps.testGitRepo, input.deps.discoverMetabaseDatabases);
1511
+ if (sourceArgs === 'back') {
1512
+ return 'back';
1513
+ }
1514
+ latestConnectionId = sourceArgs.sourceConnectionId ?? latestConnectionId;
1515
+ latestConnection = buildConnection(input.source, sourceArgs);
1516
+ latestChoice =
1517
+ latestChoice.kind === 'new'
1518
+ ? { kind: 'new', args: sourceArgs }
1519
+ : { kind: 'edited', connectionId: latestConnectionId, args: sourceArgs };
1520
+ await writeSourceConnection(input.args.projectDir, latestConnectionId, latestConnection, sourceAdapter(input.source), input.io);
1521
+ return 'configured';
1522
+ },
1523
+ validate: () => validateSourceConnectionAndMapping({
1524
+ args: input.args,
1525
+ source: input.source,
1526
+ connectionId: latestConnectionId,
1527
+ connection: latestConnection,
1528
+ prompts: input.prompts,
1529
+ io: input.io,
1530
+ deps: input.deps,
1531
+ }),
1532
+ });
1533
+ if (outcome !== 'ready') {
1534
+ return { status: outcome };
1535
+ }
1441
1536
  if (input.args.runInitialSourceIngest) {
1442
1537
  const ingestResult = await runInitialSourceIngestWithRecovery({
1443
1538
  args: input.args,
1444
- connectionId,
1539
+ connectionId: latestConnectionId,
1445
1540
  io: input.io,
1446
1541
  prompts: input.prompts,
1447
1542
  deps: input.deps,
1448
1543
  });
1449
1544
  if (ingestResult === 'failed') {
1450
- await rollback?.();
1545
+ await rollbackAfterConfigure?.();
1451
1546
  return { status: 'failed' };
1452
1547
  }
1453
1548
  if (ingestResult === 'back') {
1454
- await rollback?.();
1549
+ await rollbackAfterConfigure?.();
1455
1550
  return { status: 'back' };
1456
1551
  }
1457
1552
  }
1458
1553
  else {
1459
- input.io.stdout.write(`│ Context source ${connectionId} saved. It will be built during the context build step.\n`);
1554
+ input.io.stdout.write(`│ Context source ${latestConnectionId} saved. It will be built during the context build step.\n`);
1460
1555
  }
1461
- return { status: 'ready', connectionId };
1556
+ return { status: 'ready', connectionId: latestConnectionId };
1462
1557
  }
1463
1558
  export async function runKtxSetupSourcesStep(args, io, deps = {}) {
1464
1559
  try {
@@ -1467,6 +1562,9 @@ export async function runKtxSetupSourcesStep(args, io, deps = {}) {
1467
1562
  io.stdout.write('│ Context source setup skipped.\n');
1468
1563
  return { status: 'skipped', projectDir: args.projectDir };
1469
1564
  }
1565
+ if (args.source) {
1566
+ assertSourceCredentialFlags(args.source, args);
1567
+ }
1470
1568
  const prompts = deps.prompts ?? createPromptAdapter();
1471
1569
  const project = await loadKtxProject({ projectDir: args.projectDir });
1472
1570
  if (!hasPrimarySource(project.config)) {
@@ -1551,8 +1649,13 @@ export async function runKtxSetupSourcesStep(args, io, deps = {}) {
1551
1649
  returnToSourceSelection = true;
1552
1650
  break;
1553
1651
  }
1554
- if (!readyConnectionIds.includes(choiceResult.connectionId)) {
1555
- readyConnectionIds.push(choiceResult.connectionId);
1652
+ if (choiceResult.status === 'skip') {
1653
+ continue;
1654
+ }
1655
+ if (choiceResult.status === 'ready') {
1656
+ if (!readyConnectionIds.includes(choiceResult.connectionId)) {
1657
+ readyConnectionIds.push(choiceResult.connectionId);
1658
+ }
1556
1659
  }
1557
1660
  }
1558
1661
  if (returnToSourceSelection) {
@@ -1612,8 +1715,13 @@ export async function runKtxSetupSourcesStep(args, io, deps = {}) {
1612
1715
  if (choiceResult.status === 'back') {
1613
1716
  continue;
1614
1717
  }
1615
- if (!readyConnectionIds.includes(choiceResult.connectionId)) {
1616
- readyConnectionIds.push(choiceResult.connectionId);
1718
+ if (choiceResult.status === 'skip') {
1719
+ continue;
1720
+ }
1721
+ if (choiceResult.status === 'ready') {
1722
+ if (!readyConnectionIds.includes(choiceResult.connectionId)) {
1723
+ readyConnectionIds.push(choiceResult.connectionId);
1724
+ }
1617
1725
  }
1618
1726
  continue;
1619
1727
  }
package/dist/setup.js CHANGED
@@ -1,12 +1,12 @@
1
1
  import { existsSync } from 'node:fs';
2
2
  import { basename, join, resolve } from 'node:path';
3
3
  import { getLatestLocalIngestStatus } from './context/ingest/local-ingest.js';
4
- import { savedMemoryCountsForReport } from './context/ingest/reports.js';
4
+ import { ingestReportOutcome, savedMemoryCountsForReport } from './context/ingest/reports.js';
5
5
  import { ktxLocalStateDbPath } from './context/project/local-state-db.js';
6
6
  import { loadKtxProject } from './context/project/project.js';
7
7
  import { readKtxSetupState } from './context/project/setup-config.js';
8
8
  import { getKtxCliPackageInfo } from './cli-runtime.js';
9
- import { formatSetupNextStepLines } from './next-steps.js';
9
+ import { formatNextStepLines, formatSetupNextStepLines } from './next-steps.js';
10
10
  import { runtimeInstallPolicyFromFlags } from './managed-python-command.js';
11
11
  import { readManagedPythonRuntimeStatus } from './managed-python-runtime.js';
12
12
  import { resolveProjectRuntimeRequirements } from './runtime-requirements.js';
@@ -16,7 +16,7 @@ import { runKtxSetupDatabasesStep, } from './setup-databases.js';
16
16
  import { runKtxSetupEmbeddingsStep } from './setup-embeddings.js';
17
17
  import { isKtxSetupLlmConfigReady, runKtxSetupAnthropicModelStep, } from './setup-models.js';
18
18
  import { runKtxSetupProjectStep } from './setup-project.js';
19
- import { isKtxPreAgentSetupReady, isKtxSetupReady, runKtxSetupReadyChangeMenu, } from './setup-ready-menu.js';
19
+ import { classifyKtxSetupCompletion, runKtxSetupReadyMenu, setupHasContextTargets, } from './setup-ready-menu.js';
20
20
  import { runKtxSetupSourcesStep } from './setup-sources.js';
21
21
  import { runKtxSetupRuntimeStep, } from './setup-runtime.js';
22
22
  import { createKtxSetupPromptAdapter, createKtxSetupUiAdapter, } from './setup-prompts.js';
@@ -47,6 +47,7 @@ async function recordSetupStep(input) {
47
47
  step: input.step,
48
48
  outcome: setupTelemetryOutcome(input.status),
49
49
  durationMs: Math.max(0, performance.now() - input.startedAt),
50
+ ...(input.errorDetail ? { errorDetail: input.errorDetail } : {}),
50
51
  },
51
52
  });
52
53
  }
@@ -105,7 +106,7 @@ function sourceConnections(config) {
105
106
  .sort((left, right) => left.connectionId.localeCompare(right.connectionId));
106
107
  }
107
108
  function reportHasSavedContext(report) {
108
- if (report.body.failedWorkUnits.length > 0) {
109
+ if (ingestReportOutcome(report) === 'error') {
109
110
  return false;
110
111
  }
111
112
  const counts = savedMemoryCountsForReport(report);
@@ -286,9 +287,6 @@ function setupStatusReady(status) {
286
287
  status.sources.every((source) => source.ready) &&
287
288
  status.runtime.ready);
288
289
  }
289
- function setupHasContextTargets(status) {
290
- return status.databases.length > 0 || status.sources.length > 0;
291
- }
292
290
  function setupContextReady(status) {
293
291
  return status.context.ready;
294
292
  }
@@ -371,14 +369,22 @@ async function runKtxSetupInner(args, io, deps = {}) {
371
369
  const currentStatus = await readKtxSetupStatus(projectResult.projectDir, { cliVersion: args.cliVersion });
372
370
  let readyAction;
373
371
  if (args.inputMode !== 'disabled' && !agentsRequested) {
374
- if (isKtxSetupReady(currentStatus)) {
375
- readyAction = (await runKtxSetupReadyChangeMenu(currentStatus, deps.readyMenuDeps)).action;
376
- if (readyAction === 'exit')
372
+ const completion = classifyKtxSetupCompletion(currentStatus);
373
+ if (completion === 'ready') {
374
+ setupUi.note(formatNextStepLines().join('\n'), 'ktx is ready', io);
375
+ const choice = (await runKtxSetupReadyMenu(currentStatus, deps.readyMenuDeps)).action;
376
+ if (choice === 'exit')
377
377
  return 0;
378
+ readyAction = choice;
379
+ }
380
+ else if (completion === 'needs-context') {
381
+ // Config is done; skip the re-walk and land straight on the build prompt.
382
+ readyAction = 'context';
378
383
  }
379
- else if (isKtxPreAgentSetupReady(currentStatus)) {
384
+ else if (completion === 'needs-agents') {
380
385
  readyAction = 'agents';
381
386
  }
387
+ // 'incomplete' → readyAction stays undefined → run the full setup walk.
382
388
  }
383
389
  const runOnly = readyAction;
384
390
  const agentOnlySetup = agentsRequested || runOnly === 'agents';
@@ -467,6 +473,9 @@ async function runKtxSetupInner(args, io, deps = {}) {
467
473
  const databaseResult = await databasesRunner({
468
474
  projectDir: projectResult.projectDir,
469
475
  inputMode: args.inputMode,
476
+ yes: args.yes,
477
+ cliVersion: args.cliVersion,
478
+ runtimeInstallPolicy: setupRuntimeInstallPolicy(args),
470
479
  ...(args.databaseDrivers ? { databaseDrivers: args.databaseDrivers } : {}),
471
480
  ...(args.databaseConnectionIds ? { databaseConnectionIds: args.databaseConnectionIds } : {}),
472
481
  ...(args.databaseConnectionId ? { databaseConnectionId: args.databaseConnectionId } : {}),
@@ -565,6 +574,7 @@ async function runKtxSetupInner(args, io, deps = {}) {
565
574
  startedAt: stepStartedAt,
566
575
  io,
567
576
  cliVersion: args.cliVersion,
577
+ ...(stepResult.errorDetail ? { errorDetail: stepResult.errorDetail } : {}),
568
578
  });
569
579
  if (stepResult.status === 'failed') {
570
580
  return 1;
@@ -589,7 +599,9 @@ async function runKtxSetupInner(args, io, deps = {}) {
589
599
  }
590
600
  if (step === 'context' && stepResult.status !== 'ready') {
591
601
  if (shouldRunAgents && args.skipAgents !== true) {
592
- return 0;
602
+ // Context isn't built, so skip agent install — but still reach the
603
+ // completion screen, which states readiness and points at `ktx ingest`.
604
+ break setupLoop;
593
605
  }
594
606
  }
595
607
  forcePromptSteps.delete(step);
@@ -28,7 +28,12 @@ You have access to KTX MCP tools for data discovery, semantic-layer analysis, ra
28
28
  - Read entity details before writing SQL against an unfamiliar table. Do not assume column names.
29
29
  - Treat `sql_execution` as read-only. Writes are rejected by the server.
30
30
  - Validate value mentions with `dictionary_search` instead of guessing case or spelling. Treat a `dictionary_search` miss as non-authoritative. The index is built from profile-sampled values, so a missing value may simply have been outside the sample. Follow up with `sql_execution` against the most plausible columns before concluding the value is absent.
31
- - When `connection_list` shows multiple connections, pass an explicit `connectionId` to every tool that takes one and where user intent pins a specific warehouse. Required: `entity_details`, `sl_read_source`, and `sql_execution`. Required when user intent is warehouse-specific, including wording like "in our warehouse" or "this warehouse": `memory_ingest`; without `connectionId`, the memory agent cannot update the semantic layer and the knowledge lands as wiki-only. Pass `connectionId` when intent pins a warehouse, otherwise omit for unscoped discovery: `sl_query`, `discover_data`, and `dictionary_search`. Never pass `connectionId` to `connection_list`, `wiki_search`, `wiki_read`, or `memory_ingest_status`. If intent is ambiguous for a required-or-scoped tool, ask the user which warehouse before calling.
31
+ - `connectionId` scoping when `connection_list` shows multiple connections:
32
+ - Always pass it: `entity_details`, `sl_read_source`, `sql_execution`.
33
+ - Pass it when intent pins a warehouse, otherwise omit for unscoped discovery: `sl_query`, `discover_data`, `dictionary_search`.
34
+ - `memory_ingest`: pass it for warehouse-specific knowledge (e.g. "in our warehouse"); without it the memory lands as wiki-only and cannot update the semantic layer.
35
+ - Never pass it: `connection_list`, `wiki_search`, `wiki_read`, `memory_ingest_status`.
36
+ - If scoping is required but intent is ambiguous, ask which warehouse before calling.
32
37
  - Show compact result tables for small outputs. For broad results, summarize the top findings and mention the applied limit.
33
38
  - Ask a concise clarification only when the metric, date range, entity, or grain is genuinely ambiguous and cannot be inferred from context.
34
39
  </rules>