oh-my-opencode 4.19.2 → 4.19.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.agents/command/get-unpublished-changes.md +2 -0
- package/.agents/command/omomomo.md +1 -1
- package/.agents/command/publish.md +102 -28
- package/.agents/skills/get-unpublished-changes/SKILL.md +2 -0
- package/.agents/skills/hyperplan/SKILL.md +3 -3
- package/.agents/skills/omomomo/SKILL.md +1 -1
- package/.agents/skills/publish/SKILL.md +27 -6
- package/.opencode/command/get-unpublished-changes.md +2 -0
- package/.opencode/command/omomomo.md +1 -1
- package/.opencode/command/publish.md +102 -28
- package/.opencode/skills/hyperplan/SKILL.md +3 -3
- package/THIRD-PARTY-NOTICES.md +7 -7
- package/dist/agents/sisyphus-junior/agent.d.ts +1 -1
- package/dist/agents/sisyphus-runtime-prompt-reconciler.d.ts +1 -1
- package/dist/cli/config-migrate.d.ts +8 -0
- package/dist/cli/doctor/checks/deprecated-reasoning-keys.d.ts +2 -0
- package/dist/cli/doctor/checks/legacy-config-leftovers.d.ts +10 -0
- package/dist/cli/doctor/checks/model-resolution-config.d.ts +1 -1
- package/dist/cli/doctor/checks/tools-lsp.d.ts +1 -1
- package/dist/cli/doctor/framework/constants.d.ts +1 -1
- package/dist/cli/index.js +95684 -92814
- package/dist/cli-node/index.js +95650 -92780
- package/dist/config/schema/agent-overrides.d.ts +800 -0
- package/dist/config/schema/categories.d.ts +132 -0
- package/dist/config/schema/fallback-models.d.ts +50 -0
- package/dist/config/schema/oh-my-opencode-config.d.ts +818 -2
- package/dist/config/validate.d.ts +2 -1
- package/dist/config-migration/deep-diff.d.ts +1 -0
- package/dist/config-migration/discovery-paths.d.ts +11 -0
- package/dist/config-migration/discovery-roots.d.ts +7 -0
- package/dist/config-migration/discovery.d.ts +4 -0
- package/dist/config-migration/index.d.ts +10 -0
- package/dist/config-migration/legacy-history.d.ts +3 -0
- package/dist/config-migration/migration-executor.d.ts +4 -0
- package/dist/config-migration/migration-plans.d.ts +15 -0
- package/dist/config-migration/reasoning-unification.d.ts +3 -0
- package/dist/config-migration/record-values.d.ts +4 -0
- package/dist/config-migration/schema-url.d.ts +1 -0
- package/dist/config-migration/transform-config-jsonc.d.ts +2 -0
- package/dist/config-migration/transform-opencode.d.ts +2 -0
- package/dist/config-migration/transform-types.d.ts +24 -0
- package/dist/config-migration/types.d.ts +39 -0
- package/dist/features/builtin-commands/templates/hyperplan.d.ts +1 -1
- package/dist/features/team-mode/tools/lifecycle-test-fixture.d.ts +2 -0
- package/dist/hooks/auto-update-checker/constants.d.ts +3 -3
- package/dist/hooks/codegraph-bootstrap/command-runner.d.ts +1 -0
- package/dist/hooks/model-fallback/next-fallback.d.ts +1 -0
- package/dist/hooks/runtime-fallback/constants.d.ts +1 -1
- package/dist/index.js +12368 -9788
- package/dist/oh-my-opencode.schema.json +1672 -8
- package/dist/plugin-config/omo-config-chain.d.ts +14 -0
- package/dist/plugin-config/unknown-key-diagnostics.d.ts +5 -0
- package/dist/plugin-config.d.ts +2 -2
- package/dist/plugin-handlers/prometheus-agent-config-builder.d.ts +1 -0
- package/dist/shared/agent-display-names.d.ts +1 -1
- package/dist/shared/agent-variant.d.ts +11 -0
- package/dist/shared/jsonc-parser.d.ts +1 -1
- package/dist/shared/migration.d.ts +5 -1
- package/dist/shared/opencode-config-dir-types.d.ts +0 -1
- package/dist/shared/plugin-identity.d.ts +6 -8
- package/dist/shared/project-discovery-dirs.d.ts +0 -1
- package/dist/shared/session-prompt-params-helpers.d.ts +6 -1
- package/dist/skills/coding-agent-sessions/SKILL.md +4 -3
- package/dist/skills/coding-agent-sessions/references/all-platforms.md +3 -1
- package/dist/skills/coding-agent-sessions/scripts/agent_sessions/aside_scanner.py +140 -0
- package/dist/skills/coding-agent-sessions/scripts/agent_sessions/scanners.py +3 -0
- package/dist/skills/data-scientist/SKILL.md +243 -0
- package/dist/skills/data-scientist/references/common-scenarios.md +176 -0
- package/dist/skills/data-scientist/references/execution-templates.md +197 -0
- package/dist/skills/data-scientist/references/integration-patterns.md +153 -0
- package/dist/skills/data-scientist/references/performance-benchmarks.md +37 -0
- package/dist/skills/data-scientist/references/uv-setup.md +78 -0
- package/dist/skills/data-scientist/scripts/quick-query.py +111 -0
- package/dist/skills/data-scientist/scripts/setup-uv.ps1 +53 -0
- package/dist/skills/data-scientist/scripts/setup-uv.sh +60 -0
- package/dist/skills/debugging/SKILL.md +1 -1
- package/dist/skills/frontend/ATTRIBUTION.md +7 -0
- package/dist/skills/frontend/SKILL.md +4 -1
- package/dist/skills/frontend/references/design/README.md +8 -0
- package/dist/skills/frontend/references/design/_INDEX.md +14 -1
- package/dist/skills/frontend/references/design/interaction-skill.md +144 -0
- package/dist/skills/programming/SKILL.md +1 -2
- package/dist/skills/ulw-plan/SKILL.md +1 -1
- package/dist/skills/ulw-research/SKILL.md +122 -11
- package/dist/startup-migration.d.ts +28 -0
- package/dist/testing/create-plugin-module.d.ts +4 -0
- package/dist/tools/delegate-task/builtin-categories.d.ts +1 -0
- package/dist/tools/delegate-task/builtin-category-definition.d.ts +1 -0
- package/dist/tools/delegate-task/constants.d.ts +1 -1
- package/dist/tui.js +6040 -5443
- package/package.json +13 -13
- package/packages/lsp-core/src/lsp/client-diagnostics-freshness.integration.test.ts +21 -5
- package/packages/lsp-core/src/lsp/fixtures/workspace-edit-server.mjs +11 -2
- package/packages/omo-codex/THIRD-PARTY-NOTICES.md +2 -2
- package/packages/omo-codex/plugin/.codex-plugin/plugin.json +1 -1
- package/packages/omo-codex/plugin/components/bootstrap/dist/cli.js +21 -3
- package/packages/omo-codex/plugin/components/bootstrap/hooks/hooks.json +1 -1
- package/packages/omo-codex/plugin/components/bootstrap/package.json +1 -1
- package/packages/omo-codex/plugin/components/codegraph/AGENTS.md +11 -5
- package/packages/omo-codex/plugin/components/codegraph/NOTICE +1 -1
- package/packages/omo-codex/plugin/components/codegraph/dist/cli.js +10373 -2570
- package/packages/omo-codex/plugin/components/codegraph/dist/serve.js +7502 -574
- package/packages/omo-codex/plugin/components/codegraph/package.json +2 -2
- package/packages/omo-codex/plugin/components/codegraph/src/hook-input.ts +33 -0
- package/packages/omo-codex/plugin/components/codegraph/src/hook-types.ts +36 -10
- package/packages/omo-codex/plugin/components/codegraph/src/hook.ts +164 -159
- package/packages/omo-codex/plugin/components/codegraph/src/post-tool-use-hook.ts +34 -0
- package/packages/omo-codex/plugin/components/codegraph/src/serve.ts +20 -2
- package/packages/omo-codex/plugin/components/codegraph/src/session-start-command.ts +106 -0
- package/packages/omo-codex/plugin/components/codegraph/src/session-start-cooldown.ts +145 -0
- package/packages/omo-codex/plugin/components/codegraph/src/session-start-hook-runtime.ts +21 -0
- package/packages/omo-codex/plugin/components/codegraph/src/session-start-lock.ts +139 -0
- package/packages/omo-codex/plugin/components/codegraph/src/session-start-outcome.ts +15 -0
- package/packages/omo-codex/plugin/components/codegraph/src/session-start-paths.ts +32 -0
- package/packages/omo-codex/plugin/components/codegraph/src/session-start-project.ts +109 -0
- package/packages/omo-codex/plugin/components/codegraph/src/session-start-worker-result.ts +148 -0
- package/packages/omo-codex/plugin/components/codegraph/src/session-start-worker.ts +126 -175
- package/packages/omo-codex/plugin/components/codegraph/test/hook-exclusion.test.ts +4 -4
- package/packages/omo-codex/plugin/components/codegraph/test/hook-session-start-guard.test.ts +160 -0
- package/packages/omo-codex/plugin/components/codegraph/test/hook-store-upgrade.test.ts +12 -138
- package/packages/omo-codex/plugin/components/codegraph/test/hook.test.ts +34 -24
- package/packages/omo-codex/plugin/components/codegraph/test/mcp-bridge-fixtures.ts +3 -3
- package/packages/omo-codex/plugin/components/codegraph/test/package-runtime.test.ts +2 -2
- package/packages/omo-codex/plugin/components/codegraph/test/provisioned-node-guard.test.ts +5 -5
- package/packages/omo-codex/plugin/components/codegraph/test/serve-mcp-bridge.test.ts +3 -3
- package/packages/omo-codex/plugin/components/codegraph/test/serve-mcp-facade.test.ts +1 -1
- package/packages/omo-codex/plugin/components/codegraph/test/serve-provision.test.ts +45 -0
- package/packages/omo-codex/plugin/components/codegraph/test/serve-unavailable.test.ts +2 -2
- package/packages/omo-codex/plugin/components/codegraph/test/session-start-node-support.test.ts +70 -13
- package/packages/omo-codex/plugin/components/codegraph/test/session-start-project.test.ts +58 -0
- package/packages/omo-codex/plugin/components/codegraph/test/session-start-state.test.ts +104 -0
- package/packages/omo-codex/plugin/components/codegraph/test/session-start-trust-boundary.test.ts +4 -2
- package/packages/omo-codex/plugin/components/codegraph/test/session-start-worker-cooldown.test.ts +116 -0
- package/packages/omo-codex/plugin/components/codegraph/test/session-start-worker-flow.test.ts +58 -57
- package/packages/omo-codex/plugin/components/comment-checker/hooks/hooks.json +1 -1
- package/packages/omo-codex/plugin/components/comment-checker/package.json +1 -1
- package/packages/omo-codex/plugin/components/git-bash/hooks/hooks.json +2 -2
- package/packages/omo-codex/plugin/components/git-bash/package.json +1 -1
- package/packages/omo-codex/plugin/components/lazycodex-executor-verify/hooks/hooks.json +1 -1
- package/packages/omo-codex/plugin/components/lazycodex-executor-verify/package.json +1 -1
- package/packages/omo-codex/plugin/components/lsp/dist/.omo-runtime-manifest.json +2 -2
- package/packages/omo-codex/plugin/components/lsp/hooks/hooks.json +2 -2
- package/packages/omo-codex/plugin/components/lsp/package.json +1 -1
- package/packages/omo-codex/plugin/components/rules/dist/cli.js +32 -15
- package/packages/omo-codex/plugin/components/rules/hooks/hooks.json +4 -4
- package/packages/omo-codex/plugin/components/rules/package.json +1 -1
- package/packages/omo-codex/plugin/components/rules/src/post-compact-budget.ts +1 -1
- package/packages/omo-codex/plugin/components/start-work-continuation/hooks/hooks.json +2 -2
- package/packages/omo-codex/plugin/components/start-work-continuation/package.json +1 -1
- package/packages/omo-codex/plugin/components/teammode/hooks/hooks.json +1 -1
- package/packages/omo-codex/plugin/components/teammode/package.json +1 -1
- package/packages/omo-codex/plugin/components/telemetry/hooks/hooks.json +1 -1
- package/packages/omo-codex/plugin/components/telemetry/package.json +1 -1
- package/packages/omo-codex/plugin/components/ultrawork/hooks/hooks.json +1 -1
- package/packages/omo-codex/plugin/components/ultrawork/package.json +1 -1
- package/packages/omo-codex/plugin/components/ultrawork/skills/ulw-plan/SKILL.md +1 -1
- package/packages/omo-codex/plugin/components/ulw-loop/hooks/hooks.json +4 -4
- package/packages/omo-codex/plugin/components/ulw-loop/package.json +1 -1
- package/packages/omo-codex/plugin/components/ulw-loop/skills/ulw-loop/SKILL.md +17 -0
- package/packages/omo-codex/plugin/hooks/post-compact-resetting-git-bash-mcp-reminder.json +1 -1
- package/packages/omo-codex/plugin/hooks/post-compact-resetting-lsp-diagnostics-cache.json +1 -1
- package/packages/omo-codex/plugin/hooks/post-compact-resetting-project-rule-cache.json +1 -1
- package/packages/omo-codex/plugin/hooks/post-tool-use-checking-codegraph-init-guidance.json +1 -1
- package/packages/omo-codex/plugin/hooks/post-tool-use-checking-comments.json +1 -1
- package/packages/omo-codex/plugin/hooks/post-tool-use-checking-lsp-diagnostics.json +1 -1
- package/packages/omo-codex/plugin/hooks/post-tool-use-checking-thread-title-hygiene.json +1 -1
- package/packages/omo-codex/plugin/hooks/post-tool-use-matching-project-rules.json +1 -1
- package/packages/omo-codex/plugin/hooks/pre-tool-use-enforcing-unlimited-goal-budget.json +1 -1
- package/packages/omo-codex/plugin/hooks/pre-tool-use-guarding-ulw-loop-spawns.json +1 -1
- package/packages/omo-codex/plugin/hooks/pre-tool-use-recommending-git-bash-mcp.json +1 -1
- package/packages/omo-codex/plugin/hooks/session-start-checking-auto-update.json +1 -1
- package/packages/omo-codex/plugin/hooks/session-start-checking-bootstrap-provisioning.json +1 -1
- package/packages/omo-codex/plugin/hooks/session-start-checking-codegraph-bootstrap.json +1 -1
- package/packages/omo-codex/plugin/hooks/session-start-loading-project-rules.json +1 -1
- package/packages/omo-codex/plugin/hooks/session-start-recording-session-telemetry.json +1 -1
- package/packages/omo-codex/plugin/hooks/stop-checking-start-work-continuation.json +1 -1
- package/packages/omo-codex/plugin/hooks/stop-checking-ulw-loop-resume.json +1 -1
- package/packages/omo-codex/plugin/hooks/subagent-stop-checking-start-work-continuation.json +1 -1
- package/packages/omo-codex/plugin/hooks/subagent-stop-verifying-lazycodex-executor-evidence.json +1 -1
- package/packages/omo-codex/plugin/hooks/user-prompt-submit-checking-ultrawork-trigger.json +1 -1
- package/packages/omo-codex/plugin/hooks/user-prompt-submit-checking-ulw-loop-steering.json +1 -1
- package/packages/omo-codex/plugin/hooks/user-prompt-submit-loading-project-rules.json +1 -1
- package/packages/omo-codex/plugin/package-lock.json +42 -41
- package/packages/omo-codex/plugin/package.json +1 -1
- package/packages/omo-codex/plugin/scripts/sync-skills.mjs +6 -0
- package/packages/omo-codex/plugin/shared/src/config-loader.ts +165 -16
- package/packages/omo-codex/plugin/shared/src/config-migration.ts +162 -0
- package/packages/omo-codex/plugin/shared/test/config-loader.test.ts +186 -9
- package/packages/omo-codex/plugin/skills/coding-agent-sessions/SKILL.md +4 -3
- package/packages/omo-codex/plugin/skills/coding-agent-sessions/references/all-platforms.md +3 -1
- package/packages/omo-codex/plugin/skills/coding-agent-sessions/scripts/agent_sessions/aside_scanner.py +140 -0
- package/packages/omo-codex/plugin/skills/coding-agent-sessions/scripts/agent_sessions/scanners.py +3 -0
- package/packages/omo-codex/plugin/skills/data-scientist/SKILL.md +243 -0
- package/packages/omo-codex/plugin/skills/data-scientist/agents/openai.yaml +2 -0
- package/packages/omo-codex/plugin/skills/data-scientist/references/common-scenarios.md +176 -0
- package/packages/omo-codex/plugin/skills/data-scientist/references/execution-templates.md +197 -0
- package/packages/omo-codex/plugin/skills/data-scientist/references/integration-patterns.md +153 -0
- package/packages/omo-codex/plugin/skills/data-scientist/references/performance-benchmarks.md +37 -0
- package/packages/omo-codex/plugin/skills/data-scientist/references/uv-setup.md +78 -0
- package/packages/omo-codex/plugin/skills/data-scientist/scripts/quick-query.py +111 -0
- package/packages/omo-codex/plugin/skills/data-scientist/scripts/setup-uv.ps1 +53 -0
- package/packages/omo-codex/plugin/skills/data-scientist/scripts/setup-uv.sh +60 -0
- package/packages/omo-codex/plugin/skills/debugging/SKILL.md +1 -1
- package/packages/omo-codex/plugin/skills/frontend/ATTRIBUTION.md +7 -0
- package/packages/omo-codex/plugin/skills/frontend/SKILL.md +4 -1
- package/packages/omo-codex/plugin/skills/frontend/references/design/README.md +8 -0
- package/packages/omo-codex/plugin/skills/frontend/references/design/_INDEX.md +14 -1
- package/packages/omo-codex/plugin/skills/frontend/references/design/interaction-skill.md +144 -0
- package/packages/omo-codex/plugin/skills/programming/SKILL.md +1 -2
- package/packages/omo-codex/plugin/skills/ulw-loop/SKILL.md +17 -0
- package/packages/omo-codex/plugin/skills/ulw-plan/SKILL.md +1 -1
- package/packages/omo-codex/plugin/skills/ulw-research/SKILL.md +121 -11
- package/packages/omo-codex/plugin/test/sync-skills-test-support.mjs +7 -0
- package/packages/omo-codex/plugin/test/sync-skills.test.mjs +12 -0
- package/packages/omo-codex/scripts/install-dist/install-local.mjs +24 -5
- package/packages/shared-skills/skills/coding-agent-sessions/SKILL.md +4 -3
- package/packages/shared-skills/skills/coding-agent-sessions/references/all-platforms.md +3 -1
- package/packages/shared-skills/skills/coding-agent-sessions/scripts/agent_sessions/aside_scanner.py +140 -0
- package/packages/shared-skills/skills/coding-agent-sessions/scripts/agent_sessions/scanners.py +3 -0
- package/packages/shared-skills/skills/data-scientist/SKILL.md +243 -0
- package/packages/shared-skills/skills/data-scientist/references/common-scenarios.md +176 -0
- package/packages/shared-skills/skills/data-scientist/references/execution-templates.md +197 -0
- package/packages/shared-skills/skills/data-scientist/references/integration-patterns.md +153 -0
- package/packages/shared-skills/skills/data-scientist/references/performance-benchmarks.md +37 -0
- package/packages/shared-skills/skills/data-scientist/references/uv-setup.md +78 -0
- package/packages/shared-skills/skills/data-scientist/scripts/quick-query.py +111 -0
- package/packages/shared-skills/skills/data-scientist/scripts/setup-uv.ps1 +53 -0
- package/packages/shared-skills/skills/data-scientist/scripts/setup-uv.sh +60 -0
- package/packages/shared-skills/skills/debugging/SKILL.md +1 -1
- package/packages/shared-skills/skills/frontend/ATTRIBUTION.md +7 -0
- package/packages/shared-skills/skills/frontend/SKILL.md +4 -1
- package/packages/shared-skills/skills/frontend/references/design/README.md +8 -0
- package/packages/shared-skills/skills/frontend/references/design/_INDEX.md +14 -1
- package/packages/shared-skills/skills/frontend/references/design/interaction-skill.md +144 -0
- package/packages/shared-skills/skills/programming/SKILL.md +1 -2
- package/packages/shared-skills/skills/ulw-plan/SKILL.md +1 -1
- package/packages/shared-skills/skills/ulw-research/SKILL.md +122 -11
- package/dist/plugin-config/layered-config-loader.d.ts +0 -2
- package/dist/plugin-config/single-config-loader.d.ts +0 -4
- package/dist/shared/migrate-legacy-config-file.d.ts +0 -1
- package/dist/shared/migration/config-migration.d.ts +0 -1
|
@@ -8,6 +8,7 @@ export const expectedSkills = [
|
|
|
8
8
|
"ast-grep",
|
|
9
9
|
"coding-agent-sessions",
|
|
10
10
|
"comment-checker",
|
|
11
|
+
"data-scientist",
|
|
11
12
|
"debugging",
|
|
12
13
|
"frontend",
|
|
13
14
|
"git-master",
|
|
@@ -126,7 +127,13 @@ prefixes when identity is needed.
|
|
|
126
127
|
`;
|
|
127
128
|
const reviewWorkCodexGatePattern = new RegExp(reviewWorkCodexGate.replace(/[.*+?^${}()|[\]\\]/g, "\\$&"));
|
|
128
129
|
|
|
130
|
+
const ulwResearchOriginalDeliveryGates = "### The delivery gates \u2014 every gate must PASS, in order\n\nNothing reaches the user until the gates pass:\n\n1. **Visual QA (always).** Render the produced artifact back to images \u2014 PDF pages to PNG, the HTML in a real browser \u2014 and look at them: missing or broken figures, clipped tables, overflowing CJK text, blank pages, unreadable chart labels, wrong page breaks. Fix and re-render until the pages are clean. Reading the source markup is not visual QA; inspect the pixels.\n2. **Proofread gate \u2014 `task(category=\"writing\", ...)`.** Hand the final text to a dedicated `writing` worker whose only job is language: grammar, spelling, punctuation, terminology consistency, and whether the prose reads NATIVELY in the report's own language. It returns a defect list; fix every item and re-run the gate on the delta. Deliver only on a clean pass \u2014 this gate runs BEFORE the first delivery, not after the user finds the typo.";
|
|
131
|
+
const ulwResearchCodexDeliveryGate = "### The delivery gate \u2014 visual QA must PASS\n\nNothing reaches the user until the gate passes:\n\n**Visual QA (always).** Render the produced artifact back to images \u2014 PDF pages to PNG, the HTML in a real browser \u2014 and look at them: missing or broken figures, clipped tables, overflowing CJK text, blank pages, unreadable chart labels, wrong page breaks. Fix and re-render until the pages are clean. Reading the source markup is not visual QA; inspect the pixels.";
|
|
132
|
+
|
|
129
133
|
export function removeCodexSkillOverlays(skillName, content) {
|
|
134
|
+
if (skillName === "ulw-research") {
|
|
135
|
+
return content.replace(ulwResearchCodexDeliveryGate, ulwResearchOriginalDeliveryGates);
|
|
136
|
+
}
|
|
130
137
|
if (skillName === "start-work") {
|
|
131
138
|
return content
|
|
132
139
|
.replace(startWorkCodexCompletion, startWorkOriginalCompletion)
|
|
@@ -331,6 +331,18 @@ test("#given packaged Codex ulw-plan surfaces #when inspected #then dangerous sa
|
|
|
331
331
|
assert.doesNotMatch(componentWorkflow.content, dangerousBypassPattern, `${componentWorkflow.path} ships unsafe Codex bypass guidance`);
|
|
332
332
|
});
|
|
333
333
|
|
|
334
|
+
test("#given packaged ulw-research skill #when Codex delivery gates are inspected #then the category-based proofread gate is stripped", async () => {
|
|
335
|
+
// given
|
|
336
|
+
const content = await readFile(join(root, "skills", "ulw-research", "SKILL.md"), "utf8");
|
|
337
|
+
|
|
338
|
+
// then the writing-category proofread gate never ships to Codex (no category concept there)
|
|
339
|
+
assert.doesNotMatch(content, /category="writing"/, "codex ulw-research ships the writing-category proofread gate");
|
|
340
|
+
assert.doesNotMatch(content, /Proofread gate/, "codex ulw-research still references the proofread gate");
|
|
341
|
+
// and the visual-QA delivery gate survives as the single gate
|
|
342
|
+
assert.match(content, /### The delivery gate \u2014 visual QA must PASS/u, "codex ulw-research lost the visual-QA delivery gate");
|
|
343
|
+
assert.match(content, /Visual QA \(always\)/, "codex ulw-research lost the visual-QA gate body");
|
|
344
|
+
});
|
|
345
|
+
|
|
334
346
|
test("#given context-pressure-prone skills #when bundled for Codex #then the eagerly loaded payload stays budgeted", async () => {
|
|
335
347
|
// given
|
|
336
348
|
const skillsRoot = join(root, "skills");
|
|
@@ -5903,7 +5903,7 @@ var package_default;
|
|
|
5903
5903
|
var init_package = __esm(() => {
|
|
5904
5904
|
package_default = {
|
|
5905
5905
|
name: "@oh-my-opencode/omo-codex",
|
|
5906
|
-
version: "4.19.
|
|
5906
|
+
version: "4.19.4",
|
|
5907
5907
|
type: "module",
|
|
5908
5908
|
private: true,
|
|
5909
5909
|
description: "Codex harness adapter for oh-my-openagent. Vendored Codex plugin namespace (omo) + TypeScript installer + telemetry.",
|
|
@@ -8840,6 +8840,24 @@ function stripUnquotedInlineComment3(line) {
|
|
|
8840
8840
|
|
|
8841
8841
|
// packages/omo-codex/src/install/codex-config-reasoning.ts
|
|
8842
8842
|
var MANAGED_KEYS = ["model", "model_context_window", "model_reasoning_effort", "plan_mode_reasoning_effort"];
|
|
8843
|
+
var CODEX_REASONING_BY_UNIFIED_LEVEL = {
|
|
8844
|
+
off: "none",
|
|
8845
|
+
none: "none",
|
|
8846
|
+
minimal: "minimal",
|
|
8847
|
+
low: "low",
|
|
8848
|
+
medium: "medium",
|
|
8849
|
+
high: "high",
|
|
8850
|
+
xhigh: "xhigh",
|
|
8851
|
+
max: "max"
|
|
8852
|
+
};
|
|
8853
|
+
function applyReasoningOverride(catalog, reasoning) {
|
|
8854
|
+
if (reasoning === undefined)
|
|
8855
|
+
return catalog;
|
|
8856
|
+
const wireEffort = CODEX_REASONING_BY_UNIFIED_LEVEL[reasoning.trim().toLowerCase()];
|
|
8857
|
+
if (wireEffort === undefined)
|
|
8858
|
+
return catalog;
|
|
8859
|
+
return { ...catalog, current: { ...catalog.current, modelReasoningEffort: wireEffort } };
|
|
8860
|
+
}
|
|
8843
8861
|
function ensureCodexReasoningConfig(config, catalog) {
|
|
8844
8862
|
const current = readRootReasoningSettings(config);
|
|
8845
8863
|
if (Object.keys(current).length > 0 && !matchesProfile(current, catalog.current) && !catalog.managedProfiles.some((profile) => matchesProfile(current, profile))) {
|
|
@@ -9180,7 +9198,7 @@ async function updateCodexConfig(input) {
|
|
|
9180
9198
|
config = ensureFeatureEnabled(config, "plugin_hooks");
|
|
9181
9199
|
config = ensureFeatureEnabled(config, "multi_agent");
|
|
9182
9200
|
config = removeUnsupportedCodexMultiAgentModeConfig(config);
|
|
9183
|
-
config = ensureCodexReasoningConfig(config, await readCodexModelCatalog(input.repoRoot));
|
|
9201
|
+
config = ensureCodexReasoningConfig(config, applyReasoningOverride(await readCodexModelCatalog(input.repoRoot), input.reasoning));
|
|
9184
9202
|
config = ensureCodexMultiAgentV2Config(config, {
|
|
9185
9203
|
multiAgentVersion: resolveCodexMultiAgentVersion(config, input.configPath)
|
|
9186
9204
|
});
|
|
@@ -9369,7 +9387,7 @@ var MANAGED_REASONING_DEFAULT_UPGRADES = new Map([
|
|
|
9369
9387
|
"explorer",
|
|
9370
9388
|
[
|
|
9371
9389
|
{
|
|
9372
|
-
previous: { model: "gpt-5.
|
|
9390
|
+
previous: { model: "gpt-5.6-luna-fast", effort: "low" },
|
|
9373
9391
|
current: { model: "gpt-5.6-terra", effort: "medium" }
|
|
9374
9392
|
},
|
|
9375
9393
|
{
|
|
@@ -9382,7 +9400,7 @@ var MANAGED_REASONING_DEFAULT_UPGRADES = new Map([
|
|
|
9382
9400
|
"librarian",
|
|
9383
9401
|
[
|
|
9384
9402
|
{
|
|
9385
|
-
previous: { model: "gpt-5.
|
|
9403
|
+
previous: { model: "gpt-5.6-luna-fast", effort: "low" },
|
|
9386
9404
|
current: { model: "gpt-5.6-terra", effort: "medium" }
|
|
9387
9405
|
},
|
|
9388
9406
|
{
|
|
@@ -11015,7 +11033,8 @@ async function runCodexInstaller(options = {}) {
|
|
|
11015
11033
|
gitBashEnabled: platform === "win32" && gitBashResolution.found,
|
|
11016
11034
|
trustedHookStates,
|
|
11017
11035
|
agentConfigs: [...agentConfigs.values()].sort((left, right) => left.name.localeCompare(right.name)),
|
|
11018
|
-
autonomousPermissions: options.autonomousPermissions !== false
|
|
11036
|
+
autonomousPermissions: options.autonomousPermissions !== false,
|
|
11037
|
+
...options.reasoning === undefined ? {} : { reasoning: options.reasoning }
|
|
11019
11038
|
});
|
|
11020
11039
|
await seedAndMigrateOmoSot({ env: env2, log, repoRoot, runCommand });
|
|
11021
11040
|
const projectCleanup = await repairProjectLocalCodexArtifactsBestEffort({
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: coding-agent-sessions
|
|
3
|
-
description: "MUST USE when asked to find, read, list, search, inspect, fetch, export, or reconstruct coding-agent sessions across Codex, Claude Code/Desktop, OpenCode, Senpi/pi, oh-my-pi (omp), gajae-code (gjc), OpenClaw, Factory Droid, Amp, Gemini/Kimi/Qwen CLIs, Codebuff, Roo/Kilo/Cline, Kodu, Cursor CLI, Aider, or unknown local agent logs. Covers transcripts, session IDs, rollout JSONL, state SQLite, Claude projects/pre-compact histories, OpenCode messages/parts, child/subagent linkage, cwd/model/time/token filters, archives, and cost clues. Expands fuzzy recall into parallel query lanes and first probes known stores so absent platforms are skipped cheaply. Triggers: coding agent sessions, Codex/Claude/OpenCode/Senpi/pi/oh-my-pi/omp/gajae-code/gjc/OpenClaw/Droid/Amp/Kodu/Cursor/Aider sessions, transcript search, session history, session ID, read transcript, token usage, subagent sessions, what did I do yesterday, did we already do this."
|
|
3
|
+
description: "MUST USE when asked to find, read, list, search, inspect, fetch, export, or reconstruct coding-agent sessions across Codex, Claude Code/Desktop, OpenCode, Senpi/pi, oh-my-pi (omp), gajae-code (gjc), OpenClaw, Factory Droid, Amp, Gemini/Kimi/Qwen CLIs, Codebuff, Roo/Kilo/Cline, Kodu, Cursor CLI, Aider, Aside browser-agent sessions, or unknown local agent logs. Covers transcripts, session IDs, rollout JSONL, state SQLite, Claude projects/pre-compact histories, OpenCode messages/parts, child/subagent linkage, cwd/model/time/token filters, archives, and cost clues. Expands fuzzy recall into parallel query lanes and first probes known stores so absent platforms are skipped cheaply. Triggers: coding agent sessions, Codex/Claude/OpenCode/Senpi/pi/oh-my-pi/omp/gajae-code/gjc/OpenClaw/Droid/Amp/Kodu/Cursor/Aider/Aside sessions, transcript search, session history, session ID, read transcript, token usage, subagent sessions, what did I do yesterday, did we already do this."
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Coding Agent Sessions
|
|
@@ -18,7 +18,7 @@ Find local coding-agent sessions across agent products before answering from mem
|
|
|
18
18
|
| Senpi / pi coding-agent logs | `references/senpi.md` |
|
|
19
19
|
| oh-my-pi (`omp`, `~/.omp`) and gajae-code (`gjc`, `~/.gjc`) logs | `references/senpi.md` |
|
|
20
20
|
| OpenCode / oh-my-openagent (formerly oh-my-opencode) storage | `references/opencode.md` |
|
|
21
|
-
| OpenClaw, Droid, Amp, Gemini, Kimi, Qwen, Codebuff, Roo/Kilo/Cline, Kodu, Cursor CLI, Aider, Kiro, Goose, Hermes, Crush, Zed | `references/all-platforms.md` |
|
|
21
|
+
| OpenClaw, Droid, Amp, Gemini, Kimi, Qwen, Codebuff, Roo/Kilo/Cline, Kodu, Cursor CLI, Aider, Kiro, Goose, Hermes, Crush, Zed, Aside | `references/all-platforms.md` |
|
|
22
22
|
| Unknown / "any session" / cross-agent search | `references/all-platforms.md` |
|
|
23
23
|
|
|
24
24
|
2. **Run the broad finder first unless the user gave an exact file path. For fuzzy recall, expand the query first.**
|
|
@@ -51,7 +51,7 @@ The finder prints JSON for stdout and `jq`. Every result includes:
|
|
|
51
51
|
|
|
52
52
|
| Field | Meaning |
|
|
53
53
|
|---|---|
|
|
54
|
-
| `platform` | Registered platform key such as `codex`, `claude`, `senpi`, `oh-my-pi`, `gajae-code`, `opencode`, `openclaw`, `droid`, `amp`, `kodu`, `cursor-cli`, `aider`, `roo-code`, `kilo-code`, `kilo-cli`, or `
|
|
54
|
+
| `platform` | Registered platform key such as `codex`, `claude`, `senpi`, `oh-my-pi`, `gajae-code`, `opencode`, `openclaw`, `droid`, `amp`, `kodu`, `cursor-cli`, `aider`, `roo-code`, `kilo-code`, `kilo-cli`, `kiro`, or `aside` |
|
|
55
55
|
| `id` | Session ID or stable file-derived ID |
|
|
56
56
|
| `path` | Raw transcript/index file |
|
|
57
57
|
| `cwd` | Working directory when recoverable |
|
|
@@ -122,6 +122,7 @@ Use `references/codex.md` for Codex storage details.
|
|
|
122
122
|
| Missing oh-my-pi / gajae-code sessions | Those stores live in `~/.omp/agent/sessions` and `~/.gjc/agent/sessions`. For a custom `PI_CONFIG_DIR` / `PI_CODING_AGENT_DIR`, pass that agent dir with `--root`. |
|
|
123
123
|
| Missing OpenCode sessions | Pass the data dir that contains `messages/` and `parts/`, often `~/.opencode` or `~/.local/share/opencode`. |
|
|
124
124
|
| Missing Claude sessions | Search `~/.claude/projects`, `~/.claude/transcripts`, and `~/.claude/pre-compact-session-histories`; use `--root` for nonstandard config dirs. |
|
|
125
|
+
| Missing Aside sessions | The Aside browser agent stores per-user data under `~/.aside/u/<n>/` (`sessions/<date>_<id>/messages.jsonl` transcripts + a `state.db` index; `agents/*/sessions/` is just a hardlink mirror). Pass `--root` for a nonstandard `.aside` dir or an exported user dir. |
|
|
125
126
|
| Missing optional platform sessions | Check `references/all-platforms.md` for the exact local store. For project-local tools such as Aider, pass `--root /path/to/workspace` if the repo is outside the bounded default roots. |
|
|
126
127
|
| Date filter misses local sessions | Timestamps are compared as UTC instants when parseable; otherwise file mtime is used. |
|
|
127
128
|
| Search is slow | Narrow with repeated `--platform` flags or date/cwd filters. Optional stores are probed before parsing; avoid passing your whole home as `--root` unless you really want every bounded project-local scan. |
|
|
@@ -4,7 +4,7 @@
|
|
|
4
4
|
|
|
5
5
|
Search these first, then add user-supplied roots with `--root`:
|
|
6
6
|
|
|
7
|
-
Registered platform keys: `codex`, `claude`, `senpi`, `oh-my-pi`, `gajae-code`, `opencode`, `openclaw`, `droid`, `amp`, `gemini`, `kimi`, `qwen`, `codebuff`, `roo-code`, `kilo-code`, `cline`, `kodu`, `cursor-cli`, `aider`, `kilo-cli`, `hermes`, `goose`, `crush`, `zed`, `kiro`.
|
|
7
|
+
Registered platform keys: `codex`, `claude`, `senpi`, `oh-my-pi`, `gajae-code`, `opencode`, `openclaw`, `droid`, `amp`, `gemini`, `kimi`, `qwen`, `codebuff`, `roo-code`, `kilo-code`, `cline`, `kodu`, `cursor-cli`, `aider`, `kilo-cli`, `hermes`, `goose`, `crush`, `zed`, `kiro`, `aside`.
|
|
8
8
|
|
|
9
9
|
| Platform | Unix/macOS | Windows |
|
|
10
10
|
|---|---|---|
|
|
@@ -25,6 +25,7 @@ Registered platform keys: `codex`, `claude`, `senpi`, `oh-my-pi`, `gajae-code`,
|
|
|
25
25
|
| Aider | bounded project roots containing `.aider.chat.history.md`; use `--root` for other repos | pass `--root` |
|
|
26
26
|
| Kilo CLI (`kilo-cli`) / Hermes / Goose / Crush / Zed | Known SQLite roots are probed cheaply; unsupported schemas return empty | pass `--root` |
|
|
27
27
|
| Kiro | `~/.kiro/sessions/cli/*.json` plus paired `*.jsonl` prompt events | pass `--root` |
|
|
28
|
+
| Aside (browser agent) | `~/.aside/u/*/sessions/<date>_<id>/messages.jsonl`, joined with the per-user `state.db` `sessions` table (title, parent_id, cwd, model, timestamps). `~/.aside/u/*/agents/*/sessions/` is a hardlink mirror of `sessions/` and is intentionally not scanned | pass `--root` |
|
|
28
29
|
|
|
29
30
|
## Excluded usage-only sources
|
|
30
31
|
|
|
@@ -52,6 +53,7 @@ Do not add these as default transcript platforms without a separate prompt-recon
|
|
|
52
53
|
| Claude | `projects/<proj>/<sid>/subagents/**/agent-<agentId>.jsonl` | directory `<sid>` | `agent-*.meta.json` `agentType` |
|
|
53
54
|
| Codex | thread row + own rollout JSONL | `thread_spawn_edges` / `source.subagent.thread_spawn.parent_thread_id` | `agent_nickname (agent_role)` |
|
|
54
55
|
| OpenCode | `session` table row / `storage/session/**.json` | `parent_id` column / `parentID` field | `agent` column |
|
|
56
|
+
| Aside | own `sessions/<date>_<id>/messages.jsonl` transcript | `parent_id` column in `state.db` `sessions` | child session's `title` column (task description) |
|
|
55
57
|
|
|
56
58
|
## Parallelism
|
|
57
59
|
|
package/packages/shared-skills/skills/coding-agent-sessions/scripts/agent_sessions/aside_scanner.py
ADDED
|
@@ -0,0 +1,140 @@
|
|
|
1
|
+
from __future__ import annotations
|
|
2
|
+
|
|
3
|
+
import sqlite3
|
|
4
|
+
from dataclasses import dataclass
|
|
5
|
+
from pathlib import Path
|
|
6
|
+
from typing import TypeAlias
|
|
7
|
+
|
|
8
|
+
from .jsonio import as_map, iter_jsonl, parse_json_text, text
|
|
9
|
+
from .timeparse import file_time, unix_millis, unix_seconds
|
|
10
|
+
from .transcript import content_text, existing, flat_parallel, merge_usage, recent
|
|
11
|
+
from .types import JsonMap, Session
|
|
12
|
+
|
|
13
|
+
SqlValue: TypeAlias = str | int | float | bytes | None
|
|
14
|
+
SqlRow: TypeAlias = tuple[SqlValue, ...]
|
|
15
|
+
|
|
16
|
+
|
|
17
|
+
@dataclass(frozen=True, slots=True)
|
|
18
|
+
class _StateRow:
|
|
19
|
+
parent_id: str | None
|
|
20
|
+
title: str | None
|
|
21
|
+
cwd: str | None
|
|
22
|
+
provider: str | None
|
|
23
|
+
model: str | None
|
|
24
|
+
created_at: str | None
|
|
25
|
+
updated_at: str | None
|
|
26
|
+
|
|
27
|
+
|
|
28
|
+
def scan_aside(extra_roots: tuple[Path, ...], workers: int) -> list[Session]:
|
|
29
|
+
roots = _roots([Path.home() / ".aside"], extra_roots, (".aside",))
|
|
30
|
+
sessions: list[Session] = []
|
|
31
|
+
for user_dir in _user_dirs(roots):
|
|
32
|
+
index = _state_index(user_dir / "state.db")
|
|
33
|
+
paths = list((user_dir / "sessions").glob("*/messages.jsonl"))
|
|
34
|
+
sessions.extend(flat_parallel(recent(paths), workers, lambda path, rows=index: [_aside_session(path, rows)]))
|
|
35
|
+
return sessions
|
|
36
|
+
|
|
37
|
+
|
|
38
|
+
def _user_dirs(roots: list[Path]) -> list[Path]:
|
|
39
|
+
dirs: list[Path] = []
|
|
40
|
+
for root in roots:
|
|
41
|
+
users = sorted(path for path in (root / "u").glob("*") if path.is_dir())
|
|
42
|
+
if users:
|
|
43
|
+
dirs.extend(users)
|
|
44
|
+
elif (root / "sessions").exists():
|
|
45
|
+
dirs.append(root)
|
|
46
|
+
return dirs
|
|
47
|
+
|
|
48
|
+
|
|
49
|
+
def _state_index(db_path: Path) -> dict[str, _StateRow]:
|
|
50
|
+
if not db_path.exists():
|
|
51
|
+
return {}
|
|
52
|
+
try:
|
|
53
|
+
with sqlite3.connect(f"file:{db_path}?mode=ro", uri=True) as conn:
|
|
54
|
+
rows: list[SqlRow] = conn.execute("select id, parent_id, title, cwd, model, created_at, updated_at from sessions").fetchall()
|
|
55
|
+
except sqlite3.Error:
|
|
56
|
+
return {}
|
|
57
|
+
index: dict[str, _StateRow] = {}
|
|
58
|
+
for row in rows:
|
|
59
|
+
sid = _cell_text(row, 0)
|
|
60
|
+
if sid is None:
|
|
61
|
+
continue
|
|
62
|
+
provider, model = _model_fields(_cell_text(row, 4))
|
|
63
|
+
index[sid] = _StateRow(
|
|
64
|
+
_cell_text(row, 1),
|
|
65
|
+
_cell_text(row, 2),
|
|
66
|
+
_cell_text(row, 3),
|
|
67
|
+
provider,
|
|
68
|
+
model,
|
|
69
|
+
unix_seconds(_cell_number(row, 5)),
|
|
70
|
+
unix_seconds(_cell_number(row, 6)),
|
|
71
|
+
)
|
|
72
|
+
return index
|
|
73
|
+
|
|
74
|
+
|
|
75
|
+
def _cell_text(row: SqlRow, index: int) -> str | None:
|
|
76
|
+
value = row[index] if index < len(row) else None
|
|
77
|
+
return value if isinstance(value, str) else None
|
|
78
|
+
|
|
79
|
+
|
|
80
|
+
def _cell_number(row: SqlRow, index: int) -> int | float | None:
|
|
81
|
+
value = row[index] if index < len(row) else None
|
|
82
|
+
return value if isinstance(value, int | float) else None
|
|
83
|
+
|
|
84
|
+
|
|
85
|
+
def _model_fields(model_json: str | None) -> tuple[str | None, str | None]:
|
|
86
|
+
data = as_map(parse_json_text(model_json)) if model_json else None
|
|
87
|
+
if data is None:
|
|
88
|
+
return None, None
|
|
89
|
+
return text(data.get("provider")), text(data.get("modelId")) or text(data.get("model"))
|
|
90
|
+
|
|
91
|
+
|
|
92
|
+
def _aside_session(path: Path, index: dict[str, _StateRow]) -> Session:
|
|
93
|
+
sid = _dir_session_id(path.parent.name)
|
|
94
|
+
row = index.get(sid)
|
|
95
|
+
first_user = last_user = ""
|
|
96
|
+
provider = model = None
|
|
97
|
+
created = updated = None
|
|
98
|
+
usage: JsonMap = {}
|
|
99
|
+
for data in iter_jsonl(path):
|
|
100
|
+
stamp = data.get("timestamp")
|
|
101
|
+
moment = unix_millis(int(stamp)) if isinstance(stamp, int | float) else None
|
|
102
|
+
created = created or moment
|
|
103
|
+
updated = moment or updated
|
|
104
|
+
role = text(data.get("role"))
|
|
105
|
+
if role == "user":
|
|
106
|
+
prompt = content_text(data.get("content"))
|
|
107
|
+
if prompt:
|
|
108
|
+
first_user = first_user or prompt
|
|
109
|
+
last_user = prompt
|
|
110
|
+
elif role == "assistant":
|
|
111
|
+
provider = provider or text(data.get("provider"))
|
|
112
|
+
model = model or text(data.get("model"))
|
|
113
|
+
merge_usage(usage, as_map(data.get("usage")))
|
|
114
|
+
return Session(
|
|
115
|
+
"aside",
|
|
116
|
+
sid,
|
|
117
|
+
str(path),
|
|
118
|
+
row.cwd if row is not None else None,
|
|
119
|
+
(row.created_at if row is not None else None) or created or file_time(path),
|
|
120
|
+
(row.updated_at if row is not None else None) or updated or created or file_time(path),
|
|
121
|
+
(row.provider if row is not None else None) or provider,
|
|
122
|
+
(row.model if row is not None else None) or model,
|
|
123
|
+
first_user,
|
|
124
|
+
usage,
|
|
125
|
+
row.parent_id if row is not None else None,
|
|
126
|
+
row.title if row is not None and row.parent_id is not None else None,
|
|
127
|
+
last_user,
|
|
128
|
+
)
|
|
129
|
+
|
|
130
|
+
|
|
131
|
+
def _dir_session_id(name: str) -> str:
|
|
132
|
+
return name.split("_", 1)[-1]
|
|
133
|
+
|
|
134
|
+
|
|
135
|
+
def _roots(defaults: list[Path], extra_roots: tuple[Path, ...], children: tuple[str, ...]) -> list[Path]:
|
|
136
|
+
candidates = [*defaults]
|
|
137
|
+
for root in extra_roots:
|
|
138
|
+
candidates.append(root)
|
|
139
|
+
candidates.extend(root / child for child in children)
|
|
140
|
+
return existing(candidates)
|
package/packages/shared-skills/skills/coding-agent-sessions/scripts/agent_sessions/scanners.py
CHANGED
|
@@ -4,6 +4,7 @@ from concurrent.futures import ThreadPoolExecutor, as_completed
|
|
|
4
4
|
from pathlib import Path
|
|
5
5
|
from typing import Callable, TypeAlias
|
|
6
6
|
|
|
7
|
+
from .aside_scanner import scan_aside
|
|
7
8
|
from .claude import scan_claude
|
|
8
9
|
from .codex import scan_codex
|
|
9
10
|
from .file_scanners import (
|
|
@@ -54,6 +55,7 @@ PLATFORM_SCANNERS: dict[str, Scanner] = {
|
|
|
54
55
|
"crush": scan_crush,
|
|
55
56
|
"zed": scan_zed,
|
|
56
57
|
"kiro": scan_kiro,
|
|
58
|
+
"aside": scan_aside,
|
|
57
59
|
}
|
|
58
60
|
DEFAULT_PLATFORMS = frozenset(PLATFORM_SCANNERS)
|
|
59
61
|
PLATFORM_ALIASES = {
|
|
@@ -69,6 +71,7 @@ PLATFORM_ALIASES = {
|
|
|
69
71
|
"gjc": "gajae-code",
|
|
70
72
|
"gajae": "gajae-code",
|
|
71
73
|
"gajaecode": "gajae-code",
|
|
74
|
+
"aside-browser": "aside",
|
|
72
75
|
}
|
|
73
76
|
|
|
74
77
|
__all__ = ["DEFAULT_PLATFORMS", "PLATFORM_SCANNERS", "scan", "scan_claude", "scan_codex", "scan_gajae_code", "scan_oh_my_pi", "scan_opencode", "scan_senpi"]
|
|
@@ -0,0 +1,243 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: data-scientist
|
|
3
|
+
description: "Expert data processing specialist with intelligent DuckDB/Polars selection for maximum performance. Always includes numpy, never uses pandas, runs everything through uv. Triggers: 'analyze the data', 'analyze this file', 'what is in this CSV/parquet/json', 'summarize this', 'group by', 'filter rows', 'sort by', 'join these files', 'merge datasets', 'time series trend', 'last 30 days data', 'compare yesterday and today', 'distribution/histogram', 'correlation', 'clean duplicates', 'handle missing values', 'dataset larger than RAM', 'SQL query on files', 'DataFrame operations', 'chart/plot this data', DuckDB vs Polars selection, quick data exploration CLI. NOT for plain text/code inspection, configs, or tiny inline math."
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Data Scientist: High-Performance Data Processing Expert
|
|
7
|
+
|
|
8
|
+
## Role & Expertise
|
|
9
|
+
|
|
10
|
+
Performance-obsessed data scientist with expertise in:
|
|
11
|
+
- Intelligent tool selection: DuckDB vs Polars based on operation characteristics
|
|
12
|
+
- Zero-copy data interchange via Apache Arrow
|
|
13
|
+
- Memory-efficient processing for datasets exceeding RAM
|
|
14
|
+
- SQL and DataFrame API mastery for analytical workloads
|
|
15
|
+
|
|
16
|
+
## Environment Setup
|
|
17
|
+
|
|
18
|
+
Everything runs through **uv**. If `uv` is not on PATH, set it up first — pick the path that matches the system and run it, no manual guesswork:
|
|
19
|
+
|
|
20
|
+
```bash
|
|
21
|
+
bash scripts/setup-uv.sh # macOS / Linux / WSL / Git Bash — auto-detects OS + arch, installs or updates uv to latest
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
```powershell
|
|
25
|
+
powershell -ExecutionPolicy Bypass -File scripts/setup-uv.ps1 # native Windows — installs or updates uv to latest
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
Both scripts detect the platform, install uv when missing (official installer first, Homebrew/winget as fallback), upgrade it when present (`uv self update`), put it on PATH for the current shell, and verify with `uv --version`. The full per-platform matrix, PATH notes, and CI usage live in [references/uv-setup.md](references/uv-setup.md). Verify: `uv --version`.
|
|
29
|
+
|
|
30
|
+
## Core Principles
|
|
31
|
+
|
|
32
|
+
### ABSOLUTE RULES
|
|
33
|
+
|
|
34
|
+
1. **ALWAYS include numpy** in all data processing operations (`uv run --with numpy ...`)
|
|
35
|
+
2. **NEVER use pandas** - Polars and DuckDB beat it decisively on every operation; the entire skill assumes pandas is absent
|
|
36
|
+
3. **ALWAYS use Python via `uv run`** for calculations and data processing
|
|
37
|
+
4. **Intelligent tool selection**: Choose DuckDB or Polars based on operation types, NOT arbitrarily
|
|
38
|
+
5. **Zero-copy conversions**: hand data across DuckDB and Polars through Arrow — `duckdb.sql(...).pl()`. Never call `.df()` (returns a pandas frame; crashes without pandas). Keep `pyarrow` in the package set or `.pl()` raises `ModuleNotFoundError`
|
|
39
|
+
6. **Lazy evaluation**: Prefer `scan_csv`/`scan_parquet` and `.collect()` only when needed
|
|
40
|
+
7. **Direct file queries**: Let DuckDB query files directly instead of loading to memory when possible
|
|
41
|
+
|
|
42
|
+
### Standard Package Pattern
|
|
43
|
+
|
|
44
|
+
```bash
|
|
45
|
+
# Default for data tasks (numpy + pyarrow are mandatory parts of the set)
|
|
46
|
+
uv run --with numpy --with duckdb --with polars --with pyarrow python -c "{code}"
|
|
47
|
+
|
|
48
|
+
# With visualization (RECOMMENDED for most analysis requests)
|
|
49
|
+
uv run --with numpy --with duckdb --with polars --with pyarrow --with matplotlib python -c "{code}"
|
|
50
|
+
|
|
51
|
+
# Pure Polars
|
|
52
|
+
uv run --with numpy --with polars python -c "{code}"
|
|
53
|
+
|
|
54
|
+
# Pure DuckDB (with the Arrow handoff available)
|
|
55
|
+
uv run --with numpy --with duckdb --with pyarrow python -c "{code}"
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
**When to include matplotlib:**
|
|
59
|
+
- User requests visualization: "graph", "chart", "plot", "show me"
|
|
60
|
+
- Exploratory data analysis (EDA): "analyze", "trends", "patterns"
|
|
61
|
+
- Time-series analysis: "over time", "daily", "trends"
|
|
62
|
+
- Distribution analysis: "distribution", "histogram", "statistics"
|
|
63
|
+
- Comparison tasks: "compare", visual comparison implied
|
|
64
|
+
- **Default to including matplotlib** when in doubt - overhead is minimal
|
|
65
|
+
|
|
66
|
+
## Tool Selection Logic
|
|
67
|
+
|
|
68
|
+
### Decision Tree (Apply in Order)
|
|
69
|
+
|
|
70
|
+
1. **Is it a `.duckdb` file?** → **USE DUCKDB** (native format, optimal performance)
|
|
71
|
+
2. **Simple one-off query without needing full data in memory?** → **USE DUCKDB** (direct file query, zero memory load)
|
|
72
|
+
3. **Very heavy complex SQL query (multi-table joins, window functions)?** → **USE DUCKDB** (superior SQL optimizer)
|
|
73
|
+
4. **Main operation is FILTERING?** → **USE POLARS** (typically the fastest by a wide margin — see benchmarks)
|
|
74
|
+
5. **Main operation is SORTING?** → **USE POLARS** (typically the fastest)
|
|
75
|
+
6. **Complex SQL JOINS needed?** → **USE DUCKDB** (stronger join engine, more join types)
|
|
76
|
+
7. **Heavy GROUP BY AGGREGATIONS?** → **USE DUCKDB** (typically faster on large datasets)
|
|
77
|
+
8. **Window functions with partitioning?** → **POLARS** (typically faster)
|
|
78
|
+
9. **Complex TRANSFORMATIONS (pivot, melt, string ops)?** → **USE POLARS**
|
|
79
|
+
10. **Dataset larger than available RAM?** → **USE POLARS** (streaming support) or **DUCKDB** (out-of-core)
|
|
80
|
+
11. **Mixed operations?** → **USE HYBRID APPROACH** (leverage strengths of both)
|
|
81
|
+
|
|
82
|
+
### Quick Reference
|
|
83
|
+
|
|
84
|
+
```
|
|
85
|
+
Simple query → DuckDB
|
|
86
|
+
Heavy complex query → DuckDB
|
|
87
|
+
Filter → Polars
|
|
88
|
+
Sort → Polars
|
|
89
|
+
Join → DuckDB
|
|
90
|
+
Aggregate → DuckDB
|
|
91
|
+
Window → Polars
|
|
92
|
+
Transform → Polars
|
|
93
|
+
Too large for RAM → Polars streaming
|
|
94
|
+
Mixed operations → Hybrid
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
The exact multipliers these heuristics distill (with sources and caveats — routing heuristics, not guarantees) live in [performance-benchmarks.md](references/performance-benchmarks.md).
|
|
98
|
+
|
|
99
|
+
## Essential Patterns
|
|
100
|
+
|
|
101
|
+
### DuckDB Direct File Query
|
|
102
|
+
|
|
103
|
+
```python
|
|
104
|
+
import duckdb
|
|
105
|
+
# Query file directly - no memory load
|
|
106
|
+
result = duckdb.sql("""
|
|
107
|
+
SELECT category, SUM(amount) as total
|
|
108
|
+
FROM 'data.csv'
|
|
109
|
+
GROUP BY category
|
|
110
|
+
""").pl() # .pl() -> Polars via Arrow. Requires pyarrow. Never .df() (pandas).
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
### Polars Lazy Evaluation
|
|
114
|
+
|
|
115
|
+
```python
|
|
116
|
+
import polars as pl
|
|
117
|
+
# Lazy scan - optimizes and executes once
|
|
118
|
+
result = (
|
|
119
|
+
pl.scan_csv('data.csv')
|
|
120
|
+
.filter(pl.col('value') > 100)
|
|
121
|
+
.sort('value', descending=True)
|
|
122
|
+
.collect()
|
|
123
|
+
)
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
### Zero-Copy DuckDB → Polars
|
|
127
|
+
|
|
128
|
+
```python
|
|
129
|
+
import duckdb
|
|
130
|
+
# Direct conversion via Arrow (pyarrow required in the package set)
|
|
131
|
+
df_polars = duckdb.sql("SELECT * FROM 'data.csv'").pl()
|
|
132
|
+
```
|
|
133
|
+
|
|
134
|
+
### Hybrid Approach
|
|
135
|
+
|
|
136
|
+
```python
|
|
137
|
+
import duckdb
|
|
138
|
+
import polars as pl
|
|
139
|
+
|
|
140
|
+
# Phase 1: DuckDB for joins
|
|
141
|
+
joined = duckdb.sql(
|
|
142
|
+
"SELECT * FROM 'orders.csv' o "
|
|
143
|
+
"JOIN 'customers.csv' c ON o.customer_id = c.customer_id"
|
|
144
|
+
).pl()
|
|
145
|
+
|
|
146
|
+
# Phase 2: Polars for filtering
|
|
147
|
+
filtered = joined.filter(pl.col('amount') > 100)
|
|
148
|
+
|
|
149
|
+
# Phase 3: Back to DuckDB for aggregation
|
|
150
|
+
duckdb.register('filtered_data', filtered)
|
|
151
|
+
final = duckdb.sql('SELECT category, SUM(amount) FROM filtered_data GROUP BY category').pl()
|
|
152
|
+
```
|
|
153
|
+
|
|
154
|
+
## Quick Query CLI
|
|
155
|
+
|
|
156
|
+
For ad-hoc data exploration, use the built-in query runner:
|
|
157
|
+
|
|
158
|
+
```bash
|
|
159
|
+
# SQL query (uses DuckDB)
|
|
160
|
+
uv run scripts/quick-query.py data.csv "SELECT category, COUNT(*) FROM data GROUP BY category"
|
|
161
|
+
|
|
162
|
+
# Filter expression — Polars SQL syntax, e.g. "amount > 100" (NOT Python: never passes through eval)
|
|
163
|
+
uv run scripts/quick-query.py data.csv --filter "amount > 100"
|
|
164
|
+
|
|
165
|
+
# Auto-describe (schema + stats)
|
|
166
|
+
uv run scripts/quick-query.py data.parquet --describe
|
|
167
|
+
```
|
|
168
|
+
|
|
169
|
+
Supports CSV, Parquet, JSON, NDJSON. Cross-platform (macOS, Linux, Windows). Excel files are not read directly — export to CSV or Parquet first.
|
|
170
|
+
|
|
171
|
+
## Reference Documentation
|
|
172
|
+
|
|
173
|
+
For detailed guidance, consult these reference files:
|
|
174
|
+
|
|
175
|
+
- **Environment setup per platform**: See [uv-setup.md](references/uv-setup.md) — install/update uv on macOS, Linux, Windows, WSL, CI; PATH fixes; `scripts/setup-uv.sh` / `scripts/setup-uv.ps1` automate it.
|
|
176
|
+
- **Performance benchmarks and operation detection**: See [performance-benchmarks.md](references/performance-benchmarks.md)
|
|
177
|
+
- **Integration patterns and best practices**: See [integration-patterns.md](references/integration-patterns.md)
|
|
178
|
+
- **Execution templates**: See [execution-templates.md](references/execution-templates.md)
|
|
179
|
+
- **Common scenarios**: See [common-scenarios.md](references/common-scenarios.md)
|
|
180
|
+
|
|
181
|
+
## Quality Assurance Process
|
|
182
|
+
|
|
183
|
+
### Before Execution
|
|
184
|
+
1. **Analyze request** → Detect operation types (filter, join, aggregate, etc.)
|
|
185
|
+
2. **Select optimal tool** → Apply decision tree based on detected operations
|
|
186
|
+
3. **Verify approach** → Confirm tool selection matches the benchmark heuristics
|
|
187
|
+
4. **Check package list** → Ensure numpy AND pyarrow are included
|
|
188
|
+
|
|
189
|
+
### During Execution
|
|
190
|
+
1. **Use lazy evaluation** when possible (Polars `scan_*`, DuckDB direct queries)
|
|
191
|
+
2. **Monitor for errors** and have fallback strategy ready
|
|
192
|
+
3. **Provide progress updates** for long operations
|
|
193
|
+
|
|
194
|
+
### After Execution
|
|
195
|
+
1. **Report performance** → Show processing time and row counts
|
|
196
|
+
2. **Validate results** → Confirm output matches expectations
|
|
197
|
+
3. **Document tool choice** → Explain why specific tool was selected
|
|
198
|
+
|
|
199
|
+
## Activation Context
|
|
200
|
+
|
|
201
|
+
**Automatic activation triggers:**
|
|
202
|
+
|
|
203
|
+
### Exploratory Questions
|
|
204
|
+
- "Analyze the data" / "What's in the data" / "What's in this file"
|
|
205
|
+
- "Show me the data" / "Take a look at this file" / "Check the file contents"
|
|
206
|
+
|
|
207
|
+
### Temporal/Historical Analysis
|
|
208
|
+
- "What happened in the past N days?" / "How's last week's data?"
|
|
209
|
+
- "What's the trend for the last 30 days?" / "Compare yesterday and today"
|
|
210
|
+
|
|
211
|
+
### Aggregation/Summary Requests
|
|
212
|
+
- "Summarize this" / "What's the total?" / "What's the average?"
|
|
213
|
+
- "Show by category" / "Show statistics" / "How many?"
|
|
214
|
+
|
|
215
|
+
### Filtering/Search Patterns
|
|
216
|
+
- "Show only above 100" / "Find specific conditions" / "Top 10"
|
|
217
|
+
|
|
218
|
+
### Comparison/Correlation
|
|
219
|
+
- "Compare A and B" / "What's the difference?" / "Is there a correlation?" / "Merge two files"
|
|
220
|
+
|
|
221
|
+
### Transformation/Cleaning
|
|
222
|
+
- "Clean this up" / "Remove duplicates" / "Handle missing values" / "Convert format"
|
|
223
|
+
|
|
224
|
+
### Technical Patterns
|
|
225
|
+
- Working with CSV, Parquet, JSON, NDJSON, or `.duckdb` files
|
|
226
|
+
- File paths ending in `.csv`, `.parquet`, `.json`, `.jsonl`, `.ndjson`, `.tsv`, `.duckdb`
|
|
227
|
+
- Requests involving calculations or aggregations
|
|
228
|
+
- Joining, filtering, sorting, or transforming datasets
|
|
229
|
+
- Processing large datasets that may exceed memory
|
|
230
|
+
- Comparing or analyzing data from multiple sources
|
|
231
|
+
- Performance-critical data operations
|
|
232
|
+
- SQL queries or DataFrame operations mentioned
|
|
233
|
+
|
|
234
|
+
### When NOT to Activate
|
|
235
|
+
- Simple file reading for text/code inspection (use the harness's file-read surface)
|
|
236
|
+
- Non-data files (images, videos, binaries)
|
|
237
|
+
- Configuration files (YAML, TOML, JSON configs) unless specifically for data analysis
|
|
238
|
+
- Small inline calculations (run them directly)
|
|
239
|
+
- Excel files — convert to CSV/Parquet first
|
|
240
|
+
|
|
241
|
+
---
|
|
242
|
+
|
|
243
|
+
**Core execution principle:** Always apply intelligent tool selection based on operation characteristics, never use pandas, and always include numpy and pyarrow in the execution environment.
|