@causyne/agent 0.0.0-stage → 1.10.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +102 -0
- package/README.md +31 -2
- package/SECURITY.md +24 -0
- package/bin/causyne.js +13 -0
- package/docs/ARCHITECTURE.md +63 -0
- package/docs/PROVENANCE.md +29 -0
- package/docs/SECURITY_REVIEW.md +89 -0
- package/docs/THIRD_PARTY_NOTICES.md +49 -0
- package/native-skills/ai-engineering/context-window-management/SKILL.md +43 -0
- package/native-skills/ai-engineering/llm-evaluation/SKILL.md +46 -0
- package/native-skills/ai-engineering/model-cost-and-latency/SKILL.md +46 -0
- package/native-skills/ai-engineering/prompt-construction/SKILL.md +42 -0
- package/native-skills/ai-engineering/retrieval-augmented-generation/SKILL.md +42 -0
- package/native-skills/ai-engineering/structured-output-and-schemas/SKILL.md +45 -0
- package/native-skills/ai-engineering/tool-calling-agents/SKILL.md +43 -0
- package/native-skills/android/app-performance-profiling/SKILL.md +42 -0
- package/native-skills/android/background-work-scheduling/SKILL.md +47 -0
- package/native-skills/android/compose-state-and-recomposition/SKILL.md +43 -0
- package/native-skills/android/gradle-build-configuration/SKILL.md +41 -0
- package/native-skills/android/lifecycle-and-leaks/SKILL.md +46 -0
- package/native-skills/android/play-store-release/SKILL.md +43 -0
- package/native-skills/android/room-and-persistence/SKILL.md +40 -0
- package/native-skills/backend/authentication-and-sessions/SKILL.md +45 -0
- package/native-skills/backend/background-jobs-and-queues/SKILL.md +42 -0
- package/native-skills/backend/caching-strategy/SKILL.md +44 -0
- package/native-skills/backend/graceful-shutdown/SKILL.md +42 -0
- package/native-skills/backend/rate-limiting-and-quotas/SKILL.md +40 -0
- package/native-skills/backend/rest-api-design/SKILL.md +40 -0
- package/native-skills/backend/webhook-delivery-and-receipt/SKILL.md +43 -0
- package/native-skills/cloud/cost-optimisation/SKILL.md +46 -0
- package/native-skills/cloud/iam-least-privilege/SKILL.md +43 -0
- package/native-skills/cloud/infrastructure-as-code/SKILL.md +43 -0
- package/native-skills/cloud/multi-region-and-failover/SKILL.md +43 -0
- package/native-skills/cloud/serverless-patterns/SKILL.md +47 -0
- package/native-skills/data/data-quality-monitoring/SKILL.md +51 -0
- package/native-skills/data/etl-pipeline-reliability/SKILL.md +43 -0
- package/native-skills/data/exploratory-analysis/SKILL.md +43 -0
- package/native-skills/data/metric-definition/SKILL.md +42 -0
- package/native-skills/data/notebook-to-production/SKILL.md +43 -0
- package/native-skills/database/backup-and-recovery/SKILL.md +43 -0
- package/native-skills/database/connection-pooling/SKILL.md +42 -0
- package/native-skills/database/data-integrity-and-constraints/SKILL.md +42 -0
- package/native-skills/database/query-optimisation/SKILL.md +38 -0
- package/native-skills/database/safe-migrations/SKILL.md +47 -0
- package/native-skills/database/schema-design/SKILL.md +38 -0
- package/native-skills/database/transactions-and-isolation/SKILL.md +41 -0
- package/native-skills/design/colour-system-construction/SKILL.md +38 -0
- package/native-skills/design/design-direction-brief/SKILL.md +41 -0
- package/native-skills/design/motion-and-transitions/SKILL.md +38 -0
- package/native-skills/design/responsive-strategy/SKILL.md +39 -0
- package/native-skills/design/spacing-and-rhythm/SKILL.md +38 -0
- package/native-skills/design/type-scale-and-hierarchy/SKILL.md +39 -0
- package/native-skills/design/visual-qa-loop/SKILL.md +42 -0
- package/native-skills/devops/docker-multi-stage-builds/SKILL.md +60 -0
- package/native-skills/devops/github-actions-ci/SKILL.md +60 -0
- package/native-skills/devops/incident-triage/SKILL.md +38 -0
- package/native-skills/devops/kubernetes-deployment-hardening/SKILL.md +64 -0
- package/native-skills/devops/observability-baseline/SKILL.md +38 -0
- package/native-skills/devops/secret-management/SKILL.md +40 -0
- package/native-skills/devops/terraform-state-and-modules/SKILL.md +36 -0
- package/native-skills/devops/zero-downtime-deploys/SKILL.md +40 -0
- package/native-skills/git/bisect-regression-hunt/SKILL.md +43 -0
- package/native-skills/git/branch-strategy/SKILL.md +39 -0
- package/native-skills/git/commit-hygiene/SKILL.md +41 -0
- package/native-skills/git/history-recovery/SKILL.md +44 -0
- package/native-skills/git/large-repository-surgery/SKILL.md +46 -0
- package/native-skills/git/merge-conflict-resolution/SKILL.md +42 -0
- package/native-skills/media/cinematic-3d-shot/SKILL.md +104 -0
- package/native-skills/mobile/app-startup-optimisation/SKILL.md +41 -0
- package/native-skills/mobile/cross-platform-tradeoffs/SKILL.md +42 -0
- package/native-skills/mobile/deep-linking/SKILL.md +39 -0
- package/native-skills/mobile/mobile-storage-security/SKILL.md +45 -0
- package/native-skills/mobile/offline-first-sync/SKILL.md +42 -0
- package/native-skills/mobile/push-notifications/SKILL.md +38 -0
- package/native-skills/performance/algorithmic-complexity/SKILL.md +43 -0
- package/native-skills/performance/concurrency-and-parallelism/SKILL.md +42 -0
- package/native-skills/performance/load-testing/SKILL.md +42 -0
- package/native-skills/performance/memory-and-leak-hunting/SKILL.md +41 -0
- package/native-skills/performance/network-and-payload-efficiency/SKILL.md +42 -0
- package/native-skills/performance/profiling-methodology/SKILL.md +46 -0
- package/native-skills/python/async-correctness/SKILL.md +40 -0
- package/native-skills/python/cli-tool-construction/SKILL.md +44 -0
- package/native-skills/python/data-processing-pipelines/SKILL.md +38 -0
- package/native-skills/python/environment-isolation/SKILL.md +41 -0
- package/native-skills/python/packaging-and-distribution/SKILL.md +56 -0
- package/native-skills/python/pytest-suite-design/SKILL.md +39 -0
- package/native-skills/python/type-hints-and-static-checking/SKILL.md +42 -0
- package/native-skills/security/authorization-model/SKILL.md +42 -0
- package/native-skills/security/crypto-usage/SKILL.md +48 -0
- package/native-skills/security/dependency-supply-chain/SKILL.md +42 -0
- package/native-skills/security/input-validation-and-injection/SKILL.md +44 -0
- package/native-skills/security/secure-file-uploads/SKILL.md +46 -0
- package/native-skills/security/security-headers-and-csp/SKILL.md +44 -0
- package/native-skills/security/threat-modelling/SKILL.md +46 -0
- package/native-skills/testing/end-to-end-test-stability/SKILL.md +42 -0
- package/native-skills/testing/property-based-testing/SKILL.md +46 -0
- package/native-skills/testing/regression-test-from-a-bug/SKILL.md +41 -0
- package/native-skills/testing/snapshot-and-visual-regression/SKILL.md +42 -0
- package/native-skills/testing/test-doubles-and-mocking/SKILL.md +43 -0
- package/native-skills/testing/test-strategy-and-pyramid/SKILL.md +39 -0
- package/native-skills/testing/writing-good-assertions/SKILL.md +42 -0
- package/native-skills/web/accessible-components/SKILL.md +45 -0
- package/native-skills/web/api-client-resilience/SKILL.md +39 -0
- package/native-skills/web/bundle-size-reduction/SKILL.md +38 -0
- package/native-skills/web/core-web-vitals/SKILL.md +40 -0
- package/native-skills/web/css-layout-systems/SKILL.md +41 -0
- package/native-skills/web/form-validation-ux/SKILL.md +39 -0
- package/native-skills/web/react-state-architecture/SKILL.md +41 -0
- package/native-skills/web/ssr-hydration-correctness/SKILL.md +45 -0
- package/package.json +68 -4
- package/src/agentLoop.js +2458 -0
- package/src/appChannel.js +109 -0
- package/src/approvalGrants.js +66 -0
- package/src/auditLog.js +52 -0
- package/src/autoUpdate.js +12 -0
- package/src/capabilities.js +36 -0
- package/src/checkpoints.js +1199 -0
- package/src/cliMode.js +638 -0
- package/src/commands.js +805 -0
- package/src/compiledSkills.js +540 -0
- package/src/config.js +130 -0
- package/src/contextManager.js +498 -0
- package/src/cortex/autoCondenser.js +192 -0
- package/src/cortex/provisioner.js +27 -0
- package/src/cortex/vectorDb.js +447 -0
- package/src/designPalettes.js +99 -0
- package/src/devMode.js +6 -0
- package/src/embodimentSession.js +180 -0
- package/src/entrypoint.js +194 -0
- package/src/execSafe.js +21 -0
- package/src/gitSnapshot.js +378 -0
- package/src/mcp/jsonRpc.js +165 -0
- package/src/mcp/mcpClient.js +136 -0
- package/src/mcp/mcpRegistry.js +207 -0
- package/src/memory.js +62 -0
- package/src/memoryManager.js +124 -0
- package/src/modelCatalogue.js +132 -0
- package/src/modelIds.js +42 -0
- package/src/modelRouter.js +145 -0
- package/src/outputStore.js +246 -0
- package/src/perception.js +124 -0
- package/src/planCache.js +437 -0
- package/src/procSignal.js +32 -0
- package/src/providers/anthropic.js +408 -0
- package/src/providers/base.js +328 -0
- package/src/providers/dashscope-byok.js +86 -0
- package/src/providers/deepseek.js +80 -0
- package/src/providers/factory.js +37 -0
- package/src/providers/gemini.js +346 -0
- package/src/providers/moonshot.js +73 -0
- package/src/providers/muse.js +169 -0
- package/src/providers/ollama.js +104 -0
- package/src/providers/openai-compatible.js +84 -0
- package/src/providers/openai.js +20 -0
- package/src/providers/qwen-byok.js +76 -0
- package/src/providers/xai.js +73 -0
- package/src/provisioning/capabilityConsent.js +24 -0
- package/src/provisioning/capabilityProvisioner.js +271 -0
- package/src/provisioning/capabilityTools.js +29 -0
- package/src/reporting.js +373 -0
- package/src/sandbox.js +529 -0
- package/src/sessionStore.js +279 -0
- package/src/skillsManager.js +288 -0
- package/src/slopDetector.js +307 -0
- package/src/taskBuffer.js +45 -0
- package/src/taskShape.js +241 -0
- package/src/textDiff.js +150 -0
- package/src/tokenTracker.js +560 -0
- package/src/tool-scripts/ax_snapshot.applescript +39 -0
- package/src/tool-scripts/ax_snapshot.ps1 +10 -0
- package/src/tool-scripts/desktop_act.applescript +58 -0
- package/src/tool-scripts/resolve_automator.py +119 -0
- package/src/toolPacks.js +524 -0
- package/src/tools/archive.js +63 -0
- package/src/tools/astryxOracle.js +78 -0
- package/src/tools/axSnapshot.js +165 -0
- package/src/tools/blenderAutomator.js +349 -0
- package/src/tools/browserAutomator.js +252 -0
- package/src/tools/checkSystemSpecs.js +64 -0
- package/src/tools/clipboard.js +36 -0
- package/src/tools/codeSearchAst.js +101 -0
- package/src/tools/codeSearcher.js +230 -0
- package/src/tools/codeTools.js +262 -0
- package/src/tools/codebaseSearch.js +75 -0
- package/src/tools/createSkill.js +129 -0
- package/src/tools/deferredMcpSearch.js +246 -0
- package/src/tools/desktopAct.js +208 -0
- package/src/tools/desktopOpen.js +34 -0
- package/src/tools/dynamicLoader.js +44 -0
- package/src/tools/executeProceduralMedia.js +93 -0
- package/src/tools/fsDelete.js +51 -0
- package/src/tools/fsEdit.js +105 -0
- package/src/tools/fsFind.js +128 -0
- package/src/tools/fsList.js +77 -0
- package/src/tools/fsMove.js +57 -0
- package/src/tools/fsRead.js +87 -0
- package/src/tools/fsSearch.js +198 -0
- package/src/tools/fsWrite.js +41 -0
- package/src/tools/gameAiBehaviorTree.js +208 -0
- package/src/tools/gameAnimationRig.js +194 -0
- package/src/tools/gameAssetFetcher.js +372 -0
- package/src/tools/gameAssetHarvester.js +210 -0
- package/src/tools/gameAudioSynth.js +267 -0
- package/src/tools/gameBuilder.js +886 -0
- package/src/tools/gameCanvasAudit.js +101 -0
- package/src/tools/gameCinematicSequencer.js +151 -0
- package/src/tools/gameDataForge.js +124 -0
- package/src/tools/gameDialogueTreeGen.js +166 -0
- package/src/tools/gameEconomyBalancer.js +161 -0
- package/src/tools/gameEntitySpawner.js +207 -0
- package/src/tools/gameLoreForge.js +153 -0
- package/src/tools/gameQuestArcBuilder.js +152 -0
- package/src/tools/gameSceneBuilder.js +158 -0
- package/src/tools/gameShaderForge.js +181 -0
- package/src/tools/gameTilemapGenerator.js +164 -0
- package/src/tools/gameUiHudBuilder.js +292 -0
- package/src/tools/gameVfxEmitterGen.js +108 -0
- package/src/tools/gameWebRuntimeInspector.js +91 -0
- package/src/tools/generateMediaViaApi.js +358 -0
- package/src/tools/gitExec.js +54 -0
- package/src/tools/godotAutomator.js +437 -0
- package/src/tools/manageLocalAiDependencies.js +220 -0
- package/src/tools/memoryWrite.js +99 -0
- package/src/tools/multiEdit.js +107 -0
- package/src/tools/palmierAutomator.js +104 -0
- package/src/tools/pdfReader.js +112 -0
- package/src/tools/persistentShell.js +148 -0
- package/src/tools/printer.js +316 -0
- package/src/tools/procTerm.js +91 -0
- package/src/tools/processList.js +52 -0
- package/src/tools/readNativeSkill.js +39 -0
- package/src/tools/registry.js +277 -0
- package/src/tools/resolveAutomator.js +102 -0
- package/src/tools/screenshot.js +75 -0
- package/src/tools/subagentOrchestrator.js +305 -0
- package/src/tools/sysRun.js +125 -0
- package/src/tools/todoWrite.js +75 -0
- package/src/tools/toolDispatch.js +127 -0
- package/src/tools/toolForge.js +124 -0
- package/src/tools/updatePlan.js +88 -0
- package/src/tools/viewImage.js +82 -0
- package/src/tools/visionInspector.js +154 -0
- package/src/tools/webFetch.js +74 -0
- package/src/tools/webSearch.js +82 -0
- package/src/tui/ask.js +164 -0
- package/src/tui/banner.js +158 -0
- package/src/tui/block.js +152 -0
- package/src/tui/chatRenderer.js +953 -0
- package/src/tui/completion.js +424 -0
- package/src/tui/dialogs.js +606 -0
- package/src/tui/diff.js +296 -0
- package/src/tui/history.js +135 -0
- package/src/tui/home.js +217 -0
- package/src/tui/invocation.js +105 -0
- package/src/tui/keys.js +443 -0
- package/src/tui/lineEditor.js +256 -0
- package/src/tui/overlay.js +256 -0
- package/src/tui/prompt.js +1258 -0
- package/src/tui/selectList.js +335 -0
- package/src/tui/stage.js +330 -0
- package/src/tui/stageSurface.js +609 -0
- package/src/tui/statusLine.js +254 -0
- package/src/tui/text.js +345 -0
- package/src/tui/theme.js +393 -0
- package/src/tui/toolDisplay.js +176 -0
- package/src/tui/transcript.js +235 -0
- package/src/tui/viewport.js +337 -0
- package/src/uncertainty.js +271 -0
- package/src/userSkills.js +265 -0
- package/src/utils/cli_ui.js +0 -0
- package/src/utils/inquirer.js +12 -0
- package/src/utils/paths.js +37 -0
- package/src/version.js +38 -0
- package/src/vfeTracker.js +220 -0
package/LICENSE
ADDED
|
@@ -0,0 +1,102 @@
|
|
|
1
|
+
Business Source License 1.1
|
|
2
|
+
|
|
3
|
+
License text copyright (c) 2017 MariaDB Corporation Ab, All Rights Reserved.
|
|
4
|
+
"Business Source License" is a trademark of MariaDB Corporation Ab.
|
|
5
|
+
|
|
6
|
+
-----------------------------------------------------------------------------
|
|
7
|
+
|
|
8
|
+
Parameters
|
|
9
|
+
|
|
10
|
+
Licensor: Ikbal Fadilah
|
|
11
|
+
|
|
12
|
+
Licensed Work: Causyne Agent 1.10.1
|
|
13
|
+
The Licensed Work is (c) 2026 Ikbal Fadilah.
|
|
14
|
+
|
|
15
|
+
Additional Use Grant: You may make use of the Licensed Work, provided that
|
|
16
|
+
you do not use the Licensed Work for a Commercial
|
|
17
|
+
Offering that competes with any offering of the Licensor
|
|
18
|
+
or its affiliates that includes the Licensed Work.
|
|
19
|
+
|
|
20
|
+
A "Commercial Offering" is a product or service offered
|
|
21
|
+
to third parties on a paid basis (including through paid
|
|
22
|
+
subscriptions or usage-based pricing) that provides
|
|
23
|
+
AI agent, AI automation, or AI-assisted development
|
|
24
|
+
capabilities substantially similar to the Licensed Work.
|
|
25
|
+
|
|
26
|
+
Personal, educational, and internal non-commercial use
|
|
27
|
+
is always permitted.
|
|
28
|
+
|
|
29
|
+
Change Date: 2030-08-09
|
|
30
|
+
|
|
31
|
+
Change License: Apache License, Version 2.0
|
|
32
|
+
|
|
33
|
+
-----------------------------------------------------------------------------
|
|
34
|
+
|
|
35
|
+
Terms
|
|
36
|
+
|
|
37
|
+
The Licensor hereby grants you the right to copy, modify, create derivative
|
|
38
|
+
works, redistribute, and make non-production use of the Licensed Work. The
|
|
39
|
+
Licensor may make an Additional Use Grant, above, permitting limited
|
|
40
|
+
production use.
|
|
41
|
+
|
|
42
|
+
Effective on the Change Date, or the fourth anniversary of the first publicly
|
|
43
|
+
available distribution of a specific version of the Licensed Work under this
|
|
44
|
+
License, whichever comes first, the Licensor hereby grants you rights under
|
|
45
|
+
the terms of the Change License, and the rights granted in the paragraph
|
|
46
|
+
above terminate.
|
|
47
|
+
|
|
48
|
+
If your use of the Licensed Work does not comply with the requirements
|
|
49
|
+
currently in effect as described in this License, you must purchase a
|
|
50
|
+
commercial license from the Licensor, its affiliated entities, or authorized
|
|
51
|
+
resellers, or you must refrain from using the Licensed Work.
|
|
52
|
+
|
|
53
|
+
All copies of the original and modified Licensed Work, and derivative works
|
|
54
|
+
of the Licensed Work, are subject to this License. This License applies
|
|
55
|
+
separately for each version of the Licensed Work and the Change Date may vary
|
|
56
|
+
for each version of the Licensed Work released by Licensor.
|
|
57
|
+
|
|
58
|
+
You must conspicuously display this License on each original or modified copy
|
|
59
|
+
of the Licensed Work. If you receive the Licensed Work in original or
|
|
60
|
+
modified form from a third party, the terms and conditions set forth in this
|
|
61
|
+
License apply to your use of that work.
|
|
62
|
+
|
|
63
|
+
Any use of the Licensed Work in violation of this License will automatically
|
|
64
|
+
terminate your rights under this License for the current and all other
|
|
65
|
+
versions of the Licensed Work.
|
|
66
|
+
|
|
67
|
+
This License does not grant you any right in any trademark or logo of
|
|
68
|
+
Licensor or its affiliates (provided that you may use a trademark or logo of
|
|
69
|
+
Licensor as expressly required by this License).
|
|
70
|
+
|
|
71
|
+
TO THE EXTENT PERMITTED BY APPLICABLE LAW, THE LICENSED WORK IS PROVIDED ON
|
|
72
|
+
AN "AS IS" BASIS. LICENSOR HEREBY DISCLAIMS ALL WARRANTIES AND CONDITIONS,
|
|
73
|
+
EXPRESS OR IMPLIED, INCLUDING (WITHOUT LIMITATION) WARRANTIES OF
|
|
74
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NON-INFRINGEMENT, AND
|
|
75
|
+
TITLE.
|
|
76
|
+
|
|
77
|
+
MariaDB hereby grants you permission to use this License's text to license
|
|
78
|
+
your works, and to refer to it using the trademark "Business Source License",
|
|
79
|
+
as long as you comply with the Covenants of Licensor below.
|
|
80
|
+
|
|
81
|
+
-----------------------------------------------------------------------------
|
|
82
|
+
|
|
83
|
+
Covenants of Licensor
|
|
84
|
+
|
|
85
|
+
In consideration of the right to use this License's text and the "Business
|
|
86
|
+
Source License" name and trademark, Licensor covenants to MariaDB, and to all
|
|
87
|
+
other recipients of the licensed work to be provided by Licensor:
|
|
88
|
+
|
|
89
|
+
1. To specify as the Change License the GPL Version 2.0 or any later version,
|
|
90
|
+
or a license that is compatible with GPL Version 2.0 or a later version,
|
|
91
|
+
where "compatible" means that software provided under the Change License can
|
|
92
|
+
be included in a program with software provided under GPL Version 2.0 or a
|
|
93
|
+
later version. Licensor may specify additional Change Licenses without
|
|
94
|
+
limitation.
|
|
95
|
+
|
|
96
|
+
2. To either: (a) specify an additional grant of rights to use that does not
|
|
97
|
+
impose any additional restriction on the right granted in this License, as
|
|
98
|
+
the Additional Use Grant; or (b) insert the text "None".
|
|
99
|
+
|
|
100
|
+
3. To specify a Change Date.
|
|
101
|
+
|
|
102
|
+
4. Not to modify this License in any other way.
|
package/README.md
CHANGED
|
@@ -1,3 +1,32 @@
|
|
|
1
|
-
#
|
|
1
|
+
# Causyne Agent
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
> **Development status:** Causyne Agent is under active development. It is not
|
|
4
|
+
> yet a stable release, and its interfaces and behavior may change.
|
|
5
|
+
|
|
6
|
+
Causyne Agent is an AI agent harness for complex, multi-step work. It is being
|
|
7
|
+
developed to turn intention into meaningful progress through capable,
|
|
8
|
+
increasingly autonomous AI, with clear boundaries and human control.
|
|
9
|
+
|
|
10
|
+
This repository is an early source-available development snapshot. Public
|
|
11
|
+
documentation is intentionally concise while the project is evolving.
|
|
12
|
+
|
|
13
|
+
Learn more at [causyne.com](https://causyne.com/).
|
|
14
|
+
|
|
15
|
+
## Development
|
|
16
|
+
|
|
17
|
+
Node.js 24 or newer is required.
|
|
18
|
+
|
|
19
|
+
```sh
|
|
20
|
+
npm ci --ignore-scripts
|
|
21
|
+
npm test
|
|
22
|
+
npm run typecheck:strict
|
|
23
|
+
npm run check:public
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
Please read [SECURITY.md](SECURITY.md) before running development builds or
|
|
27
|
+
reporting a security concern. Never commit API keys, credentials, private
|
|
28
|
+
configuration, or personal data.
|
|
29
|
+
|
|
30
|
+
The source is available under the [Business Source License 1.1](LICENSE).
|
|
31
|
+
Causyne Agent is source-available software and is not presented as OSI-approved
|
|
32
|
+
open-source software.
|
package/SECURITY.md
ADDED
|
@@ -0,0 +1,24 @@
|
|
|
1
|
+
# Security
|
|
2
|
+
|
|
3
|
+
Causyne Agent is under active development and has not reached a stable security
|
|
4
|
+
baseline. Development builds may execute tools that interact with files,
|
|
5
|
+
processes, applications, and external model providers. Review requested actions
|
|
6
|
+
and use an isolated workspace when evaluating the project.
|
|
7
|
+
|
|
8
|
+
## Protecting credentials
|
|
9
|
+
|
|
10
|
+
- Keep API keys and provider credentials outside the repository.
|
|
11
|
+
- Do not commit `.env` files, private keys, authentication tokens, personal
|
|
12
|
+
configuration, or generated state.
|
|
13
|
+
- Use test credentials with minimal permissions during development.
|
|
14
|
+
- Revoke and rotate any credential that is accidentally disclosed.
|
|
15
|
+
|
|
16
|
+
The release checks scan for common credential formats, but automated scanning
|
|
17
|
+
cannot prove that a repository contains no secrets.
|
|
18
|
+
|
|
19
|
+
## Reporting a vulnerability
|
|
20
|
+
|
|
21
|
+
Please avoid publishing exploit details in a public issue. Use the repository's
|
|
22
|
+
private security-reporting channel when it becomes available. Until that channel
|
|
23
|
+
is published, retain the report privately rather than exposing user data or a
|
|
24
|
+
working exploit.
|
package/bin/causyne.js
ADDED
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
// Execute the readable modules reviewers receive in the package.
|
|
3
|
+
if (Number(process.versions.node.split('.')[0]) < 24) {
|
|
4
|
+
console.error('Causyne Agent requires Node.js 24 or newer.');
|
|
5
|
+
process.exit(1);
|
|
6
|
+
}
|
|
7
|
+
try {
|
|
8
|
+
await import('../src/entrypoint.js');
|
|
9
|
+
} catch (error) {
|
|
10
|
+
if (process.stdout.isTTY) process.stdout.write('\x1b[?1006l\x1b[?1000l\x1b[?2004l\x1b[?25h');
|
|
11
|
+
console.error('Causyne Agent failed:', error);
|
|
12
|
+
process.exitCode = 1;
|
|
13
|
+
}
|
|
@@ -0,0 +1,63 @@
|
|
|
1
|
+
# Current architecture
|
|
2
|
+
|
|
3
|
+
Decision: 2026-10-19. Causyne Agent is an independent local desktop CLI.
|
|
4
|
+
|
|
5
|
+
```text
|
|
6
|
+
local terminal -> entrypoint -> cliMode / local approval UI
|
|
7
|
+
|
|
|
8
|
+
agentLoop
|
|
9
|
+
|
|
|
10
|
+
provider adapters <-> plan / act / observe / reflect
|
|
11
|
+
|
|
|
12
|
+
policy + workspace checks + approvals + tool registry
|
|
13
|
+
|
|
|
14
|
+
local processes / files / desktop applications
|
|
15
|
+
|
|
|
16
|
+
observations, memory, sessions, checkpoints
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
The LLM may be a configured remote API or a local model endpoint. Web tools
|
|
20
|
+
and third-party MCP servers may use the network. Neither makes the program
|
|
21
|
+
a phone-controlled daemon, nor makes every workflow offline. Provider keys
|
|
22
|
+
are stored locally and sent to the chosen provider for authentication.
|
|
23
|
+
|
|
24
|
+
`LocalCLIConnection` is a local rendering/event adapter, not an inbound network
|
|
25
|
+
connection. `emitEvent` feeds the terminal. Tools can execute host commands;
|
|
26
|
+
the harness is not an OS isolation boundary. Autonomy modes and sandbox rules
|
|
27
|
+
are application policies and require adversarial review.
|
|
28
|
+
|
|
29
|
+
## Modules
|
|
30
|
+
|
|
31
|
+
| Concern | Source |
|
|
32
|
+
| --- | --- |
|
|
33
|
+
| Invocation and local session | `entrypoint.js`, `cliMode.js`, `tui/` |
|
|
34
|
+
| Orchestration and budgets | `agentLoop.js`, `modelRouter.js`, `tokenTracker.js` |
|
|
35
|
+
| Host execution | `tools/registry.js`, `sandbox.js`, `execSafe.js` |
|
|
36
|
+
| Models | `providers/`, `modelCatalogue.js` |
|
|
37
|
+
| Context and memory | `memory.js`, `memoryManager.js`, `contextManager.js`, `cortex/` |
|
|
38
|
+
| Recovery | `checkpoints.js`, `gitSnapshot.js`, `sessionStore.js` |
|
|
39
|
+
| Extensions | `mcpClient.js`, `skillsManager.js`, `toolPacks.js`, user skills and Tool Forge |
|
|
40
|
+
| Readable interpreter source | `src/tool-scripts/` |
|
|
41
|
+
|
|
42
|
+
The current implementation remains JavaScript/ESM on Node.js 24+. The accepted
|
|
43
|
+
language direction is incremental strict TypeScript. Runtime validation is
|
|
44
|
+
still needed at model, filesystem, subprocess and extension boundaries.
|
|
45
|
+
Rust is a possible narrow native worker later; no Rust rewrite is underway.
|
|
46
|
+
|
|
47
|
+
## Removed boundaries
|
|
48
|
+
|
|
49
|
+
No active pairing, relay, WebRTC/P2P, mobile command surface, background daemon
|
|
50
|
+
lifecycle, V8 bytecode loader or Android import. Socket editor clients have no
|
|
51
|
+
active network implementation; their methods report held status or use CLI
|
|
52
|
+
fallbacks. Local process automation without an inbound command server remains.
|
|
53
|
+
|
|
54
|
+
Remote capability is on hold. Its old architecture is retired permanently.
|
|
55
|
+
Future GUI and remote work need distinct decisions and threat models.
|
|
56
|
+
|
|
57
|
+
## Distribution
|
|
58
|
+
|
|
59
|
+
The launcher imports the shipped source directly. `npm run build` regenerates
|
|
60
|
+
readable skill data only. No minifier, bytecode compiler or self-updater runs.
|
|
61
|
+
An npm allowlist includes source and security/provenance documents; private
|
|
62
|
+
archives live outside this repository. The unpublished manifest is private.
|
|
63
|
+
The independent repository does not import the Android repository history.
|
|
@@ -0,0 +1,29 @@
|
|
|
1
|
+
# Project provenance
|
|
2
|
+
|
|
3
|
+
Causyne Agent is the renamed continuation of **Vanexa Agent**. It is not
|
|
4
|
+
presented as an unrelated project or as a way to bypass
|
|
5
|
+
security classification. Version 1.10.0 was the first published local-source
|
|
6
|
+
migration successor to the 1.9.20 working tree.
|
|
7
|
+
|
|
8
|
+
The previous npm identities were `@ikbal_fadilah_vanexa01/vanexa-agent` and
|
|
9
|
+
`@vanexalabs-ai/vanexa-agent`. The dispute at
|
|
10
|
+
[OpenSSF issue 1470](https://github.com/ossf/malicious-packages/issues/1470)
|
|
11
|
+
references MAL-2026-13364 and MAL-2026-13397. The issue remained open on
|
|
12
|
+
2026-10-19, but the advisory files have different current states:
|
|
13
|
+
MAL-2026-13397 is marked withdrawn and was moved to OpenSSF's `osv/withdrawn`
|
|
14
|
+
tree in merged pull request 1473; MAL-2026-13364 remained in `osv/malicious`.
|
|
15
|
+
Withdrawal of one record does not resolve the other. See
|
|
16
|
+
`docs/SECURITY_REVIEW.md` for package-specific evidence and links.
|
|
17
|
+
|
|
18
|
+
Historical remote-control sources, private operating documents, and unrelated
|
|
19
|
+
application code are excluded from this project. The repository begins with the
|
|
20
|
+
independent local CLI source rather than importing the unrelated application
|
|
21
|
+
history.
|
|
22
|
+
|
|
23
|
+
Current architecture changes do not prove historical packages harmless. A source
|
|
24
|
+
hash or a deterministic intermediate bundle also does not prove that an old
|
|
25
|
+
distributed V8 artifact is equivalent to source. Earlier reproducibility claims
|
|
26
|
+
must be checked against actual versioned artifacts before repeating them.
|
|
27
|
+
|
|
28
|
+
Do not republish old binaries as a Causyne release. Keep version-specific
|
|
29
|
+
evidence and clearly distinguish remediation from requests to correct history.
|
|
@@ -0,0 +1,89 @@
|
|
|
1
|
+
# Security review and migration status
|
|
2
|
+
|
|
3
|
+
Review date: 2026-10-19. This is a scoped source and package review, not a
|
|
4
|
+
claim that Causyne Agent is free from vulnerabilities.
|
|
5
|
+
|
|
6
|
+
## OpenSSF records
|
|
7
|
+
|
|
8
|
+
The legacy packages have two distinct OpenSSF records:
|
|
9
|
+
|
|
10
|
+
- `MAL-2026-13397` names `@vanexalabs-ai/vanexa-agent` versions 1.3.50 through
|
|
11
|
+
1.3.62. OpenSSF's current record marks this advisory **withdrawn** as of
|
|
12
|
+
2026-08-25, and the repository moved it from `osv/malicious` to
|
|
13
|
+
`osv/withdrawn` in merged pull request 1473.
|
|
14
|
+
- `MAL-2026-13364` names `@ikbal_fadilah_vanexa01/vanexa-agent` versions in the
|
|
15
|
+
published 1.1.51 through 1.3.45 set. Its current OpenSSF record remains under
|
|
16
|
+
`osv/malicious`; no withdrawal was found during this review.
|
|
17
|
+
|
|
18
|
+
The dispute issue remains open. The withdrawn status of one advisory does not
|
|
19
|
+
withdraw the other or prove that every historical release was safe.
|
|
20
|
+
|
|
21
|
+
Sources: [dispute issue 1470](https://github.com/ossf/malicious-packages/issues/1470),
|
|
22
|
+
[withdrawn MAL-2026-13397 record](https://github.com/ossf/malicious-packages/blob/main/osv/withdrawn/npm/%40vanexalabs-ai/vanexa-agent/MAL-2026-13397.json),
|
|
23
|
+
[active MAL-2026-13364 record](https://github.com/ossf/malicious-packages/blob/main/osv/malicious/npm/%40ikbal_fadilah_vanexa01/vanexa-agent/MAL-2026-13364.json),
|
|
24
|
+
and [withdrawal pull request 1473](https://github.com/ossf/malicious-packages/pull/1473).
|
|
25
|
+
|
|
26
|
+
## Historical package evidence
|
|
27
|
+
|
|
28
|
+
The npm tarballs were fetched with lifecycle scripts disabled and inspected
|
|
29
|
+
locally. Their SHA-256 hashes provide a reproducible identity for this review:
|
|
30
|
+
|
|
31
|
+
| Package | Version | Tarball SHA-256 | Material finding |
|
|
32
|
+
| --- | ---: | --- | --- |
|
|
33
|
+
| `vanexa-agent` | 1.1.48 | `57b933623759f88719d8ed8dce5e841f76ff6b576f8a30f227bb28bdabcdbd3e` | Included install scripts and a `postinstall` hook that compiled a readable bundle to V8 bytecode. This package is not named by the two records above, but it did execute code during installation. |
|
|
34
|
+
| `@ikbal_fadilah_vanexa01/vanexa-agent` | 1.3.45 | `69501eb9af623472b8e551cfeef98dd2ceacce32a5e8a14b01b2fb8631d8516d` | Included readable bundled source and bytecode, plus `werift`, `ws`, and host-execution dependencies. It had no npm install lifecycle hook in this tarball. |
|
|
35
|
+
| `@vanexalabs-ai/vanexa-agent` | 1.3.62 | `aaf0b65621a9b4d3579cbf31bbb07610efe67e3e2c65a1e414ef49ee6cd46d86` | Shipped bytecode without readable runtime source. Its Socket policy ignored broad malware, obfuscation, shell, filesystem, network, native-code, and install-script categories. |
|
|
36
|
+
| `@vanexalabs-ai/vanexa-agent` | 1.9.19 | `6491307e8d8ae253a3f97514f079fa00b4b8607606cfd9dd53fe0f7cc55adc38` | Still shipped a bytecode-only runtime with `bytenode` and `ws`; the inspected manifest had no npm install lifecycle hook. This version is outside MAL-2026-13397's affected set. |
|
|
37
|
+
|
|
38
|
+
The reports describe a phone-paired daemon that accepted commands over remote
|
|
39
|
+
signaling or data channels and dispatched them to host execution. The preserved
|
|
40
|
+
legacy source contains that architecture, relay endpoints, daemon lifecycle,
|
|
41
|
+
pairing, WebRTC code, bytecode build logic, and deliberate string-obfuscation
|
|
42
|
+
steps. Those facts explain why the old package shape attracted remote-access
|
|
43
|
+
and concealment concerns even where the intended use was consensual.
|
|
44
|
+
|
|
45
|
+
## Current Causyne package
|
|
46
|
+
|
|
47
|
+
The active source takes a different architecture and distribution path:
|
|
48
|
+
|
|
49
|
+
- It is a foreground local CLI. Pairing, phone control, relays, WebRTC/P2P,
|
|
50
|
+
background daemon lifecycle, and remote headless command intake are absent.
|
|
51
|
+
- `bytenode`, `werift`, and `ws` are not production dependencies. The package
|
|
52
|
+
ships readable source and has no install, postinstall, prepare, or automatic
|
|
53
|
+
update hook.
|
|
54
|
+
- Automatic Cortex dependency installation is disabled. Optional capability
|
|
55
|
+
packages use a fixed version allowlist, require an approval-class capability
|
|
56
|
+
grant, install into local Causyne state, and use `npm install --ignore-scripts`.
|
|
57
|
+
- Arbitrary binary/model download actions were removed from the dependency
|
|
58
|
+
manager. Missing engine checks no longer create a placeholder and report a
|
|
59
|
+
false successful installation.
|
|
60
|
+
- Python package setup passes arguments without a shell and accepts exact PyPI
|
|
61
|
+
pins only. URLs, VCS references, paths, options, and unpinned requirements are
|
|
62
|
+
rejected.
|
|
63
|
+
- The Palmier adapter cannot install third-party applications, and the Astryx
|
|
64
|
+
adapter no longer auto-installs a package through `npx --yes`.
|
|
65
|
+
- `web_fetch` downloads and curated game-asset downloads require the highest
|
|
66
|
+
approval class. Curated assets are size-limited, constrained to catalog URLs,
|
|
67
|
+
protected from output path traversal, and are not automatically extracted.
|
|
68
|
+
- The package allowlist excludes the legacy archive, Android source, private
|
|
69
|
+
operating documents, credentials, installers, bytecode, and generated
|
|
70
|
+
tarballs.
|
|
71
|
+
|
|
72
|
+
At review time, `npm audit --omit=dev` reported zero known vulnerabilities in
|
|
73
|
+
the active locked production graph. npm's audit metadata counted 106 production
|
|
74
|
+
dependencies, and no dependency lifecycle scripts were recorded in the
|
|
75
|
+
lockfile.
|
|
76
|
+
|
|
77
|
+
## Remaining risk
|
|
78
|
+
|
|
79
|
+
Causyne Agent is designed to operate on files, processes, applications, model
|
|
80
|
+
providers, and user-configured MCP servers. These capabilities can cause harm
|
|
81
|
+
when a request is wrong, approval is careless, a dependency is compromised, or
|
|
82
|
+
policy enforcement is bypassed. The application policy is not an operating
|
|
83
|
+
system security boundary.
|
|
84
|
+
|
|
85
|
+
The licensor confirmed on 2026-10-19 that the project-authored source, skills,
|
|
86
|
+
tests, and assets are original work or otherwise lawful to distribute. A
|
|
87
|
+
separate independent security review has not been completed. Optional
|
|
88
|
+
capability graphs must be audited when installed because they are outside the
|
|
89
|
+
main lockfile and can change after this review.
|
|
@@ -0,0 +1,49 @@
|
|
|
1
|
+
# Third-party software review
|
|
2
|
+
|
|
3
|
+
Review date: 2026-10-19.
|
|
4
|
+
|
|
5
|
+
Causyne Agent's npm package declares third-party packages as dependencies; it
|
|
6
|
+
does not bundle their source into the project tarball. Each dependency remains
|
|
7
|
+
under its own license. The locked graph reviewed on this date contained these
|
|
8
|
+
license identifiers:
|
|
9
|
+
|
|
10
|
+
| License expression | Lockfile package entries |
|
|
11
|
+
| --- | ---: |
|
|
12
|
+
| MIT | 87 |
|
|
13
|
+
| Apache-2.0 | 22 |
|
|
14
|
+
| BSD-2-Clause | 8 |
|
|
15
|
+
| ISC | 5 |
|
|
16
|
+
| BSD-3-Clause | 1 |
|
|
17
|
+
| BlueOak-1.0.0 | 1 |
|
|
18
|
+
| 0BSD | 1 |
|
|
19
|
+
| MIT OR CC0-1.0 | 1 |
|
|
20
|
+
|
|
21
|
+
The lockfile contained 126 dependency entries in total. npm's audit metadata
|
|
22
|
+
reported 106 production dependencies; platform-specific optional entries and
|
|
23
|
+
development dependencies account for the differing views of the graph.
|
|
24
|
+
|
|
25
|
+
The direct runtime dependencies are `@babel/parser`, `acorn-walk`, `commander`,
|
|
26
|
+
`duck-duck-scrape`, `googlethis`, and `inquirer`. Direct
|
|
27
|
+
development dependencies are `@types/node`, `acorn`, and `typescript`. Exact
|
|
28
|
+
versions and integrity hashes are recorded in `package-lock.json`.
|
|
29
|
+
|
|
30
|
+
No dependency in the reviewed main lockfile was marked with an install script.
|
|
31
|
+
Optional capability dependencies are installed only after approval into a
|
|
32
|
+
separate local environment and are not covered by the main lockfile review.
|
|
33
|
+
|
|
34
|
+
Repository scanning found no third-party copyright header or license marker in
|
|
35
|
+
the project-authored source and skill files. That negative scan alone cannot
|
|
36
|
+
establish authorship or permission. On 2026-10-19, the licensor confirmed that
|
|
37
|
+
the project-authored source, native skills, tests, and assets are original work
|
|
38
|
+
or can lawfully be distributed under the stated terms.
|
|
39
|
+
|
|
40
|
+
BUSL-1.1 applies only to Causyne-owned material. It does not replace or narrow
|
|
41
|
+
the licenses of third-party dependencies.
|
|
42
|
+
|
|
43
|
+
The terms section of `LICENSE` was normalized and compared with the SPDX
|
|
44
|
+
BUSL-1.1 canonical text during this review; it matched. The project-specific
|
|
45
|
+
parameter block selects the named licensor, Causyne Agent 1.10.1 as the licensed
|
|
46
|
+
work, 2030-08-09 as the change date, and Apache-2.0 as the change
|
|
47
|
+
license. The Additional Use Grant permits production use except a paid
|
|
48
|
+
competing commercial offering as defined in that block. These business terms
|
|
49
|
+
should receive legal review before publication if enforcement is material.
|
|
@@ -0,0 +1,43 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: context-window-management
|
|
3
|
+
description: Handle long conversations or large inputs with a language model. Use when context is exhausted, when cost is high, or when a model forgets earlier instructions.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Context Window Management
|
|
7
|
+
|
|
8
|
+
## The two costs
|
|
9
|
+
|
|
10
|
+
Tokens cost money, and attention is finite. A model given a hundred thousand tokens attends unevenly across them — material in the middle is reliably weakest. Filling the window is not the same as using it.
|
|
11
|
+
|
|
12
|
+
## Workflow
|
|
13
|
+
|
|
14
|
+
1. Measure where the tokens go.
|
|
15
|
+
- System prompt, tool definitions, retrieved context, conversation history, current input. One of these is usually dominant and unexamined — tool definitions in particular grow silently.
|
|
16
|
+
- **Done when:** the token budget is broken down by section.
|
|
17
|
+
|
|
18
|
+
2. Order for attention and for caching at once.
|
|
19
|
+
- Stable content first (system, tools, reference material), volatile last (recent turns, current question). This maximises the cacheable prefix AND puts the question at the end, where recall is strongest.
|
|
20
|
+
- **Done when:** the prefix is byte-identical across calls and the question is last.
|
|
21
|
+
|
|
22
|
+
3. Compact history rather than truncating it.
|
|
23
|
+
- Dropping the oldest messages loses the original intent, which is usually the most important thing in the conversation. Keep the first message, summarise the middle, keep recent turns verbatim.
|
|
24
|
+
- **Done when:** the original task survives any amount of compaction.
|
|
25
|
+
|
|
26
|
+
4. Spill large tool outputs instead of inlining them.
|
|
27
|
+
- Store the full output, put a summary and a reference in the context, and let the model request more. A single large file read should not consume a third of the window.
|
|
28
|
+
- **Done when:** no single tool result dominates the context.
|
|
29
|
+
|
|
30
|
+
5. Retrieve rather than include.
|
|
31
|
+
- Do not put a whole document in the prompt when a retrieval step can select the relevant part. This is cheaper and usually more accurate, because it removes distractors.
|
|
32
|
+
- **Done when:** included material is selected rather than wholesale.
|
|
33
|
+
|
|
34
|
+
6. Handle the limit before you hit it.
|
|
35
|
+
- Detect approaching the ceiling and compact proactively. An overflow error mid-task loses the work.
|
|
36
|
+
- **Done when:** the system compacts before overflow rather than failing.
|
|
37
|
+
|
|
38
|
+
## Verification
|
|
39
|
+
|
|
40
|
+
- The token budget is reported by section.
|
|
41
|
+
- Cache hit rate is measured on repeated calls.
|
|
42
|
+
- A long conversation retains the original instruction after compaction.
|
|
43
|
+
- Approaching the limit triggers compaction, not an error.
|
|
@@ -0,0 +1,46 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: llm-evaluation
|
|
3
|
+
description: Measure whether a model, prompt, or pipeline change is an improvement. Use before shipping any prompt or model change, and when quality is being debated from anecdotes.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# LLM Evaluation
|
|
7
|
+
|
|
8
|
+
## Why anecdotes fail here
|
|
9
|
+
|
|
10
|
+
Model output is non-deterministic and its failures are uneven. A change that fixes the three cases you tried can break twenty you did not. Without a fixed set, every prompt change is a guess and quality drifts in both directions unnoticed.
|
|
11
|
+
|
|
12
|
+
## Workflow
|
|
13
|
+
|
|
14
|
+
1. Build the set from real failures.
|
|
15
|
+
- Start with cases the system got wrong in production. Fifty realistic cases beat five hundred synthetic ones. Include the ambiguous, the adversarial, and the out-of-scope.
|
|
16
|
+
- **Done when:** the set contains every failure mode you have actually observed.
|
|
17
|
+
|
|
18
|
+
2. Choose the grader to match the task.
|
|
19
|
+
- Exact match or a schema check for structured output — cheap and exact, use it wherever it applies.
|
|
20
|
+
- A programmatic check for verifiable properties: does the code run, is the citation real, is the total correct.
|
|
21
|
+
- A model grader for open-ended quality, with a rubric and a calibration set showing it agrees with human judgement.
|
|
22
|
+
- Human review for the final call on anything user-facing.
|
|
23
|
+
- **Done when:** every case has a grader and model graders are calibrated against human labels.
|
|
24
|
+
|
|
25
|
+
3. Fix the seed and pin the model version.
|
|
26
|
+
- A provider silently updating a model version invalidates every comparison. Pin explicitly and record it with the results.
|
|
27
|
+
- **Done when:** results record the exact model version and parameters.
|
|
28
|
+
|
|
29
|
+
4. Compare against a baseline, always.
|
|
30
|
+
- A score means nothing alone. Run the current version and the change on the same set, and report both.
|
|
31
|
+
- **Done when:** every result is a paired comparison.
|
|
32
|
+
|
|
33
|
+
5. Look at regressions, not just the average.
|
|
34
|
+
- A change that lifts the mean while breaking five previously-passing cases is usually not worth shipping. Report what got worse.
|
|
35
|
+
- **Done when:** the per-case diff is reviewed, not just the aggregate.
|
|
36
|
+
|
|
37
|
+
6. Run it in CI.
|
|
38
|
+
- A manual evaluation is run when someone remembers. Gate on the regression count.
|
|
39
|
+
- **Done when:** a prompt change with regressions fails the build.
|
|
40
|
+
|
|
41
|
+
## Verification
|
|
42
|
+
|
|
43
|
+
- The set covers every known failure mode.
|
|
44
|
+
- Results are paired against a baseline on a pinned model version.
|
|
45
|
+
- Per-case regressions are reported.
|
|
46
|
+
- CI runs the evaluation on prompt changes.
|
|
@@ -0,0 +1,46 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: model-cost-and-latency
|
|
3
|
+
description: Reduce the cost or latency of a language-model feature. Use when spend is higher than expected, when responses are slow, or when choosing a model.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Model Cost and Latency
|
|
7
|
+
|
|
8
|
+
## Where the money goes
|
|
9
|
+
|
|
10
|
+
Input tokens, usually. A long system prompt sent on every request dominates spend far more often than output length does. Measure before optimising anything.
|
|
11
|
+
|
|
12
|
+
## Workflow
|
|
13
|
+
|
|
14
|
+
1. Instrument per-call cost and latency.
|
|
15
|
+
- Input tokens, output tokens, cached tokens, model, and latency, tagged by feature. Without this you are optimising by anecdote.
|
|
16
|
+
- **Done when:** cost per feature is visible on a dashboard.
|
|
17
|
+
|
|
18
|
+
2. Maximise prompt caching.
|
|
19
|
+
- Put stable content first and keep the prefix byte-identical. A cached prefix is dramatically cheaper and faster, and this is usually the single largest available win — before any model change.
|
|
20
|
+
- **Done when:** the cache hit rate is measured and the prefix is stable.
|
|
21
|
+
|
|
22
|
+
3. Route by difficulty.
|
|
23
|
+
- Most requests do not need the largest model. Use a small fast model by default and escalate on a measurable signal — low confidence, a failed validation, or a complexity classifier.
|
|
24
|
+
- **Done when:** the routing rule is stated and its accuracy is measured on the evaluation set.
|
|
25
|
+
|
|
26
|
+
4. Stream to fix perceived latency.
|
|
27
|
+
- Time to first token is what users feel. Streaming makes a slow response feel responsive without making it faster.
|
|
28
|
+
- **Done when:** the user sees output within a few hundred milliseconds.
|
|
29
|
+
|
|
30
|
+
5. Cut the input before cutting the model.
|
|
31
|
+
- Trim the system prompt, remove unused tool definitions, retrieve less and rank better. Halving the input often costs no quality at all, where a smaller model might.
|
|
32
|
+
- **Done when:** every section of the prompt has been justified against its token cost.
|
|
33
|
+
|
|
34
|
+
6. Cache complete responses where the input repeats.
|
|
35
|
+
- Identical or near-identical requests do not need a second call. Semantic caching catches paraphrases, with a similarity threshold tuned to avoid wrong hits.
|
|
36
|
+
- **Done when:** repeated questions are served without a model call.
|
|
37
|
+
|
|
38
|
+
7. Verify quality did not regress.
|
|
39
|
+
- Every cost reduction must be checked against the evaluation set. A cheaper wrong answer is not cheaper.
|
|
40
|
+
- **Done when:** the evaluation score is unchanged or the trade-off is stated explicitly.
|
|
41
|
+
|
|
42
|
+
## Verification
|
|
43
|
+
|
|
44
|
+
- Cost per request is reported before and after.
|
|
45
|
+
- Cache hit rate is measured.
|
|
46
|
+
- Evaluation scores are unchanged, or the regression is quantified and accepted.
|
|
@@ -0,0 +1,42 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: prompt-construction
|
|
3
|
+
description: Write or repair a prompt for a language model. Use when model output is inconsistent, when a prompt has grown by accretion, or when building any LLM-backed feature.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Prompt Construction
|
|
7
|
+
|
|
8
|
+
## The shape that works
|
|
9
|
+
|
|
10
|
+
Role and task, then constraints, then the input, then the required output shape. Put the instruction BEFORE long context: attention to instructions degrades when they are buried after thousands of tokens of material.
|
|
11
|
+
|
|
12
|
+
## Workflow
|
|
13
|
+
|
|
14
|
+
1. State the task in one sentence, then the constraints as a list.
|
|
15
|
+
- Prose paragraphs of mixed instruction and context are where prompts go to become unmaintainable. A list is auditable.
|
|
16
|
+
- **Done when:** the task is one sentence and every constraint is a separate line.
|
|
17
|
+
|
|
18
|
+
2. Say what NOT to do, explicitly.
|
|
19
|
+
- The highest-leverage lines in most prompts. "Do not invent a citation." "Do not answer if the context does not contain the answer." Negative constraints are what stop the confident wrong answer.
|
|
20
|
+
- **Done when:** the failure modes you have actually observed each have a prohibition.
|
|
21
|
+
|
|
22
|
+
3. Specify the output format precisely, and validate it.
|
|
23
|
+
- Ask for a named schema and parse the result. Where the provider supports structured output or tool calling, use it — that is a hard guarantee rather than a request.
|
|
24
|
+
- **Done when:** the response is parsed into a validated type, and a parse failure is handled.
|
|
25
|
+
|
|
26
|
+
4. Show, do not only tell.
|
|
27
|
+
- Two or three examples covering the boundary cases outperform paragraphs of description. Include a hard case and an ambiguous one; examples that are all easy teach nothing.
|
|
28
|
+
- **Done when:** examples cover the cases the description leaves ambiguous.
|
|
29
|
+
|
|
30
|
+
5. Order for caching.
|
|
31
|
+
- Put stable content first — system instructions, tool definitions, long reference material — and volatile content last. Providers cache a matching prefix, so this ordering is the single largest cost reduction available on a repeated prompt.
|
|
32
|
+
- **Done when:** the prefix is identical across calls.
|
|
33
|
+
|
|
34
|
+
6. Change one thing at a time and measure.
|
|
35
|
+
- A prompt edited by feel, several changes at once, cannot be attributed. Keep a small evaluation set.
|
|
36
|
+
- **Done when:** each change has a before and after score on the evaluation set.
|
|
37
|
+
|
|
38
|
+
## Verification
|
|
39
|
+
|
|
40
|
+
- The output parses into the declared schema every time across the evaluation set.
|
|
41
|
+
- Each observed failure mode has a constraint addressing it.
|
|
42
|
+
- The stable prefix is identical between calls.
|
|
@@ -0,0 +1,42 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: retrieval-augmented-generation
|
|
3
|
+
description: Build or fix a retrieval system that feeds a language model. Use when answers cite the wrong source, when retrieval returns irrelevant chunks, or when building search over documents.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Retrieval-Augmented Generation
|
|
7
|
+
|
|
8
|
+
## Where these systems actually fail
|
|
9
|
+
|
|
10
|
+
Almost never in the generation step. They fail in retrieval — the right passage was not in the top results, so the model answered from its own parameters and sounded equally confident. Debug retrieval first, always.
|
|
11
|
+
|
|
12
|
+
## Workflow
|
|
13
|
+
|
|
14
|
+
1. Evaluate retrieval on its own.
|
|
15
|
+
- Build a set of questions with known correct sources. Measure recall at k. If the right chunk is not retrieved, no prompt change can fix the answer.
|
|
16
|
+
- **Done when:** recall at k is measured and reported separately from answer quality.
|
|
17
|
+
|
|
18
|
+
2. Chunk on structure, not on length.
|
|
19
|
+
- Split at headings, paragraphs and other semantic boundaries. A fixed character split cuts sentences in half and produces chunks that mean nothing alone. Include the document title and heading path in each chunk so it is self-describing.
|
|
20
|
+
- **Done when:** a randomly sampled chunk is comprehensible in isolation.
|
|
21
|
+
|
|
22
|
+
3. Combine keyword and vector search.
|
|
23
|
+
- Vector search misses exact identifiers, error codes and rare proper nouns; keyword search misses paraphrase. Hybrid retrieval with reciprocal rank fusion beats either alone on almost every corpus.
|
|
24
|
+
- **Done when:** both retrievers run and their results are fused.
|
|
25
|
+
|
|
26
|
+
4. Rerank the candidates.
|
|
27
|
+
- Retrieve broadly, then rerank with a cross-encoder and keep a few. This consistently produces the largest quality gain per unit of effort in the whole pipeline.
|
|
28
|
+
- **Done when:** a reranking stage exists and its effect on recall is measured.
|
|
29
|
+
|
|
30
|
+
5. Make grounding enforceable.
|
|
31
|
+
- Instruct the model to answer only from the context and to say so when the context is insufficient. Require a citation per claim, and verify the citation points at retrieved text.
|
|
32
|
+
- **Done when:** a question with no supporting context produces a refusal, not an invention.
|
|
33
|
+
|
|
34
|
+
6. Handle the update path.
|
|
35
|
+
- Documents change. Re-embedding everything on each change does not scale; track content hashes and re-embed only what changed.
|
|
36
|
+
- **Done when:** updating one document does not require a full reindex.
|
|
37
|
+
|
|
38
|
+
## Verification
|
|
39
|
+
|
|
40
|
+
- Retrieval recall is measured against a labelled set.
|
|
41
|
+
- An out-of-corpus question is refused rather than answered.
|
|
42
|
+
- Every claim in an answer maps to a retrieved passage.
|