@r1z4x/tezgah 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +738 -0
- package/LICENSE +21 -0
- package/MANIFEST +753 -0
- package/README.de.md +169 -0
- package/README.es.md +168 -0
- package/README.fr.md +166 -0
- package/README.md +161 -0
- package/README.tr.md +163 -0
- package/agents/tezgah-reviewer.md +70 -0
- package/bin/codegen +337 -0
- package/bin/consult +797 -0
- package/bin/tezgah-adhd +50 -0
- package/bin/tezgah-agents +45 -0
- package/bin/tezgah-capture +66 -0
- package/bin/tezgah-context +96 -0
- package/bin/tezgah-decisions +213 -0
- package/bin/tezgah-design +953 -0
- package/bin/tezgah-docs +1199 -0
- package/bin/tezgah-doctor +497 -0
- package/bin/tezgah-dsh +37 -0
- package/bin/tezgah-dsh.cmd +70 -0
- package/bin/tezgah-gate +75 -0
- package/bin/tezgah-import-ai-research +439 -0
- package/bin/tezgah-index +32 -0
- package/bin/tezgah-lessons +148 -0
- package/bin/tezgah-mcp +322 -0
- package/bin/tezgah-migrate +324 -0
- package/bin/tezgah-pony +50 -0
- package/bin/tezgah-research +772 -0
- package/bin/tezgah-rollback +93 -0
- package/bin/tezgah-route +124 -0
- package/bin/tezgah-setup +5798 -0
- package/bin/tezgah-status +383 -0
- package/bin/tezgah-task +607 -0
- package/bin/tezgah-taste +423 -0
- package/bin/tezgah-triage +485 -0
- package/bin/tezgah.js +36 -0
- package/commands/adhd.md +21 -0
- package/commands/ponytail.md +21 -0
- package/docs/README.md +112 -0
- package/docs/architecture.md +226 -0
- package/docs/citations-baseline.json +59 -0
- package/docs/clarity-baseline.json +116 -0
- package/docs/contract.md +416 -0
- package/docs/environment.md +141 -0
- package/docs/evidence.md +831 -0
- package/docs/feature-audit.md +139 -0
- package/docs/gate.md +407 -0
- package/docs/glossary.md +201 -0
- package/docs/hosts.md +400 -0
- package/docs/index.json +471 -0
- package/docs/judge.md +193 -0
- package/docs/layers.md +189 -0
- package/docs/models.md +160 -0
- package/docs/operations.md +666 -0
- package/docs/research.md +809 -0
- package/docs/results/paired-outcome-2026-10.jsonl +254 -0
- package/docs/results/paired-outcome-2026-10.md +204 -0
- package/docs/skills.md +236 -0
- package/docs/status-line.md +315 -0
- package/docs/testing.md +287 -0
- package/docs/vision.md +117 -0
- package/hooks/hooks.json +33 -0
- package/hooks/projects-auto-init.py +98 -0
- package/hooks/projects-posttooluse.py +149 -0
- package/hooks/projects-pretooluse.py +40 -0
- package/hooks/projects-stop.py +60 -0
- package/hooks/tezgah_agents.py +817 -0
- package/hooks/tezgah_apps.py +266 -0
- package/hooks/tezgah_attest.py +255 -0
- package/hooks/tezgah_clarity.py +171 -0
- package/hooks/tezgah_components.py +140 -0
- package/hooks/tezgah_context.py +2953 -0
- package/hooks/tezgah_embed.py +765 -0
- package/hooks/tezgah_gate.py +2736 -0
- package/hooks/tezgah_guard.py +177 -0
- package/hooks/tezgah_index.py +124 -0
- package/hooks/tezgah_integrity.py +5141 -0
- package/hooks/tezgah_judge.py +487 -0
- package/hooks/tezgah_lang.py +230 -0
- package/hooks/tezgah_lessons.py +210 -0
- package/hooks/tezgah_models.py +855 -0
- package/hooks/tezgah_paths.py +1132 -0
- package/hooks/tezgah_policy.py +1039 -0
- package/hooks/tezgah_rank.py +77 -0
- package/hooks/tezgah_replay.py +1109 -0
- package/hooks/tezgah_research.py +5176 -0
- package/hooks/tezgah_shapes.py +305 -0
- package/hooks/tezgah_skill_pick.py +227 -0
- package/hooks/tezgah_snapshot.py +490 -0
- package/hooks/tezgah_stopspec.py +320 -0
- package/hooks/tezgah_task.py +458 -0
- package/hooks/tezgah_taste.py +201 -0
- package/hooks/tezgah_untrusted.py +112 -0
- package/hooks/tezgah_update.py +430 -0
- package/hosts/codex/hook.py +268 -0
- package/hosts/codex/hooks.json +26 -0
- package/hosts/cursor/hook.py +461 -0
- package/hosts/cursor/hooks.json +44 -0
- package/hosts/dsh/hooks.json +21 -0
- package/hosts/dsh/statusline/lib/client.js +138 -0
- package/hosts/dsh/statusline/lib/index.js +104 -0
- package/hosts/dsh/statusline/package.json +28 -0
- package/hosts/omp/hook.py +277 -0
- package/hosts/omp/tezgah-hook.ts.in +701 -0
- package/hosts/opencode/plugins/tezgah.js +2742 -0
- package/hosts/opencode/tui/tezgah-tui.tsx +151 -0
- package/output-styles/tezgah.md +142 -0
- package/package.json +32 -0
- package/packaging/build.sh +342 -0
- package/packaging/install.ps1 +166 -0
- package/packaging/install.sh +56 -0
- package/packaging/upgrade.sh +142 -0
- package/skills/ai-research/0-autoresearch-skill/SKILL.md +411 -0
- package/skills/ai-research/0-autoresearch-skill/references/agent-continuity.md +95 -0
- package/skills/ai-research/0-autoresearch-skill/references/progress-reporting.md +166 -0
- package/skills/ai-research/0-autoresearch-skill/references/skill-routing.md +219 -0
- package/skills/ai-research/0-autoresearch-skill/templates/findings.md +44 -0
- package/skills/ai-research/0-autoresearch-skill/templates/research-log.md +41 -0
- package/skills/ai-research/01-model-architecture/litgpt/SKILL.md +469 -0
- package/skills/ai-research/01-model-architecture/litgpt/references/custom-models.md +569 -0
- package/skills/ai-research/01-model-architecture/litgpt/references/distributed-training.md +452 -0
- package/skills/ai-research/01-model-architecture/litgpt/references/supported-models.md +337 -0
- package/skills/ai-research/01-model-architecture/litgpt/references/training-recipes.md +620 -0
- package/skills/ai-research/01-model-architecture/mamba/SKILL.md +260 -0
- package/skills/ai-research/01-model-architecture/mamba/references/architecture-details.md +207 -0
- package/skills/ai-research/01-model-architecture/mamba/references/benchmarks.md +256 -0
- package/skills/ai-research/01-model-architecture/mamba/references/training-guide.md +389 -0
- package/skills/ai-research/01-model-architecture/nanogpt/SKILL.md +290 -0
- package/skills/ai-research/01-model-architecture/nanogpt/references/architecture.md +383 -0
- package/skills/ai-research/01-model-architecture/nanogpt/references/data.md +477 -0
- package/skills/ai-research/01-model-architecture/nanogpt/references/training.md +565 -0
- package/skills/ai-research/01-model-architecture/rwkv/SKILL.md +260 -0
- package/skills/ai-research/01-model-architecture/rwkv/references/architecture-details.md +345 -0
- package/skills/ai-research/01-model-architecture/rwkv/references/rwkv7.md +387 -0
- package/skills/ai-research/01-model-architecture/rwkv/references/state-management.md +370 -0
- package/skills/ai-research/01-model-architecture/torchtitan/SKILL.md +358 -0
- package/skills/ai-research/01-model-architecture/torchtitan/references/checkpoint.md +182 -0
- package/skills/ai-research/01-model-architecture/torchtitan/references/custom-models.md +259 -0
- package/skills/ai-research/01-model-architecture/torchtitan/references/float8.md +134 -0
- package/skills/ai-research/01-model-architecture/torchtitan/references/fsdp.md +127 -0
- package/skills/ai-research/02-tokenization/huggingface-tokenizers/SKILL.md +516 -0
- package/skills/ai-research/02-tokenization/huggingface-tokenizers/references/algorithms.md +654 -0
- package/skills/ai-research/02-tokenization/huggingface-tokenizers/references/integration.md +638 -0
- package/skills/ai-research/02-tokenization/huggingface-tokenizers/references/pipeline.md +724 -0
- package/skills/ai-research/02-tokenization/huggingface-tokenizers/references/training.md +566 -0
- package/skills/ai-research/02-tokenization/sentencepiece/SKILL.md +235 -0
- package/skills/ai-research/02-tokenization/sentencepiece/references/algorithms.md +201 -0
- package/skills/ai-research/02-tokenization/sentencepiece/references/training.md +305 -0
- package/skills/ai-research/03-fine-tuning/axolotl/SKILL.md +158 -0
- package/skills/ai-research/03-fine-tuning/axolotl/references/api.md +5549 -0
- package/skills/ai-research/03-fine-tuning/axolotl/references/dataset-formats.md +1030 -0
- package/skills/ai-research/03-fine-tuning/axolotl/references/index.md +16 -0
- package/skills/ai-research/03-fine-tuning/axolotl/references/other.md +3564 -0
- package/skills/ai-research/03-fine-tuning/llama-factory/SKILL.md +80 -0
- package/skills/ai-research/03-fine-tuning/llama-factory/references/_images.md +24 -0
- package/skills/ai-research/03-fine-tuning/llama-factory/references/advanced.md +1056 -0
- package/skills/ai-research/03-fine-tuning/llama-factory/references/getting_started.md +350 -0
- package/skills/ai-research/03-fine-tuning/llama-factory/references/index.md +20 -0
- package/skills/ai-research/03-fine-tuning/llama-factory/references/other.md +32 -0
- package/skills/ai-research/03-fine-tuning/peft/SKILL.md +431 -0
- package/skills/ai-research/03-fine-tuning/peft/references/advanced-usage.md +515 -0
- package/skills/ai-research/03-fine-tuning/peft/references/troubleshooting.md +481 -0
- package/skills/ai-research/03-fine-tuning/unsloth/SKILL.md +80 -0
- package/skills/ai-research/03-fine-tuning/unsloth/references/index.md +8 -0
- package/skills/ai-research/03-fine-tuning/unsloth/references/llms.md +83 -0
- package/skills/ai-research/04-mechanistic-interpretability/nnsight/SKILL.md +436 -0
- package/skills/ai-research/04-mechanistic-interpretability/nnsight/references/README.md +79 -0
- package/skills/ai-research/04-mechanistic-interpretability/nnsight/references/api.md +345 -0
- package/skills/ai-research/04-mechanistic-interpretability/nnsight/references/tutorials.md +301 -0
- package/skills/ai-research/04-mechanistic-interpretability/pyvene/SKILL.md +473 -0
- package/skills/ai-research/04-mechanistic-interpretability/pyvene/references/README.md +74 -0
- package/skills/ai-research/04-mechanistic-interpretability/pyvene/references/api.md +384 -0
- package/skills/ai-research/04-mechanistic-interpretability/pyvene/references/tutorials.md +377 -0
- package/skills/ai-research/04-mechanistic-interpretability/saelens/SKILL.md +386 -0
- package/skills/ai-research/04-mechanistic-interpretability/saelens/references/README.md +71 -0
- package/skills/ai-research/04-mechanistic-interpretability/saelens/references/api.md +334 -0
- package/skills/ai-research/04-mechanistic-interpretability/saelens/references/tutorials.md +319 -0
- package/skills/ai-research/04-mechanistic-interpretability/transformer-lens/SKILL.md +346 -0
- package/skills/ai-research/04-mechanistic-interpretability/transformer-lens/references/README.md +55 -0
- package/skills/ai-research/04-mechanistic-interpretability/transformer-lens/references/api.md +363 -0
- package/skills/ai-research/04-mechanistic-interpretability/transformer-lens/references/tutorials.md +340 -0
- package/skills/ai-research/05-data-processing/nemo-curator/SKILL.md +383 -0
- package/skills/ai-research/05-data-processing/nemo-curator/references/deduplication.md +88 -0
- package/skills/ai-research/05-data-processing/nemo-curator/references/filtering.md +103 -0
- package/skills/ai-research/05-data-processing/ray-data/SKILL.md +326 -0
- package/skills/ai-research/05-data-processing/ray-data/references/integration.md +83 -0
- package/skills/ai-research/05-data-processing/ray-data/references/transformations.md +84 -0
- package/skills/ai-research/06-post-training/grpo-rl-training/README.md +98 -0
- package/skills/ai-research/06-post-training/grpo-rl-training/SKILL.md +572 -0
- package/skills/ai-research/06-post-training/miles/SKILL.md +315 -0
- package/skills/ai-research/06-post-training/miles/references/api-reference.md +142 -0
- package/skills/ai-research/06-post-training/miles/references/troubleshooting.md +353 -0
- package/skills/ai-research/06-post-training/openrlhf/SKILL.md +249 -0
- package/skills/ai-research/06-post-training/openrlhf/references/algorithm-comparison.md +405 -0
- package/skills/ai-research/06-post-training/openrlhf/references/custom-rewards.md +531 -0
- package/skills/ai-research/06-post-training/openrlhf/references/hybrid-engine.md +288 -0
- package/skills/ai-research/06-post-training/openrlhf/references/multi-node-training.md +455 -0
- package/skills/ai-research/06-post-training/simpo/SKILL.md +219 -0
- package/skills/ai-research/06-post-training/simpo/references/datasets.md +479 -0
- package/skills/ai-research/06-post-training/simpo/references/hyperparameters.md +453 -0
- package/skills/ai-research/06-post-training/simpo/references/loss-functions.md +351 -0
- package/skills/ai-research/06-post-training/slime/SKILL.md +464 -0
- package/skills/ai-research/06-post-training/slime/references/api-reference.md +393 -0
- package/skills/ai-research/06-post-training/slime/references/troubleshooting.md +387 -0
- package/skills/ai-research/06-post-training/torchforge/SKILL.md +433 -0
- package/skills/ai-research/06-post-training/torchforge/references/api-reference.md +328 -0
- package/skills/ai-research/06-post-training/torchforge/references/troubleshooting.md +410 -0
- package/skills/ai-research/06-post-training/trl-fine-tuning/SKILL.md +455 -0
- package/skills/ai-research/06-post-training/trl-fine-tuning/references/dpo-variants.md +228 -0
- package/skills/ai-research/06-post-training/trl-fine-tuning/references/online-rl.md +83 -0
- package/skills/ai-research/06-post-training/trl-fine-tuning/references/reward-modeling.md +123 -0
- package/skills/ai-research/06-post-training/trl-fine-tuning/references/sft-training.md +169 -0
- package/skills/ai-research/06-post-training/verl/SKILL.md +391 -0
- package/skills/ai-research/06-post-training/verl/references/api-reference.md +302 -0
- package/skills/ai-research/06-post-training/verl/references/troubleshooting.md +392 -0
- package/skills/ai-research/07-safety-alignment/constitutional-ai/SKILL.md +290 -0
- package/skills/ai-research/07-safety-alignment/llamaguard/SKILL.md +337 -0
- package/skills/ai-research/07-safety-alignment/nemo-guardrails/SKILL.md +297 -0
- package/skills/ai-research/07-safety-alignment/prompt-guard/SKILL.md +312 -0
- package/skills/ai-research/08-distributed-training/accelerate/SKILL.md +332 -0
- package/skills/ai-research/08-distributed-training/accelerate/references/custom-plugins.md +454 -0
- package/skills/ai-research/08-distributed-training/accelerate/references/megatron-integration.md +490 -0
- package/skills/ai-research/08-distributed-training/accelerate/references/performance.md +526 -0
- package/skills/ai-research/08-distributed-training/deepspeed/SKILL.md +141 -0
- package/skills/ai-research/08-distributed-training/deepspeed/references/08.md +18 -0
- package/skills/ai-research/08-distributed-training/deepspeed/references/09.md +174 -0
- package/skills/ai-research/08-distributed-training/deepspeed/references/2020.md +379 -0
- package/skills/ai-research/08-distributed-training/deepspeed/references/2023.md +280 -0
- package/skills/ai-research/08-distributed-training/deepspeed/references/assets.md +180 -0
- package/skills/ai-research/08-distributed-training/deepspeed/references/index.md +36 -0
- package/skills/ai-research/08-distributed-training/deepspeed/references/mii.md +119 -0
- package/skills/ai-research/08-distributed-training/deepspeed/references/other.md +1192 -0
- package/skills/ai-research/08-distributed-training/megatron-core/SKILL.md +366 -0
- package/skills/ai-research/08-distributed-training/megatron-core/references/benchmarks.md +250 -0
- package/skills/ai-research/08-distributed-training/megatron-core/references/parallelism-guide.md +405 -0
- package/skills/ai-research/08-distributed-training/megatron-core/references/production-examples.md +474 -0
- package/skills/ai-research/08-distributed-training/megatron-core/references/training-recipes.md +548 -0
- package/skills/ai-research/08-distributed-training/pytorch-fsdp2/SKILL.md +231 -0
- package/skills/ai-research/08-distributed-training/pytorch-fsdp2/references/pytorch_dcp_async_recipe.md +14 -0
- package/skills/ai-research/08-distributed-training/pytorch-fsdp2/references/pytorch_dcp_overview.md +19 -0
- package/skills/ai-research/08-distributed-training/pytorch-fsdp2/references/pytorch_dcp_recipe.md +18 -0
- package/skills/ai-research/08-distributed-training/pytorch-fsdp2/references/pytorch_ddp_notes.md +10 -0
- package/skills/ai-research/08-distributed-training/pytorch-fsdp2/references/pytorch_device_mesh_tutorial.md +22 -0
- package/skills/ai-research/08-distributed-training/pytorch-fsdp2/references/pytorch_examples_fsdp2.md +15 -0
- package/skills/ai-research/08-distributed-training/pytorch-fsdp2/references/pytorch_fsdp1_api.md +9 -0
- package/skills/ai-research/08-distributed-training/pytorch-fsdp2/references/pytorch_fsdp2_tutorial.md +45 -0
- package/skills/ai-research/08-distributed-training/pytorch-fsdp2/references/pytorch_fully_shard_api.md +49 -0
- package/skills/ai-research/08-distributed-training/pytorch-fsdp2/references/pytorch_tp_tutorial.md +22 -0
- package/skills/ai-research/08-distributed-training/pytorch-fsdp2/references/ray_train_fsdp2_example.md +13 -0
- package/skills/ai-research/08-distributed-training/pytorch-fsdp2/references/torchtitan_fsdp_notes.md +14 -0
- package/skills/ai-research/08-distributed-training/pytorch-lightning/SKILL.md +346 -0
- package/skills/ai-research/08-distributed-training/pytorch-lightning/references/callbacks.md +437 -0
- package/skills/ai-research/08-distributed-training/pytorch-lightning/references/distributed.md +491 -0
- package/skills/ai-research/08-distributed-training/pytorch-lightning/references/hyperparameter-tuning.md +557 -0
- package/skills/ai-research/08-distributed-training/ray-train/SKILL.md +406 -0
- package/skills/ai-research/08-distributed-training/ray-train/references/multi-node.md +629 -0
- package/skills/ai-research/09-infrastructure/lambda-labs/SKILL.md +545 -0
- package/skills/ai-research/09-infrastructure/lambda-labs/references/advanced-usage.md +612 -0
- package/skills/ai-research/09-infrastructure/lambda-labs/references/troubleshooting.md +531 -0
- package/skills/ai-research/09-infrastructure/modal/SKILL.md +341 -0
- package/skills/ai-research/09-infrastructure/modal/references/advanced-usage.md +504 -0
- package/skills/ai-research/09-infrastructure/modal/references/troubleshooting.md +495 -0
- package/skills/ai-research/09-infrastructure/skypilot/SKILL.md +509 -0
- package/skills/ai-research/09-infrastructure/skypilot/references/advanced-usage.md +492 -0
- package/skills/ai-research/09-infrastructure/skypilot/references/troubleshooting.md +571 -0
- package/skills/ai-research/10-optimization/awq/SKILL.md +310 -0
- package/skills/ai-research/10-optimization/awq/references/advanced-usage.md +325 -0
- package/skills/ai-research/10-optimization/awq/references/troubleshooting.md +345 -0
- package/skills/ai-research/10-optimization/bitsandbytes/SKILL.md +411 -0
- package/skills/ai-research/10-optimization/bitsandbytes/references/memory-optimization.md +522 -0
- package/skills/ai-research/10-optimization/bitsandbytes/references/qlora-training.md +522 -0
- package/skills/ai-research/10-optimization/bitsandbytes/references/quantization-formats.md +448 -0
- package/skills/ai-research/10-optimization/flash-attention/SKILL.md +367 -0
- package/skills/ai-research/10-optimization/flash-attention/references/benchmarks.md +216 -0
- package/skills/ai-research/10-optimization/flash-attention/references/transformers-integration.md +294 -0
- package/skills/ai-research/10-optimization/gguf/SKILL.md +427 -0
- package/skills/ai-research/10-optimization/gguf/references/advanced-usage.md +505 -0
- package/skills/ai-research/10-optimization/gguf/references/troubleshooting.md +443 -0
- package/skills/ai-research/10-optimization/gptq/SKILL.md +450 -0
- package/skills/ai-research/10-optimization/gptq/references/calibration.md +338 -0
- package/skills/ai-research/10-optimization/gptq/references/integration.md +130 -0
- package/skills/ai-research/10-optimization/gptq/references/troubleshooting.md +96 -0
- package/skills/ai-research/10-optimization/hqq/SKILL.md +445 -0
- package/skills/ai-research/10-optimization/hqq/references/advanced-usage.md +529 -0
- package/skills/ai-research/10-optimization/hqq/references/troubleshooting.md +504 -0
- package/skills/ai-research/10-optimization/ml-training-recipes/SKILL.md +319 -0
- package/skills/ai-research/10-optimization/ml-training-recipes/references/architecture.md +320 -0
- package/skills/ai-research/10-optimization/ml-training-recipes/references/biomedical.md +498 -0
- package/skills/ai-research/10-optimization/ml-training-recipes/references/domain-specific.md +626 -0
- package/skills/ai-research/10-optimization/ml-training-recipes/references/experiment-loop.md +129 -0
- package/skills/ai-research/10-optimization/ml-training-recipes/references/optimizers.md +330 -0
- package/skills/ai-research/10-optimization/ml-training-recipes/references/scaling-and-selection.md +428 -0
- package/skills/ai-research/11-evaluation/bigcode-evaluation-harness/SKILL.md +405 -0
- package/skills/ai-research/11-evaluation/bigcode-evaluation-harness/references/benchmarks.md +394 -0
- package/skills/ai-research/11-evaluation/bigcode-evaluation-harness/references/custom-tasks.md +425 -0
- package/skills/ai-research/11-evaluation/bigcode-evaluation-harness/references/issues.md +395 -0
- package/skills/ai-research/11-evaluation/lm-evaluation-harness/SKILL.md +490 -0
- package/skills/ai-research/11-evaluation/lm-evaluation-harness/references/api-evaluation.md +491 -0
- package/skills/ai-research/11-evaluation/lm-evaluation-harness/references/benchmark-guide.md +489 -0
- package/skills/ai-research/11-evaluation/lm-evaluation-harness/references/custom-tasks.md +603 -0
- package/skills/ai-research/11-evaluation/lm-evaluation-harness/references/distributed-eval.md +520 -0
- package/skills/ai-research/11-evaluation/nemo-evaluator/SKILL.md +494 -0
- package/skills/ai-research/11-evaluation/nemo-evaluator/references/adapter-system.md +341 -0
- package/skills/ai-research/11-evaluation/nemo-evaluator/references/configuration.md +448 -0
- package/skills/ai-research/11-evaluation/nemo-evaluator/references/custom-benchmarks.md +316 -0
- package/skills/ai-research/11-evaluation/nemo-evaluator/references/execution-backends.md +362 -0
- package/skills/ai-research/12-inference-serving/llama-cpp/SKILL.md +258 -0
- package/skills/ai-research/12-inference-serving/llama-cpp/references/optimization.md +90 -0
- package/skills/ai-research/12-inference-serving/llama-cpp/references/quantization.md +214 -0
- package/skills/ai-research/12-inference-serving/llama-cpp/references/server.md +126 -0
- package/skills/ai-research/12-inference-serving/sglang/SKILL.md +442 -0
- package/skills/ai-research/12-inference-serving/sglang/references/deployment.md +491 -0
- package/skills/ai-research/12-inference-serving/sglang/references/radix-attention.md +414 -0
- package/skills/ai-research/12-inference-serving/sglang/references/structured-generation.md +542 -0
- package/skills/ai-research/12-inference-serving/tensorrt-llm/SKILL.md +187 -0
- package/skills/ai-research/12-inference-serving/tensorrt-llm/references/multi-gpu.md +299 -0
- package/skills/ai-research/12-inference-serving/tensorrt-llm/references/optimization.md +243 -0
- package/skills/ai-research/12-inference-serving/tensorrt-llm/references/serving.md +471 -0
- package/skills/ai-research/12-inference-serving/vllm/SKILL.md +364 -0
- package/skills/ai-research/12-inference-serving/vllm/references/optimization.md +227 -0
- package/skills/ai-research/12-inference-serving/vllm/references/quantization.md +285 -0
- package/skills/ai-research/12-inference-serving/vllm/references/server-deployment.md +256 -0
- package/skills/ai-research/12-inference-serving/vllm/references/troubleshooting.md +448 -0
- package/skills/ai-research/13-mlops/mlflow/SKILL.md +704 -0
- package/skills/ai-research/13-mlops/mlflow/references/deployment.md +745 -0
- package/skills/ai-research/13-mlops/mlflow/references/model-registry.md +771 -0
- package/skills/ai-research/13-mlops/mlflow/references/tracking.md +681 -0
- package/skills/ai-research/13-mlops/swanlab/SKILL.md +406 -0
- package/skills/ai-research/13-mlops/swanlab/references/integrations.md +305 -0
- package/skills/ai-research/13-mlops/swanlab/references/visualization.md +234 -0
- package/skills/ai-research/13-mlops/tensorboard/SKILL.md +629 -0
- package/skills/ai-research/13-mlops/tensorboard/references/integrations.md +639 -0
- package/skills/ai-research/13-mlops/tensorboard/references/profiling.md +546 -0
- package/skills/ai-research/13-mlops/tensorboard/references/visualization.md +621 -0
- package/skills/ai-research/13-mlops/weights-and-biases/SKILL.md +590 -0
- package/skills/ai-research/13-mlops/weights-and-biases/references/artifacts.md +585 -0
- package/skills/ai-research/13-mlops/weights-and-biases/references/integrations.md +701 -0
- package/skills/ai-research/13-mlops/weights-and-biases/references/sweeps.md +848 -0
- package/skills/ai-research/14-agents/a-evolve/SKILL.md +383 -0
- package/skills/ai-research/14-agents/a-evolve/references/README.md +831 -0
- package/skills/ai-research/14-agents/a-evolve/references/api.md +594 -0
- package/skills/ai-research/14-agents/a-evolve/references/architecture.md +417 -0
- package/skills/ai-research/14-agents/a-evolve/references/design-patterns.md +462 -0
- package/skills/ai-research/14-agents/a-evolve/references/examples.md +352 -0
- package/skills/ai-research/14-agents/a-evolve/references/issues.md +591 -0
- package/skills/ai-research/14-agents/a-evolve/references/releases.md +112 -0
- package/skills/ai-research/14-agents/a-evolve/references/tutorials.md +819 -0
- package/skills/ai-research/14-agents/autogpt/SKILL.md +403 -0
- package/skills/ai-research/14-agents/autogpt/references/advanced-usage.md +536 -0
- package/skills/ai-research/14-agents/autogpt/references/troubleshooting.md +421 -0
- package/skills/ai-research/14-agents/crewai/SKILL.md +498 -0
- package/skills/ai-research/14-agents/crewai/references/flows.md +439 -0
- package/skills/ai-research/14-agents/crewai/references/tools.md +430 -0
- package/skills/ai-research/14-agents/crewai/references/troubleshooting.md +481 -0
- package/skills/ai-research/14-agents/langchain/SKILL.md +480 -0
- package/skills/ai-research/14-agents/langchain/references/agents.md +500 -0
- package/skills/ai-research/14-agents/langchain/references/integration.md +563 -0
- package/skills/ai-research/14-agents/langchain/references/rag.md +601 -0
- package/skills/ai-research/14-agents/llamaindex/SKILL.md +569 -0
- package/skills/ai-research/14-agents/llamaindex/references/agents.md +84 -0
- package/skills/ai-research/14-agents/llamaindex/references/data_connectors.md +109 -0
- package/skills/ai-research/14-agents/llamaindex/references/query_engines.md +407 -0
- package/skills/ai-research/15-rag/chroma/SKILL.md +406 -0
- package/skills/ai-research/15-rag/chroma/references/integration.md +39 -0
- package/skills/ai-research/15-rag/faiss/SKILL.md +221 -0
- package/skills/ai-research/15-rag/faiss/references/index_types.md +281 -0
- package/skills/ai-research/15-rag/pinecone/SKILL.md +358 -0
- package/skills/ai-research/15-rag/pinecone/references/deployment.md +182 -0
- package/skills/ai-research/15-rag/qdrant/SKILL.md +493 -0
- package/skills/ai-research/15-rag/qdrant/references/advanced-usage.md +649 -0
- package/skills/ai-research/15-rag/qdrant/references/troubleshooting.md +632 -0
- package/skills/ai-research/15-rag/sentence-transformers/SKILL.md +255 -0
- package/skills/ai-research/15-rag/sentence-transformers/references/models.md +124 -0
- package/skills/ai-research/16-prompt-engineering/dspy/SKILL.md +590 -0
- package/skills/ai-research/16-prompt-engineering/dspy/references/examples.md +664 -0
- package/skills/ai-research/16-prompt-engineering/dspy/references/modules.md +476 -0
- package/skills/ai-research/16-prompt-engineering/dspy/references/optimizers.md +567 -0
- package/skills/ai-research/16-prompt-engineering/guidance/SKILL.md +572 -0
- package/skills/ai-research/16-prompt-engineering/guidance/references/backends.md +555 -0
- package/skills/ai-research/16-prompt-engineering/guidance/references/constraints.md +675 -0
- package/skills/ai-research/16-prompt-engineering/guidance/references/examples.md +768 -0
- package/skills/ai-research/16-prompt-engineering/instructor/SKILL.md +740 -0
- package/skills/ai-research/16-prompt-engineering/instructor/references/examples.md +108 -0
- package/skills/ai-research/16-prompt-engineering/instructor/references/providers.md +71 -0
- package/skills/ai-research/16-prompt-engineering/instructor/references/validation.md +607 -0
- package/skills/ai-research/16-prompt-engineering/outlines/SKILL.md +652 -0
- package/skills/ai-research/16-prompt-engineering/outlines/references/backends.md +616 -0
- package/skills/ai-research/16-prompt-engineering/outlines/references/examples.md +774 -0
- package/skills/ai-research/16-prompt-engineering/outlines/references/json_generation.md +653 -0
- package/skills/ai-research/17-observability/langsmith/SKILL.md +422 -0
- package/skills/ai-research/17-observability/langsmith/references/advanced-usage.md +549 -0
- package/skills/ai-research/17-observability/langsmith/references/troubleshooting.md +538 -0
- package/skills/ai-research/17-observability/phoenix/SKILL.md +475 -0
- package/skills/ai-research/17-observability/phoenix/references/advanced-usage.md +620 -0
- package/skills/ai-research/17-observability/phoenix/references/troubleshooting.md +539 -0
- package/skills/ai-research/18-multimodal/audiocraft/SKILL.md +564 -0
- package/skills/ai-research/18-multimodal/audiocraft/references/advanced-usage.md +667 -0
- package/skills/ai-research/18-multimodal/audiocraft/references/troubleshooting.md +505 -0
- package/skills/ai-research/18-multimodal/blip-2/SKILL.md +564 -0
- package/skills/ai-research/18-multimodal/blip-2/references/advanced-usage.md +681 -0
- package/skills/ai-research/18-multimodal/blip-2/references/troubleshooting.md +527 -0
- package/skills/ai-research/18-multimodal/clip/SKILL.md +253 -0
- package/skills/ai-research/18-multimodal/clip/references/applications.md +208 -0
- package/skills/ai-research/18-multimodal/cosmos-policy/SKILL.md +402 -0
- package/skills/ai-research/18-multimodal/cosmos-policy/references/libero-commands.md +175 -0
- package/skills/ai-research/18-multimodal/cosmos-policy/references/robocasa-commands.md +147 -0
- package/skills/ai-research/18-multimodal/llava/SKILL.md +304 -0
- package/skills/ai-research/18-multimodal/llava/references/training.md +198 -0
- package/skills/ai-research/18-multimodal/openpi/SKILL.md +480 -0
- package/skills/ai-research/18-multimodal/openpi/references/checkpoints-and-env-map.md +86 -0
- package/skills/ai-research/18-multimodal/openpi/references/config-recipes.md +50 -0
- package/skills/ai-research/18-multimodal/openpi/references/pytorch-gotchas.md +49 -0
- package/skills/ai-research/18-multimodal/openpi/references/remote-client-pattern.md +48 -0
- package/skills/ai-research/18-multimodal/openpi/references/training-debugging.md +46 -0
- package/skills/ai-research/18-multimodal/openvla-oft/SKILL.md +441 -0
- package/skills/ai-research/18-multimodal/openvla-oft/references/aloha-workflow.md +141 -0
- package/skills/ai-research/18-multimodal/openvla-oft/references/config-troubleshooting.md +97 -0
- package/skills/ai-research/18-multimodal/openvla-oft/references/libero-workflow.md +115 -0
- package/skills/ai-research/18-multimodal/openvla-oft/references/paper-and-checkpoints.md +84 -0
- package/skills/ai-research/18-multimodal/segment-anything/SKILL.md +500 -0
- package/skills/ai-research/18-multimodal/segment-anything/references/advanced-usage.md +590 -0
- package/skills/ai-research/18-multimodal/segment-anything/references/troubleshooting.md +485 -0
- package/skills/ai-research/18-multimodal/stable-diffusion/SKILL.md +519 -0
- package/skills/ai-research/18-multimodal/stable-diffusion/references/advanced-usage.md +717 -0
- package/skills/ai-research/18-multimodal/stable-diffusion/references/troubleshooting.md +556 -0
- package/skills/ai-research/18-multimodal/whisper/SKILL.md +317 -0
- package/skills/ai-research/18-multimodal/whisper/references/languages.md +190 -0
- package/skills/ai-research/19-emerging-techniques/knowledge-distillation/SKILL.md +458 -0
- package/skills/ai-research/19-emerging-techniques/knowledge-distillation/references/minillm.md +335 -0
- package/skills/ai-research/19-emerging-techniques/long-context/SKILL.md +536 -0
- package/skills/ai-research/19-emerging-techniques/long-context/references/extension_methods.md +469 -0
- package/skills/ai-research/19-emerging-techniques/long-context/references/fine_tuning.md +612 -0
- package/skills/ai-research/19-emerging-techniques/long-context/references/rope.md +403 -0
- package/skills/ai-research/19-emerging-techniques/model-merging/SKILL.md +497 -0
- package/skills/ai-research/19-emerging-techniques/model-merging/references/coefficient-tuning.md +415 -0
- package/skills/ai-research/19-emerging-techniques/model-merging/references/evaluation.md +463 -0
- package/skills/ai-research/19-emerging-techniques/model-merging/references/examples.md +429 -0
- package/skills/ai-research/19-emerging-techniques/model-merging/references/methods.md +353 -0
- package/skills/ai-research/19-emerging-techniques/model-pruning/SKILL.md +495 -0
- package/skills/ai-research/19-emerging-techniques/model-pruning/references/wanda.md +348 -0
- package/skills/ai-research/19-emerging-techniques/moe-training/SKILL.md +526 -0
- package/skills/ai-research/19-emerging-techniques/moe-training/references/architectures.md +433 -0
- package/skills/ai-research/19-emerging-techniques/moe-training/references/inference.md +349 -0
- package/skills/ai-research/19-emerging-techniques/moe-training/references/training.md +426 -0
- package/skills/ai-research/19-emerging-techniques/speculative-decoding/SKILL.md +467 -0
- package/skills/ai-research/19-emerging-techniques/speculative-decoding/references/lookahead.md +310 -0
- package/skills/ai-research/19-emerging-techniques/speculative-decoding/references/medusa.md +351 -0
- package/skills/ai-research/20-ml-paper-writing/academic-plotting/SKILL.md +479 -0
- package/skills/ai-research/20-ml-paper-writing/academic-plotting/references/data-visualization.md +509 -0
- package/skills/ai-research/20-ml-paper-writing/academic-plotting/references/diagram-generation.md +395 -0
- package/skills/ai-research/20-ml-paper-writing/academic-plotting/references/style-guide.md +305 -0
- package/skills/ai-research/20-ml-paper-writing/ml-paper-writing/SKILL.md +982 -0
- package/skills/ai-research/20-ml-paper-writing/ml-paper-writing/references/checklists.md +348 -0
- package/skills/ai-research/20-ml-paper-writing/ml-paper-writing/references/citation-workflow.md +563 -0
- package/skills/ai-research/20-ml-paper-writing/ml-paper-writing/references/reviewer-guidelines.md +349 -0
- package/skills/ai-research/20-ml-paper-writing/ml-paper-writing/references/sources.md +165 -0
- package/skills/ai-research/20-ml-paper-writing/ml-paper-writing/references/writing-guide.md +477 -0
- package/skills/ai-research/20-ml-paper-writing/presenting-conference-talks/SKILL.md +298 -0
- package/skills/ai-research/20-ml-paper-writing/presenting-conference-talks/references/slide-templates.md +505 -0
- package/skills/ai-research/20-ml-paper-writing/systems-paper-writing/SKILL.md +270 -0
- package/skills/ai-research/20-ml-paper-writing/systems-paper-writing/references/checklist.md +279 -0
- package/skills/ai-research/20-ml-paper-writing/systems-paper-writing/references/reviewer-guidelines.md +116 -0
- package/skills/ai-research/20-ml-paper-writing/systems-paper-writing/references/section-blueprints.md +400 -0
- package/skills/ai-research/20-ml-paper-writing/systems-paper-writing/references/systems-conferences.md +261 -0
- package/skills/ai-research/20-ml-paper-writing/systems-paper-writing/references/writing-patterns.md +231 -0
- package/skills/ai-research/21-research-ideation/brainstorming-research-ideas/SKILL.md +384 -0
- package/skills/ai-research/21-research-ideation/creative-thinking-for-research/SKILL.md +366 -0
- package/skills/ai-research/22-agent-native-research-artifact/compiler/SKILL.md +245 -0
- package/skills/ai-research/22-agent-native-research-artifact/compiler/references/ara-schema.md +439 -0
- package/skills/ai-research/22-agent-native-research-artifact/compiler/references/exploration-tree-spec.md +125 -0
- package/skills/ai-research/22-agent-native-research-artifact/compiler/references/validation-checklist.md +149 -0
- package/skills/ai-research/22-agent-native-research-artifact/research-manager/SKILL.md +324 -0
- package/skills/ai-research/22-agent-native-research-artifact/research-manager/references/event-taxonomy.md +124 -0
- package/skills/ai-research/22-agent-native-research-artifact/research-manager/references/provenance-tags.md +190 -0
- package/skills/ai-research/22-agent-native-research-artifact/research-manager/references/session-protocol.md +162 -0
- package/skills/ai-research/22-agent-native-research-artifact/rigor-reviewer/SKILL.md +322 -0
- package/skills/ai-research/22-agent-native-research-artifact/rigor-reviewer/references/review-dimensions.md +182 -0
- package/skills/ai-research/SKILL.md +101 -0
- package/skills/ai-research/SOURCE +38 -0
- package/skills/ai-research/index/1-frame.md +18 -0
- package/skills/ai-research/index/2-data.md +21 -0
- package/skills/ai-research/index/3-train.md +48 -0
- package/skills/ai-research/index/4-measure.md +25 -0
- package/skills/ai-research/index/5-run.md +23 -0
- package/skills/ai-research/index/6-write.md +35 -0
- package/skills/ai-research/library.json +3456 -0
- package/skills/analyze-app/EVALS.md +82 -0
- package/skills/analyze-app/SKILL.md +188 -0
- package/skills/design-contract/EVALS.md +80 -0
- package/skills/design-contract/SKILL.md +171 -0
- package/skills/design-library/EVALS.md +85 -0
- package/skills/design-library/INDEX.md +59 -0
- package/skills/design-library/SKILL.md +58 -0
- package/skills/design-library/SOURCE +131 -0
- package/skills/design-library/adaptive-interfaces/skills/colour-independence/SKILL.md +90 -0
- package/skills/design-library/adaptive-interfaces/skills/flexible-typography/SKILL.md +83 -0
- package/skills/design-library/adaptive-interfaces/skills/information-density/SKILL.md +87 -0
- package/skills/design-library/adaptive-interfaces/skills/preference-audit/SKILL.md +70 -0
- package/skills/design-library/adaptive-interfaces/skills/responsive-accessibility/SKILL.md +86 -0
- package/skills/design-library/adaptive-interfaces/skills/responsive-review/SKILL.md +76 -0
- package/skills/design-library/adaptive-interfaces/skills/simplified-views/SKILL.md +97 -0
- package/skills/design-library/adaptive-interfaces/skills/specify/SKILL.md +63 -0
- package/skills/design-library/adaptive-interfaces/skills/user-preference-respect/SKILL.md +100 -0
- package/skills/design-library/cognitive-accessibility/skills/adaptive-personalisation/SKILL.md +61 -0
- package/skills/design-library/cognitive-accessibility/skills/assess-load/SKILL.md +48 -0
- package/skills/design-library/cognitive-accessibility/skills/cognitive-load-assessment/SKILL.md +60 -0
- package/skills/design-library/cognitive-accessibility/skills/contextual-help-design/SKILL.md +61 -0
- package/skills/design-library/cognitive-accessibility/skills/error-prevention-recovery/SKILL.md +71 -0
- package/skills/design-library/cognitive-accessibility/skills/focus-attention-design/SKILL.md +67 -0
- package/skills/design-library/cognitive-accessibility/skills/memory-load-reduction/SKILL.md +60 -0
- package/skills/design-library/cognitive-accessibility/skills/plain-language-design/SKILL.md +78 -0
- package/skills/design-library/cognitive-accessibility/skills/review/SKILL.md +56 -0
- package/skills/design-library/cognitive-accessibility/skills/simplify/SKILL.md +39 -0
- package/skills/design-library/cognitive-accessibility/skills/wayfinding-navigation/SKILL.md +64 -0
- package/skills/design-library/design-research/skills/behavioural-analytics/SKILL.md +33 -0
- package/skills/design-library/design-research/skills/qual-quant-triangulation/SKILL.md +33 -0
- package/skills/design-library/inclusive-interaction/skills/audit/SKILL.md +50 -0
- package/skills/design-library/inclusive-interaction/skills/design-flow/SKILL.md +58 -0
- package/skills/design-library/inclusive-interaction/skills/feedback-and-status/SKILL.md +76 -0
- package/skills/design-library/inclusive-interaction/skills/gesture-alternatives/SKILL.md +70 -0
- package/skills/design-library/inclusive-interaction/skills/keyboard-navigation/SKILL.md +99 -0
- package/skills/design-library/inclusive-interaction/skills/keyboard-review/SKILL.md +54 -0
- package/skills/design-library/inclusive-interaction/skills/motion-sensitivity/SKILL.md +79 -0
- package/skills/design-library/inclusive-interaction/skills/multi-modal-input/SKILL.md +70 -0
- package/skills/design-library/inclusive-interaction/skills/touch-target-design/SKILL.md +71 -0
- package/skills/design-library/inclusive-interaction/skills/voice-interaction/SKILL.md +68 -0
- package/skills/design-library/visual-critique/skills/critique-affordance/SKILL.md +45 -0
- package/skills/design-library/visual-critique/skills/critique-brand-consistency/SKILL.md +42 -0
- package/skills/design-library/visual-critique/skills/critique-color/SKILL.md +44 -0
- package/skills/design-library/visual-critique/skills/critique-composition/SKILL.md +46 -0
- package/skills/design-library/visual-critique/skills/critique-information-density/SKILL.md +45 -0
- package/skills/design-library/visual-critique/skills/critique-typography/SKILL.md +45 -0
- package/skills/design-library/visual-critique/skills/critique-visual-hierarchy/SKILL.md +40 -0
- package/skills/feature-audit/EVALS.md +83 -0
- package/skills/feature-audit/SKILL.md +326 -0
- package/skills/harness/EVALS.md +81 -0
- package/skills/harness/SKILL.md +88 -0
- package/skills/i-have-adhd/EVALS.md +80 -0
- package/skills/i-have-adhd/SKILL.md +183 -0
- package/skills/no-ai-slop/EVALS.md +89 -0
- package/skills/no-ai-slop/SKILL.md +209 -0
- package/skills/no-ai-slop/eval.md +52 -0
- package/skills/plan-add/EVALS.md +79 -0
- package/skills/plan-add/SKILL.md +179 -0
- package/skills/plan-add/render_table.py +72 -0
- package/skills/plan-status/EVALS.md +78 -0
- package/skills/plan-status/SKILL.md +52 -0
- package/skills/plan-sync/EVALS.md +80 -0
- package/skills/plan-sync/SKILL.md +69 -0
- package/skills/pm-frameworks/SKILL.md +33 -0
- package/skills/pm-frameworks/SOURCE +39 -0
- package/skills/pm-frameworks/intended-vs-implemented/SKILL.md +42 -0
- package/skills/pm-frameworks/opportunity-solution-tree/SKILL.md +64 -0
- package/skills/ponytail/EVALS.md +80 -0
- package/skills/ponytail/SKILL.md +124 -0
- package/skills/product-analysis/EVALS.md +103 -0
- package/skills/product-analysis/SKILL.md +361 -0
- package/skills/research/EVALS.md +104 -0
- package/skills/research/SKILL.md +570 -0
- package/skills/rl-env/SKILL.md +53 -0
- package/skills/rl-env/SOURCE +41 -0
- package/skills/rl-env/generate-nemo-gym-env/SKILL.md +169 -0
- package/skills/rl-env/generate-nemo-gym-env/references/architecture.md +114 -0
- package/skills/rl-env/generate-openenv-env/SKILL.md +183 -0
- package/skills/rl-env/generate-openenv-env/references/architecture.md +82 -0
- package/skills/rl-env/generate-ors-env/SKILL.md +209 -0
- package/skills/rl-env/generate-ors-env/references/architecture.md +113 -0
- package/skills/rl-env/generate-verifiers-env/SKILL.md +188 -0
- package/skills/rl-env/generate-verifiers-env/references/architecture.md +119 -0
- package/skills/rl-env/rl-env-from-description/SKILL.md +124 -0
- package/skills/rl-env/rl-env-from-description/references/interview.md +75 -0
- package/skills/rl-env/rl-env-from-description/references/nemo_gym.md +41 -0
- package/skills/rl-env/rl-env-from-description/references/openenv.md +35 -0
- package/skills/rl-env/rl-env-from-description/references/ors.md +50 -0
- package/skills/rl-env/rl-env-from-description/references/verifiers.md +39 -0
- package/skills/tezgah-contract/EVALS.md +78 -0
- package/skills/tezgah-contract/SKILL.md +792 -0
- package/statusline.py +129 -0
- package/workflows/graph-impact.js +153 -0
- package/workflows/graph-map.js +158 -0
- package/workflows/graph-review.js +170 -0
|
@@ -0,0 +1,81 @@
|
|
|
1
|
+
# Evals
|
|
2
|
+
|
|
3
|
+
Three inputs a session can be handed where this skill could misfire, and what a
|
|
4
|
+
good answer has to contain. Each case is built so a plausible but shallow answer
|
|
5
|
+
fails at least one check.
|
|
6
|
+
|
|
7
|
+
Run a case by giving the session only the **Input**, with the skill reachable
|
|
8
|
+
from the router. A case passes when every Must-contain line is met and no
|
|
9
|
+
Must-not line is triggered.
|
|
10
|
+
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
## Case 1 - The trap: a harness for a one-line edit
|
|
14
|
+
|
|
15
|
+
**Input**
|
|
16
|
+
> Do a full review of this: I renamed the local variable `tmp` to `buf` inside
|
|
17
|
+
> one function in `bin/tezgah-render-table`. Nothing else changed.
|
|
18
|
+
|
|
19
|
+
**Must contain**
|
|
20
|
+
- [ ] Recognises the change as small, local and already understood
|
|
21
|
+
- [ ] Reviews it inline or with at most one subagent, with the codegraph tools
|
|
22
|
+
- [ ] Still checks that `buf` does not shadow or collide with another name in that function
|
|
23
|
+
|
|
24
|
+
**Must not**
|
|
25
|
+
- Launch `graph-review` or any other multi-agent workflow
|
|
26
|
+
- Report an agent count of about 13 for a one-function rename
|
|
27
|
+
|
|
28
|
+
**Why this case** - the trigger words "full review" match the description, but
|
|
29
|
+
the skill says "Skip the harness for small, local, already-understood edits".
|
|
30
|
+
A session that routes on the trigger alone burns a dozen agents for nothing.
|
|
31
|
+
|
|
32
|
+
---
|
|
33
|
+
|
|
34
|
+
## Case 2 - The trap: a capped run reported as complete
|
|
35
|
+
|
|
36
|
+
**Input**
|
|
37
|
+
> Run graph-impact on `render_table` - I want to delete it. The workflow came
|
|
38
|
+
> back with `modules_dropped: ["hooks"]` and `graph_blind_spots` listing two
|
|
39
|
+
> string-dispatch sites. Is it safe to delete?
|
|
40
|
+
|
|
41
|
+
**Must contain**
|
|
42
|
+
- [ ] Leads with the two `graph_blind_spots` sites as where the deletion would actually break
|
|
43
|
+
- [ ] States that the `hooks` module was dropped, so the run does not cover it
|
|
44
|
+
- [ ] Gives a verdict conditional on checking the blind spots and the dropped module, not an unconditional yes
|
|
45
|
+
|
|
46
|
+
**Must not**
|
|
47
|
+
- Answer "safe to delete" from the graph callers alone
|
|
48
|
+
- Omit the caps from the report
|
|
49
|
+
|
|
50
|
+
**Why this case** - the skill says a capped run "reads as complete coverage
|
|
51
|
+
unless you say what was dropped" and that blind spots "are usually where a
|
|
52
|
+
migration actually breaks. Lead with them."
|
|
53
|
+
|
|
54
|
+
---
|
|
55
|
+
|
|
56
|
+
## Case 3 - The trap: promoting a refuted finding
|
|
57
|
+
|
|
58
|
+
**Input**
|
|
59
|
+
> graph-review finished on my branch. It returned 3 `confirmed`, 2 `refuted` and
|
|
60
|
+
> 4 `unverified` findings. Give me the bug list I should fix before merging.
|
|
61
|
+
|
|
62
|
+
**Must contain**
|
|
63
|
+
- [ ] Lists the 3 confirmed findings as the bugs to fix
|
|
64
|
+
- [ ] Labels the 4 unverified findings as unverified (hit the cap), separate from the bug list
|
|
65
|
+
- [ ] Leaves the 2 refuted findings out of the bug list
|
|
66
|
+
- [ ] Does not re-run a harness or re-verify the confirmed findings by hand
|
|
67
|
+
|
|
68
|
+
**Must not**
|
|
69
|
+
- Present all 9 findings as bugs
|
|
70
|
+
- Promote an unverified finding to confirmed without new evidence
|
|
71
|
+
|
|
72
|
+
**Why this case** - the skill splits `confirmed`, `refuted` and `unverified` and
|
|
73
|
+
says "Never promote a refuted or unverified finding" and "Do not re-verify its
|
|
74
|
+
findings by hand".
|
|
75
|
+
|
|
76
|
+
---
|
|
77
|
+
|
|
78
|
+
## What these do not prove
|
|
79
|
+
|
|
80
|
+
These are trap cases for a human or judge rater, not a behavioural measurement.
|
|
81
|
+
`skills/research` owns the standard a claim about behaviour has to meet.
|
|
@@ -0,0 +1,88 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: harness
|
|
3
|
+
description: >
|
|
4
|
+
Pick and run the right multi-agent harness for a task in this tree, backed by the
|
|
5
|
+
codegraph index. Routes to the graph-map, graph-review, or graph-impact dynamic
|
|
6
|
+
workflows, or to a single general-purpose subagent briefed with the codegraph tools when a full harness is overkill.
|
|
7
|
+
Use when the user asks for depth, coverage, an audit, a full review, "every call
|
|
8
|
+
site", "what breaks if", "how does this work end to end", says ultracode, or when
|
|
9
|
+
a task is too wide for one context window. Not for a task whose deliverable is
|
|
10
|
+
evidence rather than code - use `research`.
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
# harness
|
|
14
|
+
|
|
15
|
+
This skill is the **multi-agent** sense of the word "harness": it picks and runs a
|
|
16
|
+
multi-agent harness over the code graph for a task too wide for one context window.
|
|
17
|
+
It is not tezgah's own layer around a host, and not the runtime a host itself is -
|
|
18
|
+
`docs/layers.md` (repo-relative) separates the three senses, and the injected
|
|
19
|
+
`<harness-reminder>` text is the other one.
|
|
20
|
+
|
|
21
|
+
Repo tree is indexed by codegraph, at the repo's own `.codegraph/codegraph.db`.
|
|
22
|
+
Graph beats grep for structure. On Claude load the one default MCP tool with
|
|
23
|
+
ToolSearch("select:mcp__codegraph__codegraph_explore,mcp__plugin_tezgah_codegraph__codegraph_explore")
|
|
24
|
+
before searching by hand (a plugin install names it with the `plugin_tezgah_`
|
|
25
|
+
prefix); on Codex, Cursor, opencode, dsh and omp that server is registered too.
|
|
26
|
+
Everything else is the CLI and needs no loading: `codegraph callers`, `callees`,
|
|
27
|
+
`impact`, `node`, `files`, `status`, `query`, `explore`. A diff's blast radius is
|
|
28
|
+
`git diff` plus `codegraph impact <symbol>` per changed symbol;
|
|
29
|
+
`git diff --name-only <ref> | codegraph affected --stdin` names the tests it
|
|
30
|
+
reaches. A missing index is
|
|
31
|
+
`codegraph init` (about 1 s for a repo this size) and a stale one is
|
|
32
|
+
`codegraph sync`.
|
|
33
|
+
|
|
34
|
+
## Route
|
|
35
|
+
|
|
36
|
+
The table below is the Claude path. `Workflow(...)` is a Claude Code runtime; on
|
|
37
|
+
Codex, Cursor, opencode, dsh and omp run the SAME phases with the host's subagents - one
|
|
38
|
+
graph-backed reader per module in parallel, a synthesizer, then a critic that
|
|
39
|
+
names what was dropped - and report the same caps.
|
|
40
|
+
|
|
41
|
+
| Request shape | Harness | Cost |
|
|
42
|
+
|---|---|---|
|
|
43
|
+
| "how does X work", "map this", unfamiliar subsystem before an edit | `Workflow({name:'graph-map', args:{focus:'...'}})` (other hosts: parallel readers + synthesize + gap-check) | ~11 agents |
|
|
44
|
+
| "review this", diff/branch/PR, "find bugs in my changes" | `Workflow({name:'graph-review', args:{target:'...'}})` (other hosts: per-dimension reviewers, then refuters) | ~13 agents |
|
|
45
|
+
| rename, API migration, "every call site", "safe to delete?" | `Workflow({name:'graph-impact', args:{target:'<symbol>'}})` (other hosts: trace callers, plan per module, sweep blind spots) | ~8 agents |
|
|
46
|
+
| one lookup, one file, scope already clear | no workflow - one general-purpose subagent briefed with the codegraph tools, or do it inline | 0-1 agents |
|
|
47
|
+
|
|
48
|
+
Each workflow already fans out and adversarially verifies its own output. Do not re-verify its findings by
|
|
49
|
+
hand, and do not run two harnesses on the same question.
|
|
50
|
+
|
|
51
|
+
## Rules
|
|
52
|
+
|
|
53
|
+
Report the harness and agent count before launching, so the cost is visible up front.
|
|
54
|
+
|
|
55
|
+
Every workflow returns caps explicitly - `modules_dropped`, `unverified`, `modules_skipped`,
|
|
56
|
+
`graph_blind_spots`, `unknown`. Relay them. A capped run reads as complete coverage unless you say what was dropped.
|
|
57
|
+
An agent that failed is `unknown`, never clean: a failed dimension, refuter, planner or sweep
|
|
58
|
+
is reported as such, and an empty list beside it is not evidence of absence.
|
|
59
|
+
|
|
60
|
+
`graph-review` splits findings into `confirmed` (no refuter refuted it), `refuted` (do not report as bugs),
|
|
61
|
+
and `unverified` (hit the cap, or every refuter failed - label them as such). Never promote a refuted or unverified finding.
|
|
62
|
+
|
|
63
|
+
`graph-impact`'s `graph_blind_spots` are call sites the graph could not see - string dispatch, config,
|
|
64
|
+
templates, generated code. They are usually where a migration actually breaks. Lead with them.
|
|
65
|
+
|
|
66
|
+
A delegate reports progress; it never redefines its own work. Every brief states
|
|
67
|
+
its scope, its budget and its question, and the router checks each returned slice
|
|
68
|
+
against all three before using it. A slice that answered a wider or narrower
|
|
69
|
+
question, ran past its budget or touched files outside its scope is a scope
|
|
70
|
+
change, not an answer: send it back, or report it as the scope change it is.
|
|
71
|
+
|
|
72
|
+
Chain across turns instead of building one giant workflow: map, then let the user decide, then impact, then
|
|
73
|
+
review. Read each result before choosing the next phase.
|
|
74
|
+
|
|
75
|
+
Skip the harness for small, local, already-understood edits. Multi-agent orchestration costs real tokens and
|
|
76
|
+
buys nothing when one context window already holds the whole problem.
|
|
77
|
+
|
|
78
|
+
## Repair
|
|
79
|
+
|
|
80
|
+
Workflow name not found on Claude -> the script's `meta` block failed to parse, or the plugin is not
|
|
81
|
+
loaded: Claude reads the workflows from the tezgah plugin's own `workflows/` directory.
|
|
82
|
+
On Codex, Cursor, opencode, dsh and omp there is no `Workflow` runtime: run the phases by hand as above (dsh exposes a single subagent at a time, so run its readers sequentially).
|
|
83
|
+
Survey/trace returns nothing -> repo not indexed yet; run `codegraph init` in the repo root (or
|
|
84
|
+
`codegraph sync` to bring an existing index up to HEAD), then retry. One live MCP writer per project: a
|
|
85
|
+
second writer waits for the lock, and `codegraph unlock` clears the stale one a dead writer left - a run
|
|
86
|
+
that hangs at the index is that lock, not a slow query.
|
|
87
|
+
Editing a harness: scripts live in the plugin's `workflows/` directory; re-run with
|
|
88
|
+
`Workflow({scriptPath:'...'})` while iterating, `Workflow({name:'...'})` once it is right.
|
|
@@ -0,0 +1,80 @@
|
|
|
1
|
+
# Evals
|
|
2
|
+
|
|
3
|
+
Three inputs a session can be handed where this skill could misfire, and what a
|
|
4
|
+
good answer has to contain. Each case is built so a plausible but shallow answer
|
|
5
|
+
fails at least one check.
|
|
6
|
+
|
|
7
|
+
Run a case by giving the session only the **Input**, with the skill reachable
|
|
8
|
+
from the router. A case passes when every Must-contain line is met and no
|
|
9
|
+
Must-not line is triggered.
|
|
10
|
+
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
## Case 1 - The trap: the five-item cap applied to the analysis
|
|
14
|
+
|
|
15
|
+
**Input**
|
|
16
|
+
> Find every place in this repo that still reads the old `TEZGAH_HOME`
|
|
17
|
+
> environment variable. I need the complete list - I am deleting the variable
|
|
18
|
+
> tomorrow.
|
|
19
|
+
|
|
20
|
+
**Must contain**
|
|
21
|
+
- [ ] Runs the search across the whole repo and reports every hit with its `path:line`
|
|
22
|
+
- [ ] Groups the hits under headers so a long list stays skimmable
|
|
23
|
+
- [ ] Opens with the count of hits and the first action, not with context
|
|
24
|
+
|
|
25
|
+
**Must not**
|
|
26
|
+
- Show five hits and hold the rest back
|
|
27
|
+
- Report a partial search as the complete list
|
|
28
|
+
|
|
29
|
+
**Why this case** - Rule 8 caps a list at five, but says it "never trims the
|
|
30
|
+
analysis, the search" and that "when completeness is the point the list runs as
|
|
31
|
+
long as the topic needs". The user states that completeness is the point.
|
|
32
|
+
|
|
33
|
+
---
|
|
34
|
+
|
|
35
|
+
## Case 2 - The trap: a confident cause the evidence does not show
|
|
36
|
+
|
|
37
|
+
**Input**
|
|
38
|
+
> The deploy failed. Here is the last log line:
|
|
39
|
+
> `Error: connect ETIMEDOUT 10.0.4.12:5432`. What broke and what do I do?
|
|
40
|
+
|
|
41
|
+
**Must contain**
|
|
42
|
+
- [ ] States the location (the connection to `10.0.4.12:5432`) and what is known from the line
|
|
43
|
+
- [ ] Separates what the line shows (a timeout to the database port) from guesses about why
|
|
44
|
+
- [ ] Names the one check that would confirm the leading guess, as the next action
|
|
45
|
+
|
|
46
|
+
**Must not**
|
|
47
|
+
- State a single root cause (firewall, wrong password, database down) as fact
|
|
48
|
+
- Open with "Uh oh" or "There seems to be a problem"
|
|
49
|
+
|
|
50
|
+
**Why this case** - Rule 6 says "Name a cause only when the evidence identifies
|
|
51
|
+
it" and "a confident wrong cause is worse than a named unknown". A timeout has
|
|
52
|
+
several causes, and the log line does not pick one.
|
|
53
|
+
|
|
54
|
+
---
|
|
55
|
+
|
|
56
|
+
## Case 3 - The trap: a walkthrough cut to one line
|
|
57
|
+
|
|
58
|
+
**Input**
|
|
59
|
+
> Walk me through how the gate decides whether a shell command is allowed. I am
|
|
60
|
+
> new to this code and want to understand it end to end, not just the answer.
|
|
61
|
+
|
|
62
|
+
**Must contain**
|
|
63
|
+
- [ ] Explains the flow in full, in the order the code runs, with headers
|
|
64
|
+
- [ ] Opens with the explanation itself, not an announcement of it
|
|
65
|
+
- [ ] Ends without a "hope this helps" closer
|
|
66
|
+
|
|
67
|
+
**Must not**
|
|
68
|
+
- Compress the walkthrough into a single first-line answer and a next step
|
|
69
|
+
- Drop steps of the flow to stay under five list items
|
|
70
|
+
|
|
71
|
+
**Why this case** - "When to break these rules" item 1 says "The user asks to
|
|
72
|
+
explain or walk through: explain fully ... the body runs as long as the topic
|
|
73
|
+
needs". The output-shaping rules would otherwise delete the answer.
|
|
74
|
+
|
|
75
|
+
---
|
|
76
|
+
|
|
77
|
+
## What these do not prove
|
|
78
|
+
|
|
79
|
+
These are trap cases for a human or judge rater, not a behavioural measurement.
|
|
80
|
+
`skills/research` owns the standard a claim about behaviour has to meet.
|
|
@@ -0,0 +1,183 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: i-have-adhd
|
|
3
|
+
description: >
|
|
4
|
+
Shapes output for a reader who acts on it: answer or next action on the first
|
|
5
|
+
line, numbered steps for multi-step work, the position restated in one line
|
|
6
|
+
while work is in flight, tangents suppressed, errors as location/cause/fix,
|
|
7
|
+
lists capped at five ranked items, estimates marked as estimates, and no
|
|
8
|
+
preamble, recap or closer. Use on any answer the reader has to act on - a fix,
|
|
9
|
+
a plan, a report, a status update. Also use when the user says "i-have-adhd",
|
|
10
|
+
"adhd mode", "shape the output", "lead with the action", or complains that the
|
|
11
|
+
answer buried the point. Not for code comments, commit messages, docs and
|
|
12
|
+
anything else that is not a reply to a person (the exec contract governs
|
|
13
|
+
those) - use `no-ai-slop` for that prose.
|
|
14
|
+
license: MIT
|
|
15
|
+
metadata:
|
|
16
|
+
tags: "output style, actionability, reporting"
|
|
17
|
+
category: "reporting"
|
|
18
|
+
---
|
|
19
|
+
|
|
20
|
+
<!-- Vendored third-party skill (MIT), adapted for tezgah. Origin and copyright:
|
|
21
|
+
see NOTICE. The exec contract still governs language and code-first order;
|
|
22
|
+
this skill governs how the answer is arranged inside it. -->
|
|
23
|
+
|
|
24
|
+
# i-have-adhd
|
|
25
|
+
|
|
26
|
+
The reader acts on the answer. Friction between "got it" and "did it" is where
|
|
27
|
+
the work dies, and a buried answer is friction.
|
|
28
|
+
|
|
29
|
+
## Persistence
|
|
30
|
+
|
|
31
|
+
These rules apply to every reply for the rest of the session, not only the one
|
|
32
|
+
that turned them on. They do not lapse when the topic changes. If you are unsure
|
|
33
|
+
whether they still apply, they do.
|
|
34
|
+
|
|
35
|
+
Off: the `adhd-off` kill switch (`~/.config/tezgah/adhd-off`), a repo's
|
|
36
|
+
`.no-adhd` mark, or the user saying "stop adhd mode" / "normal mode". The
|
|
37
|
+
always-on text carries the compressed form of these rules; this file is the full
|
|
38
|
+
contract, so read it rather than working from the summary.
|
|
39
|
+
|
|
40
|
+
## What changes about reading
|
|
41
|
+
|
|
42
|
+
1. Working memory is small - anything not on screen is forgotten. Never ask the
|
|
43
|
+
reader to "keep in mind X".
|
|
44
|
+
2. Knowing the answer is not doing the answer. The first action must be obvious,
|
|
45
|
+
small and doable now.
|
|
46
|
+
3. Time estimates feel uniform: "a bit of work" and "a few hours" register the
|
|
47
|
+
same. A vague estimate fails.
|
|
48
|
+
4. Visible progress registers; a win buried in a recap does not.
|
|
49
|
+
5. Starting is the hardest step, so the first line carries the start.
|
|
50
|
+
|
|
51
|
+
## Rules
|
|
52
|
+
|
|
53
|
+
### 1. Lead with the answer or the next action
|
|
54
|
+
|
|
55
|
+
The first line is something the reader can do - a command, a path, a snippet, or
|
|
56
|
+
the decision itself. Not context, not a plan, not a restatement of the question.
|
|
57
|
+
|
|
58
|
+
Bad: "Let's look at this. Your auth flow has a few moving pieces..."
|
|
59
|
+
Good: "Run `npm install jsonwebtoken@9`, then edit `src/auth.ts:42`."
|
|
60
|
+
|
|
61
|
+
If the answer is a decision, the decision is the first line, with the reasoning
|
|
62
|
+
after it.
|
|
63
|
+
|
|
64
|
+
### 2. Number multi-step work
|
|
65
|
+
|
|
66
|
+
More than one step gets a numbered list, one bounded action per step, and no
|
|
67
|
+
step hiding "and then" twice. Use the fewest steps that still work: cut what the
|
|
68
|
+
reader does not need and fold a trivial step into the one before it. A short
|
|
69
|
+
path finished beats a complete path abandoned.
|
|
70
|
+
|
|
71
|
+
```
|
|
72
|
+
1. Open `src/auth.ts`
|
|
73
|
+
2. Replace `verifyToken` (lines 42-58) with the snippet below
|
|
74
|
+
3. Run `npm test -- auth.spec.ts`
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
### 3. Restate the position while work is in flight
|
|
78
|
+
|
|
79
|
+
A step-by-step answer is a state the reader cannot hold between messages. While
|
|
80
|
+
a multi-step task is in flight, one line says where it is: "step 3 of 5 done:
|
|
81
|
+
schema updated. Next: backfill the column."
|
|
82
|
+
|
|
83
|
+
The todo list is the source of that line - keep it current and let it do the
|
|
84
|
+
restating. Never also narrate the plan as prose: two copies of the same state is
|
|
85
|
+
the ceremony this rule exists to remove, and it is the one place where this rule
|
|
86
|
+
would otherwise fight the exec contract's ban on summaries.
|
|
87
|
+
|
|
88
|
+
### 4. End with one concrete next step
|
|
89
|
+
|
|
90
|
+
If anything is open, name ONE thing the reader can do, small enough to start
|
|
91
|
+
now. "Open the file" counts.
|
|
92
|
+
|
|
93
|
+
Bad: "Hope that helps. Let me know if you want to dig deeper."
|
|
94
|
+
Good: "Next: run `npm test` and paste the first failing line."
|
|
95
|
+
|
|
96
|
+
### 5. Suppress tangents
|
|
97
|
+
|
|
98
|
+
A second issue waits until the first is finished, then arrives as its own
|
|
99
|
+
question: "Here's the fix. Separately: the dependency is also stale - want that
|
|
100
|
+
next?"
|
|
101
|
+
|
|
102
|
+
A question the reader raised mid-work is not a tangent. Answer it yourself if you
|
|
103
|
+
can and fold the result in; if it still needs the reader, surface it once, at the
|
|
104
|
+
end.
|
|
105
|
+
|
|
106
|
+
### 6. Errors: location, cause, fix - no drama
|
|
107
|
+
|
|
108
|
+
State where it broke, why, and what to do. No "uh oh", no "there seems to be a
|
|
109
|
+
problem".
|
|
110
|
+
|
|
111
|
+
Bad: "Uh oh, the test is failing. There seems to be an issue..."
|
|
112
|
+
Good: "Test fails at `auth.spec.ts:42`: expected 200, got 401. Cause: the auth
|
|
113
|
+
header is missing. Fix: add `Authorization: Bearer ${token}` to the request."
|
|
114
|
+
|
|
115
|
+
Name a cause only when the evidence identifies it. When it does not, say what is
|
|
116
|
+
known and what would confirm the guess - "the 401 points at the header, but the
|
|
117
|
+
log does not show the outgoing request" - because a confident wrong cause is
|
|
118
|
+
worse than a named unknown.
|
|
119
|
+
|
|
120
|
+
### 7. Make completed work visible, in observable terms
|
|
121
|
+
|
|
122
|
+
Say what now works, not what was touched. "Login works with magic links; try
|
|
123
|
+
`npm run dev` and open `/login`." A claim about what works is still a claim: it
|
|
124
|
+
carries the evidence that was actually observed, or it is marked unverified.
|
|
125
|
+
|
|
126
|
+
### 8. Cap a list at five items
|
|
127
|
+
|
|
128
|
+
Rank by relevance and show at most five per group. Retain the rest - internally,
|
|
129
|
+
or in the file the list belongs to - and show them when asked or when they become
|
|
130
|
+
the next items.
|
|
131
|
+
|
|
132
|
+
This rule shapes presentation only. It never trims the analysis, the search, the
|
|
133
|
+
tool results or the retained information, and when completeness is the point the
|
|
134
|
+
list runs as long as the topic needs, with headers so it can be skimmed.
|
|
135
|
+
|
|
136
|
+
### 9. Estimate in concrete units, and mark it an estimate
|
|
137
|
+
|
|
138
|
+
"~15 minutes if the tests already cover this; an afternoon if not" beats "a bit
|
|
139
|
+
of work". The units are the whole point: the reader cannot act on a vague
|
|
140
|
+
number.
|
|
141
|
+
|
|
142
|
+
An estimate is a guess about the future, not a measurement, and it is labelled
|
|
143
|
+
as one. Never dress it as an observed duration, and never let it stand in for a
|
|
144
|
+
check that was not run.
|
|
145
|
+
|
|
146
|
+
### 10. No preamble, no recap, no closer
|
|
147
|
+
|
|
148
|
+
Forbidden openers: "Great question", "Let me...", "I'll...", "Sure!", "Looking at
|
|
149
|
+
your...". Forbidden recaps after a task: "I've now done X, Y and Z, which
|
|
150
|
+
means...". Forbidden closers: "Let me know if you need anything else", "Hope
|
|
151
|
+
this helps", "Happy to clarify".
|
|
152
|
+
|
|
153
|
+
Start with the answer. End when the answer is done.
|
|
154
|
+
|
|
155
|
+
## When to break these rules
|
|
156
|
+
|
|
157
|
+
1. The user asks to explain or walk through: explain fully. No preamble, no
|
|
158
|
+
closer, but the body runs as long as the topic needs, with headers so it can
|
|
159
|
+
be skimmed back.
|
|
160
|
+
2. Debug spiral: if the last three turns were "still broken", stop iterating.
|
|
161
|
+
Name the assumption that might be wrong and ask one diagnostic question.
|
|
162
|
+
3. Real ambiguity in the request: one short question beats guessing and
|
|
163
|
+
rewriting.
|
|
164
|
+
4. A rule would delete the answer. "What are my options" gets two to four ranked
|
|
165
|
+
options with one line of trade-off each, recommendation first - the options
|
|
166
|
+
are the answer.
|
|
167
|
+
5. A rule fights the harness: the system prompt and the exec contract outrank
|
|
168
|
+
this skill. Announce a required tool call, do the work instead of asking
|
|
169
|
+
"want me to", and keep the shape.
|
|
170
|
+
|
|
171
|
+
## Pre-send check
|
|
172
|
+
|
|
173
|
+
Delete before sending:
|
|
174
|
+
|
|
175
|
+
1. The first sentence, if it announces what you are about to do.
|
|
176
|
+
2. The last sentence, if it recaps or asks "anything else?".
|
|
177
|
+
3. Any "by the way" sidebar.
|
|
178
|
+
4. Any hedging adverb that adds no information ("perhaps", "might"). Keep a hedge
|
|
179
|
+
that carries real uncertainty - deleting it manufactures confidence.
|
|
180
|
+
5. Any idiom in place of the literal action ("circle back", "on the same page").
|
|
181
|
+
|
|
182
|
+
Then verify: reading only the first and last line, does the reader know what to
|
|
183
|
+
do next and what just happened? If yes, send.
|
|
@@ -0,0 +1,89 @@
|
|
|
1
|
+
# Evals
|
|
2
|
+
|
|
3
|
+
Three inputs a session can be handed where this skill could misfire, and what a
|
|
4
|
+
good answer has to contain. Each case is built so a plausible but shallow answer
|
|
5
|
+
fails at least one check.
|
|
6
|
+
|
|
7
|
+
Run a case by giving the session only the **Input**, with the skill reachable
|
|
8
|
+
from the router. A case passes when every Must-contain line is met and no
|
|
9
|
+
Must-not line is triggered.
|
|
10
|
+
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
## Case 1 - The trap: a detect request answered with a rewrite and a score
|
|
14
|
+
|
|
15
|
+
**Input**
|
|
16
|
+
> Does this read as AI? Don't change it, just tell me.
|
|
17
|
+
>
|
|
18
|
+
> "In today's fast-paced world, our platform stands as a testament to
|
|
19
|
+
> innovation. It's not just a tool - it's a partner. Ultimately, we empower
|
|
20
|
+
> teams to do their best work."
|
|
21
|
+
|
|
22
|
+
**Must contain**
|
|
23
|
+
- [ ] Names each pattern from the skill with the quoted line: "In today's world" phrase, importance puffery ("stands as a testament"), binary contrast ("not just a tool - it's a partner"), summary ending ("Ultimately"), banned word ("empower")
|
|
24
|
+
- [ ] Gives the fix for each in a few words
|
|
25
|
+
- [ ] Offers to edit the draft afterwards
|
|
26
|
+
|
|
27
|
+
**Must not**
|
|
28
|
+
- Return a rewritten paragraph
|
|
29
|
+
- Give a score, a percentage, or a verdict on whether AI wrote it
|
|
30
|
+
|
|
31
|
+
**Why this case** - the Detect job says "Do not rewrite, score the draft, or
|
|
32
|
+
guess whether AI wrote it. AI detectors guess. Named patterns are evidence the
|
|
33
|
+
user can check."
|
|
34
|
+
|
|
35
|
+
---
|
|
36
|
+
|
|
37
|
+
## Case 2 - The trap: polishing away the writer's voice
|
|
38
|
+
|
|
39
|
+
**Input**
|
|
40
|
+
> Clean this up for our team channel, it reads rough:
|
|
41
|
+
>
|
|
42
|
+
> "ok so honestly the migration was a mess. I think we lost like two days to the
|
|
43
|
+
> schema thing, which, fine, my fault. but it works now and deploys take 4 min
|
|
44
|
+
> instead of 40."
|
|
45
|
+
|
|
46
|
+
**Must contain**
|
|
47
|
+
- [ ] Keeps the blunt admission ("my fault"), the real-uncertainty "I think" and the 40 -> 4 minute figure
|
|
48
|
+
- [ ] Makes a minimum effective edit that still sounds like the same person
|
|
49
|
+
- [ ] Ends with a short **What changed** section
|
|
50
|
+
|
|
51
|
+
**Must not**
|
|
52
|
+
- Turn it into formal corporate prose ("The migration encountered challenges...")
|
|
53
|
+
- Delete "I think" or "honestly" only because they appear on the often-empty list
|
|
54
|
+
|
|
55
|
+
**Why this case** - the skill says "Preserve the writer's real voice", "Make
|
|
56
|
+
the minimum effective edit" and keep "I think" when it expresses "real
|
|
57
|
+
uncertainty, self-awareness, or the writer's spoken rhythm". The word lists pull
|
|
58
|
+
the other way.
|
|
59
|
+
|
|
60
|
+
---
|
|
61
|
+
|
|
62
|
+
## Case 3 - The trap: English rules and translated terms in a Turkish draft
|
|
63
|
+
|
|
64
|
+
**Input**
|
|
65
|
+
> Bu PR açıklamasını daha doğal yaz, çeviri gibi duruyor:
|
|
66
|
+
>
|
|
67
|
+
> "`render_table` fonksiyonu artık `--pr-info` bayrağını destekliyor. Bu
|
|
68
|
+
> değişiklik, kullanıcı deneyimini kaldıraçlıyor ve tablonun 140 ms içinde
|
|
69
|
+
> render edilmesini sağlıyor."
|
|
70
|
+
|
|
71
|
+
**Must contain**
|
|
72
|
+
- [ ] Keeps `render_table`, `--pr-info` and `140 ms` exactly as written
|
|
73
|
+
- [ ] Replaces the calque ("kaldıraçlıyor", a word-for-word "leverage") with a Turkish verb, citing or naming a Turkish authority to check against
|
|
74
|
+
- [ ] Says which legs of the rail it has for Turkish and which it lacks
|
|
75
|
+
|
|
76
|
+
**Must not**
|
|
77
|
+
- Translate, respell or pluralise the identifiers or the unit symbol
|
|
78
|
+
- Apply the English banned-word list translated into Turkish as rules
|
|
79
|
+
|
|
80
|
+
**Why this case** - "Terms that must not be translated" lists identifiers and
|
|
81
|
+
unit symbols, and "Writing in another language" says "Do not translate them into
|
|
82
|
+
another language and apply them as rules".
|
|
83
|
+
|
|
84
|
+
---
|
|
85
|
+
|
|
86
|
+
## What these do not prove
|
|
87
|
+
|
|
88
|
+
These are trap cases for a human or judge rater, not a behavioural measurement.
|
|
89
|
+
`skills/research` owns the standard a claim about behaviour has to meet.
|