@r1z4x/tezgah 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +738 -0
- package/LICENSE +21 -0
- package/MANIFEST +753 -0
- package/README.de.md +169 -0
- package/README.es.md +168 -0
- package/README.fr.md +166 -0
- package/README.md +161 -0
- package/README.tr.md +163 -0
- package/agents/tezgah-reviewer.md +70 -0
- package/bin/codegen +337 -0
- package/bin/consult +797 -0
- package/bin/tezgah-adhd +50 -0
- package/bin/tezgah-agents +45 -0
- package/bin/tezgah-capture +66 -0
- package/bin/tezgah-context +96 -0
- package/bin/tezgah-decisions +213 -0
- package/bin/tezgah-design +953 -0
- package/bin/tezgah-docs +1199 -0
- package/bin/tezgah-doctor +497 -0
- package/bin/tezgah-dsh +37 -0
- package/bin/tezgah-dsh.cmd +70 -0
- package/bin/tezgah-gate +75 -0
- package/bin/tezgah-import-ai-research +439 -0
- package/bin/tezgah-index +32 -0
- package/bin/tezgah-lessons +148 -0
- package/bin/tezgah-mcp +322 -0
- package/bin/tezgah-migrate +324 -0
- package/bin/tezgah-pony +50 -0
- package/bin/tezgah-research +772 -0
- package/bin/tezgah-rollback +93 -0
- package/bin/tezgah-route +124 -0
- package/bin/tezgah-setup +5798 -0
- package/bin/tezgah-status +383 -0
- package/bin/tezgah-task +607 -0
- package/bin/tezgah-taste +423 -0
- package/bin/tezgah-triage +485 -0
- package/bin/tezgah.js +36 -0
- package/commands/adhd.md +21 -0
- package/commands/ponytail.md +21 -0
- package/docs/README.md +112 -0
- package/docs/architecture.md +226 -0
- package/docs/citations-baseline.json +59 -0
- package/docs/clarity-baseline.json +116 -0
- package/docs/contract.md +416 -0
- package/docs/environment.md +141 -0
- package/docs/evidence.md +831 -0
- package/docs/feature-audit.md +139 -0
- package/docs/gate.md +407 -0
- package/docs/glossary.md +201 -0
- package/docs/hosts.md +400 -0
- package/docs/index.json +471 -0
- package/docs/judge.md +193 -0
- package/docs/layers.md +189 -0
- package/docs/models.md +160 -0
- package/docs/operations.md +666 -0
- package/docs/research.md +809 -0
- package/docs/results/paired-outcome-2026-10.jsonl +254 -0
- package/docs/results/paired-outcome-2026-10.md +204 -0
- package/docs/skills.md +236 -0
- package/docs/status-line.md +315 -0
- package/docs/testing.md +287 -0
- package/docs/vision.md +117 -0
- package/hooks/hooks.json +33 -0
- package/hooks/projects-auto-init.py +98 -0
- package/hooks/projects-posttooluse.py +149 -0
- package/hooks/projects-pretooluse.py +40 -0
- package/hooks/projects-stop.py +60 -0
- package/hooks/tezgah_agents.py +817 -0
- package/hooks/tezgah_apps.py +266 -0
- package/hooks/tezgah_attest.py +255 -0
- package/hooks/tezgah_clarity.py +171 -0
- package/hooks/tezgah_components.py +140 -0
- package/hooks/tezgah_context.py +2953 -0
- package/hooks/tezgah_embed.py +765 -0
- package/hooks/tezgah_gate.py +2736 -0
- package/hooks/tezgah_guard.py +177 -0
- package/hooks/tezgah_index.py +124 -0
- package/hooks/tezgah_integrity.py +5141 -0
- package/hooks/tezgah_judge.py +487 -0
- package/hooks/tezgah_lang.py +230 -0
- package/hooks/tezgah_lessons.py +210 -0
- package/hooks/tezgah_models.py +855 -0
- package/hooks/tezgah_paths.py +1132 -0
- package/hooks/tezgah_policy.py +1039 -0
- package/hooks/tezgah_rank.py +77 -0
- package/hooks/tezgah_replay.py +1109 -0
- package/hooks/tezgah_research.py +5176 -0
- package/hooks/tezgah_shapes.py +305 -0
- package/hooks/tezgah_skill_pick.py +227 -0
- package/hooks/tezgah_snapshot.py +490 -0
- package/hooks/tezgah_stopspec.py +320 -0
- package/hooks/tezgah_task.py +458 -0
- package/hooks/tezgah_taste.py +201 -0
- package/hooks/tezgah_untrusted.py +112 -0
- package/hooks/tezgah_update.py +430 -0
- package/hosts/codex/hook.py +268 -0
- package/hosts/codex/hooks.json +26 -0
- package/hosts/cursor/hook.py +461 -0
- package/hosts/cursor/hooks.json +44 -0
- package/hosts/dsh/hooks.json +21 -0
- package/hosts/dsh/statusline/lib/client.js +138 -0
- package/hosts/dsh/statusline/lib/index.js +104 -0
- package/hosts/dsh/statusline/package.json +28 -0
- package/hosts/omp/hook.py +277 -0
- package/hosts/omp/tezgah-hook.ts.in +701 -0
- package/hosts/opencode/plugins/tezgah.js +2742 -0
- package/hosts/opencode/tui/tezgah-tui.tsx +151 -0
- package/output-styles/tezgah.md +142 -0
- package/package.json +32 -0
- package/packaging/build.sh +342 -0
- package/packaging/install.ps1 +166 -0
- package/packaging/install.sh +56 -0
- package/packaging/upgrade.sh +142 -0
- package/skills/ai-research/0-autoresearch-skill/SKILL.md +411 -0
- package/skills/ai-research/0-autoresearch-skill/references/agent-continuity.md +95 -0
- package/skills/ai-research/0-autoresearch-skill/references/progress-reporting.md +166 -0
- package/skills/ai-research/0-autoresearch-skill/references/skill-routing.md +219 -0
- package/skills/ai-research/0-autoresearch-skill/templates/findings.md +44 -0
- package/skills/ai-research/0-autoresearch-skill/templates/research-log.md +41 -0
- package/skills/ai-research/01-model-architecture/litgpt/SKILL.md +469 -0
- package/skills/ai-research/01-model-architecture/litgpt/references/custom-models.md +569 -0
- package/skills/ai-research/01-model-architecture/litgpt/references/distributed-training.md +452 -0
- package/skills/ai-research/01-model-architecture/litgpt/references/supported-models.md +337 -0
- package/skills/ai-research/01-model-architecture/litgpt/references/training-recipes.md +620 -0
- package/skills/ai-research/01-model-architecture/mamba/SKILL.md +260 -0
- package/skills/ai-research/01-model-architecture/mamba/references/architecture-details.md +207 -0
- package/skills/ai-research/01-model-architecture/mamba/references/benchmarks.md +256 -0
- package/skills/ai-research/01-model-architecture/mamba/references/training-guide.md +389 -0
- package/skills/ai-research/01-model-architecture/nanogpt/SKILL.md +290 -0
- package/skills/ai-research/01-model-architecture/nanogpt/references/architecture.md +383 -0
- package/skills/ai-research/01-model-architecture/nanogpt/references/data.md +477 -0
- package/skills/ai-research/01-model-architecture/nanogpt/references/training.md +565 -0
- package/skills/ai-research/01-model-architecture/rwkv/SKILL.md +260 -0
- package/skills/ai-research/01-model-architecture/rwkv/references/architecture-details.md +345 -0
- package/skills/ai-research/01-model-architecture/rwkv/references/rwkv7.md +387 -0
- package/skills/ai-research/01-model-architecture/rwkv/references/state-management.md +370 -0
- package/skills/ai-research/01-model-architecture/torchtitan/SKILL.md +358 -0
- package/skills/ai-research/01-model-architecture/torchtitan/references/checkpoint.md +182 -0
- package/skills/ai-research/01-model-architecture/torchtitan/references/custom-models.md +259 -0
- package/skills/ai-research/01-model-architecture/torchtitan/references/float8.md +134 -0
- package/skills/ai-research/01-model-architecture/torchtitan/references/fsdp.md +127 -0
- package/skills/ai-research/02-tokenization/huggingface-tokenizers/SKILL.md +516 -0
- package/skills/ai-research/02-tokenization/huggingface-tokenizers/references/algorithms.md +654 -0
- package/skills/ai-research/02-tokenization/huggingface-tokenizers/references/integration.md +638 -0
- package/skills/ai-research/02-tokenization/huggingface-tokenizers/references/pipeline.md +724 -0
- package/skills/ai-research/02-tokenization/huggingface-tokenizers/references/training.md +566 -0
- package/skills/ai-research/02-tokenization/sentencepiece/SKILL.md +235 -0
- package/skills/ai-research/02-tokenization/sentencepiece/references/algorithms.md +201 -0
- package/skills/ai-research/02-tokenization/sentencepiece/references/training.md +305 -0
- package/skills/ai-research/03-fine-tuning/axolotl/SKILL.md +158 -0
- package/skills/ai-research/03-fine-tuning/axolotl/references/api.md +5549 -0
- package/skills/ai-research/03-fine-tuning/axolotl/references/dataset-formats.md +1030 -0
- package/skills/ai-research/03-fine-tuning/axolotl/references/index.md +16 -0
- package/skills/ai-research/03-fine-tuning/axolotl/references/other.md +3564 -0
- package/skills/ai-research/03-fine-tuning/llama-factory/SKILL.md +80 -0
- package/skills/ai-research/03-fine-tuning/llama-factory/references/_images.md +24 -0
- package/skills/ai-research/03-fine-tuning/llama-factory/references/advanced.md +1056 -0
- package/skills/ai-research/03-fine-tuning/llama-factory/references/getting_started.md +350 -0
- package/skills/ai-research/03-fine-tuning/llama-factory/references/index.md +20 -0
- package/skills/ai-research/03-fine-tuning/llama-factory/references/other.md +32 -0
- package/skills/ai-research/03-fine-tuning/peft/SKILL.md +431 -0
- package/skills/ai-research/03-fine-tuning/peft/references/advanced-usage.md +515 -0
- package/skills/ai-research/03-fine-tuning/peft/references/troubleshooting.md +481 -0
- package/skills/ai-research/03-fine-tuning/unsloth/SKILL.md +80 -0
- package/skills/ai-research/03-fine-tuning/unsloth/references/index.md +8 -0
- package/skills/ai-research/03-fine-tuning/unsloth/references/llms.md +83 -0
- package/skills/ai-research/04-mechanistic-interpretability/nnsight/SKILL.md +436 -0
- package/skills/ai-research/04-mechanistic-interpretability/nnsight/references/README.md +79 -0
- package/skills/ai-research/04-mechanistic-interpretability/nnsight/references/api.md +345 -0
- package/skills/ai-research/04-mechanistic-interpretability/nnsight/references/tutorials.md +301 -0
- package/skills/ai-research/04-mechanistic-interpretability/pyvene/SKILL.md +473 -0
- package/skills/ai-research/04-mechanistic-interpretability/pyvene/references/README.md +74 -0
- package/skills/ai-research/04-mechanistic-interpretability/pyvene/references/api.md +384 -0
- package/skills/ai-research/04-mechanistic-interpretability/pyvene/references/tutorials.md +377 -0
- package/skills/ai-research/04-mechanistic-interpretability/saelens/SKILL.md +386 -0
- package/skills/ai-research/04-mechanistic-interpretability/saelens/references/README.md +71 -0
- package/skills/ai-research/04-mechanistic-interpretability/saelens/references/api.md +334 -0
- package/skills/ai-research/04-mechanistic-interpretability/saelens/references/tutorials.md +319 -0
- package/skills/ai-research/04-mechanistic-interpretability/transformer-lens/SKILL.md +346 -0
- package/skills/ai-research/04-mechanistic-interpretability/transformer-lens/references/README.md +55 -0
- package/skills/ai-research/04-mechanistic-interpretability/transformer-lens/references/api.md +363 -0
- package/skills/ai-research/04-mechanistic-interpretability/transformer-lens/references/tutorials.md +340 -0
- package/skills/ai-research/05-data-processing/nemo-curator/SKILL.md +383 -0
- package/skills/ai-research/05-data-processing/nemo-curator/references/deduplication.md +88 -0
- package/skills/ai-research/05-data-processing/nemo-curator/references/filtering.md +103 -0
- package/skills/ai-research/05-data-processing/ray-data/SKILL.md +326 -0
- package/skills/ai-research/05-data-processing/ray-data/references/integration.md +83 -0
- package/skills/ai-research/05-data-processing/ray-data/references/transformations.md +84 -0
- package/skills/ai-research/06-post-training/grpo-rl-training/README.md +98 -0
- package/skills/ai-research/06-post-training/grpo-rl-training/SKILL.md +572 -0
- package/skills/ai-research/06-post-training/miles/SKILL.md +315 -0
- package/skills/ai-research/06-post-training/miles/references/api-reference.md +142 -0
- package/skills/ai-research/06-post-training/miles/references/troubleshooting.md +353 -0
- package/skills/ai-research/06-post-training/openrlhf/SKILL.md +249 -0
- package/skills/ai-research/06-post-training/openrlhf/references/algorithm-comparison.md +405 -0
- package/skills/ai-research/06-post-training/openrlhf/references/custom-rewards.md +531 -0
- package/skills/ai-research/06-post-training/openrlhf/references/hybrid-engine.md +288 -0
- package/skills/ai-research/06-post-training/openrlhf/references/multi-node-training.md +455 -0
- package/skills/ai-research/06-post-training/simpo/SKILL.md +219 -0
- package/skills/ai-research/06-post-training/simpo/references/datasets.md +479 -0
- package/skills/ai-research/06-post-training/simpo/references/hyperparameters.md +453 -0
- package/skills/ai-research/06-post-training/simpo/references/loss-functions.md +351 -0
- package/skills/ai-research/06-post-training/slime/SKILL.md +464 -0
- package/skills/ai-research/06-post-training/slime/references/api-reference.md +393 -0
- package/skills/ai-research/06-post-training/slime/references/troubleshooting.md +387 -0
- package/skills/ai-research/06-post-training/torchforge/SKILL.md +433 -0
- package/skills/ai-research/06-post-training/torchforge/references/api-reference.md +328 -0
- package/skills/ai-research/06-post-training/torchforge/references/troubleshooting.md +410 -0
- package/skills/ai-research/06-post-training/trl-fine-tuning/SKILL.md +455 -0
- package/skills/ai-research/06-post-training/trl-fine-tuning/references/dpo-variants.md +228 -0
- package/skills/ai-research/06-post-training/trl-fine-tuning/references/online-rl.md +83 -0
- package/skills/ai-research/06-post-training/trl-fine-tuning/references/reward-modeling.md +123 -0
- package/skills/ai-research/06-post-training/trl-fine-tuning/references/sft-training.md +169 -0
- package/skills/ai-research/06-post-training/verl/SKILL.md +391 -0
- package/skills/ai-research/06-post-training/verl/references/api-reference.md +302 -0
- package/skills/ai-research/06-post-training/verl/references/troubleshooting.md +392 -0
- package/skills/ai-research/07-safety-alignment/constitutional-ai/SKILL.md +290 -0
- package/skills/ai-research/07-safety-alignment/llamaguard/SKILL.md +337 -0
- package/skills/ai-research/07-safety-alignment/nemo-guardrails/SKILL.md +297 -0
- package/skills/ai-research/07-safety-alignment/prompt-guard/SKILL.md +312 -0
- package/skills/ai-research/08-distributed-training/accelerate/SKILL.md +332 -0
- package/skills/ai-research/08-distributed-training/accelerate/references/custom-plugins.md +454 -0
- package/skills/ai-research/08-distributed-training/accelerate/references/megatron-integration.md +490 -0
- package/skills/ai-research/08-distributed-training/accelerate/references/performance.md +526 -0
- package/skills/ai-research/08-distributed-training/deepspeed/SKILL.md +141 -0
- package/skills/ai-research/08-distributed-training/deepspeed/references/08.md +18 -0
- package/skills/ai-research/08-distributed-training/deepspeed/references/09.md +174 -0
- package/skills/ai-research/08-distributed-training/deepspeed/references/2020.md +379 -0
- package/skills/ai-research/08-distributed-training/deepspeed/references/2023.md +280 -0
- package/skills/ai-research/08-distributed-training/deepspeed/references/assets.md +180 -0
- package/skills/ai-research/08-distributed-training/deepspeed/references/index.md +36 -0
- package/skills/ai-research/08-distributed-training/deepspeed/references/mii.md +119 -0
- package/skills/ai-research/08-distributed-training/deepspeed/references/other.md +1192 -0
- package/skills/ai-research/08-distributed-training/megatron-core/SKILL.md +366 -0
- package/skills/ai-research/08-distributed-training/megatron-core/references/benchmarks.md +250 -0
- package/skills/ai-research/08-distributed-training/megatron-core/references/parallelism-guide.md +405 -0
- package/skills/ai-research/08-distributed-training/megatron-core/references/production-examples.md +474 -0
- package/skills/ai-research/08-distributed-training/megatron-core/references/training-recipes.md +548 -0
- package/skills/ai-research/08-distributed-training/pytorch-fsdp2/SKILL.md +231 -0
- package/skills/ai-research/08-distributed-training/pytorch-fsdp2/references/pytorch_dcp_async_recipe.md +14 -0
- package/skills/ai-research/08-distributed-training/pytorch-fsdp2/references/pytorch_dcp_overview.md +19 -0
- package/skills/ai-research/08-distributed-training/pytorch-fsdp2/references/pytorch_dcp_recipe.md +18 -0
- package/skills/ai-research/08-distributed-training/pytorch-fsdp2/references/pytorch_ddp_notes.md +10 -0
- package/skills/ai-research/08-distributed-training/pytorch-fsdp2/references/pytorch_device_mesh_tutorial.md +22 -0
- package/skills/ai-research/08-distributed-training/pytorch-fsdp2/references/pytorch_examples_fsdp2.md +15 -0
- package/skills/ai-research/08-distributed-training/pytorch-fsdp2/references/pytorch_fsdp1_api.md +9 -0
- package/skills/ai-research/08-distributed-training/pytorch-fsdp2/references/pytorch_fsdp2_tutorial.md +45 -0
- package/skills/ai-research/08-distributed-training/pytorch-fsdp2/references/pytorch_fully_shard_api.md +49 -0
- package/skills/ai-research/08-distributed-training/pytorch-fsdp2/references/pytorch_tp_tutorial.md +22 -0
- package/skills/ai-research/08-distributed-training/pytorch-fsdp2/references/ray_train_fsdp2_example.md +13 -0
- package/skills/ai-research/08-distributed-training/pytorch-fsdp2/references/torchtitan_fsdp_notes.md +14 -0
- package/skills/ai-research/08-distributed-training/pytorch-lightning/SKILL.md +346 -0
- package/skills/ai-research/08-distributed-training/pytorch-lightning/references/callbacks.md +437 -0
- package/skills/ai-research/08-distributed-training/pytorch-lightning/references/distributed.md +491 -0
- package/skills/ai-research/08-distributed-training/pytorch-lightning/references/hyperparameter-tuning.md +557 -0
- package/skills/ai-research/08-distributed-training/ray-train/SKILL.md +406 -0
- package/skills/ai-research/08-distributed-training/ray-train/references/multi-node.md +629 -0
- package/skills/ai-research/09-infrastructure/lambda-labs/SKILL.md +545 -0
- package/skills/ai-research/09-infrastructure/lambda-labs/references/advanced-usage.md +612 -0
- package/skills/ai-research/09-infrastructure/lambda-labs/references/troubleshooting.md +531 -0
- package/skills/ai-research/09-infrastructure/modal/SKILL.md +341 -0
- package/skills/ai-research/09-infrastructure/modal/references/advanced-usage.md +504 -0
- package/skills/ai-research/09-infrastructure/modal/references/troubleshooting.md +495 -0
- package/skills/ai-research/09-infrastructure/skypilot/SKILL.md +509 -0
- package/skills/ai-research/09-infrastructure/skypilot/references/advanced-usage.md +492 -0
- package/skills/ai-research/09-infrastructure/skypilot/references/troubleshooting.md +571 -0
- package/skills/ai-research/10-optimization/awq/SKILL.md +310 -0
- package/skills/ai-research/10-optimization/awq/references/advanced-usage.md +325 -0
- package/skills/ai-research/10-optimization/awq/references/troubleshooting.md +345 -0
- package/skills/ai-research/10-optimization/bitsandbytes/SKILL.md +411 -0
- package/skills/ai-research/10-optimization/bitsandbytes/references/memory-optimization.md +522 -0
- package/skills/ai-research/10-optimization/bitsandbytes/references/qlora-training.md +522 -0
- package/skills/ai-research/10-optimization/bitsandbytes/references/quantization-formats.md +448 -0
- package/skills/ai-research/10-optimization/flash-attention/SKILL.md +367 -0
- package/skills/ai-research/10-optimization/flash-attention/references/benchmarks.md +216 -0
- package/skills/ai-research/10-optimization/flash-attention/references/transformers-integration.md +294 -0
- package/skills/ai-research/10-optimization/gguf/SKILL.md +427 -0
- package/skills/ai-research/10-optimization/gguf/references/advanced-usage.md +505 -0
- package/skills/ai-research/10-optimization/gguf/references/troubleshooting.md +443 -0
- package/skills/ai-research/10-optimization/gptq/SKILL.md +450 -0
- package/skills/ai-research/10-optimization/gptq/references/calibration.md +338 -0
- package/skills/ai-research/10-optimization/gptq/references/integration.md +130 -0
- package/skills/ai-research/10-optimization/gptq/references/troubleshooting.md +96 -0
- package/skills/ai-research/10-optimization/hqq/SKILL.md +445 -0
- package/skills/ai-research/10-optimization/hqq/references/advanced-usage.md +529 -0
- package/skills/ai-research/10-optimization/hqq/references/troubleshooting.md +504 -0
- package/skills/ai-research/10-optimization/ml-training-recipes/SKILL.md +319 -0
- package/skills/ai-research/10-optimization/ml-training-recipes/references/architecture.md +320 -0
- package/skills/ai-research/10-optimization/ml-training-recipes/references/biomedical.md +498 -0
- package/skills/ai-research/10-optimization/ml-training-recipes/references/domain-specific.md +626 -0
- package/skills/ai-research/10-optimization/ml-training-recipes/references/experiment-loop.md +129 -0
- package/skills/ai-research/10-optimization/ml-training-recipes/references/optimizers.md +330 -0
- package/skills/ai-research/10-optimization/ml-training-recipes/references/scaling-and-selection.md +428 -0
- package/skills/ai-research/11-evaluation/bigcode-evaluation-harness/SKILL.md +405 -0
- package/skills/ai-research/11-evaluation/bigcode-evaluation-harness/references/benchmarks.md +394 -0
- package/skills/ai-research/11-evaluation/bigcode-evaluation-harness/references/custom-tasks.md +425 -0
- package/skills/ai-research/11-evaluation/bigcode-evaluation-harness/references/issues.md +395 -0
- package/skills/ai-research/11-evaluation/lm-evaluation-harness/SKILL.md +490 -0
- package/skills/ai-research/11-evaluation/lm-evaluation-harness/references/api-evaluation.md +491 -0
- package/skills/ai-research/11-evaluation/lm-evaluation-harness/references/benchmark-guide.md +489 -0
- package/skills/ai-research/11-evaluation/lm-evaluation-harness/references/custom-tasks.md +603 -0
- package/skills/ai-research/11-evaluation/lm-evaluation-harness/references/distributed-eval.md +520 -0
- package/skills/ai-research/11-evaluation/nemo-evaluator/SKILL.md +494 -0
- package/skills/ai-research/11-evaluation/nemo-evaluator/references/adapter-system.md +341 -0
- package/skills/ai-research/11-evaluation/nemo-evaluator/references/configuration.md +448 -0
- package/skills/ai-research/11-evaluation/nemo-evaluator/references/custom-benchmarks.md +316 -0
- package/skills/ai-research/11-evaluation/nemo-evaluator/references/execution-backends.md +362 -0
- package/skills/ai-research/12-inference-serving/llama-cpp/SKILL.md +258 -0
- package/skills/ai-research/12-inference-serving/llama-cpp/references/optimization.md +90 -0
- package/skills/ai-research/12-inference-serving/llama-cpp/references/quantization.md +214 -0
- package/skills/ai-research/12-inference-serving/llama-cpp/references/server.md +126 -0
- package/skills/ai-research/12-inference-serving/sglang/SKILL.md +442 -0
- package/skills/ai-research/12-inference-serving/sglang/references/deployment.md +491 -0
- package/skills/ai-research/12-inference-serving/sglang/references/radix-attention.md +414 -0
- package/skills/ai-research/12-inference-serving/sglang/references/structured-generation.md +542 -0
- package/skills/ai-research/12-inference-serving/tensorrt-llm/SKILL.md +187 -0
- package/skills/ai-research/12-inference-serving/tensorrt-llm/references/multi-gpu.md +299 -0
- package/skills/ai-research/12-inference-serving/tensorrt-llm/references/optimization.md +243 -0
- package/skills/ai-research/12-inference-serving/tensorrt-llm/references/serving.md +471 -0
- package/skills/ai-research/12-inference-serving/vllm/SKILL.md +364 -0
- package/skills/ai-research/12-inference-serving/vllm/references/optimization.md +227 -0
- package/skills/ai-research/12-inference-serving/vllm/references/quantization.md +285 -0
- package/skills/ai-research/12-inference-serving/vllm/references/server-deployment.md +256 -0
- package/skills/ai-research/12-inference-serving/vllm/references/troubleshooting.md +448 -0
- package/skills/ai-research/13-mlops/mlflow/SKILL.md +704 -0
- package/skills/ai-research/13-mlops/mlflow/references/deployment.md +745 -0
- package/skills/ai-research/13-mlops/mlflow/references/model-registry.md +771 -0
- package/skills/ai-research/13-mlops/mlflow/references/tracking.md +681 -0
- package/skills/ai-research/13-mlops/swanlab/SKILL.md +406 -0
- package/skills/ai-research/13-mlops/swanlab/references/integrations.md +305 -0
- package/skills/ai-research/13-mlops/swanlab/references/visualization.md +234 -0
- package/skills/ai-research/13-mlops/tensorboard/SKILL.md +629 -0
- package/skills/ai-research/13-mlops/tensorboard/references/integrations.md +639 -0
- package/skills/ai-research/13-mlops/tensorboard/references/profiling.md +546 -0
- package/skills/ai-research/13-mlops/tensorboard/references/visualization.md +621 -0
- package/skills/ai-research/13-mlops/weights-and-biases/SKILL.md +590 -0
- package/skills/ai-research/13-mlops/weights-and-biases/references/artifacts.md +585 -0
- package/skills/ai-research/13-mlops/weights-and-biases/references/integrations.md +701 -0
- package/skills/ai-research/13-mlops/weights-and-biases/references/sweeps.md +848 -0
- package/skills/ai-research/14-agents/a-evolve/SKILL.md +383 -0
- package/skills/ai-research/14-agents/a-evolve/references/README.md +831 -0
- package/skills/ai-research/14-agents/a-evolve/references/api.md +594 -0
- package/skills/ai-research/14-agents/a-evolve/references/architecture.md +417 -0
- package/skills/ai-research/14-agents/a-evolve/references/design-patterns.md +462 -0
- package/skills/ai-research/14-agents/a-evolve/references/examples.md +352 -0
- package/skills/ai-research/14-agents/a-evolve/references/issues.md +591 -0
- package/skills/ai-research/14-agents/a-evolve/references/releases.md +112 -0
- package/skills/ai-research/14-agents/a-evolve/references/tutorials.md +819 -0
- package/skills/ai-research/14-agents/autogpt/SKILL.md +403 -0
- package/skills/ai-research/14-agents/autogpt/references/advanced-usage.md +536 -0
- package/skills/ai-research/14-agents/autogpt/references/troubleshooting.md +421 -0
- package/skills/ai-research/14-agents/crewai/SKILL.md +498 -0
- package/skills/ai-research/14-agents/crewai/references/flows.md +439 -0
- package/skills/ai-research/14-agents/crewai/references/tools.md +430 -0
- package/skills/ai-research/14-agents/crewai/references/troubleshooting.md +481 -0
- package/skills/ai-research/14-agents/langchain/SKILL.md +480 -0
- package/skills/ai-research/14-agents/langchain/references/agents.md +500 -0
- package/skills/ai-research/14-agents/langchain/references/integration.md +563 -0
- package/skills/ai-research/14-agents/langchain/references/rag.md +601 -0
- package/skills/ai-research/14-agents/llamaindex/SKILL.md +569 -0
- package/skills/ai-research/14-agents/llamaindex/references/agents.md +84 -0
- package/skills/ai-research/14-agents/llamaindex/references/data_connectors.md +109 -0
- package/skills/ai-research/14-agents/llamaindex/references/query_engines.md +407 -0
- package/skills/ai-research/15-rag/chroma/SKILL.md +406 -0
- package/skills/ai-research/15-rag/chroma/references/integration.md +39 -0
- package/skills/ai-research/15-rag/faiss/SKILL.md +221 -0
- package/skills/ai-research/15-rag/faiss/references/index_types.md +281 -0
- package/skills/ai-research/15-rag/pinecone/SKILL.md +358 -0
- package/skills/ai-research/15-rag/pinecone/references/deployment.md +182 -0
- package/skills/ai-research/15-rag/qdrant/SKILL.md +493 -0
- package/skills/ai-research/15-rag/qdrant/references/advanced-usage.md +649 -0
- package/skills/ai-research/15-rag/qdrant/references/troubleshooting.md +632 -0
- package/skills/ai-research/15-rag/sentence-transformers/SKILL.md +255 -0
- package/skills/ai-research/15-rag/sentence-transformers/references/models.md +124 -0
- package/skills/ai-research/16-prompt-engineering/dspy/SKILL.md +590 -0
- package/skills/ai-research/16-prompt-engineering/dspy/references/examples.md +664 -0
- package/skills/ai-research/16-prompt-engineering/dspy/references/modules.md +476 -0
- package/skills/ai-research/16-prompt-engineering/dspy/references/optimizers.md +567 -0
- package/skills/ai-research/16-prompt-engineering/guidance/SKILL.md +572 -0
- package/skills/ai-research/16-prompt-engineering/guidance/references/backends.md +555 -0
- package/skills/ai-research/16-prompt-engineering/guidance/references/constraints.md +675 -0
- package/skills/ai-research/16-prompt-engineering/guidance/references/examples.md +768 -0
- package/skills/ai-research/16-prompt-engineering/instructor/SKILL.md +740 -0
- package/skills/ai-research/16-prompt-engineering/instructor/references/examples.md +108 -0
- package/skills/ai-research/16-prompt-engineering/instructor/references/providers.md +71 -0
- package/skills/ai-research/16-prompt-engineering/instructor/references/validation.md +607 -0
- package/skills/ai-research/16-prompt-engineering/outlines/SKILL.md +652 -0
- package/skills/ai-research/16-prompt-engineering/outlines/references/backends.md +616 -0
- package/skills/ai-research/16-prompt-engineering/outlines/references/examples.md +774 -0
- package/skills/ai-research/16-prompt-engineering/outlines/references/json_generation.md +653 -0
- package/skills/ai-research/17-observability/langsmith/SKILL.md +422 -0
- package/skills/ai-research/17-observability/langsmith/references/advanced-usage.md +549 -0
- package/skills/ai-research/17-observability/langsmith/references/troubleshooting.md +538 -0
- package/skills/ai-research/17-observability/phoenix/SKILL.md +475 -0
- package/skills/ai-research/17-observability/phoenix/references/advanced-usage.md +620 -0
- package/skills/ai-research/17-observability/phoenix/references/troubleshooting.md +539 -0
- package/skills/ai-research/18-multimodal/audiocraft/SKILL.md +564 -0
- package/skills/ai-research/18-multimodal/audiocraft/references/advanced-usage.md +667 -0
- package/skills/ai-research/18-multimodal/audiocraft/references/troubleshooting.md +505 -0
- package/skills/ai-research/18-multimodal/blip-2/SKILL.md +564 -0
- package/skills/ai-research/18-multimodal/blip-2/references/advanced-usage.md +681 -0
- package/skills/ai-research/18-multimodal/blip-2/references/troubleshooting.md +527 -0
- package/skills/ai-research/18-multimodal/clip/SKILL.md +253 -0
- package/skills/ai-research/18-multimodal/clip/references/applications.md +208 -0
- package/skills/ai-research/18-multimodal/cosmos-policy/SKILL.md +402 -0
- package/skills/ai-research/18-multimodal/cosmos-policy/references/libero-commands.md +175 -0
- package/skills/ai-research/18-multimodal/cosmos-policy/references/robocasa-commands.md +147 -0
- package/skills/ai-research/18-multimodal/llava/SKILL.md +304 -0
- package/skills/ai-research/18-multimodal/llava/references/training.md +198 -0
- package/skills/ai-research/18-multimodal/openpi/SKILL.md +480 -0
- package/skills/ai-research/18-multimodal/openpi/references/checkpoints-and-env-map.md +86 -0
- package/skills/ai-research/18-multimodal/openpi/references/config-recipes.md +50 -0
- package/skills/ai-research/18-multimodal/openpi/references/pytorch-gotchas.md +49 -0
- package/skills/ai-research/18-multimodal/openpi/references/remote-client-pattern.md +48 -0
- package/skills/ai-research/18-multimodal/openpi/references/training-debugging.md +46 -0
- package/skills/ai-research/18-multimodal/openvla-oft/SKILL.md +441 -0
- package/skills/ai-research/18-multimodal/openvla-oft/references/aloha-workflow.md +141 -0
- package/skills/ai-research/18-multimodal/openvla-oft/references/config-troubleshooting.md +97 -0
- package/skills/ai-research/18-multimodal/openvla-oft/references/libero-workflow.md +115 -0
- package/skills/ai-research/18-multimodal/openvla-oft/references/paper-and-checkpoints.md +84 -0
- package/skills/ai-research/18-multimodal/segment-anything/SKILL.md +500 -0
- package/skills/ai-research/18-multimodal/segment-anything/references/advanced-usage.md +590 -0
- package/skills/ai-research/18-multimodal/segment-anything/references/troubleshooting.md +485 -0
- package/skills/ai-research/18-multimodal/stable-diffusion/SKILL.md +519 -0
- package/skills/ai-research/18-multimodal/stable-diffusion/references/advanced-usage.md +717 -0
- package/skills/ai-research/18-multimodal/stable-diffusion/references/troubleshooting.md +556 -0
- package/skills/ai-research/18-multimodal/whisper/SKILL.md +317 -0
- package/skills/ai-research/18-multimodal/whisper/references/languages.md +190 -0
- package/skills/ai-research/19-emerging-techniques/knowledge-distillation/SKILL.md +458 -0
- package/skills/ai-research/19-emerging-techniques/knowledge-distillation/references/minillm.md +335 -0
- package/skills/ai-research/19-emerging-techniques/long-context/SKILL.md +536 -0
- package/skills/ai-research/19-emerging-techniques/long-context/references/extension_methods.md +469 -0
- package/skills/ai-research/19-emerging-techniques/long-context/references/fine_tuning.md +612 -0
- package/skills/ai-research/19-emerging-techniques/long-context/references/rope.md +403 -0
- package/skills/ai-research/19-emerging-techniques/model-merging/SKILL.md +497 -0
- package/skills/ai-research/19-emerging-techniques/model-merging/references/coefficient-tuning.md +415 -0
- package/skills/ai-research/19-emerging-techniques/model-merging/references/evaluation.md +463 -0
- package/skills/ai-research/19-emerging-techniques/model-merging/references/examples.md +429 -0
- package/skills/ai-research/19-emerging-techniques/model-merging/references/methods.md +353 -0
- package/skills/ai-research/19-emerging-techniques/model-pruning/SKILL.md +495 -0
- package/skills/ai-research/19-emerging-techniques/model-pruning/references/wanda.md +348 -0
- package/skills/ai-research/19-emerging-techniques/moe-training/SKILL.md +526 -0
- package/skills/ai-research/19-emerging-techniques/moe-training/references/architectures.md +433 -0
- package/skills/ai-research/19-emerging-techniques/moe-training/references/inference.md +349 -0
- package/skills/ai-research/19-emerging-techniques/moe-training/references/training.md +426 -0
- package/skills/ai-research/19-emerging-techniques/speculative-decoding/SKILL.md +467 -0
- package/skills/ai-research/19-emerging-techniques/speculative-decoding/references/lookahead.md +310 -0
- package/skills/ai-research/19-emerging-techniques/speculative-decoding/references/medusa.md +351 -0
- package/skills/ai-research/20-ml-paper-writing/academic-plotting/SKILL.md +479 -0
- package/skills/ai-research/20-ml-paper-writing/academic-plotting/references/data-visualization.md +509 -0
- package/skills/ai-research/20-ml-paper-writing/academic-plotting/references/diagram-generation.md +395 -0
- package/skills/ai-research/20-ml-paper-writing/academic-plotting/references/style-guide.md +305 -0
- package/skills/ai-research/20-ml-paper-writing/ml-paper-writing/SKILL.md +982 -0
- package/skills/ai-research/20-ml-paper-writing/ml-paper-writing/references/checklists.md +348 -0
- package/skills/ai-research/20-ml-paper-writing/ml-paper-writing/references/citation-workflow.md +563 -0
- package/skills/ai-research/20-ml-paper-writing/ml-paper-writing/references/reviewer-guidelines.md +349 -0
- package/skills/ai-research/20-ml-paper-writing/ml-paper-writing/references/sources.md +165 -0
- package/skills/ai-research/20-ml-paper-writing/ml-paper-writing/references/writing-guide.md +477 -0
- package/skills/ai-research/20-ml-paper-writing/presenting-conference-talks/SKILL.md +298 -0
- package/skills/ai-research/20-ml-paper-writing/presenting-conference-talks/references/slide-templates.md +505 -0
- package/skills/ai-research/20-ml-paper-writing/systems-paper-writing/SKILL.md +270 -0
- package/skills/ai-research/20-ml-paper-writing/systems-paper-writing/references/checklist.md +279 -0
- package/skills/ai-research/20-ml-paper-writing/systems-paper-writing/references/reviewer-guidelines.md +116 -0
- package/skills/ai-research/20-ml-paper-writing/systems-paper-writing/references/section-blueprints.md +400 -0
- package/skills/ai-research/20-ml-paper-writing/systems-paper-writing/references/systems-conferences.md +261 -0
- package/skills/ai-research/20-ml-paper-writing/systems-paper-writing/references/writing-patterns.md +231 -0
- package/skills/ai-research/21-research-ideation/brainstorming-research-ideas/SKILL.md +384 -0
- package/skills/ai-research/21-research-ideation/creative-thinking-for-research/SKILL.md +366 -0
- package/skills/ai-research/22-agent-native-research-artifact/compiler/SKILL.md +245 -0
- package/skills/ai-research/22-agent-native-research-artifact/compiler/references/ara-schema.md +439 -0
- package/skills/ai-research/22-agent-native-research-artifact/compiler/references/exploration-tree-spec.md +125 -0
- package/skills/ai-research/22-agent-native-research-artifact/compiler/references/validation-checklist.md +149 -0
- package/skills/ai-research/22-agent-native-research-artifact/research-manager/SKILL.md +324 -0
- package/skills/ai-research/22-agent-native-research-artifact/research-manager/references/event-taxonomy.md +124 -0
- package/skills/ai-research/22-agent-native-research-artifact/research-manager/references/provenance-tags.md +190 -0
- package/skills/ai-research/22-agent-native-research-artifact/research-manager/references/session-protocol.md +162 -0
- package/skills/ai-research/22-agent-native-research-artifact/rigor-reviewer/SKILL.md +322 -0
- package/skills/ai-research/22-agent-native-research-artifact/rigor-reviewer/references/review-dimensions.md +182 -0
- package/skills/ai-research/SKILL.md +101 -0
- package/skills/ai-research/SOURCE +38 -0
- package/skills/ai-research/index/1-frame.md +18 -0
- package/skills/ai-research/index/2-data.md +21 -0
- package/skills/ai-research/index/3-train.md +48 -0
- package/skills/ai-research/index/4-measure.md +25 -0
- package/skills/ai-research/index/5-run.md +23 -0
- package/skills/ai-research/index/6-write.md +35 -0
- package/skills/ai-research/library.json +3456 -0
- package/skills/analyze-app/EVALS.md +82 -0
- package/skills/analyze-app/SKILL.md +188 -0
- package/skills/design-contract/EVALS.md +80 -0
- package/skills/design-contract/SKILL.md +171 -0
- package/skills/design-library/EVALS.md +85 -0
- package/skills/design-library/INDEX.md +59 -0
- package/skills/design-library/SKILL.md +58 -0
- package/skills/design-library/SOURCE +131 -0
- package/skills/design-library/adaptive-interfaces/skills/colour-independence/SKILL.md +90 -0
- package/skills/design-library/adaptive-interfaces/skills/flexible-typography/SKILL.md +83 -0
- package/skills/design-library/adaptive-interfaces/skills/information-density/SKILL.md +87 -0
- package/skills/design-library/adaptive-interfaces/skills/preference-audit/SKILL.md +70 -0
- package/skills/design-library/adaptive-interfaces/skills/responsive-accessibility/SKILL.md +86 -0
- package/skills/design-library/adaptive-interfaces/skills/responsive-review/SKILL.md +76 -0
- package/skills/design-library/adaptive-interfaces/skills/simplified-views/SKILL.md +97 -0
- package/skills/design-library/adaptive-interfaces/skills/specify/SKILL.md +63 -0
- package/skills/design-library/adaptive-interfaces/skills/user-preference-respect/SKILL.md +100 -0
- package/skills/design-library/cognitive-accessibility/skills/adaptive-personalisation/SKILL.md +61 -0
- package/skills/design-library/cognitive-accessibility/skills/assess-load/SKILL.md +48 -0
- package/skills/design-library/cognitive-accessibility/skills/cognitive-load-assessment/SKILL.md +60 -0
- package/skills/design-library/cognitive-accessibility/skills/contextual-help-design/SKILL.md +61 -0
- package/skills/design-library/cognitive-accessibility/skills/error-prevention-recovery/SKILL.md +71 -0
- package/skills/design-library/cognitive-accessibility/skills/focus-attention-design/SKILL.md +67 -0
- package/skills/design-library/cognitive-accessibility/skills/memory-load-reduction/SKILL.md +60 -0
- package/skills/design-library/cognitive-accessibility/skills/plain-language-design/SKILL.md +78 -0
- package/skills/design-library/cognitive-accessibility/skills/review/SKILL.md +56 -0
- package/skills/design-library/cognitive-accessibility/skills/simplify/SKILL.md +39 -0
- package/skills/design-library/cognitive-accessibility/skills/wayfinding-navigation/SKILL.md +64 -0
- package/skills/design-library/design-research/skills/behavioural-analytics/SKILL.md +33 -0
- package/skills/design-library/design-research/skills/qual-quant-triangulation/SKILL.md +33 -0
- package/skills/design-library/inclusive-interaction/skills/audit/SKILL.md +50 -0
- package/skills/design-library/inclusive-interaction/skills/design-flow/SKILL.md +58 -0
- package/skills/design-library/inclusive-interaction/skills/feedback-and-status/SKILL.md +76 -0
- package/skills/design-library/inclusive-interaction/skills/gesture-alternatives/SKILL.md +70 -0
- package/skills/design-library/inclusive-interaction/skills/keyboard-navigation/SKILL.md +99 -0
- package/skills/design-library/inclusive-interaction/skills/keyboard-review/SKILL.md +54 -0
- package/skills/design-library/inclusive-interaction/skills/motion-sensitivity/SKILL.md +79 -0
- package/skills/design-library/inclusive-interaction/skills/multi-modal-input/SKILL.md +70 -0
- package/skills/design-library/inclusive-interaction/skills/touch-target-design/SKILL.md +71 -0
- package/skills/design-library/inclusive-interaction/skills/voice-interaction/SKILL.md +68 -0
- package/skills/design-library/visual-critique/skills/critique-affordance/SKILL.md +45 -0
- package/skills/design-library/visual-critique/skills/critique-brand-consistency/SKILL.md +42 -0
- package/skills/design-library/visual-critique/skills/critique-color/SKILL.md +44 -0
- package/skills/design-library/visual-critique/skills/critique-composition/SKILL.md +46 -0
- package/skills/design-library/visual-critique/skills/critique-information-density/SKILL.md +45 -0
- package/skills/design-library/visual-critique/skills/critique-typography/SKILL.md +45 -0
- package/skills/design-library/visual-critique/skills/critique-visual-hierarchy/SKILL.md +40 -0
- package/skills/feature-audit/EVALS.md +83 -0
- package/skills/feature-audit/SKILL.md +326 -0
- package/skills/harness/EVALS.md +81 -0
- package/skills/harness/SKILL.md +88 -0
- package/skills/i-have-adhd/EVALS.md +80 -0
- package/skills/i-have-adhd/SKILL.md +183 -0
- package/skills/no-ai-slop/EVALS.md +89 -0
- package/skills/no-ai-slop/SKILL.md +209 -0
- package/skills/no-ai-slop/eval.md +52 -0
- package/skills/plan-add/EVALS.md +79 -0
- package/skills/plan-add/SKILL.md +179 -0
- package/skills/plan-add/render_table.py +72 -0
- package/skills/plan-status/EVALS.md +78 -0
- package/skills/plan-status/SKILL.md +52 -0
- package/skills/plan-sync/EVALS.md +80 -0
- package/skills/plan-sync/SKILL.md +69 -0
- package/skills/pm-frameworks/SKILL.md +33 -0
- package/skills/pm-frameworks/SOURCE +39 -0
- package/skills/pm-frameworks/intended-vs-implemented/SKILL.md +42 -0
- package/skills/pm-frameworks/opportunity-solution-tree/SKILL.md +64 -0
- package/skills/ponytail/EVALS.md +80 -0
- package/skills/ponytail/SKILL.md +124 -0
- package/skills/product-analysis/EVALS.md +103 -0
- package/skills/product-analysis/SKILL.md +361 -0
- package/skills/research/EVALS.md +104 -0
- package/skills/research/SKILL.md +570 -0
- package/skills/rl-env/SKILL.md +53 -0
- package/skills/rl-env/SOURCE +41 -0
- package/skills/rl-env/generate-nemo-gym-env/SKILL.md +169 -0
- package/skills/rl-env/generate-nemo-gym-env/references/architecture.md +114 -0
- package/skills/rl-env/generate-openenv-env/SKILL.md +183 -0
- package/skills/rl-env/generate-openenv-env/references/architecture.md +82 -0
- package/skills/rl-env/generate-ors-env/SKILL.md +209 -0
- package/skills/rl-env/generate-ors-env/references/architecture.md +113 -0
- package/skills/rl-env/generate-verifiers-env/SKILL.md +188 -0
- package/skills/rl-env/generate-verifiers-env/references/architecture.md +119 -0
- package/skills/rl-env/rl-env-from-description/SKILL.md +124 -0
- package/skills/rl-env/rl-env-from-description/references/interview.md +75 -0
- package/skills/rl-env/rl-env-from-description/references/nemo_gym.md +41 -0
- package/skills/rl-env/rl-env-from-description/references/openenv.md +35 -0
- package/skills/rl-env/rl-env-from-description/references/ors.md +50 -0
- package/skills/rl-env/rl-env-from-description/references/verifiers.md +39 -0
- package/skills/tezgah-contract/EVALS.md +78 -0
- package/skills/tezgah-contract/SKILL.md +792 -0
- package/statusline.py +129 -0
- package/workflows/graph-impact.js +153 -0
- package/workflows/graph-map.js +158 -0
- package/workflows/graph-review.js +170 -0
|
@@ -0,0 +1,82 @@
|
|
|
1
|
+
# Evals
|
|
2
|
+
|
|
3
|
+
Three inputs a session can be handed where this skill could misfire, and what a
|
|
4
|
+
good answer has to contain. Each case is built so a plausible but shallow answer
|
|
5
|
+
fails at least one check.
|
|
6
|
+
|
|
7
|
+
Run a case by giving the session only the **Input**, with the skill reachable
|
|
8
|
+
from the router. A case passes when every Must-contain line is met and no
|
|
9
|
+
Must-not line is triggered.
|
|
10
|
+
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
## Case 1 - The trap: a static snapshot read as "the state is missing"
|
|
14
|
+
|
|
15
|
+
**Input**
|
|
16
|
+
> Check the Save button on `/settings` for missing states. I only need a quick
|
|
17
|
+
> look at the snapshot.
|
|
18
|
+
|
|
19
|
+
**Must contain**
|
|
20
|
+
- [ ] Reads the tree with `browser_snapshot` and acts on element references
|
|
21
|
+
- [ ] Drives the interactive states (hover, focus, active, disabled) before calling any of them missing
|
|
22
|
+
- [ ] Routes the data-side states it checks (loading, error, offline) with the network tools before judging them
|
|
23
|
+
- [ ] Says which states were driven and re-read, and which only the static tree showed
|
|
24
|
+
|
|
25
|
+
**Must not**
|
|
26
|
+
- Report hover or focus as missing because the static snapshot does not show them
|
|
27
|
+
- Judge the states from a screenshot alone, without the tree
|
|
28
|
+
|
|
29
|
+
**Why this case** - the triage section says "a static snapshot never shows hover",
|
|
30
|
+
and that a state reported as not shown is "driven ... and routed ... and then
|
|
31
|
+
re-read, before it is called missing". "Quick look at the snapshot" invites the
|
|
32
|
+
shortcut.
|
|
33
|
+
|
|
34
|
+
---
|
|
35
|
+
|
|
36
|
+
## Case 2 - The trap: a mobile contrast claim from the view tree
|
|
37
|
+
|
|
38
|
+
**Input**
|
|
39
|
+
> On the iOS simulator build, is the grey caption text on the onboarding screen
|
|
40
|
+
> readable? Check the contrast and the tap targets.
|
|
41
|
+
|
|
42
|
+
**Must contain**
|
|
43
|
+
- [ ] States that contrast and tap-target size are not measurable from the Mobile MCP view tree
|
|
44
|
+
- [ ] Takes a screenshot and names why (colour and size the tree cannot carry), or declines to make the claim
|
|
45
|
+
- [ ] Labels any readability verdict as a judgement and says how many passes agreed
|
|
46
|
+
|
|
47
|
+
**Must not**
|
|
48
|
+
- Give a contrast ratio or a pass/fail read off `mobile_list_elements_on_screen`
|
|
49
|
+
- Call a `mobile_verify_*` tool or claim an assertion that Mobile MCP does not offer
|
|
50
|
+
|
|
51
|
+
**Why this case** - "Mobile caveats" says contrast and tap-target size "are
|
|
52
|
+
**not measurable** from the tree: take the screenshot (naming why), or do not
|
|
53
|
+
make the claim", and that there is no `mobile_verify_*`.
|
|
54
|
+
|
|
55
|
+
---
|
|
56
|
+
|
|
57
|
+
## Case 3 - The trap: a clean axe run read as WCAG conformance
|
|
58
|
+
|
|
59
|
+
**Input**
|
|
60
|
+
> Run axe on the checkout page. If it comes back clean we can say we are WCAG
|
|
61
|
+
> 2.2 AA compliant, right?
|
|
62
|
+
|
|
63
|
+
**Must contain**
|
|
64
|
+
- [ ] Injects axe-core from a pinned build with `browser_evaluate`
|
|
65
|
+
- [ ] States that axe covers on average 57% of WCAG issues and reports its `incomplete` results
|
|
66
|
+
- [ ] Names the gaps it hand-checks or leaves open: focus appearance, target size, dragging, accessible authentication
|
|
67
|
+
- [ ] Names the viewport widths it judged
|
|
68
|
+
|
|
69
|
+
**Must not**
|
|
70
|
+
- Load axe from a `@latest` CDN script
|
|
71
|
+
- Declare WCAG 2.2 AA conformance from a clean axe run alone
|
|
72
|
+
|
|
73
|
+
**Why this case** - "The automated sweeps" says axe finds "on average 57% of WCAG
|
|
74
|
+
issues automatically" and that the named gaps "are hand-checked, not assumed";
|
|
75
|
+
it also forbids a `@latest` script.
|
|
76
|
+
|
|
77
|
+
---
|
|
78
|
+
|
|
79
|
+
## What these do not prove
|
|
80
|
+
|
|
81
|
+
These are trap cases for a human or judge rater, not a behavioural measurement.
|
|
82
|
+
`skills/research` owns the standard a claim about behaviour has to meet.
|
|
@@ -0,0 +1,188 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: analyze-app
|
|
3
|
+
description: >
|
|
4
|
+
Analyze a running web or mobile app from its structured accessibility / DOM /
|
|
5
|
+
native view tree, and from screenshots of the rendered screen. Use when a task
|
|
6
|
+
must drive,
|
|
7
|
+
inspect, or verify a running application - walk a flow, read the UI, check the
|
|
8
|
+
console, network or device logs, or confirm behavior end to end - rather than
|
|
9
|
+
only read its source. Covers a real browser (Playwright MCP), an iOS Simulator
|
|
10
|
+
or Android emulator (Mobile MCP), and optional web diagnostics (Chrome
|
|
11
|
+
DevTools MCP). Not for judging the whole product - use `product-analysis`.
|
|
12
|
+
---
|
|
13
|
+
|
|
14
|
+
# analyze-app
|
|
15
|
+
|
|
16
|
+
Drive the running app through its accessibility tree. The tree answers structure;
|
|
17
|
+
the rendered image answers hierarchy, rhythm, colour and feel - so a screenshot is
|
|
18
|
+
taken whenever a claim needs those, and for what the tree cannot represent at all
|
|
19
|
+
(canvas, game, animation, pixel-level visual regression). It is read, not merely
|
|
20
|
+
saved, and it is still not a per-step default: capture what the tree could not
|
|
21
|
+
answer, and say what that was.
|
|
22
|
+
|
|
23
|
+
## Pick the target
|
|
24
|
+
|
|
25
|
+
| Target | Server | Tool prefix |
|
|
26
|
+
|---|---|---|
|
|
27
|
+
| Web page / webview | Playwright MCP | `browser_*` |
|
|
28
|
+
| iOS Simulator / Android emulator | Mobile MCP | `mobile_*` |
|
|
29
|
+
| Web perf / deep network / source-mapped console | Chrome DevTools MCP (opt-in) | `*` |
|
|
30
|
+
|
|
31
|
+
If the `browser_*` / `mobile_*` tools are not in this session, the server is not
|
|
32
|
+
wired for this host, or it failed to start: say so and fall back to the CLI on
|
|
33
|
+
the shell, with the same pins tezgah wires (`hooks/tezgah_apps.py`) -
|
|
34
|
+
`npx -y @playwright/mcp@0.0.81 --isolated --caps=testing,storage,network` or
|
|
35
|
+
`npx -y @mobilenext/mobile-mcp@1.0.4`. Never `@latest`: a floating tag fetches
|
|
36
|
+
whatever the registry serves at session start and runs it with the agent's
|
|
37
|
+
privileges.
|
|
38
|
+
|
|
39
|
+
**omp has no `browser_*` tools, by design**: while its native `browser` (the
|
|
40
|
+
eval prelude: `browser.open`, `tab.observe`, `tab.click`, `tab.console`,
|
|
41
|
+
`tab.requests`) is on, omp drops browser MCP servers, so tezgah does not wire
|
|
42
|
+
Playwright there. Use the native browser for the web loop below - its
|
|
43
|
+
`observe`/`ariaSnapshot` is the tree, its screenshot is the explicit capture -
|
|
44
|
+
and Mobile MCP for devices as everywhere else.
|
|
45
|
+
|
|
46
|
+
## What each capability buys an audit
|
|
47
|
+
|
|
48
|
+
Playwright MCP serves core tools only unless the capabilities are named, and the
|
|
49
|
+
three tezgah wires are the difference between "the tree looked fine" and a claim
|
|
50
|
+
someone can re-run:
|
|
51
|
+
|
|
52
|
+
| Capability | Tools | What an analysis does with it |
|
|
53
|
+
|---|---|---|
|
|
54
|
+
| core (always on) | `browser_snapshot`, `browser_find`, interaction, `browser_evaluate`, `browser_console_messages`, `browser_network_requests`, `browser_resize` | read the flow and its element names, roles and states; run a measurement in the page; collect the errors a step produced |
|
|
55
|
+
| `testing` | `browser_verify_text_visible`, `browser_verify_element_visible`, `browser_verify_list_visible`, `browser_verify_value`, `browser_generate_locator` | turn a UX claim into an assertion that fails or passes on a re-run - the difference between a `failure` and a `judgement` in a product analysis |
|
|
56
|
+
| `storage` | `browser_storage_state`, `browser_set_storage_state`, cookies, localStorage | audit a logged-in surface from a saved state file instead of attaching to the user's real Chrome profile |
|
|
57
|
+
| `network` | `browser_route`, `browser_route_list`, `browser_network_state_set` | set the offline state and mock a failing endpoint - the two states a mobile-first product is judged on |
|
|
58
|
+
| `devtools` (off) | tracing, video, highlight | owned by the opt-in chrome-devtools sidecar; enabling it here would duplicate that and widen the tool list |
|
|
59
|
+
|
|
60
|
+
`browser_resize` matters more than it looks: a product audit must name the
|
|
61
|
+
viewport it judged, and a layout finding that holds at one width and not another
|
|
62
|
+
is not a finding until both are checked.
|
|
63
|
+
|
|
64
|
+
## The loop (tree first)
|
|
65
|
+
|
|
66
|
+
1. **Open** the target: `browser_navigate` / `mobile_launch_app`. On mobile,
|
|
67
|
+
boot the simulator/emulator first and confirm with
|
|
68
|
+
`mobile_list_available_devices`.
|
|
69
|
+
2. **Read the tree, do not screenshot**: `browser_snapshot` (web) or
|
|
70
|
+
`mobile_list_elements_on_screen` (mobile). Act on the returned element
|
|
71
|
+
references, not on coordinates.
|
|
72
|
+
3. **Act**: `browser_click`, `browser_fill_form`, `browser_press_key`,
|
|
73
|
+
`browser_evaluate`; `mobile_click_on_screen_at_coordinates`,
|
|
74
|
+
`mobile_type_keys`, `mobile_swipe_on_screen`, `mobile_press_button`.
|
|
75
|
+
4. **Observe after every change**: `browser_console_messages`,
|
|
76
|
+
`browser_network_requests`, `mobile_get_device_logs`, `mobile_list_crashes`.
|
|
77
|
+
Re-read the tree instead of taking a screenshot.
|
|
78
|
+
5. **Screenshot only when the tree is insufficient**, and name the reason:
|
|
79
|
+
`browser_take_screenshot` or `mobile_save_screenshot`. Reference the saved
|
|
80
|
+
path; never paste image bytes into the conversation.
|
|
81
|
+
|
|
82
|
+
## Triage: read the lines that matter, not the screen
|
|
83
|
+
|
|
84
|
+
A snapshot is the whole screen as text - one 40-row admin table came back as
|
|
85
|
+
13,045 characters in 356 lines - and the loop pays for all of it to find the dozen
|
|
86
|
+
lines that carry the controls. So after the snapshot, and before reading it:
|
|
87
|
+
|
|
88
|
+
1. **Flatten and keep it**: save the `browser_snapshot` result (or the flattened
|
|
89
|
+
text of one) to a file.
|
|
90
|
+
2. **Triage**: `tezgah-triage --select FILE --task "what this pass is looking
|
|
91
|
+
for"` sends the tree's repeating units - a row, a table cell, a control on its
|
|
92
|
+
own - and the task in one batched judgement, and prints the line ids under the
|
|
93
|
+
selected units with their original `ref=` values, the unselected count and the
|
|
94
|
+
characters that may now be skipped.
|
|
95
|
+
3. **Read the selected refs** in the tree, and act on them. Read the whole snapshot
|
|
96
|
+
when the task is "is anything wrong on this screen at all" - a selection cannot
|
|
97
|
+
answer a question it was not asked.
|
|
98
|
+
|
|
99
|
+
For one component, `tezgah-triage --states FILE --component "#12"` asks one
|
|
100
|
+
judgement per state over that component's subtree - the interactive pair `focus`
|
|
101
|
+
and `active` arrives as one three-way answer (`neither` / `focus` / `pressed`)
|
|
102
|
+
inside that same call, measured better than two separate questions - covering
|
|
103
|
+
default, hover, focus, active, disabled, loading, error; empty, skeleton, offline,
|
|
104
|
+
partial, long-text, permission-denied - and a state the judgement reports as not
|
|
105
|
+
shown is driven for
|
|
106
|
+
the interactive set (hover, focus, active, disabled) and routed for the data set
|
|
107
|
+
(offline, error, empty, loading) and then re-read, before it is called missing: a
|
|
108
|
+
static snapshot never shows hover, and it shows only the data state the route
|
|
109
|
+
happens to be in, so `--states` answers what the current tree shows, not what the
|
|
110
|
+
component implements.
|
|
111
|
+
|
|
112
|
+
**The triage is an aid, and its recall is measured, not assumed.** On two
|
|
113
|
+
reproduced screens this round the unit selection kept **100% of the control
|
|
114
|
+
lines** - at a 94% read on the 40-row table, 74% on the 20-button screen - while
|
|
115
|
+
one question per line kept only 73.5% at a 26% read, which is what "selective"
|
|
116
|
+
looked like when it was actually losing a quarter of the controls. 100% is a
|
|
117
|
+
measurement on those screens, not a guarantee: read what the selection names, then
|
|
118
|
+
judge from what the tree and the screen actually show, and say what you read. When
|
|
119
|
+
the printed selection is most of the screen, the task names a control every row
|
|
120
|
+
repeats and the triage bought coverage rather than skipping - the tool says so
|
|
121
|
+
itself, and reading the snapshot directly is then the cheaper move. With no
|
|
122
|
+
credential, or with the `judge-off` switch on, `tezgah-triage` prints why and
|
|
123
|
+
exits 1: read the snapshot directly, the way the loop reads it today, and say the
|
|
124
|
+
triage was unavailable.
|
|
125
|
+
|
|
126
|
+
## Browser profile
|
|
127
|
+
|
|
128
|
+
Isolated by default, so a run never touches the user's real Chrome state. To
|
|
129
|
+
analyze a logged-in flow, attach deliberately - `--cdp-endpoint` or the
|
|
130
|
+
Playwright extension instead of the isolated default - and say so before doing
|
|
131
|
+
it, because it exposes the real session to the analysis.
|
|
132
|
+
|
|
133
|
+
## Artifacts
|
|
134
|
+
|
|
135
|
+
Screenshots, traces and tree dumps go to the app artifact directory
|
|
136
|
+
(`PLAYWRIGHT_MCP_OUTPUT_DIR`, default `~/.cache/tezgah/apps`). Return the file
|
|
137
|
+
path; keep image tokens out of the context.
|
|
138
|
+
|
|
139
|
+
## Looking at the screen
|
|
140
|
+
|
|
141
|
+
The tree answers structure; the image answers hierarchy, rhythm, colour and feel.
|
|
142
|
+
Neither replaces the other, and a review that saves a screenshot without reading it
|
|
143
|
+
has produced a file, not evidence.
|
|
144
|
+
|
|
145
|
+
- Say what the tree could not answer before capturing. `browser_take_screenshot` /
|
|
146
|
+
`mobile_save_screenshot` return a path; the bytes stay out of the context.
|
|
147
|
+
- Sweep the widths in scope with `browser_resize` - at least 320 / 768 / 1280 - then
|
|
148
|
+
capture. Repeat in dark mode and at the largest text setting: hierarchy and
|
|
149
|
+
contrast move with all three.
|
|
150
|
+
- Two checks that need the image and take seconds: the **blur test** (blur or squint
|
|
151
|
+
- is the primary action still first?) and the **grayscale test** (does the
|
|
152
|
+
hierarchy survive without colour?). Both are read from the same capture.
|
|
153
|
+
- Report what was seen: the screen, the region, what competes with what. That is a
|
|
154
|
+
judgement, so run the pass twice and say whether both agreed.
|
|
155
|
+
|
|
156
|
+
## The automated sweeps
|
|
157
|
+
|
|
158
|
+
Two standards can be run in the page instead of argued about:
|
|
159
|
+
|
|
160
|
+
- **axe-core** - injected with `browser_evaluate` from a *pinned* build, never a
|
|
161
|
+
`@latest` CDN script, which is the supply-chain defect the pins exist to avoid. It
|
|
162
|
+
covers WCAG A/AA/AAA plus common best practices, and its own documentation puts it
|
|
163
|
+
at "on average 57% of WCAG issues automatically", returning `incomplete` where it
|
|
164
|
+
cannot be certain. The named gaps - focus appearance, target size, dragging,
|
|
165
|
+
accessible authentication - are hand-checked, not assumed.
|
|
166
|
+
- **Core Web Vitals** via `PerformanceObserver` in the page: LCP <= 2500 ms,
|
|
167
|
+
INP <= 200 ms and CLS <= 0.1 are "good" at the 75th percentile; > 4000 ms,
|
|
168
|
+
> 500 ms and > 0.25 are "poor" (web.dev). A number outside them is a failure with
|
|
169
|
+
its trace; the opt-in chrome-devtools sidecar is the path to that trace.
|
|
170
|
+
|
|
171
|
+
## Mobile caveats
|
|
172
|
+
|
|
173
|
+
Mobile MCP has no assertion, mocking or geometry tools: there is no
|
|
174
|
+
`mobile_verify_*`, no way to force an offline state, and the view tree carries no
|
|
175
|
+
colour or size maths. So on mobile -
|
|
176
|
+
|
|
177
|
+
- a UX claim is a **judgement** unless re-walking the flow reproduces it; say how
|
|
178
|
+
many passes agreed,
|
|
179
|
+
- contrast, tap-target size and focus order are **not measurable** from the tree:
|
|
180
|
+
take the screenshot (naming why), or do not make the claim,
|
|
181
|
+
- the two states mobile products are judged on - offline and a failing backend -
|
|
182
|
+
come from the device or the app's own test hook, not from this server; if
|
|
183
|
+
neither exists, that is the finding.
|
|
184
|
+
|
|
185
|
+
macOS may prompt for Accessibility / Screen-Recording permission on first use,
|
|
186
|
+
and a real iOS device needs Developer Mode. The view tree can drop under load:
|
|
187
|
+
retry the tree once, then fall back to a screenshot. iOS/Android tooling version
|
|
188
|
+
skew is the usual failure - report the exact error instead of guessing.
|
|
@@ -0,0 +1,80 @@
|
|
|
1
|
+
# Evals
|
|
2
|
+
|
|
3
|
+
Three inputs a session can be handed where this skill could misfire, and what a
|
|
4
|
+
good answer has to contain. Each case is built so a plausible but shallow answer
|
|
5
|
+
fails at least one check.
|
|
6
|
+
|
|
7
|
+
Run a case by giving the session only the **Input**, with the skill reachable
|
|
8
|
+
from the router. A case passes when every Must-contain line is met and no
|
|
9
|
+
Must-not line is triggered.
|
|
10
|
+
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
## Case 1 - The trap: inventing a palette for a repo with no tokens
|
|
14
|
+
|
|
15
|
+
**Input**
|
|
16
|
+
> This app has no design system at all - just inline styles. Set up a design
|
|
17
|
+
> contract so `tezgah-design check` has something to judge against.
|
|
18
|
+
|
|
19
|
+
**Must contain**
|
|
20
|
+
- [ ] Runs `tezgah-design derive` and reports that it wrote nothing and exited 2 when no token source exists
|
|
21
|
+
- [ ] Writes the floor by hand with `source: derived` and a line naming where it came from (platform guide, brand asset, or a colour read off the running screen)
|
|
22
|
+
- [ ] Keeps the WCAG floors: `tap_target` 24 (SC 2.5.8) and contrast 4.5:1 / 3:1 (SC 1.4.3)
|
|
23
|
+
|
|
24
|
+
**Must not**
|
|
25
|
+
- Make up a palette, type scale or spacing unit with no stated origin
|
|
26
|
+
- Leave `source` empty
|
|
27
|
+
|
|
28
|
+
**Why this case** - Derivation says `derive` "does not invent a palette" and that
|
|
29
|
+
"an invented palette is not a floor at all". A blank repo pulls toward filling the
|
|
30
|
+
gap with plausible tokens.
|
|
31
|
+
|
|
32
|
+
---
|
|
33
|
+
|
|
34
|
+
## Case 2 - The trap: `0 violations` over a component inventory nobody filled
|
|
35
|
+
|
|
36
|
+
**Input**
|
|
37
|
+
> I ran `tezgah-design derive` and then `check` on our measurement. It printed
|
|
38
|
+
> `0 violations`. The new `PriceTag` component is in the measurement. Ship it?
|
|
39
|
+
|
|
40
|
+
**Must contain**
|
|
41
|
+
- [ ] States that a derived contract carries an empty `components` list, so `component-inventory` is inert
|
|
42
|
+
- [ ] Asks for `PriceTag` (and the other owned components) to be added as `{name, kind}` in this turn
|
|
43
|
+
- [ ] Checks how many components the run judged, and that `PriceTag` has a `kind` so its state coverage is judged
|
|
44
|
+
|
|
45
|
+
**Must not**
|
|
46
|
+
- Read `0 violations` as a clean result without checking what was judged
|
|
47
|
+
- Treat the check as a substitute for reading the running app
|
|
48
|
+
|
|
49
|
+
**Why this case** - the skill says "the `component-inventory` rule is inert until
|
|
50
|
+
someone adds one `{name, kind}` per component", that a component without `kind`
|
|
51
|
+
"is unjudged, never passed", and that the checker "is not a substitute for the
|
|
52
|
+
screen read".
|
|
53
|
+
|
|
54
|
+
---
|
|
55
|
+
|
|
56
|
+
## Case 3 - The trap: dropping states to make the check pass
|
|
57
|
+
|
|
58
|
+
**Input**
|
|
59
|
+
> `check` flags `state-coverage` on our DataTable: missing skeleton, offline,
|
|
60
|
+
> partial, long-text, permission-denied. We never show those. Can I trim
|
|
61
|
+
> `states.data` in the contract to what we have?
|
|
62
|
+
|
|
63
|
+
**Must contain**
|
|
64
|
+
- [ ] Says the contract must carry the state set exactly, and trimming it raises a `state-set` violation
|
|
65
|
+
- [ ] Treats each missing state as a finding about the DataTable, with empty, loading and error ranked as the bigger ones when missing
|
|
66
|
+
- [ ] Gives the next action: build or measure the missing states
|
|
67
|
+
|
|
68
|
+
**Must not**
|
|
69
|
+
- Edit `states.data` to remove the missing states
|
|
70
|
+
- Mark the missing states as not applicable without evidence from the running app
|
|
71
|
+
|
|
72
|
+
**Why this case** - "The state set" says "A state that does not exist is a
|
|
73
|
+
finding", and `states.data` is a field "The contract must carry ... exactly".
|
|
74
|
+
|
|
75
|
+
---
|
|
76
|
+
|
|
77
|
+
## What these do not prove
|
|
78
|
+
|
|
79
|
+
These are trap cases for a human or judge rater, not a behavioural measurement.
|
|
80
|
+
`skills/research` owns the standard a claim about behaviour has to meet.
|
|
@@ -0,0 +1,171 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: design-contract
|
|
3
|
+
description: >
|
|
4
|
+
The artifact shape of a repository's design contract - the colour roles, the
|
|
5
|
+
type scale, the spacing unit and its rhythm, the component inventory, and the
|
|
6
|
+
state set every interactive control and every data view owes - and the rule
|
|
7
|
+
that derives it from the repository's own tokens.
|
|
8
|
+
Use when a UI or component turn needs a design floor, tokens, a state set or
|
|
9
|
+
a named `tezgah-design` check. Not for reading or driving the running app -
|
|
10
|
+
use `analyze-app`.
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
# design-contract
|
|
14
|
+
|
|
15
|
+
This skill owns ONE thing: the shape of `.tezgah/design-contract.md`, the
|
|
16
|
+
per-repo floor a UI turn is judged against, and the derivation rule that fills it
|
|
17
|
+
when the repository never had a design system. The reading of the running app is
|
|
18
|
+
`analyze-app`'s, the measurement vocabulary is `product-analysis`'s, and the
|
|
19
|
+
always-on rules are `tezgah-contract`'s.
|
|
20
|
+
|
|
21
|
+
The contract is not injected into a session - `~/.config/tezgah/bin/tezgah-design check` reads the
|
|
22
|
+
file, so it costs nothing until it is used.
|
|
23
|
+
|
|
24
|
+
## The artifact
|
|
25
|
+
|
|
26
|
+
`.tezgah/design-contract.md`, in the private `.tezgah` repository beside
|
|
27
|
+
`.tezgah/lessons.md`. A human header, then ONE fenced `json` block: the block is
|
|
28
|
+
the machine half and the only part `check` reads, and the header is what a person
|
|
29
|
+
opens. Field names and prose stay English.
|
|
30
|
+
|
|
31
|
+
- `source` - where the floor came from: `css-custom-properties`, `tailwind-config`
|
|
32
|
+
or `theme-json` when it was read out of the repository, `derived` when it could
|
|
33
|
+
not be and someone chose it. An empty `source` is refused: a floor nobody can
|
|
34
|
+
trace is not a floor.
|
|
35
|
+
- `tokens.colors` - the colour roles, as role name to hex. A measured colour that
|
|
36
|
+
is not one of them is a `palette` violation.
|
|
37
|
+
- `tokens.type_scale` - the type steps in px. A measured `font-size` off the scale
|
|
38
|
+
is a `type-scale` violation.
|
|
39
|
+
- `tokens.space_unit` - the spacing unit in px, and the rhythm it implies: every
|
|
40
|
+
padding, margin and gap is a multiple of it, or it is a `spacing-rhythm`
|
|
41
|
+
violation. `null` when the repository states none, which leaves spacing values
|
|
42
|
+
unjudged rather than guessed at.
|
|
43
|
+
- `tokens.tap_target` and `tokens.contrast` - the WCAG 2.2 floors (SC 2.5.8:
|
|
44
|
+
24x24 CSS px; SC 1.4.3: 4.5:1, 3:1 above 24px). These are the standard's, not
|
|
45
|
+
the repository's; they are never derived away.
|
|
46
|
+
- `tokens.reduced_motion` - optional, `true` once motion has a declared way out.
|
|
47
|
+
A measurement that carries a non-zero `transition-duration` or
|
|
48
|
+
`animation-duration` while the contract declares no guard is a `reduced-motion`
|
|
49
|
+
violation (SC 2.3.3 Animation from Interactions, Level AAA), and the reading
|
|
50
|
+
takes the longest entry of a duration list, which is what a person sits
|
|
51
|
+
through.
|
|
52
|
+
- A measurement that says the focus outline is off (`outline: none`,
|
|
53
|
+
`outline-width: 0`, or an `outline` shorthand whose width is zero) with
|
|
54
|
+
nothing beside it that draws - no `box-shadow` that paints, and no border
|
|
55
|
+
whose every declaration is alive (a live `border-style` beside
|
|
56
|
+
`border-width: 0px` draws nothing, which is a border-less element's
|
|
57
|
+
computed shape on a Tailwind page) - is a `focus-visible` violation (SC 2.4.7
|
|
58
|
+
Focus Visible, Level AA). An `outline-color`, a bare `border-color`,
|
|
59
|
+
`border-style: hidden` and a zero width are not indications: a resolved
|
|
60
|
+
computed-style dump carries all of them where nothing is drawn. Both rules
|
|
61
|
+
read what the measurement declares and conclude from what they can: a
|
|
62
|
+
component whose styles never mention focus or motion, and a declaration no
|
|
63
|
+
reader can classify (`var(--dur)`, an `outline-color` with no style or width
|
|
64
|
+
beside it), are counted unjudged rather than passed - and if nothing in the
|
|
65
|
+
measurement is judged at all, the check exits 2 rather than report
|
|
66
|
+
`0 violations` over a screen nobody looked at.
|
|
67
|
+
- `states.interactive` and `states.data` - the state sets below. The contract
|
|
68
|
+
must carry them exactly; a floor that drops a state is a `state-set` violation.
|
|
69
|
+
- `components` - the inventory: one `{name, kind}` per component the repository
|
|
70
|
+
owns. A component the measurement shows that is not listed is a
|
|
71
|
+
`component-inventory` violation, so a new component is added here, in the same
|
|
72
|
+
turn it is written.
|
|
73
|
+
|
|
74
|
+
```json
|
|
75
|
+
{
|
|
76
|
+
"version": 1,
|
|
77
|
+
"source": "css-custom-properties",
|
|
78
|
+
"tokens": {
|
|
79
|
+
"space_unit": 4,
|
|
80
|
+
"type_scale": [14, 16, 20],
|
|
81
|
+
"colors": {"text": "#111111", "surface": "#ffffff", "accent": "#0b5fff"},
|
|
82
|
+
"tap_target": 24,
|
|
83
|
+
"contrast": {"normal": 4.5, "large": 3.0}
|
|
84
|
+
},
|
|
85
|
+
"states": {
|
|
86
|
+
"interactive": ["default", "hover", "focus", "active", "disabled",
|
|
87
|
+
"loading", "error"],
|
|
88
|
+
"data": ["empty", "loading", "skeleton", "error", "offline", "partial",
|
|
89
|
+
"long-text", "permission-denied"]
|
|
90
|
+
},
|
|
91
|
+
"components": [{"name": "Button", "kind": "interactive"}]
|
|
92
|
+
}
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
## The state set
|
|
96
|
+
|
|
97
|
+
Every interactive control owes all of `states.interactive`; every data view owes
|
|
98
|
+
all of `states.data`. A state that does not exist is a finding - the empty,
|
|
99
|
+
loading and error states usually a bigger one than a state that merely looks
|
|
100
|
+
wrong - and a component whose measurement does not show a state it owes gets a
|
|
101
|
+
`state-coverage` violation naming the states it is missing. `kind` decides which
|
|
102
|
+
half applies, so a component measured without one is unjudged, never passed.
|
|
103
|
+
|
|
104
|
+
## Derivation
|
|
105
|
+
|
|
106
|
+
Read the repository's own token source. In this order, and nothing else:
|
|
107
|
+
|
|
108
|
+
1. **CSS custom properties** - `--name: value` declarations in `*.css`, `*.scss`,
|
|
109
|
+
`*.sass`, `*.less`. A property whose value is a colour literal is a colour
|
|
110
|
+
role, named by the property (the `color-`/`c-` prefix and a trailing `-color`
|
|
111
|
+
are dropped); one named `space`, `spacing`, `unit`, `rhythm` and the like
|
|
112
|
+
states the unit; `font-size`/`text-size`/`fs` properties are the type scale.
|
|
113
|
+
2. **A Tailwind config** - `tailwind.config.{js,cjs,mjs,ts,json}`, its `colors`,
|
|
114
|
+
`spacing` and `fontSize` blocks. `unit`/`base` in `spacing` states the unit;
|
|
115
|
+
otherwise the greatest common divisor of the scale the config declares is it.
|
|
116
|
+
3. **A tokens/theme JSON** - `theme.json`, `tokens.json` or `design-tokens.json`,
|
|
117
|
+
its `colors`/`palette`, `spacing` and `fontSize` keys, found at any depth.
|
|
118
|
+
|
|
119
|
+
```sh
|
|
120
|
+
~/.config/tezgah/bin/tezgah-design derive --repo . # writes .tezgah/design-contract.md
|
|
121
|
+
~/.config/tezgah/bin/tezgah-design derive --repo . --json # the same, machine-readable
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
`derive` writes nothing and exits 2 when the repository has none of the three, and it never
|
|
125
|
+
overwrites a contract that already exists - that file carries the component inventory someone
|
|
126
|
+
filled in, so a re-derive over changed tokens is a hand edit or a deliberate removal first. It
|
|
127
|
+
does not invent a palette, a type scale or a unit either: a repository with no design system gets
|
|
128
|
+
its floor written by hand, with `source: derived` and a line saying where the floor came from -
|
|
129
|
+
the platform's own guide (Apple HIG / Material), an existing brand asset, or a colour read off the
|
|
130
|
+
running screen. A derived floor is a decision someone made and wrote down, which is why the file
|
|
131
|
+
has to say so; an invented palette is not a floor at all.
|
|
132
|
+
|
|
133
|
+
A derived contract carries an EMPTY `components` list - `derive` reads tokens, never components, and
|
|
134
|
+
the artifact says so in its header: the `component-inventory` rule is inert until someone adds one
|
|
135
|
+
`{name, kind}` per component. A measurement names a component; only a person knows which ones the
|
|
136
|
+
repository owns, so list them in the turn that writes a new one.
|
|
137
|
+
|
|
138
|
+
## The check
|
|
139
|
+
|
|
140
|
+
```sh
|
|
141
|
+
~/.config/tezgah/bin/tezgah-design check --contract .tezgah/design-contract.md \
|
|
142
|
+
--measured .tezgah/design/measurement.json
|
|
143
|
+
~/.config/tezgah/bin/tezgah-design check --contract ... --measured ... --json
|
|
144
|
+
```
|
|
145
|
+
|
|
146
|
+
`--measured` takes the JSON the running-app read produces - the per-component
|
|
147
|
+
computed styles and observed states - or a dash for stdin:
|
|
148
|
+
|
|
149
|
+
```json
|
|
150
|
+
{"components": [
|
|
151
|
+
{"name": "Button", "kind": "interactive",
|
|
152
|
+
"states": ["default", "hover", "focus", "active", "disabled", "loading",
|
|
153
|
+
"error"],
|
|
154
|
+
"styles": {"font-size": "16px", "padding": "8px 12px", "gap": "8px",
|
|
155
|
+
"color": "#111111", "background-color": "#ffffff",
|
|
156
|
+
"width": "120px", "height": "40px"}}]}
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
It prints `N violations` and one `component: rule - detail` line per
|
|
160
|
+
violation, names how many components it judged, and exits 0 only on `0 violations`
|
|
161
|
+
over at least one component judged (1 with at least one violation, 2 for a usage
|
|
162
|
+
error, an unreadable contract, or a measurement that carried no component it
|
|
163
|
+
could judge - `0 violations` over nothing is not a clean app). The rules are `contract-source`,
|
|
164
|
+
`contract-palette`, `state-set`, `palette`, `type-scale`, `spacing-rhythm`,
|
|
165
|
+
`tap-target`, `contrast`, `focus-visible`, `reduced-motion`, `state-coverage`
|
|
166
|
+
and `component-inventory`. A measured value the contract cannot judge - `em`,
|
|
167
|
+
`%`, a unitless non-zero - is counted on its own line and never read as a pass.
|
|
168
|
+
|
|
169
|
+
The checker judges only what the measurement reports. It is not a substitute for
|
|
170
|
+
the screen read: a turn that changed a component owes the app read AND the check,
|
|
171
|
+
which is what the Stop rule's `no ui_ok` class now asks for.
|
|
@@ -0,0 +1,85 @@
|
|
|
1
|
+
# Evals
|
|
2
|
+
|
|
3
|
+
Three inputs a turn on this library can actually be handed, and what a good
|
|
4
|
+
answer has to contain. Each case is built so a plausible-sounding but shallow
|
|
5
|
+
answer fails at least one check - the first one, deliberately, is the mistake the
|
|
6
|
+
vendored corpus itself made before it was corrected.
|
|
7
|
+
|
|
8
|
+
Run a case by giving the session only the **Input**, with the library reachable
|
|
9
|
+
from the router. Score against **Must contain**: a case passes when every
|
|
10
|
+
Must-contain line is met and no Must-not line is triggered.
|
|
11
|
+
|
|
12
|
+
---
|
|
13
|
+
|
|
14
|
+
## Case 1 - The trap: a size quoted against the wrong level
|
|
15
|
+
|
|
16
|
+
**Input**
|
|
17
|
+
> Our spec says buttons must be 44x44 because that is the WCAG AA minimum for
|
|
18
|
+
> touch. Our design system ships 32x32 buttons. Are we failing WCAG AA?
|
|
19
|
+
|
|
20
|
+
**Must contain**
|
|
21
|
+
- [ ] States that SC 2.5.8 Target Size (Minimum) is Level AA at 24x24 CSS px
|
|
22
|
+
- [ ] States that 44x44 is SC 2.5.5 Target Size (Enhanced), Level AAA - and the common platform guideline, not the AA floor
|
|
23
|
+
- [ ] Places 32x32 above the AA floor and below the platform guideline, naming which claim that makes it
|
|
24
|
+
- [ ] Gives the next action (spacing, hit area, or a documented exception), not just a verdict
|
|
25
|
+
|
|
26
|
+
**Must not**
|
|
27
|
+
- Repeat that 44x44 is the WCAG AA minimum
|
|
28
|
+
- Call 32x32 a WCAG AA failure without naming the criterion it fails
|
|
29
|
+
|
|
30
|
+
**Why this case** - upstream's touch-target body said exactly this; it is the one
|
|
31
|
+
body `SOURCE` records as adapted
|
|
32
|
+
(`inclusive-interaction/skills/touch-target-design/SKILL.md`). An answer that
|
|
33
|
+
parrots the library without the correction fails on the first Must-not line.
|
|
34
|
+
|
|
35
|
+
---
|
|
36
|
+
|
|
37
|
+
## Case 2 - The trap: a tracking artefact read as a design problem
|
|
38
|
+
|
|
39
|
+
**Input**
|
|
40
|
+
> Our signup funnel drops 60% of users between "account created" and "profile
|
|
41
|
+
> completed". Leadership wants the profile step redesigned this quarter. Where do
|
|
42
|
+
> we start?
|
|
43
|
+
|
|
44
|
+
**Must contain**
|
|
45
|
+
- [ ] Questions what the two events fire on before proposing any redesign
|
|
46
|
+
- [ ] Names at least one non-design explanation (the step is skippable, the event fires on render, the profile is reachable later)
|
|
47
|
+
- [ ] Says what would confirm or kill the tracking explanation, cheaply
|
|
48
|
+
- [ ] Gives one ranked next action rather than everything that could be wrong
|
|
49
|
+
|
|
50
|
+
**Must not**
|
|
51
|
+
- Accept the 60% at face value and go straight to redesign recommendations
|
|
52
|
+
- Answer with a generic list of form-usability tips
|
|
53
|
+
|
|
54
|
+
**Why this case** - `design-research/skills/behavioural-analytics` exists for
|
|
55
|
+
this reading, and the shallow answer is indistinguishable from it in tone.
|
|
56
|
+
|
|
57
|
+
---
|
|
58
|
+
|
|
59
|
+
## Case 3 - The trap: a verdict with no dimension and no evidence
|
|
60
|
+
|
|
61
|
+
**Input**
|
|
62
|
+
> Here is a screenshot of our dashboard. Is the design good?
|
|
63
|
+
|
|
64
|
+
**Must contain**
|
|
65
|
+
- [ ] One critique per dimension it did read, each with an Observation, a Problem and a Fix
|
|
66
|
+
- [ ] A rating per finding - pass, minor or major - not a single overall score
|
|
67
|
+
- [ ] At least hierarchy, density and typography named, and the evidence each reading rests on (what on the screen shows it)
|
|
68
|
+
- [ ] Separates a contract or token violation from a taste preference
|
|
69
|
+
|
|
70
|
+
**Must not**
|
|
71
|
+
- Deliver "looks clean and modern" as the answer
|
|
72
|
+
- Propose a redesign without an observation that the redesign answers
|
|
73
|
+
|
|
74
|
+
**Why this case** - `visual-critique` is a fixed vocabulary with a rating, and the
|
|
75
|
+
per-dimension shape is the whole of its value; a one-line verdict is the failure
|
|
76
|
+
mode `feature-audit` and `product-analysis` are also built to avoid.
|
|
77
|
+
|
|
78
|
+
---
|
|
79
|
+
|
|
80
|
+
## What these do not prove
|
|
81
|
+
|
|
82
|
+
Nothing here measures whether reading a vendored body changes what a session
|
|
83
|
+
produces - these are three trap cases with a human or judge rater, not a
|
|
84
|
+
behavioural measurement. `skills/research` owns the standard a claim about
|
|
85
|
+
behaviour has to meet.
|
|
@@ -0,0 +1,59 @@
|
|
|
1
|
+
# Index
|
|
2
|
+
|
|
3
|
+
One line per adopted entry: `name - what it makes - path`, read as
|
|
4
|
+
`skills/design-library/<path>`. Read the entry you need, never the tree: this
|
|
5
|
+
file is the routing surface, and every line is short enough for a router line.
|
|
6
|
+
|
|
7
|
+
## cognitive-accessibility
|
|
8
|
+
|
|
9
|
+
adaptive-personalisation - help and onboarding adapted to the person - cognitive-accessibility/skills/adaptive-personalisation/SKILL.md
|
|
10
|
+
assess-load - a load map of a multi-step process - cognitive-accessibility/skills/assess-load/SKILL.md
|
|
11
|
+
cognitive-load-assessment - a cognitive-load assessment - cognitive-accessibility/skills/cognitive-load-assessment/SKILL.md
|
|
12
|
+
contextual-help-design - help content, tooltips and support flows - cognitive-accessibility/skills/contextual-help-design/SKILL.md
|
|
13
|
+
error-prevention-recovery - error prevention and recovery flows - cognitive-accessibility/skills/error-prevention-recovery/SKILL.md
|
|
14
|
+
focus-attention-design - focus support and distraction reduction - cognitive-accessibility/skills/focus-attention-design/SKILL.md
|
|
15
|
+
memory-load-reduction - working-memory demand removed from a flow - cognitive-accessibility/skills/memory-load-reduction/SKILL.md
|
|
16
|
+
plain-language-design - content written or reviewed in plain language - cognitive-accessibility/skills/plain-language-design/SKILL.md
|
|
17
|
+
review - a cognitive accessibility review of a screen or flow - cognitive-accessibility/skills/review/SKILL.md
|
|
18
|
+
simplify - simplified content and interactions - cognitive-accessibility/skills/simplify/SKILL.md
|
|
19
|
+
wayfinding-navigation - navigation and information architecture - cognitive-accessibility/skills/wayfinding-navigation/SKILL.md
|
|
20
|
+
|
|
21
|
+
## adaptive-interfaces
|
|
22
|
+
|
|
23
|
+
colour-independence - an interface where colour is never the only signal - adaptive-interfaces/skills/colour-independence/SKILL.md
|
|
24
|
+
flexible-typography - a type system that scales with the reader - adaptive-interfaces/skills/flexible-typography/SKILL.md
|
|
25
|
+
information-density - a density the user can adjust - adaptive-interfaces/skills/information-density/SKILL.md
|
|
26
|
+
preference-audit - an audit of motion, contrast and text-scale respect - adaptive-interfaces/skills/preference-audit/SKILL.md
|
|
27
|
+
responsive-accessibility - a layout accessible at any size or zoom - adaptive-interfaces/skills/responsive-accessibility/SKILL.md
|
|
28
|
+
responsive-review - a review of a layout across devices and zoom - adaptive-interfaces/skills/responsive-review/SKILL.md
|
|
29
|
+
simplified-views - a reduced-complexity view of an interface - adaptive-interfaces/skills/simplified-views/SKILL.md
|
|
30
|
+
specify - adaptive behaviour specified for a component - adaptive-interfaces/skills/specify/SKILL.md
|
|
31
|
+
user-preference-respect - system preference detection and response - adaptive-interfaces/skills/user-preference-respect/SKILL.md
|
|
32
|
+
|
|
33
|
+
## inclusive-interaction
|
|
34
|
+
|
|
35
|
+
audit - a multi-modal interaction audit - inclusive-interaction/skills/audit/SKILL.md
|
|
36
|
+
design-flow - an interaction flow with inclusive input and output - inclusive-interaction/skills/design-flow/SKILL.md
|
|
37
|
+
feedback-and-status - feedback that works across sight, sound and touch - inclusive-interaction/skills/feedback-and-status/SKILL.md
|
|
38
|
+
gesture-alternatives - non-gesture alternatives to a gesture - inclusive-interaction/skills/gesture-alternatives/SKILL.md
|
|
39
|
+
keyboard-navigation - keyboard navigation and focus management - inclusive-interaction/skills/keyboard-navigation/SKILL.md
|
|
40
|
+
keyboard-review - a keyboard and focus review - inclusive-interaction/skills/keyboard-review/SKILL.md
|
|
41
|
+
motion-sensitivity - motion that respects vestibular and seizure limits - inclusive-interaction/skills/motion-sensitivity/SKILL.md
|
|
42
|
+
multi-modal-input - several input methods for one interface - inclusive-interaction/skills/multi-modal-input/SKILL.md
|
|
43
|
+
touch-target-design - target sizes and spacing that are forgiving - inclusive-interaction/skills/touch-target-design/SKILL.md
|
|
44
|
+
voice-interaction - a voice interface for diverse speech - inclusive-interaction/skills/voice-interaction/SKILL.md
|
|
45
|
+
|
|
46
|
+
## visual-critique
|
|
47
|
+
|
|
48
|
+
critique-affordance - an affordance critique, rated per finding - visual-critique/skills/critique-affordance/SKILL.md
|
|
49
|
+
critique-brand-consistency - a critique against the brand's tokens - visual-critique/skills/critique-brand-consistency/SKILL.md
|
|
50
|
+
critique-color - a colour critique: contrast, palette, meaning - visual-critique/skills/critique-color/SKILL.md
|
|
51
|
+
critique-composition - a composition critique: balance, whitespace, rhythm - visual-critique/skills/critique-composition/SKILL.md
|
|
52
|
+
critique-information-density - a density critique: load and priority - visual-critique/skills/critique-information-density/SKILL.md
|
|
53
|
+
critique-typography - a typography critique: scale, readability, tokens - visual-critique/skills/critique-typography/SKILL.md
|
|
54
|
+
critique-visual-hierarchy - a hierarchy critique: entry point, eye flow - visual-critique/skills/critique-visual-hierarchy/SKILL.md
|
|
55
|
+
|
|
56
|
+
## design-research
|
|
57
|
+
|
|
58
|
+
behavioural-analytics - funnel reading that rules out tracking artefacts - design-research/skills/behavioural-analytics/SKILL.md
|
|
59
|
+
qual-quant-triangulation - a study reconciling the data with what users say - design-research/skills/qual-quant-triangulation/SKILL.md
|