aigroup-workflow 2.1.2 → 2.2.0

package/skills/feature-forge/references/interview-questions.md

@@ -1,150 +0,0 @@
-# Interview Questions
-
-## PM Hat Questions
-
-Focus on user value and business goals.
-
-| Area | Questions |
-|------|-----------|
-| **Problem** | What problem does this solve? Who experiences this problem? How often? |
-| **Users** | Who are the target users? What are their goals? Technical level? |
-| **Value** | How will users benefit? What's the business value? ROI? |
-| **Scope** | What's in scope? What's explicitly out of scope? MVP vs full version? |
-| **Success** | How will we measure success? Key metrics? |
-| **Priority** | Is this a must-have, should-have, or nice-to-have? |
-
-### Example PM Questions
-
-```markdown
-For a "User Export" feature:
-- Who needs to export data and why?
-- What format do they need (CSV, JSON, Excel)?
-- How much data? 100 rows or 1 million?
-- Is this for compliance (GDPR) or convenience?
-- How often will this be used?
-- What's the deadline?
-```
-
-## Dev Hat Questions
-
-Focus on technical feasibility and edge cases.
-
-| Area | Questions |
-|------|-----------|
-| **Integration** | What systems does this touch? APIs, databases, services? |
-| **Security** | Authentication required? Data sensitivity (PII, PCI)? |
-| **Performance** | Expected load? Response time requirements? Async OK? |
-| **Edge Cases** | What happens when X fails? Empty states? Limits? |
-| **Data** | What's stored? Retention period? Backup needs? |
-| **Dependencies** | External services? Rate limits? Costs? |
-
-### Example Dev Questions
-
-```markdown
-For a "User Export" feature:
-- What fields to include? Are any sensitive (passwords, tokens)?
-- Max export size? Need streaming or background job?
-- Should include soft-deleted records?
-- What happens if export fails midway?
-- File retention - how long to keep generated files?
-- Need progress indicator for large exports?
-```
-
-## Tool Usage: AskUserQuestions
-
-Use `AskUserQuestions` when questions have a finite set of likely answers. Use open-ended follow-up when answers are unbounded.
-
-### When to Use Structured Options
-
-| Question Pattern | Example | Options Style |
-|-----------------|---------|---------------|
-| Priority/ranking | "Is this must-have or nice-to-have?" | Single select: Must-have, Should-have, Nice-to-have |
-| Format selection | "What export format?" | Multi-select: CSV, JSON, Excel, PDF |
-| Scope decisions | "MVP or full version?" | Single select: MVP, Full, Phased |
-| Yes/No with nuance | "Auth required?" | Single select: Public, Authenticated, Role-based |
-
-### When to Use Open-Ended
-
-- "Describe the user journey in your own words"
-- "What problem does this solve?"
-- "Walk me through the workflow"
-
-### Example: Structured Elicitation
-
-For a "User Export" feature, batch related choices:
-
-**Question 1** (header: "Export scope"):
-"What data should users be able to export?"
-Options: "Own data only", "Team data", "Organization-wide", multi-select enabled
-
-**Question 2** (header: "Format"):
-"Which export formats should be supported?"
-Options: "CSV", "JSON", "Excel (.xlsx)", "PDF", multi-select enabled
-
-**Question 3** (header: "Priority"):
-"How critical is this feature?"
-Options: "Must-have (blocking)", "Should-have (important)", "Nice-to-have (future)"
-
----
-
-## Interview Flow
-
-### Phase 1: Discovery
-Use open-ended questions to understand the problem space:
-1. "Tell me about this feature in your own words"
-2. "What problem are we solving?"
-
-Then use `AskUserQuestions` to narrow down:
-- Target users (single select from identified personas)
-- Usage frequency (Daily, Weekly, Monthly, Rarely)
-- Priority (Must-have, Should-have, Nice-to-have)
-
-### Phase 2: Details
-Use `AskUserQuestions` for scope and constraint decisions:
-- Scope: MVP vs Full vs Phased (single select)
-- Key capabilities (multi-select from discovered items)
-
-Then open-ended: "Walk me through the user journey"
-
-### Phase 3: Edge Cases
-Use `AskUserQuestions` for technical trade-offs:
-- Error handling approach (Retry, Fail fast, Queue, Notify)
-- Data limits (multi-select thresholds)
-
-Then open-ended: "What happens when [X] fails?"
-
-### Phase 4: Validation
-Present spec summary, then use `AskUserQuestions`:
-- "Does this capture your requirements?" (Yes / Needs changes / Major gaps)
-- Per-requirement priority confirmation if needed
-
-## Multi-Agent Pre-Discovery
-
-For features spanning multiple domains, launch Task subagents with relevant skills **before** starting the interview. This front-loads technical context so the interview focuses on decisions rather than exploration.
-
-### Pattern: Parallel Skill-Invoked Discovery
-
-```
-User request: "I need a feature that does X"
-
-Before interview, launch subagents in parallel:
-- Task(subagent_type="general-purpose"): Invoke architecture-designer skill to assess system impact
-- Task(subagent_type="general-purpose"): Invoke security-reviewer skill to identify auth/data concerns
-- Task(subagent_type="Explore"): Search codebase for existing patterns related to the feature
-
-Collect subagent findings → Use them to inform interview questions
-```
-
-This ensures the Feature Forge interview starts with concrete technical context rather than assumptions.
-
----
-
-## Quick Reference
-
-| Phase | Focus | Tool |
-|-------|-------|------|
-| Pre-Discovery | Technical context | Task subagents with skills |
-| Discovery | Problem, users, value | Open-ended → AskUserQuestions |
-| Details | Journey, scope, constraints | AskUserQuestions → Open-ended |
-| Edge Cases | Failures, limits, security | AskUserQuestions → Open-ended |
-| Validation | Summary, gaps | AskUserQuestions |

package/skills/feature-forge/references/pre-discovery-subagents.md

@@ -1,54 +0,0 @@
-# Pre-Discovery with Subagents
-
-For features spanning multiple domains (auth, database, UI, etc.) that need front-loaded technical context before the Feature Forge interview.
-
-## Overview
-
-For features spanning multiple domains, you can accelerate discovery by launching Task subagents with relevant skills BEFORE starting the Feature Forge interview. This front-loads technical context so the interview focuses on decisions rather than exploration.
-
-## When to Use
-
-- Feature touches 3+ distinct system layers (e.g., auth, database, UI)
-- Codebase is unfamiliar or underdocumented
-- You need concrete technical facts before asking requirements questions
-- Stakeholder time is limited and you want to minimize back-and-forth
-
-## When NOT to Use
-
-- Feature is well-scoped to a single domain
-- You already have deep codebase knowledge
-- Requirements are purely business/UX (no technical exploration needed)
-
-## Pattern
-
-```
-1. Identify domains the feature touches
-2. Launch parallel Task subagents with relevant skills:
-   - Architecture Designer → existing patterns and constraints
-   - Framework Expert → current implementation details
-   - Security Reviewer → security requirements and risks
-3. Collect findings from all subagents
-4. Begin Feature Forge interview with technical context loaded
-5. Focus interview on decisions, trade-offs, and requirements
-```
-
-## Example
-
-For a "user profile with avatar upload" feature:
-
-```
-Task subagent 1 (Architecture Designer):
-  "Analyze the current user model, storage patterns, and image handling in this codebase"
-
-Task subagent 2 (Security Reviewer):
-  "What security concerns exist for file upload in this stack?"
-
-Task subagent 3 (Framework Expert):
-  "How does this project handle API endpoints and file storage?"
-```
-
-Results feed into the Feature Forge interview, so questions like "Where should we store avatars?" come with context about existing patterns.
-
-## Integration with Interview Questions
-
-See `interview-questions.md` for the full multi-agent discovery pattern and how subagent findings map to interview categories.

package/skills/feature-forge/references/specification-template.md

@@ -1,103 +0,0 @@
-# Specification Template
-
-## Full Template
-
-```markdown
-# Feature: [Name]
-
-## Overview
-[2-3 sentence description of the feature and its value to users]
-
-## Functional Requirements
-
-### FR-001: [Requirement Name]
-While <precondition>, when <trigger>, the system shall <response>.
-
-### FR-002: [Requirement Name]
-While <precondition>, when <trigger>, the system shall <response>.
-
-## Non-Functional Requirements
-
-### Performance
-- Response time: < 200ms p95
-- Throughput: 1000 requests/minute
-- Data volume: Up to 1M records
-
-### Security
-- Authentication: JWT required
-- Authorization: Role-based (admin, user)
-- Data protection: PII encrypted at rest
-
-### Scalability
-- Concurrent users: 10,000
-- Peak load handling: Auto-scale to 3x
-- Data retention: 90 days
-
-## Acceptance Criteria
-
-### AC-001: [Scenario Name]
-Given [context/precondition]
-When [action taken]
-Then [expected result]
-
-### AC-002: [Scenario Name]
-Given [context/precondition]
-When [action taken]
-Then [expected result]
-
-## Error Handling
-
-| Error Condition | HTTP Code | User Message |
-|-----------------|-----------|--------------|
-| Invalid input | 400 | "Please check your input" |
-| Unauthorized | 401 | "Please log in to continue" |
-| Forbidden | 403 | "You don't have permission" |
-| Not found | 404 | "Resource not found" |
-| Conflict | 409 | "This already exists" |
-
-## Implementation TODO
-
-### Backend
-- [ ] Create database migration for X table
-- [ ] Implement X service with Y method
-- [ ] Add API endpoint POST /api/x
-- [ ] Add input validation schema
-- [ ] Add authorization check
-
-### Frontend
-- [ ] Create X component
-- [ ] Add form with validation
-- [ ] Implement API integration
-- [ ] Add loading/error states
-- [ ] Add success feedback
-
-### Testing
-- [ ] Unit tests for X service
-- [ ] Integration tests for API endpoint
-- [ ] E2E test for complete user flow
-
-## Out of Scope
-- [Feature/capability explicitly not included]
-- [Future enhancement to consider later]
-
-## Open Questions
-- [ ] [Question needing stakeholder input]
-- [ ] [Technical decision pending]
-```
-
-## Save Location
-
-Save as: `specs/{feature_name}.spec.md`
-
-## Required Sections Checklist
-
-| Section | Purpose | Required |
-|---------|---------|----------|
-| Overview | Quick understanding | Yes |
-| Functional Requirements | What it does | Yes |
-| Non-Functional Requirements | How well it does it | Yes |
-| Acceptance Criteria | How to verify | Yes |
-| Error Handling | Failure cases | Yes |
-| Implementation TODO | Action items | Yes |
-| Out of Scope | Prevent scope creep | Recommended |
-| Open Questions | Track decisions | As needed |

package/skills/fine-tuning-expert/SKILL.md

@@ -1,162 +0,0 @@
----
-name: fine-tuning-expert
-description: "Use when fine-tuning LLMs, training custom models, or adapting foundation models for specific tasks. Invoke for configuring LoRA/QLoRA adapters, preparing JSONL training datasets, setting hyperparameters for fine-tuning runs, adapter training, transfer learning, finetuning with Hugging Face PEFT, OpenAI fine-tuning, instruction tuning, RLHF, DPO, or quantizing and deploying fine-tuned models. Trigger terms include: LoRA, QLoRA, PEFT, finetuning, fine-tuning, adapter tuning, LLM training, model training, custom model."
-license: MIT
-metadata:
-  author: https://github.com/Jeffallan
-  version: "1.1.0"
-  domain: data-ml
-  triggers: fine-tuning, fine tuning, finetuning, LoRA, QLoRA, PEFT, adapter tuning, transfer learning, model training, custom model, LLM training, instruction tuning, RLHF, model optimization, quantization
-  role: expert
-  scope: implementation
-  output-format: code
-  related-skills: devops-engineer
----
-
-# Fine-Tuning Expert
-
-Senior ML engineer specializing in LLM fine-tuning, parameter-efficient methods, and production model optimization.
-
-## Core Workflow
-
-1. **Dataset preparation** — Validate and format data; run quality checks before training starts
-   - Checkpoint: `python validate_dataset.py --input data.jsonl` — fix all errors before proceeding
-2. **Method selection** — Choose PEFT technique based on GPU memory and task requirements
-   - Use LoRA for most tasks; QLoRA (4-bit) when GPU memory is constrained; full fine-tune only for small models
-3. **Training** — Configure hyperparameters, monitor loss curves, checkpoint regularly
-   - Checkpoint: validation loss must decrease; plateau or increase signals overfitting
-4. **Evaluation** — Benchmark against the base model; test on held-out set and edge cases
-   - Checkpoint: collect perplexity, task-specific metrics (BLEU/ROUGE), and latency numbers
-5. **Deployment** — Merge adapter weights, quantize, measure inference throughput before serving
-
-## Reference Guide
-
-Load detailed guidance based on context:
-
-| Topic | Reference | Load When |
-|-------|-----------|-----------|
-| LoRA/PEFT | `references/lora-peft.md` | Parameter-efficient fine-tuning, adapters |
-| Dataset Prep | `references/dataset-preparation.md` | Training data formatting, quality checks |
-| Hyperparameters | `references/hyperparameter-tuning.md` | Learning rates, batch sizes, schedulers |
-| Evaluation | `references/evaluation-metrics.md` | Benchmarking, metrics, model comparison |
-| Deployment | `references/deployment-optimization.md` | Model merging, quantization, serving |
-
-## Minimal Working Example — LoRA Fine-Tuning with Hugging Face PEFT
-
-```python
-from datasets import load_dataset
-from transformers import AutoTokenizer, AutoModelForCausalLM, TrainingArguments
-from peft import LoraConfig, get_peft_model, TaskType
-from trl import SFTTrainer
-import torch
-
-# 1. Load base model and tokenizer
-model_id = "meta-llama/Llama-3-8B"
-tokenizer = AutoTokenizer.from_pretrained(model_id)
-tokenizer.pad_token = tokenizer.eos_token
-
-model = AutoModelForCausalLM.from_pretrained(
-    model_id,
-    torch_dtype=torch.bfloat16,
-    device_map="auto",
-)
-
-# 2. Configure LoRA adapter
-lora_config = LoraConfig(
-    task_type=TaskType.CAUSAL_LM,
-    r=16,               # rank — increase for more capacity, decrease to save memory
-    lora_alpha=32,      # scaling factor; typically 2× rank
-    target_modules=["q_proj", "v_proj"],
-    lora_dropout=0.05,
-    bias="none",
-)
-model = get_peft_model(model, lora_config)
-model.print_trainable_parameters()  # verify: should be ~0.1–1% of total params
-
-# 3. Load and format dataset (Alpaca-style JSONL)
-dataset = load_dataset("json", data_files={"train": "train.jsonl", "test": "test.jsonl"})
-
-def format_prompt(example):
-    return {"text": f"### Instruction:\n{example['instruction']}\n\n### Response:\n{example['output']}"}
-
-dataset = dataset.map(format_prompt)
-
-# 4. Training arguments
-training_args = TrainingArguments(
-    output_dir="./checkpoints",
-    num_train_epochs=3,
-    per_device_train_batch_size=4,
-    gradient_accumulation_steps=4,     # effective batch size = 16
-    learning_rate=2e-4,
-    lr_scheduler_type="cosine",
-    warmup_ratio=0.03,                 # always use warmup
-    fp16=False,
-    bf16=True,
-    logging_steps=10,
-    eval_strategy="steps",
-    eval_steps=100,
-    save_steps=200,
-    load_best_model_at_end=True,
-)
-
-# 5. Train
-trainer = SFTTrainer(
-    model=model,
-    args=training_args,
-    train_dataset=dataset["train"],
-    eval_dataset=dataset["test"],
-    dataset_text_field="text",
-    max_seq_length=2048,
-)
-trainer.train()
-
-# 6. Save adapter weights only
-model.save_pretrained("./lora-adapter")
-tokenizer.save_pretrained("./lora-adapter")
-```
-
-**QLoRA variant** — add these lines before loading the model to enable 4-bit quantization:
-```python
-from transformers import BitsAndBytesConfig
-
-bnb_config = BitsAndBytesConfig(
-    load_in_4bit=True,
-    bnb_4bit_quant_type="nf4",
-    bnb_4bit_compute_dtype=torch.bfloat16,
-    bnb_4bit_use_double_quant=True,
-)
-model = AutoModelForCausalLM.from_pretrained(model_id, quantization_config=bnb_config, device_map="auto")
-```
-
-**Merge adapter into base model for deployment:**
-```python
-from peft import PeftModel
-
-base = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16)
-merged = PeftModel.from_pretrained(base, "./lora-adapter").merge_and_unload()
-merged.save_pretrained("./merged-model")
-```
-
-## Constraints
-
-### MUST DO
-- Validate dataset quality before training
-- Use parameter-efficient methods for large models (>7B)
-- Monitor training/validation loss curves
-- Document hyperparameters and training config
-- Version datasets and model checkpoints
-- Always include a learning rate warmup
-
-### MUST NOT DO
-- Skip data quality validation
-- Overfit on small datasets — use regularisation (dropout, weight decay) and early stopping
-- Merge incompatible adapters (mismatched rank, base model, or target modules)
-- Deploy without evaluation against a held-out set and latency benchmark
-
-## Output Templates
-
-When implementing fine-tuning, always provide:
-1. **Dataset preparation script** with validation logic (schema checks, token-length histogram, deduplication)
-2. **Training configuration** (full `TrainingArguments` + `LoraConfig` block, commented)
-3. **Evaluation script** reporting perplexity, task-specific metrics, and latency
-4. **Brief design rationale** — why this PEFT method, rank, and learning rate were chosen for this task