@groupby/ai-dev 0.5.5 → 0.5.8

package/teams/fhr-ai-team/skills/algo-test-planning/references/pipeline-registry.md

@@ -0,0 +1,280 @@
+# Pipeline Registry
+
+Complete registry of all AI pipelines from `attraqt-kubeflow-pipelines`.
+Source: `kubeflow_pipelines/pipelines/ai/__init__.py`
+
+---
+
+## Base Pipelines (Single-Step Templates)
+
+These are generic pipeline wrappers for running a single batch job step.
+
+| Pipeline | Type | Use case |
+|----------|------|----------|
+| `python_batch_pipeline` | Python | Standard Python batch jobs |
+| `large_python_batch_pipeline` | Python | GPU/high-memory Python batch jobs |
+| `scala_batch_pipeline` | Scala | Scala-based batch jobs |
+| `spark_scala_batch_pipeline` | Spark Scala | Spark Scala batch jobs |
+
+**Config differences:**
+- Python pipelines use `batch_config.arguments` for custom params
+- Scala pipelines use `batch_config.custom_params` (NOT `arguments`)
+- `large_python_batch_pipeline` supports GPU: set `gpu_vendor: "nvidia.com/gpu"` and `gpu_accelerator_name: "nvidia-l4"`
+
+---
+
+## Semantic Search
+
+| Pipeline | Type | Description |
+|----------|------|-------------|
+| `semantic_search_learning_pipeline` | Full | Learning only |
+| `semantic_search_learning_with_generated_analytics_pipeline` | Full | Learning + analytics generation (most common) |
+| `semantic_search_item_encoding_pipeline` | Full | Item encoding after training |
+| `export_huggingface_sentence_transformer_model_pipeline` | Script | Export HuggingFace sentence transformer model |
+
+---
+
+## Search (Text-only, no images)
+
+| Pipeline | Type | Description |
+|----------|------|-------------|
+| `search_rnn_learning_pipeline_without_images` | Full | RNN-based search learning |
+| `search_llm_learning_pipeline_without_images` | Full | LLM-based search learning |
+| `search_item_encoding_pipeline_without_images` | Full | Item encoding (text only) |
+
+---
+
+## Search + CLIP (Text + Images)
+
+| Pipeline | Type | Description |
+|----------|------|-------------|
+| `clip_search_rnn_learning_pipeline` | Full | CLIP + RNN search learning |
+| `clip_search_llm_learning_pipeline` | Full | CLIP + LLM search learning |
+| `clip_search_llm_learning_pipeline_with_data_augmentation` | Full | CLIP + LLM with data augmentation |
+| `clip_search_vertical_rnn_learning_pipeline` | Full | Vertical (per-tenant) CLIP + RNN learning |
+| `clip_search_vertical_llm_learning_pipeline` | Full | Vertical (per-tenant) CLIP + LLM learning |
+| `clip_large_search_vertical_rnn_learning_pipeline` | Full | Large vertical CLIP + RNN (GPU) |
+| `clip_large_search_vertical_llm_learning_pipeline` | Full | Large vertical CLIP + LLM (GPU) |
+| `clip_search_item_encoding_pipeline` | Full | CLIP search item encoding |
+
+---
+
+## Search + Image Encoder
+
+| Pipeline | Type | Description |
+|----------|------|-------------|
+| `image_encoder_search_item_encoding_pipeline_with_images` | Full | Image encoder search encoding (with images) |
+| `image_encoder_search_item_encoding_pipeline_without_images` | Full | Image encoder search encoding (without images) |
+
+---
+
+## Search Evaluation
+
+| Pipeline | Type | Description |
+|----------|------|-------------|
+| `search_evaluation_pipeline` | Full | Standard search evaluation |
+| `search_llm_evaluation_pipeline` | Full | LLM-based search evaluation |
+
+---
+
+## Computer Vision - CLIP
+
+| Pipeline | Type | Description |
+|----------|------|-------------|
+| `clip_learning_pipeline` | Full | CLIP model learning |
+| `clip_vertical_learning_pipeline` | Full | Per-tenant CLIP learning |
+| `large_clip_learning_pipeline` | Full | Large CLIP learning (GPU) |
+| `clip_item_images_single_encoding_pipeline` | Full | CLIP single image encoding |
+| `export_huggingface_clip_model_pipeline` | Script | Export HuggingFace CLIP model |
+
+---
+
+## Computer Vision - Image Encoder
+
+| Pipeline | Type | Description |
+|----------|------|-------------|
+| `computer_vision_learning_pipeline` | Full | Image encoder learning |
+| `computer_vision_vertical_learning_pipeline` | Full | Per-tenant image encoder learning |
+| `large_computer_vision_vertical_learning_pipeline` | Full | Large vertical learning (GPU) |
+| `computer_vision_item_images_single_encoding_pipeline` | Full | Image encoding |
+
+---
+
+## Computer Vision - SAM
+
+| Pipeline | Type | Description |
+|----------|------|-------------|
+| `export_huggingface_sam_model_pipeline` | Script | Export HuggingFace SAM model |
+
+---
+
+## FM (Factorization Machines / Recommendations)
+
+| Pipeline | Type | Description |
+|----------|------|-------------|
+| `fm_global_initialization_pipeline` | Full | FM global model initialization |
+| `fm_global_incremental_pipeline` | Full | FM global incremental update |
+| `fm_complementarity_initialization_pipeline` | Full | FM complementarity initialization |
+| `fm_complementarity_incremental_pipeline` | Full | FM complementarity incremental update |
+
+---
+
+## GPT (Generative)
+
+| Pipeline | Type | Description |
+|----------|------|-------------|
+| `gpt_initialization_pipeline_with_images` | Full | GPT init (with images) |
+| `gpt_initialization_pipeline_without_images` | Full | GPT init (text only) |
+| `gpt_incremental_pipeline_with_images` | Full | GPT incremental (with images) |
+| `gpt_incremental_pipeline_without_images` | Full | GPT incremental (text only) |
+| `gpt_item_encoding_pipeline_with_images` | Full | GPT item encoding (with images) |
+| `gpt_item_encoding_pipeline_without_images` | Full | GPT item encoding (text only) |
+
+---
+
+## Tagging
+
+| Pipeline | Type | Description |
+|----------|------|-------------|
+| `tagging_learning_pipeline` | Full | Tagging model learning |
+| `tagging_item_tagging_pipeline` | Full | Apply tagging to items |
+| `tagging_item_macro_tagging_pipeline` | Full | Apply macro tagging to items |
+
+---
+
+## Shop the Look
+
+| Pipeline | Type | Description |
+|----------|------|-------------|
+| `shop_the_look_recommendation_pipeline` | Full | STL recommendations |
+| `shop_the_look_recommendation_with_segmentation_pipeline` | Full | STL with image segmentation |
+| `shop_the_look_recommendation_without_segmentation_pipeline` | Full | STL without segmentation |
+| `shop_the_look_recommendation_with_outfit_detection_pipeline` | Full | STL with outfit detection |
+| `outfit_image_classification_learning_pipeline` | Full | Outfit classifier learning |
+| `outfit_image_classification_vertical_learning_pipeline` | Full | Per-tenant outfit classifier |
+
+---
+
+## YOLO (Object Detection)
+
+| Pipeline | Type | Description |
+|----------|------|-------------|
+| `yolo_model_fine_tuning_pipeline` | Full | YOLO model fine-tuning |
+| `yolo_model_fine_tuning_vertical_pipeline` | Full | Per-tenant YOLO fine-tuning |
+| `export_ultralytics_yolo_model_pipeline` | Script | Export Ultralytics YOLO model |
+
+---
+
+## Item and Analytic Data
+
+| Pipeline | Type | Description |
+|----------|------|-------------|
+| `xo_item_data_pipeline` | Full | XO item data ingestion |
+| `fhr_item_data_pipeline` | Full | FHR item data ingestion |
+| `fhr_item_data_pipeline_legacy` | Full | FHR item data (legacy) |
+| `cidp_item_data_pipeline` | Full | CIDP item data ingestion |
+| `fhr_analytic_incremental_data_pipeline` | Full | FHR analytics incremental |
+| `fhr_analytic_incremental_data_pipeline_legacy` | Full | FHR analytics incremental (legacy) |
+| `fhr_analytic_data_pipeline_legacy` | Full | FHR analytics (legacy) |
+
+---
+
+## NLP
+
+| Pipeline | Type | Description |
+|----------|------|-------------|
+| `nlp_word_tokenizer_pipeline` | Full | Word tokenizer training |
+| `nlp_character_tokenizer_pipeline` | Full | Character tokenizer training |
+
+---
+
+## Content-Based
+
+| Pipeline | Type | Description |
+|----------|------|-------------|
+| `content_based_word2vec_pipeline` | Full | Word2Vec content-based recommendations |
+
+---
+
+## ALS (Alternating Least Squares)
+
+| Pipeline | Type | Description |
+|----------|------|-------------|
+| `als_pipeline` | Full | ALS collaborative filtering |
+
+---
+
+## FP-Growth
+
+| Pipeline | Type | Description |
+|----------|------|-------------|
+| `fp_growth_items_pipeline` | Full | FP-Growth item associations |
+| `fp_growth_categories_pipeline` | Full | FP-Growth category associations |
+
+---
+
+## Pass-Through (Graph)
+
+| Pipeline | Type | Description |
+|----------|------|-------------|
+| `pass_through_scored_graph_pipeline` | Full | Scored graph pass-through |
+| `pass_through_unscored_graph_1_pipeline` | Full | Unscored graph variant 1 |
+| `pass_through_unscored_graph_2_pipeline` | Full | Unscored graph variant 2 |
+| `pass_through_source_to_items_unscored_graph_pipeline` | Full | Source-to-items unscored graph |
+
+---
+
+## Autocomplete
+
+| Pipeline | Type | Description |
+|----------|------|-------------|
+| `autocomplete_pipeline` | Full | Autocomplete model training |
+
+---
+
+## Miscellaneous
+
+| Pipeline | Type | Description |
+|----------|------|-------------|
+| `basic_pipeline` | Full | Basic/generic pipeline template |
+| `sessions_pipeline` | Full | Session data processing |
+| `bigquery_cleanup_pipeline` | Full | BigQuery data cleanup |
+| `gibberish_pipeline` | Full | Gibberish detection |
+| `dummy_ai_scores_pipeline` | Full | Dummy AI scores (testing) |
+| `item_tagging_pipeline` | Full | Items enrichment tagging |
+| `merch_agent_data_pipeline` | Full | Merch agent data preparation |
+| `lakefs_garbage_collection_pipeline` | Full | LakeFS garbage collection |
+
+---
+
+## Label Studio
+
+| Pipeline | Type | Description |
+|----------|------|-------------|
+| `outfit_tasks_import_pipeline` | Script | Import outfit tasks to Label Studio |
+| `outfit_annotations_export_pipeline` | Script | Export outfit annotations from Label Studio |
+| `yolo_tasks_import_pipeline` | Script | Import YOLO tasks to Label Studio |
+| `yolo_annotations_export_pipeline` | Script | Export YOLO annotations from Label Studio |
+
+---
+
+## Monitoring and Maintenance
+
+| Pipeline | Type | Description |
+|----------|------|-------------|
+| `activity_monitoring` | Monitoring | Activity monitoring |
+| `experiments_with_consecutive_failed_runs_monitoring_pipeline` | Monitoring | Failed experiments monitoring |
+| `runs_with_abnormal_duration_cleaning_pipeline` | Monitoring | Abnormal duration cleanup |
+| `gcs_cleaning_pipeline` | Script | GCS storage cleanup |
+| `gcs_activities_copy_pipeline` | Script | GCS activities data copy |
+| `image_download_pipeline` | Script | Image download utility |
+| `inference_data_cleaning_pipeline` | Script | Inference data cleanup |
+
+---
+
+## Total: ~93 pipelines
+
+- 4 base pipelines
+- ~70 full (multi-step) pipelines
+- ~12 script/utility pipelines
+- ~7 monitoring/maintenance pipelines

package/teams/fhr-ai-team/skills/brainstorming/SKILL.md

@@ -0,0 +1,111 @@
+---
+name: brainstorming
+description: >
+  Use when the user wants to brainstorm, design, or explore a new feature, improvement,
+  or architecture decision. Discovers AI team repos via gh, searches existing code before
+  proposing solutions, and gathers requirements interactively via AskUserQuestion.
+---
+
+# Codebase-Aware Brainstorming
+
+## Hard Gate
+
+Do NOT invoke any implementation skill, write any code, scaffold any project, or take any
+implementation action until you have presented a design and the user has approved it.
+
+## Process
+
+### Step 1: Discover AI Team Repos
+
+Run the following to get the current repo landscape:
+
+```bash
+gh repo list Attraqt --json name,description --limit 200 --no-archived
+```
+
+Filter results for repos matching `ai.*`, `algo.*`, `ebap-*`, `attraqt-kubeflow-*`, and `*-toolbox` patterns.
+Present the user with a summary of the relevant repos grouped by category:
+
+| Category | Pattern | Purpose |
+|----------|---------|---------|
+| ML algorithms | `algo.*` | Model training, inference, evaluation |
+| ML training | `algo.*-ml` | Kubeflow-based model training/fine-tuning |
+| AI services | `ai.*` | FastAPI/Streamlit microservices |
+| Toolboxes | `*-toolbox` | Shared Python libraries |
+| Kubeflow infra | `attraqt-kubeflow-*` | Pipeline configs and definitions |
+| Platform infra | `ebap-*` | Early Birds AI Platform |
+
+### Step 2: Explore Project Context
+
+For repos relevant to the brainstorm topic:
+- Read their `CLAUDE.md` or `README.md` for architecture context
+- Check recent git history (`git log --oneline -20`) for active development areas
+- Scan directory structure to understand component layout
+
+### Step 3: Search Before Proposing
+
+**MANDATORY:** Before proposing any solution, search the codebase for existing utilities,
+patterns, and implementations related to the topic.
+
+- Use Grep/Glob across relevant repos
+- Check shared libraries: `earlybirds_commons`, `torch_toolbox`, `item-toolbox`, `nlp-toolbox`, `eb_tensorflow`
+- Report findings to the user: "I found X in repo Y that does something similar"
+
+If existing code covers part of the need, build on it rather than proposing greenfield work.
+
+### Step 4: Gather Requirements
+
+Use the AskUserQuestion tool to gather requirements interactively.
+
+Rules:
+- **One question per message.** Do not batch multiple questions.
+- **Prefer multiple-choice** over open-ended questions. Provide 2-4 concrete options based on what you found in the codebase.
+- Cover these dimensions (not all at once; ask only what is relevant):
+  - Scope: what is in/out
+  - Target repos: which repos are affected
+  - Constraints: performance, compatibility, timeline
+  - Dependencies: what must exist first
+  - Users: who benefits from this
+
+### Step 5: Propose 2-3 Approaches
+
+For each approach, include:
+- **Summary:** one-sentence description
+- **Trade-offs:** pros, cons, effort
+- **Repos affected:** which repos need changes
+- **Reuse opportunities:** what existing code can be leveraged
+- **Concrete code references:** point to specific files/functions in real repos
+
+### Step 6: Present Design in Sections
+
+Break the design into focused sections, each covering one concern.
+Wait for user feedback between sections. Sections might include:
+- Data model / schema changes
+- API contracts
+- Pipeline configuration
+- Integration points with existing code
+- Testing strategy
+
+### Step 7: Write Design Document
+
+After user approval, save the design to `docs/specs/YYYY-MM-DD-<topic>-design.md`
+in the relevant project repo. Include:
+- Problem statement
+- Chosen approach (with rationale)
+- Detailed design per section
+- Open questions (if any remain)
+- References to existing code being reused
+
+### Step 8: Self-Review
+
+Before presenting the final spec, review it for:
+- Placeholders or vague language ("TBD", "as appropriate", "handle errors")
+- Contradictions between sections
+- Scope creep beyond what was agreed
+- Missing error paths or edge cases
+- Naming convention violations (invoke `ai.pierre:naming-conventions-reviewer` if code is shown)
+
+### Step 9: User Review and Transition
+
+Present the spec for final user review. After approval, offer to invoke `/plan` to create
+implementation tasks from the approved design.

package/teams/fhr-ai-team/skills/e2e-testing/SKILL.md

@@ -0,0 +1,163 @@
+---
+name: e2e-testing
+description: >
+  Use when the user wants to run end-to-end tests of ML code, launch pipeline test runs,
+  or verify model outputs. Covers local pytest execution, Kubeflow pipeline launches,
+  MLflow metric validation, and pod-level debugging of failures.
+---
+
+# End-to-End ML Testing
+
+## Overview
+
+This skill handles all testing workflows for ML code, from local unit tests to full
+Kubeflow pipeline end-to-end runs and model validation.
+
+## Step 1: Determine Test Scope
+
+Use AskUserQuestion to ask:
+
+**Question:** "What type of testing do you want to run?"
+
+| Option | Description |
+|--------|-------------|
+| Local tests | Run pytest (unit and integration tests) |
+| Pipeline end-to-end | Launch a Kubeflow pipeline run and monitor it |
+| Model validation | Check MLflow metrics and model outputs |
+
+---
+
+## Local Tests
+
+### Identify the target
+- Determine which repo and test directory to run
+- Check for `pytest.ini`, `setup.cfg`, or `pyproject.toml` for test configuration
+- Check for test markers (e.g., `@pytest.mark.integration`, `@pytest.mark.slow`)
+
+### Run tests
+```bash
+pytest tests/ -v --tb=short
+```
+
+For specific test files or functions:
+```bash
+pytest tests/test_specific.py::test_function -v
+```
+
+With coverage:
+```bash
+pytest tests/ --cov=<package_name> --cov-report=term-missing
+```
+
+### Analyze failures
+- Read the full traceback
+- Check if it is a test environment issue (missing deps, wrong Python version, missing .env)
+- Check if it is a real code bug
+- Suggest fixes with exact code changes
+
+---
+
+## Pipeline End-to-End
+
+### Prerequisites check
+Before launching, verify:
+
+1. **`.env` file exists** in `attraqt-kubeflow-configs`:
+   ```bash
+   ls /Users/mehdi/dev/projects/attraqt-kubeflow-configs/.env
+   ```
+   If missing: `cp .env.dev .env`
+
+2. **`version_name` is valid**:
+   ```bash
+   python3 scripts/kf_query.py --pipeline-versions <pipeline_name>
+   ```
+
+3. **MongoDB config is correct** (for learning pipelines):
+   ```bash
+   mongosh "mongodb://10.11.96.21:27017/earlybirds" --quiet --eval '
+   const doc = db.predictors.findOne({"_id": ObjectId("<PREDICTOR_ID>")});
+   print(JSON.stringify(doc.config.batch, null, 2));
+   '
+   ```
+
+### Launch
+```bash
+cd /Users/mehdi/dev/projects/attraqt-kubeflow-configs/scripts
+python -m run -c <absolute_path_to_config>
+```
+
+### Monitor
+Poll the run status:
+```bash
+python3 scripts/kf_query.py <run_id>
+```
+
+Check for failures:
+```bash
+python3 scripts/kf_query.py <run_id> --failed
+```
+
+### Verify
+- All steps should show status "Succeeded"
+- Check output artifacts exist in GCS
+- For learning pipelines: verify MLflow run was created
+- For encoding pipelines: verify output encodings exist at expected GCS path
+
+### Debug Failures
+When a step fails:
+
+1. **Find the pod:**
+   ```bash
+   kubectl get pods -n kubeflow | grep <workflow-name>
+   ```
+
+2. **Read logs:**
+   ```bash
+   kubectl logs -n kubeflow <pod-name> --tail=200
+   kubectl logs -n kubeflow <pod-name> --previous  # if crashed
+   ```
+
+3. **Check events (OOM, scheduling, image pull):**
+   ```bash
+   kubectl describe pod -n kubeflow <pod-name>
+   ```
+
+4. **Common failure patterns:**
+   - OOM: increase memory in config, or reduce batch size in MongoDB
+   - Image pull error: wrong image version; verify with `kf_query.py --pipeline-versions`
+   - Config error: wrong arguments format (check `arguments` vs `custom_params`)
+   - GPU scheduling: check node availability with `kubectl get nodes`
+
+See `skills/ml-tooling-dev/references/kubectl-debug.md` for the full debugging reference.
+
+---
+
+## Model Validation
+
+### Fetch metrics
+```bash
+python3 scripts/mlflow_query.py run <run_id>
+```
+
+### Compare against baseline
+- Check key metrics (loss, accuracy, recall, precision) against previous runs
+- Use `mlflow_query.py runs <experiment_name>` to list recent runs for comparison
+
+### Check registered models
+```bash
+python3 scripts/mlflow_query.py model-for-predictor <predictor_id>
+python3 scripts/mlflow_query.py model <model_name>
+```
+
+Verify:
+- Model version was registered
+- Aliases are set correctly (e.g., "champion", "challenger")
+- Model artifact exists in the registry
+
+---
+
+## Skill Dependencies
+
+This skill invokes `ai.pierre:ml-tooling-dev` for all Kubeflow, MLflow, and MongoDB operations.
+For pipeline config generation, use `/plan-algo-tests` (invokes `ai.pierre:algo-test-planning`).

package/teams/fhr-ai-team/skills/grill-me/SKILL.md

@@ -0,0 +1,10 @@
+---
+name: grill-me
+description: Interview the user relentlessly about a plan or design until reaching shared understanding, resolving each branch of the decision tree. Use when user wants to stress-test a plan, get grilled on their design, or mentions "grill me".
+---
+
+Interview me relentlessly about every aspect of this plan until we reach a shared understanding. Walk down each branch of the design tree, resolving dependencies between decisions one-by-one. For each question, provide your recommended answer.
+
+Ask the questions one at a time.
+
+If a question can be answered by exploring the codebase, explore the codebase instead.