@monoes/monomindcli 2.15.3 → 2.15.5

package/org-skills/huggingface-transformers/references/models.md

@@ -0,0 +1,371 @@
+# Model Loading and Management
+
+## Overview
+
+The transformers library provides flexible model loading with automatic architecture detection, device management, and configuration control.
+
+## Loading Models
+
+### AutoModel Classes
+
+Use AutoModel classes for automatic architecture selection:
+
+```python
+from transformers import AutoModel, AutoModelForSequenceClassification, AutoModelForCausalLM
+
+# Base model (no task head)
+model = AutoModel.from_pretrained("bert-base-uncased")
+
+# Sequence classification
+model = AutoModelForSequenceClassification.from_pretrained("distilbert-base-uncased")
+
+# Causal language modeling (GPT-style)
+model = AutoModelForCausalLM.from_pretrained("gpt2")
+
+# Masked language modeling (BERT-style)
+from transformers import AutoModelForMaskedLM
+model = AutoModelForMaskedLM.from_pretrained("bert-base-uncased")
+
+# Sequence-to-sequence (T5-style)
+from transformers import AutoModelForSeq2SeqLM
+model = AutoModelForSeq2SeqLM.from_pretrained("t5-small")
+```
+
+### Common AutoModel Classes
+
+**NLP Tasks:**
+- `AutoModelForSequenceClassification`: Text classification, sentiment analysis
+- `AutoModelForTokenClassification`: NER, POS tagging
+- `AutoModelForQuestionAnswering`: Extractive QA
+- `AutoModelForCausalLM`: Text generation (GPT, Llama)
+- `AutoModelForMaskedLM`: Masked language modeling (BERT)
+- `AutoModelForSeq2SeqLM`: Translation, summarization (T5, BART)
+
+**Vision Tasks:**
+- `AutoModelForImageClassification`: Image classification
+- `AutoModelForObjectDetection`: Object detection
+- `AutoModelForImageSegmentation`: Image segmentation
+
+**Audio Tasks:**
+- `AutoModelForAudioClassification`: Audio classification
+- `AutoModelForSpeechSeq2Seq`: Speech recognition
+
+**Multimodal:**
+- `AutoModelForVision2Seq`: Image captioning, VQA
+
+## Loading Parameters
+
+### Basic Parameters
+
+**pretrained_model_name_or_path**: Model identifier or local path
+```python
+model = AutoModel.from_pretrained("bert-base-uncased")  # From Hub
+model = AutoModel.from_pretrained("./local/model/path")  # From disk
+```
+
+**num_labels**: Number of output labels for classification
+```python
+model = AutoModelForSequenceClassification.from_pretrained(
+    "bert-base-uncased",
+    num_labels=3
+)
+```
+
+**cache_dir**: Custom cache location
+```python
+model = AutoModel.from_pretrained("model-id", cache_dir="./my_cache")
+```
+
+### Device Management
+
+**device_map**: Automatic device allocation for large models
+```python
+# Automatically distribute across GPUs and CPU
+model = AutoModelForCausalLM.from_pretrained(
+    "meta-llama/Llama-2-7b-hf",
+    device_map="auto"
+)
+
+# Sequential placement
+model = AutoModelForCausalLM.from_pretrained(
+    "model-id",
+    device_map="sequential"
+)
+
+# Custom device map
+device_map = {
+    "transformer.layers.0": 0,      # GPU 0
+    "transformer.layers.1": 1,      # GPU 1
+    "transformer.layers.2": "cpu",  # CPU
+}
+model = AutoModel.from_pretrained("model-id", device_map=device_map)
+```
+
+Manual device placement:
+```python
+import torch
+model = AutoModel.from_pretrained("model-id")
+model.to("cuda:0")  # Move to GPU 0
+model.to(torch.device("cuda" if torch.cuda.is_available() else "cpu"))
+```
+
+### Precision Control
+
+**dtype**: Set model precision (preferred in v5; `torch_dtype` still works but is deprecated)
+```python
+import torch
+
+# Float16 (half precision)
+model = AutoModel.from_pretrained("model-id", dtype=torch.float16)
+
+# BFloat16 (better range than float16)
+model = AutoModel.from_pretrained("model-id", dtype=torch.bfloat16)
+
+# Auto (use original dtype)
+model = AutoModel.from_pretrained("model-id", dtype="auto")
+```
+
+### Attention Implementation
+
+**attn_implementation**: Choose attention mechanism
+```python
+# Scaled Dot Product Attention (PyTorch 2.0+, fastest)
+model = AutoModel.from_pretrained("model-id", attn_implementation="sdpa")
+
+# Flash Attention 2 (requires flash-attn package)
+model = AutoModel.from_pretrained("model-id", attn_implementation="flash_attention_2")
+
+# Eager (default, most compatible)
+model = AutoModel.from_pretrained("model-id", attn_implementation="eager")
+```
+
+### Memory Optimization
+
+**low_cpu_mem_usage**: Reduce CPU memory during loading
+```python
+model = AutoModelForCausalLM.from_pretrained(
+    "large-model-id",
+    low_cpu_mem_usage=True,
+    device_map="auto"
+)
+```
+
+**BitsAndBytesConfig**: 8-bit and 4-bit quantization (requires optional `bitsandbytes`; `uv pip install bitsandbytes==0.49.2`)
+```python
+from transformers import BitsAndBytesConfig
+
+quantization_config = BitsAndBytesConfig(load_in_8bit=True)
+
+model = AutoModelForCausalLM.from_pretrained(
+    "model-id",
+    device_map="auto",
+    quantization_config=quantization_config
+)
+```
+
+**4-bit QLoRA-style loading**: use `BitsAndBytesConfig` instead of direct `load_in_4bit` arguments
+```python
+import torch
+from transformers import BitsAndBytesConfig
+
+quantization_config = BitsAndBytesConfig(
+    load_in_4bit=True,
+    bnb_4bit_compute_dtype=torch.bfloat16,
+    bnb_4bit_quant_type="nf4"
+)
+
+model = AutoModelForCausalLM.from_pretrained(
+    "model-id",
+    quantization_config=quantization_config,
+    device_map="auto"
+)
+```
+
+## Model Configuration
+
+### Loading with Custom Config
+
+```python
+from transformers import AutoConfig, AutoModel
+
+# Load and modify config
+config = AutoConfig.from_pretrained("bert-base-uncased")
+config.hidden_dropout_prob = 0.2
+config.attention_probs_dropout_prob = 0.2
+
+# Initialize model with custom config
+model = AutoModel.from_pretrained("bert-base-uncased", config=config)
+```
+
+### Initializing from Config Only
+
+```python
+config = AutoConfig.from_pretrained("gpt2")
+model = AutoModelForCausalLM.from_config(config)  # Random weights
+```
+
+## Model Modes
+
+### Training vs Evaluation Mode
+
+Models load in evaluation mode by default:
+
+```python
+model = AutoModel.from_pretrained("model-id")
+print(model.training)  # False
+
+# Switch to training mode
+model.train(True)
+
+# Switch back to evaluation mode (equivalent to eval mode on nn.Module)
+model.train(False)
+```
+
+Evaluation mode disables dropout and uses batch norm statistics. `model.train(False)` is equivalent to `model.eval()` in PyTorch.
+
+## Saving Models
+
+### Save Locally
+
+```python
+model.save_pretrained("./my_model")
+```
+
+This creates:
+- `config.json`: Model configuration
+- `pytorch_model.bin` or `model.safetensors`: Model weights
+
+### Save to Hugging Face Hub
+
+```python
+model.push_to_hub("username/model-name")
+
+# With custom commit message
+model.push_to_hub("username/model-name", commit_message="Update model")
+
+# Private repository
+model.push_to_hub("username/model-name", private=True)
+```
+
+## Model Inspection
+
+### Parameter Count
+
+```python
+# Total parameters
+total_params = model.num_parameters()
+
+# Trainable parameters only
+trainable_params = model.num_parameters(only_trainable=True)
+
+print(f"Total: {total_params:,}")
+print(f"Trainable: {trainable_params:,}")
+```
+
+### Memory Footprint
+
+```python
+memory_bytes = model.get_memory_footprint()
+memory_mb = memory_bytes / 1024**2
+print(f"Memory: {memory_mb:.2f} MB")
+```
+
+### Model Architecture
+
+```python
+print(model)  # Print full architecture
+
+# Access specific components
+print(model.config)
+print(model.base_model)
+```
+
+## Forward Pass
+
+Basic inference:
+
+```python
+from transformers import AutoTokenizer
+
+tokenizer = AutoTokenizer.from_pretrained("model-id")
+model = AutoModelForSequenceClassification.from_pretrained("model-id")
+
+inputs = tokenizer("Sample text", return_tensors="pt")
+outputs = model(**inputs)
+
+logits = outputs.logits
+predictions = logits.argmax(dim=-1)
+```
+
+## Model Formats
+
+### SafeTensors vs PyTorch
+
+SafeTensors is faster and safer:
+
+```python
+# Save as safetensors (recommended)
+model.save_pretrained("./model", safe_serialization=True)
+
+# Load either format automatically
+model = AutoModel.from_pretrained("./model")
+```
+
+### ONNX Export
+
+Export for optimized inference:
+
+```python
+from transformers.onnx import export
+
+# Export to ONNX
+export(
+    tokenizer=tokenizer,
+    model=model,
+    config=config,
+    output=Path("model.onnx")
+)
+```
+
+## Best Practices
+
+1. **Use AutoModel classes**: Automatic architecture detection
+2. **Specify `dtype` explicitly**: Control precision and memory (avoid deprecated `torch_dtype` in new code)
+3. **Use device_map="auto"**: For large models
+4. **Enable low_cpu_mem_usage**: When loading large models
+5. **Use safetensors format**: Faster and safer serialization
+6. **Check model.training**: Ensure correct mode for task
+7. **Consider quantization**: For deployment on resource-constrained devices
+8. **Cache models locally**: Set `HF_HOME` (Hub cache at `$HF_HOME/hub`)
+
+## Common Issues
+
+**CUDA out of memory:**
+```python
+import torch
+from transformers import BitsAndBytesConfig
+
+# Use smaller precision
+model = AutoModel.from_pretrained("model-id", dtype=torch.float16)
+
+# Or use quantization
+quantization_config = BitsAndBytesConfig(load_in_8bit=True)
+model = AutoModel.from_pretrained("model-id", quantization_config=quantization_config)
+
+# Or use CPU
+model = AutoModel.from_pretrained("model-id", device_map="cpu")
+```
+
+**Slow loading:**
+```python
+# Enable low CPU memory mode
+model = AutoModel.from_pretrained("model-id", low_cpu_mem_usage=True)
+```
+
+**Model not found:**
+```python
+# Verify model ID on hub.co
+# Check authentication for private models
+from huggingface_hub import login
+login()
+```

package/org-skills/huggingface-transformers/references/pipelines.md

@@ -0,0 +1,334 @@
+# Pipeline API Reference
+
+## Overview
+
+Pipelines provide the simplest way to use pre-trained models for inference. They abstract away tokenization, model loading, and post-processing, offering a unified interface for dozens of tasks.
+
+## Basic Usage
+
+Create a pipeline by specifying a task:
+
+```python
+from transformers import pipeline
+
+# Auto-select default model for task
+pipe = pipeline("text-classification")
+result = pipe("This is great!")
+```
+
+Or specify a model:
+
+```python
+pipe = pipeline("text-classification", model="distilbert-base-uncased-finetuned-sst-2-english")
+```
+
+## Supported Tasks
+
+### Natural Language Processing
+
+**text-generation**: Generate text continuations
+```python
+generator = pipeline("text-generation", model="gpt2")
+output = generator("Once upon a time", max_new_tokens=50, num_return_sequences=2)
+```
+
+**text-classification**: Classify text into categories
+```python
+classifier = pipeline("text-classification")
+result = classifier("I love this product!")  # Returns label and score
+```
+
+**token-classification**: Label individual tokens (NER, POS tagging)
+```python
+ner = pipeline("token-classification", model="dslim/bert-base-NER")
+entities = ner("Hugging Face is based in New York City")
+```
+
+**question-answering**: Extract answers from context
+```python
+qa = pipeline("question-answering")
+result = qa(question="What is the capital?", context="Paris is the capital of France.")
+```
+
+**fill-mask**: Predict masked tokens
+```python
+unmasker = pipeline("fill-mask", model="bert-base-uncased")
+result = unmasker("Paris is the [MASK] of France")
+```
+
+**summarization**: Summarize long texts
+```python
+summarizer = pipeline("summarization", model="facebook/bart-large-cnn")
+summary = summarizer("Long article text...", max_length=130, min_length=30)
+```
+
+**translation**: Translate between languages
+```python
+translator = pipeline("translation_en_to_fr", model="Helsinki-NLP/opus-mt-en-fr")
+result = translator("Hello, how are you?")
+```
+
+**zero-shot-classification**: Classify without training data
+```python
+classifier = pipeline("zero-shot-classification", model="facebook/bart-large-mnli")
+result = classifier(
+    "This is a course about Python programming",
+    candidate_labels=["education", "politics", "business"]
+)
+```
+
+**sentiment-analysis**: Alias for text-classification focused on sentiment
+```python
+sentiment = pipeline("sentiment-analysis")
+result = sentiment("This product exceeded my expectations!")
+```
+
+### Computer Vision
+
+**image-classification**: Classify images
+```python
+classifier = pipeline("image-classification", model="google/vit-base-patch16-224")
+result = classifier("path/to/image.jpg")
+# Or use PIL Image or URL
+from PIL import Image
+result = classifier(Image.open("image.jpg"))
+```
+
+**object-detection**: Detect objects in images
+```python
+detector = pipeline("object-detection", model="facebook/detr-resnet-50")
+results = detector("image.jpg")  # Returns bounding boxes and labels
+```
+
+**image-segmentation**: Segment images
+```python
+segmenter = pipeline("image-segmentation", model="facebook/detr-resnet-50-panoptic")
+segments = segmenter("image.jpg")
+```
+
+**depth-estimation**: Estimate depth from images
+```python
+depth = pipeline("depth-estimation", model="Intel/dpt-large")
+result = depth("image.jpg")
+```
+
+**zero-shot-image-classification**: Classify images without training
+```python
+classifier = pipeline("zero-shot-image-classification", model="openai/clip-vit-base-patch32")
+result = classifier("image.jpg", candidate_labels=["cat", "dog", "bird"])
+```
+
+### Audio
+
+**automatic-speech-recognition**: Transcribe speech
+```python
+asr = pipeline("automatic-speech-recognition", model="openai/whisper-base")
+text = asr("audio.mp3")
+```
+
+**audio-classification**: Classify audio
+```python
+classifier = pipeline("audio-classification", model="MIT/ast-finetuned-audioset-10-10-0.4593")
+result = classifier("audio.wav")
+```
+
+**text-to-speech**: Generate speech from text (with specific models)
+```python
+tts = pipeline("text-to-speech", model="microsoft/speecht5_tts")
+audio = tts("Hello, this is a test")
+```
+
+### Multimodal
+
+**visual-question-answering**: Answer questions about images
+```python
+vqa = pipeline("visual-question-answering", model="dandelin/vilt-b32-finetuned-vqa")
+result = vqa(image="image.jpg", question="What color is the car?")
+```
+
+**document-question-answering**: Answer questions about documents
+```python
+doc_qa = pipeline("document-question-answering", model="impira/layoutlm-document-qa")
+result = doc_qa(image="document.png", question="What is the invoice number?")
+```
+
+**image-to-text**: Generate captions for images
+```python
+captioner = pipeline("image-to-text", model="Salesforce/blip-image-captioning-base")
+caption = captioner("image.jpg")
+```
+
+## Pipeline Parameters
+
+### Common Parameters
+
+**model**: Model identifier or path
+```python
+pipe = pipeline("task", model="model-id")
+```
+
+**device**: GPU device index (-1 for CPU, 0+ for GPU)
+```python
+pipe = pipeline("task", device=0)  # Use first GPU
+```
+
+**device_map**: Automatic device allocation for large models
+```python
+pipe = pipeline("task", model="large-model", device_map="auto")
+```
+
+**dtype**: Model precision (reduces memory; `torch_dtype` is deprecated but still accepted)
+```python
+import torch
+pipe = pipeline("task", dtype=torch.float16)
+```
+
+**batch_size**: Process multiple inputs at once
+```python
+pipe = pipeline("task", batch_size=8)
+results = pipe(["text1", "text2", "text3"])
+```
+
+**Backend**: Transformers v5 pipelines use PyTorch only (TensorFlow/JAX backends were removed in v5).
+
+## Batch Processing
+
+Process multiple inputs efficiently:
+
+```python
+classifier = pipeline("text-classification")
+texts = ["Great product!", "Terrible experience", "Just okay"]
+results = classifier(texts)
+```
+
+For large datasets, use generators or KeyDataset:
+
+```python
+from transformers.pipelines.pt_utils import KeyDataset
+import datasets
+
+dataset = datasets.load_dataset("dataset-name", split="test")
+pipe = pipeline("task", device=0)
+
+for output in pipe(KeyDataset(dataset, "text")):
+    print(output)
+```
+
+## Performance Optimization
+
+### GPU Acceleration
+
+Always specify device for GPU usage:
+```python
+pipe = pipeline("task", device=0)
+```
+
+### Mixed Precision
+
+Use float16 for 2x speedup on supported GPUs:
+```python
+import torch
+pipe = pipeline("task", dtype=torch.float16, device=0)
+```
+
+### Batching Guidelines
+
+- **CPU**: Usually skip batching
+- **GPU with variable lengths**: May reduce efficiency
+- **GPU with similar lengths**: Significant speedup
+- **Real-time applications**: Skip batching (increases latency)
+
+```python
+# Good for throughput
+pipe = pipeline("task", batch_size=32, device=0)
+results = pipe(list_of_texts)
+```
+
+### Streaming Output
+
+For text generation, stream tokens as they're generated:
+
+```python
+from transformers import AutoTokenizer, TextStreamer, pipeline
+
+tokenizer = AutoTokenizer.from_pretrained("gpt2")
+streamer = TextStreamer(tokenizer)
+generator = pipeline("text-generation", model="gpt2", streamer=streamer)
+generator("The future of AI", max_new_tokens=100)
+```
+
+## Custom Pipeline Configuration
+
+Specify tokenizer and model separately:
+
+```python
+from transformers import AutoTokenizer, AutoModelForSequenceClassification
+
+tokenizer = AutoTokenizer.from_pretrained("model-id")
+model = AutoModelForSequenceClassification.from_pretrained("model-id")
+pipe = pipeline("text-classification", model=model, tokenizer=tokenizer)
+```
+
+Use custom pipeline classes:
+
+```python
+from transformers import TextClassificationPipeline
+
+class CustomPipeline(TextClassificationPipeline):
+    def postprocess(self, model_outputs, **kwargs):
+        # Custom post-processing
+        return super().postprocess(model_outputs, **kwargs)
+
+pipe = pipeline("text-classification", model="model-id", pipeline_class=CustomPipeline)
+```
+
+## Input Formats
+
+Pipelines accept various input types:
+
+**Text tasks**: Strings or lists of strings
+```python
+pipe("single text")
+pipe(["text1", "text2"])
+```
+
+**Image tasks**: URLs, file paths, PIL Images, or numpy arrays
+```python
+pipe("https://example.com/image.jpg")
+pipe("local/path/image.png")
+pipe(PIL.Image.open("image.jpg"))
+pipe(numpy_array)
+```
+
+**Audio tasks**: File paths, numpy arrays, or raw waveforms
+```python
+pipe("audio.mp3")
+pipe(audio_array)
+```
+
+## Error Handling
+
+Handle common issues:
+
+```python
+try:
+    result = pipe(input_data)
+except Exception as e:
+    if "CUDA out of memory" in str(e):
+        # Reduce batch size or use CPU
+        pipe = pipeline("task", device=-1)
+    elif "does not appear to have a file named" in str(e):
+        # Model not found
+        print("Check model identifier")
+    else:
+        raise
+```
+
+## Best Practices
+
+1. **Use pipelines for prototyping**: Fast iteration without boilerplate
+2. **Specify models explicitly**: Default models may change
+3. **Enable GPU when available**: Significant speedup
+4. **Use batching for throughput**: When processing many inputs
+5. **Consider memory usage**: Use float16 or smaller models for large batches
+6. **Cache models locally**: Avoid repeated downloads