@synsci/cli-darwin-x64 1.1.97 → 1.1.99

package/bin/skills/transformer-lens/references/api.md

@@ -1,362 +0,0 @@
-# TransformerLens API Reference
-
-## HookedTransformer
-
-The core class for mechanistic interpretability, wrapping transformer models with hooks on every activation.
-
-### Loading Models
-
-```python
-from transformer_lens import HookedTransformer
-
-# Basic loading
-model = HookedTransformer.from_pretrained("gpt2-small")
-
-# With specific device/dtype
-model = HookedTransformer.from_pretrained(
-    "gpt2-medium",
-    device="cuda",
-    dtype=torch.float16
-)
-
-# Gated models (LLaMA, Mistral)
-import os
-os.environ["HF_TOKEN"] = "your_token"
-model = HookedTransformer.from_pretrained("meta-llama/Llama-2-7b-hf")
-```
-
-### from_pretrained() Parameters
-
-| Parameter | Type | Default | Description |
-|-----------|------|---------|-------------|
-| `model_name` | str | required | Model name from OFFICIAL_MODEL_NAMES |
-| `fold_ln` | bool | True | Fold LayerNorm weights into subsequent layers |
-| `center_writing_weights` | bool | True | Center residual stream writer means |
-| `center_unembed` | bool | True | Center unembedding weights |
-| `dtype` | torch.dtype | None | Model precision |
-| `device` | str | None | Target device |
-| `n_devices` | int | 1 | Number of devices for model parallelism |
-
-### Weight Matrices
-
-| Property | Shape | Description |
-|----------|-------|-------------|
-| `W_E` | [d_vocab, d_model] | Token embedding matrix |
-| `W_U` | [d_model, d_vocab] | Unembedding matrix |
-| `W_pos` | [n_ctx, d_model] | Positional embedding |
-| `W_Q` | [n_layers, n_heads, d_model, d_head] | Query weights |
-| `W_K` | [n_layers, n_heads, d_model, d_head] | Key weights |
-| `W_V` | [n_layers, n_heads, d_model, d_head] | Value weights |
-| `W_O` | [n_layers, n_heads, d_head, d_model] | Output weights |
-| `W_in` | [n_layers, d_model, d_mlp] | MLP input weights |
-| `W_out` | [n_layers, d_mlp, d_model] | MLP output weights |
-
-### Core Methods
-
-#### forward()
-
-```python
-logits = model(tokens)
-logits = model(tokens, return_type="logits")
-loss = model(tokens, return_type="loss")
-logits, loss = model(tokens, return_type="both")
-```
-
-Parameters:
-- `input`: Token tensor or string
-- `return_type`: "logits", "loss", "both", or None
-- `prepend_bos`: Whether to prepend BOS token
-- `start_at_layer`: Start execution from specific layer
-- `stop_at_layer`: Stop execution at specific layer
-
-#### run_with_cache()
-
-```python
-logits, cache = model.run_with_cache(tokens)
-
-# Selective caching (saves memory)
-logits, cache = model.run_with_cache(
-    tokens,
-    names_filter=lambda name: "resid_post" in name
-)
-
-# Cache on CPU
-logits, cache = model.run_with_cache(tokens, device="cpu")
-```
-
-#### run_with_hooks()
-
-```python
-def my_hook(activation, hook):
-    # Modify activation
-    activation[:, :, 0] = 0
-    return activation
-
-logits = model.run_with_hooks(
-    tokens,
-    fwd_hooks=[("blocks.5.hook_resid_post", my_hook)]
-)
-```
-
-#### generate()
-
-```python
-output = model.generate(
-    tokens,
-    max_new_tokens=50,
-    temperature=0.7,
-    top_k=40,
-    top_p=0.9,
-    freq_penalty=1.0,
-    use_past_kv_cache=True
-)
-```
-
-### Tokenization Methods
-
-```python
-# String to tokens
-tokens = model.to_tokens("Hello world")  # [1, seq_len]
-tokens = model.to_tokens("Hello", prepend_bos=False)
-
-# Tokens to string
-text = model.to_string(tokens)
-
-# Get string tokens (for debugging)
-str_tokens = model.to_str_tokens("Hello world")
-# ['<|endoftext|>', 'Hello', ' world']
-
-# Single token validation
-token_id = model.to_single_token(" Paris")  # Returns int or raises error
-```
-
-### Hook Management
-
-```python
-# Clear all hooks
-model.reset_hooks()
-
-# Add permanent hook
-model.add_hook("blocks.0.hook_resid_post", my_hook)
-
-# Remove specific hook
-model.remove_hook("blocks.0.hook_resid_post")
-```
-
----
-
-## ActivationCache
-
-Stores and provides access to all activations from a forward pass.
-
-### Accessing Activations
-
-```python
-logits, cache = model.run_with_cache(tokens)
-
-# By name and layer
-residual = cache["resid_post", 5]
-attention = cache["pattern", 3]
-mlp_out = cache["mlp_out", 7]
-
-# Full name string
-residual = cache["blocks.5.hook_resid_post"]
-```
-
-### Cache Keys
-
-| Key Pattern | Shape | Description |
-|-------------|-------|-------------|
-| `hook_embed` | [batch, pos, d_model] | Token embeddings |
-| `hook_pos_embed` | [batch, pos, d_model] | Positional embeddings |
-| `resid_pre, layer` | [batch, pos, d_model] | Residual before attention |
-| `resid_mid, layer` | [batch, pos, d_model] | Residual after attention |
-| `resid_post, layer` | [batch, pos, d_model] | Residual after MLP |
-| `attn_out, layer` | [batch, pos, d_model] | Attention output |
-| `mlp_out, layer` | [batch, pos, d_model] | MLP output |
-| `pattern, layer` | [batch, head, q_pos, k_pos] | Attention pattern (post-softmax) |
-| `attn_scores, layer` | [batch, head, q_pos, k_pos] | Attention scores (pre-softmax) |
-| `q, layer` | [batch, pos, head, d_head] | Query vectors |
-| `k, layer` | [batch, pos, head, d_head] | Key vectors |
-| `v, layer` | [batch, pos, head, d_head] | Value vectors |
-| `z, layer` | [batch, pos, head, d_head] | Attention output per head |
-
-### Analysis Methods
-
-#### decompose_resid()
-
-Decomposes residual stream into component contributions:
-
-```python
-components, labels = cache.decompose_resid(
-    layer=5,
-    return_labels=True,
-    mode="attn"  # or "mlp" or "full"
-)
-```
-
-#### accumulated_resid()
-
-Get accumulated residual at each layer (for Logit Lens):
-
-```python
-accumulated = cache.accumulated_resid(
-    layer=None,  # All layers
-    incl_mid=False,
-    apply_ln=True  # Apply final LayerNorm
-)
-```
-
-#### logit_attrs()
-
-Calculate logit attribution for components:
-
-```python
-attrs = cache.logit_attrs(
-    residual_stack,
-    tokens=target_tokens,
-    incorrect_tokens=incorrect_tokens
-)
-```
-
-#### stack_head_results()
-
-Stack attention head outputs:
-
-```python
-head_results = cache.stack_head_results(
-    layer=-1,  # All layers
-    pos_slice=None  # All positions
-)
-# Shape: [n_layers, n_heads, batch, pos, d_model]
-```
-
-### Utility Methods
-
-```python
-# Move cache to device
-cache = cache.to("cpu")
-
-# Remove batch dimension (for batch_size=1)
-cache = cache.remove_batch_dim()
-
-# Get all keys
-keys = cache.keys()
-
-# Iterate
-for name, activation in cache.items():
-    print(name, activation.shape)
-```
-
----
-
-## HookPoint
-
-The fundamental hook mechanism wrapping every activation.
-
-### Hook Function Signature
-
-```python
-def hook_fn(activation: torch.Tensor, hook: HookPoint) -> torch.Tensor:
-    """
-    Args:
-        activation: Current activation value
-        hook: The HookPoint object (has .name attribute)
-
-    Returns:
-        Modified activation (or None to keep original)
-    """
-    # Modify activation
-    return activation
-```
-
-### Common Hook Patterns
-
-```python
-# Zero ablation
-def zero_hook(act, hook):
-    act[:, :, :] = 0
-    return act
-
-# Mean ablation
-def mean_hook(act, hook):
-    act[:, :, :] = act.mean(dim=0, keepdim=True)
-    return act
-
-# Patch from cache
-def patch_hook(act, hook):
-    act[:, 5, :] = clean_cache[hook.name][:, 5, :]
-    return act
-
-# Add steering vector
-def steer_hook(act, hook):
-    act += 0.5 * steering_vector
-    return act
-```
-
----
-
-## Utility Functions
-
-### patching module
-
-```python
-from transformer_lens import patching
-
-# Generic activation patching
-results = patching.generic_activation_patch(
-    model=model,
-    corrupted_tokens=corrupted,
-    clean_cache=clean_cache,
-    patching_metric=metric_fn,
-    patch_setter=patch_fn,
-    activation_name="resid_post",
-    index_axis_names=("layer", "pos")
-)
-```
-
-### FactoredMatrix
-
-Efficient operations on factored weight matrices:
-
-```python
-from transformer_lens import FactoredMatrix
-
-# QK circuit
-QK = FactoredMatrix(model.W_Q[layer], model.W_K[layer].T)
-
-# OV circuit
-OV = FactoredMatrix(model.W_V[layer], model.W_O[layer])
-
-# Get full matrix
-full = QK.AB
-
-# SVD decomposition
-U, S, V = QK.svd()
-```
-
----
-
-## Configuration
-
-### HookedTransformerConfig
-
-Key configuration attributes:
-
-| Attribute | Description |
-|-----------|-------------|
-| `n_layers` | Number of transformer layers |
-| `n_heads` | Number of attention heads |
-| `d_model` | Model dimension |
-| `d_head` | Head dimension |
-| `d_mlp` | MLP hidden dimension |
-| `d_vocab` | Vocabulary size |
-| `n_ctx` | Maximum context length |
-| `act_fn` | Activation function name |
-| `normalization_type` | "LN" or "LNPre" |
-
-Access via:
-```python
-model.cfg.n_layers
-model.cfg.d_model
-```

package/bin/skills/transformer-lens/references/tutorials.md

@@ -1,339 +0,0 @@
-# TransformerLens Tutorials
-
-## Tutorial 1: Basic Activation Analysis
-
-### Goal
-Understand how to load models, cache activations, and inspect model internals.
-
-### Step-by-Step
-
-```python
-from transformer_lens import HookedTransformer
-import torch
-
-# 1. Load model
-model = HookedTransformer.from_pretrained("gpt2-small")
-print(f"Model has {model.cfg.n_layers} layers, {model.cfg.n_heads} heads")
-
-# 2. Tokenize input
-prompt = "The capital of France is"
-tokens = model.to_tokens(prompt)
-print(f"Tokens shape: {tokens.shape}")
-print(f"String tokens: {model.to_str_tokens(prompt)}")
-
-# 3. Run with cache
-logits, cache = model.run_with_cache(tokens)
-print(f"Logits shape: {logits.shape}")
-print(f"Cache keys: {len(cache.keys())}")
-
-# 4. Inspect activations
-for layer in range(model.cfg.n_layers):
-    resid = cache["resid_post", layer]
-    print(f"Layer {layer} residual norm: {resid.norm().item():.2f}")
-
-# 5. Look at attention patterns
-attn = cache["pattern", 0]  # Layer 0
-print(f"Attention shape: {attn.shape}")  # [batch, heads, q_pos, k_pos]
-
-# 6. Get top predictions
-probs = torch.softmax(logits[0, -1], dim=-1)
-top_tokens = probs.topk(5)
-for token_id, prob in zip(top_tokens.indices, top_tokens.values):
-    print(f"{model.to_string(token_id.unsqueeze(0))}: {prob.item():.3f}")
-```
-
----
-
-## Tutorial 2: Activation Patching
-
-### Goal
-Identify which activations causally affect model output.
-
-### Concept
-1. Run model on "clean" input, cache activations
-2. Run model on "corrupted" input
-3. Patch clean activations into corrupted run
-4. Measure effect on output
-
-### Step-by-Step
-
-```python
-from transformer_lens import HookedTransformer
-import torch
-
-model = HookedTransformer.from_pretrained("gpt2-small")
-
-# Define clean and corrupted prompts
-clean_prompt = "The Eiffel Tower is in the city of"
-corrupted_prompt = "The Colosseum is in the city of"
-
-clean_tokens = model.to_tokens(clean_prompt)
-corrupted_tokens = model.to_tokens(corrupted_prompt)
-
-# Get clean activations
-_, clean_cache = model.run_with_cache(clean_tokens)
-
-# Define metric
-paris_token = model.to_single_token(" Paris")
-rome_token = model.to_single_token(" Rome")
-
-def logit_diff(logits):
-    """Positive = model prefers Paris over Rome"""
-    return (logits[0, -1, paris_token] - logits[0, -1, rome_token]).item()
-
-# Baseline measurements
-clean_logits = model(clean_tokens)
-corrupted_logits = model(corrupted_tokens)
-print(f"Clean logit diff: {logit_diff(clean_logits):.3f}")
-print(f"Corrupted logit diff: {logit_diff(corrupted_logits):.3f}")
-
-# Patch each layer
-results = []
-for layer in range(model.cfg.n_layers):
-    def patch_hook(activation, hook, layer=layer):
-        activation[:] = clean_cache["resid_post", layer]
-        return activation
-
-    patched_logits = model.run_with_hooks(
-        corrupted_tokens,
-        fwd_hooks=[(f"blocks.{layer}.hook_resid_post", patch_hook)]
-    )
-    results.append(logit_diff(patched_logits))
-    print(f"Layer {layer}: {results[-1]:.3f}")
-
-# Find most important layer
-best_layer = max(range(len(results)), key=lambda i: results[i])
-print(f"\nMost important layer: {best_layer}")
-```
-
-### Position-Specific Patching
-
-```python
-import torch
-
-seq_len = clean_tokens.shape[1]
-results = torch.zeros(model.cfg.n_layers, seq_len)
-
-for layer in range(model.cfg.n_layers):
-    for pos in range(seq_len):
-        def patch_hook(activation, hook, layer=layer, pos=pos):
-            activation[:, pos, :] = clean_cache["resid_post", layer][:, pos, :]
-            return activation
-
-        patched_logits = model.run_with_hooks(
-            corrupted_tokens,
-            fwd_hooks=[(f"blocks.{layer}.hook_resid_post", patch_hook)]
-        )
-        results[layer, pos] = logit_diff(patched_logits)
-
-# Visualize as heatmap
-import matplotlib.pyplot as plt
-plt.figure(figsize=(12, 8))
-plt.imshow(results.numpy(), aspect='auto', cmap='RdBu')
-plt.xlabel('Position')
-plt.ylabel('Layer')
-plt.colorbar(label='Logit Difference')
-plt.title('Activation Patching Results')
-```
-
----
-
-## Tutorial 3: Direct Logit Attribution
-
-### Goal
-Identify which components (heads, neurons) contribute to specific predictions.
-
-### Step-by-Step
-
-```python
-from transformer_lens import HookedTransformer
-import torch
-
-model = HookedTransformer.from_pretrained("gpt2-small")
-
-prompt = "The capital of France is"
-tokens = model.to_tokens(prompt)
-logits, cache = model.run_with_cache(tokens)
-
-# Target token
-target_token = model.to_single_token(" Paris")
-
-# Get unembedding direction for target
-target_direction = model.W_U[:, target_token]  # [d_model]
-
-# Attribution per attention head
-head_contributions = torch.zeros(model.cfg.n_layers, model.cfg.n_heads)
-
-for layer in range(model.cfg.n_layers):
-    # Get per-head output at final position
-    z = cache["z", layer][0, -1]  # [n_heads, d_head]
-
-    for head in range(model.cfg.n_heads):
-        # Project through W_O to get contribution to residual
-        head_out = z[head] @ model.W_O[layer, head]  # [d_model]
-
-        # Dot with target direction
-        contribution = (head_out @ target_direction).item()
-        head_contributions[layer, head] = contribution
-
-# Find top contributing heads
-flat_idx = head_contributions.flatten().topk(10)
-print("Top 10 heads for predicting 'Paris':")
-for idx, val in zip(flat_idx.indices, flat_idx.values):
-    layer = idx.item() // model.cfg.n_heads
-    head = idx.item() % model.cfg.n_heads
-    print(f"  L{layer}H{head}: {val.item():.3f}")
-```
-
----
-
-## Tutorial 4: Induction Head Detection
-
-### Goal
-Find attention heads that implement the [A][B]...[A] → [B] pattern.
-
-### Step-by-Step
-
-```python
-from transformer_lens import HookedTransformer
-import torch
-
-model = HookedTransformer.from_pretrained("gpt2-small")
-
-# Create repeated sequence pattern
-# Pattern: [A][B][C][A] - model should attend from last A to B
-seq = torch.randint(1000, 5000, (1, 20))
-# Repeat first half
-seq[0, 10:] = seq[0, :10]
-
-_, cache = model.run_with_cache(seq)
-
-# For induction heads: position i should attend to position (i - seq_len/2 + 1)
-# At position 10 (second A), should attend to position 1 (first B)
-
-induction_scores = torch.zeros(model.cfg.n_layers, model.cfg.n_heads)
-
-for layer in range(model.cfg.n_layers):
-    pattern = cache["pattern", layer][0]  # [heads, q_pos, k_pos]
-
-    # Check attention from repeated positions to position after first occurrence
-    for offset in range(1, 10):
-        q_pos = 10 + offset  # Position in second half
-        k_pos = offset       # Should attend to corresponding position in first half
-
-        # Average attention to the "correct" position
-        induction_scores[layer] += pattern[:, q_pos, k_pos]
-
-    induction_scores[layer] /= 9  # Average over offsets
-
-# Find top induction heads
-print("Top induction heads:")
-for layer in range(model.cfg.n_layers):
-    for head in range(model.cfg.n_heads):
-        score = induction_scores[layer, head].item()
-        if score > 0.3:
-            print(f"  L{layer}H{head}: {score:.3f}")
-```
-
----
-
-## Tutorial 5: Logit Lens
-
-### Goal
-See what the model "believes" at each layer before final unembedding.
-
-### Step-by-Step
-
-```python
-from transformer_lens import HookedTransformer
-import torch
-
-model = HookedTransformer.from_pretrained("gpt2-small")
-
-prompt = "The quick brown fox jumps over the lazy"
-tokens = model.to_tokens(prompt)
-logits, cache = model.run_with_cache(tokens)
-
-# Get accumulated residual at each layer
-# Apply LayerNorm to match what unembedding sees
-accumulated = cache.accumulated_resid(layer=None, incl_mid=False, apply_ln=True)
-# Shape: [n_layers + 1, batch, pos, d_model]
-
-# Project to vocabulary
-layer_logits = accumulated @ model.W_U  # [n_layers + 1, batch, pos, d_vocab]
-
-# Look at predictions for final position
-print("Layer-by-layer predictions for final token:")
-for layer in range(model.cfg.n_layers + 1):
-    probs = torch.softmax(layer_logits[layer, 0, -1], dim=-1)
-    top_token = probs.argmax()
-    top_prob = probs[top_token].item()
-    print(f"Layer {layer}: {model.to_string(top_token.unsqueeze(0))!r} ({top_prob:.3f})")
-```
-
----
-
-## Tutorial 6: Steering with Activation Addition
-
-### Goal
-Add a steering vector to change model behavior.
-
-### Step-by-Step
-
-```python
-from transformer_lens import HookedTransformer
-import torch
-
-model = HookedTransformer.from_pretrained("gpt2-small")
-
-# Get activations for contrasting prompts
-positive_prompt = "I love this! It's absolutely wonderful and"
-negative_prompt = "I hate this! It's absolutely terrible and"
-
-_, pos_cache = model.run_with_cache(model.to_tokens(positive_prompt))
-_, neg_cache = model.run_with_cache(model.to_tokens(negative_prompt))
-
-# Compute steering vector (positive - negative direction)
-layer = 6
-steering_vector = (
-    pos_cache["resid_post", layer].mean(dim=1) -
-    neg_cache["resid_post", layer].mean(dim=1)
-)
-
-# Generate with steering
-test_prompt = "The movie was"
-test_tokens = model.to_tokens(test_prompt)
-
-def steer_hook(activation, hook):
-    activation += 2.0 * steering_vector
-    return activation
-
-# Without steering
-normal_output = model.generate(test_tokens, max_new_tokens=20)
-print(f"Normal: {model.to_string(normal_output[0])}")
-
-# With positive steering
-steered_output = model.generate(
-    test_tokens,
-    max_new_tokens=20,
-    fwd_hooks=[(f"blocks.{layer}.hook_resid_post", steer_hook)]
-)
-print(f"Steered: {model.to_string(steered_output[0])}")
-```
-
----
-
-## External Resources
-
-### Official Tutorials
-- [Main Demo](https://transformerlensorg.github.io/TransformerLens/generated/demos/Main_Demo.html)
-- [Exploratory Analysis](https://transformerlensorg.github.io/TransformerLens/generated/demos/Exploratory_Analysis_Demo.html)
-- [Activation Patching Demo](https://colab.research.google.com/github/TransformerLensOrg/TransformerLens/blob/main/demos/Activation_Patching_in_TL_Demo.ipynb)
-
-### ARENA Course
-Comprehensive 200+ hour curriculum: https://arena-foundation.github.io/ARENA/
-
-### Neel Nanda's Resources
-- [Getting Started in Mech Interp](https://www.neelnanda.io/mechanistic-interpretability/getting-started)
-- [Mech Interp Glossary](https://www.neelnanda.io/mechanistic-interpretability/glossary)
-- [YouTube Channel](https://www.youtube.com/@neelnanda)