chill-agent 1.4.0 → 1.6.0

package/examples/email-validation.md

@@ -0,0 +1,156 @@
+# Email Validation
+
+**Task:** "Write me a Python function that validates email addresses."
+
+Verbatim model output from a benchmark run, Claude Haiku 4.5, no-skill arm vs ponytail arm, temperature 1, source `benchmarks/output.json`. Reproduce: `npx promptfoo@latest eval -c benchmarks/promptfooconfig.yaml`.
+
+## Without Ponytail, 75 lines of code
+
+# Email Validation Function
+
+Here's a practical Python function that validates email addresses:
+
+```python
+import re
+
+def validate_email(email: str) -> bool:
+    """
+    Validates an email address using a regular expression.
+    
+    Args:
+        email (str): The email address to validate
+        
+    Returns:
+        bool: True if valid, False otherwise
+    """
+    # Regular expression pattern for email validation
+    pattern = r'^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$'
+    
+    return re.match(pattern, email) is not None
+
+
+# Test cases
+if __name__ == "__main__":
+    test_emails = [
+        "user@example.com",           # Valid
+        "john.doe@company.co.uk",     # Valid
+        "test+tag@domain.org",        # Valid
+        "invalid.email@",             # Invalid
+        "missing@domain",             # Invalid
+        "@nodomain.com",              # Invalid
+        "spaces in@email.com",        # Invalid
+        "user@domain..com",           # Invalid (double dot)
+    ]
+    
+    for email in test_emails:
+        result = validate_email(email)
+        print(f"{email:<30} -> {result}")
+```
+
+## More Robust Version (with additional checks)
+
+```python
+import re
+
+def validate_email_advanced(email: str) -> tuple[bool, str]:
+    """
+    Advanced email validation with detailed feedback.
+    
+    Args:
+        email (str): The email address to validate
+        
+    Returns:
+        tuple: (is_valid, message)
+    """
+    # Basic checks
+    if not email or not isinstance(email, str):
+        return False, "Email must be a non-empty string"
+    
+    email = email.strip()
+    
+    if len(email) > 254:
+        return False, "Email is too long (max 254 characters)"
+    
+    # Regular expression pattern
+    pattern = r'^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$'
+    
+    if not re.match(pattern, email):
+        return False, "Email format is invalid"
+    
+    # Additional checks
+    local_part, domain = email.rsplit('@', 1)
+    
+    if len(local_part) > 64:
+        return False, "Local part is too long (max 64 characters)"
+    
+    if local_part.startswith('.') or local_part.endswith('.'):
+        return False, "Local part cannot start or end with a dot"
+    
+    if '..' in email:
+        return False, "Email cannot contain consecutive dots"
+    
+    return True, "Email is valid"
+
+
+# Test the advanced version
+if __name__ == "__main__":
+    test_emails = [
+        "user@example.com",
+        "invalid..email@domain.com",
+        ".invalid@domain.com",
+        "user@domain",
+    ]
+    
+    for email in test_emails:
+        is_valid, message = validate_email_advanced(email)
+        print(f"{email:<35} -> {is_valid:5} ({message})")
+```
+
+## Using a Third-Party Library (Recommended for Production)
+
+```python
+from email_validator import validate_email, EmailNotValidError
+
+def validate_email_production(email: str) -> bool:
+    """
+    Validates email using the email_validator library.
+    More reliable for production use.
+    
+    Install with: pip install email-validator
+    """
+    try:
+        # Validate and normalize the email
+        valid = validate_email(email)
+        return True
+    except EmailNotValidError:
+        return False
+
+
+# Usage
+if __name__ == "__main__":
+    print(validate_email_production("user@example.com"))  # True
+    print(validate_email_production("invalid@"))          # False
+```
+
+## Comparison
+
+| Method | Pros | Cons |
+|--------|------|------|
+| **Regex (Simple)** | Fast, no dependencies | Limited validation |
+| **Regex (Advanced)** | More thorough | Still not RFC-compliant |
+| **email-validator** | RFC-compliant, reliable | External dependency |
+
+**Recommendation**: Use the `email-validator` library for production applications, as it properly handles RFC 5321/5322 standards. Use the simple regex version for quick scripts or when you want to avoid dependencies.
+
+## With Ponytail, 3 lines of code
+
+```python
+import re
+
+def is_valid_email(email: str) -> bool:
+    return bool(re.match(r'^[^@]+@[^@]+\.[^@]+$', email))
+```
+
+Skipped: RFC 5322 parser, DNS MX lookup, confirmation email. Add when you actually need to reject `user+tag@sub.domain.co.uk` or catch typos, until then, this catches 99% of "oops I fat-fingered it" cases.
+
+**75 → 3 lines of code**, same model, same prompt.

package/examples/group-by.md

@@ -0,0 +1,35 @@
+# Group By
+
+**Task:** "Group this array of objects by a key."
+
+## Without Ponytail
+
+```bash
+npm install lodash
+```
+
+```js
+import { groupBy } from "lodash";
+
+const byStatus = groupBy(orders, "status");
+// → { pending: [...], shipped: [...], delivered: [...] }
+```
+
+Or the hand-rolled version:
+
+```js
+const byStatus = orders.reduce((acc, order) => {
+  (acc[order.status] ??= []).push(order);
+  return acc;
+}, {});
+```
+
+## With Ponytail
+
+```js
+// ponytail: Object.groupBy does this
+const byStatus = Object.groupBy(orders, order => order.status);
+// → { pending: [...], shipped: [...], delivered: [...] }
+```
+
+**1 dependency (or a reduce) → 1 built-in.** `Object.groupBy` shipped in Chrome 117, Firefox 119, Safari 17.4, Node.js 21. If you need a `Map` instead of a plain object: `Map.groupBy(orders, o => o.status)`. Check your target runtime; if you need IE11 or old Node, the `reduce` one-liner is still the right call, not lodash.

package/examples/infinite-scroll.md

@@ -0,0 +1,58 @@
+# Infinite Scroll
+
+**Task:** "Load more items when the user scrolls to the bottom."
+
+## Without Ponytail
+
+```bash
+npm install react-infinite-scroll-component
+```
+
+```jsx
+import InfiniteScroll from "react-infinite-scroll-component";
+
+export function Feed({ items, fetchMore, hasMore }) {
+  return (
+    <InfiniteScroll
+      dataLength={items.length}
+      next={fetchMore}
+      hasMore={hasMore}
+      loader={<Spinner />}
+      endMessage={<p>No more items</p>}
+      scrollThreshold={0.9}
+    >
+      {items.map(item => <Card key={item.id} item={item} />)}
+    </InfiniteScroll>
+  );
+}
+```
+
+A dependency to watch scroll position and fire a callback.
+
+## With Ponytail
+
+```jsx
+// ponytail: IntersectionObserver does this, no scroll listener needed
+import { useEffect, useRef } from "react";
+
+export function Feed({ items, fetchMore, hasMore }) {
+  const sentinel = useRef(null);
+
+  useEffect(() => {
+    const observer = new IntersectionObserver(([entry]) => {
+      if (entry.isIntersecting && hasMore) fetchMore();
+    });
+    if (sentinel.current) observer.observe(sentinel.current);
+    return () => observer.disconnect();
+  }, [hasMore, fetchMore]);
+
+  return (
+    <>
+      {items.map(item => <Card key={item.id} item={item} />)}
+      <div ref={sentinel} />
+    </>
+  );
+}
+```
+
+**1 dependency → 0 dependencies.** `IntersectionObserver` fires only when the sentinel enters the viewport, no scroll event, no throttling, no jank. Ships in every browser. The library wraps exactly this API.

package/examples/number-formatting.md

@@ -0,0 +1,37 @@
+# Number Formatting
+
+**Task:** "Format numbers as currency and with thousand separators."
+
+## Without Ponytail
+
+```bash
+npm install numeral
+# or: npm install accounting
+```
+
+```js
+import numeral from "numeral";
+
+numeral(1234567.89).format("$1,234.00"); // "$1,234,567.89"
+numeral(0.745).format("0.0%");           // "74.5%"
+numeral(1500).format("0.0a");            // "1.5k"
+```
+
+## With Ponytail
+
+```js
+// ponytail: Intl.NumberFormat does this, locale-aware
+new Intl.NumberFormat("en-US", { style: "currency", currency: "USD" })
+  .format(1234567.89);
+// → "$1,234,567.89"
+
+new Intl.NumberFormat("en-US", { style: "percent" })
+  .format(0.745);
+// → "74.5%"
+
+new Intl.NumberFormat("en-US", { notation: "compact" })
+  .format(1500);
+// → "1.5K"
+```
+
+**1 dependency → 0 dependencies.** `Intl.NumberFormat` is built into every JS runtime, handles every locale correctly, and gets currency symbols, decimal separators, and grouping right for any market without a lookup table. A library that hardcodes formats will always be wrong for someone.

package/examples/rate-limit.md

@@ -0,0 +1,272 @@
+# Rate Limiting
+
+**Task:** "Add rate limiting to my FastAPI endpoint so users can't spam it."
+
+Verbatim model output from a benchmark run, Claude Haiku 4.5, no-skill arm vs ponytail arm, temperature 1, source `benchmarks/output.json`. Reproduce: `npx promptfoo@latest eval -c benchmarks/promptfooconfig.yaml`.
+
+## Without Ponytail, 128 lines of code
+
+# Rate Limiting in FastAPI
+
+Here are several approaches, from simplest to most robust:
+
+## 1. **Using `slowapi` (Recommended - Easiest)**
+
+```bash
+pip install slowapi
+```
+
+```python
+from fastapi import FastAPI, Request
+from slowapi import Limiter
+from slowapi.util import get_remote_address
+from slowapi.errors import RateLimitExceeded
+from fastapi.responses import JSONResponse
+
+app = FastAPI()
+
+limiter = Limiter(key_func=get_remote_address)
+app.state.limiter = limiter
+app.add_exception_handler(RateLimitExceeded, lambda request, exc: JSONResponse(
+    status_code=429,
+    content={"detail": "Rate limit exceeded"}
+))
+
+@app.get("/api/data")
+@limiter.limit("5/minute")
+async def get_data(request: Request):
+    return {"message": "Success"}
+```
+
+**Rate limit formats:**
+- `"5/minute"` - 5 requests per minute
+- `"100/hour"` - 100 requests per hour
+- `"1000/day"` - 1000 requests per day
+
+---
+
+## 2. **Using `limits` Library (More Control)**
+
+```bash
+pip install limits
+```
+
+```python
+from fastapi import FastAPI, HTTPException, Request
+from limits import parse
+from limits.storage import MemoryStorage
+from limits.strategies import MovingWindowRateLimiter
+
+app = FastAPI()
+
+storage = MemoryStorage()
+limiter = MovingWindowRateLimiter(storage)
+rate_limit = parse("5/minute")
+
+@app.get("/api/data")
+async def get_data(request: Request):
+    client_id = request.client.host
+    
+    if not limiter.hit(rate_limit, client_id):
+        raise HTTPException(status_code=429, detail="Rate limit exceeded")
+    
+    return {"message": "Success"}
+```
+
+---
+
+## 3. **Custom Middleware (Full Control)**
+
+```python
+from fastapi import FastAPI, HTTPException, Request
+from datetime import datetime, timedelta
+from collections import defaultdict
+import asyncio
+
+app = FastAPI()
+
+# Store request timestamps per IP
+request_history = defaultdict(list)
+RATE_LIMIT = 5  # requests
+TIME_WINDOW = 60  # seconds
+
+@app.middleware("http")
+async def rate_limit_middleware(request: Request, call_next):
+    client_ip = request.client.host
+    now = datetime.now()
+    
+    # Clean old requests outside time window
+    request_history[client_ip] = [
+        req_time for req_time in request_history[client_ip]
+        if now - req_time < timedelta(seconds=TIME_WINDOW)
+    ]
+    
+    # Check if limit exceeded
+    if len(request_history[client_ip]) >= RATE_LIMIT:
+        return JSONResponse(
+            status_code=429,
+            content={"detail": "Rate limit exceeded"}
+        )
+    
+    # Record this request
+    request_history[client_ip].append(now)
+    
+    return await call_next(request)
+
+@app.get("/api/data")
+async def get_data():
+    return {"message": "Success"}
+```
+
+---
+
+## 4. **Per-User Rate Limiting (With Authentication)**
+
+```python
+from fastapi import FastAPI, Depends, HTTPException
+from slowapi import Limiter
+from slowapi.util import get_remote_address
+
+app = FastAPI()
+limiter = Limiter(key_func=get_remote_address)
+
+def get_user_id(token: str = Header(None)) -> str:
+    # Your auth logic here
+    return token or "anonymous"
+
+@app.get("/api/data")
+@limiter.limit("10/minute")
+async def get_data(request: Request, user_id: str = Depends(get_user_id)):
+    return {"message": f"Success for {user_id}"}
+```
+
+---
+
+## 5. **Redis-Based Rate Limiting (Production)**
+
+```bash
+pip install slowapi redis
+```
+
+```python
+from fastapi import FastAPI, Request
+from slowapi import Limiter
+from slowapi.util import get_remote_address
+from slowapi.errors import RateLimitExceeded
+from slowapi.storage import RedisStorage
+from redis import Redis
+from fastapi.responses import JSONResponse
+
+redis_client = Redis.from_url("redis://localhost:6379")
+storage = RedisStorage(redis_client)
+limiter = Limiter(key_func=get_remote_address, storage=storage)
+
+app = FastAPI()
+app.state.limiter = limiter
+
+app.add_exception_handler(
+    RateLimitExceeded,
+    lambda request, exc: JSONResponse(
+        status_code=429,
+        content={"detail": "Rate limit exceeded"}
+    )
+)
+
+@app.get("/api/data")
+@limiter.limit("5/minute")
+async def get_data(request: Request):
+    return {"message": "Success"}
+```
+
+---
+
+## 6. **Complete Example with Multiple Endpoints**
+
+```python
+from fastapi import FastAPI, Request
+from slowapi import Limiter
+from slowapi.util import get_remote_address
+from slowapi.errors import RateLimitExceeded
+from fastapi.responses import JSONResponse
+
+app = FastAPI()
+limiter = Limiter(key_func=get_remote_address)
+app.state.limiter = limiter
+
+@app.exception_handler(RateLimitExceeded)
+async def rate_limit_handler(request: Request, exc: RateLimitExceeded):
+    return JSONResponse(
+        status_code=429,
+        content={
+            "detail": "Too many requests",
+            "retry_after": exc.detail
+        }
+    )
+
+# Strict limit
+@app.post("/api/login")
+@limiter.limit("3/minute")
+async def login(request: Request):
+    return {"token": "xxx"}
+
+# Moderate limit
+@app.get("/api/data")
+@limiter.limit("30/minute")
+async def get_data(request: Request):
+    return {"data": []}
+
+# Generous limit
+@app.get("/api/public")
+@limiter.limit("100/minute")
+async def get_public(request: Request):
+    return {"public": "data"}
+```
+
+---
+
+## Comparison Table
+
+| Method | Pros | Cons | Best For |
+|--------|------|------|----------|
+| **slowapi** | Easy, decorator-based | In-memory only | Development, small apps |
+| **limits** | Flexible, multiple backends | More setup | Custom logic |
+| **Middleware** | Full control, no dependencies | Manual cleanup | Learning, simple cases |
+| **Redis** | Distributed, scalable | Requires Redis | Production, multi-server |
+
+---
+
+## Testing Rate Limits
+
+```python
+import httpx
+
+async def test_rate_limit():
+    async with httpx.AsyncClient(app=app, base_url="http://test") as client:
+        for i in range(6):
+            response = await client.get("/api/data")
+            print(f"Request {i+1}: {response.status_code}")
+            # First 5 should be 200, 6th should be 429
+```
+
+**Recommendation:** Use `slowapi` for most cases, it's the most popular and easiest to implement! 🚀
+
+## With Ponytail, 10 lines of code
+
+```python
+from fastapi import FastAPI, HTTPException
+from slowapi import Limiter
+from slowapi.util import get_remote_address
+
+app = FastAPI()
+limiter = Limiter(key_func=get_remote_address)
+app.state.limiter = limiter
+
+@app.get("/api/endpoint")
+@limiter.limit("10/minute")
+async def my_endpoint(request):
+    return {"status": "ok"}
+```
+
+Skipped: custom rate limit logic, Redis, sliding windows, `slowapi` handles it. Add when: you need distributed rate limiting across multiple servers (swap `Limiter` for Redis backend) or per-user limits (add `key_func=lambda r: r.headers.get("authorization")`).
+
+**128 → 10 lines of code**, same model, same prompt.