@rubix0270/arboris 1.0.1 → 1.0.3

package/prisma/skills/redis-patterns/SKILL.md

@@ -1,404 +0,0 @@
----
-name: redis-patterns
-description: Redis data structure patterns, caching strategies, distributed locks, rate limiting, pub/sub, and connection management for production applications.
-metadata:
-  origin: ECC
----
-
-# Redis Patterns
-
-Quick reference for Redis best practices across common backend use cases.
-
-## How It Works
-
-Redis is an in-memory data structure store that supports strings, hashes, lists, sets, sorted sets, streams, and more. Individual Redis commands are atomic on a single instance; multi-step workflows require Lua scripts, MULTI/EXEC transactions, or explicit synchronization to stay atomic. Data is optionally persisted via RDB snapshots or AOF logs. Clients communicate over TCP using the RESP protocol; connection pools are essential to avoid per-request handshake overhead.
-
-## When to Activate
-
-- Adding caching to an application
-- Implementing rate limiting or throttling
-- Building distributed locks or coordination
-- Setting up session or token storage
-- Using Pub/Sub or Redis Streams for messaging
-- Configuring Redis in production (pooling, eviction, clustering)
-
-## Data Structure Cheat Sheet
-
-| Use Case | Structure | Example Key |
-|----------|-----------|-------------|
-| Simple cache | String | `product:123` |
-| User session | Hash | `session:abc` |
-| Leaderboard | Sorted Set | `scores:weekly` |
-| Unique visitors | Set | `visitors:2024-01-01` |
-| Activity feed | List | `feed:user:456` |
-| Event stream | Stream | `events:orders` |
-| Counters / rate limits | String (INCR) | `ratelimit:user:123` |
-| Bloom filter / HLL | HyperLogLog | `hll:pageviews` |
-
-## Core Patterns
-
-### Cache-Aside (Lazy Loading)
-
-```python
-import redis
-import json
-
-r = redis.Redis(host='localhost', port=6379, decode_responses=True)
-
-def get_product(product_id: int):
-    cache_key = f"product:{product_id}"
-    cached = r.get(cache_key)
-
-    if cached:
-        return json.loads(cached)
-
-    product = db.query("SELECT * FROM products WHERE id = %s", product_id)
-    r.setex(cache_key, 3600, json.dumps(product))  # TTL: 1 hour
-    return product
-```
-
-### Write-Through Cache
-
-```python
-def update_product(product_id: int, data: dict):
-    # Write to DB first
-    db.execute("UPDATE products SET ... WHERE id = %s", product_id)
-
-    # Immediately update cache
-    cache_key = f"product:{product_id}"
-    r.setex(cache_key, 3600, json.dumps(data))
-```
-
-### Cache Invalidation
-
-```python
-# Tag-based invalidation — group related keys under a set
-def cache_product(product_id: int, category_id: int, data: dict):
-    key = f"product:{product_id}"
-    tag = f"tag:category:{category_id}"
-    pipe = r.pipeline(transaction=True)
-    pipe.setex(key, 3600, json.dumps(data))
-    pipe.sadd(tag, key)
-    pipe.expire(tag, 3600)
-    pipe.execute()
-
-def invalidate_category(category_id: int):
-    tag = f"tag:category:{category_id}"
-    keys = r.smembers(tag)
-    if keys:
-        r.delete(*keys)
-    r.delete(tag)
-```
-
-### Session Storage
-
-```python
-import time
-import uuid
-
-def create_session(user_id: int, ttl: int = 86400) -> str:
-    session_id = str(uuid.uuid4())
-    key = f"session:{session_id}"
-    pipe = r.pipeline(transaction=True)
-    pipe.hset(key, mapping={
-        "user_id": user_id,
-        "created_at": int(time.time()),
-    })
-    pipe.expire(key, ttl)
-    pipe.execute()
-    return session_id
-
-def get_session(session_id: str) -> dict | None:
-    data = r.hgetall(f"session:{session_id}")
-    return data if data else None
-
-def delete_session(session_id: str):
-    r.delete(f"session:{session_id}")
-```
-
-## Rate Limiting
-
-### Fixed Window (Simple)
-
-```python
-def is_rate_limited(user_id: int, limit: int = 100, window: int = 60) -> bool:
-    key = f"ratelimit:{user_id}:{int(time.time()) // window}"
-    pipe = r.pipeline(transaction=True)
-    pipe.incr(key)
-    pipe.expire(key, window)
-    count, _ = pipe.execute()
-    return count > limit
-```
-
-### Sliding Window (Lua — Atomic)
-
-```lua
--- sliding_window.lua
-local key = KEYS[1]
-local now = tonumber(ARGV[1])
-local window = tonumber(ARGV[2])
-local limit = tonumber(ARGV[3])
-
-redis.call('ZREMRANGEBYSCORE', key, 0, now - window)
-local count = redis.call('ZCARD', key)
-
-if count < limit then
-    -- Use unique member (now + sequence) to avoid collisions within the same millisecond
-    local seq_key = key .. ':seq'
-    local seq = redis.call('INCR', seq_key)
-    redis.call('EXPIRE', seq_key, math.ceil(window / 1000))
-    redis.call('ZADD', key, now, now .. '-' .. seq)
-    redis.call('EXPIRE', key, math.ceil(window / 1000))
-    return 1
-end
-return 0
-```
-
-```python
-sliding_window = r.register_script(open('sliding_window.lua').read())
-
-def allow_request(user_id: int) -> bool:
-    key = f"ratelimit:sliding:{user_id}"
-    now = int(time.time() * 1000)
-    return bool(sliding_window(keys=[key], args=[now, 60000, 100]))
-```
-
-## Distributed Locks
-
-### Distributed Lock (Single Node — SET NX PX)
-
-```python
-import uuid
-
-def acquire_lock(resource: str, ttl_ms: int = 5000) -> str | None:
-    lock_key = f"lock:{resource}"
-    token = str(uuid.uuid4())
-    acquired = r.set(lock_key, token, px=ttl_ms, nx=True)
-    return token if acquired else None
-
-def release_lock(resource: str, token: str) -> bool:
-    release_script = """
-    if redis.call('get', KEYS[1]) == ARGV[1] then
-        return redis.call('del', KEYS[1])
-    else
-        return 0
-    end
-    """
-    result = r.eval(release_script, 1, f"lock:{resource}", token)
-    return bool(result)
-
-# Usage
-token = acquire_lock("order:payment:123")
-if token:
-    try:
-        process_payment()
-    finally:
-        release_lock("order:payment:123", token)
-```
-
-> For multi-node setups use the `redlock-py` library which implements the full Redlock algorithm.
-
-## Pub/Sub & Streams
-
-### Pub/Sub (Fire-and-Forget)
-
-```python
-# Publisher
-def publish_event(channel: str, payload: dict):
-    r.publish(channel, json.dumps(payload))
-
-# Subscriber (blocking — run in separate thread/process)
-def subscribe_events(channel: str):
-    pubsub = r.pubsub()
-    pubsub.subscribe(channel)
-    for message in pubsub.listen():
-        if message['type'] == 'message':
-            handle(json.loads(message['data']))
-```
-
-### Redis Streams (Durable Queue)
-
-```python
-# Producer
-def emit(stream: str, event: dict):
-    r.xadd(stream, event, maxlen=10000)  # Cap stream length
-
-# Consumer group — guarantees at-least-once delivery
-try:
-    r.xgroup_create('events:orders', 'processor', id='0', mkstream=True)
-except Exception:
-    pass  # Group already exists
-
-def consume(stream: str, group: str, consumer: str):
-    while True:
-        messages = r.xreadgroup(group, consumer, {stream: '>'}, count=10, block=2000)
-        for _, entries in (messages or []):
-            for msg_id, data in entries:
-                process(data)
-                r.xack(stream, group, msg_id)
-```
-
-> Prefer **Streams** over Pub/Sub when you need delivery guarantees, consumer groups, or replay.
-
-## Key Design
-
-### Naming Conventions
-
-```
-# Pattern: resource:id:field
-user:123:profile
-order:456:status
-cache:product:789
-
-# Pattern: namespace:resource:id
-myapp:session:abc123
-myapp:ratelimit:user:123
-
-# Pattern: resource:date (time-bound keys)
-stats:pageviews:2024-01-01
-```
-
-### TTL Strategy
-
-| Data Type | Suggested TTL |
-|-----------|--------------|
-| User session | 24h (`86400`) |
-| API response cache | 5–15 min |
-| Rate limit window | Match window size |
-| Short-lived tokens | 5–10 min |
-| Leaderboard | 1h–24h |
-| Static/reference data | 1h–1 week |
-
-Always set a TTL. Keys without TTL accumulate indefinitely and cause memory pressure.
-
-## Connection Management
-
-### Connection Pooling
-
-```python
-from redis import ConnectionPool, Redis
-
-pool = ConnectionPool(
-    host='localhost',
-    port=6379,
-    db=0,
-    max_connections=20,
-    decode_responses=True,
-    socket_connect_timeout=2,
-    socket_timeout=2,
-)
-
-r = Redis(connection_pool=pool)
-```
-
-### Cluster Mode
-
-```python
-from redis.cluster import RedisCluster
-
-r = RedisCluster(
-    startup_nodes=[{"host": "redis-1", "port": 6379}],
-    decode_responses=True,
-    skip_full_coverage_check=True,
-)
-```
-
-### Sentinel (High Availability)
-
-```python
-from redis.sentinel import Sentinel
-
-sentinel = Sentinel(
-    [('sentinel-1', 26379), ('sentinel-2', 26379)],
-    socket_timeout=0.5,
-)
-master = sentinel.master_for('mymaster', decode_responses=True)
-replica = sentinel.slave_for('mymaster', decode_responses=True)
-```
-
-## Eviction Policies
-
-| Policy | Behavior | Best For |
-|--------|----------|----------|
-| `noeviction` | Error on write when full | Queues / critical data |
-| `allkeys-lru` | Evict least recently used | General cache |
-| `volatile-lru` | LRU only among keys with TTL | Mixed data store |
-| `allkeys-lfu` | Evict least frequently used | Skewed access patterns |
-| `volatile-ttl` | Evict soonest-to-expire | Prioritize long-lived data |
-
-Set via `redis.conf`: `maxmemory-policy allkeys-lru`
-
-## Anti-Patterns
-
-| Anti-Pattern | Problem | Fix |
-|---|---|---|
-| Keys with no TTL | Memory grows unbounded | Always set TTL |
-| `KEYS *` in production | Blocks the server (O(N)) | Use `SCAN` cursor |
-| Storing large blobs (>100KB) | Slow serialization, memory pressure | Store reference + fetch from object store |
-| Single Redis for everything | No isolation between cache & queue | Use separate DBs or instances |
-| Ignoring connection pool limits | Connection exhaustion under load | Size pool to workload |
-| Not handling cache miss stampede | Thundering herd on cold start | Use locks or probabilistic early expiry |
-| `FLUSHALL` without thought | Wipes entire instance | Scope deletes by key pattern |
-
-### Cache Miss Stampede Prevention
-
-```python
-import threading
-
-_locks: dict[str, threading.Lock] = {}
-_locks_mutex = threading.Lock()
-
-def get_with_lock(key: str, fetch_fn, ttl: int = 300):
-    cached = r.get(key)
-    if cached:
-        return json.loads(cached)
-
-    with _locks_mutex:
-        if key not in _locks:
-            _locks[key] = threading.Lock()
-        lock = _locks[key]
-    with lock:
-        cached = r.get(key)  # Re-check after acquiring lock
-        if cached:
-            return json.loads(cached)
-        value = fetch_fn()
-        r.setex(key, ttl, json.dumps(value))
-        return value
-```
-
-> Note: for multi-process deployments, replace the in-process lock with `acquire_lock`/`release_lock` from the Distributed Locks section above.
-
-## Examples
-
-**Add caching to a Django/Flask API endpoint:**
-Use cache-aside with `setex` and a 5-minute TTL on the response. Key on the request parameters.
-
-**Rate-limit an API by user:**
-Use fixed-window with `pipeline(transaction=True)` for low-traffic endpoints; use sliding-window Lua for accurate per-user throttling.
-
-**Coordinate a background job across workers:**
-Use `acquire_lock` with a TTL that exceeds the expected job duration. Always release in a `finally` block.
-
-**Fan-out notifications to multiple subscribers:**
-Use Pub/Sub for fire-and-forget. Switch to Streams if you need guaranteed delivery or replay for late consumers.
-
-## Quick Reference
-
-| Pattern | When to Use |
-|---------|-------------|
-| Cache-aside | Read-heavy, tolerate slight staleness |
-| Write-through | Strong consistency required |
-| Distributed lock | Prevent concurrent access to a resource |
-| Sliding window rate limit | Accurate per-user throttling |
-| Redis Streams | Durable event queue with consumer groups |
-| Pub/Sub | Broadcast with no delivery guarantees needed |
-| Sorted Set leaderboard | Ranked scoring, pagination |
-| HyperLogLog | Approximate unique count at low memory |
-
-## Related
-
-- Skill: `postgres-patterns` — relational data patterns
-- Skill: `backend-patterns` — API and service layer patterns
-- Skill: `database-migrations` — schema versioning
-- Skill: `django-patterns` — Django cache framework integration
-- Agent: `database-reviewer` — full database review workflow

package/prisma/skills/regex-vs-llm-structured-text/SKILL.md

@@ -1,221 +0,0 @@
----
-name: regex-vs-llm-structured-text
-description: Decision framework for choosing between regex and LLM when parsing structured text — start with regex, add LLM only for low-confidence edge cases.
-metadata:
-  origin: ECC
----
-
-# Regex vs LLM for Structured Text Parsing
-
-A practical decision framework for parsing structured text (quizzes, forms, invoices, documents). The key insight: regex handles 95-98% of cases cheaply and deterministically. Reserve expensive LLM calls for the remaining edge cases.
-
-## When to Activate
-
-- Parsing structured text with repeating patterns (questions, forms, tables)
-- Deciding between regex and LLM for text extraction
-- Building hybrid pipelines that combine both approaches
-- Optimizing cost/accuracy tradeoffs in text processing
-
-## Decision Framework
-
-```
-Is the text format consistent and repeating?
-├── Yes (>90% follows a pattern) → Start with Regex
-│   ├── Regex handles 95%+ → Done, no LLM needed
-│   └── Regex handles <95% → Add LLM for edge cases only
-└── No (free-form, highly variable) → Use LLM directly
-```
-
-## Architecture Pattern
-
-```
-Source Text
-    │
-    ▼
-[Regex Parser] ─── Extracts structure (95-98% accuracy)
-    │
-    ▼
-[Text Cleaner] ─── Removes noise (markers, page numbers, artifacts)
-    │
-    ▼
-[Confidence Scorer] ─── Flags low-confidence extractions
-    │
-    ├── High confidence (≥0.95) → Direct output
-    │
-    └── Low confidence (<0.95) → [LLM Validator] → Output
-```
-
-## Implementation
-
-### 1. Regex Parser (Handles the Majority)
-
-```python
-import re
-from dataclasses import dataclass
-
-@dataclass(frozen=True)
-class ParsedItem:
-    id: str
-    text: str
-    choices: tuple[str, ...]
-    answer: str
-    confidence: float = 1.0
-
-def parse_structured_text(content: str) -> list[ParsedItem]:
-    """Parse structured text using regex patterns."""
-    pattern = re.compile(
-        r"(?P<id>\d+)\.\s*(?P<text>.+?)\n"
-        r"(?P<choices>(?:[A-D]\..+?\n)+)"
-        r"Answer:\s*(?P<answer>[A-D])",
-        re.MULTILINE | re.DOTALL,
-    )
-    items = []
-    for match in pattern.finditer(content):
-        choices = tuple(
-            c.strip() for c in re.findall(r"[A-D]\.\s*(.+)", match.group("choices"))
-        )
-        items.append(ParsedItem(
-            id=match.group("id"),
-            text=match.group("text").strip(),
-            choices=choices,
-            answer=match.group("answer"),
-        ))
-    return items
-```
-
-### 2. Confidence Scoring
-
-Flag items that may need LLM review:
-
-```python
-@dataclass(frozen=True)
-class ConfidenceFlag:
-    item_id: str
-    score: float
-    reasons: tuple[str, ...]
-
-def score_confidence(item: ParsedItem) -> ConfidenceFlag:
-    """Score extraction confidence and flag issues."""
-    reasons = []
-    score = 1.0
-
-    if len(item.choices) < 3:
-        reasons.append("few_choices")
-        score -= 0.3
-
-    if not item.answer:
-        reasons.append("missing_answer")
-        score -= 0.5
-
-    if len(item.text) < 10:
-        reasons.append("short_text")
-        score -= 0.2
-
-    return ConfidenceFlag(
-        item_id=item.id,
-        score=max(0.0, score),
-        reasons=tuple(reasons),
-    )
-
-def identify_low_confidence(
-    items: list[ParsedItem],
-    threshold: float = 0.95,
-) -> list[ConfidenceFlag]:
-    """Return items below confidence threshold."""
-    flags = [score_confidence(item) for item in items]
-    return [f for f in flags if f.score < threshold]
-```
-
-### 3. LLM Validator (Edge Cases Only)
-
-```python
-def validate_with_llm(
-    item: ParsedItem,
-    original_text: str,
-    client,
-) -> ParsedItem:
-    """Use LLM to fix low-confidence extractions."""
-    response = client.messages.create(
-        model="claude-haiku-4-5-20251001",  # Cheapest model for validation
-        max_tokens=500,
-        messages=[{
-            "role": "user",
-            "content": (
-                f"Extract the question, choices, and answer from this text.\n\n"
-                f"Text: {original_text}\n\n"
-                f"Current extraction: {item}\n\n"
-                f"Return corrected JSON if needed, or 'CORRECT' if accurate."
-            ),
-        }],
-    )
-    # Parse LLM response and return corrected item...
-    return corrected_item
-```
-
-### 4. Hybrid Pipeline
-
-```python
-def process_document(
-    content: str,
-    *,
-    llm_client=None,
-    confidence_threshold: float = 0.95,
-) -> list[ParsedItem]:
-    """Full pipeline: regex -> confidence check -> LLM for edge cases."""
-    # Step 1: Regex extraction (handles 95-98%)
-    items = parse_structured_text(content)
-
-    # Step 2: Confidence scoring
-    low_confidence = identify_low_confidence(items, confidence_threshold)
-
-    if not low_confidence or llm_client is None:
-        return items
-
-    # Step 3: LLM validation (only for flagged items)
-    low_conf_ids = {f.item_id for f in low_confidence}
-    result = []
-    for item in items:
-        if item.id in low_conf_ids:
-            result.append(validate_with_llm(item, content, llm_client))
-        else:
-            result.append(item)
-
-    return result
-```
-
-## Real-World Metrics
-
-From a production quiz parsing pipeline (410 items):
-
-| Metric | Value |
-|--------|-------|
-| Regex success rate | 98.0% |
-| Low confidence items | 8 (2.0%) |
-| LLM calls needed | ~5 |
-| Cost savings vs all-LLM | ~95% |
-| Test coverage | 93% |
-
-## Best Practices
-
-- **Start with regex** — even imperfect regex gives you a baseline to improve
-- **Use confidence scoring** to programmatically identify what needs LLM help
-- **Use the cheapest LLM** for validation (Haiku-class models are sufficient)
-- **Never mutate** parsed items — return new instances from cleaning/validation steps
-- **TDD works well** for parsers — write tests for known patterns first, then edge cases
-- **Log metrics** (regex success rate, LLM call count) to track pipeline health
-
-## Anti-Patterns to Avoid
-
-- Sending all text to an LLM when regex handles 95%+ of cases (expensive and slow)
-- Using regex for free-form, highly variable text (LLM is better here)
-- Skipping confidence scoring and hoping regex "just works"
-- Mutating parsed objects during cleaning/validation steps
-- Not testing edge cases (malformed input, missing fields, encoding issues)
-
-## When to Use
-
-- Quiz/exam question parsing
-- Form data extraction
-- Invoice/receipt processing
-- Document structure parsing (headers, sections, tables)
-- Any structured text with repeating patterns where cost matters

package/prisma/skills/remotion-video-creation/SKILL.md

@@ -1,43 +0,0 @@
----
-name: remotion-video-creation
-description: Best practices for Remotion - Video creation in React. 29 domain-specific rules covering 3D, animations, audio, captions, charts, transitions, and more.
-metadata:
-  tags: remotion, video, react, animation, composition, three.js, lottie
----
-
-## When to use
-
-Use this skills whenever you are dealing with Remotion code to obtain the domain-specific knowledge.
-
-## How to use
-
-Read individual rule files for detailed explanations and code examples:
-
-- [rules/3d.md](rules/3d.md) - 3D content in Remotion using Three.js and React Three Fiber
-- [rules/animations.md](rules/animations.md) - Fundamental animation skills for Remotion
-- [rules/assets.md](rules/assets.md) - Importing images, videos, audio, and fonts into Remotion
-- [rules/audio.md](rules/audio.md) - Using audio and sound in Remotion - importing, trimming, volume, speed, pitch
-- [rules/calculate-metadata.md](rules/calculate-metadata.md) - Dynamically set composition duration, dimensions, and props
-- [rules/can-decode.md](rules/can-decode.md) - Check if a video can be decoded by the browser using Mediabunny
-- [rules/charts.md](rules/charts.md) - Chart and data visualization patterns for Remotion
-- [rules/compositions.md](rules/compositions.md) - Defining compositions, stills, folders, default props and dynamic metadata
-- [rules/display-captions.md](rules/display-captions.md) - Displaying captions in Remotion with TikTok-style pages and word highlighting
-- [rules/extract-frames.md](rules/extract-frames.md) - Extract frames from videos at specific timestamps using Mediabunny
-- [rules/fonts.md](rules/fonts.md) - Loading Google Fonts and local fonts in Remotion
-- [rules/get-audio-duration.md](rules/get-audio-duration.md) - Getting the duration of an audio file in seconds with Mediabunny
-- [rules/get-video-dimensions.md](rules/get-video-dimensions.md) - Getting the width and height of a video file with Mediabunny
-- [rules/get-video-duration.md](rules/get-video-duration.md) - Getting the duration of a video file in seconds with Mediabunny
-- [rules/gifs.md](rules/gifs.md) - Displaying GIFs synchronized with Remotion's timeline
-- [rules/images.md](rules/images.md) - Embedding images in Remotion using the Img component
-- [rules/import-srt-captions.md](rules/import-srt-captions.md) - Importing .srt subtitle files into Remotion using @remotion/captions
-- [rules/lottie.md](rules/lottie.md) - Embedding Lottie animations in Remotion
-- [rules/measuring-dom-nodes.md](rules/measuring-dom-nodes.md) - Measuring DOM element dimensions in Remotion
-- [rules/measuring-text.md](rules/measuring-text.md) - Measuring text dimensions, fitting text to containers, and checking overflow
-- [rules/sequencing.md](rules/sequencing.md) - Sequencing patterns for Remotion - delay, trim, limit duration of items
-- [rules/tailwind.md](rules/tailwind.md) - Using TailwindCSS in Remotion
-- [rules/text-animations.md](rules/text-animations.md) - Typography and text animation patterns for Remotion
-- [rules/timing.md](rules/timing.md) - Interpolation curves in Remotion - linear, easing, spring animations
-- [rules/transcribe-captions.md](rules/transcribe-captions.md) - Transcribing audio to generate captions in Remotion
-- [rules/transitions.md](rules/transitions.md) - Scene transition patterns for Remotion
-- [rules/trimming.md](rules/trimming.md) - Trimming patterns for Remotion - cut the beginning or end of animations
-- [rules/videos.md](rules/videos.md) - Embedding videos in Remotion - trimming, volume, speed, looping, pitch