vibes-plug 1.0.0 → 2.5.0

package/skills/web-3d-graphics-expert/SKILL.md

@@ -0,0 +1,131 @@
+---
+name: web-3d-graphics-expert
+description: "Expert guide for WebGL and 3D graphics in the browser using Three.js, Babylon.js, React Three Fiber (R3F), and TresJS. Covers scene optimization, shaders, lighting, 3D model loading (GLTF/GLB), and performance tuning."
+author: "Roedy Rustam"
+---
+
+# Web 3D Graphics Expert (Three.js & Babylon.js)
+
+[English](#english) | [Bahasa Indonesia](#bahasa-indonesia)
+
+---
+
+<a name="english"></a>
+## English
+
+### Description
+Production-level guidance for developing high-performance 3D graphics in the browser. Covers the raw APIs of **Three.js** and **Babylon.js**, as well as framework-specific wrappers like **React Three Fiber (R3F)** for React and **TresJS** for Vue. Focuses on rendering performance, efficient memory management, and realistic lighting/material setups.
+
+### Trigger Conditions
+Activate this skill when the user is:
+- Initializing a WebGL/WebGPU 3D scene.
+- Using `three`, `@react-three/fiber`, `@babylonjs/core`, or `@tresjs/core`.
+- Loading 3D models (`.glb`, `.gltf`, `.obj`).
+- Adding advanced lighting, shadows, or Post-Processing effects (Bloom, Depth of Field).
+- Asking about framerate drops, memory leaks, or rendering performance in 3D apps.
+
+---
+
+### Core Concepts & Framework Selection
+
+#### 1. React Three Fiber (R3F) vs Vanilla Three.js
+When building with React, always use **React Three Fiber**. It manages the scene graph, object lifecycles, and `requestAnimationFrame` natively within the React component tree.
+
+```tsx
+import { Canvas } from '@react-three/fiber'
+import { OrbitControls, Environment, useGLTF } from '@react-three/drei'
+
+export default function App() {
+  const { scene } = useGLTF('/model.glb')
+  return (
+    <Canvas camera={{ position: [0, 2, 5] }}>
+      <ambientLight intensity={0.5} />
+      <directionalLight position={[10, 10, 5]} castShadow />
+      <primitive object={scene} />
+      <Environment preset="city" />
+      <OrbitControls />
+    </Canvas>
+  )
+}
+```
+*Note: Always use `@react-three/drei` for common helpers (OrbitControls, Text, Environment).*
+
+#### 2. Babylon.js vs Three.js
+- **Three.js**: Lighter, more flexible, vast community, excellent for creative coding, landing page animations, and React/Vue integrations.
+- **Babylon.js**: Robust, game-engine-like architecture, built-in physics, excellent for web-based games, AR/VR (WebXR), and complex interactive simulations. Fully written in TypeScript.
+
+```typescript
+// Basic Babylon.js Setup
+import { Engine, Scene, FreeCamera, Vector3, MeshBuilder, HemisphericLight } from '@babylonjs/core'
+
+const canvas = document.getElementById('renderCanvas') as HTMLCanvasElement
+const engine = new Engine(canvas, true)
+const scene = new Scene(engine)
+
+const camera = new FreeCamera('camera1', new Vector3(0, 5, -10), scene)
+camera.setTarget(Vector3.Zero())
+camera.attachControl(canvas, true)
+
+const light = new HemisphericLight('light', new Vector3(0, 1, 0), scene)
+const sphere = MeshBuilder.CreateSphere('sphere', { diameter: 2 }, scene)
+
+engine.runRenderLoop(() => {
+  scene.render()
+})
+```
+
+#### 3. Asset Loading & Optimization
+- **Format:** ALWAYS use `.glb` or `.gltf`. Avoid `.obj` or `.fbx` in the browser as they are unoptimized.
+- **Compression:** Use Draco compression (`glTF-Transform` or `gltf-pipeline`) to dramatically reduce file sizes.
+- **Loading:** Preload models if they are critical to the first paint.
+
+---
+
+### Best Practices for Performance
+
+1. **Geometry and Material Reuse:**
+   Never create new Geometries or Materials inside the render loop. Instantiate them once and reuse them. Use `InstancedMesh` (Three.js) or `Thin Instances` (Babylon.js) when rendering hundreds of identical objects.
+2. **Memory Management (Disposing):**
+   WebGL does not automatically garbage collect textures and geometries. When an object is removed from the scene, you MUST call `.dispose()` on its geometry and material.
+   *(Note: R3F handles this automatically when components unmount).*
+3. **Shadows & Lighting:**
+   Limit the number of shadow-casting lights to 1 or 2. Use baked shadows (lightmaps) for static scenes. Use `Environment` maps (HDRI) instead of multiple lights for realistic reflections.
+4. **Post-Processing:**
+   Post-processing requires rendering the scene multiple times. Use it sparingly. Group passes together when possible.
+
+---
+
+### Integration with Other Skills (MANDATORY)
+
+This skill works best when combined with:
+- `senior-frontend` — When integrating React Three Fiber into Next.js/React applications.
+- `vue-frontend-expert` — When integrating TresJS into Vue 3/Nuxt 3 applications.
+- `performance-web-vitals` — To optimize the bundle size of 3D libraries and reduce Total Blocking Time (TBT).
+
+### Referenced By Orchestrators (MANDATORY)
+
+This skill should be referenced by the following orchestrators:
+- `brainstorming` — Add to "UI/UX & Design Systems" or create a "3D & Graphics" category.
+- `zero-to-prod-orchestrator` — Phase 5 (Frontend Implementation).
+
+---
+
+<a name="bahasa-indonesia"></a>
+## Bahasa Indonesia
+
+### Deskripsi
+Panduan produksi untuk pengembangan grafis 3D performa tinggi di browser. Mencakup **Three.js** dan **Babylon.js**, serta integrasi framework seperti **React Three Fiber (R3F)** untuk React dan **TresJS** untuk Vue. Fokus pada performa render, manajemen memori, dan pencahayaan realistis.
+
+### Kondisi Pemicu
+Aktifkan skill ini ketika pengguna sedang:
+- Menginisialisasi *scene* WebGL 3D.
+- Menggunakan library `three`, `@react-three/fiber`, atau `@babylonjs/core`.
+- Memuat model 3D (terutama format `.glb`/`.gltf`).
+- Mengoptimalkan FPS (frame-rate) atau menangani masalah *memory leak* pada aplikasi 3D.
+
+### Panduan Singkat
+- **Pemilihan Alat:** Gunakan **React Three Fiber** jika menggunakan React. Gunakan **TresJS** jika menggunakan Vue. Gunakan **Babylon.js** jika membuat game berbasis web yang membutuhkan mesin fisika kuat (physics engine).
+- **Format Aset:** Selalu gunakan format `.glb` dengan kompresi Draco.
+- **Performa:** Jangan membuat *Geometry* atau *Material* baru di dalam *render loop* (`requestAnimationFrame`). 
+- **Manajemen Memori:** Panggil method `.dispose()` pada geometry, material, dan texture yang sudah tidak digunakan di vanilla Three.js/Babylon.js untuk mencegah kebocoran memori.
+- **Pencahayaan:** Gunakan *Environment map* (HDRI) untuk pencahayaan realistis yang lebih ringan diproses daripada banyak lampu *directional/point*. Batasi lampu yang memancarkan bayangan maksimal 1-2 saja.

package/skills/web-game-engine-expert/SKILL.md

@@ -0,0 +1,96 @@
+---
+name: web-game-engine-expert
+description: "Expert guide for web-based game development. Covers Entity Component System (ECS) architectures, physics engines (Rapier, Havok, Cannon-es), collision detection, and game loop optimization."
+author: "Roedy Rustam"
+---
+
+# Web Game Engine Expert (Physics & ECS)
+
+[English](#english) | [Bahasa Indonesia](#bahasa-indonesia)
+
+---
+
+<a name="english"></a>
+## English
+
+### Description
+Production-grade guidance for building highly interactive 3D simulations and web games. Focuses on integrating modern WebAssembly-based **Physics Engines** (Havok, Rapier, Cannon-es) and structuring application state using the **Entity Component System (ECS)** pattern (e.g., bitECS, Miniplex).
+
+### Trigger Conditions
+Activate this skill when the user is:
+- Adding physics (gravity, collisions, rigid bodies) to a 3D scene (Three.js/Babylon.js).
+- Dealing with performance issues in web games with thousands of objects.
+- Structuring a complex game or simulation architecture.
+- Using `@dimforge/rapier3d`, `havok`, or `cannon-es`.
+
+### Core Concepts
+
+#### 1. Physics Engine Integration
+Avoid calculating collisions manually in JavaScript. Use a dedicated physics engine.
+- **Babylon.js + Havok:** Babylon.js natively supports Havok Physics (the same engine used in AAA titles like Zelda BotW).
+- **Three.js / React Three Fiber + Rapier:** Use `@react-three/rapier` for seamless integration in R3F. It is written in Rust and compiled to WebAssembly.
+
+```tsx
+// React Three Fiber + Rapier Example
+import { Physics, RigidBody } from '@react-three/rapier';
+
+function GameScene() {
+  return (
+    <Physics gravity={[0, -9.81, 0]}>
+      {/* Falling Object */}
+      <RigidBody colliders="box" position={[0, 10, 0]}>
+        <mesh>
+          <boxGeometry />
+          <meshStandardMaterial color="hotpink" />
+        </mesh>
+      </RigidBody>
+
+      {/* Static Floor */}
+      <RigidBody type="fixed" colliders="cuboid">
+        <mesh position={[0, -1, 0]}>
+          <boxGeometry args={[10, 1, 10]} />
+          <meshStandardMaterial color="gray" />
+        </mesh>
+      </RigidBody>
+    </Physics>
+  );
+}
+```
+
+#### 2. Entity Component System (ECS)
+Traditional Object-Oriented Programming (OOP) can cause performance bottlenecks in games due to poor cache locality. ECS stores data in contiguous memory arrays (Components) and processes them in tight loops (Systems).
+- Use **bitECS** for extreme performance (uses TypedArrays).
+- Use **Miniplex** for an easier, developer-friendly React integration.
+
+### Best Practices
+- **Decouple Logic from Rendering:** The physics engine should update at a fixed timestep (e.g., 60Hz), while rendering should happen at the screen's refresh rate (e.g., 144Hz) using interpolation to prevent stuttering.
+- **Use Simple Colliders:** Never use complex mesh colliders (Convex/Trimesh) for moving objects unless absolutely necessary. Use primitives (Spheres, Boxes, Capsules).
+
+---
+
+### Integration with Other Skills (MANDATORY)
+This skill works best when combined with:
+- `web-3d-graphics-expert` — For the visual rendering of the physics world.
+- `webxr-ar-vr-expert` — For adding grabbable physics objects in VR space.
+- `realtime-collaboration-expert` — For synchronizing physics state across multiple players using WebRTC.
+
+### Referenced By Orchestrators (MANDATORY)
+- `brainstorming` — Add to "3D & Graphics" or "Game Development".
+- `zero-to-prod-orchestrator` — Phase 5 (Frontend / Simulation).
+
+---
+
+<a name="bahasa-indonesia"></a>
+## Bahasa Indonesia
+
+### Deskripsi
+Panduan pengembangan *game* dan simulasi 3D di web. Mencakup integrasi *Physics Engine* berbasis WebAssembly (Rapier, Havok, Cannon) dan arsitektur *Entity Component System* (ECS).
+
+### Kondisi Pemicu
+- Saat menambahkan gravitasi, tabrakan (collision), atau benda padat (rigid body) ke dalam aplikasi 3D.
+- Saat membuat game web yang membutuhkan performa tinggi dengan ribuan entitas.
+
+### Panduan Singkat
+- **Pilih Mesin Fisika:** Gunakan **Havok** jika menggunakan Babylon.js. Gunakan **Rapier** (`@react-three/rapier`) jika menggunakan Three.js/R3F.
+- **Gunakan Collider Sederhana:** Untuk objek bergerak, selalu gunakan *primitive colliders* (Kotak, Bola, Kapsul) alih-alih *mesh collider* agar perhitungan CPU/WASM tetap ringan.
+- **Pola ECS (Entity Component System):** Untuk game kompleks, pisahkan logika dan data menggunakan arsitektur ECS (seperti `miniplex` atau `bitECS`) untuk memaksimalkan performa *cache* memori.

package/skills/web-scraper/SKILL.md

@@ -1,205 +1,207 @@
----
-name: web-scraper
-description: "Smart web data extraction capability with multi-strategy scraping (Crawl4AI, Playwright, BeautifulSoup), LLM extraction, pagination support, and structured export / Kemampuan ekstraksi data web cerdas dengan strategi scraping modern (Crawl4AI, Playwright, BeautifulSoup), ekstraksi LLM, paginasi, dan ekspor terstruktur."
-author: "Roedy Rustam"
----
-
-# Web Scraper Expert (2026 Edition)
-
-[English](#english) | [Bahasa Indonesia](#bahasa-indonesia)
-
----
-
-<a name="english"></a>
-## English
-
-### Description
-Smart web data extraction using modern scraping strategies — **Firecrawl** (LLM-ready structured extraction), **Jina Reader API** (clean markdown from any URL), **Crawl4AI** (async Python scraping), **Playwright** (browser automation), and **BeautifulSoup** (lightweight HTML parsing). Supports pagination, dynamic content, structured export (JSON, CSV, Markdown), and AI-powered data extraction.
-
-### Trigger Conditions
-- Extracting structured data from websites for analysis, training data, or content pipelines.
-- Scraping dynamic JavaScript-rendered pages (SPAs, React apps).
-- Converting web pages to clean Markdown for LLM context or RAG pipelines.
-- Building an automated data pipeline that scrapes, transforms, and stores data.
-- Extracting data at scale with rate limiting and proxy rotation.
-
-### Strategy Selection Guide
-
-| Strategy | Tool | Best For | JS Required |
-|---|---|---|---|
-| **Managed API** | Firecrawl / Jina Reader | LLM-ready content, clean Markdown | Handled |
-| **Async Python** | Crawl4AI | High-volume, AI-powered extraction | Optional |
-| **Browser Automation** | Playwright | Complex SPAs, auth-required pages | ✅ |
-| **Lightweight HTML** | BeautifulSoup + httpx | Static HTML, fast extraction | ❌ |
-
-### Firecrawl — LLM-Ready Web Scraping (2026 Standard)
-Firecrawl converts any URL to clean, LLM-ready Markdown with a single API call:
-```python
-from firecrawl import FirecrawlApp
-
-app = FirecrawlApp(api_key="fc-xxxx")
-
-# Single page — clean Markdown
-result = app.scrape_url(
-    "https://example.com/article",
-    formats=["markdown", "html"],
-    only_main_content=True,  # removes nav, footer, ads
-)
-print(result.markdown)
-
-# Full site crawl
-crawl_result = app.crawl_url(
-    "https://docs.example.com",
-    limit=50,
-    scrape_options={"formats": ["markdown"]},
-)
-
-# LLM-powered structured extraction with JSON schema
-from pydantic import BaseModel
-
-class ProductInfo(BaseModel):
-    name: str
-    price: float
-    description: str
-    in_stock: bool
-
-result = app.scrape_url(
-    "https://shop.example.com/product/123",
-    formats=["extract"],
-    extract={"schema": ProductInfo.model_json_schema()},
-)
-product = ProductInfo(**result.extract)
-```
-
-### Jina Reader API — Clean Markdown from Any URL
-```python
-import httpx
-
-async def url_to_markdown(url: str) -> str:
-    """Convert any URL to clean Markdown via Jina Reader API."""
-    async with httpx.AsyncClient() as client:
-        response = await client.get(
-            f"https://r.jina.ai/{url}",
-            headers={
-                "Accept": "application/json",
-                "X-Return-Format": "markdown",
-                "Authorization": f"Bearer {JINA_API_KEY}",
-            }
-        )
-        data = response.json()
-        return data["data"]["content"]
-
-# Search and get results as Markdown
-async def search_to_markdown(query: str) -> str:
-    async with httpx.AsyncClient() as client:
-        response = await client.get(
-            f"https://s.jina.ai/{query}",
-            headers={"Accept": "application/json"}
-        )
-        return response.json()["data"]
-```
-
-### Crawl4AI — Async Python Web Scraper
-```python
-import asyncio
-from crawl4ai import AsyncWebCrawler, BrowserConfig, CrawlerRunConfig, CacheMode
-
-async def scrape_with_ai_extraction():
-    browser_config = BrowserConfig(headless=True, verbose=False)
-    
-    async with AsyncWebCrawler(config=browser_config) as crawler:
-        result = await crawler.arun(
-            url="https://news.example.com",
-            config=CrawlerRunConfig(
-                cache_mode=CacheMode.ENABLED,       # cache results
-                word_count_threshold=50,            # skip short content
-                exclude_external_links=True,
-                remove_overlay_elements=True,       # remove popups/modals
-            ),
-        )
-        
-        print(result.markdown.fit_markdown)  # cleaned, AI-optimized Markdown
-        print(result.links)                  # extracted links
-```
-
-### Playwright — Complex Dynamic Pages
-```python
-from playwright.async_api import async_playwright
-import asyncio
-
-async def scrape_spa(url: str) -> dict:
-    async with async_playwright() as pw:
-        browser = await pw.chromium.launch(headless=True)
-        page = await browser.new_page()
-        
-        # Block images/CSS to speed up
-        await page.route("**/*.{png,jpg,jpeg,gif,css,woff2}", lambda r: r.abort())
-        
-        await page.goto(url, wait_until="networkidle")
-        
-        # Wait for dynamic content
-        await page.wait_for_selector("[data-testid='product-list']")
-        
-        # Extract structured data via JavaScript
-        products = await page.evaluate("""() => {
-            return Array.from(document.querySelectorAll('.product-card')).map(card => ({
-                name: card.querySelector('.product-name')?.textContent?.trim(),
-                price: card.querySelector('.price')?.textContent?.trim(),
-            }));
-        }""")
-        
-        await browser.close()
-        return {"products": products, "url": url}
-```
-
-### Ethical Scraping Checklist
-- [ ] Check `robots.txt` and respect `Disallow` rules.
-- [ ] Implement rate limiting — minimum 1-2 seconds between requests.
-- [ ] Use descriptive `User-Agent` header with contact email.
-- [ ] Cache results to avoid repeated requests.
-- [ ] Prefer public APIs or official data feeds when available.
-- [ ] Do not scrape personal/private data without consent.
-
----
-
-<a name="bahasa-indonesia"></a>
-## Bahasa Indonesia
-
-### Deskripsi
-Ekstraksi data web yang cerdas menggunakan strategi scraping modern — **Firecrawl** (ekstraksi terstruktur siap-LLM), **Jina Reader API** (Markdown bersih dari URL manapun), **Crawl4AI** (scraping Python async), **Playwright** (otomasi browser), dan **BeautifulSoup** (parsing HTML ringan). Mendukung paginasi, konten dinamis, ekspor terstruktur, dan ekstraksi data bertenaga AI.
-
-### Kondisi Pemicu
-- Mengekstrak data terstruktur dari website untuk analisis, data pelatihan, atau pipeline konten.
-- Scraping halaman yang dirender JavaScript secara dinamis (SPA, aplikasi React).
-- Mengonversi halaman web menjadi Markdown bersih untuk konteks LLM atau pipeline RAG.
-- Membangun pipeline data otomatis yang melakukan scraping, transformasi, dan penyimpanan.
-- Mengekstrak data dalam skala besar dengan rate limiting dan rotasi proxy.
-
-### Panduan Pemilihan Strategi
-
-| Strategi | Tool | Terbaik Untuk | Perlu JS |
-|---|---|---|---|
-| **API Terkelola** | Firecrawl / Jina Reader | Konten siap LLM, Markdown bersih | Ditangani |
-| **Python Async** | Crawl4AI | Volume tinggi, ekstraksi AI | Opsional |
-| **Otomasi Browser** | Playwright | SPA kompleks, halaman butuh auth | ✅ |
-| **HTML Ringan** | BeautifulSoup + httpx | HTML statis, ekstraksi cepat | ❌ |
-
-### Firecrawl — Scraping Web Siap LLM
-Firecrawl mengonversi URL apapun menjadi Markdown bersih siap LLM dengan satu panggilan API. Mendukung crawling seluruh situs, ekstraksi terstruktur berbasis skema JSON, dan penghapusan konten yang tidak relevan (navigasi, footer, iklan).
-
-### Jina Reader API — Markdown Bersih dari URL Manapun
-Jina Reader (`r.jina.ai/{url}`) mengonversi halaman web manapun menjadi Markdown yang dioptimalkan untuk LLM. Jina Search (`s.jina.ai/{query}`) melakukan pencarian web dan mengembalikan hasilnya sebagai Markdown.
-
-### Crawl4AI — Web Scraper Python Async
-Crawl4AI mendukung caching hasil, ekstraksi Markdown yang dioptimalkan AI, dan konfigurasi browser yang fleksibel untuk scraping konten dinamis.
-
-### Playwright — Halaman Dinamis Kompleks
-Gunakan Playwright untuk scraping SPA yang memerlukan eksekusi JavaScript, menunggu elemen dinamis, atau interaksi dengan halaman (klik, scroll, isi form).
-
-### Checklist Scraping Etis
-- [ ] Periksa `robots.txt` dan hormati aturan `Disallow`.
-- [ ] Implementasikan rate limiting — minimal 1-2 detik antar permintaan.
-- [ ] Gunakan header `User-Agent` deskriptif dengan email kontak.
-- [ ] Cache hasil untuk menghindari permintaan berulang.
-- [ ] Utamakan API publik atau feed data resmi jika tersedia.
-- [ ] Jangan scraping data pribadi/privat tanpa izin.
+---
+name: web-scraper
+description: "Smart web data extraction capability with multi-strategy scraping (Crawl4AI, Playwright, BeautifulSoup), LLM extraction, pagination support, and structured export / Kemampuan ekstraksi data web cerdas dengan strategi scraping modern (Crawl4AI, Playwright, BeautifulSoup), ekstraksi LLM, paginasi, dan ekspor terstruktur."
+author: "Roedy Rustam"
+---
+
+# Web Scraper Expert (2026 Edition)
+
+[English](#english) | [Bahasa Indonesia](#bahasa-indonesia)
+
+---
+
+<a name="english"></a>
+## English
+
+### Description
+Smart web data extraction using modern scraping strategies — **Firecrawl** (LLM-ready structured extraction), **Jina Reader API** (clean markdown from any URL), **Crawl4AI** (async Python scraping), **Playwright** (browser automation), and **BeautifulSoup** (lightweight HTML parsing). Supports pagination, dynamic content, structured export (JSON, CSV, Markdown), and AI-powered data extraction.
+
+### Trigger Conditions
+- Extracting structured data from websites for analysis, training data, or content pipelines.
+- Scraping dynamic JavaScript-rendered pages (SPAs, React apps).
+- Converting web pages to clean Markdown for LLM context or RAG pipelines.
+- Analyzing or cloning website UI design, templates, and components directly from a URL (Handoff to `website-design-cloner`).
+- Building an automated data pipeline that scrapes, transforms, and stores data.
+- Extracting data at scale with rate limiting and proxy rotation.
+
+### Strategy Selection Guide
+
+| Strategy | Tool | Best For | JS Required |
+|---|---|---|---|
+| **Managed API** | Firecrawl / Jina Reader | LLM-ready content, clean Markdown | Handled |
+| **Async Python** | Crawl4AI | High-volume, AI-powered extraction | Optional |
+| **Browser Automation** | Playwright | Complex SPAs, auth-required pages | ✅ |
+| **Lightweight HTML** | BeautifulSoup + httpx | Static HTML, fast extraction | ❌ |
+
+### Firecrawl — LLM-Ready Web Scraping (2026 Standard)
+Firecrawl converts any URL to clean, LLM-ready Markdown with a single API call:
+```python
+from firecrawl import FirecrawlApp
+
+app = FirecrawlApp(api_key="fc-xxxx")
+
+# Single page — clean Markdown
+result = app.scrape_url(
+    "https://example.com/article",
+    formats=["markdown", "html"],
+    only_main_content=True,  # removes nav, footer, ads
+)
+print(result.markdown)
+
+# Full site crawl
+crawl_result = app.crawl_url(
+    "https://docs.example.com",
+    limit=50,
+    scrape_options={"formats": ["markdown"]},
+)
+
+# LLM-powered structured extraction with JSON schema
+from pydantic import BaseModel
+
+class ProductInfo(BaseModel):
+    name: str
+    price: float
+    description: str
+    in_stock: bool
+
+result = app.scrape_url(
+    "https://shop.example.com/product/123",
+    formats=["extract"],
+    extract={"schema": ProductInfo.model_json_schema()},
+)
+product = ProductInfo(**result.extract)
+```
+
+### Jina Reader API — Clean Markdown from Any URL
+```python
+import httpx
+
+async def url_to_markdown(url: str) -> str:
+    """Convert any URL to clean Markdown via Jina Reader API."""
+    async with httpx.AsyncClient() as client:
+        response = await client.get(
+            f"https://r.jina.ai/{url}",
+            headers={
+                "Accept": "application/json",
+                "X-Return-Format": "markdown",
+                "Authorization": f"Bearer {JINA_API_KEY}",
+            }
+        )
+        data = response.json()
+        return data["data"]["content"]
+
+# Search and get results as Markdown
+async def search_to_markdown(query: str) -> str:
+    async with httpx.AsyncClient() as client:
+        response = await client.get(
+            f"https://s.jina.ai/{query}",
+            headers={"Accept": "application/json"}
+        )
+        return response.json()["data"]
+```
+
+### Crawl4AI — Async Python Web Scraper
+```python
+import asyncio
+from crawl4ai import AsyncWebCrawler, BrowserConfig, CrawlerRunConfig, CacheMode
+
+async def scrape_with_ai_extraction():
+    browser_config = BrowserConfig(headless=True, verbose=False)
+    
+    async with AsyncWebCrawler(config=browser_config) as crawler:
+        result = await crawler.arun(
+            url="https://news.example.com",
+            config=CrawlerRunConfig(
+                cache_mode=CacheMode.ENABLED,       # cache results
+                word_count_threshold=50,            # skip short content
+                exclude_external_links=True,
+                remove_overlay_elements=True,       # remove popups/modals
+            ),
+        )
+        
+        print(result.markdown.fit_markdown)  # cleaned, AI-optimized Markdown
+        print(result.links)                  # extracted links
+```
+
+### Playwright — Complex Dynamic Pages
+```python
+from playwright.async_api import async_playwright
+import asyncio
+
+async def scrape_spa(url: str) -> dict:
+    async with async_playwright() as pw:
+        browser = await pw.chromium.launch(headless=True)
+        page = await browser.new_page()
+        
+        # Block images/CSS to speed up
+        await page.route("**/*.{png,jpg,jpeg,gif,css,woff2}", lambda r: r.abort())
+        
+        await page.goto(url, wait_until="networkidle")
+        
+        # Wait for dynamic content
+        await page.wait_for_selector("[data-testid='product-list']")
+        
+        # Extract structured data via JavaScript
+        products = await page.evaluate("""() => {
+            return Array.from(document.querySelectorAll('.product-card')).map(card => ({
+                name: card.querySelector('.product-name')?.textContent?.trim(),
+                price: card.querySelector('.price')?.textContent?.trim(),
+            }));
+        }""")
+        
+        await browser.close()
+        return {"products": products, "url": url}
+```
+
+### Ethical Scraping Checklist
+- [ ] Check `robots.txt` and respect `Disallow` rules.
+- [ ] Implement rate limiting — minimum 1-2 seconds between requests.
+- [ ] Use descriptive `User-Agent` header with contact email.
+- [ ] Cache results to avoid repeated requests.
+- [ ] Prefer public APIs or official data feeds when available.
+- [ ] Do not scrape personal/private data without consent.
+
+---
+
+<a name="bahasa-indonesia"></a>
+## Bahasa Indonesia
+
+### Deskripsi
+Ekstraksi data web yang cerdas menggunakan strategi scraping modern — **Firecrawl** (ekstraksi terstruktur siap-LLM), **Jina Reader API** (Markdown bersih dari URL manapun), **Crawl4AI** (scraping Python async), **Playwright** (otomasi browser), dan **BeautifulSoup** (parsing HTML ringan). Mendukung paginasi, konten dinamis, ekspor terstruktur, dan ekstraksi data bertenaga AI.
+
+### Kondisi Pemicu
+- Mengekstrak data terstruktur dari website untuk analisis, data pelatihan, atau pipeline konten.
+- Scraping halaman yang dirender JavaScript secara dinamis (SPA, aplikasi React).
+- Mengonversi halaman web menjadi Markdown bersih untuk konteks LLM atau pipeline RAG.
+- Mempelajari atau menduplikasi desain UI, template, dan komponen situs web langsung dari URL (Delegasikan ke `website-design-cloner`).
+- Membangun pipeline data otomatis yang melakukan scraping, transformasi, dan penyimpanan.
+- Mengekstrak data dalam skala besar dengan rate limiting dan rotasi proxy.
+
+### Panduan Pemilihan Strategi
+
+| Strategi | Tool | Terbaik Untuk | Perlu JS |
+|---|---|---|---|
+| **API Terkelola** | Firecrawl / Jina Reader | Konten siap LLM, Markdown bersih | Ditangani |
+| **Python Async** | Crawl4AI | Volume tinggi, ekstraksi AI | Opsional |
+| **Otomasi Browser** | Playwright | SPA kompleks, halaman butuh auth | ✅ |
+| **HTML Ringan** | BeautifulSoup + httpx | HTML statis, ekstraksi cepat | ❌ |
+
+### Firecrawl — Scraping Web Siap LLM
+Firecrawl mengonversi URL apapun menjadi Markdown bersih siap LLM dengan satu panggilan API. Mendukung crawling seluruh situs, ekstraksi terstruktur berbasis skema JSON, dan penghapusan konten yang tidak relevan (navigasi, footer, iklan).
+
+### Jina Reader API — Markdown Bersih dari URL Manapun
+Jina Reader (`r.jina.ai/{url}`) mengonversi halaman web manapun menjadi Markdown yang dioptimalkan untuk LLM. Jina Search (`s.jina.ai/{query}`) melakukan pencarian web dan mengembalikan hasilnya sebagai Markdown.
+
+### Crawl4AI — Web Scraper Python Async
+Crawl4AI mendukung caching hasil, ekstraksi Markdown yang dioptimalkan AI, dan konfigurasi browser yang fleksibel untuk scraping konten dinamis.
+
+### Playwright — Halaman Dinamis Kompleks
+Gunakan Playwright untuk scraping SPA yang memerlukan eksekusi JavaScript, menunggu elemen dinamis, atau interaksi dengan halaman (klik, scroll, isi form).
+
+### Checklist Scraping Etis
+- [ ] Periksa `robots.txt` dan hormati aturan `Disallow`.
+- [ ] Implementasikan rate limiting — minimal 1-2 detik antar permintaan.
+- [ ] Gunakan header `User-Agent` deskriptif dengan email kontak.
+- [ ] Cache hasil untuk menghindari permintaan berulang.
+- [ ] Utamakan API publik atau feed data resmi jika tersedia.
+- [ ] Jangan scraping data pribadi/privat tanpa izin.