@wenathlan/saddle 1.8.17 → 2.0.2

package/docs/plans/25.research.server.api.md

@@ -1,342 +0,0 @@
-# Pesquisa: Server, API e Distribuição (2026)
-
-## 1. Framework Comparison
-
-### Performance Benchmarks (2026)
-
-| Framework | Runtime | req/s (Node.js) | req/s (Bun) | Bundle Size | TypeScript | GitHub Stars | npm Weekly Downloads |
-|-----------|---------|-----------------|-------------|-------------|------------|--------------|---------------------|
-| **Elysia** | Bun-native | N/A | ~184K (peak) | ~30 KB | Native (best) | ~12K | ~300K-700K |
-| **Hono** | Multi-runtime | ~45K-78K | ~55K-62K | ~12-15 KB | First-class | ~22-28K | ~1.2-9.3M |
-| **Fastify** | Node.js | ~42K-114K | ~85K | ~50 KB+ | First-class | ~33-36K | ~4-11M |
-| **Express** | Node.js | ~8K-21K | ~15-21K | ~200 KB+ | Via @types | ~65-69K | ~30-35M |
-
-### Recommendation Matrix for Scraping APIs
-
-| Use Case | Top Pick | Runner-up | Reason |
-|----------|----------|-----------|--------|
-| **Cloudflare Workers / Edge API** | Hono | — | Native support, 14 KB bundle, runs everywhere |
-| **New Node.js REST API** | Fastify / Hono | Fastify | 3-10x faster than Express, schema validation |
-| **Bun production API** | Elysia | Hono | Peak Bun throughput, end-to-end type safety |
-| **Existing Express codebase** | Express | Fastify | Lowest migration cost, largest ecosystem |
-| **Multi-runtime / portable** | Hono | — | Same code on Node, Bun, Deno, CF Workers, Lambda |
-| **Enterprise large-scale** | Fastify | Express | Plugin system, stability, 300+ plugins |
-
-### Framework-Specific Middleware
-
-| Capability | Express | Fastify | Hono | Elysia |
-|-----------|---------|---------|------|--------|
-| CORS | `cors` (npm) | `@fastify/cors` | Built-in `hono/cors` | `@elysiajs/cors` |
-| Rate limiting | `express-rate-limit` | `@fastify/rate-limit` | `hono-rate-limiter` | Custom |
-| JWT Auth | `passport`, `jsonwebtoken` | `@fastify/jwt` | `hono/jwt` | `@elysiajs/jwt` |
-| WebSocket | `ws` (standalone) | `@fastify/websocket` | `hono/ws` | Native |
-| OpenAPI/Swagger | `swagger-jsdoc` | `@fastify/swagger` | `@hono/swagger-ui` | `@elysiajs/swagger` |
-
-## 2. Rate Limiting on the API Layer
-
-### Packages by Framework
-
-| Package | Framework | Algorithm |
-|---------|-----------|-----------|
-| `express-rate-limit` 8.5.x | Express | Fixed window, pluggable stores |
-| `@fastify/rate-limit` | Fastify | Fixed window, Redis-backed |
-| `hono-rate-limiter` 0.5.x | Hono | Fixed window, Redis/CF KV |
-| `universal-rate-limit` | All | Fixed/sliding window, token bucket |
-
-### Hono Rate Limiting Example
-
-```typescript
-import { rateLimiter } from 'hono-rate-limiter';
-import { RedisStore } from '@hono-rate-limiter/redis';
-import { Redis } from '@upstash/redis';
-
-const client = new Redis({ url: REDIS_URL, token: REDIS_TOKEN });
-
-app.use('/api/*', rateLimiter({
-  windowMs: 60_000,
-  limit: 100,
-  store: new RedisStore({ client }),
-  keyGenerator: (c) => c.var.accountId,
-  standardHeaders: 'draft-7',
-}));
-```
-
-### Tiered Rate Limiting
-
-```typescript
-// Global limit
-app.use(rateLimiter({ windowMs: 60_000, limit: 1000 }));
-
-// Per-endpoint for expensive operations
-app.post('/v1/scrape', rateLimiter({
-  windowMs: 60_000,
-  limit: 10,
-  keyGenerator: (c) => c.var.userId,
-}));
-```
-
-## 3. Streaming Responses
-
-### Chunked Transfer Encoding
-
-```typescript
-app.get('/api/export', async (req, res) => {
-  res.writeHead(200, {
-    'Content-Type': 'application/json',
-    'Transfer-Encoding': 'chunked',
-  });
-  res.write('[\n');
-  let first = true;
-  for await (const batch of scraper.streamResults(url)) {
-    for (const row of batch) {
-      res.write(first ? '' : ',\n');
-      res.write(JSON.stringify(row));
-      first = false;
-    }
-    await new Promise(resolve => setImmediate(resolve));
-  }
-  res.write('\n]');
-  res.end();
-});
-```
-
-### Server-Sent Events (SSE)
-
-```typescript
-app.get('/api/scrape/stream', async (req, res) => {
-  res.setHeader('Content-Type', 'text/event-stream');
-  res.setHeader('Cache-Control', 'no-cache');
-  res.setHeader('Connection', 'keep-alive');
-  res.setHeader('X-Accel-Buffering', 'no');
-  res.flushHeaders();
-
-  const scraper = scrapeWithProgress(url);
-  let eventId = 0;
-
-  for await (const event of scraper) {
-    res.write(`id: ${++eventId}\n`);
-    res.write(`event: ${event.type}\n`);
-    res.write(`data: ${JSON.stringify(event.data)}\n\n`);
-  }
-
-  res.write('event: done\ndata: {"status":"complete"}\n\n');
-  res.end();
-  req.on('close', () => scraper.abort());
-});
-```
-
-### SSE Gotchas
-- Nginx buffers by default — add `proxy_buffering off;` and `X-Accel-Buffering: no`
-- Cloudflare free tier has 100s idle timeout — send `: keepalive\n\n` every 30s
-- Browsers limit 6 concurrent SSE connections per origin (HTTP/1.1)
-- Skip streaming for responses under 50KB
-
-## 4. WebSocket for Real-Time Progress
-
-```typescript
-import { WebSocketServer } from 'ws';
-
-const wss = new WebSocketServer({ port: 3001 });
-
-wss.on('connection', (ws) => {
-  ws.isAlive = true;
-  ws.on('pong', () => { ws.isAlive = true; });
-  ws.on('message', (data) => {
-    const msg = JSON.parse(data);
-    if (msg.type === 'subscribe') ws.jobId = msg.jobId;
-  });
-});
-
-// Heartbeat
-setInterval(() => {
-  wss.clients.forEach(ws => {
-    if (!ws.isAlive) return ws.terminate();
-    ws.isAlive = false;
-    ws.ping();
-  });
-}, 30_000);
-
-function broadcastJobUpdate(jobId: string, data: unknown) {
-  wss.clients.forEach(ws => {
-    if (ws.jobId === jobId && ws.readyState === 1) {
-      ws.send(JSON.stringify({ type: 'job_update', ...data }));
-    }
-  });
-}
-```
-
-## 5. Authentication
-
-### API Key Pattern
-
-```typescript
-async function verifyApiKey(request: any, reply: any) {
-  const apiKey = request.headers['x-api-key'];
-  if (!apiKey) return reply.code(401).send({ error: 'API key required' });
-  const user = await redis.hgetall(`apikey:${apiKey}`);
-  if (!user.user_id) return reply.code(403).send({ error: 'Invalid API key' });
-  request.user = { userId: user.user_id };
-}
-```
-
-### JWT Pattern
-
-```typescript
-import jwt from 'jsonwebtoken';
-
-function validateJwt(request: any, reply: any) {
-  const token = request.headers.authorization?.replace('Bearer ', '');
-  if (!token) return reply.code(401).send({ error: 'Token required' });
-  try {
-    request.user = jwt.verify(token, SECRET, { algorithms: ['HS256'] });
-  } catch (e) {
-    return reply.code(401).send({ error: 'Invalid token' });
-  }
-}
-```
-
-## 6. REST API Design
-
-### Standard Endpoints
-
-```
-POST   /v1/scrape                  # Sync scrape
-POST   /v1/scrape/async            # Async scrape (returns task ID)
-GET    /v1/scrape/:taskId          # Get async job result
-POST   /v1/batch                   # Batch scrape multiple URLs
-POST   /v1/crawl                   # Start recursive crawl
-GET    /v1/crawl/:crawlId          # Get crawl status/results
-GET    /health                     # Health check
-```
-
-### Request/Response Schema
-
-```json
-// POST /v1/scrape
-{
-  "url": "https://example.com",
-  "useBrowser": false,
-  "format": "markdown",
-  "waitFor": "h1",
-  "timeout": 30,
-  "proxy": "http://user:pass@proxy:port"
-}
-
-// Response
-{
-  "status": "ok",
-  "data": {
-    "url": "https://example.com",
-    "markdown": "# Content...",
-    "metadata": { "title": "...", "description": "..." },
-    "links": [...],
-    "images": [...]
-  },
-  "timing": { "duration_ms": 1234 }
-}
-```
-
-### Error Response Format
-
-```json
-{
-  "error": {
-    "code": "RATE_LIMIT_EXCEEDED",
-    "message": "Rate limit exceeded. Try again in 60 seconds.",
-    "retry_after": 60,
-    "request_id": "req_abc123"
-  }
-}
-```
-
-## 7. Docker Deployment
-
-### Multi-Stage Dockerfile
-
-```dockerfile
-# syntax=docker/dockerfile:1.4
-FROM node:22-alpine AS deps
-WORKDIR /app
-COPY package.json package-lock.json ./
-RUN npm ci --frozen-lockfile
-
-FROM node:22-alpine AS builder
-WORKDIR /app
-COPY --from=deps /app/node_modules ./node_modules
-COPY . .
-RUN npm run build
-
-FROM node:22-alpine AS runner
-WORKDIR /app
-RUN addgroup --system --gid 1001 nodejs && adduser --system --uid 1001 appuser
-ENV NODE_ENV=production
-COPY package.json package-lock.json ./
-RUN npm ci --frozen-lockfile --omit=dev && npm cache clean --force
-COPY --from=builder /app/dist ./dist
-RUN chown -R appuser:nodejs /app
-USER appuser
-EXPOSE 3000
-HEALTHCHECK --interval=30s --timeout=5s --start-period=30s --retries=3 \
-    CMD node -e "require('http').get('http://localhost:3000/health', (r) => process.exit(r.statusCode === 200 ? 0 : 1)).on('error', () => process.exit(1))"
-CMD ["node", "--enable-source-maps", "dist/index.js"]
-```
-
-### Docker Compose
-
-```yaml
-services:
-  api:
-    build: .
-    ports: ["3000:3000"]
-    environment:
-      - DATABASE_URL=postgresql://user:pass@postgres:5432/scraper
-      - REDIS_URL=redis://redis:6379
-    depends_on: [postgres, redis]
-
-  postgres:
-    image: postgres:16-alpine
-    volumes: [pgdata:/var/lib/postgresql/data]
-    environment:
-      POSTGRES_DB: scraper
-      POSTGRES_USER: user
-      POSTGRES_PASSWORD: pass
-
-  redis:
-    image: redis:7-alpine
-    volumes: [redis_data:/data]
-
-volumes:
-  pgdata:
-  redis_data:
-```
-
-## 8. Serverless Deployment
-
-| Platform | Best For | Notes |
-|----------|----------|-------|
-| AWS Lambda (container) | Event-driven scraping | Up to 15 min timeout |
-| Google Cloud Run | Serverless pipeline | Separate scraper + API services |
-| Cloudflare Workers | Edge scraping API | Hono is best choice, 14 KB bundle |
-| Fly.io | Persistent containers | Good for browser-based scraping |
-| Render | Full-stack | Postgres + Redis included |
-
-## 9. Observability
-
-- Structured JSON logging (Pino for Fastify, built-in for Hono)
-- OpenTelemetry for distributed tracing
-- Metrics: request count, latency p50/p95/p99, scrape success rate
-- Per-user usage dashboards
-- Alerting on anomalous patterns
-
-## 10. Security Checklist
-
-- [ ] HTTPS everywhere
-- [ ] API key or JWT authentication
-- [ ] Rate limiting per user/API key
-- [ ] Input validation on all endpoints
-- [ ] SSRF protection (prevent scraping internal networks)
-- [ ] CORS configured properly
-- [ ] No secrets in Docker images
-- [ ] Request ID tracking for audit trails
-- [ ] Respect robots.txt
-- [ ] Data retention limits and deletion schedules

package/docs/plans/26.research.compilation.md

@@ -1,249 +0,0 @@
-# Pesquisa: Compilação e Build de Bibliotecas (2026)
-
-## 1. Bun `--compile` — Standalone Executables
-
-Bun's bundler has a mature `--compile` flag that generates standalone binaries with the Bun runtime embedded.
-
-### Basic Usage
-
-```bash
-bun build ./cli.ts --compile --outfile mycli
-./mycli
-```
-
-### Cross-Compilation Targets
-
-| Target | OS | Arch |
-|--------|-----|------|
-| `bun-linux-x64` | Linux | x64 (glibc) |
-| `bun-linux-arm64` | Linux | arm64 |
-| `bun-windows-x64` | Windows | x64 (.exe) |
-| `bun-windows-arm64` | Windows | arm64 |
-| `bun-darwin-x64` | macOS | x64 |
-| `bun-darwin-arm64` | macOS | arm64 |
-| `bun-linux-x64-musl` | Linux | x64 (Alpine) |
-| `bun-linux-arm64-musl` | Linux | arm64 (musl) |
-
-### Advanced Features
-
-```bash
-# Bytecode compilation (2x faster startup)
-bun build --compile --bytecode ./app.ts --outfile myapp
-
-# Cross-compile
-bun build --compile --target=bun-linux-x64 ./src/cli.ts --outfile myapp-linux
-
-# Embed files
-bun build --compile ./index.ts ./public/**/*.png
-
-# Build-time constants
-bun build --compile --define BUILD_VERSION='"1.2.3"' ./app.ts --outfile myapp
-
-# Minification + sourcemaps
-bun build --compile --minify --sourcemap ./app.ts --outfile myapp
-```
-
-### Key Features (2026)
-- Build-time constants via `--define`
-- Bytecode compilation for fast startup
-- Embed files/assets with `with { type: "file" }` import attributes
-- Embed SQLite databases with `with { type: "sqlite", embed: "true" }`
-- Embed N-API addons (`.node` files) directly
-- Full-stack executables (v1.2.17+): embed HTML/CSS/JS frontend
-- Windows metadata (icon, hideConsole, title)
-- macOS codesigning support
-
-## 2. tsup vs tsdown
-
-### tsup (legacy, v8.5.1 — no longer actively maintained)
-
-```typescript
-// tsup.config.ts
-import { defineConfig } from 'tsup'
-export default defineConfig({
-  entry: ['src/index.ts', 'src/cli.ts'],
-  format: ['cjs', 'esm'],
-  dts: true,
-  splitting: true,
-  clean: true,
-  minify: true,
-})
-```
-
-### tsdown (recommended successor, Rolldown-based)
-
-```bash
-npm i tsdown -D
-npx tsdown-migrate  # migrate from tsup
-```
-
-| Option | tsup | tsdown |
-|--------|------|--------|
-| Default format | `'cjs'` | `'esm'` |
-| Clean | `false` | `true` |
-| DTS | `false` | Auto-enabled |
-| Target | none | Auto-reads from `engines.node` |
-| Backend | esbuild | Rolldown (Rust) |
-
-### tsdown Unique Features
-- **`exe: true`** — bundle as Node.js SEA
-- **`exports: true`** — auto-generate `package.json` exports
-- **`publint`/`attw`** — validate package
-- **`devtools`** — Vite DevTools integration
-- **Full CSS pipeline** with preprocessors
-- **Lifecycle hooks**: `build:prepare`, `build:before`, `build:done`
-
-```typescript
-// tsdown.config.ts
-import { defineConfig } from 'tsdown'
-export default defineConfig({
-  entry: ['src/index.ts', 'src/cli.ts'],
-  format: ['cjs', 'esm'],
-  dts: true,
-  exe: false,
-  minify: true,
-  exports: true,
-  publint: true,
-})
-```
-
-## 3. Node.js SEA (Single Executable Applications)
-
-### Step-by-Step Process
-
-```bash
-# 1. Write your app
-echo 'console.log("Hello!");' > hello.js
-
-# 2. Bundle to single CJS file (esbuild/ncc/webpack)
-npx esbuild hello.ts --bundle --platform=node --outfile=hello.js --format=cjs
-
-# 3. Create sea-config.json
-echo '{"main":"hello.js","output":"sea-prep.blob","disableExperimentalSEAWarning":true}' > sea-config.json
-
-# 4. Generate the blob
-node --experimental-sea-config sea-config.json
-
-# 5. Copy node binary
-cp $(command -v node) hello
-
-# 6. Remove signature (macOS)
-codesign --remove-signature hello
-
-# 7. Inject blob with postject
-npx postject hello NODE_SEA_BLOB sea-prep.blob \
-    --sentinel-fuse NODE_SEA_FUSE_fce680ab2cc467b6e072b8b5df1996b2
-
-# 8. Re-sign (macOS)
-codesign --sign - hello
-
-# 9. Run
-./hello
-```
-
-### SEA v22+ Features
-- **Assets** — embed files via `"assets"` config field
-- **Startup snapshots** — `"useSnapshot": true`
-- **V8 code cache** — `"useCodeCache": true`
-- **Embedded `execArgv`** — `"execArgv": ["--no-warnings"]`
-
-**Key limitation:** SEA only supports CommonJS. Must bundle everything into a single JS file first.
-
-## 4. esbuild for Fast Bundling
-
-```bash
-# Browser bundle
-esbuild app.jsx --bundle --minify --sourcemap --target=chrome58 --outfile=out.js
-
-# Node.js bundle
-esbuild app.js --bundle --platform=node --target=node18 --outfile=out.js
-
-# Leave dependencies external (for libraries)
-esbuild app.jsx --bundle --platform=node --packages=external --outfile=out.js
-```
-
-### Key Features
-- Extremely fast (written in Go, parallelized)
-- TypeScript/JSX/TSX out of the box
-- CSS bundling support
-- Plugin system (`onResolve`, `onLoad`, `onStart`, `onEnd`)
-- WASM version available (`esbuild-wasm`)
-
-## 5. Creating CLI Tools Without Node.js
-
-### Options Ranked
-
-1. **Bun `--compile`** (recommended): True cross-compilation, supports Bun + Node.js APIs
-2. **Node.js SEA**: Embeds full Node.js runtime, CJS only
-3. **Deno compile**: Alternative, good for Deno-first projects
-
-## 6. Native Dependencies in Compiled Binaries
-
-### Strategies
-
-1. **Replace with pure-JS alternatives:**
-   - `better-sqlite3` → `bun:sqlite` (Bun) or `sql.js` (WASM)
-   - `bcrypt` → `bcryptjs`
-   - `sharp` → pure-JS image libs
-
-2. **Use WASM-based alternatives** — Many native libs now have WASM builds
-
-3. **Bundle with `ncc` and mark as external:**
-   ```bash
-   ncc build src/index.ts -o dist --external better-sqlite3
-   ```
-
-4. **With Bun compile**, embed `.node` file directly if you control the require path
-
-## 7. WASM Builds
-
-### Using tsdown with rolldown-plugin-wasm
-
-```typescript
-// tsdown.config.ts
-import { wasm } from 'rolldown-plugin-wasm'
-import { defineConfig } from 'tsdown'
-
-export default defineConfig({
-  entry: ['./src/index.ts'],
-  plugins: [wasm({
-    maxFileSize: 14 * 1024,
-    targetEnv: 'auto',
-  })],
-})
-```
-
-### Importing WASM Modules
-
-```typescript
-// Direct import
-import { add } from './add.wasm'
-add(1, 2)
-
-// Async init
-import init from './add.wasm?init'
-const instance = await init()
-instance.exports.add(1, 2)
-```
-
-## 8. Distribution Strategy Matrix
-
-| Target | Tool | Output | Needs Runtime? |
-|--------|------|--------|----------------|
-| npm package | tsdown | ESM + CJS + DTS | Yes (consumer's bundler) |
-| npm CLI tool | tsdown + bin field | JS + bin script | Yes (Node.js) |
-| Standalone binary | Bun compile | Native executable | No |
-| Node.js SEA | esbuild + postject | Modified node binary | Embedded Node.js |
-| Browser bundle | esbuild/tsdown | ESM/IIFE JS | No (browser) |
-| Edge/Workers | tsdown + WASM plugin | JS + .wasm | No (edge runtime) |
-| Serverless | @vercel/ncc | Single JS file | Yes (Node.js Lambda) |
-| Deno | deno compile | Native executable | No |
-
-## 9. Recommended 2026 Toolchain
-
-1. **tsdown** for library building (replaces tsup, uses Rolldown)
-2. **esbuild** for fast development bundling
-3. **Bun `--compile`** for standalone executables
-4. **Node.js SEA** when Node.js-based standalone is needed
-5. **rolldown-plugin-wasm** for WASM builds via tsdown
-6. **@vercel/ncc** for single-file Node.js deployments