@tinkcarlos/skillora 0.2.0

package/.claude/skills/api-testing-observability/SKILL.md

@@ -0,0 +1,583 @@
+---
+name: api-testing-observability
+description: |
+  API testing and observability with contract validation, monitoring, and log analysis.
+
+  Use when:
+  - Testing API endpoints (unit/integration/contract)
+  - Setting up API monitoring and observability
+  - Creating mock servers for development
+  - Validating API contracts
+  - Setting up structured logging and log aggregation
+  - Implementing error tracking (Sentry)
+
+  Key Features:
+  - Contract testing: OpenAPI validation
+  - Mock server: Realistic API simulation
+  - Observability: Logging, metrics, tracing
+  - Log aggregation: ELK/Loki setup
+  - Error tracking: Sentry integration
+  - Integration with fullstack-developer workflow
+allowed-tools: [read, write, execute, grep, glob]
+---
+
+# API Testing & Observability
+
+Test contracts, mock dependencies, observe everything.
+
+## The Iron Law
+
+```
+NO DEPLOY WITHOUT CONTRACT TEST. NO EXTERNAL CALL WITHOUT MOCK. NO PRODUCTION WITHOUT OBSERVABILITY.
+```
+
+---
+
+## Quick Start Decision Tree
+
+```
+API 测试需求
+    │
+    ▼
+┌─────────────────────────────────────────────────────────────┐
+│ Step 1: 确定测试类型                                         │
+│ ─────────────────────────────────────────────────────────── │
+│ 单元测试 → 测试单个函数/方法                                 │
+│ 集成测试 → 测试 API 端点 + 数据库                           │
+│ 契约测试 → 验证 API 符合 OpenAPI 规范                       │
+│ E2E 测试 → 完整用户流程                                     │
+└─────────────────────────────────────────────────────────────┘
+    │
+    ▼
+┌─────────────────────────────────────────────────────────────┐
+│ Step 2: 选择工具                                             │
+│ ─────────────────────────────────────────────────────────── │
+│ Python: pytest + httpx + pytest-asyncio                     │
+│ Node.js: Jest + Supertest                                   │
+│ 契约测试: Schemathesis / Dredd                              │
+│ Mock: WireMock / MSW / 自定义 Mock Server                   │
+└─────────────────────────────────────────────────────────────┘
+    │
+    ▼
+┌─────────────────────────────────────────────────────────────┐
+│ Step 3: 设置可观测性                                         │
+│ ─────────────────────────────────────────────────────────── │
+│ Logging: 结构化日志 + 请求 ID                               │
+│ Metrics: RED 指标 (Rate, Errors, Duration)                  │
+│ Tracing: OpenTelemetry 分布式追踪                           │
+└─────────────────────────────────────────────────────────────┘
+```
+
+---
+
+## 🔴 Testing Pyramid
+
+```
+                    ┌─────────┐
+                    │  E2E    │  少量，验证关键流程
+                    │  Tests  │
+                   ─┴─────────┴─
+                  ┌─────────────┐
+                  │ Integration │  中等，测试 API 端点
+                  │   Tests     │
+                 ─┴─────────────┴─
+                ┌─────────────────┐
+                │  Contract Tests │  验证 API 契约
+               ─┴─────────────────┴─
+              ┌─────────────────────┐
+              │    Unit Tests       │  大量，测试业务逻辑
+              └─────────────────────┘
+```
+
+---
+
+## 🔴 Contract Testing Protocol (MANDATORY)
+
+### Step 1: 契约验证
+
+**每个 API 必须通过契约测试:**
+
+```python
+# tests/contract/test_openapi_compliance.py
+import schemathesis
+
+schema = schemathesis.from_path("docs/openapi.yaml")
+
+@schema.parametrize()
+def test_api_contract(case):
+    """验证 API 符合 OpenAPI 规范"""
+    response = case.call()
+    case.validate_response(response)
+```
+
+### Step 2: 契约测试清单
+
+| 检查项 | 验证内容 |
+|--------|----------|
+| 响应状态码 | 符合 OpenAPI 定义 |
+| 响应结构 | 字段类型、必填字段 |
+| 请求验证 | 参数类型、格式 |
+| 错误响应 | 错误格式一致 |
+| 边界条件 | 空值、超长、特殊字符 |
+
+---
+
+## 🔴 Integration Testing
+
+### FastAPI 集成测试模板
+
+```python
+# tests/integration/test_users_api.py
+import pytest
+from httpx import AsyncClient
+from app.main import app
+
+@pytest.fixture
+async def client():
+    async with AsyncClient(app=app, base_url="http://test") as ac:
+        yield ac
+
+@pytest.fixture
+async def auth_headers(client):
+    """获取认证 token"""
+    response = await client.post("/auth/login", json={
+        "username": "test_user",
+        "password": "test_password"
+    })
+    token = response.json()["access_token"]
+    return {"Authorization": f"Bearer {token}"}
+
+class TestUsersAPI:
+    async def test_create_user_success(self, client, auth_headers):
+        """测试创建用户成功"""
+        response = await client.post(
+            "/api/v1/users",
+            json={"username": "newuser", "email": "new@example.com"},
+            headers=auth_headers
+        )
+        assert response.status_code == 201
+        assert response.json()["username"] == "newuser"
+
+    async def test_create_user_duplicate(self, client, auth_headers):
+        """测试重复创建用户"""
+        # 先创建一个用户
+        await client.post("/api/v1/users", json={...}, headers=auth_headers)
+        # 再次创建相同用户
+        response = await client.post("/api/v1/users", json={...}, headers=auth_headers)
+        assert response.status_code == 409
+
+    async def test_get_user_not_found(self, client, auth_headers):
+        """测试获取不存在的用户"""
+        response = await client.get("/api/v1/users/99999", headers=auth_headers)
+        assert response.status_code == 404
+
+    async def test_unauthorized_access(self, client):
+        """测试未认证访问"""
+        response = await client.get("/api/v1/users/1")
+        assert response.status_code == 401
+```
+
+### 测试覆盖要求
+
+| 场景 | 必须测试 |
+|------|----------|
+| Happy Path | 正常流程成功 |
+| 认证失败 | 401 Unauthorized |
+| 授权失败 | 403 Forbidden |
+| 资源不存在 | 404 Not Found |
+| 输入验证 | 400/422 错误 |
+| 资源冲突 | 409 Conflict |
+| 边界条件 | 空值、极值 |
+
+---
+
+## 🔴 Mock Server Protocol
+
+### 何时使用 Mock
+
+| 场景 | 使用 Mock |
+|------|-----------|
+| 外部 API 依赖 | ✅ 必须 Mock |
+| 数据库 | ❌ 使用测试数据库 |
+| 第三方服务 | ✅ 必须 Mock |
+| 内部微服务 | ✅ 契约测试时 Mock |
+
+### Mock Server 实现
+
+```python
+# tests/mocks/mock_server.py
+from fastapi import FastAPI
+from typing import Dict, Any
+
+class MockServer:
+    def __init__(self):
+        self.app = FastAPI()
+        self.stubs: Dict[str, Any] = {}
+        self._setup_routes()
+
+    def stub(self, method: str, path: str, response: dict, status: int = 200):
+        """添加 stub"""
+        key = f"{method.upper()}:{path}"
+        self.stubs[key] = {"response": response, "status": status}
+
+    def reset(self):
+        """重置所有 stubs"""
+        self.stubs.clear()
+
+    def _setup_routes(self):
+        @self.app.api_route("/{path:path}", methods=["GET", "POST", "PUT", "DELETE"])
+        async def handle_request(path: str, request):
+            key = f"{request.method}:/{path}"
+            if key in self.stubs:
+                stub = self.stubs[key]
+                return JSONResponse(stub["response"], status_code=stub["status"])
+            return JSONResponse({"error": "No stub found"}, status_code=404)
+
+# 使用示例
+@pytest.fixture
+async def mock_payment_service():
+    server = MockServer()
+    server.stub("POST", "/payments", {"id": "pay_123", "status": "success"}, 201)
+    # 启动 mock server...
+    yield server
+    server.reset()
+```
+
+### Scenario-Based Mocking
+
+```python
+# tests/mocks/scenarios.py
+SCENARIOS = {
+    "happy_path": {
+        "POST:/auth/login": {"status": 200, "body": {"token": "valid_token"}},
+        "GET:/users/1": {"status": 200, "body": {"id": 1, "name": "Test User"}}
+    },
+    "auth_failure": {
+        "POST:/auth/login": {"status": 401, "body": {"error": "Invalid credentials"}}
+    },
+    "rate_limited": {
+        "GET:/api/*": {"status": 429, "body": {"error": "Rate limit exceeded"}}
+    },
+    "degraded": {
+        "GET:/api/*": {"delay": 5000, "status": 200}  # 5秒延迟
+    }
+}
+```
+
+---
+
+## 🔴 Observability Setup (MANDATORY)
+
+### 三大支柱
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│                    Observability                            │
+├─────────────────┬─────────────────┬─────────────────────────┤
+│    Logging      │    Metrics      │       Tracing           │
+│  (What happened)│  (How much)     │  (Where/How long)       │
+├─────────────────┼─────────────────┼─────────────────────────┤
+│ 结构化日志       │ RED 指标        │ 分布式追踪              │
+│ 请求 ID 关联     │ Prometheus      │ OpenTelemetry           │
+│ 错误堆栈         │ Grafana         │ Jaeger/Zipkin           │
+└─────────────────┴─────────────────┴─────────────────────────┘
+```
+
+### Structured Logging
+
+```python
+# core/logging.py
+import structlog
+from uuid import uuid4
+
+def setup_logging():
+    structlog.configure(
+        processors=[
+            structlog.processors.TimeStamper(fmt="iso"),
+            structlog.processors.add_log_level,
+            structlog.processors.JSONRenderer()
+        ]
+    )
+
+# middleware/request_logging.py
+@app.middleware("http")
+async def log_requests(request: Request, call_next):
+    request_id = str(uuid4())
+    request.state.request_id = request_id
+
+    logger = structlog.get_logger().bind(
+        request_id=request_id,
+        method=request.method,
+        path=request.url.path
+    )
+
+    start_time = time.time()
+    response = await call_next(request)
+    duration = time.time() - start_time
+
+    logger.info(
+        "request_completed",
+        status_code=response.status_code,
+        duration_ms=round(duration * 1000, 2)
+    )
+
+    response.headers["X-Request-ID"] = request_id
+    return response
+```
+
+### RED Metrics
+
+```python
+# core/metrics.py
+from prometheus_client import Counter, Histogram
+
+# Rate: 请求速率
+REQUEST_COUNT = Counter(
+    'http_requests_total',
+    'Total HTTP requests',
+    ['method', 'endpoint', 'status']
+)
+
+# Errors: 错误率
+ERROR_COUNT = Counter(
+    'http_errors_total',
+    'Total HTTP errors',
+    ['method', 'endpoint', 'error_type']
+)
+
+# Duration: 响应时间
+REQUEST_DURATION = Histogram(
+    'http_request_duration_seconds',
+    'HTTP request duration',
+    ['method', 'endpoint'],
+    buckets=[0.01, 0.05, 0.1, 0.5, 1.0, 5.0]
+)
+
+# 使用
+@app.middleware("http")
+async def metrics_middleware(request: Request, call_next):
+    start = time.time()
+    response = await call_next(request)
+    duration = time.time() - start
+
+    REQUEST_COUNT.labels(
+        method=request.method,
+        endpoint=request.url.path,
+        status=response.status_code
+    ).inc()
+
+    REQUEST_DURATION.labels(
+        method=request.method,
+        endpoint=request.url.path
+    ).observe(duration)
+
+    return response
+```
+
+### Distributed Tracing
+
+```python
+# core/tracing.py
+from opentelemetry import trace
+from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
+from opentelemetry.sdk.trace import TracerProvider
+from opentelemetry.sdk.trace.export import BatchSpanProcessor
+
+def setup_tracing(service_name: str):
+    provider = TracerProvider()
+    processor = BatchSpanProcessor(OTLPSpanExporter())
+    provider.add_span_processor(processor)
+    trace.set_tracer_provider(provider)
+
+tracer = trace.get_tracer(__name__)
+
+# 使用
+async def get_user(user_id: int):
+    with tracer.start_as_current_span("get_user") as span:
+        span.set_attribute("user_id", user_id)
+        user = await user_repo.get(user_id)
+        if not user:
+            span.set_attribute("error", "user_not_found")
+        return user
+```
+
+---
+
+## 🔴 Health Check Endpoints
+
+```python
+# api/health.py
+from fastapi import APIRouter
+from pydantic import BaseModel
+
+router = APIRouter()
+
+class HealthResponse(BaseModel):
+    status: str
+    version: str
+    dependencies: dict
+
+@router.get("/health/live")
+async def liveness():
+    """Kubernetes liveness probe"""
+    return {"status": "ok"}
+
+@router.get("/health/ready")
+async def readiness():
+    """Kubernetes readiness probe"""
+    checks = {
+        "database": await check_database(),
+        "redis": await check_redis(),
+        "external_api": await check_external_api()
+    }
+
+    all_healthy = all(c["healthy"] for c in checks.values())
+    status = "ok" if all_healthy else "degraded"
+
+    return HealthResponse(
+        status=status,
+        version=settings.VERSION,
+        dependencies=checks
+    )
+
+async def check_database():
+    try:
+        await db.execute("SELECT 1")
+        return {"healthy": True, "latency_ms": 5}
+    except Exception as e:
+        return {"healthy": False, "error": str(e)}
+```
+
+---
+
+## 🔴 Deep Logging (from log-analysis)
+
+### 日志级别规范
+
+| 级别 | 用途 | 示例 |
+|------|------|------|
+| **ERROR** | 需要立即处理 | 数据库连接失败、支付失败 |
+| **WARN** | 潜在问题 | 重试成功、降级处理 |
+| **INFO** | 关键业务事件 | 用户登录、订单创建 |
+| **DEBUG** | 调试信息 | 函数参数、中间状态 |
+
+### 敏感信息脱敏
+
+```python
+SENSITIVE_FIELDS = {'password', 'token', 'secret', 'credit_card'}
+
+def mask_sensitive(event_dict):
+    for key in event_dict:
+        if any(s in key.lower() for s in SENSITIVE_FIELDS):
+            event_dict[key] = "***MASKED***"
+    return event_dict
+```
+
+### 日志聚合 (ELK/Loki)
+
+```yaml
+# ELK Stack
+services:
+  elasticsearch:
+    image: elasticsearch:8.11.0
+  logstash:
+    image: logstash:8.11.0
+  kibana:
+    image: kibana:8.11.0
+
+# Loki + Grafana
+services:
+  loki:
+    image: grafana/loki:2.9.0
+  grafana:
+    image: grafana/grafana:latest
+```
+
+### 日志查询
+
+```
+# Kibana KQL
+level: "error" AND service: "api"
+duration_ms > 1000
+
+# Loki LogQL
+{app="api"} | json | level="error"
+sum(rate({app="api"} |= "error" [5m]))
+```
+
+### 错误追踪 (Sentry)
+
+```python
+import sentry_sdk
+from sentry_sdk.integrations.fastapi import FastApiIntegration
+
+sentry_sdk.init(
+    dsn="https://xxx@sentry.io/xxx",
+    integrations=[FastApiIntegration()],
+    traces_sample_rate=0.1,
+)
+```
+
+---
+
+## Agent 路由
+
+| 任务类型 | 调用 Agent/Command |
+|----------|-------------------|
+| API 文档生成 | `agents/api-documenter.md` |
+| Mock Server | `commands/api-mock.md` |
+
+---
+
+## 与 fullstack-developer 集成
+
+### 调用时机
+
+```
+fullstack-developer Phase 5 (测试验证)
+    │
+    ├── API 开发完成?
+    │       ↓
+    │   api-testing-observability
+    │       │
+    │       ├── 契约测试
+    │       ├── 集成测试
+    │       ├── 设置可观测性
+    │       └── 返回: 测试报告 + 覆盖率
+    │
+    └── 继续 Phase 6 (code-review)
+```
+
+### 输出要求
+
+```markdown
+## API Testing 完成报告
+
+### 测试覆盖
+| 类型 | 通过 | 失败 | 覆盖率 |
+|------|------|------|--------|
+| 单元测试 | 45 | 0 | 85% |
+| 集成测试 | 20 | 0 | 100% |
+| 契约测试 | 15 | 0 | 100% |
+
+### 可观测性
+- [ ] 结构化日志: ✅
+- [ ] RED 指标: ✅
+- [ ] 分布式追踪: ✅
+- [ ] Health Check: ✅
+
+### Mock 配置
+- 外部服务 Mock: 3 个
+- 场景覆盖: happy_path, error, degraded
+```
+
+---
+
+## Reference Navigation
+
+| Situation | Read This |
+|-----------|-----------|
+| API 文档 | `agents/api-documenter.md` |
+| Mock Server | `commands/api-mock.md` |
+| API 脚手架 | `../api-scaffolding/SKILL.md` |
+| 测试模式 | `../test-driven-development/SKILL.md` |