@pzy560117/codex-harness 0.1.2 → 0.1.4

package/package-source/docs/codex-harness-engineering/templates/package-assets/skills/error-handling/SKILL.md

@@ -1,245 +0,0 @@
----
-name: error-handling
-description: Error Handling Skill
----
-
-# Error Handling Skill
-
-异常处理、日志记录和监控最佳实践。
-
-## 错误处理原则
-
-### 错误分类
-| 类型 | 示例 | 处理方式 |
-|------|------|----------|
-| 业务错误 | 用户不存在、余额不足 | 返回明确错误码和消息 |
-| 验证错误 | 参数格式错误 | 返回字段级错误详情 |
-| 系统错误 | 数据库连接失败 | 记录日志，返回通用错误 |
-| 未知错误 | 未捕获异常 | 记录完整堆栈，告警 |
-
-### 错误响应格式
-```json
-{
-  "code": 40001,
-  "message": "用户名或密码错误",
-  "details": null,
-  "requestId": "req-abc123",
-  "timestamp": "2024-01-01T12:00:00Z"
-}
-```
-
-### 验证错误响应
-```json
-{
-  "code": 42201,
-  "message": "参数验证失败",
-  "details": [
-    {
-      "field": "email",
-      "message": "邮箱格式不正确"
-    },
-    {
-      "field": "password",
-      "message": "密码长度至少8位"
-    }
-  ]
-}
-```
-
-## 异常处理模式
-
-### 全局异常处理
-```python
-# Python FastAPI
-from fastapi import Request
-from fastapi.responses import JSONResponse
-
-@app.exception_handler(Exception)
-async def global_exception_handler(request: Request, exc: Exception):
-    logger.error(f"Unhandled exception: {exc}", exc_info=True)
-    return JSONResponse(
-        status_code=500,
-        content={
-            "code": 50000,
-            "message": "服务器内部错误",
-            "requestId": request.state.request_id
-        }
-    )
-```
-
-```javascript
-// Node.js Express
-app.use((err, req, res, next) => {
-  logger.error('Unhandled error', {
-    error: err.message,
-    stack: err.stack,
-    requestId: req.id
-  });
-  
-  res.status(500).json({
-    code: 50000,
-    message: '服务器内部错误',
-    requestId: req.id
-  });
-});
-```
-
-### 自定义业务异常
-```python
-class BusinessError(Exception):
-    def __init__(self, code: int, message: str):
-        self.code = code
-        self.message = message
-
-class UserNotFoundError(BusinessError):
-    def __init__(self, user_id: int):
-        super().__init__(40401, f"用户 {user_id} 不存在")
-
-# 使用
-raise UserNotFoundError(user_id=123)
-```
-
-## 日志规范
-
-### 日志级别
-| 级别 | 用途 |
-|------|------|
-| DEBUG | 调试信息，生产环境关闭 |
-| INFO | 关键业务流程记录 |
-| WARN | 潜在问题，不影响功能 |
-| ERROR | 错误，需要关注 |
-| FATAL | 严重错误，服务不可用 |
-
-### 结构化日志
-```json
-{
-  "timestamp": "2024-01-01T12:00:00.000Z",
-  "level": "ERROR",
-  "message": "Failed to process order",
-  "service": "order-service",
-  "requestId": "req-abc123",
-  "userId": 12345,
-  "orderId": "ORD-001",
-  "error": {
-    "type": "PaymentError",
-    "message": "Insufficient balance",
-    "stack": "..."
-  },
-  "duration": 150
-}
-```
-
-### 日志配置
-```javascript
-// Winston 配置
-const logger = winston.createLogger({
-  level: process.env.LOG_LEVEL || 'info',
-  format: winston.format.combine(
-    winston.format.timestamp(),
-    winston.format.json()
-  ),
-  defaultMeta: { service: 'my-service' },
-  transports: [
-    new winston.transports.Console(),
-    new winston.transports.File({ filename: 'error.log', level: 'error' }),
-    new winston.transports.File({ filename: 'combined.log' })
-  ]
-});
-```
-
-## 监控告警
-
-### 关键指标
-```
-错误率 = 错误请求数 / 总请求数
-P99 延迟 = 99% 请求的响应时间
-可用性 = 正常运行时间 / 总时间
-```
-
-### 告警规则
-```yaml
-# Prometheus AlertManager 示例
-groups:
-  - name: api-alerts
-    rules:
-      - alert: HighErrorRate
-        expr: rate(http_requests_total{status=~"5.."}[5m]) / rate(http_requests_total[5m]) > 0.05
-        for: 5m
-        labels:
-          severity: critical
-        annotations:
-          summary: "API 错误率超过 5%"
-          
-      - alert: HighLatency
-        expr: histogram_quantile(0.99, rate(http_request_duration_seconds_bucket[5m])) > 1
-        for: 5m
-        labels:
-          severity: warning
-        annotations:
-          summary: "P99 延迟超过 1 秒"
-```
-
-## 前端错误处理
-
-### 全局错误捕获
-```javascript
-// React Error Boundary
-class ErrorBoundary extends React.Component {
-  state = { hasError: false };
-  
-  static getDerivedStateFromError(error) {
-    return { hasError: true };
-  }
-  
-  componentDidCatch(error, errorInfo) {
-    logger.error('React error', { error, errorInfo });
-  }
-  
-  render() {
-    if (this.state.hasError) {
-      return <ErrorFallback />;
-    }
-    return this.props.children;
-  }
-}
-```
-
-### API 错误处理
-```javascript
-// Axios 拦截器
-axios.interceptors.response.use(
-  response => response,
-  error => {
-    const { response } = error;
-    
-    if (response?.status === 401) {
-      // 跳转登录
-      router.push('/login');
-    } else if (response?.status >= 500) {
-      // 显示通用错误
-      toast.error('服务器错误，请稍后重试');
-    }
-    
-    return Promise.reject(error);
-  }
-);
-```
-
-## 重试策略
-
-```javascript
-// 指数退避重试
-async function retryWithBackoff(fn, maxRetries = 3) {
-  for (let i = 0; i < maxRetries; i++) {
-    try {
-      return await fn();
-    } catch (error) {
-      if (i === maxRetries - 1) throw error;
-      
-      const delay = Math.min(1000 * Math.pow(2, i), 10000);
-      await new Promise(resolve => setTimeout(resolve, delay));
-    }
-  }
-}
-```
-

package/package-source/docs/codex-harness-engineering/templates/package-assets/skills/eval/SKILL.md

@@ -1,125 +0,0 @@
----
-name: eval
-description: Workflow skill from eval.md
----
-
-# Eval Command
-
-Manage eval-driven development workflow.
-
-## Usage
-
-`/eval [define|check|report|list] [feature-name]`
-
-## Define Evals
-
-`/eval define feature-name`
-
-Create a new eval definition:
-
-1. Create `.claude/evals/feature-name.md` with template:
-
-```markdown
-## EVAL: feature-name
-Created: $(date)
-
-### Capability Evals
-- [ ] [Description of capability 1]
-- [ ] [Description of capability 2]
-
-### Regression Evals
-- [ ] [Existing behavior 1 still works]
-- [ ] [Existing behavior 2 still works]
-
-### Success Criteria
-- pass@3 > 90% for capability evals
-- pass^3 = 100% for regression evals
-```
-
-2. Prompt user to fill in specific criteria
-
-## Check Evals
-
-`/eval check feature-name`
-
-Run evals for a feature:
-
-1. Read eval definition from `.claude/evals/feature-name.md`
-2. For each capability eval:
-   - Attempt to verify criterion
-   - Record PASS/FAIL
-   - Log attempt in `.claude/evals/feature-name.log`
-3. For each regression eval:
-   - Run relevant tests
-   - Compare against baseline
-   - Record PASS/FAIL
-4. Report current status:
-
-```
-EVAL CHECK: feature-name
-========================
-Capability: X/Y passing
-Regression: X/Y passing
-Status: IN PROGRESS / READY
-```
-
-## Report Evals
-
-`/eval report feature-name`
-
-Generate comprehensive eval report:
-
-```
-EVAL REPORT: feature-name
-=========================
-Generated: $(date)
-
-CAPABILITY EVALS
-----------------
-[eval-1]: PASS (pass@1)
-[eval-2]: PASS (pass@2) - required retry
-[eval-3]: FAIL - see notes
-
-REGRESSION EVALS
-----------------
-[test-1]: PASS
-[test-2]: PASS
-[test-3]: PASS
-
-METRICS
--------
-Capability pass@1: 67%
-Capability pass@3: 100%
-Regression pass^3: 100%
-
-NOTES
------
-[Any issues, edge cases, or observations]
-
-RECOMMENDATION
---------------
-[SHIP / NEEDS WORK / BLOCKED]
-```
-
-## List Evals
-
-`/eval list`
-
-Show all eval definitions:
-
-```
-EVAL DEFINITIONS
-================
-feature-auth      [3/5 passing] IN PROGRESS
-feature-search    [5/5 passing] READY
-feature-export    [0/4 passing] NOT STARTED
-```
-
-## Arguments
-
-$ARGUMENTS:
-- `define <name>` - Create new eval definition
-- `check <name>` - Run and check evals
-- `report <name>` - Generate full report
-- `list` - Show all evals
-- `clean` - Remove old eval logs (keeps last 10 runs)

package/package-source/docs/codex-harness-engineering/templates/package-assets/skills/eval-harness/SKILL.md

@@ -1,226 +0,0 @@
----
-name: eval-harness
-description: A formal evaluation framework implementing eval-driven development principles.
----
-
-# Eval Harness Skill
-
-A formal evaluation framework for Claude Code sessions, implementing eval-driven development (EDD) principles.
-
-## Philosophy
-
-Eval-Driven Development treats evals as the "unit tests of AI development":
-- Define expected behavior BEFORE implementation
-- Run evals continuously during development
-- Track regressions with each change
-- Use pass@k metrics for reliability measurement
-
-## Eval Types
-
-### Capability Evals
-Test if Claude can do something it couldn't before:
-```markdown
-[CAPABILITY EVAL: feature-name]
-Task: Description of what Claude should accomplish
-Success Criteria:
-  - [ ] Criterion 1
-  - [ ] Criterion 2
-  - [ ] Criterion 3
-Expected Output: Description of expected result
-```
-
-### Regression Evals
-Ensure changes don't break existing functionality:
-```markdown
-[REGRESSION EVAL: feature-name]
-Baseline: SHA or checkpoint name
-Tests:
-  - existing-test-1: PASS/FAIL
-  - existing-test-2: PASS/FAIL
-  - existing-test-3: PASS/FAIL
-Result: X/Y passed (previously Y/Y)
-```
-
-## Grader Types
-
-### 1. Code-Based Grader
-Deterministic checks using code:
-```bash
-# Check if file contains expected pattern
-grep -q "export function handleAuth" src/auth.ts && echo "PASS" || echo "FAIL"
-
-# Check if tests pass
-npm test -- --testPathPattern="auth" && echo "PASS" || echo "FAIL"
-
-# Check if build succeeds
-npm run build && echo "PASS" || echo "FAIL"
-```
-
-### 2. Model-Based Grader
-Use Claude to evaluate open-ended outputs:
-```markdown
-[MODEL GRADER PROMPT]
-Evaluate the following code change:
-1. Does it solve the stated problem?
-2. Is it well-structured?
-3. Are edge cases handled?
-4. Is error handling appropriate?
-
-Score: 1-5 (1=poor, 5=excellent)
-Reasoning: [explanation]
-```
-
-### 3. Human Grader
-Flag for manual review:
-```markdown
-[HUMAN REVIEW REQUIRED]
-Change: Description of what changed
-Reason: Why human review is needed
-Risk Level: LOW/MEDIUM/HIGH
-```
-
-## Metrics
-
-### pass@k
-"At least one success in k attempts"
-- pass@1: First attempt success rate
-- pass@3: Success within 3 attempts
-- Typical target: pass@3 > 90%
-
-### pass^k
-"All k trials succeed"
-- Higher bar for reliability
-- pass^3: 3 consecutive successes
-- Use for critical paths
-
-## Eval Workflow
-
-### 1. Define (Before Coding)
-```markdown
-## EVAL DEFINITION: feature-xyz
-
-### Capability Evals
-1. Can create new user account
-2. Can validate email format
-3. Can hash password securely
-
-### Regression Evals
-1. Existing login still works
-2. Session management unchanged
-3. Logout flow intact
-
-### Success Metrics
-- pass@3 > 90% for capability evals
-- pass^3 = 100% for regression evals
-```
-
-### 2. Implement
-Write code to pass the defined evals.
-
-### 3. Evaluate
-```bash
-# Run capability evals
-[Run each capability eval, record PASS/FAIL]
-
-# Run regression evals
-npm test -- --testPathPattern="existing"
-
-# Generate report
-```
-
-### 4. Report
-```markdown
-EVAL REPORT: feature-xyz
-========================
-
-Capability Evals:
-  create-user:     PASS (pass@1)
-  validate-email:  PASS (pass@2)
-  hash-password:   PASS (pass@1)
-  Overall:         3/3 passed
-
-Regression Evals:
-  login-flow:      PASS
-  session-mgmt:    PASS
-  logout-flow:     PASS
-  Overall:         3/3 passed
-
-Metrics:
-  pass@1: 67% (2/3)
-  pass@3: 100% (3/3)
-
-Status: READY FOR REVIEW
-```
-
-## Integration Patterns
-
-### Pre-Implementation
-```
-/eval define feature-name
-```
-Creates eval definition file at `.claude/evals/feature-name.md`
-
-### During Implementation
-```
-/eval check feature-name
-```
-Runs current evals and reports status
-
-### Post-Implementation
-```
-/eval report feature-name
-```
-Generates full eval report
-
-## Eval Storage
-
-Store evals in project:
-```
-.claude/
-  evals/
-    feature-xyz.md      # Eval definition
-    feature-xyz.log     # Eval run history
-    baseline.json       # Regression baselines
-```
-
-## Best Practices
-
-1. **Define evals BEFORE coding** - Forces clear thinking about success criteria
-2. **Run evals frequently** - Catch regressions early
-3. **Track pass@k over time** - Monitor reliability trends
-4. **Use code graders when possible** - Deterministic > probabilistic
-5. **Human review for security** - Never fully automate security checks
-6. **Keep evals fast** - Slow evals don't get run
-7. **Version evals with code** - Evals are first-class artifacts
-
-## Example: Adding Authentication
-
-```markdown
-## EVAL: add-authentication
-
-### Phase 1: Define (10 min)
-Capability Evals:
-- [ ] User can register with email/password
-- [ ] User can login with valid credentials
-- [ ] Invalid credentials rejected with proper error
-- [ ] Sessions persist across page reloads
-- [ ] Logout clears session
-
-Regression Evals:
-- [ ] Public routes still accessible
-- [ ] API responses unchanged
-- [ ] Database schema compatible
-
-### Phase 2: Implement (varies)
-[Write code]
-
-### Phase 3: Evaluate
-Run: /eval check add-authentication
-
-### Phase 4: Report
-EVAL REPORT: add-authentication
-==============================
-Capability: 5/5 passed (pass@3: 100%)
-Regression: 3/3 passed (pass^3: 100%)
-Status: SHIP IT
-```

package/package-source/docs/codex-harness-engineering/templates/package-assets/skills/example-skill/README.md

@@ -1,37 +0,0 @@
-# Example Skill
-
-这是一个示例skill，展示了完整的skill文件结构。
-
-## 目录结构
-
-```
-example-skill/
-├── SKILL.md              # 主要技能文档（必需）
-├── README.md             # 本文件
-├── scripts/              # 辅助脚本目录
-│   └── helper.py         # 示例辅助脚本
-├── examples/             # 使用示例
-│   └── usage.md          # 使用示例文档
-└── resources/            # 资源文件
-    └── config.json       # 配置模板
-```
-
-## 使用方法
-
-1. 查看 `SKILL.md` 了解技能详细说明
-2. 参考 `examples/usage.md` 查看使用示例
-3. 使用 `scripts/` 目录中的辅助工具
-4. 根据 `resources/` 中的模板进行配置
-
-## 快速开始
-
-```bash
-# 查看主要文档
-cat SKILL.md
-
-# 运行示例脚本
-python scripts/helper.py
-
-# 查看使用示例
-cat examples/usage.md
-```