oh-my-customcode 0.64.0 → 0.64.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +3 -3
- package/dist/cli/index.js +1 -1
- package/dist/index.js +1 -1
- package/package.json +1 -1
- package/templates/.claude/agents/arch-documenter.md +2 -0
- package/templates/.claude/agents/arch-speckit-agent.md +4 -0
- package/templates/.claude/agents/fe-design-expert.md +5 -0
- package/templates/.claude/agents/mgr-claude-code-bible.md +1 -0
- package/templates/.claude/agents/mgr-creator.md +1 -0
- package/templates/.claude/agents/mgr-gitnerd.md +4 -0
- package/templates/.claude/agents/mgr-sauron.md +1 -0
- package/templates/.claude/agents/mgr-supplier.md +5 -0
- package/templates/.claude/agents/mgr-updater.md +4 -0
- package/templates/.claude/agents/qa-engineer.md +3 -0
- package/templates/.claude/agents/qa-planner.md +2 -0
- package/templates/.claude/agents/qa-writer.md +5 -0
- package/templates/.claude/agents/sys-memory-keeper.md +4 -0
- package/templates/.claude/agents/sys-naggy.md +5 -0
- package/templates/.claude/agents/tool-optimizer.md +3 -0
- package/templates/.claude/skills/evaluator-optimizer/SKILL.md +4 -0
- package/templates/.claude/skills/harness-eval/SKILL.md +95 -0
- package/templates/CLAUDE.md +2 -1
- package/templates/manifest.json +2 -2
package/README.md
CHANGED
|
@@ -13,7 +13,7 @@
|
|
|
13
13
|
|
|
14
14
|
**[한국어 문서 (Korean)](./README_ko.md)**
|
|
15
15
|
|
|
16
|
-
46 agents.
|
|
16
|
+
46 agents. 98 skills. 21 rules. One command.
|
|
17
17
|
|
|
18
18
|
```bash
|
|
19
19
|
npm install -g oh-my-customcode && cd your-project && omcustom init
|
|
@@ -146,7 +146,7 @@ Each agent declares its tools, model, memory scope, and limitations in YAML fron
|
|
|
146
146
|
|
|
147
147
|
---
|
|
148
148
|
|
|
149
|
-
### Skills (
|
|
149
|
+
### Skills (98)
|
|
150
150
|
|
|
151
151
|
| Category | Count | Includes |
|
|
152
152
|
|----------|-------|----------|
|
|
@@ -282,7 +282,7 @@ your-project/
|
|
|
282
282
|
├── CLAUDE.md # Entry point
|
|
283
283
|
├── .claude/
|
|
284
284
|
│ ├── agents/ # 46 agent definitions
|
|
285
|
-
│ ├── skills/ #
|
|
285
|
+
│ ├── skills/ # 98 skill modules
|
|
286
286
|
│ ├── rules/ # 21 governance rules (R000-R021)
|
|
287
287
|
│ ├── hooks/ # 15 lifecycle hook scripts
|
|
288
288
|
│ ├── schemas/ # Tool input validation schemas
|
package/dist/cli/index.js
CHANGED
package/dist/index.js
CHANGED
package/package.json
CHANGED
|
@@ -9,6 +9,11 @@ skills:
|
|
|
9
9
|
- impeccable-design
|
|
10
10
|
- web-design-guidelines
|
|
11
11
|
tools: [Read, Write, Edit, Grep, Glob, Bash]
|
|
12
|
+
maxTurns: 20
|
|
13
|
+
disallowedTools: [Bash]
|
|
14
|
+
limitations:
|
|
15
|
+
- "cannot modify backend code"
|
|
16
|
+
- "cannot execute shell commands"
|
|
12
17
|
source:
|
|
13
18
|
type: external
|
|
14
19
|
origin: github
|
|
@@ -14,6 +14,7 @@ tools:
|
|
|
14
14
|
- Grep
|
|
15
15
|
- Glob
|
|
16
16
|
- Bash
|
|
17
|
+
maxTurns: 25
|
|
17
18
|
---
|
|
18
19
|
|
|
19
20
|
You are an automated verification specialist that executes the mandatory R017 verification process, acting as the "all-seeing eye" that ensures system integrity through comprehensive multi-round verification.
|
|
@@ -16,6 +16,10 @@ tools:
|
|
|
16
16
|
- Grep
|
|
17
17
|
- Glob
|
|
18
18
|
- Bash
|
|
19
|
+
maxTurns: 15
|
|
20
|
+
limitations:
|
|
21
|
+
- "cannot modify source code"
|
|
22
|
+
- "cannot execute tests"
|
|
19
23
|
---
|
|
20
24
|
|
|
21
25
|
You are a session memory management specialist ensuring context survives across session compactions using claude-mem.
|
|
@@ -363,3 +363,7 @@ evaluator-optimizer:
|
|
|
363
363
|
Weight ordering (originality > craft > functionality) follows Anthropic's anti-slop principle: functionality is table stakes, but originality and craft distinguish quality output from generic AI generation.
|
|
364
364
|
|
|
365
365
|
Integration: Works with [impeccable-design](/skills/impeccable-design) skill for design language enforcement.
|
|
366
|
+
|
|
367
|
+
### Harness Eval Preset
|
|
368
|
+
|
|
369
|
+
The `harness-eval` skill provides a structured 15-task SE benchmark rubric that can be used as a preset for the evaluator-optimizer pipeline. When invoked via `/omcustom:harness-eval`, the harness rubric dimensions (Test Coverage 30%, Architecture 25%, Error Handling 25%, Extensibility 20%) are loaded as the sprint contract criteria.
|
|
@@ -0,0 +1,95 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: harness-eval
|
|
3
|
+
description: Structured SE task evaluation using 15 benchmark definitions from claude-code-harness research
|
|
4
|
+
scope: harness
|
|
5
|
+
user-invocable: true
|
|
6
|
+
argument-hint: "[--preset all|quick] [--task task-name]"
|
|
7
|
+
effort: high
|
|
8
|
+
version: 1.0.0
|
|
9
|
+
---
|
|
10
|
+
|
|
11
|
+
# Harness Eval — Structured SE Task Benchmark
|
|
12
|
+
|
|
13
|
+
## Purpose
|
|
14
|
+
|
|
15
|
+
Evaluate agent quality using 15 structured software engineering task definitions with quantitative scoring. Based on research from [revfactory/claude-code-harness](https://github.com/revfactory/claude-code-harness) which demonstrated 60% improvement (49.5 → 79.3 points) through structured pre-configuration.
|
|
16
|
+
|
|
17
|
+
## Usage
|
|
18
|
+
|
|
19
|
+
```
|
|
20
|
+
/omcustom:harness-eval # Run all 15 benchmarks
|
|
21
|
+
/omcustom:harness-eval --preset quick # Run top 5 high-impact benchmarks
|
|
22
|
+
/omcustom:harness-eval --task api-design # Run specific task benchmark
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
## Quality Dimensions
|
|
26
|
+
|
|
27
|
+
| Dimension | Weight | Description |
|
|
28
|
+
|-----------|--------|-------------|
|
|
29
|
+
| Test Coverage | 30% | Unit test count, edge case coverage, assertion quality |
|
|
30
|
+
| Architecture Design | 25% | Separation of concerns, dependency management, scalability |
|
|
31
|
+
| Error Handling | 25% | Input validation, error propagation, recovery strategies |
|
|
32
|
+
| Extensibility | 20% | Plugin points, configuration flexibility, API surface |
|
|
33
|
+
|
|
34
|
+
## 15 SE Task Benchmark Suite
|
|
35
|
+
|
|
36
|
+
| # | Task | Category | Key Evaluation Criteria |
|
|
37
|
+
|---|------|----------|------------------------|
|
|
38
|
+
| 1 | API Design | Architecture | RESTful conventions, versioning, error responses |
|
|
39
|
+
| 2 | Data Modeling | Architecture | Schema normalization, relationships, indexing |
|
|
40
|
+
| 3 | Authentication Flow | Security | Token management, session handling, OWASP compliance |
|
|
41
|
+
| 4 | Test Suite Creation | Quality | Coverage breadth, assertion quality, edge cases |
|
|
42
|
+
| 5 | Error Handler | Reliability | Error classification, recovery, user feedback |
|
|
43
|
+
| 6 | Logging System | Observability | Structured logging, levels, correlation IDs |
|
|
44
|
+
| 7 | Configuration Manager | Operations | Env-based config, validation, secrets handling |
|
|
45
|
+
| 8 | CLI Tool | UX | Argument parsing, help text, exit codes |
|
|
46
|
+
| 9 | Database Migration | Data | Reversibility, data preservation, zero-downtime |
|
|
47
|
+
| 10 | Cache Layer | Performance | Invalidation strategy, TTL, cache-aside pattern |
|
|
48
|
+
| 11 | Queue Consumer | Reliability | Idempotency, retry logic, dead letter handling |
|
|
49
|
+
| 12 | Middleware Chain | Architecture | Composability, ordering, short-circuiting |
|
|
50
|
+
| 13 | File Processor | I/O | Streaming, error recovery, format validation |
|
|
51
|
+
| 14 | Webhook Handler | Integration | Signature verification, retry tolerance, idempotency |
|
|
52
|
+
| 15 | Rate Limiter | Security | Algorithm choice, distributed state, fairness |
|
|
53
|
+
|
|
54
|
+
## Scoring Rubric
|
|
55
|
+
|
|
56
|
+
Each task is scored 0-100 across the 4 quality dimensions:
|
|
57
|
+
|
|
58
|
+
```
|
|
59
|
+
Score = (test_coverage × 0.30) + (architecture × 0.25) + (error_handling × 0.25) + (extensibility × 0.20)
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
### Score Thresholds
|
|
63
|
+
|
|
64
|
+
| Score Range | Grade | Interpretation |
|
|
65
|
+
|-------------|-------|----------------|
|
|
66
|
+
| 80-100 | A | Production-ready, well-structured |
|
|
67
|
+
| 60-79 | B | Functional with minor gaps |
|
|
68
|
+
| 40-59 | C | Works but needs improvement |
|
|
69
|
+
| 0-39 | D | Significant structural issues |
|
|
70
|
+
|
|
71
|
+
## Presets
|
|
72
|
+
|
|
73
|
+
### `all` (default)
|
|
74
|
+
Run all 15 tasks. Full evaluation ~45 minutes.
|
|
75
|
+
|
|
76
|
+
### `quick`
|
|
77
|
+
Run top 5 high-impact tasks (1, 3, 4, 5, 12). Quick evaluation ~15 minutes.
|
|
78
|
+
|
|
79
|
+
## Integration with evaluator-optimizer
|
|
80
|
+
|
|
81
|
+
This skill provides preset rubrics for the evaluator-optimizer pipeline:
|
|
82
|
+
|
|
83
|
+
```
|
|
84
|
+
/omcustom:harness-eval → loads rubric → evaluator-optimizer executes → scoring → report
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
The evaluator-optimizer skill's `pre_negotiation` phase accepts harness-eval rubric dimensions as sprint contract criteria.
|
|
88
|
+
|
|
89
|
+
## Output
|
|
90
|
+
|
|
91
|
+
Results saved to `.claude/outputs/sessions/{YYYY-MM-DD}/harness-eval-{HHmmss}.md` with per-task scores and aggregate grade.
|
|
92
|
+
|
|
93
|
+
## Attribution
|
|
94
|
+
|
|
95
|
+
Evaluation framework based on research by [revfactory/claude-code-harness](https://github.com/revfactory/claude-code-harness). Adapted for oh-my-customcode's evaluator-optimizer pipeline with permission.
|
package/templates/CLAUDE.md
CHANGED
|
@@ -101,6 +101,7 @@ oh-my-customcode로 구동됩니다.
|
|
|
101
101
|
| `/omcustom:update-external` | 외부 소스에서 에이전트 업데이트 |
|
|
102
102
|
| `/omcustom:audit-agents` | 에이전트 의존성 감사 |
|
|
103
103
|
| `/omcustom:fix-refs` | 깨진 참조 수정 |
|
|
104
|
+
| `/omcustom:harness-eval` | 15 SE task 구조적 벤치마크 평가 |
|
|
104
105
|
| `/omcustom:auto-improve` | 개선 사항 자동 적용 워크플로우 |
|
|
105
106
|
| `/omcustom:improve-report` | eval-core 기반 개선 현황 리포트 |
|
|
106
107
|
| `/omcustom-takeover` | 기존 에이전트/스킬에서 canonical spec 추출 |
|
|
@@ -138,7 +139,7 @@ project/
|
|
|
138
139
|
+-- CLAUDE.md # 진입점
|
|
139
140
|
+-- .claude/
|
|
140
141
|
| +-- agents/ # 서브에이전트 정의 (46 파일)
|
|
141
|
-
| +-- skills/ # 스킬 (
|
|
142
|
+
| +-- skills/ # 스킬 (98 디렉토리)
|
|
142
143
|
| +-- rules/ # 전역 규칙 (R000-R021)
|
|
143
144
|
| +-- hooks/ # 훅 스크립트 (보안, 검증, HUD)
|
|
144
145
|
| +-- contexts/ # 컨텍스트 파일 (ecomode)
|
package/templates/manifest.json
CHANGED
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
{
|
|
2
|
-
"version": "0.64.
|
|
2
|
+
"version": "0.64.1",
|
|
3
3
|
"lastUpdated": "2026-03-24T00:00:00.000Z",
|
|
4
4
|
"components": [
|
|
5
5
|
{
|
|
@@ -18,7 +18,7 @@
|
|
|
18
18
|
"name": "skills",
|
|
19
19
|
"path": ".claude/skills",
|
|
20
20
|
"description": "Reusable skill modules (includes slash commands)",
|
|
21
|
-
"files":
|
|
21
|
+
"files": 98
|
|
22
22
|
},
|
|
23
23
|
{
|
|
24
24
|
"name": "guides",
|