@nurix/etna 0.19.32 → 0.19.34
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/data/harness-index.json +5 -1
- package/data/harnesses/code/harness.json +2 -2
- package/data/harnesses/code/rules/changelog.md +5 -23
- package/data/harnesses/code/rules/database.md +77 -20
- package/data/harnesses/code/rules/express.md +7 -2
- package/data/harnesses/code/rules/frontend.md +14 -17
- package/data/harnesses/code/rules/general.md +47 -47
- package/data/harnesses/code/rules/nomenclature.md +25 -25
- package/data/harnesses/code/rules/pnpm.md +31 -28
- package/data/harnesses/code/rules/testing.md +11 -13
- package/data/harnesses/delivery/.prettierignore +29 -0
- package/data/harnesses/delivery/.prettierrc.json +3 -0
- package/data/harnesses/delivery/.vscode/settings.json +19 -0
- package/data/harnesses/delivery/CLAUDE.md +123 -0
- package/data/harnesses/delivery/agents/brainstorm.md +144 -0
- package/data/harnesses/delivery/agents/data-generator.md +23 -0
- package/data/harnesses/delivery/agents/eval-runner.md +23 -0
- package/data/harnesses/delivery/agents/judge.md +23 -0
- package/data/harnesses/delivery/agents/lld-mode.md +56 -0
- package/data/harnesses/delivery/agents/maintenance.md +229 -0
- package/data/harnesses/delivery/agents/prompt-iterator.md +23 -0
- package/data/harnesses/delivery/agents/red-team.md +23 -0
- package/data/harnesses/delivery/agents/regression-runner.md +28 -0
- package/data/harnesses/delivery/agents/retro.md +46 -0
- package/data/harnesses/delivery/codex/agents/brainstorm.toml +142 -0
- package/data/harnesses/delivery/codex/agents/data-generator.toml +22 -0
- package/data/harnesses/delivery/codex/agents/eval-runner.toml +22 -0
- package/data/harnesses/delivery/codex/agents/judge.toml +22 -0
- package/data/harnesses/delivery/codex/agents/lld-mode.toml +52 -0
- package/data/harnesses/delivery/codex/agents/maintenance.toml +228 -0
- package/data/harnesses/delivery/codex/agents/prompt-iterator.toml +22 -0
- package/data/harnesses/delivery/codex/agents/red-team.toml +22 -0
- package/data/harnesses/delivery/codex/agents/regression-runner.toml +27 -0
- package/data/harnesses/delivery/codex/agents/retro.toml +45 -0
- package/data/harnesses/delivery/harness.json +11 -0
- package/data/harnesses/delivery/hooks/generate-ownership-index.mjs +148 -0
- package/data/harnesses/delivery/hooks/hooks.json +16 -0
- package/data/harnesses/delivery/rules/changelog.md +19 -0
- package/data/harnesses/delivery/rules/evals.md +23 -0
- package/data/harnesses/delivery/rules/model-tiers.md +18 -0
- package/data/harnesses/delivery/rules/nomenclature.md +34 -0
- package/data/harnesses/delivery/rules/prompting.md +23 -0
- package/data/harnesses/delivery/skills/brainstorm/README.md +56 -0
- package/data/harnesses/delivery/skills/brainstorm/SKILL.md +173 -0
- package/data/harnesses/delivery/skills/brainstorm/references/doctrine.md +98 -0
- package/data/harnesses/delivery/skills/brainstorm/references/harvest.md +86 -0
- package/data/harnesses/delivery/skills/brainstorm/references/session-schema.md +186 -0
- package/data/harnesses/delivery/skills/brainstorm/scripts/detector.mjs +460 -0
- package/data/harnesses/delivery/skills/cost-report/SKILL.md +24 -0
- package/data/harnesses/delivery/skills/documentation/SKILL.md +129 -0
- package/data/harnesses/delivery/skills/documentation/references/examples.md +279 -0
- package/data/harnesses/delivery/skills/documentation/references/frontmatter.md +44 -0
- package/data/harnesses/delivery/skills/documentation/references/style-guide.md +340 -0
- package/data/harnesses/delivery/skills/documentation/references/untemplated.md +38 -0
- package/data/harnesses/delivery/skills/documentation/scripts/doc-lint.mjs +140 -0
- package/data/harnesses/delivery/skills/documentation/templates/Readme.md +31 -0
- package/data/harnesses/delivery/skills/documentation/templates/api-reference.md +20 -0
- package/data/harnesses/delivery/skills/documentation/templates/architecture-decision.md +121 -0
- package/data/harnesses/delivery/skills/documentation/templates/backlog.md +33 -0
- package/data/harnesses/delivery/skills/documentation/templates/changelog-entry.md +57 -0
- package/data/harnesses/delivery/skills/documentation/templates/changelog-index.md +41 -0
- package/data/harnesses/delivery/skills/documentation/templates/docs-readme.md +72 -0
- package/data/harnesses/delivery/skills/documentation/templates/domain-model.md +30 -0
- package/data/harnesses/delivery/skills/documentation/templates/environment-registry.md +15 -0
- package/data/harnesses/delivery/skills/documentation/templates/feature.md +149 -0
- package/data/harnesses/delivery/skills/documentation/templates/glossary.md +11 -0
- package/data/harnesses/delivery/skills/documentation/templates/memory-main.md +25 -0
- package/data/harnesses/delivery/skills/documentation/templates/migration-runbook.md +34 -0
- package/data/harnesses/delivery/skills/documentation/templates/non-goals.md +11 -0
- package/data/harnesses/delivery/skills/documentation/templates/operations-procedure.md +36 -0
- package/data/harnesses/delivery/skills/documentation/templates/plan.md +64 -0
- package/data/harnesses/delivery/skills/documentation/templates/protected-files.md +37 -0
- package/data/harnesses/delivery/skills/documentation/templates/root-readme.md +104 -0
- package/data/harnesses/delivery/skills/documentation/templates/security-overview.md +27 -0
- package/data/harnesses/delivery/skills/documentation/templates/structure.md +65 -0
- package/data/harnesses/delivery/skills/documentation/templates/system-overview.md +32 -0
- package/data/harnesses/delivery/skills/documentation/templates/testing-strategy.md +29 -0
- package/data/harnesses/delivery/skills/escalate/SKILL.md +25 -0
- package/data/harnesses/delivery/skills/prompt-eval/SKILL.md +22 -0
- package/data/harnesses/delivery/skills/prompt-library/SKILL.md +36 -0
- package/data/harnesses/delivery/skills/route/SKILL.md +21 -0
- package/data/harnesses/etna/harness.json +2 -2
- package/data/harnesses/etna/rules/changelog.md +5 -23
- package/data/harnesses/etna/rules/general.md +47 -47
- package/data/harnesses/etna/rules/kb.md +4 -11
- package/data/harnesses/etna/rules/nomenclature.md +25 -25
- package/data/harnesses/etna/rules/testing.md +11 -13
- package/data/harnesses/hr/harness.json +2 -2
- package/data/harnesses/hr/rules/code-eval-batcher.md +24 -28
- package/package.json +1 -1
|
@@ -1,61 +1,61 @@
|
|
|
1
1
|
# General Standards
|
|
2
2
|
|
|
3
|
-
Runtime versions
|
|
3
|
+
Runtime versions, the canonical package list, and GitHub Actions pins — the workspace baseline. Web-frontend/styling conventions are [`frontend.md`](frontend.md); the Express server and the API response envelope are [`express.md`](express.md).
|
|
4
4
|
|
|
5
5
|
## Runtime Versions
|
|
6
6
|
|
|
7
7
|
- **Node.js:** 24 LTS
|
|
8
|
-
- **Package manager:** detect from lockfile
|
|
8
|
+
- **Package manager:** detect from the lockfile — **never** hardcode one.
|
|
9
9
|
- **PostgreSQL:** 17 · **Redis:** 7.x (latest)
|
|
10
10
|
|
|
11
|
-
##
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
| Package
|
|
16
|
-
|
|
|
17
|
-
| react
|
|
18
|
-
| vite
|
|
19
|
-
| tailwindcss
|
|
20
|
-
|
|
|
21
|
-
| react-router-dom
|
|
22
|
-
|
|
|
23
|
-
| react-redux
|
|
24
|
-
| axios
|
|
25
|
-
| date-fns
|
|
26
|
-
|
|
|
27
|
-
|
|
|
28
|
-
| motion
|
|
29
|
-
| sonner
|
|
30
|
-
| react-window
|
|
31
|
-
| nanoid
|
|
32
|
-
| express
|
|
33
|
-
| drizzle-orm
|
|
34
|
-
| drizzle-kit
|
|
35
|
-
| zod
|
|
36
|
-
| jose
|
|
37
|
-
| microdiff
|
|
38
|
-
| react18-json-view
|
|
39
|
-
|
|
|
40
|
-
| shiki
|
|
41
|
-
| lucide-react
|
|
42
|
-
|
|
|
43
|
-
| cmdk
|
|
44
|
-
|
|
|
45
|
-
| localforage
|
|
46
|
-
| react-markdown
|
|
47
|
-
| remark-gfm
|
|
48
|
-
| fast-deep-equal
|
|
49
|
-
|
|
|
50
|
-
| pino
|
|
11
|
+
## Packages
|
|
12
|
+
|
|
13
|
+
**Only** these — no alternatives.
|
|
14
|
+
|
|
15
|
+
| Package | Version | Use | Don't use |
|
|
16
|
+
| --- | --- | --- | --- |
|
|
17
|
+
| `react` | ^19.2.3 | UI components | — |
|
|
18
|
+
| `vite` | ^8.0.0 | Build tool, dev server | `webpack`, CRA |
|
|
19
|
+
| `tailwindcss` | ^4.1.18 | Styling | `styled-components` |
|
|
20
|
+
| `@vitejs/plugin-react-swc` | ^4.2.2 | JSX compilation | `@vitejs/plugin-react` |
|
|
21
|
+
| `react-router-dom` | ^7.11.0 | Routing | — |
|
|
22
|
+
| `@reduxjs/toolkit` | ^2.11.2 | State management | `zustand`, `jotai` |
|
|
23
|
+
| `react-redux` | ^9.2.0 | Redux React bindings | — |
|
|
24
|
+
| `axios` | ^1.13.2 | HTTP requests | `fetch` |
|
|
25
|
+
| `date-fns` | ^4.1.0 | Date formatting | `moment`, `dayjs` |
|
|
26
|
+
| `@dnd-kit/core` | ^6.3.1 | Drag and drop | `react-beautiful-dnd` |
|
|
27
|
+
| `@xyflow/react` | ^12.11.0 | Node-based flow charts | `reactflow`, `react-flow-renderer` |
|
|
28
|
+
| `motion` | ^12.23.26 | Animations | `react-spring` |
|
|
29
|
+
| `sonner` | ^2.0.7 | Toast notifications | `react-toastify` |
|
|
30
|
+
| `react-window` | ^2.2.3 | List virtualization | `react-virtualized` |
|
|
31
|
+
| `nanoid` | ^6.0.0 | ID generation | `uuid` |
|
|
32
|
+
| `express` | ^5.2.1 | HTTP server | `fastify`, `koa` |
|
|
33
|
+
| `drizzle-orm` | ^0.45.2 | Database ORM | `prisma`, `sequelize`, `typeorm` |
|
|
34
|
+
| `drizzle-kit` | ^0.31.10 | Schema tooling | — |
|
|
35
|
+
| `zod` | ^4.2.1 | Schema validation | `yup`, `joi` |
|
|
36
|
+
| `jose` | ^6.1.3 | JWT handling | `jsonwebtoken` |
|
|
37
|
+
| `microdiff` | ^1.5.0 | JSON object diffing | `deep-diff`, `just-diff` |
|
|
38
|
+
| `react18-json-view` | ^0.2.9 | JSON tree view | `react-json-view` |
|
|
39
|
+
| `@tiptap/react` | ^3.11.1 | Rich text editor | `slate`, `quill`, `draft-js` |
|
|
40
|
+
| `shiki` | ^4.3.1 | Syntax highlighting | `prism`, `highlight.js` |
|
|
41
|
+
| `lucide-react` | ^1.26.0 | Icons | `react-icons`, `heroicons` |
|
|
42
|
+
| `@pdfme/generator` | ^6.1.12 | PDF generation | `jspdf`, `pdfkit` |
|
|
43
|
+
| `cmdk` | ^1.1.1 | Command palette | `kbar`, `ninja-keys` |
|
|
44
|
+
| `@monaco-editor/react` | ^4.7.0 | Code editor | `codemirror`, `ace` |
|
|
45
|
+
| `localforage` | ^1.10.0 | Client-side storage | `idb`, `dexie` |
|
|
46
|
+
| `react-markdown` | ^10.1.0 | Markdown rendering | `marked`, `markdown-it` |
|
|
47
|
+
| `remark-gfm` | ^4.0.1 | GFM (tables, tasklists) | `remark-github`, custom |
|
|
48
|
+
| `fast-deep-equal` | ^3.1.3 | Deep equality checks | `lodash.isEqual` |
|
|
49
|
+
| `@ngneat/falso` | latest | Fake data generation | `@faker-js/faker`, `chance` |
|
|
50
|
+
| `pino` | ^10.1.0 | Logging | `winston` |
|
|
51
51
|
|
|
52
52
|
## GitHub Actions
|
|
53
53
|
|
|
54
54
|
Pin the major, and check it against the action's releases — a major behind still runs, so a stale pin never fails, it only warns.
|
|
55
55
|
|
|
56
|
-
| Action
|
|
57
|
-
|
|
|
58
|
-
| actions/checkout
|
|
59
|
-
| actions/setup-node
|
|
56
|
+
| Action | Version | Use |
|
|
57
|
+
| --- | --- | --- |
|
|
58
|
+
| `actions/checkout` | v7 | Repo checkout |
|
|
59
|
+
| `actions/setup-node` | v7 | Node toolchain + npm cache |
|
|
60
60
|
|
|
61
|
-
Set `node-version` from the Runtime Versions block above,
|
|
61
|
+
Set `node-version` from the Runtime Versions block above, **never** from the runner's default.
|
|
@@ -1,13 +1,6 @@
|
|
|
1
|
-
#
|
|
1
|
+
# Knowledge-Base Folders
|
|
2
2
|
|
|
3
|
-
Conventions for any folder that is majority markdown — `docs/`, `research/`, an Obsidian vault,
|
|
3
|
+
Conventions for any folder that is majority markdown — `docs/`, `research/`, an Obsidian vault, any knowledge base. Folder shape and linking only.
|
|
4
4
|
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
### 1. Every topic gets a `Readme.md`
|
|
8
|
-
|
|
9
|
-
When you start documenting a topic, create `Readme.md` as the folder's landing page. Don't use `index.md` or unscoped filenames — `Readme.md` is what readers and tools (GitHub, Obsidian) look for first, and it makes the folder self-introducing.
|
|
10
|
-
|
|
11
|
-
### 2. Always link documents to each other
|
|
12
|
-
|
|
13
|
-
When one document references an idea, concept, or topic covered in another, link to it. Wikilinks (`[[other-doc]]`) inside Obsidian-native vaults; relative markdown links (`[label](../other-doc.md)`) elsewhere. If you mention it, you link it — that's how the folder becomes a knowledge graph instead of a pile of files.
|
|
5
|
+
- **Every topic folder carries a `Readme.md`**, created with the topic's first document — **never** `index.md` or an unscoped filename. `Readme.md` is what GitHub and Obsidian surface first, so every folder introduces itself.
|
|
6
|
+
- **Every mention is a link** — wikilinks (`[[other-doc]]`) inside Obsidian-native vaults, relative markdown links (`[label](../other-doc.md)`) everywhere else. Unlinked mentions decay a knowledge graph into a pile of files.
|
|
@@ -1,34 +1,34 @@
|
|
|
1
|
-
# Nomenclature
|
|
1
|
+
# Nomenclature
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
Naming and casing for code — identifiers, files, folders. The rationale throughout is consistency: one concept, one name, one casing shape, in every repo that loads this rule. Database naming is not here — see § Database.
|
|
4
4
|
|
|
5
|
-
|
|
6
|
-
- Names should explain why something exists and how it's used
|
|
7
|
-
- Favor clarity over brevity
|
|
8
|
-
- Maintain consistency within the codebase
|
|
9
|
-
- Avoid abbreviations unless they're universally understood
|
|
5
|
+
Names state intent in full words — clarity over brevity; no abbreviations unless universally understood (`id`, `url` ✅ · `cfg`, `btn`, `idx` ❌).
|
|
10
6
|
|
|
11
|
-
##
|
|
7
|
+
## Casing
|
|
12
8
|
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
-
|
|
19
|
-
|
|
20
|
-
- Interfaces & Types : Use Consider prefixing with I for interfaces (if preferred). Examples: `IUserData`, `IConfigOptions`, `AuthResponse`, `RequestParams`
|
|
21
|
-
- Promises & Async Functions : Use Consider using future tense or suffixes like Async. Examples: `fetchUserData`, `getUserProfileAsync`, `loadConfigAsync`, `processPayment`
|
|
22
|
-
- File & Folder Structure : Use lowercase with descriptive names. Group by feature or function type. Example: `auth/login_service.js`
|
|
9
|
+
| Casing | Used for | Example |
|
|
10
|
+
| --- | --- | --- |
|
|
11
|
+
| `camelCase` | variables, functions, methods, properties; a module file exporting no component | `calculateTotal`, `sessionClient.ts` |
|
|
12
|
+
| `PascalCase` | classes, type definitions, components; the file exporting one | `PaymentProcessor`, `AppsListPage.tsx` |
|
|
13
|
+
| `UPPER_SNAKE_CASE` | constants, environment variables | `MAX_RETRY_COUNT` |
|
|
14
|
+
| `kebab-case` | URLs, CSS class names, HTML IDs, multi-word folders | `user-profile`, `workspace-provisioning/` |
|
|
15
|
+
| `snake_case` | database tables, columns, indexes — § Database | `created_at` |
|
|
23
16
|
|
|
24
|
-
|
|
17
|
+
**A file exporting one component carries that component's name exactly**, so an import names the file it came from; a single-word folder is plain lowercase (`lib`, `surfaces`). ❌ Never `snake_case` a file or folder — that casing belongs to the database.
|
|
25
18
|
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
19
|
+
## Identifier shape
|
|
20
|
+
|
|
21
|
+
| Identifier | Shape | Example |
|
|
22
|
+
| --- | --- | --- |
|
|
23
|
+
| Variable | noun phrase | `paymentStatus` |
|
|
24
|
+
| Function / method | verb phrase naming the action | `validateInput` |
|
|
25
|
+
| Boolean | `is` / `has` / `can` / `should` prefix — **never** a bare adjective | `hasPermission` |
|
|
26
|
+
| Array / collection | plural noun | `errorMessages` |
|
|
27
|
+
| Class / component | singular noun phrase | `RequestHandler` |
|
|
28
|
+
| Event handler | `handle` or `on` prefix | `handleSubmit`, `onPageLoad` |
|
|
29
|
+
| Interface | `I` prefix acceptable; type aliases stay plain | `IConfigOptions`, `AuthResponse` |
|
|
30
|
+
| Async function | verb phrase; `Async` suffix acceptable | `loadConfigAsync` |
|
|
31
31
|
|
|
32
32
|
## Database
|
|
33
33
|
|
|
34
|
-
Table, column, index, and key naming for relational schemas — the `id` / `slug` / `label` vocabulary, junction `_map` tables, and Drizzle mirrors — lives in [`database.md`](database.md),
|
|
34
|
+
Table, column, index, and key naming for relational schemas — the `id` / `slug` / `label` vocabulary, junction `_map` tables, and Drizzle mirrors — lives in [`database.md`](database.md), which routes general code naming back here.
|
|
@@ -1,25 +1,23 @@
|
|
|
1
1
|
# Testing
|
|
2
2
|
|
|
3
|
-
Framework choice is never ad hoc and never the user's problem mid-task: detect the stack, use its row
|
|
3
|
+
Test technology and placement — framework per stack, suite layout, mocking policy; the suite-first policy (what gets tests, run-and-report discipline) is the harness `CLAUDE.md` Tests guardrail, not here. Framework choice is **never** ad hoc and never the user's problem mid-task: detect the stack, use its row; if the repo already has a runner, that runner wins — **never** introduce a second framework beside it.
|
|
4
4
|
|
|
5
5
|
## Frameworks by stack
|
|
6
6
|
|
|
7
|
-
No alternatives. Just use these.
|
|
8
|
-
|
|
9
7
|
| Stack | Unit / integration | Component | E2E | HTTP mocking | Don't use |
|
|
10
8
|
| --- | --- | --- | --- | --- | --- |
|
|
11
|
-
| Node + Vite / React (web) | vitest |
|
|
12
|
-
| Express APIs | vitest | — | supertest against the app instance | msw (outbound calls) | jest
|
|
13
|
-
| React Native / Expo | jest via the jest-expo preset |
|
|
14
|
-
| Cloudflare Workers | vitest +
|
|
9
|
+
| Node + Vite / React (web) | `vitest` | `@testing-library/react` (+ `@testing-library/user-event`, `happy-dom`) | `@playwright/test` | `msw` | `jest`, `mocha`, `jasmine`, `cypress`, `enzyme` |
|
|
10
|
+
| Express APIs | `vitest` | — | `supertest` against the app instance | `msw` (outbound calls) | `jest`, `chai-http` |
|
|
11
|
+
| React Native / Expo | `jest` via the `jest-expo` preset | `@testing-library/react-native` | `maestro` | `msw` | `detox`, `enzyme` |
|
|
12
|
+
| Cloudflare Workers | `vitest` + `@cloudflare/vitest-pool-workers` | — | `@playwright/test` against `wrangler dev` | `msw` | `jest` |
|
|
15
13
|
| Rust | `cargo test` (built-in) | — | — | — | — |
|
|
16
14
|
|
|
17
|
-
|
|
15
|
+
Test packages install at `latest` — they sit outside [`general.md`](general.md)'s pinned table. Fake data comes from `@ngneat/falso` — **never** `@faker-js/faker` / `chance` (its row there).
|
|
18
16
|
|
|
19
17
|
## Conventions
|
|
20
18
|
|
|
21
|
-
- **Suite, not scripts.** Tests live in the canonical test tree — `tests/` mirroring `src/`, or the convention the repo already uses
|
|
22
|
-
- **Wire the canonical command.** `"test"` in package.json runs the whole unit suite (`vitest run`); e2e
|
|
23
|
-
- **Unit and E2E trees stay separate.** Vitest/Jest suites in `tests/`; Playwright/Maestro suites in `e2e/` with their own config — never mixed into the unit runner's glob.
|
|
24
|
-
- **No test doubles for what you own.** Prefer real instances — in-memory SQLite for drizzle, the actual Express app via supertest — over mocks; mock only true externals (network via msw
|
|
25
|
-
- **Determinism.** No real network, no real time, no shared mutable state between tests; every test must pass alone and in any order.
|
|
19
|
+
- **Suite, not scripts.** Tests live in the canonical test tree — `tests/` mirroring `src/`, or the convention the repo already uses — named `<subject>.test.ts`; shared fixtures, factories, and helpers in `tests/helpers/` (or the suite's existing shared home).
|
|
20
|
+
- **Wire the canonical command.** `"test"` in `package.json` runs the whole unit suite (`vitest run`); e2e under `"test:e2e"`. A repo where `pnpm test` / `npm test` doesn't run the suite is misconfigured — fix the wiring, **never** route around it.
|
|
21
|
+
- **Unit and E2E trees stay separate.** Vitest/Jest suites in `tests/`; Playwright/Maestro suites in `e2e/` with their own config — **never** mixed into the unit runner's glob.
|
|
22
|
+
- **No test doubles for what you own.** Prefer real instances — in-memory SQLite for drizzle, the actual Express app via `supertest` — over mocks; mock **only** true externals (network via `msw`, time via the runner's fake timers).
|
|
23
|
+
- **Determinism.** No real network, no real time, no shared mutable state between tests; every test **must** pass alone and in any order.
|
|
@@ -1,45 +1,45 @@
|
|
|
1
1
|
# Code-Eval Batcher
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
This is distinct from `interview-eval` (which scores how a person *steered a session*). This rule scores the *project's code and product*, across many projects, and ranks them.
|
|
3
|
+
Evaluating and ranking **multiple code projects** — candidate take-homes, submissions, a folder of repos — against the production-readiness + product-quality rubric. The main agent drives this directly: it can dispatch sub-agents in parallel and a sub-agent cannot, so this is a rule to perform, not an agent to delegate to. A single project does **not** need this rule — invoke the `evaluator` skill directly. Scoring how a person *steered a session* is `interview-eval`, not this rule.
|
|
6
4
|
|
|
7
5
|
## Division of labor
|
|
8
6
|
|
|
9
|
-
Per-project scoring — the rubric, gates, absolute banding, the 8 sub-criteria, every recipe — belongs to the **`evaluator` skill** (`.claude/skills/evaluator/`)
|
|
7
|
+
Per-project scoring — the rubric, gates, absolute banding, the 8 sub-criteria, every recipe — belongs to the **`evaluator` skill** (`.claude/skills/evaluator/`); never re-derive it here. This rule owns only what the skill deliberately doesn't: **enumeration, parallel dispatch, weighting, and the cross-project rollup.** **Never re-band a project against the others** — cohort-relative banding is the anti-pattern this design removed; the only cross-project levers live in the weights block.
|
|
10
8
|
|
|
11
9
|
## Procedure
|
|
12
10
|
|
|
13
11
|
| # | Step |
|
|
14
12
|
| --- | --- |
|
|
15
|
-
| 0 | **Preflight once:** `bash .claude/skills/evaluator/preflight.sh`. Halt on non-zero and report the missing tools —
|
|
16
|
-
| 1 | **Enumerate** the projects →
|
|
17
|
-
| 2 | **Dispatch in parallel — rolling window of 4.**
|
|
18
|
-
| 3 | **Collect** every `<out_dir>/*/scorecard.yaml`. Re-dispatch
|
|
13
|
+
| 0 | **Preflight once:** `bash .claude/skills/evaluator/preflight.sh`. Halt on non-zero and report the missing tools — never score with degraded signals. |
|
|
14
|
+
| 1 | **Enumerate** the projects → local paths, one slug each (its dir name). Choose an `out_dir` (default `./code-eval-out`). |
|
|
15
|
+
| 2 | **Dispatch in parallel — rolling window of 4.** Per project: `Agent(subagent_type="general-purpose", run_in_background=True, …)` instructing it to invoke the `evaluator` skill on that one path, write the workspace to `<out_dir>/<slug>/`, validate it (`python3 .claude/skills/evaluator/validate_scorecard.py <out_dir>/<slug>/scorecard.yaml`), and return the workspace path + the scorecard's `normalized`. Fire a replacement the moment a slot frees. |
|
|
16
|
+
| 3 | **Collect** every `<out_dir>/*/scorecard.yaml`. Re-dispatch a validation failure once; on the second failure mark the project `VALIDATION_FAILED` and continue. |
|
|
19
17
|
| 4 | **Weight + normalize** every project (math below) using the weights block. |
|
|
20
|
-
| 5 | **Rollup** → `<out_dir>/results/{scorecard.csv, ranking.md, summary.md}
|
|
21
|
-
|
|
22
|
-
A single project does **not** need this rule — invoke the `evaluator` skill directly.
|
|
18
|
+
| 5 | **Rollup** → `<out_dir>/results/{scorecard.csv, ranking.md, summary.md}` (contents below), sorted by rank score desc. Leave everything uncommitted. |
|
|
23
19
|
|
|
24
20
|
## Weighting math
|
|
25
21
|
|
|
26
|
-
For each project `p`, read
|
|
22
|
+
For each project `p`, read each sub-criterion `s`'s raw 0–5 `score` (the evaluator's absolute band) from its `scorecard.yaml`:
|
|
27
23
|
|
|
28
|
-
1. **Weighted (0–100)** over sub-criteria with a non-null
|
|
29
|
-
`weighted_p = 100 × Σ_s (
|
|
24
|
+
1. **Weighted (0–100)** over sub-criteria with a non-null `score` and `weight > 0`:
|
|
25
|
+
`weighted_p = 100 × Σ_s (score_{p,s} / 5 × weight_s) / Σ_s weight_s`
|
|
30
26
|
2. **Gate penalty:** `adjusted_p = max(0, weighted_p − Σ_{g ∈ gates_triggered_p} gate_penalties[g])`.
|
|
31
|
-
3. **Normalize across the set
|
|
32
|
-
- `percent_of_max` → `adjusted_p` unchanged
|
|
33
|
-
- `min_max` → `100 × (adjusted_p − min) / (max − min)`;
|
|
34
|
-
- `z_score` → `clamp(0, 100, 50 + 10 × (adjusted_p − mean) / sd)`;
|
|
27
|
+
3. **Normalize** across the set `{adjusted_p}` per `normalization`:
|
|
28
|
+
- `percent_of_max` (default) → `adjusted_p` unchanged — absolute, comparable across runs.
|
|
29
|
+
- `min_max` → `100 × (adjusted_p − min) / (max − min)`; `max == min` ⇒ all 100. Spreads a tight set; not comparable across runs.
|
|
30
|
+
- `z_score` → `clamp(0, 100, 50 + 10 × (adjusted_p − mean) / sd)`; `sd == 0` or n < 2 ⇒ fall back to `adjusted_p`.
|
|
35
31
|
|
|
36
|
-
`VALIDATION_FAILED` projects are excluded from min/max/mean/sd and listed separately. A null sub-criterion drops out of that project's weighted average —
|
|
32
|
+
`VALIDATION_FAILED` projects are excluded from min/max/mean/sd and listed separately. A null sub-criterion drops out of that project's weighted average — never impute a 0. Compute with a short throwaway script over the scorecards; round reported scores to 2 decimals.
|
|
37
33
|
|
|
38
|
-
|
|
34
|
+
| Output | Contents |
|
|
35
|
+
| --- | --- |
|
|
36
|
+
| `scorecard.csv` | `rank, project, <8 scores>, gates, short_circuited, evaluator_total, weighted, penalty, rank_score` |
|
|
37
|
+
| `ranking.md` | ranked table headed with the normalization method — the sort key |
|
|
38
|
+
| `summary.md` | n scored, the exact weights / penalties / normalization used, top-5 / bottom-5, gate frequency, any `VALIDATION_FAILED` |
|
|
39
39
|
|
|
40
40
|
## Weights — edit this block to change the ranking
|
|
41
41
|
|
|
42
|
-
|
|
42
|
+
The math above consumes these values verbatim. Edits here are overwritten on a harness re-install — to persist a custom weighting, drop a `code-eval-batcher.weights.json` with the same shape in the `out_dir` and read that instead.
|
|
43
43
|
|
|
44
44
|
```json
|
|
45
45
|
{
|
|
@@ -61,11 +61,7 @@ These are the only cross-project knobs. Edit the values to re-rank; the math abo
|
|
|
61
61
|
}
|
|
62
62
|
```
|
|
63
63
|
|
|
64
|
-
- **`weights`** — relative importance
|
|
65
|
-
- **`gate_penalties`** — flat 0–100-scale points docked per triggered gate, on top of the band caps the evaluator already applied. Default 0 = rely
|
|
66
|
-
- **`normalization`** — `percent_of_max` (absolute, default) · `min_max` (spreads a tight set; not comparable across runs) · `z_score` (distance from the set mean).
|
|
67
|
-
|
|
68
|
-
## Discipline
|
|
64
|
+
- **`weights`** — relative importance; any non-negative numbers, normalized to sum 100 over the present, weighted sub-criteria. `0` drops a sub-criterion. The all-12.5 default reproduces the evaluator's own equal-weight total.
|
|
65
|
+
- **`gate_penalties`** — flat 0–100-scale points docked per triggered gate, on top of the band caps the evaluator already applied. Default 0 = rely on the evaluator's caps alone. E.g. `SECRETS: 15` docks 15 from any project with a committed secret.
|
|
69
66
|
|
|
70
|
-
|
|
71
|
-
- **You weight; the evaluator bands.** Changing a project's band because of how others scored is the cohort-relative anti-pattern this design removed. Your only cross-project levers are the weights block.
|
|
67
|
+
**No silent fallback.** A malformed weights block, a failed preflight, or a scorecard that won't validate is reported loudly — never worked around.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@nurix/etna",
|
|
3
|
-
"version": "0.19.
|
|
3
|
+
"version": "0.19.34",
|
|
4
4
|
"description": "Bootstrap a project with a Nurix harness — copies the harness's CLAUDE.md, skills, agents, rules, and hooks directly into the project from a bundled payload (no marketplace fetch, no network).",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|