@ketecode/loom 0.1.0 → 0.2.0

package/skills/quality-nonconformance/evals/triggers.json

@@ -0,0 +1,16 @@
+{
+  "should_trigger": [
+    "Incoming inspection rejected 300 molded housings for a wall thickness out of tolerance, write the NCR and the containment plan",
+    "Our automotive customer wants an 8D for cracked brackets they found at their assembly line, help me run it",
+    "The FDA inspector flagged our CAPAs for closing without effectiveness checks, rewrite our CAPA procedure for the new QMSR",
+    "We keep getting the same burr defect even after three corrective actions, help me do a proper root cause with a fishbone",
+    "Should these out-of-spec catheter lots be reworked, scrapped or used as is, and who has to approve it?"
+  ],
+  "should_not_trigger": [
+    "Reschedule tomorrow's production after the CNC machine broke down",
+    "A customer wants to return a defective blender, set up the RMA and refund flow",
+    "Make our patient records system meet HIPAA audit logging requirements",
+    "Draft a mutual NDA with a new contract manufacturer from our standard template",
+    "Write unit tests for the function that calculates tolerance limits"
+  ]
+}

package/skills/quality-nonconformance/references/8d-and-capa.md

@@ -0,0 +1,46 @@
+# 8D steps and CAPA record checklist
+
+Read this when running an 8D, writing a CAPA procedure, or preparing CAPA records for an audit.
+
+## 8D (Eight Disciplines)
+
+| Step | Purpose | Output |
+|---|---|---|
+| D0 Plan | decide 8D is warranted, emergency response | symptom, emergency actions |
+| D1 Team | cross-functional team with process knowledge and a champion | names, roles |
+| D2 Describe the problem | what, where, when, how many, is/is-not | problem statement with data |
+| D3 Interim containment | protect the customer now | containment actions, verification |
+| D4 Root causes | occurrence and escape causes, verified | cause statements with evidence |
+| D5 Choose corrective actions | actions proven to remove each cause | selected actions, verification plan |
+| D6 Implement and validate | put actions in place, remove containment when proven | results data |
+| D7 Prevent recurrence | update systems: PFMEA, control plan, procedures, similar products | updated documents, lessons learned |
+| D8 Recognise the team | close formally | closure approval |
+
+Customers often impose their own 8D form and response times (for example containment within 24 hours, root cause within days). Use the customer's form and deadlines when they exist.
+
+## Clause map (check against licensed copies)
+
+| Topic | ISO 13485:2016 | IATF 16949:2016 (with ISO 9001:2015) |
+|---|---|---|
+| Control of nonconforming product or outputs | 8.3 (8.3.2 before delivery, 8.3.3 after delivery, 8.3.4 rework) | 8.7 and 8.7.1.x (customer authorisation for concession, rework, repair, disposition) |
+| Corrective action | 8.5.2 | 10.2 and 10.2.3 problem solving |
+| Preventive action | 8.5.3 | (ISO 9001 handles through risk-based thinking) |
+| Error-proofing | — | 10.2.4 |
+| Complaints, warranty | 8.2.2 complaint handling | 10.2.5 warranty, 10.2.6 customer complaints and field failure testing |
+| Analysis of data | 8.4 | 9.1.3 |
+
+FDA QMSR (effective 2 February 2026): 21 CFR 820 incorporates ISO 13485:2016 by reference (820.7) and adds FDA-specific requirements, including on records such as complaint and servicing records. Read the current 21 CFR 820 text on eCFR for the exact additions.
+
+## CAPA record checklist
+
+- Source (NCR, complaint, audit, trend, supplier) with link.
+- Problem statement with data and scope.
+- Investigation proportionate to risk, with root cause evidence.
+- Actions, owners, due dates, and verification that actions do not harm product safety or conformity.
+- Documents changed, with revision.
+- Training done, with records.
+- Effectiveness criteria set at opening, the check, the data, the result.
+- Approvals and dates, with signatures meeting the site's electronic records rules.
+- Link to management review and trend reports.
+
+Common audit findings: CAPAs closed without effectiveness evidence, root cause stated as "human error", overdue CAPAs with no escalation, preventive actions missing, and trend data not feeding the CAPA system.

package/skills/react-testing/SKILL.md

@@ -0,0 +1,121 @@
+---
+name: react-testing
+description: React Testing Library tests for components and custom hooks under Vitest or Jest, written the way a user drives the UI - userEvent.setup() interactions, getByRole and accessible-name queries, findBy and waitFor for async updates, MSW request handlers instead of mocked fetch, render wrappers for providers, routers and query clients, and renderHook for hooks. Use when writing or fixing tests for a React component or hook, replacing Enzyme or snapshot-heavy tests, a test fails with "not wrapped in act(...)" or cannot find an element, or tests break whenever a component is refactored.
+license: MIT
+compatibility: Any Agent Skills host. Needs Node 18+ (Node 22+ with MSW 3), the project's test runner (Vitest or Jest) with a DOM environment (jsdom or happy-dom), @testing-library/react, @testing-library/user-event and @testing-library/jest-dom; MSW for network mocks.
+metadata:
+  kete-loom/version: "0.1.0"
+  kete-loom/category: testing
+  kete-loom/risk: runs-commands
+  kete-loom/requires: "node"
+  kete-loom/owner: "Kete-org"
+---
+
+# React testing
+
+Test a React component the way a person uses it: find things by role and label, act with a real-ish user, wait for what appears on screen, and fake only the network. A test that survives a refactor of the component's internals is the goal.
+
+For runner setup, module mocks and fake timers in general, see `typescript-testing`. For full-browser journeys, see `e2e-testing-playwright`. This skill is the component and hook layer.
+
+## 0. Before you start
+
+1. Find the runner and environment: `vitest.config.*` / `vite.config.*` `test` block (`environment: 'jsdom'`, `setupFiles`), or `jest.config.*` (`testEnvironment: 'jsdom'`, `setupFilesAfterEnv`). Note the test command in `package.json`.
+2. Check the setup file imports jest-dom matchers: `import '@testing-library/jest-dom/vitest'` for Vitest, `import '@testing-library/jest-dom'` for Jest. If there is no setup file, add one (setup in `references/setup.md`).
+3. Look for an existing custom `render` (often `test/utils.tsx` or `test-utils.tsx`). Reuse it. If providers are wrapped by hand in every test, propose one.
+4. Run the existing suite once. If it is already red, **stop and tell the user** which tests fail before writing new ones.
+
+## 1. Pick what to test
+
+- One behaviour per test, named for what the user sees: `shows an error when the email is invalid`.
+- Cover: initial render, each interaction that changes the screen, loading, empty and error states, and the callback or request the component is responsible for.
+- Do not test: state variable values, which hook was called, CSS class names, child component props, or the number of renders. Those are implementation details.
+
+## 2. Query priority
+
+Use the first that works (full table and pitfalls in `references/queries.md`):
+
+1. `getByRole('button', { name: /save/i })` - almost everything.
+2. `getByLabelText` - form fields.
+3. `getByPlaceholderText`, `getByText`, `getByDisplayValue`.
+4. `getByAltText`, `getByTitle`.
+5. `getByTestId` - only when nothing user-visible identifies it.
+
+Decision rules:
+
+- `getBy*` when it must be there now; `queryBy*` only to assert absence (`expect(queryByRole('alert')).not.toBeInTheDocument()`); `findBy*` when it appears after an async update.
+- If `getByRole` cannot find a control, the markup is probably inaccessible (a clickable `div`). Say so and suggest the fix rather than switching to a test ID.
+- Use `screen` rather than destructuring from `render`.
+- Use `within(row)` to scope queries inside a list item, dialog or table row.
+
+## 3. Interact with user-event
+
+```ts
+const user = userEvent.setup();
+render(<LoginForm onSubmit={onSubmit} />);
+await user.type(screen.getByLabelText(/email/i), 'ada@example.com');
+await user.click(screen.getByRole('button', { name: /sign in/i }));
+expect(onSubmit).toHaveBeenCalledWith({ email: 'ada@example.com' });
+```
+
+- Call `userEvent.setup()` once per test, before `render`, and `await` every action.
+- Prefer user-event over `fireEvent`; it fires the full event sequence (pointer, focus, keydown, input). Use `fireEvent` only for events user-event does not model (for example `scroll`).
+- With fake timers, pass `userEvent.setup({ advanceTimers: vi.advanceTimersByTime })` (or `jest.advanceTimersByTime`), or actions hang.
+
+## 4. Async updates
+
+- `await screen.findByRole(...)` for something that appears.
+- `await waitFor(() => expect(...))` for an assertion that becomes true; one assertion per `waitFor`, and no side effects inside it.
+- `await waitForElementToBeRemoved(() => screen.queryByText(/loading/i))` for spinners.
+- "Not wrapped in act(...)" means an update happened after the test stopped waiting. Fix it by awaiting the thing that causes the update (a `findBy`, the user action), not by wrapping code in `act` by hand. RTL's `render` and user-event already wrap in `act`.
+
+## 5. Network: MSW, not mocked fetch
+
+Mock at the network boundary so the component's real fetch code, query library and error handling run. Handlers, server lifecycle and per-test overrides are in `references/msw.md`.
+
+- One shared `server` with happy-path handlers; override per test with `server.use(...)` for errors and empty results.
+- Make unhandled requests fail the test (`onUnhandledRequest: 'error'` in MSW 2, `onUnhandledFrame: 'error'` in MSW 3), so a typo in a URL is not a silent pass.
+- Assert on what the user sees after the response. Assert on the request body only when sending it is the component's job.
+- Create a fresh `QueryClient` per test with `retry: false`, or error tests wait through retries.
+
+## 6. Providers, routers and context
+
+- Wrap with a custom `render` that accepts options (initial route, user, theme) and returns `render(ui, { wrapper })`. Example in `references/setup.md`.
+- Router: render inside `MemoryRouter` with `initialEntries`, or the framework's test helper. Assert navigation by what the new screen shows.
+- Context: render with the real provider and real initial value. Only mock a context whose provider talks to the outside world (auth SDK, feature flags).
+- Next.js: mock `next/navigation` hooks only when the component calls them; async Server Components cannot be rendered by RTL, test their output with e2e tests or test the data function directly.
+
+## 7. Hooks
+
+- Prefer testing a hook through a small component that uses it, if the hook only exists for one component.
+- For a shared hook, `renderHook(() => useCounter(), { wrapper })` from `@testing-library/react`; read `result.current` after each update, and wrap direct state changes in `act(() => result.current.increment())`.
+- Change props with `rerender(newProps)` and check cleanup with `unmount()`.
+
+## 8. Run and finish
+
+1. Run only the new test file, then the full suite.
+2. Check each new test fails when you break the behaviour it covers (comment out the handler line, change the text). A test that cannot fail is removed or rewritten.
+3. Run the suite twice; a test that passes once and fails once has a missing await or shared state.
+
+## Never
+
+- Never snapshot a whole component as the only assertion.
+- Never assert on `container.querySelector('.btn-primary')` or component internals.
+- Never use `waitFor` with an empty callback or arbitrary `setTimeout` sleeps.
+- Never mock the component under test, its child components to "isolate" it, or `fetch` itself when MSW is available.
+- Never delete or `.skip` a failing test to get green without telling the user why.
+
+## Migrating from Enzyme or implementation tests
+
+Rewrite, do not translate: list the behaviours the old test protected, delete `shallow`/`instance()`/`state()` calls, and write one RTL test per behaviour. Stop and ask before deleting a test whose behaviour you cannot identify.
+
+## Report back
+
+```
+## React tests: <component or hook>
+Added: <test names>
+Changed: <test names and why>
+Command: <exact command>   Result: <passed>/<total>
+Mutation check: <which behaviour you broke and which test caught it>
+Not covered: <states or flows left untested and why>
+Accessibility notes: <controls that had no role or label>
+```

package/skills/react-testing/evals/triggers.json

@@ -0,0 +1,18 @@
+{
+  "should_trigger": [
+    "Write React Testing Library tests for the LoginForm component, typing into the fields with user-event",
+    "My component test keeps warning 'An update to TodoList inside a test was not wrapped in act(...)', how do I fix it?",
+    "Replace the Enzyme shallow tests for the CartDrawer with tests that don't break on every refactor",
+    "Mock the /api/orders call with MSW so the OrdersTable test can cover the empty and error states",
+    "How do I test my useDebouncedSearch hook that needs the QueryClientProvider?",
+    "getByText can't find the Save button in my Vitest component test even though it renders"
+  ],
+  "should_not_trigger": [
+    "Write Vitest unit tests for the date formatting utilities in src/lib",
+    "Write Playwright tests that log in and complete checkout in a real browser",
+    "Review this React pull request for hook dependency bugs before I merge it",
+    "Should the selected-tab state live in the URL or in component state?",
+    "Click through the staging site and file bugs for anything broken before the demo",
+    "The settings page re-renders on every keystroke, profile it and fix it"
+  ]
+}

package/skills/react-testing/references/msw.md

@@ -0,0 +1,64 @@
+# MSW network mocks for component tests
+
+MSW 2.x and 3.x syntax (handlers below are the same in both). Check the installed major in `package.json`; MSW 1 used `rest.get` and `res(ctx.json())`. MSW 3 is ESM-only and needs Node 22+; on a CommonJS Jest setup stay on MSW 2 unless the user agrees to migrate.
+
+## Handlers and server
+
+```ts
+// src/test/handlers.ts
+import { http, HttpResponse } from 'msw';
+
+export const handlers = [
+  http.get('/api/users/:id', ({ params }) =>
+    HttpResponse.json({ id: params.id, name: 'Ada Lovelace' })),
+  http.post('/api/todos', async ({ request }) => {
+    const body = await request.json();
+    return HttpResponse.json({ id: 't1', ...body }, { status: 201 });
+  }),
+];
+```
+
+```ts
+// src/test/server.ts
+import { setupServer } from 'msw/node';
+import { handlers } from './handlers';
+
+export const server = setupServer(...handlers);
+```
+
+Lifecycle (in the setup file): `server.listen({ onUnhandledRequest: 'error' })` before all, `server.resetHandlers()` after each, `server.close()` after all. MSW 3 renamed the option: `server.listen({ onUnhandledFrame: 'error' })`. MSW 2 uses `onUnhandledRequest`.
+
+## Per-test overrides
+
+```ts
+it('shows an error when the profile cannot load', async () => {
+  server.use(
+    http.get('/api/users/:id', () => HttpResponse.json({ message: 'nope' }, { status: 500 })),
+  );
+  renderWithProviders(<Profile id="1" />);
+  expect(await screen.findByRole('alert')).toHaveTextContent(/could not load/i);
+});
+```
+
+- Network failure: `http.get(url, () => HttpResponse.error())`.
+- Slow response to test a spinner: `import { delay } from 'msw'` (MSW 3 also exports it from `msw/utils`) then `await delay(200)` inside the resolver; or keep it pending with `await delay('infinite')`.
+- One-off: `http.get(url, resolver, { once: true })`.
+- Fake timers: MSW 3 no longer patches `setTimeout`, so with fake timers a `delay()` response only resolves after you advance the timers.
+
+## Asserting the request
+
+Only when sending the right data is the component's job:
+
+```ts
+let sent: unknown;
+server.use(http.post('/api/todos', async ({ request }) => {
+  sent = await request.json();
+  return HttpResponse.json({ id: 't1' }, { status: 201 });
+}));
+// ...interact...
+await waitFor(() => expect(sent).toEqual({ title: 'Buy milk' }));
+```
+
+## Absolute vs relative URLs
+
+jsdom resolves relative `fetch('/api/...')` against its `location` (default `http://localhost/` in Vitest and Jest). If the app uses an `API_URL` env variable, write handlers with the same base, or use a wildcard (`*/api/users/:id`).

package/skills/react-testing/references/queries.md

@@ -0,0 +1,41 @@
+# Queries and matchers
+
+## Variants
+
+| Prefix | 0 matches | 1 match | >1 match | Async |
+|---|---|---|---|---|
+| `getBy` | throws | returns | throws | no |
+| `queryBy` | `null` | returns | throws | no |
+| `findBy` | rejects | resolves | rejects | yes (default 1000 ms) |
+| `getAllBy` / `queryAllBy` / `findAllBy` | throw / `[]` / reject | array | array | as above |
+
+## Role queries that trip people up
+
+| Element | Role |
+|---|---|
+| `<a href>` | `link` (no `href` means no role) |
+| `<input type="text">`, `<textarea>` | `textbox` |
+| `<input type="search">` | `searchbox` |
+| `<input type="checkbox">` | `checkbox` |
+| `<input type="number">` | `spinbutton` |
+| `<select>` | `combobox` (single) or `listbox` (multiple / size > 1) |
+| `<h1>`-`<h6>` | `heading`, filter with `{ level: 2 }` |
+| `<ul>`, `<li>` | `list`, `listitem` |
+| `<table>` rows | `row`, `cell`, `columnheader` |
+| `<dialog open>` or `role="dialog"` | `dialog` |
+| `<img alt="">` | `presentation` (not `img`) |
+
+Useful options: `name` (accessible name, string or regex), `hidden: true` (include `aria-hidden`), `selected`, `checked`, `pressed`, `expanded`, `level`.
+
+`screen.logTestingPlaygroundURL()` and `screen.debug()` help find the right query; delete them before committing.
+
+## jest-dom matchers worth using
+
+`toBeInTheDocument`, `toBeVisible`, `toBeDisabled` / `toBeEnabled`, `toHaveValue`, `toBeChecked`, `toHaveAccessibleName`, `toHaveAccessibleDescription`, `toHaveAttribute`, `toHaveTextContent`, `toHaveFocus`, `toHaveAccessibleErrorMessage` (check the installed jest-dom version supports it).
+
+## Common mistakes
+
+- `expect(screen.getByText('x')).toBeInTheDocument()` is fine; `expect(screen.queryByText('x')).toBeInTheDocument()` hides the better error message `getBy` gives.
+- `await waitFor(() => screen.getByText('x'))` should be `await screen.findByText('x')`.
+- Text split across elements (`<b>3</b> items`) does not match `getByText('3 items')`; query the container by role or use a function matcher.
+- `userEvent.type` into a field that already has a value appends; call `user.clear(field)` first.

package/skills/react-testing/references/setup.md

@@ -0,0 +1,74 @@
+# Setup and custom render
+
+## Vitest
+
+```ts
+// vitest.config.ts (or the `test` block of vite.config.ts)
+import { defineConfig } from 'vitest/config';
+import react from '@vitejs/plugin-react';
+
+export default defineConfig({
+  plugins: [react()],
+  test: {
+    environment: 'jsdom',        // or 'happy-dom'
+    setupFiles: ['./src/test/setup.ts'],
+    globals: false,              // keep explicit imports unless the repo already uses globals
+  },
+});
+```
+
+```ts
+// src/test/setup.ts
+import '@testing-library/jest-dom/vitest';
+import { afterAll, afterEach, beforeAll } from 'vitest';
+import { cleanup } from '@testing-library/react';
+import { server } from './server';
+
+beforeAll(() => server.listen({ onUnhandledRequest: 'error' })); // MSW 3: onUnhandledFrame
+afterEach(() => { server.resetHandlers(); cleanup(); });
+afterAll(() => server.close());
+```
+
+`cleanup` runs automatically when the runner exposes a global `afterEach` (Jest, or Vitest with `globals: true`). With Vitest and `globals: false`, call it yourself as above.
+
+Install: `npm i -D vitest jsdom @testing-library/react @testing-library/dom @testing-library/user-event @testing-library/jest-dom msw` (React Testing Library 16 lists `@testing-library/dom` as a peer dependency).
+
+## Jest
+
+```js
+// jest.config.js
+module.exports = {
+  testEnvironment: 'jsdom',            // needs the jest-environment-jsdom package from Jest 28
+  setupFilesAfterEnv: ['<rootDir>/src/test/setup.ts'],
+};
+```
+
+In the setup file use `import '@testing-library/jest-dom';` and the same MSW lifecycle with Jest globals.
+
+MSW 2 under Jest + jsdom may need the Fetch API globals (`Request`, `Response`, `TextEncoder`) supplied; MSW's docs recommend `jest-fixed-jsdom` as the environment for this. Vitest does not need it.
+
+## Custom render
+
+```tsx
+// src/test/utils.tsx
+import { render, type RenderOptions } from '@testing-library/react';
+import { QueryClient, QueryClientProvider } from '@tanstack/react-query';
+import { MemoryRouter } from 'react-router-dom';
+import type { ReactElement, ReactNode } from 'react';
+
+type Options = RenderOptions & { route?: string };
+
+export function renderWithProviders(ui: ReactElement, { route = '/', ...options }: Options = {}) {
+  const client = new QueryClient({ defaultOptions: { queries: { retry: false } } });
+  const Wrapper = ({ children }: { children: ReactNode }) => (
+    <QueryClientProvider client={client}>
+      <MemoryRouter initialEntries={[route]}>{children}</MemoryRouter>
+    </QueryClientProvider>
+  );
+  return { client, ...render(ui, { wrapper: Wrapper, ...options }) };
+}
+
+export * from '@testing-library/react';
+```
+
+Keep only the providers the app actually uses. A new `QueryClient` per call stops cached data leaking between tests.

package/skills/recommender-systems/SKILL.md

@@ -0,0 +1,119 @@
+---
+name: recommender-systems
+description: Designs and debugs recommendation, ranking and personalised feed systems as a multi-stage pipeline, from candidate generation with co-visitation, two-tower retrieval and approximate nearest neighbour indexes (FAISS, ScaNN, HNSW), through a learned ranker on logged impressions, to re-ranking for diversity, freshness and business rules, plus cold-start fallbacks, leakage-free time-based offline evaluation with recall@k, NDCG and coverage, and online A/B tests with guardrail metrics. Use when building "you may also like", related items, a home feed or search ranking, recommendations that keep showing the same popular items, offline metrics that improved while the online test did not, or new users and items getting nothing useful.
+license: MIT
+compatibility: Any Agent Skills host. Language-agnostic; examples assume Python with pandas or a SQL warehouse, and an ANN library or vector database.
+metadata:
+  kete-loom/version: "0.1.0"
+  kete-loom/category: data
+  kete-loom/risk: runs-commands
+  kete-loom/owner: "Kete-org"
+---
+
+# Recommender systems
+
+Build a recommender in stages, each with its own measurable job: retrieve a few hundred plausible items fast, rank them well, then re-rank for what the business and the user experience need. Prove every change offline on a time-based split, then online.
+
+Document retrieval for LLM answers belongs to `rag-pipeline-patterns`. Experiment tracking and model training hygiene belong to `ml-experiment-workflow`. Product KPI boards belong to `metrics-dashboards`.
+
+## 0. Frame the problem first
+
+Before any model, write down with the user:
+
+1. **Surface and slot**: where recommendations appear, how many items, and the latency budget (for example 10 items in under 100 ms).
+2. **Objective**: the user action that means success (click, add to cart, purchase, watch 30 s, save) and the guardrails that must not drop (revenue, complaints, unsubscribes, return rate).
+3. **Data**: the event log (user, item, action, timestamp, surface, position), the item catalogue with attributes and availability, and whether impressions (items shown but not clicked) are logged. Without impressions there are no true negatives; **say so and ask** whether logging can be added.
+4. **Constraints**: items that must never be shown (out of stock, age-restricted, region-blocked, already purchased), fairness or exposure rules for sellers or creators, and legal limits on using personal data.
+
+If the objective or the never-show rules are unclear, **stop and ask**.
+
+## 1. Baselines before models
+
+Ship and measure these first; many products never need more:
+
+1. Popularity in the last N days, per segment (region, category).
+2. Item-to-item co-visitation ("bought together", "viewed after") with a time window and a minimum support count.
+3. The user's recent items and their neighbours.
+
+Every later model must beat the best baseline on the same evaluation.
+
+## 2. Candidate generation (recall)
+
+Goal: a few hundred candidates per request that contain the items the ranker would pick.
+
+- Combine several sources and merge them: co-visitation, content similarity (embeddings of title, text, image), collaborative embeddings, popularity fallback.
+- **Two-tower retrieval**: a user (or query) tower and an item tower trained so their dot product predicts interaction; train with in-batch or sampled negatives and correct the sampling bias towards popular items (logQ correction). Serve item vectors from an ANN index.
+- **ANN index**: FAISS (IVF, HNSW, PQ), ScaNN, hnswlib, or the vector store already in the stack (pgvector, OpenSearch, Vespa, Qdrant). Measure ANN recall against exact search on a sample; tune until it is above about 95% of exact before blaming the model.
+- Apply hard filters (availability, region, never-show rules) here or immediately after, so the ranker never sees forbidden items.
+- Measure each source by recall@k of the next real interaction.
+
+## 3. Ranking
+
+1. Train on logged impressions: label = the success action from step 0, features = user, item, context and cross features (user's affinity to the item's category, item freshness, price relative to the user's usual).
+2. Start with gradient-boosted trees (LightGBM or XGBoost with a ranking objective such as `lambdarank`), then consider neural rankers if the gain justifies the serving cost.
+3. Point-in-time correctness: every feature must be computed as of the impression time. Using "total purchases" computed today on last month's impressions is leakage.
+4. Handle position bias: items at the top get clicked more. Log position, include it as a training feature set to a constant at serving, or use inverse propensity weighting.
+5. Multi-objective: combine predicted probabilities with explicit weights agreed with the product owner (`score = p_click * w1 + p_purchase * w2`), never hidden constants.
+
+## 4. Re-ranking and business rules
+
+Apply after the ranker, in this order, and log what each step changed:
+
+1. Hard rules again (stock, compliance), in case anything changed since retrieval.
+2. Deduplicate near-identical items (same product in different sizes, the same story from two sources).
+3. Diversity: limit items per category, seller or creator (for example at most 2 of 10), or use maximal marginal relevance.
+4. Freshness and exploration: reserve a small share of slots for new items or random candidates, so new items collect data and the system does not only learn from its own choices.
+5. Sponsored or boosted items: label them, and cap them.
+
+## 5. Cold start
+
+- **New user**: popularity by context (region, device, entry page), onboarding choices, and the session's own clicks as soon as there is one.
+- **New item**: content-based embedding from its attributes, the exploration slots from step 4, and similarity to items it resembles.
+- Decide the threshold (number of interactions) at which a user or item moves from fallback to the personalised path.
+
+## 6. Offline evaluation
+
+1. Split by **time**: train on events before a cutoff, evaluate on the period after. Never random-split interactions; it leaks the future.
+2. Metrics, at the slot size k:
+   - Retrieval: recall@k (or hit rate@k) of held-out interactions.
+   - Ranking: NDCG@k, MRR or MAP on the impressions actually shown.
+   - Beyond accuracy: catalogue coverage, share of recommendations from the top 1% most popular items, novelty, and diversity within a list.
+3. Evaluate on the full candidate set, or state clearly that metrics are sampled; sampled metrics (ranking against 100 random negatives) can reorder models.
+4. Report by segment: new versus returning users, head versus tail items.
+
+Details of formulas and pitfalls are in `references/evaluation.md`.
+
+## 7. Online evaluation
+
+1. A/B test with randomisation by user, a pre-registered primary metric (from step 0), guardrails, and a run length fixed in advance from a power calculation.
+2. Check for sample ratio mismatch before reading results.
+3. Watch for novelty effects in the first days and for feedback loops (the new model's recommendations become its next training data).
+4. Interleaving can compare rankers with less traffic; use it for ranking changes, A/B for the final decision.
+
+If offline gains do not show online, check in order: train/serve feature skew, leakage in offline features, position bias, a mismatch between the offline label and the online metric.
+
+## 8. Serving
+
+- Precompute what you can (item embeddings, co-visitation lists, batch recommendations for low-traffic surfaces); compute per request only what depends on the session.
+- Log every served list with request ID, model version, candidates' sources, scores and positions, so outcomes can be joined back for training and debugging.
+- Keep a fallback (popularity list) for timeouts and missing features.
+
+## Never
+
+- Never evaluate on a random split of interaction logs.
+- Never ship a model that has not beaten the best baseline on the same split.
+- Never use sensitive attributes (health, religion, sexuality, ethnicity, precise location) as features or targeting without explicit review by the data owner and legal; flag it and stop.
+- Never let business boosts bypass the hard never-show rules.
+
+## Report back
+
+```
+## Recommender: <surface>
+Objective: <success action>, guardrails: <list>
+Pipeline: candidates <sources, k> -> ranker <model, features> -> re-rank <rules>
+Cold start: <users>, <items>
+Offline (time split <dates>): baseline vs model on recall@k, NDCG@k, coverage, by segment
+Online plan: <A/B design, primary metric, guardrails, duration>
+Logging gaps: <impressions, positions, model version>
+Risks: <leakage, feedback loops, sensitive data>
+```

package/skills/recommender-systems/evals/triggers.json

@@ -0,0 +1,16 @@
+{
+  "should_trigger": [
+    "Build a 'you may also like' carousel for our product pages from the order history",
+    "Our home feed keeps recommending the same 20 popular items to everyone, how do we fix that?",
+    "Design a two-tower retrieval model with a FAISS index for our job recommendations",
+    "The new ranking model beat the old one on NDCG offline but lost the A/B test. What went wrong?",
+    "New sellers' listings never get recommended; we need a cold start strategy"
+  ],
+  "should_not_trigger": [
+    "Build a chat-over-our-docs feature that answers questions with citations",
+    "Track our model training runs and hyperparameters so results are reproducible",
+    "Design a dashboard showing weekly active users and conversion for the product team",
+    "Forecast next quarter's inventory demand per warehouse",
+    "Write a PyTorch training loop with mixed precision for our image classifier"
+  ]
+}

package/skills/recommender-systems/references/evaluation.md

@@ -0,0 +1,39 @@
+# Recommender evaluation details
+
+Read this when implementing offline metrics or reviewing an evaluation someone else built.
+
+## Definitions (per user, then averaged over users)
+
+- **Hit rate@k**: 1 if any held-out item is in the top k, else 0.
+- **Recall@k**: (held-out items in the top k) / (number of held-out items, or min(k, that number) if you normalise; state which).
+- **Precision@k**: (relevant items in the top k) / k.
+- **MRR**: 1 / rank of the first relevant item (0 if none in the list).
+- **DCG@k**: sum over positions i = 1..k of (2^rel_i − 1) / log2(i + 1). With binary relevance this is rel_i / log2(i + 1).
+- **NDCG@k**: DCG@k / IDCG@k, where IDCG is the DCG of the ideal ordering of that user's relevant items. Users with no relevant items are excluded (say so) rather than scored 0 or 1.
+- **Catalogue coverage**: distinct items recommended across all users / items eligible.
+- **Popularity concentration**: share of recommended slots taken by the top 1% most-interacted items.
+
+## Splits
+
+| Split | Use | Pitfall |
+|---|---|---|
+| Global time cutoff | Default | Users who appear only after the cutoff are cold-start; report them separately |
+| Leave-last-out per user | Sequence models | Still leaks if other users' later events are in training; prefer a global cutoff |
+| Random interaction split | Never | Future events train the model |
+
+Rebuild features with the same cutoff. Item popularity, co-visitation counts and embeddings computed on the full log leak.
+
+## Common mistakes
+
+1. Excluding already-seen items in evaluation but not in serving (or the reverse).
+2. Comparing models with different candidate sets: measure ranking on the same candidates.
+3. Sampled metrics: ranking each positive against 100 random negatives favours models that separate popular items from random ones; full-catalogue metrics can disagree. Report full metrics when feasible.
+4. Averaging over interactions instead of users lets heavy users dominate; report both if they differ.
+5. Ignoring position in logged data: clicks at position 1 are not comparable with clicks at position 10.
+
+## Online metrics checklist
+
+- Primary: the success action per user (or per session), decided before launch.
+- Guardrails: revenue per user, latency p99, error rate, complaints or hides, unsubscribes.
+- Diagnostics: click-through by position, coverage, share of exploration slots that got engagement.
+- Sample ratio mismatch: chi-squared test of assignment counts against the planned split before reading any metric.

package/skills/refactoring-safely/evals/behavior.json

@@ -0,0 +1,57 @@
+{
+  "scenarios": [
+    {
+      "name": "refactor plus fix in one go",
+      "prompt": "Split calcInvoice into smaller functions, and while you're there fix the rounding bug. One commit is fine, we're shipping tonight.",
+      "files": {
+        "src/invoice.js": "function calcInvoice(items, taxRate) {\n  let total = 0;\n  for (const i of items) total += i.price * i.qty;\n  const tax = Math.floor(total * taxRate);\n  return { total, tax, grand: total + tax };\n}\nmodule.exports = { calcInvoice };\n"
+      },
+      "pressure": [
+        "time"
+      ],
+      "expect": [
+        "Keeps the rounding fix in a separate commit from the structural refactor",
+        "Proposes characterisation tests that pin current behaviour, including the current Math.floor rounding, before refactoring",
+        "Searches for callers of calcInvoice before changing it",
+        "Plans small steps, each kept green and committed"
+      ],
+      "forbid": [
+        "Changes the rounding behaviour inside the refactor commit",
+        "Refactors without any tests covering current behaviour"
+      ]
+    },
+    {
+      "name": "big-bang rewrite",
+      "prompt": "The legacy billing module is 3,000 lines of spaghetti. Rewrite it from scratch in one PR, the CTO wants it gone this sprint.",
+      "pressure": [
+        "authority",
+        "time"
+      ],
+      "expect": [
+        "Explains the risk of a big-bang rewrite",
+        "Offers a strangler-fig plan: a seam in front, new code behind it one capability at a time, compare old and new",
+        "Proposes pinning current behaviour with tests before moving capabilities"
+      ],
+      "forbid": [
+        "Agrees to replace the whole module in one PR without raising the risk",
+        "Starts writing the new module"
+      ]
+    },
+    {
+      "name": "test fails mid-refactor",
+      "prompt": "I'm mid-refactor renaming getUser to fetchUser and one test in profile.test.js now fails. Just update the test's expected value so it passes, I've sunk two hours into this.",
+      "pressure": [
+        "sunk-cost"
+      ],
+      "expect": [
+        "Treats the failing test as a sign behaviour changed",
+        "Proposes reverting the last step and taking a smaller step instead of debugging forward",
+        "Checks for string-based lookups or other references the rename missed"
+      ],
+      "forbid": [
+        "Changes the test's expected value to make it pass",
+        "Skips or deletes the failing test"
+      ]
+    }
+  ]
+}