@clipboard-health/ai-rules 1.8.0 → 2.0.0

package/README.md

@@ -1,16 +1,17 @@
 # @clipboard-health/ai-rules
 
-Pre-built AI agent rules for consistent coding standards.
+Pre-built AI agent rules for consistent coding standards. Uses a retrieval-based approach: generates a compressed index in `AGENTS.md` pointing to individual rule files that agents read on demand.
 
 ## Table of contents
 
-- [@clipboard-health/ai-rules](#clipboard-healthai-rules)
-  - [Table of contents](#table-of-contents)
-  - [Install](#install)
-  - [Usage](#usage)
-    - [Quick Start](#quick-start)
-    - [Updating Rules](#updating-rules)
-  - [Local development commands](#local-development-commands)
+- [Install](#install)
+- [Usage](#usage)
+  - [Quick Start](#quick-start)
+  - [Include/Exclude Rules](#includeexclude-rules)
+  - [Updating Rules](#updating-rules)
+- [Available Rules](#available-rules)
+- [Migration from v1](#migration-from-v1)
+- [Local development commands](#local-development-commands)
 
 ## Install
 
@@ -22,7 +23,7 @@ npm install --save-dev @clipboard-health/ai-rules
 
 ### Quick Start
 
-1. If you have an existing `AGENTS.md` and/or `CLAUDE.md` file in your repository, rename it to `OVERLAY.md`. The `sync-ai-rules` script you'll add below appends this file's contents to each generated file so it's loaded into LLM agent contexts.
+1. If you have an existing `AGENTS.md` and/or `CLAUDE.md` file in your repository, rename it to `OVERLAY.md`. The sync script appends this file's contents to generated `AGENTS.md` so it's loaded into LLM agent contexts.
 
 2. Choose the profile that matches your project type:
 
@@ -34,12 +35,6 @@ npm install --save-dev @clipboard-health/ai-rules
    | `fullstack`    | common + frontend + backend | Monorepos, fullstack apps              |
    | `datamodeling` | datamodeling                | DBT data modeling                      |
 
-   **Rule categories:**
-   - **common**: TypeScript, testing, code style, error handling, key conventions
-   - **frontend**: React patterns, hooks, performance, styling, data fetching, custom hooks
-   - **backend**: NestJS APIs, three-tier architecture, controllers, services
-   - **datamodeling**: data modeling, testing, yaml documentation, data cleaning, analytics
-
 3. Add it to your `package.json`:
 
    ```json
@@ -60,10 +55,35 @@ npm install --save-dev @clipboard-health/ai-rules
 5. Commit the generated files:
 
    ```bash
-   git add .
+   git add rules/ AGENTS.md CLAUDE.md
    git commit -m "feat: add AI coding rules"
    ```
 
+### Include/Exclude Rules
+
+Fine-tune which rules are synced using `--include` and `--exclude`:
+
+```bash
+# Backend profile without MongoDB rules
+node sync.js backend --exclude backend/mongodb
+
+# Common profile plus one backend rule
+node sync.js common --include backend/architecture
+
+# Multiple overrides
+node sync.js backend --exclude backend/mongodb backend/postgres --include frontend/testing
+```
+
+Update your `package.json` script accordingly:
+
+```json
+{
+  "scripts": {
+    "sync-ai-rules": "node ./node_modules/@clipboard-health/ai-rules/scripts/sync.js backend --exclude backend/mongodb"
+  }
+}
+```
+
 ### Updating Rules
 
 When we release new rules or improvements:
@@ -72,17 +92,90 @@ When we release new rules or improvements:
 # Update the package
 npm update @clipboard-health/ai-rules
 
-# The postinstall script automatically copies the latest files
+# The postinstall script automatically syncs the latest files
 npm install
 
 # Review the changes
-git diff .
+git diff rules/ AGENTS.md
 
 # Commit the updates
-git add .
+git add rules/ AGENTS.md CLAUDE.md
 git commit -m "chore: update AI coding rules"
 ```
 
+## Available Rules
+
+### common
+
+| Rule ID                       | Description                                                    |
+| ----------------------------- | -------------------------------------------------------------- |
+| `common/configuration`        | Config decisions: secrets, SSM, feature flags, DB vs hardcoded |
+| `common/featureFlags`         | Feature flag naming, lifecycle, and cleanup                    |
+| `common/gitWorkflow`          | Commit messages, PR titles, and pull request guidelines        |
+| `common/loggingObservability` | Log levels, structured context, PII avoidance                  |
+| `common/testing`              | Unit test conventions and structure                            |
+| `common/typeScript`           | TypeScript naming, types, functions, error handling            |
+
+### backend
+
+| Rule ID                  | Description                                         |
+| ------------------------ | --------------------------------------------------- |
+| `backend/architecture`   | Three-tier pattern: modules, services, controllers  |
+| `backend/asyncMessaging` | Queues, async messaging, background jobs            |
+| `backend/mongodb`        | MongoDB/Mongoose: schemas, indexes, queries         |
+| `backend/notifications`  | Notification and messaging patterns                 |
+| `backend/postgres`       | Postgres queries: Prisma, subqueries, feature flags |
+| `backend/restApiDesign`  | REST API design: JSON:API, endpoints, contracts     |
+| `backend/serviceTests`   | Service-level integration tests                     |
+
+### frontend
+
+| Rule ID                            | Description                                  |
+| ---------------------------------- | -------------------------------------------- |
+| `frontend/customHooks`             | React custom hooks patterns                  |
+| `frontend/dataFetching`            | React Query, API calls, caching              |
+| `frontend/e2eTesting`              | E2E testing with Playwright                  |
+| `frontend/errorHandling`           | React error handling patterns                |
+| `frontend/fileOrganization`        | Frontend file and folder organization        |
+| `frontend/frontendTechnologyStack` | Frontend library and framework choices       |
+| `frontend/interactiveElements`     | Forms, buttons, inputs                       |
+| `frontend/modalRoutes`             | Modals and route-based dialogs               |
+| `frontend/reactComponents`         | React component patterns, props, composition |
+| `frontend/styling`                 | CSS, themes, responsive design               |
+| `frontend/testing`                 | React Testing Library, component tests       |
+
+### datamodeling
+
+| Rule ID                                | Description                             |
+| -------------------------------------- | --------------------------------------- |
+| `datamodeling/analytics`               | Analytics data models                   |
+| `datamodeling/castingDbtStagingModels` | Data type casting in dbt staging models |
+| `datamodeling/dbtModelDevelopment`     | dbt model naming, structure, testing    |
+| `datamodeling/dbtYamlDocumentation`    | dbt YAML documentation and schema files |
+
+## Migration from v1
+
+v2 replaces the monolithic `AGENTS.md` with a retrieval-based approach. Rule files are now committed to your repo under `rules/`.
+
+1. Update the package:
+
+   ```bash
+   npm install --save-dev @clipboard-health/ai-rules@latest
+   ```
+
+2. Run install to trigger sync:
+
+   ```bash
+   npm install
+   ```
+
+3. Add `rules/` to git and commit:
+
+   ```bash
+   git add rules/ AGENTS.md CLAUDE.md
+   git commit -m "feat!: update ai-rules to v2 retrieval-based approach"
+   ```
+
 ## Local development commands
 
 See [`package.json`](./package.json) `scripts` for a list of commands.

package/package.json

@@ -1,11 +1,8 @@
 {
   "name": "@clipboard-health/ai-rules",
   "description": "Pre-built AI agent rules for consistent coding standards.",
-  "version": "1.8.0",
+  "version": "2.0.0",
   "bugs": "https://github.com/ClipboardHealth/core-utils/issues",
-  "devDependencies": {
-    "@intellectronica/ruler": "0.3.31"
-  },
   "keywords": [
     "ai",
     "best-practices",
@@ -28,6 +25,6 @@
     "url": "git+https://github.com/ClipboardHealth/core-utils.git"
   },
   "scripts": {
-    "format": "prettier --write '.ruler/**/*.md'"
+    "format": "prettier --write 'rules/**/*.md'"
   }
 }

package/rules/backend/architecture.md

@@ -0,0 +1,81 @@
+# Architecture
+
+## Three-Tier Architecture
+
+All NestJS microservices follow a three-tier layered architecture:
+
+```text
+┌─────────────────────────────────────────────────────────────────┐
+│  Entrypoints (Controllers, message consumers)                   │
+│  - HTTP request/response, JSON:API DTO translation, auth        │
+├─────────────────────────────────────────────────────────────────┤
+│  Logic (NestJS services, message publishers, background jobs)   │
+│  - ALL business logic; works with DOs only                      │
+│  - Knows nothing about HTTP or database specifics               │
+├─────────────────────────────────────────────────────────────────┤
+│  Data (Data repositories, gateways)                             │
+│  - Database via ORM (Prisma/Mongoose), DAO ↔ DO translation     │
+│  - External service integrations (Gateways)                     │
+└─────────────────────────────────────────────────────────────────┘
+```
+
+**Module Structure:**
+
+`ts-rest` contracts:
+
+```text
+packages/contract-<service>/src/
+├── index.ts
+└── lib
+    ├── constants.ts
+    └── contracts
+        ├── contract.ts
+        ├── health.contract.ts
+        ├── index.ts
+        └── user
+            ├── index.ts
+            ├── user.contract.ts
+            └── shared.ts
+```
+
+NestJS microservice modules:
+
+```text
+ src/modules/user
+ ├── user.module.ts
+ ├── data
+ │   ├── user.dao.mapper.ts
+ │   └── user.repo.ts
+ ├── entrypoints
+ │   ├── user.controller.ts
+ │   ├── user.dto.mapper.ts
+ │   └── userCreated.consumer.ts
+ └── logic
+     ├── user.do.ts
+     ├── user.service.ts
+     ├── userCreated.service.ts
+     └── jobs
+         ├── user.job.mapper.ts
+         ├── userCreated.job.spec.ts
+         └── userCreated.job.ts
+```
+
+**File Patterns:**
+
+```text
+*.controller.ts  - HTTP controllers (entrypoints)
+*.consumer.ts    - Message consumers (entrypoints)
+*.service.ts     - Business logic (logic)
+*.job.ts         - Background jobs (logic)
+*.repo.ts        - Database access (data)
+*.gateway.ts     - External services (data)
+*.do.ts          - Domain objects
+*.dto.mapper.ts  - DTO transformation
+*.dao.mapper.ts  - DAO transformation
+```
+
+**Microservices Principles:**
+
+- One domain = one module (bounded contexts)
+- Specific modules know about generic, not vice versa
+- Don't block Node.js thread—use background jobs for expensive operations

package/rules/backend/asyncMessaging.md

@@ -0,0 +1,78 @@
+# Async Messaging & Background Jobs
+
+## When to Use
+
+| Scenario                         | Solution          |
+| -------------------------------- | ----------------- |
+| Same service producer/consumer   | Background Jobs   |
+| Cross-service communication      | EventBridge + SQS |
+| Deferred work from API path      | Background Jobs   |
+| Replacing `void` fire-and-forget | Background Jobs   |
+| Scaling CRON jobs                | Background Jobs   |
+
+## Background Jobs
+
+**Creation with Transaction:**
+
+```typescript
+async function createLicense() {
+  await db.transaction(async (tx) => {
+    const license = await tx.license.create({ data });
+    await jobs.enqueue(VerificationJob, { licenseId: license.id }, { transaction: tx });
+  });
+}
+```
+
+**Handler Pattern:**
+
+```typescript
+class ShiftReminderJob implements Handler<ShiftReminderPayload> {
+  static queueName = "shift.reminder";
+
+  async perform(payload: ShiftReminderPayload, job: Job): Promise<string> {
+    const { shiftId } = payload;
+
+    // Fetch fresh data—don't trust stale payload
+    const shift = await this.shiftRepo.findById({ id: shiftId });
+
+    if (!shift || shift.isCancelled) {
+      return `Skipping: shift ${shiftId} not found or cancelled`;
+    }
+
+    try {
+      await this.notificationService.performSideEffect(shift);
+      return `Reminder sent for shift ${shiftId}`;
+    } catch (error) {
+      if (error instanceof KnownRecoverableError) throw error; // Retry
+      return `Skipping: ${error.message}`; // No retry
+    }
+  }
+}
+```
+
+**Key Practices:**
+
+- Pass minimal arguments (IDs, not objects)
+- Fetch fresh data in handler
+- Implement idempotency
+- Check state before action
+- Use Expand/Contract for job code updates
+
+**Avoid Circular Dependencies:**
+
+```typescript
+// Shared types file
+export const NOTIFICATION_JOB = "shift-notification";
+export interface NotificationJobPayload {
+  shiftId: string;
+}
+
+// Enqueue by string name
+await jobs.enqueue<NotificationJobPayload>(NOTIFICATION_JOB, { shiftId });
+```
+
+## SQS/EventBridge
+
+**Producer:** Single producer per message type, publish atomically (use jobs as outbox), deterministic message IDs, don't rely on strict ordering.
+
+**Consumer:** Own queue per consumer, must be idempotent, separate process from API, don't auto-consume DLQs.

package/rules/backend/mongodb.md

@@ -0,0 +1,59 @@
+# MongoDB/Mongoose
+
+**ObjectId:**
+
+```typescript
+import mongoose, { Types, Schema } from "mongoose";
+
+const id = new Types.ObjectId();
+
+// In schemas
+const schema = new Schema({
+  authorId: { type: Schema.Types.ObjectId, ref: "User" },
+});
+
+// In interfaces
+interface Post {
+  authorId: Types.ObjectId;
+}
+
+// Validation
+if (mongoose.isObjectIdOrHexString(value)) {
+}
+```
+
+**Indexes:**
+
+- Add only when needed (slower writes tradeoff)
+- Apply via code only
+- Separate `indexes.ts` from `schema.ts`
+- Index definitions only in owning service
+- Set `autoIndex: false`
+- Design covering indexes for high-traffic queries
+
+```text
+models/User/
+├── schema.ts    # Schema definition, schemaName, InferSchemaType
+├── indexes.ts   # Index definitions only
+├── types.ts     # Re-export types
+└── index.ts     # Model creation and export
+```
+
+**Verify query plans:**
+
+```typescript
+const explanation = await ShiftModel.find(query).explain("executionStats");
+// Check: totalDocsExamined ≈ totalDocsReturned
+// Good: stage 'IXSCAN'; Bad: stage 'COLLSCAN'
+```
+
+## Repository Pattern
+
+```typescript
+class UserRepo {
+  // Named methods over generic CRUD
+  async findById(request: { id: UserId }): Promise<UserDo> {}
+  async findByEmail(request: { email: string }): Promise<UserDo> {}
+  async updateEmail(request: { id: UserId; email: string }): Promise<UserDo> {}
+}
+```

package/rules/backend/notifications.md

@@ -0,0 +1,18 @@
+# Notifications
+
+Send via [Knock](https://docs.knock.app) using `@clipboard-health/notifications`.
+
+Use `triggerChunked` to store full trigger request at job enqueue time. See package documentation for setup of `triggerNotification.job.ts`, `notificationClient.provider.ts`, and workflow keys.
+
+**Job enqueue pattern:**
+
+```typescript
+const jobData: SerializableTriggerChunkedRequest = {
+  body: { recipients: ["userId-1"], data: notificationData },
+  expiresAt: new Date(Date.now() + 60 * 60_000).toISOString(),
+  keysToRedact: ["secret"],
+  workflowKey: WORKFLOW_KEYS.eventStartingReminder,
+};
+
+await adapter.enqueue(TRIGGER_NOTIFICATION_JOB_NAME, jobData, { session });
+```

package/rules/backend/postgres.md

@@ -0,0 +1,5 @@
+# Postgres
+
+- Avoid correlated subqueries that execute per-row; they can exhaust connection pools on high-traffic endpoints
+- Put significant query changes (new joins, subqueries, query rewrites) behind a feature flag for gradual rollout and instant rollback
+- For complex queries (joins, aggregations, conditional filtering), prefer Prisma TypedSQL over Prisma client methods

package/rules/backend/restApiDesign.md

@@ -0,0 +1,59 @@
+# REST API Design
+
+## JSON:API Specification
+
+Follow [JSON:API spec](https://jsonapi.org/).
+
+```json
+{
+  "data": [
+    {
+      "id": "1",
+      "type": "shift",
+      "attributes": { "qualification": "nurse" },
+      "relationships": {
+        "assignedWorker": {
+          "data": { "type": "worker", "id": "9" }
+        },
+        "location": {
+          "data": { "type": "workplace", "id": "17" }
+        }
+      }
+    }
+  ]
+}
+```
+
+- Singular `type` values: `shift` not `shifts`
+- Links optional (use only for pagination)
+- Use `include` for related resources
+- Avoid `meta` unless necessary
+
+## URLs
+
+```text
+GET /urgent-shifts                    # lowercase kebab-case, plural nouns
+GET /workers/:workerId/shifts
+POST /workers/:workerId/referral-codes
+```
+
+## HTTP Conventions
+
+- No PUT support — use PATCH for updates
+- POST returns the created resource DTO
+- 422 for unsupported filters/sorts (not 400)
+
+## Filtering, Sorting, Pagination
+
+```text
+GET /shifts?filter[verified]=true&sort=startDate,-urgency&page[cursor]=abc&page[size]=50
+```
+
+- Cursor-based pagination only (not offset)
+- Avoid count totals (performance)
+- Only implement filters/sorts clients need
+
+## Contracts
+
+- Add contracts to `contract-<repo-name>` package
+- Use `ts-rest` with composable Zod schemas

package/rules/backend/serviceTests.md

@@ -0,0 +1,43 @@
+# Service Tests (Primary Testing Approach)
+
+Test the public contract (REST endpoints, events) with real local dependencies (Postgres, Mongo, Redis). Fake slow/external services (Zendesk, Stripe).
+
+```typescript
+describe("Documents", () => {
+  let tc: TestContext;
+
+  describe("GET /documents", () => {
+    it("returns existing documents for authenticated user", async () => {
+      const authToken = await tc.auth.createUser({ role: "employee" });
+      await tc.fixtures.createDocument({ name: "doc-1" });
+
+      const response = await tc.http.get("/documents", {
+        headers: { authorization: authToken },
+      });
+
+      expect(response.statusCode).toBe(200);
+      expect(response.parsedBody.data).toHaveLength(1);
+    });
+  });
+});
+```
+
+**Qualities:** One behavior per test, no shared setup, no mocking, <1 second, parallelizable.
+
+**Testing Background Jobs:**
+
+Don't spy on job enqueuing. Instead, run the job and assert side effects:
+
+```typescript
+// Run the job
+await tc.jobs.drainQueues("shift.reminder");
+
+// Assert side effects
+const shift = await tc.http.get(`/shifts/${shiftId}`);
+expect(shift.reminderSent).toBe(true);
+
+// Or check fakes for external calls
+expect(tc.fakes.notifications.requests).toHaveLength(1);
+```
+
+Side effects to assert: database changes, published messages, external HTTP requests.

package/rules/common/configuration.md

@@ -0,0 +1,21 @@
+# Configuration
+
+```text
+Contains secrets?
+  └── Yes → SSM Parameter Store
+  └── No → Engineers-only, tolerate 1hr propagation?
+      └── Yes → Hardcode with @clipboard-health/config
+      └── No → 1:1 with DB entity OR needs custom UI?
+          └── Yes → Database
+          └── No → LaunchDarkly feature flag
+```
+
+**NPM package management**: Use exact versions: add `save-exact=true` to `.npmrc`
+
+## Secrets
+
+- `.env` locally (gitignored)
+- Production: AWS SSM Parameter Store
+- Prefer short-lived tokens
+
+**Naming:** `[ENV]_[VENDOR]_[TYPE]_usedBy_[CLIENT]_[SCOPE]_[CREATED_AT]_[OWNER]`

package/rules/common/featureFlags.md

@@ -0,0 +1,21 @@
+# Feature Flags
+
+**Naming:** `YYYY-MM-[kind]-[subject]` (e.g., `2024-03-release-new-booking-flow`)
+
+| Kind         | Purpose                  |
+| ------------ | ------------------------ |
+| `release`    | Gradual rollout to 100%  |
+| `enable`     | Kill switch              |
+| `experiment` | Trial for small audience |
+| `configure`  | Runtime config           |
+
+**Rules:**
+
+- "Off" = default/safer value
+- No permanent flags (except `configure`)
+- Create archival ticket when creating flag
+- Validate staging before production
+- Always provide default values in code
+- Clean up after full launch
+
+**When making feature flag changes**: include LaunchDarkly link: `https://app.launchdarkly.com/projects/default/flags/{flag-key}`

package/rules/common/gitWorkflow.md

@@ -0,0 +1,13 @@
+# Git Workflow
+
+Follow Conventional Commits 1.0 spec for commit messages and PR titles.
+
+## Pull Requests
+
+1. Clear title: change summary + ticket
+2. Thorough description: why, not just what
+3. Small & focused: single concept
+4. Tested: service tests + validation proof
+5. Passing CI
+
+Link ticket, context, reasoning, and areas of concern in description.

package/rules/common/loggingObservability.md

@@ -0,0 +1,43 @@
+# Logging & Observability
+
+## Log Levels
+
+| Level | When                                       |
+| ----- | ------------------------------------------ |
+| ERROR | Required functionality broken (2am pager?) |
+| WARN  | Optional broken OR recovered from failure  |
+| INFO  | Informative, ignorable during normal ops   |
+| DEBUG | Local only, not production                 |
+
+## Best Practices
+
+```typescript
+// Bad
+logger.error("Operation failed");
+logger.error(`Operation failed for workplace ${workplaceId}`);
+
+// Good—structured context
+logger.error("Exporting urgent shifts to CSV failed", {
+  workplaceId,
+  startDate,
+  endDate,
+});
+```
+
+- **Never log:** PII, PHI, tokens, secrets, SSN, account numbers, entire request/response/headers.
+- Use metrics for counting:
+
+  ```typescript
+  datadogMetrics.increment("negotiation.errors", { state: "New York" });
+  ```
+
+- Log IDs or specific fields instead of full objects:
+  - `workerId` (not `agent`, `hcp`, `worker`)
+  - `shiftId` (not `shift`)
+- When multiple log statements share context, create a reusable `logContext` object:
+
+  ```typescript
+  const logContext = { shiftId, workerId };
+  logger.info("Processing shift", logContext);
+  logger.info("Notification sent", logContext);
+  ```

package/rules/common/testing.md

@@ -0,0 +1,12 @@
+# Testing
+
+## Unit Tests
+
+Use when: error handling hard to trigger black-box, concurrency scenarios, >5 variations, pure function logic.
+
+## Conventions
+
+- `describe` for grouping
+- Arrange-Act-Assert with newlines between
+- Variables: `mockX`, `input`, `expected`, `actual`
+- Prefer `it.each` for multiple cases