@ivannikov-pro/ai-context-surgeon 1.0.0

package/strategy/monorepo_strategy.md

@@ -0,0 +1,726 @@
+<!-- FNH: Strategy — Monorepo management for AI agents (Architectural Guide) | SECTIONS: Context Management, Models, Tools | DEPS: none -->
+
+# 🏗️ Strategy for working with AI Agents in a Large Monorepo
+
+> Complete guide: architecture, models, techniques, antipatterns | SECTIONS: Context Management, Task Breakdown, Model Selection, Tool Usage | DEPS: none
+
+---
+
+## Table of Contents
+
+1. [Philosophy: Why Context is Everything](#1-philosophy)
+2. [Monorepo Architecture for AI](#2-monorepo-architecture-for-ai)
+3. [File Navigation: The Agent's Roadmap](#3-file-navigation)
+4. [Models: Which One for What](#4-models)
+5. [AGENTS.md — The Law of the Monorepo](#5-agentsmd)
+6. [Knowledge Items — Project Memory](#6-knowledge-items)
+7. [Skills — Agent Abilities](#7-skills)
+8. [Workflows — Routine Automation](#8-workflows)
+9. [MCP Servers — External Tools](#9-mcp-servers)
+10. [Context Saving Techniques](#10-context-saving-techniques)
+11. [Antipatterns](#11-antipatterns)
+12. [Checklist: Ideal Monorepo State](#12-checklist)
+
+---
+
+## 1. Philosophy: The AI AGENT FIRST Manifesto
+
+> [!IMPORTANT]
+> **We do not optimize for human readability.**
+> Humans ideally will never look at this code. YOU (the AI Agent) write the code, and YOU read the code. Therefore, the codebase must be formatted exclusively to improve **your** "life". Everything — from FNH headers to file structures — exists solely to make parsing, navigating, and reasoning as easy as possible for a machine interpreter.
+
+### The Problem with Context
+
+```text
+Agent context ≈ 128K–200K tokens
+System prompt ≈ 20K–35K tokens (already consumed)
+Tools (MCP) ≈ 10K–20K tokens (already consumed)
+Actual budget for your task ≈ 70K–150K tokens
+```
+
+> [!CAUTION]
+> Every unnecessary file loaded into context is a penalty to reasoning accuracy.
+> AI does not "forget" — it loses **focus**. The more irrelevant noise there is, the worse the response quality.
+
+### Four Pillars of Agentic Architecture
+
+| # | Pillar | Essence                  |
+| --- | --- | --- |
+| 1 | **Machine-Oriented** | Format for parsers, not people (e.g., single-line FNHs instead of bloated human comments). |
+| 2 | **Modularity** | The agent must only see the package it is currently working with. |
+| 3 | **Navigability** | The agent must find the needed file in 1–2 steps, without full directory scanning. |
+| 4 | **Self-documenting** | The code and structure must explain themselves without external Notion/Confluence docs. |
+
+---
+
+## 2. Monorepo Architecture for AI
+
+### Ideal Structure
+
+```
+monorepo/
+├── .agents/                          # 🧠 Agent's brain
+│   ├── AGENTS.md                     # Laws — HIGHEST PRIORITY
+│   ├── knowledge/                    # Project memory (Knowledge Items)
+│   │   ├── api-conventions/
+│   │   ├── database-schema/
+│   │   └── deployment-rules/
+│   ├── skills/                       # Agent skills
+│   │   ├── db-migration/
+│   │   └── api-endpoint/
+│   └── workflows/                    # Automation
+│       ├── deploy.md
+│       ├── test.md
+│       └── lint-fix.md
+│
+├── packages/                         # 📦 Packages (each is an isolated world)
+│   ├── core/
+│   │   ├── README.md                 # ← Package roadmap
+│   │   ├── package.json
+│   │   ├── src/
+│   │   │   ├── index.ts              # ← Public API
+│   │   │   ├── types.ts              # ← Contracts
+│   │   │   └── internal/             # ← Implementation details
+│   │   └── tests/
+│   ├── api/
+│   ├── web/
+│   └── shared/
+│
+├── apps/                             # 🚀 Applications
+│   ├── dashboard/
+│   └── mobile-api/
+│
+├── tools/                            # 🔧 Infrastructure utilities
+│   ├── scripts/
+│   └── generators/
+│
+├── README.md                         # Global project map
+├── turbo.json / nx.json              # Dependency graph
+└── package.json                      # Root manifest
+```
+
+### Package Isolation Rule
+
+> [!IMPORTANT]
+> Every package in `packages/` must be a **completely autonomous context**.
+> An agent working on `packages/api` should not need to read `packages/web`.
+
+How to ensure this:
+
+1. **Clear Interfaces** — every package exports only through `index.ts`
+2. **Typed Contracts** — shared types live in `packages/shared/types/`
+3. **README per package** — describes the API, dependencies, and ownership
+4. **Separate `tsconfig.json`** — the package compiles independently
+
+---
+
+## 3. File Navigation: The Agent's Roadmap
+
+> [!TIP]
+> The agent is a blind navigator. It doesn't see the project as a whole, it relies on `list_dir` → `view_file` → `grep_search`.
+> Every such call consumes context. Your job is to reduce the number of steps to a minimum.
+
+### Level 1: Root README.md
+
+```markdown
+# Project Name
+
+## Architecture
+```
+
+packages/
+core/ ← Domain logic, entities, use cases
+api/ ← REST API, Express handlers, middleware
+web/ ← React SPA, pages, components
+shared/ ← Shared types, utils, constants
+notifications/ ← Email, push, SMS services
+apps/
+dashboard/ ← Admin panel (Next.js)
+mobile-api/ ← Mobile BFF
+tools/
+scripts/ ← Build, deploy, migration scripts
+
+```
+
+## Key Decisions
+- Monorepo manager: Turborepo
+- DB: PostgreSQL + Prisma
+- Auth: JWT + refresh tokens
+- API style: REST, versioned (v1, v2)
+```
+
+### Level 2: Package README
+
+```markdown
+# packages/api
+
+REST API server. Express + Zod validation.
+
+## Source Structure
+```
+
+src/
+index.ts ← Entry point, app setup
+routes/ ← Route definitions
+v1/
+users.ts ← CRUD /api/v1/users
+orders.ts ← CRUD /api/v1/orders
+auth.ts ← Login, register, refresh
+middleware/
+auth.ts ← JWT verification
+validation.ts ← Zod schema middleware
+error-handler.ts ← Global error handler
+services/
+user.service.ts ← Business logic for users
+order.service.ts ← Business logic for orders
+utils/
+logger.ts ← Winston logger setup
+response.ts ← Standardized API responses
+
+```
+
+## Dependencies
+- `@monorepo/core` — domain entities
+- `@monorepo/shared` — types, constants
+
+## Environment
+- PORT=3000
+- DATABASE_URL=postgresql://...
+```
+
+### Level 3: JSDoc Navigation (20+ files in a directory)
+
+```typescript
+/** Route: GET/POST /api/v1/users — User CRUD operations */
+import { Router } from "express";
+// ...
+```
+
+```typescript
+/** Middleware: JWT verification — validates Bearer token, sets req.user */
+import { verify } from "jsonwebtoken";
+// ...
+```
+
+### Threshold Rule
+
+| Files in Directory | Action |
+| --- | --- |
+| ≤ 10 | Semantic names are sufficient (`user.service.ts`) |
+| 10–20 | Add Source Structure in the package README |
+| 20+ | Source Structure + JSDoc headers in every file |
+
+---
+
+## 4. Models: Which One for What
+
+### Selection Matrix
+
+| Task | Model | Mode | Why |
+| --- | --- | --- | --- |
+| **Architectural decision**, refactoring, complex migration | **Claude Opus 4.6** | Planning | Best reasoning, deep analysis, thinking mode |
+| **Code generation**, feature implementation | **Gemini 3.1 Pro (High)** | Planning | Strong code generation, massive context window |
+| **Quick edits**, minor fixes, formatting | **Gemini 3 Flash** | Fast | Minimum latency, cost savings |
+| **Bulk operations**, batch refactoring | **Gemini 3.1 Pro (Low)** | Fast | Balance of speed/quality at scale |
+| **Code review**, PR descriptions, docs | **Claude Sonnet 4.6** | Planning | Good linguistic qualities, cheaper than Opus |
+| **Prompt testing**, A/B comparisons | **GPT-OSS 120B** | Planning | Alternative viewpoint, cross-validation |
+| **Automated pipeline tasks** (CI/Orchestrator) | **Gemini 3.1 Pro (Low)** or **GPT-OSS 120B** | Fast | Predictability, cost efficiency |
+
+### The "Model Cascade" Strategy
+
+```mermaid
+graph TD
+    A[New Task] --> B{Complexity?}
+    B -->|Trivial: typo, format, rename| C[Gemini 3 Flash — Fast]
+    B -->|Standard: CRUD, endpoint, test| D[Gemini 3.1 Pro High — Fast]
+    B -->|Complex: architecture, migration| E[Claude Opus 4.6 — Planning]
+
+    E --> F[Opus creates plan]
+    F --> G[Gemini 3.1 Pro implements plan]
+    G --> H[Claude Sonnet reviews result]
+
+    style C fill:#4ade80
+    style D fill:#60a5fa
+    style E fill:#f472b6
+    style F fill:#f472b6
+    style G fill:#60a5fa
+    style H fill:#a78bfa
+```
+
+> [!TIP]
+> **Golden Rule**: Use Opus for **thinking**, Gemini for **doing**, Flash for **small details**.
+> Don't waste Opus on writing CRUD endpoints. Don't let Flash design architecture.
+
+### Model Switching Rule
+
+Switch the model **between tasks**, not in the middle of a task. Switching models resets conversation context.
+
+If a task requires different models (planning + implementation), use:
+
+1. Opus → creates `implementation_plan.md`
+2. Gemini → new conversation, reads plan, implements
+3. Sonnet → new conversation, reviews result
+
+---
+
+## 5. AGENTS.md — The Law of the Monorepo
+
+> [!IMPORTANT]
+> `AGENTS.md` is the **constitution** of the project. Everything written here has the highest priority.
+> The agent is **obligated** to follow these rules, even if its instructions say otherwise.
+
+### Structure of an Ideal AGENTS.md
+
+```markdown
+# Project Rules
+
+## Language & Style
+
+- All code comments and commit messages in English
+- Respond to the user in Russian (or their language)
+- Use TypeScript strict mode. No `any`.
+
+## Architecture Rules
+
+- Every package must have a README.md with Source Structure
+- No cross-package imports except through index.ts
+- Shared types go to packages/shared/types/
+- Services never import from routes (dependency direction: routes → services → repositories)
+
+## Code Conventions
+
+- Use Zod for all input validation
+- Use Result<T, E> pattern, no thrown exceptions in business logic
+- Logging: use structured JSON via Winston
+- Tests: Vitest, minimum 80% coverage per package
+
+## File Operations
+
+- Never modify files in packages/ you are not explicitly asked to work on
+- Always run `turbo build --filter=<package>` after changes
+- Run `turbo test --filter=<package>` before signaling done
+
+## Context Rules
+
+- Before modifying a package, ALWAYS read its README.md first
+- Do not scan entire monorepo. Work within the target package only.
+- If you need types from another package, read only its index.ts
+```
+
+### Why Each Rule Exists
+
+| Rule | Context Savings |
+| --- | --- |
+| "Work within the target package only" | The agent avoids reading 200 files from neighboring packages |
+| "Read README.md first" | 1 file instead of 20 `list_dir` + `view_file` |
+| "No cross-package imports except through index.ts" | The agent needs only 1 file to understand a package's API |
+| "Dependency direction" | The agent knows where to look and where not to |
+
+---
+
+## 6. Knowledge Items — Project Memory
+
+KI is the **encyclopedia** that the agent checks at the beginning of every conversation.
+
+### What to put in KI in a Monorepo
+
+| KI | Content | When needed |
+| --- | --- | --- |
+| `database-schema` | All tables, relations, indexes | When working with Prisma, migrations, queries |
+| `api-conventions` | Response formatting, error codes, pagination | When creating/editing endpoints |
+| `auth-flow` | JWT lifecycle, refresh logic, roles | When handling authentication |
+| `deployment-pipeline` | How to deploy, staging vs prod, env vars | When dealing with CI/CD |
+| `known-bugs` | Known bugs, workarounds | When debugging weird behavior |
+| `package-ownership` | Who owns what, areas of responsibility | When routing tasks in orchestrator |
+| `tech-debt-registry` | List of tech debt with priorities | When planning refactoring |
+
+### Rules for Writing KI
+
+```
+✅ Good:
+"API responses are always wrapped in { data: T, error: null } | { data: null, error: { code, message } }"
+
+❌ Bad:
+"We have a flexible and modern API architecture supporting many formats"
+```
+
+> [!TIP]
+> **Atomicity Rule**: One KI = One Topic. Don't write "everything about the project" into one file.
+> The agent only loads relevant KIs. The smaller the KI, the more accurate the selection.
+
+---
+
+## 7. Skills — Agent Abilities
+
+Skills are **instructions** on how to execute a specific, repeatable task.
+
+### Which Skills to Create in a Monorepo
+
+| Skill | Purpose | Example |
+| --- | --- | --- |
+| `create-package` | Scaffold a new package | tsconfig, package.json, README, template files |
+| `create-api-endpoint` | Scaffold a REST endpoint | route + service + validation + test |
+| `db-migration` | Prisma migration | schema change → prisma migrate → seed |
+| `add-feature-flag` | Setup a feature flag | config + check + rollout |
+| `create-shared-type` | New shared type | where to put, how to export |
+
+### Example SKILL.md
+
+```markdown
+---
+name: create-api-endpoint
+description: >-
+  Create a new REST API endpoint following project conventions.
+  Use when adding new routes to packages/api.
+---
+
+## When to Use
+
+- User asks to add a new API endpoint
+- User asks to add a new route
+
+## When NOT to Use
+
+- Modifying existing endpoints (just edit the file)
+- WebSocket endpoints (different pattern)
+
+## Steps
+
+1. Read packages/api/README.md for Source Structure
+2. Create route file in src/routes/v1/{resource}.ts
+3. Create service in src/services/{resource}.service.ts
+4. Add Zod validation schemas
+5. Register route in src/routes/v1/index.ts
+6. Create test in tests/routes/{resource}.test.ts
+7. Run: turbo test --filter=@monorepo/api
+
+## Template
+
+### Route (src/routes/v1/{resource}.ts)
+
+\`\`\`typescript
+/\*_ Route: GET/POST /api/v1/{resource} — {Resource} CRUD _/
+import { Router } from 'express';
+import { z } from 'zod';
+import { validate } from '../../middleware/validation';
+import { {Resource}Service } from '../../services/{resource}.service';
+
+const router = Router();
+
+const createSchema = z.object({
+name: z.string().min(1).max(255),
+});
+
+router.get('/', async (req, res, next) => { /_ ... _/ });
+router.post('/', validate(createSchema), async (req, res, next) => { /_ ... _/ });
+
+export default router;
+\`\`\`
+```
+
+---
+
+## 8. Workflows — Routine Automation
+
+Workflows are **step-by-step instructions** for frequently executed procedures.
+
+### Key Workflows for a Monorepo
+
+````markdown
+## <!-- .agents/workflows/test.md -->
+
+## description: Run tests for a specific package
+
+// turbo-all
+
+1. Determine the target package from user request
+2. Run tests:
+
+```bash
+cd /path/to/monorepo && npx turbo test --filter=@monorepo/{package}
+```
+````
+
+3. If tests fail, analyze the output and suggest fixes
+
+````
+
+```markdown
+<!-- .agents/workflows/build-check.md -->
+---
+description: Verify build for affected packages
+---
+// turbo-all
+
+1. Run type check:
+```bash
+cd /path/to/monorepo && npx turbo build --dry-run
+````
+
+2. Build only affected:
+
+```bash
+cd /path/to/monorepo && npx turbo build --filter=...[HEAD~1]
+```
+
+````
+
+```markdown
+<!-- .agents/workflows/new-package.md -->
+---
+description: Create a new package in the monorepo
+---
+1. Ask the user for: package name, description, dependencies
+2. Create directory structure (see create-package skill)
+3. Run: npm install
+4. Run: turbo build --filter=@monorepo/{name}
+5. Verify it builds successfully
+````
+
+> [!TIP]
+> **`// turbo-all`** is a magic annotation. It tells the agent to auto-approve all commands in the workflow.
+> Only use this for safe, read-only, or idempotent operations.
+
+---
+
+## 9. MCP Servers — External Tools
+
+> [!WARNING]
+> **Budget: ~100 tools** combined across all MCP servers, ~20K tokens for definitions.
+> Every unnecessary tool deteriorates the tool choice model. One `search` tool with routing is better than 100 endpoint wrapper tools.
+
+### Recommended MCPs for a Monorepo
+
+| MCP Server | Purpose | Context Cost |
+| --- | --- | --- |
+| `github-mcp-server` | PRs, issues, code search | ~5K tokens (many tools) |
+| `context7` | Framework documentation | ~2K tokens |
+| `docs-cache` | Documentation cache + notes | ~3K tokens |
+| Custom `project-search` | Semantic search over your repo | ~1K tokens |
+
+### Advice: Custom MCP for Navigation
+
+Create an MCP server that understands your monorepo structure:
+
+```typescript
+// tools:
+// - find_package(name) → returns README.md content + file list
+// - find_type(typeName) → returns file path + definition
+// - find_service(domain) → returns service files for a domain
+// - get_dependency_graph(packageName) → returns deps
+```
+
+This is **one MCP with 4 tools** instead of the agent doing 10+ `list_dir` + `grep_search`.
+
+---
+
+## 10. Context Saving Techniques
+
+### Technique 1: "Lazy Loading"
+
+```
+❌ Agent: view_file on 15 files sequentially, then starts thinking
+✅ Agent: reads README → understands structure → conditionally reads ONLY 2-3 files
+```
+
+Ensured by an AGENTS.md rule:
+
+```
+Before modifying a package, ALWAYS read its README.md first.
+Do not read more than 3 files before starting work.
+```
+
+### Technique 2: "Targeted Grep over Full Read"
+
+```
+❌ view_file("src/services/user.service.ts") — 500 lines
+✅ grep_search("createUser", "src/services/") → found line 42 → view_file(lines 35-65)
+```
+
+### Technique 3: "Types as Contracts"
+
+Instead of reading implementation — read only the types:
+
+```
+❌ view_file("packages/core/src/services/payment.service.ts") — 300 lines of implementation
+✅ view_file("packages/core/src/types.ts") — 50 lines of interfaces
+```
+
+This requires:
+
+- All public types extracted to `types.ts` or `types/`
+- `index.ts` re-exports only the public API
+- Implementation is inside `internal/`
+
+### Technique 4: "Context via Conventions"
+
+The stricter the conventions, the less the agent has to read:
+
+```
+If the agent knows:
+  - "All services are in src/services/{domain}.service.ts"
+  - "All tests mirror src/ inside tests/"
+  - "Validation is via Zod in src/validation/{domain}.schema.ts"
+
+Then it can predict file paths without running list_dir.
+```
+
+### Technique 5: "Package Barrier"
+
+```
+AGENTS.md:
+  "Never read files outside the target package unless explicitly asked.
+   If you need information from another package, read ONLY its:
+   1. README.md
+   2. src/index.ts (public API)
+   3. src/types.ts (type contracts)"
+```
+
+This is context restriction **by design**. The agent won't wander around 10 packages.
+
+### Technique 6: "Cascading Task Decomposition"
+
+For a complex task spanning 5+ packages:
+
+```
+Conversation 1 (Opus, Planning):
+  → Analyzes the task, creates implementation_plan.md
+  → Breaks it down into 5 subtasks, one per package
+
+Conversations 2–6 (Gemini Pro, Fast):
+  → Each conversation handles exactly one package
+  → Reads plan + package README
+  → Implements changes within one package only
+
+Conversation 7 (Sonnet, Planning):
+  → Integration testing
+  → turbo build, turbo test
+```
+
+### Technique 7: "Knowledge Item instead of Re-reading"
+
+If the agent reads the exact same database schema 3 times on the same day — turn it into a KI:
+
+```
+.agents/knowledge/database-schema/artifacts/schema.md
+```
+
+The KI is injected automatically, saving the agent a `view_file` call for the Prisma schema.
+
+### Technique 8: "Explicit Ownership in Pipeline"
+
+```json
+// pipeline.json — orchestrator knows who touches what
+{
+  "tasks": [
+    {
+      "id": "api-users-crud",
+      "owns": ["packages/api/src/routes/v1/users.ts", "packages/api/src/services/user.*"],
+      "role": "back"
+    }
+  ]
+}
+```
+
+The `owns` pattern prevents the agent from deviating out of bounds.
+
+---
+
+## 11. Antipatterns
+
+### ❌ Structural Antipatterns
+
+| Antipattern | Consequence | Solution |
+| --- | --- | --- |
+| All files in one `src/` folder | Agent reads 50+ files | Separate into domains / packages |
+| No README in packages | Agent wanders around wasting context | README + Source Structure |
+| Circular dependencies | Agent cannot determine scope | Strict dependency direction |
+| "God-file" (3000+ lines) | Agent loads massive amount of tokens | Split by responsibility |
+| Inconsistent names | Agent cannot predict paths | Strict naming conventions |
+
+### ❌ AI Antipatterns
+
+| Antipattern | Consequence | Solution |
+| --- | --- | --- |
+| "Do everything in one conversation" | Context overflows, quality drops | Cascading decomposition |
+| Opus for CRUD | Expensive, slow, overkill | Flash / Gemini Pro |
+| Flash for Architecture | Bad decisions | Opus in Planning mode |
+| No AGENTS.md | Agent invents conventions | Provide AGENTS.md |
+| All docs in Notion | Agent can't see them | Move to .agents/knowledge/ |
+| 50 MCP tools | Tool selection degrades | ≤20 tools, better routing |
+
+### ❌ Context Antipatterns
+
+| Antipattern | Consequence | Solution |
+| --- | --- | --- |
+| Opening 20 files in tabs | They pollute `ADDITIONAL_METADATA` | Close irrelevant files |
+| Long conversation (30+ prompts) | Checkpoints truncate, specifics lost | New conversation + handoff |
+| Copy-pasting raw trace | 500 lines of stack trace | Feed only core 10-15 lines |
+| Framing the task vaguely | Agent wastes context on clarification | Provide exact file scopes |
+
+---
+
+## 12. Checklist: Ideal Monorepo State
+
+### 🏗️ Structure
+
+- [ ] Every package is isolated and has its own `package.json`, `tsconfig.json`
+- [ ] Public API is exported via `index.ts`
+- [ ] Types are extracted to `types.ts` or `types/`
+- [ ] Implementation is in `internal/` (not imported directly)
+- [ ] Root `README.md` includes an architectural map
+- [ ] `README.md` in each package contains Source Structure
+
+### 🧠 Agent
+
+- [ ] `.agents/AGENTS.md` holds project rules
+- [ ] Knowledge Items for persistent knowledge (DB schema, API conventions, auth)
+- [ ] Skills for repeatable tasks (create endpoint, migration)
+- [ ] Workflows for routine procedures (test, build, deploy)
+- [ ] JSDoc headers in source files (requires when 20+ files in dir)
+
+### 📊 Models
+
+- [ ] Opus/Sonnet used for planning and reviewing
+- [ ] Gemini Pro used for implementation
+- [ ] Flash used for minor edits
+- [ ] Cascading strategy utilized for complex tasks
+
+### 🛡️ Context
+
+- [ ] "Read README first" rule in AGENTS.md
+- [ ] "Stay within package" rule in AGENTS.md
+- [ ] Close irrelevant tabs before the task
+- [ ] Decompose tasks into separated conversations (1 package = 1 convo)
+- [ ] Use `owns` property in pipeline to restrict scope
+
+---
+
+## Final Formula
+
+```
+Ideal Context Consumption =
+  Package README (≈200 tokens)
+  + 2–3 target files (≈2K tokens)
+  + Thematic KI (≈500 tokens)
+  + AGENTS.md rules (≈300 tokens)
+  ────────────────────────────
+  ≈ 3K tokens to grasp the task
+
+vs.
+
+Typical Context Unprepared =
+  15 × list_dir (≈3K)
+  + 20 × view_file (≈30K)
+  + 5 × grep_search (≈2K)
+  ────────────────────────────
+  ≈ 35K tokens wandering
+```
+
+> [!CAUTION]
+> **10x difference** in context consumption. With a budget of 70K–150K tokens per task,
+> this translates to the difference between "The agent solved the task flawlessly" and "The agent got confused by the context".
+
+---
+
+_Created: 2026-04-03 | Based on real-world experience using Antigravity IDE, 6 models, and AUTO_DEV pipelines._