ai-engineering-loop 1.10.0 → 1.11.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.agents/devil-advocate.md +13 -1
- package/.agents/judge.md +10 -0
- package/.agents/skills/backend-development/SKILL.md +272 -0
- package/.agents/skills/backend-development/references/api-design-guidelines.md +95 -0
- package/.agents/skills/backend-development/references/query-and-database-best-practices.md +53 -0
- package/.agents/skills/backend-development/references/security-checklist.md +60 -0
- package/.agents/workflows/ai-engineering-loop.md +4 -4
- package/.claude/agents/devil-advocate.md +13 -1
- package/.claude/agents/judge.md +10 -0
- package/.claude/skills/ai-engineering-loop/SKILL.md +4 -4
- package/.claude/skills/backend-development/SKILL.md +272 -0
- package/.claude/skills/backend-development/references/api-design-guidelines.md +95 -0
- package/.claude/skills/backend-development/references/query-and-database-best-practices.md +53 -0
- package/.claude/skills/backend-development/references/security-checklist.md +60 -0
- package/.gemini/skills/ai-engineering-loop/SKILL.md +4 -4
- package/.gemini/skills/backend-development/SKILL.md +272 -0
- package/.gemini/skills/backend-development/references/api-design-guidelines.md +95 -0
- package/.gemini/skills/backend-development/references/query-and-database-best-practices.md +53 -0
- package/.gemini/skills/backend-development/references/security-checklist.md +60 -0
- package/.grok/agents/devil-advocate.md +13 -1
- package/.grok/agents/judge.md +10 -0
- package/.grok/skills/ai-engineering-loop/SKILL.md +4 -4
- package/.grok/skills/backend-development/SKILL.md +272 -0
- package/.grok/skills/backend-development/references/api-design-guidelines.md +95 -0
- package/.grok/skills/backend-development/references/query-and-database-best-practices.md +53 -0
- package/.grok/skills/backend-development/references/security-checklist.md +60 -0
- package/README.md +33 -1
- package/README.npm.md +8 -1
- package/agents/shared/devil-advocate.body.md +13 -1
- package/agents/shared/judge.body.md +10 -0
- package/bin/ai-engineering-loop.js +207 -2
- package/core/runtime-safety.md +43 -0
- package/lib/budget.js +35 -0
- package/lib/doctor.js +4 -0
- package/lib/run-compaction.js +170 -0
- package/lib/runtime-policy.js +24 -1
- package/lib/safe-context.js +32 -7
- package/lib/smart-context.js +643 -0
- package/lib/studio-server.js +19 -2
- package/lib/sync-hosts.js +20 -0
- package/lib/update-manager.js +147 -0
- package/lib/verification-summary.js +130 -0
- package/package.json +1 -1
- package/policies/review-budget.md +12 -0
- package/studio/app.js +5 -2
|
@@ -43,20 +43,20 @@ Run `npx ai-engineering-loop state --json` to get the current `runId` and use `.
|
|
|
43
43
|
|
|
44
44
|
Read `core/runtime-safety.md`. The ledger mode is immutable. Before every model or Task call, run `npx ai-engineering-loop budget status --run <runId>`; after it returns, record actual provider-reported usage with `budget record --input <n> --output <n> --model <id> --run <runId>`. Never invent actual token counts. If usage metadata is unavailable, UNATTENDED stops and escalates; ASSISTED discloses the gap and asks the human whether to continue. `budget pause` is an emergency kill switch.
|
|
45
45
|
|
|
46
|
-
Inherited invariants remain mandatory: read `core/grill-policy.md`, `policies/tdd-policy.md`, and `.ai-engineering-loop/glossary.md`. On `adapter_type: dot`, grill includes blast radius: lifecycle sketch and load `task-impact-inquiry`. If the user prompt has no numbered AC, draft the contract, list a short menu and wait. Chat agreement is not freeze. AC is a failure table. One red test per AC row. Do not freeze sunny-path-only. Write claimed-vs-reality.md. Do not spawn Devil's Advocate if the file is missing or any Claimed row lacks Reality. Report Spec and Standards separately. DA prompt includes "at most 8 tool calls; read the diff file; skip css and generated files". Judge prompt includes "at most 4 tool calls; ledger and contract only; skip css; do not re-review the whole diff".
|
|
46
|
+
Inherited invariants remain mandatory: read `core/grill-policy.md`, `policies/tdd-policy.md`, and `.ai-engineering-loop/glossary.md`. On `adapter_type: dot`, grill includes blast radius: lifecycle sketch and load `task-impact-inquiry`. If the user prompt has no numbered AC, draft the contract, list a short menu and wait. Chat agreement is not freeze. AC is a failure table. One red test per AC row. Do not freeze sunny-path-only. Write claimed-vs-reality.md. Do not spawn Devil's Advocate if the file is missing or any Claimed row lacks Reality. Report Spec and Standards separately. Use `npx ai-engineering-loop context query <term>` and `context related <file>` before opening broad source files. Use the runtime policy `reviewProfile` (default `lean`) for context packs; do not upgrade to `standard` or `thorough` unless the human asks or the budget gate requires escalation. DA prompt includes "at most 8 tool calls; read the diff file; skip css and generated files". Judge prompt includes "at most 4 tool calls; ledger and contract only; skip css; do not re-review the whole diff".
|
|
47
47
|
|
|
48
48
|
1. Stage 0: `npx ai-engineering-loop sync-hosts`, `status`, then `run [--mode report-only|assisted|unattended]` to create or resume the ledger. Run `state --json` and `budget status`. Read `.ai-engineering-loop/glossary.md`. Read `.ai-engineering-loop/lessons.md` and `.ai-engineering-loop/workflow.md` if they exist. Follow workflow.md hooks (`before_grill`, `after_freeze`, `after_pass`). Do not skip required gates even if workflow.md asks. If workflow.md is missing, run the default 8-stage loop. If sync-hosts copied files, tell the user a new session is needed; keep going with this session.
|
|
49
49
|
2. Stage 1: Goal Contract (`core/goal-contract.md`). If the user asks for ideas, list a short menu and wait; do not implement. If the task is ambiguous and the user can answer, grill first (`core/grill-policy.md`): design tree, recommended answers, do not ask look-up facts. On business-logic change, and always on `adapter_type: dot`, include blast radius. Chat agreement is not freeze. Emit `npx ai-engineering-loop activity goal-drafting --run <runId> --message <summary>` before drafting. Write the contract JSON to a repository-local temporary file, then publish it with `npx ai-engineering-loop goal draft --run <runId> --file <file> --revision <currentRevision>` so Workflow Studio sees the exact draft and its ready state. Wait. After the user explicitly freezes, use `npx ai-engineering-loop goal freeze --run <runId> --revision <reviewedRevision> --hash <reviewedHash> --by human`; do not edit production code unless it passes. `gate goal` remains the legacy compatibility command, not the preferred revision-bound path.
|
|
50
50
|
3. After Goal freeze, branch on mode. REPORT_ONLY: do not edit repository files; write `RUN_DIR/report.json` with summary and recommendations, then run `gate report` and stop. ASSISTED: run the normal Maker and later require human approval. UNATTENDED: run `sandbox create --run <runId>`, set Maker's working directory to the returned worktreePath, never edit the parent checkout, then run `sandbox capture --run <runId>`. For Maker, read `maker_intern` from workflow.md (missing or none = parent is Maker). Do not type a model name. Confirm it through `/models` when available; otherwise parent is Maker and report INVOCATION_UNAVAILABLE. Intern cannot skip later gates. Bugs: red repro first. Features: one red test per AC row. In ASSISTED write the exact review diff to `RUN_DIR/diff.patch`; UNATTENDED capture writes it. Then run `npx ai-engineering-loop gate maker`.
|
|
51
|
-
4. Stage 5: run commands from `.ai-engineering-loop/verification.md`. Write `RUN_DIR/verification.json` with current git revision, gated diff hash, command, execution identity, times, exit code, stdout, timeout status, and test counts. Vague "seems green" is invalid. Keep `.ai-engineering-loop/tasks/claimed-vs-reality.md`. Run `npx ai-engineering-loop gate verification`; do not spawn Devil's Advocate unless it passes.
|
|
51
|
+
4. Stage 5: run commands from `.ai-engineering-loop/verification.md`. Write `RUN_DIR/verification.json` with current git revision, gated diff hash, command, execution identity, times, exit code, stdout, timeout status, and test counts. Then run `npx ai-engineering-loop verification summarize --run <runId>` and `npx ai-engineering-loop context compact --run <runId>`; pass run-summary.md and verification-summary.json to reviewers instead of raw logs whenever possible. Vague "seems green" is invalid. Keep `.ai-engineering-loop/tasks/claimed-vs-reality.md`. Run `npx ai-engineering-loop gate verification`; do not spawn Devil's Advocate unless it passes.
|
|
52
52
|
5. Write artifacts, then spawn. Before Devil's Advocate:
|
|
53
53
|
- Write `git diff` to a file (for example `.ai-engineering-loop/tasks/current.diff`).
|
|
54
54
|
- Write changed paths (`git diff --name-only`) into the Task prompt as a short list.
|
|
55
55
|
- Put those paths in the child prompt. Do not paste Maker rationale.
|
|
56
56
|
- Write `.ai-engineering-loop/tasks/claimed-vs-reality.md` if missing. Do not spawn DA without it.
|
|
57
57
|
- If stopping mid-loop, write `.ai-engineering-loop/tasks/handoff.md` (`core/handoff-policy.md`).
|
|
58
|
-
6. Stage 6: Build `npx ai-engineering-loop context devil-advocate <files...>`
|
|
59
|
-
7. Stage 7: Run `npx ai-engineering-loop escalation --json` before selecting Judge. Resolve the required tier through `/models`; if unavailable, escalate to the user. Build Judge context from only
|
|
58
|
+
6. Stage 6: Run `npx ai-engineering-loop context fast-path --json`; keep the default lean path when eligible. Build `npx ai-engineering-loop context diff-hunks --run <runId>` plus `npx ai-engineering-loop context devil-advocate <files...>` only for files whose hunks are insufficient. In iteration 2+, prefer the diff-hunk pack's unresolved findings and new hunks instead of replaying the whole diff. Recheck `budget status`, then Task `subagent_type: devil-advocate`. Use `general-purpose` only if rejected. Wait and immediately `budget record` its actual usage. Normalize its ledger into `RUN_DIR/findings.json` with runId and diffHash, then run `npx ai-engineering-loop gate review`.
|
|
59
|
+
7. Stage 7: Run `npx ai-engineering-loop escalation --json` before selecting Judge. Resolve the required tier through `/models`; if unavailable, escalate to the user. Build Judge context from only run-summary.md, verification-summary.json (or verification.json if the summary is missing), open-findings.json/findings.json, and the Goal hash; Judge must not read the full diff unless those summaries contradict each other. Recheck `budget status`, spawn Judge as a sibling and wait, then immediately `budget record` actual usage. Normalize its output into `RUN_DIR/verdict.json`, include `modelTier`, then run `npx ai-engineering-loop gate judge`. A gate failure overrides optimistic prose.
|
|
60
60
|
8. ITERATE with iteration under 3: fix in the parent, re-verify, spawn a **new** Devil's Advocate (do not resume the previous child).
|
|
61
61
|
9. After Judge PASS: run the `after_pass` hook, then Stage 8 from `.ai-engineering-loop/adapter.md`. PR/MR body includes `Steps to Reproduce & Testing (QA)`; bugfix placeholders must not stay empty. Preview remote payloads; do not expose context packs, usage ledgers, raw logs, or credentials. In ASSISTED, wait for explicit human approval and write `humanApproved: true`; never infer approval. Write `RUN_DIR/delivery.json` with destination and summary, then run `npx ai-engineering-loop gate delivery`. Only this gate completes the run.
|
|
62
62
|
|
|
@@ -0,0 +1,272 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: backend-development
|
|
3
|
+
description: Framework-agnostic backend best practices for clean code, naming, queries/databases, REST APIs, errors, security, performance, scalability, testing, and structure. Trigger on backend implementation, architecture, code quality, naming, magic values, complex conditionals, query optimization, caching, error handling, or security tasks.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Backend Development Guidelines
|
|
7
|
+
|
|
8
|
+
Universal principles for reliable, maintainable, performant backends — regardless of framework or language.
|
|
9
|
+
|
|
10
|
+
Framework-specific conventions in `CLAUDE.md`, `AGENTS.md`, README files, or project docs take precedence for implementation details. This skill governs the underlying engineering principles.
|
|
11
|
+
|
|
12
|
+
## How to use this skill
|
|
13
|
+
|
|
14
|
+
- Apply these rules during backend implementation, review, refactoring, architecture, naming, database, API, error-handling, security, performance, and testing work.
|
|
15
|
+
- Prefer the project’s established patterns when they are coherent. Use this skill to improve unclear, unsafe, or over-complex code.
|
|
16
|
+
- Do not introduce speculative architecture. Fix the rule causing the problem, with the smallest coherent design.
|
|
17
|
+
- Read reference files only when depth is needed:
|
|
18
|
+
- `references/query-and-database-best-practices.md`
|
|
19
|
+
- `references/api-design-guidelines.md`
|
|
20
|
+
- `references/security-checklist.md`
|
|
21
|
+
|
|
22
|
+
---
|
|
23
|
+
|
|
24
|
+
## 1. Clean Code Principles
|
|
25
|
+
|
|
26
|
+
### Single Responsibility
|
|
27
|
+
|
|
28
|
+
Every function, class, and module should have one reason to exist. If you need “and” to describe it — validates input and saves to database and sends notification — split it.
|
|
29
|
+
|
|
30
|
+
- Functions do one thing. A data fetcher should not format presentation output.
|
|
31
|
+
- Services/classes own one domain concept. `UserService` handles user logic, not email delivery.
|
|
32
|
+
- Modules encapsulate one bounded context. Cross-module access flows through well-defined interfaces.
|
|
33
|
+
|
|
34
|
+
### Meaningful Names
|
|
35
|
+
|
|
36
|
+
Names reveal intent without requiring a comment.
|
|
37
|
+
|
|
38
|
+
- Variables: `activeUserCount`, `invoiceDueDate`, `orderItems`.
|
|
39
|
+
- Functions: verb-first and specific, e.g. `calculateShippingCost`, `archiveExpiredCoupons`, `validateEmailFormat`.
|
|
40
|
+
- Classes/types: role-suffixed nouns, e.g. `UserService`, `OrderRepository`, `PaymentController`, `ProductDto`.
|
|
41
|
+
- Abbreviations: only universally understood ones like `id`, `url`, `dto`, `http`.
|
|
42
|
+
|
|
43
|
+
### Boolean naming
|
|
44
|
+
|
|
45
|
+
Boolean names must read as yes/no propositions. Use prefixes that reveal the type and intent:
|
|
46
|
+
|
|
47
|
+
| Prefix | Reads as | Example |
|
|
48
|
+
|---|---|---|
|
|
49
|
+
| `is` | Is it in this state? | `isActive`, `isEmailVerified` |
|
|
50
|
+
| `has` | Does it own this? | `hasPermission`, `hasSubscription` |
|
|
51
|
+
| `can` | Is it allowed or able? | `canRetry`, `canRefund` |
|
|
52
|
+
| `should` | Ought it happen? | `shouldNotify`, `shouldRetry` |
|
|
53
|
+
| `was` | Did it happen? | `wasDeleted`, `wasCharged` |
|
|
54
|
+
| `will` | Is it going to happen? | `willExpire`, `willRenew` |
|
|
55
|
+
|
|
56
|
+
Avoid ambiguous booleans like `active`, `verified`, `enabled`, `permission`, or `status` unless the type is intentionally not boolean.
|
|
57
|
+
|
|
58
|
+
```typescript
|
|
59
|
+
// Bad: string? number? boolean?
|
|
60
|
+
const active = user.status
|
|
61
|
+
const verification = checkEmail(email)
|
|
62
|
+
|
|
63
|
+
// Good: yes/no at a glance
|
|
64
|
+
const isActive = user.status === 'active'
|
|
65
|
+
const isEmailVerified = checkEmailVerification(email)
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
### DRY, KISS, YAGNI
|
|
69
|
+
|
|
70
|
+
- Extract repeated logic when changing one copy would force changes everywhere.
|
|
71
|
+
- Do not abstract merely because code looks similar twice; similar code may diverge.
|
|
72
|
+
- Prefer straightforward solutions. Patterns and abstractions must earn their complexity.
|
|
73
|
+
- Build for today’s requirements. Add extension points when the requirement arrives.
|
|
74
|
+
|
|
75
|
+
### Guard clauses and early returns
|
|
76
|
+
|
|
77
|
+
Handle edge cases at the top and return early. Keep the happy path at the lowest indentation level.
|
|
78
|
+
|
|
79
|
+
### Extract complex conditions
|
|
80
|
+
|
|
81
|
+
A condition should read as intent, not mechanics. Name the rule before testing it.
|
|
82
|
+
|
|
83
|
+
```typescript
|
|
84
|
+
const meetsAgeRequirement = user.age >= MIN_AGE
|
|
85
|
+
const isIdentityVerified = user.kycStatus === 'verified'
|
|
86
|
+
const isInGoodStanding = !user.isBanned && user.region !== 'restricted'
|
|
87
|
+
const hasPremiumAccess = user.accountType === 'premium'
|
|
88
|
+
|| user.loyaltyPoints > LOYALTY_THRESHOLD
|
|
89
|
+
|
|
90
|
+
const isEligibleForAccess = meetsAgeRequirement
|
|
91
|
+
&& isIdentityVerified
|
|
92
|
+
&& isInGoodStanding
|
|
93
|
+
&& hasPremiumAccess
|
|
94
|
+
|
|
95
|
+
if (isEligibleForAccess) {
|
|
96
|
+
grantAccess()
|
|
97
|
+
}
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
Rule of thumb: if the reader must parse operators, extract a named variable. This applies to a single domain comparison too when the mechanism hides the business rule.
|
|
101
|
+
|
|
102
|
+
### Function size
|
|
103
|
+
|
|
104
|
+
Aim for 20–30 lines. If a function exceeds 40 lines, look for extraction opportunities. A block that needs a section comment is a candidate for its own function.
|
|
105
|
+
|
|
106
|
+
### No magic numbers or strings
|
|
107
|
+
|
|
108
|
+
Extract domain-meaningful literals into named constants: status codes, thresholds, retry counts, timeouts, role IDs, page limits.
|
|
109
|
+
|
|
110
|
+
```typescript
|
|
111
|
+
const ROLE_ADMIN = 3
|
|
112
|
+
const ONE_DAY_MS = 24 * 60 * 60 * 1000
|
|
113
|
+
const MAX_ITEMS_PER_ORDER = 50
|
|
114
|
+
|
|
115
|
+
const isAdmin = user.role === ROLE_ADMIN
|
|
116
|
+
const isSessionStale = Date.now() - lastLogin > ONE_DAY_MS
|
|
117
|
+
const exceedsItemLimit = order.items.length > MAX_ITEMS_PER_ORDER
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
### Comments
|
|
121
|
+
|
|
122
|
+
A comment is often a failure to make the code say it itself. Before adding one, try renaming or extracting.
|
|
123
|
+
|
|
124
|
+
Keep comments for intent, protocol details, warning of consequences, public API docs, legal headers, and living TODOs. Delete comments that restate code, drift out of date, journal changes, attribute authors, or preserve commented-out code.
|
|
125
|
+
|
|
126
|
+
### Command-Query Separation
|
|
127
|
+
|
|
128
|
+
Functions that return data should avoid side effects. Functions that mutate state should make mutation explicit. Reads are safe; writes are deliberate.
|
|
129
|
+
|
|
130
|
+
---
|
|
131
|
+
|
|
132
|
+
## 2. Naming Conventions
|
|
133
|
+
|
|
134
|
+
| Context | Convention | Examples |
|
|
135
|
+
|---|---|---|
|
|
136
|
+
| Variables, parameters | Descriptive nouns, language-standard casing | `userId`, `orderItems`, `isValid` |
|
|
137
|
+
| Functions, methods | Verb-first | `getUser`, `createOrder`, `validateInput`, `isExpired` |
|
|
138
|
+
| Classes, types | PascalCase nouns, role suffix | `UserService`, `OrderRepository`, `PaymentController` |
|
|
139
|
+
| Constants | UPPER_SNAKE_CASE | `MAX_RETRY_COUNT`, `DEFAULT_PAGE_SIZE` |
|
|
140
|
+
| Database columns | snake_case | `created_at`, `user_id`, `is_active` |
|
|
141
|
+
| Database tables | snake_case plural | `users`, `order_items`, `payment_transactions` |
|
|
142
|
+
| API endpoints | lowercase plural nouns, kebab-case | `/users`, `/order-items` |
|
|
143
|
+
| Enum values | UPPER_SNAKE_CASE or language convention | `PENDING`, `IN_PROGRESS` |
|
|
144
|
+
|
|
145
|
+
Foreign keys: `{referenced_table_singular}_id`, e.g. `user_id`.
|
|
146
|
+
|
|
147
|
+
Boolean columns: `is_active`, `is_verified`, `has_subscription`.
|
|
148
|
+
|
|
149
|
+
Timestamp columns: suffix `_at`, e.g. `created_at`, `updated_at`, `deleted_at`, `expires_at`.
|
|
150
|
+
|
|
151
|
+
---
|
|
152
|
+
|
|
153
|
+
## 3. Query and Database Best Practices
|
|
154
|
+
|
|
155
|
+
Poor query patterns are a common backend bottleneck.
|
|
156
|
+
|
|
157
|
+
- Prevent N+1 queries with joins, eager loading, or batch loading.
|
|
158
|
+
- Select only needed columns. Avoid `SELECT *` in application queries.
|
|
159
|
+
- Index foreign keys and frequent `WHERE` / `ORDER BY` fields.
|
|
160
|
+
- Composite indexes follow leftmost prefix rules.
|
|
161
|
+
- Paginate list endpoints; never return unbounded lists.
|
|
162
|
+
- Use transactions for multi-step writes. Keep transactions short.
|
|
163
|
+
- Use connection pools sized to expected concurrency.
|
|
164
|
+
|
|
165
|
+
Read `references/query-and-database-best-practices.md` for schema, query, pagination, migration, and pool-sizing details.
|
|
166
|
+
|
|
167
|
+
---
|
|
168
|
+
|
|
169
|
+
## 4. API Design Principles
|
|
170
|
+
|
|
171
|
+
A good API is consistent and predictable.
|
|
172
|
+
|
|
173
|
+
- Use resource nouns, not verbs: `POST /orders`, not `/create-order`.
|
|
174
|
+
- Use plural collections: `/users`, `/orders`, `/products`.
|
|
175
|
+
- Nest sub-resources at most two levels; flatten deeper models.
|
|
176
|
+
- Keep response envelopes consistent for success and error.
|
|
177
|
+
- Version breaking API contracts explicitly, usually `/v1`, `/v2`.
|
|
178
|
+
- Validate filters and sort fields against allow-lists.
|
|
179
|
+
- Include pagination metadata.
|
|
180
|
+
- For long-running work, return `202 Accepted` with a task/status resource.
|
|
181
|
+
|
|
182
|
+
Read `references/api-design-guidelines.md` for endpoint patterns, envelopes, pagination, filtering, bulk operations, and versioning.
|
|
183
|
+
|
|
184
|
+
---
|
|
185
|
+
|
|
186
|
+
## 5. Error Handling
|
|
187
|
+
|
|
188
|
+
Treat error paths with the same care as success paths.
|
|
189
|
+
|
|
190
|
+
- Use consistent error shape: HTTP status, machine-readable code, human-readable message, optional details.
|
|
191
|
+
- Do not expose stack traces, internal paths, raw SQL errors, secret values, or implementation details.
|
|
192
|
+
- Validate at the API boundary and fail fast.
|
|
193
|
+
- User-facing messages must answer what went wrong and what to do next.
|
|
194
|
+
|
|
195
|
+
```text
|
|
196
|
+
Bad: ValidationError: field 'shipping_zone_id' violates constraint fk_order_zone
|
|
197
|
+
Good: Cannot save this order. The selected shipping zone is no longer available. Choose a different shipping zone and save again.
|
|
198
|
+
```
|
|
199
|
+
|
|
200
|
+
Use specific status codes: `400`, `401`, `403`, `404`, `409`, `422`, `429`, `500`, `502`, `503`.
|
|
201
|
+
|
|
202
|
+
Structured logs should include request ID, user ID when available, action, and sanitized error context. Never log passwords, tokens, API keys, or PII.
|
|
203
|
+
|
|
204
|
+
---
|
|
205
|
+
|
|
206
|
+
## 6. Security
|
|
207
|
+
|
|
208
|
+
Security is a design constraint.
|
|
209
|
+
|
|
210
|
+
- Validate all external input at boundaries.
|
|
211
|
+
- Use parameterized queries or query builders; never concatenate user input into SQL, shell commands, or template expressions.
|
|
212
|
+
- Store secrets in env vars or secret managers, never source code.
|
|
213
|
+
- Hash passwords with bcrypt, argon2, or scrypt.
|
|
214
|
+
- Enforce authorization at API boundaries and verify resource ownership.
|
|
215
|
+
- Never rely solely on frontend checks.
|
|
216
|
+
- Mask sensitive fields in API responses and logs.
|
|
217
|
+
- Use HTTPS everywhere.
|
|
218
|
+
|
|
219
|
+
Read `references/security-checklist.md` for auth, authorization, validation, headers, dependency security, and vulnerability prevention.
|
|
220
|
+
|
|
221
|
+
---
|
|
222
|
+
|
|
223
|
+
## 7. Performance and Scalability
|
|
224
|
+
|
|
225
|
+
Design for predictable performance under load.
|
|
226
|
+
|
|
227
|
+
- Keep services stateless; externalize session/state to Redis/database when needed.
|
|
228
|
+
- Cache expensive operations with explicit invalidation strategy.
|
|
229
|
+
- Use eager loading when related data is always needed; lazy loading only when rare and safe.
|
|
230
|
+
- Use bulk operations for mass writes/deletes.
|
|
231
|
+
- Offload long-running work to queues/background jobs.
|
|
232
|
+
- Enable response compression.
|
|
233
|
+
- Limit request body size and paginate/stream large payloads.
|
|
234
|
+
|
|
235
|
+
---
|
|
236
|
+
|
|
237
|
+
## 8. Code Organization
|
|
238
|
+
|
|
239
|
+
Use layers unless the operation is trivially simple:
|
|
240
|
+
|
|
241
|
+
1. Controller/handler: request parsing, validation, response mapping. No business logic.
|
|
242
|
+
2. Service: business logic and orchestration. Framework-agnostic where possible.
|
|
243
|
+
3. Repository/data access: persistence and schema-aware queries.
|
|
244
|
+
|
|
245
|
+
Rules:
|
|
246
|
+
|
|
247
|
+
- Services should not know HTTP status codes.
|
|
248
|
+
- Controllers should not contain SQL.
|
|
249
|
+
- Repositories should not enforce business workflows.
|
|
250
|
+
- Use dependency injection; avoid `new` inside services for dependencies.
|
|
251
|
+
- Domain modules own their entities, DTOs, services, and repositories.
|
|
252
|
+
- Cross-module communication goes through service interfaces.
|
|
253
|
+
- Shared utilities must be truly shared; avoid tangled abstractions.
|
|
254
|
+
- Externalize configuration into typed config objects.
|
|
255
|
+
|
|
256
|
+
---
|
|
257
|
+
|
|
258
|
+
## 9. Testing Strategy
|
|
259
|
+
|
|
260
|
+
Test behavior and business rules, not framework internals.
|
|
261
|
+
|
|
262
|
+
- Unit tests: business logic and edge cases.
|
|
263
|
+
- Integration tests: repositories, real database behavior, HTTP endpoints.
|
|
264
|
+
- E2E tests: critical flows only.
|
|
265
|
+
|
|
266
|
+
Cover happy paths, empty inputs, boundary values, invalid inputs, authorization failures, and business rule violations.
|
|
267
|
+
|
|
268
|
+
Avoid testing private methods, simple getters, third-party behavior, and framework plumbing unless your code configures it in a risky way.
|
|
269
|
+
|
|
270
|
+
Name tests by scenario and outcome: `should return 404 when user does not exist`.
|
|
271
|
+
|
|
272
|
+
Use factories/builders for test data. Each test owns its setup and cleanup.
|
|
@@ -0,0 +1,95 @@
|
|
|
1
|
+
# API Design Guidelines
|
|
2
|
+
|
|
3
|
+
Reference for designing consistent, predictable REST APIs.
|
|
4
|
+
|
|
5
|
+
## HTTP method semantics
|
|
6
|
+
|
|
7
|
+
| Method | Purpose | Idempotent | Safe | Typical Success |
|
|
8
|
+
|---|---|---|---|---|
|
|
9
|
+
| GET | Retrieve resources | Yes | Yes | 200 |
|
|
10
|
+
| POST | Create resource | No | No | 201 |
|
|
11
|
+
| PUT | Replace resource | Yes | No | 200 |
|
|
12
|
+
| PATCH | Partially update | Yes | No | 200 |
|
|
13
|
+
| DELETE | Remove resource | Yes | No | 200 or 204 |
|
|
14
|
+
|
|
15
|
+
## Resource URL patterns
|
|
16
|
+
|
|
17
|
+
```text
|
|
18
|
+
GET /v1/products
|
|
19
|
+
POST /v1/products
|
|
20
|
+
GET /v1/products/{id}
|
|
21
|
+
PUT /v1/products/{id}
|
|
22
|
+
PATCH /v1/products/{id}
|
|
23
|
+
DELETE /v1/products/{id}
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
Sub-resources should usually stop at two levels:
|
|
27
|
+
|
|
28
|
+
```text
|
|
29
|
+
GET /v1/users/{id}/orders
|
|
30
|
+
GET /v1/users/{id}/orders/{order_id}
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
For non-CRUD operations, use action sub-resources sparingly:
|
|
34
|
+
|
|
35
|
+
```text
|
|
36
|
+
POST /v1/orders/{id}/cancel
|
|
37
|
+
POST /v1/users/{id}/verify-email
|
|
38
|
+
POST /v1/reports/generate
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
## Response envelopes
|
|
42
|
+
|
|
43
|
+
Success:
|
|
44
|
+
|
|
45
|
+
```json
|
|
46
|
+
{ "status_code": 200, "message": "Success", "data": {} }
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
Paginated success:
|
|
50
|
+
|
|
51
|
+
```json
|
|
52
|
+
{
|
|
53
|
+
"status_code": 200,
|
|
54
|
+
"message": "Success",
|
|
55
|
+
"data": [],
|
|
56
|
+
"meta": { "total": 150, "page": 3, "limit": 20, "total_pages": 8 }
|
|
57
|
+
}
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
Error:
|
|
61
|
+
|
|
62
|
+
```json
|
|
63
|
+
{
|
|
64
|
+
"status_code": 422,
|
|
65
|
+
"message": "Validation failed",
|
|
66
|
+
"error": "VALIDATION_ERROR",
|
|
67
|
+
"details": [{ "field": "email", "message": "must be a valid email address" }]
|
|
68
|
+
}
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
## Pagination
|
|
72
|
+
|
|
73
|
+
Offset pagination: `page`, `limit`. Cap `limit`.
|
|
74
|
+
|
|
75
|
+
Cursor pagination: `after`, `limit`, response `next_cursor`. Prefer for feeds and large datasets.
|
|
76
|
+
|
|
77
|
+
## Filtering and sorting
|
|
78
|
+
|
|
79
|
+
Filters as query params: `?status=active&role=admin`.
|
|
80
|
+
|
|
81
|
+
Sort with allow-listed fields: `?sort=-created_at,name`.
|
|
82
|
+
|
|
83
|
+
Reject unsupported filters/sorts instead of silently ignoring them.
|
|
84
|
+
|
|
85
|
+
## Bulk operations
|
|
86
|
+
|
|
87
|
+
Batch create/delete should report partial success when appropriate. Include per-item errors with indexes or IDs.
|
|
88
|
+
|
|
89
|
+
## Long-running operations
|
|
90
|
+
|
|
91
|
+
Return `202 Accepted` with a task ID, process asynchronously, and expose a status endpoint.
|
|
92
|
+
|
|
93
|
+
## Versioning
|
|
94
|
+
|
|
95
|
+
Use URL path versioning (`/v1`). Never introduce breaking changes to a published version. Adding optional fields/endpoints is usually non-breaking; removing fields or changing types is breaking.
|
|
@@ -0,0 +1,53 @@
|
|
|
1
|
+
# Query and Database Best Practices
|
|
2
|
+
|
|
3
|
+
Reference for database design, query optimization, and data access patterns.
|
|
4
|
+
|
|
5
|
+
## N+1 query prevention
|
|
6
|
+
|
|
7
|
+
N+1 happens when code fetches N parent records, then one query per parent for related data. Prevent it with joins, eager loading, or batch loading using `WHERE parent_id IN (...)`. Enable query logging in development to catch repeated query shapes.
|
|
8
|
+
|
|
9
|
+
## Index strategy
|
|
10
|
+
|
|
11
|
+
- B-tree indexes support equality, ranges, sorting, and prefix matching.
|
|
12
|
+
- Index foreign keys, frequent `WHERE` columns, and `ORDER BY` columns used in pagination.
|
|
13
|
+
- Composite indexes follow leftmost prefix: `(status, created_at)` supports `status` and `status + created_at`, not `created_at` alone.
|
|
14
|
+
- Partial indexes help when queries target a minority subset.
|
|
15
|
+
- Covering indexes help read-heavy queries when the index contains all selected columns.
|
|
16
|
+
- Avoid over-indexing write-heavy tables.
|
|
17
|
+
|
|
18
|
+
## Query anti-patterns
|
|
19
|
+
|
|
20
|
+
| Anti-pattern | Problem | Fix |
|
|
21
|
+
|---|---|---|
|
|
22
|
+
| `SELECT *` | Wastes bandwidth and may expose data | Select needed columns |
|
|
23
|
+
| Function in WHERE | Prevents index use | Use range predicates |
|
|
24
|
+
| Leading wildcard LIKE | Cannot use normal B-tree | Full-text or suffix strategy |
|
|
25
|
+
| Implicit conversion | Forces row conversion | Match types |
|
|
26
|
+
| OR across columns | Often skips indexes | UNION indexed queries |
|
|
27
|
+
| Unbounded list | Memory/latency risk | Pagination/limit |
|
|
28
|
+
| DISTINCT as fix | Hides bad joins | Fix join logic |
|
|
29
|
+
| Correlated subquery per row | Repeated work | JOIN/lateral/batch |
|
|
30
|
+
|
|
31
|
+
## Pagination
|
|
32
|
+
|
|
33
|
+
Offset: simple, supports jump to page N, degrades at high offsets.
|
|
34
|
+
|
|
35
|
+
Cursor: stable performance at depth, better for feeds/large datasets, cannot jump to arbitrary page.
|
|
36
|
+
|
|
37
|
+
## Transactions
|
|
38
|
+
|
|
39
|
+
Keep transactions short. Compute and validate before opening. Open, write, commit. Always rollback on failure.
|
|
40
|
+
|
|
41
|
+
Use the lowest isolation level that satisfies consistency. Higher isolation reduces throughput.
|
|
42
|
+
|
|
43
|
+
## Bulk operations
|
|
44
|
+
|
|
45
|
+
Use batch inserts/updates/deletes instead of per-row loops. Start with batches around 500–1000 rows and tune.
|
|
46
|
+
|
|
47
|
+
## Migrations
|
|
48
|
+
|
|
49
|
+
Prefer additive migrations for zero-downtime deploys. For renames/removals: add new, migrate data, deploy code, then remove old later. Never edit migrations already applied to shared environments.
|
|
50
|
+
|
|
51
|
+
## Connection pooling
|
|
52
|
+
|
|
53
|
+
Start with conservative pool sizes, monitor utilization and latency, then adjust. Too few connections queue requests; too many exhaust database resources.
|
|
@@ -0,0 +1,60 @@
|
|
|
1
|
+
# Security Checklist
|
|
2
|
+
|
|
3
|
+
Audit guide for authentication, authorization, input handling, and data protection.
|
|
4
|
+
|
|
5
|
+
## Input validation
|
|
6
|
+
|
|
7
|
+
Validate at API boundaries: type, length, range, format, enum membership, array size, uniqueness, file type by content, file size, UUID shape, and pagination limits.
|
|
8
|
+
|
|
9
|
+
Use schema validation libraries. Never trust frontend validation.
|
|
10
|
+
|
|
11
|
+
## Authentication
|
|
12
|
+
|
|
13
|
+
- Use short-lived access tokens.
|
|
14
|
+
- Use refresh tokens securely and rotate on use when appropriate.
|
|
15
|
+
- Support revocation for logout/incidents.
|
|
16
|
+
- Hash passwords with bcrypt, argon2, or scrypt.
|
|
17
|
+
- Store secrets in env vars or secret managers, not source code.
|
|
18
|
+
|
|
19
|
+
## Authorization
|
|
20
|
+
|
|
21
|
+
- Define specific roles/permissions.
|
|
22
|
+
- Check permissions at API boundaries.
|
|
23
|
+
- Verify resource ownership to prevent IDOR.
|
|
24
|
+
- Prevent mass assignment by accepting explicit DTO fields only.
|
|
25
|
+
- Test that users cannot access or mutate others’ resources.
|
|
26
|
+
|
|
27
|
+
## Data protection
|
|
28
|
+
|
|
29
|
+
- HTTPS everywhere.
|
|
30
|
+
- Encrypt sensitive data at rest where required.
|
|
31
|
+
- Minimize PII collection.
|
|
32
|
+
- Mask PII/secrets in logs and responses.
|
|
33
|
+
- Implement retention/deletion policies where applicable.
|
|
34
|
+
|
|
35
|
+
## HTTP security headers
|
|
36
|
+
|
|
37
|
+
Configure HSTS, X-Content-Type-Options, X-Frame-Options or CSP frame policy, Content-Security-Policy, Referrer-Policy, and Permissions-Policy according to the app surface.
|
|
38
|
+
|
|
39
|
+
## Dependency security
|
|
40
|
+
|
|
41
|
+
Audit dependencies regularly, update vulnerable packages, pin major versions, and review new dependencies for maintenance and risk.
|
|
42
|
+
|
|
43
|
+
## Common vulnerability prevention
|
|
44
|
+
|
|
45
|
+
| Vulnerability | Prevention |
|
|
46
|
+
|---|---|
|
|
47
|
+
| SQL injection | Parameterized queries / query builders |
|
|
48
|
+
| NoSQL injection | Validate operators and input shape |
|
|
49
|
+
| XSS | Escape output, sanitize HTML, set CSP |
|
|
50
|
+
| CSRF | SameSite cookies and CSRF tokens for browser state changes |
|
|
51
|
+
| IDOR | Ownership/permission checks |
|
|
52
|
+
| Mass assignment | Explicit DTO allow-lists |
|
|
53
|
+
| SSRF | URL allow-list and internal network blocking |
|
|
54
|
+
| Command injection | Avoid shell; pass args safely |
|
|
55
|
+
| Path traversal | Canonicalize and enforce allowed root |
|
|
56
|
+
| Open redirect | Allow-list redirect targets |
|
|
57
|
+
|
|
58
|
+
## Logging security
|
|
59
|
+
|
|
60
|
+
Log auth events, authorization failures, validation failures, system errors, and admin actions. Do not log passwords, full tokens, API keys, card data, health data, or full request/response bodies with PII.
|
|
@@ -36,20 +36,20 @@ Run `npx ai-engineering-loop state --json` to get the current runId and use `.ai
|
|
|
36
36
|
|
|
37
37
|
Read `core/runtime-safety.md`. Before every model or subagent call, run `budget status --run <runId>` and afterward run `budget record --input <actual> --output <actual> --model <id> --run <runId>` using provider-reported usage. Never invent usage. Missing usage stops UNATTENDED and requires human direction in ASSISTED. `budget pause` is the kill switch.
|
|
38
38
|
|
|
39
|
-
Inherited invariants remain mandatory: read `core/grill-policy.md`, `policies/tdd-policy.md`, and `.ai-engineering-loop/glossary.md`. On `adapter_type: dot`, grill includes blast radius: lifecycle sketch and load `task-impact-inquiry`. If the user prompt has no numbered AC, draft the contract, list a short menu and wait. Chat agreement is not freeze. AC is a failure table. One red test per AC row. Do not freeze sunny-path-only. Write claimed-vs-reality.md. Do not spawn Devil's Advocate if the file is missing or any Claimed row lacks Reality. Report Spec and Standards separately. DA prompt includes "at most 8 tool calls; read the diff file; skip css and generated files". Judge prompt includes "at most 4 tool calls; ledger and contract only; skip css; do not re-review the whole diff". `generate-workflow` still reads workflow.md and lessons.md; missing maker_intern means INVOCATION_UNAVAILABLE. Do not type a model name.
|
|
39
|
+
Inherited invariants remain mandatory: read `core/grill-policy.md`, `policies/tdd-policy.md`, and `.ai-engineering-loop/glossary.md`. On `adapter_type: dot`, grill includes blast radius: lifecycle sketch and load `task-impact-inquiry`. If the user prompt has no numbered AC, draft the contract, list a short menu and wait. Chat agreement is not freeze. AC is a failure table. One red test per AC row. Do not freeze sunny-path-only. Write claimed-vs-reality.md. Do not spawn Devil's Advocate if the file is missing or any Claimed row lacks Reality. Report Spec and Standards separately. Use `npx ai-engineering-loop context query <term>` and `context related <file>` before opening broad source files. Use the runtime policy `reviewProfile` (default `lean`) for context packs; do not upgrade to `standard` or `thorough` unless the human asks or the budget gate requires escalation. DA prompt includes "at most 8 tool calls; read the diff file; skip css and generated files". Judge prompt includes "at most 4 tool calls; ledger and contract only; skip css; do not re-review the whole diff". `generate-workflow` still reads workflow.md and lessons.md; missing maker_intern means INVOCATION_UNAVAILABLE. Do not type a model name.
|
|
40
40
|
|
|
41
41
|
1. Stage 0: sync hosts, check status, then run `npx ai-engineering-loop run [--mode report-only|assisted|unattended]` to create or resume the ledger. Run `state --json` and `budget status`. Read glossary, lessons, and workflow overlays. Required gates cannot be skipped.
|
|
42
42
|
2. Stage 1: grill ambiguous decisions and business blast radius first. Emit `npx ai-engineering-loop activity goal-drafting --run <runId> --message <summary>`. Write the Goal Contract JSON to a repository-local temporary file, then publish it with `npx ai-engineering-loop goal draft --run <runId> --file <file> --revision <currentRevision>` so Workflow Studio sees it and its ready state. Wait. Chat agreement is not freeze. After explicit human freeze, use `npx ai-engineering-loop goal freeze --run <runId> --revision <reviewedRevision> --hash <reviewedHash> --by human`; do not edit production code unless it passes. `gate goal` remains the legacy compatibility command, not the preferred revision-bound path.
|
|
43
43
|
3. After Goal freeze, branch on mode. REPORT_ONLY: never edit repository files; write `RUN_DIR/report.json`, run `gate report`, and stop. ASSISTED: use the normal Maker. UNATTENDED: run `sandbox create --run <runId>`, run Maker only in the returned worktreePath, never edit the parent checkout, then run `sandbox capture --run <runId>`. Resolve maker_intern only through the host catalog; otherwise parent is Maker. Intern cannot skip gates. Use red repro or TDD. ASSISTED writes `RUN_DIR/diff.patch`; UNATTENDED capture writes it. Then run `npx ai-engineering-loop gate maker`.
|
|
44
|
-
4. Stage 5: write `RUN_DIR/verification.json` with current git revision, gated diff hash, command, execution identity, times, exit code, stdout, timeout status, and test counts. Keep claimed-vs-reality.md. Run `npx ai-engineering-loop gate verification`; do not spawn DA unless it passes.
|
|
44
|
+
4. Stage 5: write `RUN_DIR/verification.json` with current git revision, gated diff hash, command, execution identity, times, exit code, stdout, timeout status, and test counts. Run `npx ai-engineering-loop verification summarize --run <runId>` and `npx ai-engineering-loop context compact --run <runId>`; pass run-summary.md and verification-summary.json to reviewers instead of raw logs whenever possible. Keep claimed-vs-reality.md. Run `npx ai-engineering-loop gate verification`; do not spawn DA unless it passes.
|
|
45
45
|
5. Write artifacts, then spawn. Before Devil's Advocate:
|
|
46
46
|
- Write `git diff` to a file (for example `.ai-engineering-loop/tasks/current.diff`).
|
|
47
47
|
- Write changed paths (`git diff --name-only`) into the child prompt as a short list.
|
|
48
48
|
- Put those paths in the child prompt. Do not paste Maker rationale.
|
|
49
49
|
- Write `.ai-engineering-loop/tasks/claimed-vs-reality.md` if missing. Do not spawn DA without it.
|
|
50
50
|
- If stopping mid-loop, write `.ai-engineering-loop/tasks/handoff.md` (`core/handoff-policy.md`).
|
|
51
|
-
6. Stage 6: Build `npx ai-engineering-loop context devil-advocate <files
|
|
52
|
-
7. Stage 7: Run `npx ai-engineering-loop escalation --json`, resolve the required tier through the host catalog, or escalate to the user. Build Judge context from
|
|
51
|
+
6. Stage 6: Run `npx ai-engineering-loop context fast-path --json`; keep the default lean path when eligible. Build `npx ai-engineering-loop context diff-hunks --run <runId>`, then `context devil-advocate <files...>` only when hunks are insufficient. Use unresolved findings and new hunks instead of replaying accepted old findings. Check `budget status`, spawn DA with the pack path, wait, and `budget record` actual usage. Normalize its output into `RUN_DIR/findings.json` with runId and diffHash, then run `npx ai-engineering-loop gate review`.
|
|
52
|
+
7. Stage 7: Run `npx ai-engineering-loop escalation --json`, resolve the required tier through the host catalog, or escalate to the user. Build Judge context from only run-summary.md, verification-summary.json (or verification.json if the summary is missing), open-findings.json/findings.json, and the Goal hash; Judge must not read the full diff unless those summaries contradict each other. Check `budget status`, spawn Judge, wait, and `budget record` actual usage. Write `RUN_DIR/verdict.json` with modelTier, then run `npx ai-engineering-loop gate judge`.
|
|
53
53
|
8. ITERATE with iteration under 3: fix in the parent, re-verify, spawn a **new** Devil's Advocate.
|
|
54
54
|
9. After Judge PASS: run after_pass, then the configured Stage 8 adapter. PR/MR body includes `Steps to Reproduce & Testing (QA)`; bugfix placeholders must not stay empty. Preview remote payloads and never include context packs, usage ledgers, raw logs, or credentials. In ASSISTED, wait for explicit human approval and put `humanApproved: true` in `RUN_DIR/delivery.json`; never infer it. Then run `npx ai-engineering-loop gate delivery`.
|
|
55
55
|
|