@rely-ai/caliber 1.21.0 → 1.22.0-dev.1773744838
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +28 -13
- package/dist/bin.js +194 -67
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -32,26 +32,25 @@ No API key required — works with your existing **Claude Code** or **Cursor** s
|
|
|
32
32
|
| New team members start with no AI context | `caliber init` gives any contributor a complete setup in seconds |
|
|
33
33
|
| Configs diverge across AI tools | Cross-platform parity — Claude, Cursor, and Codex stay consistent |
|
|
34
34
|
| No idea if your config is actually helping | Score your setup (A–F grade) and see exactly what to improve |
|
|
35
|
+
| AI keeps making the same mistakes | Session learning captures patterns and corrections automatically |
|
|
35
36
|
|
|
36
37
|
## ⚙️ How It Works
|
|
37
38
|
|
|
38
39
|
```
|
|
39
40
|
caliber init
|
|
40
41
|
│
|
|
41
|
-
├─ 1. 🔌
|
|
42
|
+
├─ 1. 🔌 Setup Choose your LLM provider — Claude Code seat, Cursor seat,
|
|
42
43
|
│ or an API key (Anthropic, OpenAI, Vertex AI)
|
|
43
44
|
│
|
|
44
|
-
├─ 2.
|
|
45
|
-
│
|
|
45
|
+
├─ 2. 🛠️ Engine Fingerprint your project, generate configs in parallel,
|
|
46
|
+
│ search community skills, and auto-refine against
|
|
47
|
+
│ deterministic scoring checks (up to 2 iterations)
|
|
46
48
|
│
|
|
47
|
-
├─ 3.
|
|
48
|
-
│ (heavy model for docs, fast model for skills)
|
|
49
|
-
│
|
|
50
|
-
├─ 4. 👀 Review See a diff of every proposed change — accept, refine
|
|
49
|
+
├─ 3. 👀 Review See a diff of every proposed change — accept, refine
|
|
51
50
|
│ via chat, or decline. All originals are backed up
|
|
52
51
|
│
|
|
53
|
-
└─
|
|
54
|
-
|
|
52
|
+
└─ 4. ✅ Finalize Write files, install auto-refresh hooks, and set up
|
|
53
|
+
session learning for continuous improvement
|
|
55
54
|
```
|
|
56
55
|
|
|
57
56
|
Already have a setup? If your existing config scores **95+**, Caliber skips full regeneration and applies targeted fixes to the specific checks that are failing.
|
|
@@ -60,6 +59,7 @@ Already have a setup? If your existing config scores **95+**, Caliber skips full
|
|
|
60
59
|
|
|
61
60
|
**Claude Code**
|
|
62
61
|
- `CLAUDE.md` — Project context, build/test commands, architecture, conventions
|
|
62
|
+
- `CALIBER_LEARNINGS.md` — Patterns learned from your AI coding sessions
|
|
63
63
|
- `.claude/skills/*/SKILL.md` — Reusable skills ([OpenSkills](https://agentskills.io) format)
|
|
64
64
|
- `.mcp.json` — Auto-discovered MCP server configurations
|
|
65
65
|
- `.claude/settings.json` — Permissions and hooks
|
|
@@ -126,17 +126,31 @@ Every failing check includes structured fix data — when `caliber init` runs, t
|
|
|
126
126
|
|
|
127
127
|
</details>
|
|
128
128
|
|
|
129
|
+
### 🧠 Session Learning
|
|
130
|
+
Caliber watches your AI coding sessions and learns from them. Hooks capture tool usage, failures, and your corrections — then an LLM distills operational patterns into `CALIBER_LEARNINGS.md`.
|
|
131
|
+
|
|
132
|
+
```bash
|
|
133
|
+
caliber learn install # Install hooks for Claude Code and Cursor
|
|
134
|
+
caliber learn status # View hook status, event count, and ROI summary
|
|
135
|
+
caliber learn finalize # Manually trigger analysis (auto-runs on session end)
|
|
136
|
+
caliber learn remove # Remove hooks
|
|
137
|
+
```
|
|
138
|
+
|
|
139
|
+
Learned items are categorized by type — **[correction]**, **[gotcha]**, **[fix]**, **[pattern]**, **[env]**, **[convention]** — and automatically deduplicated. ROI tracking shows how much time and tokens the learnings save across sessions.
|
|
140
|
+
|
|
129
141
|
### 🔄 Auto-Refresh
|
|
130
142
|
Keep configs in sync with your codebase automatically:
|
|
131
143
|
|
|
132
144
|
| Hook | Trigger | What it does |
|
|
133
145
|
|---|---|---|
|
|
134
|
-
| **Claude Code** | End of each session | Runs `caliber refresh` and updates docs |
|
|
135
146
|
| **Git pre-commit** | Before each commit | Refreshes docs and stages updated files |
|
|
147
|
+
| **Claude Code session end** | End of each session | Runs `caliber refresh` and updates docs |
|
|
148
|
+
| **Learning hooks** | During each session | Captures events for session learning |
|
|
136
149
|
|
|
137
150
|
```bash
|
|
138
|
-
caliber hooks --install # Enable
|
|
139
|
-
caliber hooks --remove # Disable
|
|
151
|
+
caliber hooks --install # Enable refresh hooks
|
|
152
|
+
caliber hooks --remove # Disable refresh hooks
|
|
153
|
+
caliber learn install # Enable learning hooks
|
|
140
154
|
```
|
|
141
155
|
|
|
142
156
|
The `refresh` command analyzes your git diff (committed, staged, and unstaged changes) and updates config files to reflect what changed. Works across multiple repos when run from a parent directory.
|
|
@@ -152,11 +166,12 @@ Every change Caliber makes can be undone:
|
|
|
152
166
|
|
|
153
167
|
| Command | Description |
|
|
154
168
|
|---|---|
|
|
155
|
-
| `caliber init` | Full setup wizard — analyze, generate, review, install
|
|
169
|
+
| `caliber init` | Full setup wizard — analyze, generate, review, install hooks |
|
|
156
170
|
| `caliber score` | Score config quality (deterministic, no LLM) |
|
|
157
171
|
| `caliber regenerate` | Re-analyze and regenerate configs (aliases: `regen`, `re`) |
|
|
158
172
|
| `caliber refresh` | Update docs based on recent code changes |
|
|
159
173
|
| `caliber skills` | Discover and install community skills |
|
|
174
|
+
| `caliber learn` | Session learning — install hooks, view status, finalize analysis |
|
|
160
175
|
| `caliber hooks` | Manage auto-refresh hooks |
|
|
161
176
|
| `caliber config` | Configure LLM provider, API key, and model |
|
|
162
177
|
| `caliber status` | Show current setup status |
|
package/dist/bin.js
CHANGED
|
@@ -1856,6 +1856,12 @@ Safety: Never include API keys, tokens, or credentials in config files.
|
|
|
1856
1856
|
Note: Permissions, hooks, freshness tracking, and OpenSkills frontmatter are scored automatically by caliber \u2014 do not optimize for them.`;
|
|
1857
1857
|
var OUTPUT_SIZE_CONSTRAINTS = `OUTPUT SIZE CONSTRAINTS \u2014 these are critical:
|
|
1858
1858
|
- CLAUDE.md / AGENTS.md: MUST be under 150 lines for maximum score. Aim for 100-140 lines. Be concise \u2014 commands, architecture overview, and key conventions. Use bullet points and tables, not prose.
|
|
1859
|
+
|
|
1860
|
+
Pack project references densely in architecture sections \u2014 use inline paths, not prose paragraphs:
|
|
1861
|
+
GOOD: **Entry**: \`src/bin.ts\` \u2192 \`src/cli.ts\` \xB7 **LLM** (\`src/llm/\`): \`anthropic.ts\` \xB7 \`vertex.ts\` \xB7 \`openai-compat.ts\`
|
|
1862
|
+
BAD: The entry point of the application is located in the src directory. The LLM module handles different providers.
|
|
1863
|
+
For command sections, use code blocks with one command per line.
|
|
1864
|
+
|
|
1859
1865
|
- Each skill content: max 150 lines. Focus on patterns and examples, not exhaustive docs.
|
|
1860
1866
|
- Cursor rules: max 5 .mdc files.
|
|
1861
1867
|
- If the project is large, prioritize depth on the 3-4 most critical tools over breadth across everything.`;
|
|
@@ -4597,6 +4603,29 @@ function countTreeLines(content) {
|
|
|
4597
4603
|
}
|
|
4598
4604
|
return count;
|
|
4599
4605
|
}
|
|
4606
|
+
function calculateDuplicatePercent(content1, content2) {
|
|
4607
|
+
const lines1 = new Set(content1.split("\n").map((l) => l.trim()).filter((l) => l.length > 10));
|
|
4608
|
+
const lines2 = content2.split("\n").map((l) => l.trim()).filter((l) => l.length > 10);
|
|
4609
|
+
const overlapping = lines2.filter((l) => lines1.has(l)).length;
|
|
4610
|
+
return lines2.length > 0 ? Math.round(overlapping / lines2.length * 100) : 0;
|
|
4611
|
+
}
|
|
4612
|
+
function calculateDensityPoints(density, maxPoints) {
|
|
4613
|
+
if (density >= 40) return maxPoints;
|
|
4614
|
+
if (density >= 25) return Math.round(maxPoints * 0.75);
|
|
4615
|
+
if (density >= 15) return Math.round(maxPoints * 0.5);
|
|
4616
|
+
if (density >= 5) return Math.round(maxPoints * 0.25);
|
|
4617
|
+
return 0;
|
|
4618
|
+
}
|
|
4619
|
+
function isEntryMentioned(entry, contentLower) {
|
|
4620
|
+
const entryLower = entry.toLowerCase();
|
|
4621
|
+
const variants = [entryLower, entryLower.replace(/\\/g, "/")];
|
|
4622
|
+
const lastSegment = entry.split("/").pop()?.toLowerCase();
|
|
4623
|
+
if (lastSegment && lastSegment.length > 3) variants.push(lastSegment);
|
|
4624
|
+
return variants.some((v) => {
|
|
4625
|
+
const escaped = v.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
|
|
4626
|
+
return new RegExp(`(?:^|[\\s\`/"'\\.,(])${escaped}(?:[\\s\`/"'.,;:!?)\\\\]|$)`, "i").test(contentLower);
|
|
4627
|
+
});
|
|
4628
|
+
}
|
|
4600
4629
|
function classifyLine(line, inCodeBlock) {
|
|
4601
4630
|
if (inCodeBlock) return "concrete";
|
|
4602
4631
|
const trimmed = line.trim();
|
|
@@ -4706,15 +4735,7 @@ function checkQuality(dir) {
|
|
|
4706
4735
|
instruction: "Remove directory tree listings from code blocks. Reference key directories inline instead."
|
|
4707
4736
|
} : void 0
|
|
4708
4737
|
});
|
|
4709
|
-
|
|
4710
|
-
if (claudeMd && cursorrules) {
|
|
4711
|
-
const claudeLines = new Set(
|
|
4712
|
-
claudeMd.split("\n").map((l) => l.trim()).filter((l) => l.length > 10)
|
|
4713
|
-
);
|
|
4714
|
-
const cursorLines = cursorrules.split("\n").map((l) => l.trim()).filter((l) => l.length > 10);
|
|
4715
|
-
const overlapping = cursorLines.filter((l) => claudeLines.has(l)).length;
|
|
4716
|
-
duplicatePercent = cursorLines.length > 0 ? Math.round(overlapping / cursorLines.length * 100) : 0;
|
|
4717
|
-
}
|
|
4738
|
+
const duplicatePercent = claudeMd && cursorrules ? calculateDuplicatePercent(claudeMd, cursorrules) : 0;
|
|
4718
4739
|
const hasDuplicates = duplicatePercent > 50;
|
|
4719
4740
|
checks.push({
|
|
4720
4741
|
id: "no_duplicate_content",
|
|
@@ -4764,20 +4785,7 @@ function checkGrounding(dir) {
|
|
|
4764
4785
|
const mentioned = [];
|
|
4765
4786
|
const notMentioned = [];
|
|
4766
4787
|
for (const entry of meaningfulEntries) {
|
|
4767
|
-
|
|
4768
|
-
const variants = [
|
|
4769
|
-
entryLower,
|
|
4770
|
-
entryLower.replace(/\\/g, "/")
|
|
4771
|
-
];
|
|
4772
|
-
const lastSegment = entry.split("/").pop()?.toLowerCase();
|
|
4773
|
-
if (lastSegment && lastSegment.length > 3) {
|
|
4774
|
-
variants.push(lastSegment);
|
|
4775
|
-
}
|
|
4776
|
-
const ismentioned = variants.some((v) => {
|
|
4777
|
-
const escaped = v.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
|
|
4778
|
-
return new RegExp(`(?:^|[\\s\`/"'\\.,(])${escaped}(?:[\\s\`/"'.,;:!?)\\\\]|$)`, "i").test(configLower);
|
|
4779
|
-
});
|
|
4780
|
-
if (ismentioned) {
|
|
4788
|
+
if (isEntryMentioned(entry, configLower)) {
|
|
4781
4789
|
mentioned.push(entry);
|
|
4782
4790
|
} else {
|
|
4783
4791
|
notMentioned.push(entry);
|
|
@@ -4787,12 +4795,8 @@ function checkGrounding(dir) {
|
|
|
4787
4795
|
const groundingThreshold = GROUNDING_THRESHOLDS.find((t) => groundingRatio >= t.minRatio);
|
|
4788
4796
|
const groundingPoints = meaningfulEntries.length === 0 ? 0 : groundingThreshold?.points ?? 0;
|
|
4789
4797
|
const topDirs = projectStructure.dirs.filter((d) => !d.includes("/")).filter((d) => d.length > 2);
|
|
4790
|
-
const
|
|
4791
|
-
|
|
4792
|
-
return new RegExp(`(?:^|[\\s\`/"'\\.,(])${escaped}(?:[\\s\`/"'.,;:!?)\\\\]|$)`, "i").test(configLower);
|
|
4793
|
-
};
|
|
4794
|
-
const unmentionedTopDirs = topDirs.filter((d) => !matchesConfig(d));
|
|
4795
|
-
const mentionedTopDirs = topDirs.filter((d) => matchesConfig(d));
|
|
4798
|
+
const unmentionedTopDirs = topDirs.filter((d) => !isEntryMentioned(d, configLower));
|
|
4799
|
+
const mentionedTopDirs = topDirs.filter((d) => isEntryMentioned(d, configLower));
|
|
4796
4800
|
checks.push({
|
|
4797
4801
|
id: "project_grounding",
|
|
4798
4802
|
name: "Project grounding",
|
|
@@ -4817,18 +4821,7 @@ function checkGrounding(dir) {
|
|
|
4817
4821
|
const mdStructure = analyzeMarkdownStructure(configContent);
|
|
4818
4822
|
const totalSpecificRefs = refs.length + mdStructure.inlineCodeCount;
|
|
4819
4823
|
const density = mdStructure.nonEmptyLines > 0 ? totalSpecificRefs / mdStructure.nonEmptyLines * 100 : 0;
|
|
4820
|
-
|
|
4821
|
-
if (configContent.length === 0) {
|
|
4822
|
-
densityPoints = 0;
|
|
4823
|
-
} else if (density >= 40) {
|
|
4824
|
-
densityPoints = POINTS_REFERENCE_DENSITY;
|
|
4825
|
-
} else if (density >= 25) {
|
|
4826
|
-
densityPoints = Math.round(POINTS_REFERENCE_DENSITY * 0.75);
|
|
4827
|
-
} else if (density >= 15) {
|
|
4828
|
-
densityPoints = Math.round(POINTS_REFERENCE_DENSITY * 0.5);
|
|
4829
|
-
} else if (density >= 5) {
|
|
4830
|
-
densityPoints = Math.round(POINTS_REFERENCE_DENSITY * 0.25);
|
|
4831
|
-
}
|
|
4824
|
+
const densityPoints = configContent.length === 0 ? 0 : calculateDensityPoints(density, POINTS_REFERENCE_DENSITY);
|
|
4832
4825
|
checks.push({
|
|
4833
4826
|
id: "reference_density",
|
|
4834
4827
|
name: "Reference density",
|
|
@@ -6449,14 +6442,46 @@ var MAX_REFINE_ITERATIONS = 2;
|
|
|
6449
6442
|
function extractConfigContent(setup) {
|
|
6450
6443
|
const claude = setup.claude;
|
|
6451
6444
|
const codex = setup.codex;
|
|
6445
|
+
const cursor = setup.cursor;
|
|
6446
|
+
let cursorrules = null;
|
|
6447
|
+
if (typeof cursor?.cursorrules === "string" && cursor.cursorrules.length > 0) {
|
|
6448
|
+
cursorrules = cursor.cursorrules;
|
|
6449
|
+
}
|
|
6450
|
+
const skills = [];
|
|
6451
|
+
for (const [platform, obj] of [["claude", claude], ["codex", codex], ["cursor", cursor]]) {
|
|
6452
|
+
const platformSkills = obj?.skills;
|
|
6453
|
+
if (Array.isArray(platformSkills)) {
|
|
6454
|
+
for (const skill of platformSkills) {
|
|
6455
|
+
if (typeof skill.content === "string" && skill.content.length > 0) {
|
|
6456
|
+
skills.push({ name: skill.name, content: skill.content, platform });
|
|
6457
|
+
}
|
|
6458
|
+
}
|
|
6459
|
+
}
|
|
6460
|
+
}
|
|
6452
6461
|
return {
|
|
6453
6462
|
claudeMd: claude?.claudeMd ?? null,
|
|
6454
|
-
agentsMd: codex?.agentsMd ?? null
|
|
6463
|
+
agentsMd: codex?.agentsMd ?? null,
|
|
6464
|
+
cursorrules,
|
|
6465
|
+
skills
|
|
6455
6466
|
};
|
|
6456
6467
|
}
|
|
6457
|
-
function
|
|
6468
|
+
function buildGroundingFixInstruction(unmentionedTopDirs, projectStructure) {
|
|
6469
|
+
const dirDescriptions = unmentionedTopDirs.slice(0, 8).map((dir) => {
|
|
6470
|
+
const subdirs = projectStructure.dirs.filter((d) => d.startsWith(`${dir}/`) && !d.includes("/", dir.length + 1));
|
|
6471
|
+
const files = projectStructure.files.filter((f) => f.startsWith(`${dir}/`) && !f.includes("/", dir.length + 1));
|
|
6472
|
+
const children = [...subdirs.slice(0, 4), ...files.slice(0, 2)];
|
|
6473
|
+
const childList = children.map((c) => c.split("/").pop()).join(", ");
|
|
6474
|
+
return childList ? `- \`${dir}/\` (contains: ${childList})` : `- \`${dir}/\``;
|
|
6475
|
+
});
|
|
6476
|
+
return [
|
|
6477
|
+
"Reference these project directories with descriptions of their contents:",
|
|
6478
|
+
...dirDescriptions,
|
|
6479
|
+
"Mention them naturally in architecture descriptions using dense inline references."
|
|
6480
|
+
].join("\n");
|
|
6481
|
+
}
|
|
6482
|
+
function validateSetup(setup, dir, checkExists = existsSync9, projectStructure) {
|
|
6458
6483
|
const issues = [];
|
|
6459
|
-
const { claudeMd, agentsMd } = extractConfigContent(setup);
|
|
6484
|
+
const { claudeMd, agentsMd, cursorrules, skills } = extractConfigContent(setup);
|
|
6460
6485
|
const primaryContent = [claudeMd, agentsMd].filter(Boolean).join("\n");
|
|
6461
6486
|
if (!primaryContent) return issues;
|
|
6462
6487
|
const refs = validateFileReferences(primaryContent, dir, checkExists);
|
|
@@ -6487,15 +6512,15 @@ function validateSetup(setup, dir, checkExists = existsSync9) {
|
|
|
6487
6512
|
}
|
|
6488
6513
|
const content = claudeMd ?? agentsMd ?? "";
|
|
6489
6514
|
if (content) {
|
|
6490
|
-
const
|
|
6491
|
-
const blockThreshold = CODE_BLOCK_THRESHOLDS.find((t) =>
|
|
6515
|
+
const structure2 = analyzeMarkdownStructure(content);
|
|
6516
|
+
const blockThreshold = CODE_BLOCK_THRESHOLDS.find((t) => structure2.codeBlockCount >= t.minBlocks);
|
|
6492
6517
|
const blockPoints = blockThreshold?.points ?? 0;
|
|
6493
6518
|
const maxBlockPoints = CODE_BLOCK_THRESHOLDS[0].points;
|
|
6494
|
-
if (blockPoints < maxBlockPoints &&
|
|
6519
|
+
if (blockPoints < maxBlockPoints && structure2.codeBlockCount < CODE_BLOCK_THRESHOLDS[0].minBlocks) {
|
|
6495
6520
|
issues.push({
|
|
6496
6521
|
check: "Executable content",
|
|
6497
|
-
detail: `${
|
|
6498
|
-
fixInstruction: `Add ${CODE_BLOCK_THRESHOLDS[0].minBlocks -
|
|
6522
|
+
detail: `${structure2.codeBlockCount} code block${structure2.codeBlockCount === 1 ? "" : "s"} (need \u2265${CODE_BLOCK_THRESHOLDS[0].minBlocks} for full points)`,
|
|
6523
|
+
fixInstruction: `Add ${CODE_BLOCK_THRESHOLDS[0].minBlocks - structure2.codeBlockCount} more code blocks with actual project commands (build, test, lint, deploy).`,
|
|
6499
6524
|
pointsLost: maxBlockPoints - blockPoints
|
|
6500
6525
|
});
|
|
6501
6526
|
}
|
|
@@ -6522,15 +6547,86 @@ function validateSetup(setup, dir, checkExists = existsSync9) {
|
|
|
6522
6547
|
pointsLost: POINTS_NO_DIR_TREE
|
|
6523
6548
|
});
|
|
6524
6549
|
}
|
|
6525
|
-
if (
|
|
6550
|
+
if (structure2.h2Count < 3 || structure2.listItemCount < 3) {
|
|
6526
6551
|
const parts = [];
|
|
6527
|
-
if (
|
|
6528
|
-
if (
|
|
6552
|
+
if (structure2.h2Count < 3) parts.push(`add ${3 - structure2.h2Count} more ## sections`);
|
|
6553
|
+
if (structure2.listItemCount < 3) parts.push("use bullet lists for multi-item instructions");
|
|
6529
6554
|
issues.push({
|
|
6530
6555
|
check: "Structured with headings",
|
|
6531
|
-
detail: `${
|
|
6556
|
+
detail: `${structure2.h2Count} sections, ${structure2.listItemCount} list items`,
|
|
6532
6557
|
fixInstruction: `Improve structure: ${parts.join(" and ")}.`,
|
|
6533
|
-
pointsLost: POINTS_HAS_STRUCTURE - ((
|
|
6558
|
+
pointsLost: POINTS_HAS_STRUCTURE - ((structure2.h2Count >= 3 ? 1 : 0) + (structure2.listItemCount >= 3 ? 1 : 0))
|
|
6559
|
+
});
|
|
6560
|
+
}
|
|
6561
|
+
}
|
|
6562
|
+
const structure = projectStructure ?? collectProjectStructure(dir);
|
|
6563
|
+
const allEntries = [...structure.dirs, ...structure.files].filter((e) => e.length > 2);
|
|
6564
|
+
if (allEntries.length > 0) {
|
|
6565
|
+
const contentLower = primaryContent.toLowerCase();
|
|
6566
|
+
const mentionedEntries = allEntries.filter((e) => isEntryMentioned(e, contentLower));
|
|
6567
|
+
const groundingRatio = mentionedEntries.length / allEntries.length;
|
|
6568
|
+
const groundingThreshold = GROUNDING_THRESHOLDS.find((t) => groundingRatio >= t.minRatio);
|
|
6569
|
+
const groundingPoints = groundingThreshold?.points ?? 0;
|
|
6570
|
+
const groundingLost = POINTS_PROJECT_GROUNDING - groundingPoints;
|
|
6571
|
+
if (groundingLost > 0) {
|
|
6572
|
+
const topDirs = structure.dirs.filter((d) => !d.includes("/") && d.length > 2);
|
|
6573
|
+
const unmentionedTopDirs = topDirs.filter((d) => !isEntryMentioned(d, contentLower));
|
|
6574
|
+
if (unmentionedTopDirs.length > 0) {
|
|
6575
|
+
issues.push({
|
|
6576
|
+
check: "Project grounding",
|
|
6577
|
+
detail: `${mentionedEntries.length}/${allEntries.length} project entries referenced (${Math.round(groundingRatio * 100)}%)`,
|
|
6578
|
+
fixInstruction: buildGroundingFixInstruction(unmentionedTopDirs, structure),
|
|
6579
|
+
pointsLost: groundingLost
|
|
6580
|
+
});
|
|
6581
|
+
}
|
|
6582
|
+
}
|
|
6583
|
+
}
|
|
6584
|
+
const allRefs = extractReferences(primaryContent);
|
|
6585
|
+
const primaryStructure = analyzeMarkdownStructure(primaryContent);
|
|
6586
|
+
const totalSpecificRefs = allRefs.length + primaryStructure.inlineCodeCount;
|
|
6587
|
+
const density = primaryStructure.nonEmptyLines > 0 ? totalSpecificRefs / primaryStructure.nonEmptyLines * 100 : 0;
|
|
6588
|
+
const densityPoints = calculateDensityPoints(density, POINTS_REFERENCE_DENSITY);
|
|
6589
|
+
const densityLost = POINTS_REFERENCE_DENSITY - densityPoints;
|
|
6590
|
+
if (densityLost > 0) {
|
|
6591
|
+
issues.push({
|
|
6592
|
+
check: "Reference density",
|
|
6593
|
+
detail: `${totalSpecificRefs} references across ${primaryStructure.nonEmptyLines} lines (${Math.round(density)}% density, need \u226540% for full points)`,
|
|
6594
|
+
fixInstruction: `Add more backtick references around file paths, commands, and identifiers. Use the dense reference style: \`src/api/\` routes \xB7 \`src/models/\` data. Current density: ${Math.round(density)}%, target: \u226540%.`,
|
|
6595
|
+
pointsLost: densityLost
|
|
6596
|
+
});
|
|
6597
|
+
}
|
|
6598
|
+
if (claudeMd && cursorrules) {
|
|
6599
|
+
const duplicatePercent = calculateDuplicatePercent(claudeMd, cursorrules);
|
|
6600
|
+
if (duplicatePercent > 50) {
|
|
6601
|
+
issues.push({
|
|
6602
|
+
check: "No duplicate content",
|
|
6603
|
+
detail: `${duplicatePercent}% overlap between CLAUDE.md and .cursorrules`,
|
|
6604
|
+
fixInstruction: "Deduplicate content. Keep shared instructions in CLAUDE.md only. Make .cursorrules contain only Cursor-specific settings and platform differences.",
|
|
6605
|
+
pointsLost: POINTS_NO_DUPLICATES
|
|
6606
|
+
});
|
|
6607
|
+
}
|
|
6608
|
+
}
|
|
6609
|
+
for (const skill of skills) {
|
|
6610
|
+
const skillIssues = [];
|
|
6611
|
+
const skillRefs = validateFileReferences(skill.content, dir, checkExists);
|
|
6612
|
+
if (skillRefs.invalid.length > 0) {
|
|
6613
|
+
skillIssues.push(`invalid refs: ${skillRefs.invalid.slice(0, 3).join(", ")}`);
|
|
6614
|
+
}
|
|
6615
|
+
const skillStructure = analyzeMarkdownStructure(skill.content);
|
|
6616
|
+
if (skillStructure.codeBlockCount === 0 && skill.content.length > 200) {
|
|
6617
|
+
skillIssues.push("no code blocks");
|
|
6618
|
+
}
|
|
6619
|
+
const { concrete, abstract } = countConcreteness(skill.content);
|
|
6620
|
+
const total = concrete + abstract;
|
|
6621
|
+
if (total > 3 && concrete / total < 0.3) {
|
|
6622
|
+
skillIssues.push("low concreteness");
|
|
6623
|
+
}
|
|
6624
|
+
if (skillIssues.length > 0) {
|
|
6625
|
+
issues.push({
|
|
6626
|
+
check: `Skill quality: ${skill.name}`,
|
|
6627
|
+
detail: skillIssues.join("; "),
|
|
6628
|
+
fixInstruction: `Fix skill "${skill.name}": ${skillIssues.join(", ")}.${skillRefs.invalid.length > 0 ? ` Remove invalid paths: ${skillRefs.invalid.join(", ")}.` : ""} Add code blocks and specific file references.`,
|
|
6629
|
+
pointsLost: 0
|
|
6534
6630
|
});
|
|
6535
6631
|
}
|
|
6536
6632
|
}
|
|
@@ -6560,23 +6656,25 @@ async function scoreAndRefine(setup, dir, sessionHistory, callbacks) {
|
|
|
6560
6656
|
existsCache.set(path29, result);
|
|
6561
6657
|
return result;
|
|
6562
6658
|
};
|
|
6659
|
+
const projectStructure = collectProjectStructure(dir);
|
|
6563
6660
|
let currentSetup = setup;
|
|
6564
6661
|
let bestSetup = setup;
|
|
6565
6662
|
let bestLostPoints = Infinity;
|
|
6566
6663
|
for (let iteration = 0; iteration < MAX_REFINE_ITERATIONS; iteration++) {
|
|
6567
|
-
const issues = validateSetup(currentSetup, dir, cachedExists);
|
|
6664
|
+
const issues = validateSetup(currentSetup, dir, cachedExists, projectStructure);
|
|
6568
6665
|
const lostPoints = countIssuePoints(issues);
|
|
6569
6666
|
if (lostPoints < bestLostPoints) {
|
|
6570
6667
|
bestSetup = currentSetup;
|
|
6571
6668
|
bestLostPoints = lostPoints;
|
|
6572
6669
|
}
|
|
6573
|
-
if (
|
|
6670
|
+
if (lostPoints === 0) {
|
|
6574
6671
|
if (callbacks?.onStatus) callbacks.onStatus("Setup passes all scoring checks");
|
|
6575
6672
|
return bestSetup;
|
|
6576
6673
|
}
|
|
6674
|
+
const pointIssues = issues.filter((i) => i.pointsLost > 0);
|
|
6675
|
+
const pointIssueNames = pointIssues.map((i) => i.check).join(", ");
|
|
6577
6676
|
if (callbacks?.onStatus) {
|
|
6578
|
-
|
|
6579
|
-
callbacks.onStatus(`Fixing ${issues.length} scoring issue${issues.length === 1 ? "" : "s"}: ${issueNames}...`);
|
|
6677
|
+
callbacks.onStatus(`Fixing ${pointIssues.length} scoring issue${pointIssues.length === 1 ? "" : "s"}: ${pointIssueNames}...`);
|
|
6580
6678
|
}
|
|
6581
6679
|
const refined = await applyTargetedFixes(currentSetup, issues);
|
|
6582
6680
|
if (!refined) {
|
|
@@ -6585,15 +6683,15 @@ async function scoreAndRefine(setup, dir, sessionHistory, callbacks) {
|
|
|
6585
6683
|
}
|
|
6586
6684
|
sessionHistory.push({
|
|
6587
6685
|
role: "user",
|
|
6588
|
-
content: `Fix scoring issues: ${
|
|
6686
|
+
content: `Fix scoring issues: ${pointIssueNames}`
|
|
6589
6687
|
});
|
|
6590
6688
|
sessionHistory.push({
|
|
6591
6689
|
role: "assistant",
|
|
6592
|
-
content: `Applied scoring fixes for: ${
|
|
6690
|
+
content: `Applied scoring fixes for: ${pointIssueNames}`
|
|
6593
6691
|
});
|
|
6594
6692
|
currentSetup = refined;
|
|
6595
6693
|
}
|
|
6596
|
-
const finalIssues = validateSetup(currentSetup, dir, cachedExists);
|
|
6694
|
+
const finalIssues = validateSetup(currentSetup, dir, cachedExists, projectStructure);
|
|
6597
6695
|
const finalLostPoints = countIssuePoints(finalIssues);
|
|
6598
6696
|
if (finalLostPoints < bestLostPoints) {
|
|
6599
6697
|
bestSetup = currentSetup;
|
|
@@ -6601,10 +6699,19 @@ async function scoreAndRefine(setup, dir, sessionHistory, callbacks) {
|
|
|
6601
6699
|
return bestSetup;
|
|
6602
6700
|
}
|
|
6603
6701
|
async function applyTargetedFixes(setup, issues) {
|
|
6604
|
-
const { claudeMd, agentsMd } = extractConfigContent(setup);
|
|
6702
|
+
const { claudeMd, agentsMd, cursorrules, skills } = extractConfigContent(setup);
|
|
6605
6703
|
const targets = [];
|
|
6606
6704
|
if (claudeMd) targets.push({ key: "claudeMd", label: "CLAUDE.md", content: claudeMd });
|
|
6607
6705
|
if (agentsMd) targets.push({ key: "agentsMd", label: "AGENTS.md", content: agentsMd });
|
|
6706
|
+
const failingChecks = new Set(issues.map((i) => i.check));
|
|
6707
|
+
if (cursorrules && failingChecks.has("No duplicate content")) {
|
|
6708
|
+
targets.push({ key: "cursorrules", label: ".cursorrules", content: cursorrules });
|
|
6709
|
+
}
|
|
6710
|
+
for (const skill of skills) {
|
|
6711
|
+
if (failingChecks.has(`Skill quality: ${skill.name}`)) {
|
|
6712
|
+
targets.push({ key: `skill:${skill.name}`, label: `Skill: ${skill.name}`, content: skill.content });
|
|
6713
|
+
}
|
|
6714
|
+
}
|
|
6608
6715
|
if (targets.length === 0) return null;
|
|
6609
6716
|
const feedbackMessage = buildFeedbackMessage(issues);
|
|
6610
6717
|
const contentBlock = targets.map(
|
|
@@ -6621,11 +6728,12 @@ ${t.content}
|
|
|
6621
6728
|
`
|
|
6622
6729
|
Return ONLY the fixed content as a JSON object with keys ${targets.map((t) => `"${t.key}"`).join(", ")}. Each value is the fixed markdown string. No code fences, no explanations.`
|
|
6623
6730
|
].join("\n");
|
|
6731
|
+
const maxTokens = Math.min(12e3, 4e3 + targets.length * 2e3);
|
|
6624
6732
|
try {
|
|
6625
6733
|
const raw = await llmCall({
|
|
6626
|
-
system: "You fix scoring issues in AI agent configuration files. Return only a JSON object with the fixed content \u2014 no explanations, no code fences.",
|
|
6734
|
+
system: "You fix scoring issues in AI agent configuration files. You may receive CLAUDE.md, AGENTS.md, .cursorrules, and/or skill files. Return only a JSON object with the fixed content \u2014 no explanations, no code fences.",
|
|
6627
6735
|
prompt,
|
|
6628
|
-
maxTokens
|
|
6736
|
+
maxTokens
|
|
6629
6737
|
});
|
|
6630
6738
|
const cleaned = stripMarkdownFences(raw);
|
|
6631
6739
|
const jsonStart = cleaned.indexOf("{");
|
|
@@ -6633,9 +6741,28 @@ Return ONLY the fixed content as a JSON object with keys ${targets.map((t) => `"
|
|
|
6633
6741
|
const fixes = JSON.parse(jsonToParse);
|
|
6634
6742
|
const patched = structuredClone(setup);
|
|
6635
6743
|
for (const target of targets) {
|
|
6636
|
-
|
|
6637
|
-
|
|
6638
|
-
|
|
6744
|
+
const fixedValue = fixes[target.key];
|
|
6745
|
+
if (typeof fixedValue !== "string" || fixedValue.length < 50) continue;
|
|
6746
|
+
if (target.key === "claudeMd") {
|
|
6747
|
+
const parent = patched.claude;
|
|
6748
|
+
if (parent) parent.claudeMd = fixedValue;
|
|
6749
|
+
} else if (target.key === "agentsMd") {
|
|
6750
|
+
const parent = patched.codex;
|
|
6751
|
+
if (parent) parent.agentsMd = fixedValue;
|
|
6752
|
+
} else if (target.key === "cursorrules") {
|
|
6753
|
+
const parent = patched.cursor;
|
|
6754
|
+
if (parent) parent.cursorrules = fixedValue;
|
|
6755
|
+
} else if (target.key.startsWith("skill:")) {
|
|
6756
|
+
const skillName = target.key.slice(6);
|
|
6757
|
+
for (const platform of ["claude", "cursor", "codex"]) {
|
|
6758
|
+
const platformObj = patched[platform];
|
|
6759
|
+
const platformSkills = platformObj?.skills;
|
|
6760
|
+
const skill = platformSkills?.find((s) => s.name === skillName);
|
|
6761
|
+
if (skill) {
|
|
6762
|
+
skill.content = fixedValue;
|
|
6763
|
+
break;
|
|
6764
|
+
}
|
|
6765
|
+
}
|
|
6639
6766
|
}
|
|
6640
6767
|
}
|
|
6641
6768
|
return patched;
|
package/package.json
CHANGED