@rely-ai/caliber 1.21.0 → 1.22.0-dev.1773744838

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (3) hide show
  1. package/README.md +28 -13
  2. package/dist/bin.js +194 -67
  3. package/package.json +1 -1
package/README.md CHANGED
@@ -32,26 +32,25 @@ No API key required — works with your existing **Claude Code** or **Cursor** s
32
32
  | New team members start with no AI context | `caliber init` gives any contributor a complete setup in seconds |
33
33
  | Configs diverge across AI tools | Cross-platform parity — Claude, Cursor, and Codex stay consistent |
34
34
  | No idea if your config is actually helping | Score your setup (A–F grade) and see exactly what to improve |
35
+ | AI keeps making the same mistakes | Session learning captures patterns and corrections automatically |
35
36
 
36
37
  ## ⚙️ How It Works
37
38
 
38
39
  ```
39
40
  caliber init
40
41
 
41
- ├─ 1. 🔌 Connect Choose your LLM provider — Claude Code seat, Cursor seat,
42
+ ├─ 1. 🔌 Setup Choose your LLM provider — Claude Code seat, Cursor seat,
42
43
  │ or an API key (Anthropic, OpenAI, Vertex AI)
43
44
 
44
- ├─ 2. 🔍 Discover Fingerprint your project: languages, frameworks, dependencies,
45
- file structure, and existing agent configs
45
+ ├─ 2. 🛠️ Engine Fingerprint your project, generate configs in parallel,
46
+ search community skills, and auto-refine against
47
+ │ deterministic scoring checks (up to 2 iterations)
46
48
 
47
- ├─ 3. 🛠️ Generate Build optimized configs with parallel LLM calls
48
- │ (heavy model for docs, fast model for skills)
49
-
50
- ├─ 4. 👀 Review See a diff of every proposed change — accept, refine
49
+ ├─ 3. 👀 Review See a diff of every proposed change — accept, refine
51
50
  │ via chat, or decline. All originals are backed up
52
51
 
53
- └─ 5. 🧩 Skills Search community registries and install relevant
54
- skills for your tech stack
52
+ └─ 4. Finalize Write files, install auto-refresh hooks, and set up
53
+ session learning for continuous improvement
55
54
  ```
56
55
 
57
56
  Already have a setup? If your existing config scores **95+**, Caliber skips full regeneration and applies targeted fixes to the specific checks that are failing.
@@ -60,6 +59,7 @@ Already have a setup? If your existing config scores **95+**, Caliber skips full
60
59
 
61
60
  **Claude Code**
62
61
  - `CLAUDE.md` — Project context, build/test commands, architecture, conventions
62
+ - `CALIBER_LEARNINGS.md` — Patterns learned from your AI coding sessions
63
63
  - `.claude/skills/*/SKILL.md` — Reusable skills ([OpenSkills](https://agentskills.io) format)
64
64
  - `.mcp.json` — Auto-discovered MCP server configurations
65
65
  - `.claude/settings.json` — Permissions and hooks
@@ -126,17 +126,31 @@ Every failing check includes structured fix data — when `caliber init` runs, t
126
126
 
127
127
  </details>
128
128
 
129
+ ### 🧠 Session Learning
130
+ Caliber watches your AI coding sessions and learns from them. Hooks capture tool usage, failures, and your corrections — then an LLM distills operational patterns into `CALIBER_LEARNINGS.md`.
131
+
132
+ ```bash
133
+ caliber learn install # Install hooks for Claude Code and Cursor
134
+ caliber learn status # View hook status, event count, and ROI summary
135
+ caliber learn finalize # Manually trigger analysis (auto-runs on session end)
136
+ caliber learn remove # Remove hooks
137
+ ```
138
+
139
+ Learned items are categorized by type — **[correction]**, **[gotcha]**, **[fix]**, **[pattern]**, **[env]**, **[convention]** — and automatically deduplicated. ROI tracking shows how much time and tokens the learnings save across sessions.
140
+
129
141
  ### 🔄 Auto-Refresh
130
142
  Keep configs in sync with your codebase automatically:
131
143
 
132
144
  | Hook | Trigger | What it does |
133
145
  |---|---|---|
134
- | **Claude Code** | End of each session | Runs `caliber refresh` and updates docs |
135
146
  | **Git pre-commit** | Before each commit | Refreshes docs and stages updated files |
147
+ | **Claude Code session end** | End of each session | Runs `caliber refresh` and updates docs |
148
+ | **Learning hooks** | During each session | Captures events for session learning |
136
149
 
137
150
  ```bash
138
- caliber hooks --install # Enable all hooks
139
- caliber hooks --remove # Disable all hooks
151
+ caliber hooks --install # Enable refresh hooks
152
+ caliber hooks --remove # Disable refresh hooks
153
+ caliber learn install # Enable learning hooks
140
154
  ```
141
155
 
142
156
  The `refresh` command analyzes your git diff (committed, staged, and unstaged changes) and updates config files to reflect what changed. Works across multiple repos when run from a parent directory.
@@ -152,11 +166,12 @@ Every change Caliber makes can be undone:
152
166
 
153
167
  | Command | Description |
154
168
  |---|---|
155
- | `caliber init` | Full setup wizard — analyze, generate, review, install skills |
169
+ | `caliber init` | Full setup wizard — analyze, generate, review, install hooks |
156
170
  | `caliber score` | Score config quality (deterministic, no LLM) |
157
171
  | `caliber regenerate` | Re-analyze and regenerate configs (aliases: `regen`, `re`) |
158
172
  | `caliber refresh` | Update docs based on recent code changes |
159
173
  | `caliber skills` | Discover and install community skills |
174
+ | `caliber learn` | Session learning — install hooks, view status, finalize analysis |
160
175
  | `caliber hooks` | Manage auto-refresh hooks |
161
176
  | `caliber config` | Configure LLM provider, API key, and model |
162
177
  | `caliber status` | Show current setup status |
package/dist/bin.js CHANGED
@@ -1856,6 +1856,12 @@ Safety: Never include API keys, tokens, or credentials in config files.
1856
1856
  Note: Permissions, hooks, freshness tracking, and OpenSkills frontmatter are scored automatically by caliber \u2014 do not optimize for them.`;
1857
1857
  var OUTPUT_SIZE_CONSTRAINTS = `OUTPUT SIZE CONSTRAINTS \u2014 these are critical:
1858
1858
  - CLAUDE.md / AGENTS.md: MUST be under 150 lines for maximum score. Aim for 100-140 lines. Be concise \u2014 commands, architecture overview, and key conventions. Use bullet points and tables, not prose.
1859
+
1860
+ Pack project references densely in architecture sections \u2014 use inline paths, not prose paragraphs:
1861
+ GOOD: **Entry**: \`src/bin.ts\` \u2192 \`src/cli.ts\` \xB7 **LLM** (\`src/llm/\`): \`anthropic.ts\` \xB7 \`vertex.ts\` \xB7 \`openai-compat.ts\`
1862
+ BAD: The entry point of the application is located in the src directory. The LLM module handles different providers.
1863
+ For command sections, use code blocks with one command per line.
1864
+
1859
1865
  - Each skill content: max 150 lines. Focus on patterns and examples, not exhaustive docs.
1860
1866
  - Cursor rules: max 5 .mdc files.
1861
1867
  - If the project is large, prioritize depth on the 3-4 most critical tools over breadth across everything.`;
@@ -4597,6 +4603,29 @@ function countTreeLines(content) {
4597
4603
  }
4598
4604
  return count;
4599
4605
  }
4606
+ function calculateDuplicatePercent(content1, content2) {
4607
+ const lines1 = new Set(content1.split("\n").map((l) => l.trim()).filter((l) => l.length > 10));
4608
+ const lines2 = content2.split("\n").map((l) => l.trim()).filter((l) => l.length > 10);
4609
+ const overlapping = lines2.filter((l) => lines1.has(l)).length;
4610
+ return lines2.length > 0 ? Math.round(overlapping / lines2.length * 100) : 0;
4611
+ }
4612
+ function calculateDensityPoints(density, maxPoints) {
4613
+ if (density >= 40) return maxPoints;
4614
+ if (density >= 25) return Math.round(maxPoints * 0.75);
4615
+ if (density >= 15) return Math.round(maxPoints * 0.5);
4616
+ if (density >= 5) return Math.round(maxPoints * 0.25);
4617
+ return 0;
4618
+ }
4619
+ function isEntryMentioned(entry, contentLower) {
4620
+ const entryLower = entry.toLowerCase();
4621
+ const variants = [entryLower, entryLower.replace(/\\/g, "/")];
4622
+ const lastSegment = entry.split("/").pop()?.toLowerCase();
4623
+ if (lastSegment && lastSegment.length > 3) variants.push(lastSegment);
4624
+ return variants.some((v) => {
4625
+ const escaped = v.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
4626
+ return new RegExp(`(?:^|[\\s\`/"'\\.,(])${escaped}(?:[\\s\`/"'.,;:!?)\\\\]|$)`, "i").test(contentLower);
4627
+ });
4628
+ }
4600
4629
  function classifyLine(line, inCodeBlock) {
4601
4630
  if (inCodeBlock) return "concrete";
4602
4631
  const trimmed = line.trim();
@@ -4706,15 +4735,7 @@ function checkQuality(dir) {
4706
4735
  instruction: "Remove directory tree listings from code blocks. Reference key directories inline instead."
4707
4736
  } : void 0
4708
4737
  });
4709
- let duplicatePercent = 0;
4710
- if (claudeMd && cursorrules) {
4711
- const claudeLines = new Set(
4712
- claudeMd.split("\n").map((l) => l.trim()).filter((l) => l.length > 10)
4713
- );
4714
- const cursorLines = cursorrules.split("\n").map((l) => l.trim()).filter((l) => l.length > 10);
4715
- const overlapping = cursorLines.filter((l) => claudeLines.has(l)).length;
4716
- duplicatePercent = cursorLines.length > 0 ? Math.round(overlapping / cursorLines.length * 100) : 0;
4717
- }
4738
+ const duplicatePercent = claudeMd && cursorrules ? calculateDuplicatePercent(claudeMd, cursorrules) : 0;
4718
4739
  const hasDuplicates = duplicatePercent > 50;
4719
4740
  checks.push({
4720
4741
  id: "no_duplicate_content",
@@ -4764,20 +4785,7 @@ function checkGrounding(dir) {
4764
4785
  const mentioned = [];
4765
4786
  const notMentioned = [];
4766
4787
  for (const entry of meaningfulEntries) {
4767
- const entryLower = entry.toLowerCase();
4768
- const variants = [
4769
- entryLower,
4770
- entryLower.replace(/\\/g, "/")
4771
- ];
4772
- const lastSegment = entry.split("/").pop()?.toLowerCase();
4773
- if (lastSegment && lastSegment.length > 3) {
4774
- variants.push(lastSegment);
4775
- }
4776
- const ismentioned = variants.some((v) => {
4777
- const escaped = v.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
4778
- return new RegExp(`(?:^|[\\s\`/"'\\.,(])${escaped}(?:[\\s\`/"'.,;:!?)\\\\]|$)`, "i").test(configLower);
4779
- });
4780
- if (ismentioned) {
4788
+ if (isEntryMentioned(entry, configLower)) {
4781
4789
  mentioned.push(entry);
4782
4790
  } else {
4783
4791
  notMentioned.push(entry);
@@ -4787,12 +4795,8 @@ function checkGrounding(dir) {
4787
4795
  const groundingThreshold = GROUNDING_THRESHOLDS.find((t) => groundingRatio >= t.minRatio);
4788
4796
  const groundingPoints = meaningfulEntries.length === 0 ? 0 : groundingThreshold?.points ?? 0;
4789
4797
  const topDirs = projectStructure.dirs.filter((d) => !d.includes("/")).filter((d) => d.length > 2);
4790
- const matchesConfig = (name) => {
4791
- const escaped = name.toLowerCase().replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
4792
- return new RegExp(`(?:^|[\\s\`/"'\\.,(])${escaped}(?:[\\s\`/"'.,;:!?)\\\\]|$)`, "i").test(configLower);
4793
- };
4794
- const unmentionedTopDirs = topDirs.filter((d) => !matchesConfig(d));
4795
- const mentionedTopDirs = topDirs.filter((d) => matchesConfig(d));
4798
+ const unmentionedTopDirs = topDirs.filter((d) => !isEntryMentioned(d, configLower));
4799
+ const mentionedTopDirs = topDirs.filter((d) => isEntryMentioned(d, configLower));
4796
4800
  checks.push({
4797
4801
  id: "project_grounding",
4798
4802
  name: "Project grounding",
@@ -4817,18 +4821,7 @@ function checkGrounding(dir) {
4817
4821
  const mdStructure = analyzeMarkdownStructure(configContent);
4818
4822
  const totalSpecificRefs = refs.length + mdStructure.inlineCodeCount;
4819
4823
  const density = mdStructure.nonEmptyLines > 0 ? totalSpecificRefs / mdStructure.nonEmptyLines * 100 : 0;
4820
- let densityPoints = 0;
4821
- if (configContent.length === 0) {
4822
- densityPoints = 0;
4823
- } else if (density >= 40) {
4824
- densityPoints = POINTS_REFERENCE_DENSITY;
4825
- } else if (density >= 25) {
4826
- densityPoints = Math.round(POINTS_REFERENCE_DENSITY * 0.75);
4827
- } else if (density >= 15) {
4828
- densityPoints = Math.round(POINTS_REFERENCE_DENSITY * 0.5);
4829
- } else if (density >= 5) {
4830
- densityPoints = Math.round(POINTS_REFERENCE_DENSITY * 0.25);
4831
- }
4824
+ const densityPoints = configContent.length === 0 ? 0 : calculateDensityPoints(density, POINTS_REFERENCE_DENSITY);
4832
4825
  checks.push({
4833
4826
  id: "reference_density",
4834
4827
  name: "Reference density",
@@ -6449,14 +6442,46 @@ var MAX_REFINE_ITERATIONS = 2;
6449
6442
  function extractConfigContent(setup) {
6450
6443
  const claude = setup.claude;
6451
6444
  const codex = setup.codex;
6445
+ const cursor = setup.cursor;
6446
+ let cursorrules = null;
6447
+ if (typeof cursor?.cursorrules === "string" && cursor.cursorrules.length > 0) {
6448
+ cursorrules = cursor.cursorrules;
6449
+ }
6450
+ const skills = [];
6451
+ for (const [platform, obj] of [["claude", claude], ["codex", codex], ["cursor", cursor]]) {
6452
+ const platformSkills = obj?.skills;
6453
+ if (Array.isArray(platformSkills)) {
6454
+ for (const skill of platformSkills) {
6455
+ if (typeof skill.content === "string" && skill.content.length > 0) {
6456
+ skills.push({ name: skill.name, content: skill.content, platform });
6457
+ }
6458
+ }
6459
+ }
6460
+ }
6452
6461
  return {
6453
6462
  claudeMd: claude?.claudeMd ?? null,
6454
- agentsMd: codex?.agentsMd ?? null
6463
+ agentsMd: codex?.agentsMd ?? null,
6464
+ cursorrules,
6465
+ skills
6455
6466
  };
6456
6467
  }
6457
- function validateSetup(setup, dir, checkExists = existsSync9) {
6468
+ function buildGroundingFixInstruction(unmentionedTopDirs, projectStructure) {
6469
+ const dirDescriptions = unmentionedTopDirs.slice(0, 8).map((dir) => {
6470
+ const subdirs = projectStructure.dirs.filter((d) => d.startsWith(`${dir}/`) && !d.includes("/", dir.length + 1));
6471
+ const files = projectStructure.files.filter((f) => f.startsWith(`${dir}/`) && !f.includes("/", dir.length + 1));
6472
+ const children = [...subdirs.slice(0, 4), ...files.slice(0, 2)];
6473
+ const childList = children.map((c) => c.split("/").pop()).join(", ");
6474
+ return childList ? `- \`${dir}/\` (contains: ${childList})` : `- \`${dir}/\``;
6475
+ });
6476
+ return [
6477
+ "Reference these project directories with descriptions of their contents:",
6478
+ ...dirDescriptions,
6479
+ "Mention them naturally in architecture descriptions using dense inline references."
6480
+ ].join("\n");
6481
+ }
6482
+ function validateSetup(setup, dir, checkExists = existsSync9, projectStructure) {
6458
6483
  const issues = [];
6459
- const { claudeMd, agentsMd } = extractConfigContent(setup);
6484
+ const { claudeMd, agentsMd, cursorrules, skills } = extractConfigContent(setup);
6460
6485
  const primaryContent = [claudeMd, agentsMd].filter(Boolean).join("\n");
6461
6486
  if (!primaryContent) return issues;
6462
6487
  const refs = validateFileReferences(primaryContent, dir, checkExists);
@@ -6487,15 +6512,15 @@ function validateSetup(setup, dir, checkExists = existsSync9) {
6487
6512
  }
6488
6513
  const content = claudeMd ?? agentsMd ?? "";
6489
6514
  if (content) {
6490
- const structure = analyzeMarkdownStructure(content);
6491
- const blockThreshold = CODE_BLOCK_THRESHOLDS.find((t) => structure.codeBlockCount >= t.minBlocks);
6515
+ const structure2 = analyzeMarkdownStructure(content);
6516
+ const blockThreshold = CODE_BLOCK_THRESHOLDS.find((t) => structure2.codeBlockCount >= t.minBlocks);
6492
6517
  const blockPoints = blockThreshold?.points ?? 0;
6493
6518
  const maxBlockPoints = CODE_BLOCK_THRESHOLDS[0].points;
6494
- if (blockPoints < maxBlockPoints && structure.codeBlockCount < CODE_BLOCK_THRESHOLDS[0].minBlocks) {
6519
+ if (blockPoints < maxBlockPoints && structure2.codeBlockCount < CODE_BLOCK_THRESHOLDS[0].minBlocks) {
6495
6520
  issues.push({
6496
6521
  check: "Executable content",
6497
- detail: `${structure.codeBlockCount} code block${structure.codeBlockCount === 1 ? "" : "s"} (need \u2265${CODE_BLOCK_THRESHOLDS[0].minBlocks} for full points)`,
6498
- fixInstruction: `Add ${CODE_BLOCK_THRESHOLDS[0].minBlocks - structure.codeBlockCount} more code blocks with actual project commands (build, test, lint, deploy).`,
6522
+ detail: `${structure2.codeBlockCount} code block${structure2.codeBlockCount === 1 ? "" : "s"} (need \u2265${CODE_BLOCK_THRESHOLDS[0].minBlocks} for full points)`,
6523
+ fixInstruction: `Add ${CODE_BLOCK_THRESHOLDS[0].minBlocks - structure2.codeBlockCount} more code blocks with actual project commands (build, test, lint, deploy).`,
6499
6524
  pointsLost: maxBlockPoints - blockPoints
6500
6525
  });
6501
6526
  }
@@ -6522,15 +6547,86 @@ function validateSetup(setup, dir, checkExists = existsSync9) {
6522
6547
  pointsLost: POINTS_NO_DIR_TREE
6523
6548
  });
6524
6549
  }
6525
- if (structure.h2Count < 3 || structure.listItemCount < 3) {
6550
+ if (structure2.h2Count < 3 || structure2.listItemCount < 3) {
6526
6551
  const parts = [];
6527
- if (structure.h2Count < 3) parts.push(`add ${3 - structure.h2Count} more ## sections`);
6528
- if (structure.listItemCount < 3) parts.push("use bullet lists for multi-item instructions");
6552
+ if (structure2.h2Count < 3) parts.push(`add ${3 - structure2.h2Count} more ## sections`);
6553
+ if (structure2.listItemCount < 3) parts.push("use bullet lists for multi-item instructions");
6529
6554
  issues.push({
6530
6555
  check: "Structured with headings",
6531
- detail: `${structure.h2Count} sections, ${structure.listItemCount} list items`,
6556
+ detail: `${structure2.h2Count} sections, ${structure2.listItemCount} list items`,
6532
6557
  fixInstruction: `Improve structure: ${parts.join(" and ")}.`,
6533
- pointsLost: POINTS_HAS_STRUCTURE - ((structure.h2Count >= 3 ? 1 : 0) + (structure.listItemCount >= 3 ? 1 : 0))
6558
+ pointsLost: POINTS_HAS_STRUCTURE - ((structure2.h2Count >= 3 ? 1 : 0) + (structure2.listItemCount >= 3 ? 1 : 0))
6559
+ });
6560
+ }
6561
+ }
6562
+ const structure = projectStructure ?? collectProjectStructure(dir);
6563
+ const allEntries = [...structure.dirs, ...structure.files].filter((e) => e.length > 2);
6564
+ if (allEntries.length > 0) {
6565
+ const contentLower = primaryContent.toLowerCase();
6566
+ const mentionedEntries = allEntries.filter((e) => isEntryMentioned(e, contentLower));
6567
+ const groundingRatio = mentionedEntries.length / allEntries.length;
6568
+ const groundingThreshold = GROUNDING_THRESHOLDS.find((t) => groundingRatio >= t.minRatio);
6569
+ const groundingPoints = groundingThreshold?.points ?? 0;
6570
+ const groundingLost = POINTS_PROJECT_GROUNDING - groundingPoints;
6571
+ if (groundingLost > 0) {
6572
+ const topDirs = structure.dirs.filter((d) => !d.includes("/") && d.length > 2);
6573
+ const unmentionedTopDirs = topDirs.filter((d) => !isEntryMentioned(d, contentLower));
6574
+ if (unmentionedTopDirs.length > 0) {
6575
+ issues.push({
6576
+ check: "Project grounding",
6577
+ detail: `${mentionedEntries.length}/${allEntries.length} project entries referenced (${Math.round(groundingRatio * 100)}%)`,
6578
+ fixInstruction: buildGroundingFixInstruction(unmentionedTopDirs, structure),
6579
+ pointsLost: groundingLost
6580
+ });
6581
+ }
6582
+ }
6583
+ }
6584
+ const allRefs = extractReferences(primaryContent);
6585
+ const primaryStructure = analyzeMarkdownStructure(primaryContent);
6586
+ const totalSpecificRefs = allRefs.length + primaryStructure.inlineCodeCount;
6587
+ const density = primaryStructure.nonEmptyLines > 0 ? totalSpecificRefs / primaryStructure.nonEmptyLines * 100 : 0;
6588
+ const densityPoints = calculateDensityPoints(density, POINTS_REFERENCE_DENSITY);
6589
+ const densityLost = POINTS_REFERENCE_DENSITY - densityPoints;
6590
+ if (densityLost > 0) {
6591
+ issues.push({
6592
+ check: "Reference density",
6593
+ detail: `${totalSpecificRefs} references across ${primaryStructure.nonEmptyLines} lines (${Math.round(density)}% density, need \u226540% for full points)`,
6594
+ fixInstruction: `Add more backtick references around file paths, commands, and identifiers. Use the dense reference style: \`src/api/\` routes \xB7 \`src/models/\` data. Current density: ${Math.round(density)}%, target: \u226540%.`,
6595
+ pointsLost: densityLost
6596
+ });
6597
+ }
6598
+ if (claudeMd && cursorrules) {
6599
+ const duplicatePercent = calculateDuplicatePercent(claudeMd, cursorrules);
6600
+ if (duplicatePercent > 50) {
6601
+ issues.push({
6602
+ check: "No duplicate content",
6603
+ detail: `${duplicatePercent}% overlap between CLAUDE.md and .cursorrules`,
6604
+ fixInstruction: "Deduplicate content. Keep shared instructions in CLAUDE.md only. Make .cursorrules contain only Cursor-specific settings and platform differences.",
6605
+ pointsLost: POINTS_NO_DUPLICATES
6606
+ });
6607
+ }
6608
+ }
6609
+ for (const skill of skills) {
6610
+ const skillIssues = [];
6611
+ const skillRefs = validateFileReferences(skill.content, dir, checkExists);
6612
+ if (skillRefs.invalid.length > 0) {
6613
+ skillIssues.push(`invalid refs: ${skillRefs.invalid.slice(0, 3).join(", ")}`);
6614
+ }
6615
+ const skillStructure = analyzeMarkdownStructure(skill.content);
6616
+ if (skillStructure.codeBlockCount === 0 && skill.content.length > 200) {
6617
+ skillIssues.push("no code blocks");
6618
+ }
6619
+ const { concrete, abstract } = countConcreteness(skill.content);
6620
+ const total = concrete + abstract;
6621
+ if (total > 3 && concrete / total < 0.3) {
6622
+ skillIssues.push("low concreteness");
6623
+ }
6624
+ if (skillIssues.length > 0) {
6625
+ issues.push({
6626
+ check: `Skill quality: ${skill.name}`,
6627
+ detail: skillIssues.join("; "),
6628
+ fixInstruction: `Fix skill "${skill.name}": ${skillIssues.join(", ")}.${skillRefs.invalid.length > 0 ? ` Remove invalid paths: ${skillRefs.invalid.join(", ")}.` : ""} Add code blocks and specific file references.`,
6629
+ pointsLost: 0
6534
6630
  });
6535
6631
  }
6536
6632
  }
@@ -6560,23 +6656,25 @@ async function scoreAndRefine(setup, dir, sessionHistory, callbacks) {
6560
6656
  existsCache.set(path29, result);
6561
6657
  return result;
6562
6658
  };
6659
+ const projectStructure = collectProjectStructure(dir);
6563
6660
  let currentSetup = setup;
6564
6661
  let bestSetup = setup;
6565
6662
  let bestLostPoints = Infinity;
6566
6663
  for (let iteration = 0; iteration < MAX_REFINE_ITERATIONS; iteration++) {
6567
- const issues = validateSetup(currentSetup, dir, cachedExists);
6664
+ const issues = validateSetup(currentSetup, dir, cachedExists, projectStructure);
6568
6665
  const lostPoints = countIssuePoints(issues);
6569
6666
  if (lostPoints < bestLostPoints) {
6570
6667
  bestSetup = currentSetup;
6571
6668
  bestLostPoints = lostPoints;
6572
6669
  }
6573
- if (issues.length === 0) {
6670
+ if (lostPoints === 0) {
6574
6671
  if (callbacks?.onStatus) callbacks.onStatus("Setup passes all scoring checks");
6575
6672
  return bestSetup;
6576
6673
  }
6674
+ const pointIssues = issues.filter((i) => i.pointsLost > 0);
6675
+ const pointIssueNames = pointIssues.map((i) => i.check).join(", ");
6577
6676
  if (callbacks?.onStatus) {
6578
- const issueNames = issues.map((i) => i.check).join(", ");
6579
- callbacks.onStatus(`Fixing ${issues.length} scoring issue${issues.length === 1 ? "" : "s"}: ${issueNames}...`);
6677
+ callbacks.onStatus(`Fixing ${pointIssues.length} scoring issue${pointIssues.length === 1 ? "" : "s"}: ${pointIssueNames}...`);
6580
6678
  }
6581
6679
  const refined = await applyTargetedFixes(currentSetup, issues);
6582
6680
  if (!refined) {
@@ -6585,15 +6683,15 @@ async function scoreAndRefine(setup, dir, sessionHistory, callbacks) {
6585
6683
  }
6586
6684
  sessionHistory.push({
6587
6685
  role: "user",
6588
- content: `Fix scoring issues: ${issues.map((i) => i.check).join(", ")}`
6686
+ content: `Fix scoring issues: ${pointIssueNames}`
6589
6687
  });
6590
6688
  sessionHistory.push({
6591
6689
  role: "assistant",
6592
- content: `Applied scoring fixes for: ${issues.map((i) => i.check).join(", ")}`
6690
+ content: `Applied scoring fixes for: ${pointIssueNames}`
6593
6691
  });
6594
6692
  currentSetup = refined;
6595
6693
  }
6596
- const finalIssues = validateSetup(currentSetup, dir, cachedExists);
6694
+ const finalIssues = validateSetup(currentSetup, dir, cachedExists, projectStructure);
6597
6695
  const finalLostPoints = countIssuePoints(finalIssues);
6598
6696
  if (finalLostPoints < bestLostPoints) {
6599
6697
  bestSetup = currentSetup;
@@ -6601,10 +6699,19 @@ async function scoreAndRefine(setup, dir, sessionHistory, callbacks) {
6601
6699
  return bestSetup;
6602
6700
  }
6603
6701
  async function applyTargetedFixes(setup, issues) {
6604
- const { claudeMd, agentsMd } = extractConfigContent(setup);
6702
+ const { claudeMd, agentsMd, cursorrules, skills } = extractConfigContent(setup);
6605
6703
  const targets = [];
6606
6704
  if (claudeMd) targets.push({ key: "claudeMd", label: "CLAUDE.md", content: claudeMd });
6607
6705
  if (agentsMd) targets.push({ key: "agentsMd", label: "AGENTS.md", content: agentsMd });
6706
+ const failingChecks = new Set(issues.map((i) => i.check));
6707
+ if (cursorrules && failingChecks.has("No duplicate content")) {
6708
+ targets.push({ key: "cursorrules", label: ".cursorrules", content: cursorrules });
6709
+ }
6710
+ for (const skill of skills) {
6711
+ if (failingChecks.has(`Skill quality: ${skill.name}`)) {
6712
+ targets.push({ key: `skill:${skill.name}`, label: `Skill: ${skill.name}`, content: skill.content });
6713
+ }
6714
+ }
6608
6715
  if (targets.length === 0) return null;
6609
6716
  const feedbackMessage = buildFeedbackMessage(issues);
6610
6717
  const contentBlock = targets.map(
@@ -6621,11 +6728,12 @@ ${t.content}
6621
6728
  `
6622
6729
  Return ONLY the fixed content as a JSON object with keys ${targets.map((t) => `"${t.key}"`).join(", ")}. Each value is the fixed markdown string. No code fences, no explanations.`
6623
6730
  ].join("\n");
6731
+ const maxTokens = Math.min(12e3, 4e3 + targets.length * 2e3);
6624
6732
  try {
6625
6733
  const raw = await llmCall({
6626
- system: "You fix scoring issues in AI agent configuration files. Return only a JSON object with the fixed content \u2014 no explanations, no code fences.",
6734
+ system: "You fix scoring issues in AI agent configuration files. You may receive CLAUDE.md, AGENTS.md, .cursorrules, and/or skill files. Return only a JSON object with the fixed content \u2014 no explanations, no code fences.",
6627
6735
  prompt,
6628
- maxTokens: 8e3
6736
+ maxTokens
6629
6737
  });
6630
6738
  const cleaned = stripMarkdownFences(raw);
6631
6739
  const jsonStart = cleaned.indexOf("{");
@@ -6633,9 +6741,28 @@ Return ONLY the fixed content as a JSON object with keys ${targets.map((t) => `"
6633
6741
  const fixes = JSON.parse(jsonToParse);
6634
6742
  const patched = structuredClone(setup);
6635
6743
  for (const target of targets) {
6636
- if (typeof fixes[target.key] === "string" && fixes[target.key].length > 50) {
6637
- const parent = target.key === "claudeMd" ? patched.claude : patched.codex;
6638
- if (parent) parent[target.key] = fixes[target.key];
6744
+ const fixedValue = fixes[target.key];
6745
+ if (typeof fixedValue !== "string" || fixedValue.length < 50) continue;
6746
+ if (target.key === "claudeMd") {
6747
+ const parent = patched.claude;
6748
+ if (parent) parent.claudeMd = fixedValue;
6749
+ } else if (target.key === "agentsMd") {
6750
+ const parent = patched.codex;
6751
+ if (parent) parent.agentsMd = fixedValue;
6752
+ } else if (target.key === "cursorrules") {
6753
+ const parent = patched.cursor;
6754
+ if (parent) parent.cursorrules = fixedValue;
6755
+ } else if (target.key.startsWith("skill:")) {
6756
+ const skillName = target.key.slice(6);
6757
+ for (const platform of ["claude", "cursor", "codex"]) {
6758
+ const platformObj = patched[platform];
6759
+ const platformSkills = platformObj?.skills;
6760
+ const skill = platformSkills?.find((s) => s.name === skillName);
6761
+ if (skill) {
6762
+ skill.content = fixedValue;
6763
+ break;
6764
+ }
6765
+ }
6639
6766
  }
6640
6767
  }
6641
6768
  return patched;
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@rely-ai/caliber",
3
- "version": "1.21.0",
3
+ "version": "1.22.0-dev.1773744838",
4
4
  "description": "Analyze your codebase and generate optimized AI agent configs (CLAUDE.md, .cursorrules, skills) — no API key needed",
5
5
  "type": "module",
6
6
  "bin": {