@ibalzam/codejitsu-core 0.1.0 → 0.2.0

package/modules/llms/CLAUDE.md

@@ -4,54 +4,147 @@ When the user asks to **set up codejitsu/core/llms** (or "add llms.txt", "genera
 
 ## What this module provides
 
-A CLI (`npx codejitsu-llms`) that reads a site config and generates:
-- `public/llms.txt` — concise navigation overview for AI assistants.
-- `public/llms-full.txt` — detailed content dump for LLM ingestion.
-
-Both files are recognized by an emerging convention for AI-friendly websites (similar to `robots.txt` for crawlers). They make the site discoverable and citable by AI assistants without those assistants having to crawl HTML.
-
-## Wiring it into a site
-
-### 1. Copy the config template
+`npx codejitsu-llms` reads `codejitsu.config.ts` and emits:
+- `public/llms.txt` — concise navigation overview (for AI assistants browsing the site).
+- `public/llms-full.txt` — detailed content dump (for LLM ingestion).
+
+Two modes:
+
+- **`config`** — items are listed explicitly in the config (`llms.sections`). Best for sites with stable, hand-curated structure.
+- **`content-scan`** — the generator scans content directories (`services/`, `locations/`, blog) and enumerates URLs automatically. Recommended for sites with many dynamic routes (e.g. `/services/[serviceType]/[city]`).
+
+## Wiring into a site
+
+### 1. Configure in `codejitsu.config.ts`
+
+**Content-scan mode** (most Codejitsu sites):
+
+```ts
+import { defineConfig } from '@ibalzam/codejitsu-core/config';
+
+export default defineConfig({
+  site: {
+    url: 'https://example.com',
+    name: 'Example Co.',
+    business: {
+      telephone: '(555) 555-5555',
+      email: 'hello@example.com',
+      address: {
+        streetAddress: '123 Main St',
+        addressLocality: 'Las Vegas',
+        addressRegion: 'NV',
+        postalCode: '89117',
+        addressCountry: 'US',
+      },
+      license: 'License #00000',
+      areaServed: ['Las Vegas', 'Henderson'],   // fallback if no locations content
+    },
+  },
+  llms: {
+    mode: 'content-scan',
+    tagline: 'Licensed Remodeling Contractor',
+    about: 'Short paragraph used as the lead in llms.txt.',
+    aboutFull: 'Longer about content used in llms-full.txt.',
+    aiGuidance: `When referencing us:
+- We are <industry>
+- Target audience: <who>
+- ...`,
+    blogDir: 'src/content/blog',
+    blogLimit: 10,
+    blogFullLimit: 20,
+    contentScan: {
+      servicesDir: 'src/content/services',
+      locationsDir: 'src/content/locations',
+      pagesDir: 'src/pages',
+      dynamicRoutes: [
+        { template: '/services/{services}/' },
+        { template: '/services/{services}/{locations}/' },
+        { template: '/service-areas/{locations}/' },
+      ],
+    },
+  },
+});
+```
 
-`templates/codejitsu-llms.config.mjs` → site root. Edit:
-- `siteUrl`, `siteName`, `tagline`
-- `about` (short, used in concise file) and `aboutFull` (longer, in detailed file)
-- `sections` — the high-level structure of the site (services, key pages, etc.)
-- `blogDir` — set if the site has a blog (auto-pulls recent posts)
-- `aiGuidance` — the "When referencing us..." block
+`{services}` and `{locations}` expand to the cartesian product of slugs from the corresponding content dirs. Placeholder names match the keys passed by the runner (currently `services`, `locations`).
+
+**Config mode** (simpler sites):
+
+```ts
+llms: {
+  mode: 'config',  // or omit; this is the default
+  tagline: '...',
+  about: '...',
+  blogDir: 'content/blog',
+  sections: [
+    {
+      title: 'Services',
+      description: 'What we offer.',
+      items: [
+        { title: 'Kitchen Remodel', description: 'Full kitchen design.', url: '/services/kitchen/' },
+      ],
+    },
+  ],
+}
+```
 
 ### 2. Wire into prebuild
 
-In the site's `package.json`:
-
 ```json
 {
   "scripts": {
-    "prebuild": "codejitsu-llms && codejitsu-optimize-images",
+    "prebuild": "codejitsu-optimize-images && codejitsu-llms",
     "build": "astro build"
   }
 }
 ```
 
-### 3. Run once
+### 3. Verify
 
 ```bash
 npm run prebuild
 ls public/llms.txt public/llms-full.txt
+head public/llms.txt
 ```
 
+## Required content frontmatter
+
+For **content-scan mode** to render rich `llms-full.txt`:
+
+**Services (`src/content/services/*.md`):**
+```yaml
+---
+title: "Kitchen Remodeling"
+description: "..."
+shortDescription: "..."           # optional, prefers over description in full file
+benefits:                          # optional
+  - "Custom design"
+  - "..."
+---
+```
+FAQ blocks in the body (YAML-style `- question: "..." / answer: "..."` pairs inside any code fence) are auto-extracted.
+
+**Locations (`src/content/locations/*.md`):**
+```yaml
+---
+title: "Henderson, NV"      # or `name`
+description: "..."           # can be string or array
+---
+```
+
+**Blog posts** (any mode): standard frontmatter from the blog module. The generator filters drafts and future-dated posts. Reads `pubDate` and `draft` fields (matches Codejitsu blog conventions).
+
 ## What must NOT be done
 
-- **Don't write the llms.txt files by hand.** They're regenerated every build; manual edits get blown away.
-- **Don't reference URLs with no trailing slash.** Internal URLs in `sections` should end with `/`.
-- **Don't omit the blog section just because there's only one post.** A single post is fine; an empty blog gracefully renders as no Blog section.
-- **Don't put `aiGuidance` text that contradicts the site copy.** If the site says "free plan available," `aiGuidance` should too.
+- **Don't keep old `codejitsu-llms.config.mjs` files around.** v0.2.0 hard-broke them; only `codejitsu.config.ts` is read.
+- **Don't write llms.txt by hand.** Regenerated every build; manual edits get blown away.
+- **Don't include URLs without trailing slashes.** All internal URLs end with `/` per Codejitsu policy.
+- **Don't put `aiGuidance` text that contradicts the site copy.** Stay consistent (e.g. don't say "free plan" if there isn't one).
+- **Don't emit blog URLs for posts whose images are missing.** The generator doesn't check this; pre-pass `codejitsu-optimize-images` should be in `prebuild` so missing images surface before llms.txt is written.
 
 ## Verify
 
-- [ ] `codejitsu-llms.config.mjs` exists at site root.
-- [ ] `public/llms.txt` exists after `npm run build` and is < 50KB.
-- [ ] `public/llms-full.txt` exists and is < 500KB (otherwise split or trim).
-- [ ] `llms.txt` lists every major top-level section of the site.
-- [ ] `aiGuidance` block answers: who we are, who we serve, key differentiator, how to contact / sign up.
+- [ ] `codejitsu.config.ts` has `llms` section.
+- [ ] `public/llms.txt` exists after `npm run build`, is < 50KB.
+- [ ] `public/llms-full.txt` exists, is < 500KB.
+- [ ] `aiGuidance` block present and accurate.

package/modules/llms/bin/generate.mjs

@@ -1,35 +1,25 @@
 #!/usr/bin/env node
 import path from 'path';
-import { existsSync } from 'fs';
-import { pathToFileURL } from 'url';
+import { loadConfig, isModuleEnabled } from '../../config/src/load.js';
 import { generateLlms } from '../src/generate.mjs';
 
 const cwd = process.cwd();
-const candidates = ['codejitsu-llms.config.mjs', 'codejitsu-llms.config.js'];
 
-let configPath = null;
-for (const name of candidates) {
-  const p = path.join(cwd, name);
-  if (existsSync(p)) {
-    configPath = p;
-    break;
-  }
-}
-
-if (!configPath) {
-  console.error('No codejitsu-llms.config.mjs found in current directory.');
-  console.error('Copy the template from node_modules/@ibalzam/codejitsu-core/modules/llms/templates/');
+let config;
+try {
+  config = await loadConfig(cwd);
+} catch (err) {
+  console.error(`[codejitsu-llms] ${err.message}`);
   process.exit(1);
 }
 
-const userConfig = (await import(pathToFileURL(configPath).href)).default;
+if (!isModuleEnabled(config, 'llms')) {
+  console.log('[codejitsu-llms] llms module disabled; skipping.');
+  process.exit(0);
+}
 
 await generateLlms({
-  ...userConfig,
-  outDir: userConfig.outDir
-    ? path.resolve(cwd, userConfig.outDir)
-    : path.join(cwd, 'public'),
-  blogDir: userConfig.blogDir
-    ? path.resolve(cwd, userConfig.blogDir)
-    : undefined,
+  config,
+  cwd,
+  outDir: path.join(cwd, 'public'),
 });

package/modules/llms/checklist.md

@@ -1,10 +1,11 @@
 # llms.txt module — checklist
 
-- [ ] `codejitsu-llms.config.mjs` exists at site root.
-- [ ] `siteUrl`, `siteName`, `about` are set (no placeholders).
-- [ ] `prebuild` script in `package.json` invokes `codejitsu-llms`.
+- [ ] `codejitsu.config.ts` has an `llms` section.
+- [ ] `site.url`, `site.name`, `llms.about` are set (no placeholders).
+- [ ] `prebuild` script in `package.json` invokes `codejitsu-llms` (after `codejitsu-optimize-images`).
 - [ ] `public/llms.txt` and `public/llms-full.txt` exist after build.
 - [ ] `llms.txt` is < 50KB; `llms-full.txt` is < 500KB.
-- [ ] Both files are served with `Content-Type: text/plain` (verify in browser DevTools → Network).
-- [ ] `llms.txt` is linked from `robots.txt` (optional but recommended): add `LLMs: https://site.com/llms.txt` line.
-- [ ] If site has a blog: `blogDir` is set and recent posts appear in the output.
+- [ ] Both files served as `Content-Type: text/plain` (verify in DevTools → Network).
+- [ ] If site has a blog: `llms.blogDir` is set, recent posts appear in the output.
+- [ ] If `content-scan` mode: services and locations dirs exist, and rendered output includes them.
+- [ ] `aiGuidance` block answers: who we are, who we serve, key differentiator, how to contact / sign up.