@slybridges/kiss 0.9.5 → 0.10.0

package/.claude/settings.local.json

@@ -0,0 +1,10 @@
+{
+  "permissions": {
+    "allow": [
+      "Bash(npx eslint:*)",
+      "Bash(grep:*)"
+    ],
+    "deny": [],
+    "ask": []
+  }
+}

package/CLAUDE.md

@@ -0,0 +1,127 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project Overview
+
+KISS (Keep It Simple and Static) is a low-tech static site generator built with Node.js. It transforms markdown, HTML, JSON, and JavaScript content into static websites using Nunjucks templates.
+
+## Development Commands
+
+### Build the site
+```bash
+npx kiss build
+# Production build:
+NODE_ENV=production npx kiss build
+```
+
+### Development server with auto-reload
+```bash
+npx kiss start
+# Launches build, watches for changes, and auto-reloads browser
+```
+
+### Watch mode only (no server)
+```bash
+npx kiss watch
+# Use --incremental flag for experimental incremental builds
+```
+
+### Serve only (no watch)
+```bash
+npx kiss serve
+```
+
+### Linting and formatting
+```bash
+npx eslint .
+npx prettier --write .
+# Note: Prettier config uses no semicolons
+```
+
+## Architecture
+
+### Build Pipeline
+
+The build process (`src/build.js`) follows this lifecycle:
+
+1. **Config Loading**: Loads `kiss.config.js` with defaults from `src/config/defaultConfig.js`
+2. **Content Loading**: Scans `content/` directory recursively, loading files via loaders
+3. **Data Cascade**: Parent data cascades to children, with dynamic computations at each level
+4. **Content Transformation**: Applies transforms (Nunjucks, @attributes, images)
+5. **Data Views**: Computes derived data (collections, categories, site metadata)
+6. **Writing**: Outputs HTML, optimized images, RSS, sitemap, and JSON context
+
+### Content Organization
+
+- Content lives in `content/` directory
+- Folder structure maps to URL structure
+- Index files (`index.js/md/html`) provide data cascade to children
+- Single files or directories with `post.md/html` become pages
+
+### Key Data Flow
+
+Pages have a data structure with:
+- User content fields (title, description, created, etc.)
+- Computed fields via dynamic functions (permalink, URL, image optimization)
+- `_meta` object with internal metadata (id, parent, children, paths)
+- `_html` populated by content transformers
+
+### Dynamic Computations
+
+Located in `src/data/`, these functions run during data cascade to compute values:
+- `computePermalink`: Generates URL path from file location
+- `computeTitle`: Auto-generates from filename if not provided
+- `computeImage`: Handles responsive image generation
+- `computeDescription`: Extracts from content if not provided
+
+### Content Loaders
+
+Located in `src/loaders/`:
+- `markdownLoader`: Processes .md files with front-matter
+- `jsLoader`: Executes .js files that export data
+- `jsonLoader`: Loads .json data files
+- `textLoader`: Loads .html and other text files
+- `staticLoader`: Copies static assets as-is
+
+### Content Transforms
+
+Located in `src/transforms/`:
+- `nunjucksContentTransform`: Processes Nunjucks templates in content
+- `atAttributesContentTransform`: Handles @attribute syntax for dynamic data insertion
+- `imageContextTransform`: Optimizes and creates responsive images
+
+### Writers
+
+Located in `src/writers/`:
+- `htmlWriter`: Renders pages using Nunjucks templates
+- `imageWriter`: Generates optimized image variants with Sharp
+- `rssContextWriter`: Creates RSS feed
+- `sitemapContextWriter`: Generates sitemap.xml
+- `jsonContextWriter`: Dumps full site context for debugging
+
+## Configuration
+
+Main config file is `kiss.config.js`. Key settings:
+
+- `context.site`: Site-wide metadata (url, title, description)
+- `dirs`: Directory paths (content, public, theme, templates)
+- `hooks`: Extensibility points for custom loaders, transforms, writers
+- `defaults`: Default values and behaviors
+- `dataViews`: Functions to compute derived data
+
+## Template System
+
+Uses Nunjucks templates in `theme/templates/`:
+- Templates have access to full page data and site context
+- Default templates: `default.njk`, `collection.njk`, `post.njk`
+- Custom filters loaded via `src/libs/loadNunjucksFilters.js`
+
+## Important Conventions
+
+- Code style: No semicolons (per Prettier config)
+- Node.js 20+ required
+- Uses CommonJS modules (require/module.exports)
+- Async/await for asynchronous operations
+- Lodash for utility functions
+- Fast-glob for file system operations

package/README.md

@@ -12,11 +12,12 @@
 ## What kiss is
 
 - **Low-tech**: VanillaJS, small codebase, little abstractions.
-- **Minimal**: No framework, no code transpiler, little dependencies.
-- **Batteries included**: Comes out of the box with everything you need to make an SEO friendly website
+- **Minimal**: No framework, no code transpiler, few dependencies.
+- **Batteries included**: Comes out of the box with everything you need to make an SEO-friendly website.
+- **Built for performance**: Optimized for large sites with thousands of pages (one site we run has 5,000+ pages and builds under 6 minutes in the cloud and 3 minutes locally).
 - **Developer friendly**: `kiss start` will watch your changes and reload the browser after every build so that you can iterate quickly.
 - **Powerful**: Dynamic data computations, page data cascade and derived content generation.
-- **Extensible**: Easily add support for more content types, dynamic computations, writers, post build commands via hooks, etc.
+- **Extensible**: Easily add support for more content types, dynamic computations, writers, post-build commands via hooks, etc.
 
 ## How it works:
 
@@ -28,10 +29,10 @@
 - Create custom pages derived from the main data (e.g list of articles by tags or articles by author's)
 - Pre-compute derived data views based (e.g. compute the list of categories and subcategories for generating the navigation bar)
 
-kiss will automatically make your site SEO friendly by default:
+kiss will automatically make your site SEO-friendly by default:
 
 - Optimize images and make them responsive
-- Data cascade makes it trivial to generate meta and Open Graph tags
+- Data cascade makes it trivial to generate different sections of your site with different layouts and data content
 - Generate RSS feed
 - Generate sitemap
 - Generate a dump of your full site as JSON for debug or to implement actions via workers (like site search)

package/TODO.md

@@ -0,0 +1,5 @@
+TODO
+- review code
+- try to get read of findBy duplication in atAttributesContentTransform.js
+- test deployment on cloudflare pages
+- test batch writes

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@slybridges/kiss",
-  "version": "0.9.5",
+  "version": "0.10.0",
   "description": "Keep It Simple and Static site generator",
   "main": "src/index.js",
   "bin": {

package/src/build.js

@@ -12,6 +12,7 @@ const {
   getPageFromInputPath,
   relativeToAbsoluteAttributes,
 } = require("./helpers")
+const { buildPageIndexes } = require("./indexing")
 const { baseLoader } = require("./loaders")
 const { setGlobalLogger } = require("./logger")
 
@@ -97,6 +98,26 @@ const build = async (options = {}, lastBuild = {}, version = 0) => {
     context = computeAllPagesData(context, config, buildFlags)
   }
 
+  // Build indexes for O(1) lookups during transforms after dynamic data is computed
+  if (config.defaults.enablePageIndexes) {
+    global.logger.info("Building page indexes for fast lookups")
+    context._pageIndexes = buildPageIndexes(context.pages)
+    global.logger.log(
+      `- Built ${Object.keys(context._pageIndexes).length} indexes for ${Object.keys(context.pages).length} pages`,
+    )
+  } else {
+    // Indexes disabled - transforms will use O(n) fallbacks
+    // This is supported but not recommended for large sites
+    global.logger.info(
+      "Page indexes disabled - transforms will use O(n) page searches",
+    )
+    if (Object.keys(context.pages).length > 1000) {
+      global.logger.warn(
+        "Consider enabling page indexes for better performance (set config.defaults.enablePageIndexes = true)",
+      )
+    }
+  }
+
   if (buildFlags.dataViews) {
     global.logger.section("Computing data views")
     context = computeDataViews(context, config)

package/src/config/defaultConfig.js

@@ -75,6 +75,9 @@ const defaultConfig = {
     // https://moz.com/learn/seo/meta-description
     // This settings only applies to description automatically generated. If you provide your own description, it will be used as is.
     descriptionLength: 160,
+    // PERFORMANCE OPTIMIZATION: Enable centralized page indexes for O(1) lookups during transforms
+    // Set to false only for extremely memory-constrained environments
+    enablePageIndexes: true,
     maxComputingRounds: 10,
     pageData: initialPageData,
     pagePublishedAttribute: "created",

package/src/helpers.js

@@ -1,6 +1,11 @@
 const _ = require("lodash")
 const fs = require("fs")
 const path = require("path")
+const {
+  findPageByPermalink,
+  findPageByDerivative,
+  findParentByPermalink,
+} = require("./indexing")
 
 // @ attribute format: @<attribute>:<value><terminator>
 // terminators: space, comma, newline, end of string, ', ",<, >, ), ], }, #, \
@@ -150,13 +155,13 @@ const getFullPath = (pathname, basePath, options = {}) => {
 
 /** Computes the input path based on the permalink by checking if the parent
  *  had a permalink different than their input path */
-const getInputPath = (permalink, pages, baseContentPath) => {
+const getInputPath = (permalink, pages, baseContentPath, indexes) => {
   const pathObject = path.parse(permalink)
+  const parentPermalink = pathObject.dir + "/"
+
   // search if a have a parent corresponding to this permalink's dir
-  const parent = _.find(
-    pages,
-    (page) => page.permalink === pathObject.dir + "/",
-  )
+  const parent = findParentByPermalink(indexes, parentPermalink, pages)
+
   if (!parent) {
     // no result: assume inputPath same as permalink
     return path.join(baseContentPath, permalink)
@@ -185,31 +190,29 @@ const getLocale = (context, sep = "-") => {
 }
 
 const getPageFromInputPath = (inputPath, pages) => {
-  const pageValues = Object.values(pages)
-  let page = pageValues.find((p) =>
-    p._meta.inputSources.map((s) => s.path).includes(inputPath),
-  )
-  return page
+  // O(n) search - this is only used in build.js where indexes aren't available
+  return Object.values(pages).find((page) => {
+    return page._meta?.inputSources?.some((source) => source.path === inputPath)
+  })
 }
 
 // Tries to find the page corresponding to the source
-// @attributes should have been resolved by mow
+// @attributes should have been resolved by now
 // Supports absolute, and relative paths
 const getPageFromSource = (source, parentPage, pages, config, options = {}) => {
   // source may have URL entities encoded. Decode them
   source = decodeURI(source)
-  const { throwIfNotFound = true } = options
+  const { throwIfNotFound = true, indexes } = options
   // value is a path: compute the permalink in case it is a relative path
   const permalink = getFullPath(source, parentPage.permalink, {
     throwIfInvalid: true,
   })
-  const page = Object.values(pages).find(
-    (p) =>
-      p.permalink === permalink ||
-      // in incremental builds,
-      // also search in derivatives in case the image source was already replaced during previous build
-      p.derivatives?.find((d) => d.permalink === permalink),
-  )
+
+  const page =
+    findPageByPermalink(indexes, permalink, pages) ||
+    // in incremental builds, we also search in derivatives in case the image source was already replaced during previous build
+    findPageByDerivative(indexes, permalink, pages)
+
   if (!page) {
     if (throwIfNotFound) {
       throw new Error(
@@ -355,6 +358,101 @@ const sortPages = (pages, sortBy, { skipUndefinedSort } = {}) => {
   return pages
 }
 
+// Placeholder constants used by jsonSafeStringify/jsonSafeParse
+// These unique strings replace special JS values that JSON can't handle
+const JSON_PLACEHOLDERS = {
+  FUNCTION: "__KISS_FUNCTION__",
+  UNDEFINED: "__KISS_UNDEFINED__",
+  DATE: "__KISS_DATE__",
+}
+
+/**
+ * Converts a JavaScript object to a JSON string while preserving special values
+ * that JSON.stringify normally can't handle (functions, dates, undefined).
+ *
+ * @param {Object} obj - The object to stringify
+ * @returns {Object} { jsonStr: string, specialValues: Map } - The JSON string and a map of special values
+ */
+const jsonSafeStringify = (obj) => {
+  const specialValues = new Map()
+
+  const replacer = (key, value) => {
+    // Handle functions - store them and replace with placeholder
+    if (typeof value === "function") {
+      const id = `${JSON_PLACEHOLDERS.FUNCTION}_${specialValues.size}`
+      specialValues.set(id, value)
+      return id
+    }
+    // Handle dates - store them and replace with placeholder
+    if (value instanceof Date) {
+      const id = `${JSON_PLACEHOLDERS.DATE}_${specialValues.size}`
+      specialValues.set(id, value)
+      return id
+    }
+    // Handle undefined - store it and replace with placeholder
+    if (value === undefined) {
+      const id = `${JSON_PLACEHOLDERS.UNDEFINED}_${specialValues.size}`
+      specialValues.set(id, undefined)
+      return id
+    }
+    return value
+  }
+
+  const jsonStr = JSON.stringify(obj, replacer)
+  return { jsonStr, specialValues }
+}
+
+/**
+ * Parses a JSON string back to an object while restoring special values
+ * that were replaced with placeholders during jsonSafeStringify.
+ *
+ * JSON.parse's reviver function DELETES properties when undefined is returned
+ * This causes properties with undefined values to be lost entirely, so we don't use this feature.
+ * Instead, We use a two-phase approach to get around this issue:
+ * 1. Parse the JSON normally to get object structure
+ * 2. Walk the object and restore special values in-place
+ * This preserves properties that have undefined values.
+ *
+ * @param {string} jsonStr - The JSON string to parse
+ * @param {Map} specialValues - Map of placeholders to their original values
+ * @returns {Object} - The restored JavaScript object with all special values intact
+ */
+const jsonSafeParse = (jsonStr, specialValues) => {
+  // First parse normally to get the object structure
+  const parsed = JSON.parse(jsonStr)
+
+  // Then walk through and restore special values
+  const restoreSpecialValues = (obj) => {
+    if (obj === null || typeof obj !== "object") {
+      return obj
+    }
+
+    for (const key in obj) {
+      const value = obj[key]
+
+      if (typeof value === "string") {
+        // Check if this is a placeholder that needs restoration
+        if (
+          value.startsWith(JSON_PLACEHOLDERS.FUNCTION) ||
+          value.startsWith(JSON_PLACEHOLDERS.DATE) ||
+          value.startsWith(JSON_PLACEHOLDERS.UNDEFINED)
+        ) {
+          if (specialValues.has(value)) {
+            obj[key] = specialValues.get(value)
+          }
+        }
+      } else if (typeof value === "object" && value !== null) {
+        // Recursively process nested objects
+        restoreSpecialValues(value)
+      }
+    }
+
+    return obj
+  }
+
+  return restoreSpecialValues(parsed)
+}
+
 module.exports = {
   AT_FILE_ATTRIBUTE_REGEX,
   AT_GENERIC_ATTRIBUTE_REGEX,
@@ -373,6 +471,8 @@ module.exports = {
   getParentPage,
   isChild,
   isValidURL,
+  jsonSafeParse,
+  jsonSafeStringify,
   omitDeep,
   relativeToAbsoluteAttributes,
   sortPageIds,

package/src/indexing/buildPageIndexes.js

@@ -0,0 +1,73 @@
+/**
+ * Builds centralized indexes for O(1) page lookups throughout the build process.
+ *
+ * FALLBACK SUPPORT:
+ * When indexes are disabled (config.defaults.enablePageIndexes = false),
+ * all lookups fall back to O(n) searches. This is supported but not recommended
+ * for sites with 1000+ pages.
+ *
+ * @param {Object} pages - All pages with resolved data (post-cascade)
+ * @returns {Object} Six specialized indexes for different lookup patterns:
+ *   - byPermalink: page.permalink → page
+ *   - byInputPath: page._meta.inputPath → page
+ *   - byIdAndLang: "id:lang" → page
+ *   - byDerivative: derivative.permalink → page
+ *   - byParentPermalink: directory permalinks → page
+ *   - byInputSource: source.path → page
+ */
+const buildPageIndexes = (pages) => {
+  const indexes = {
+    byPermalink: new Map(),
+    byInputPath: new Map(),
+    byIdAndLang: new Map(),
+    byDerivative: new Map(),
+    byParentPermalink: new Map(),
+    byInputSource: new Map(),
+  }
+
+  // Build all indexes in a single pass through pages
+  for (const page of Object.values(pages)) {
+    // Permalink index
+    if (page.permalink) {
+      indexes.byPermalink.set(page.permalink, page)
+
+      // Parent permalink index for directory lookups
+      // Used by getInputPath to find parent directories
+      if (page.permalink.endsWith("/")) {
+        indexes.byParentPermalink.set(page.permalink, page)
+      }
+    }
+
+    // InputPath index - available after content loading
+    if (page._meta?.inputPath) {
+      indexes.byInputPath.set(page._meta.inputPath, page)
+    }
+
+    // ID and language index for @id resolution
+    if (page.id && page.lang) {
+      indexes.byIdAndLang.set(`${page.id}:${page.lang}`, page)
+    }
+
+    // Derivatives index for image permalinks
+    if (page.derivatives) {
+      for (const derivative of page.derivatives) {
+        if (derivative.permalink) {
+          indexes.byDerivative.set(derivative.permalink, page)
+        }
+      }
+    }
+
+    // Input sources index for getPageFromInputPath
+    if (page._meta?.inputSources) {
+      for (const source of page._meta.inputSources) {
+        if (source.path) {
+          indexes.byInputSource.set(source.path, page)
+        }
+      }
+    }
+  }
+
+  return indexes
+}
+
+module.exports = buildPageIndexes

package/src/indexing/index.js

@@ -0,0 +1,35 @@
+/**
+ * Centralized indexing module for O(1) page lookups.
+ *
+ * ARCHITECTURE DECISION:
+ * On large sites, walking through and finding pages during the transform phase
+ * can be a major performance bottleneck. To address this, we build centralized
+ * indexes for common lookup patterns (by permalink, inputPath, id+lang, etc.)
+ *
+ * BENCHMARKS (5000 page site, 200 @attributes each):
+ * - No indexes: 5+ minutes
+ * - With indexes + helpers: 57 seconds
+ * - With indexes + direct access: 41 seconds (28% faster)
+ */
+
+const buildPageIndexes = require("./buildPageIndexes")
+const {
+  findInIndex,
+  findPageByPermalink,
+  findPageByInputPath,
+  findPageByIdAndLang,
+  findPageByDerivative,
+  findParentByPermalink,
+  findPageByInputSource,
+} = require("./lookupHelpers")
+
+module.exports = {
+  buildPageIndexes,
+  findInIndex,
+  findPageByPermalink,
+  findPageByInputPath,
+  findPageByIdAndLang,
+  findPageByDerivative,
+  findParentByPermalink,
+  findPageByInputSource,
+}

package/src/indexing/lookupHelpers.js

@@ -0,0 +1,114 @@
+/**
+ * Helper functions for efficient index-based lookups with O(n) fallback.
+ *
+ * IMPORTANT PERFORMANCE NOTE:
+ * These helpers add ~0.001ms overhead per call due to function invocation.
+ * For hot paths with 1M+ calls (like @attribute resolution), use direct Map.get() instead.
+ * Benchmark data: 41 seconds with direct access vs 57 seconds with these helpers (40% slower).
+ *
+ * WHEN TO USE THESE HELPERS:
+ * - One-off lookups during page processing
+ * - Non-performance-critical paths
+ * - When code clarity is more important than microsecond optimization
+ *
+ * WHEN NOT TO USE:
+ * - Inside tight loops processing thousands of items
+ * - @attribute resolution (uses inline helpers instead)
+ * - Any path that executes more than 10,000 times per build
+ */
+
+/**
+ * Generic lookup with index (O(1)) and fallback (O(n)) support.
+ *
+ * @param {Object} indexes - The indexes object containing Maps for lookups
+ * @param {string} indexName - Name of the index to use (e.g., 'byPermalink')
+ * @param {*} key - The key to look up in the index
+ * @param {Function} fallbackFn - Optional fallback function for O(n) search
+ * @returns {*} The found item or null
+ */
+const findInIndex = (indexes, indexName, key, fallbackFn) => {
+  // Try index first (O(1))
+  if (indexes && indexes[indexName]) {
+    const result = indexes[indexName].get(key)
+    if (result) {
+      return result
+    }
+  }
+
+  // Fall back to O(n) search if provided
+  // This is expected during dynamic data cascade before indexes are built
+  if (fallbackFn) {
+    return fallbackFn()
+  }
+
+  return null
+}
+
+/**
+ * Find page by permalink using index or fallback.
+ */
+const findPageByPermalink = (indexes, permalink, pages) => {
+  return findInIndex(indexes, "byPermalink", permalink, () =>
+    Object.values(pages).find((p) => p.permalink === permalink),
+  )
+}
+
+/**
+ * Find page by input path using index or fallback.
+ */
+const findPageByInputPath = (indexes, inputPath, pages) => {
+  return findInIndex(indexes, "byInputPath", inputPath, () =>
+    Object.values(pages).find((p) => p._meta?.inputPath === inputPath),
+  )
+}
+
+/**
+ * Find page by ID and language using index or fallback.
+ */
+const findPageByIdAndLang = (indexes, id, lang, pages) => {
+  const key = `${id}:${lang}`
+  return findInIndex(indexes, "byIdAndLang", key, () =>
+    Object.values(pages).find((p) => p.id === id && p.lang === lang),
+  )
+}
+
+/**
+ * Find page by derivative permalink using index or fallback.
+ */
+const findPageByDerivative = (indexes, derivativePermalink, pages) => {
+  return findInIndex(indexes, "byDerivative", derivativePermalink, () =>
+    Object.values(pages).find((p) =>
+      p.derivatives?.find((d) => d.permalink === derivativePermalink),
+    ),
+  )
+}
+
+/**
+ * Find parent page by directory permalink using index or fallback.
+ */
+const findParentByPermalink = (indexes, parentPermalink, pages) => {
+  return findInIndex(indexes, "byParentPermalink", parentPermalink, () =>
+    Object.values(pages).find((p) => p.permalink === parentPermalink),
+  )
+}
+
+/**
+ * Find page by input source path using index or fallback.
+ */
+const findPageByInputSource = (indexes, inputSourcePath, pages) => {
+  return findInIndex(indexes, "byInputSource", inputSourcePath, () =>
+    Object.values(pages).find((p) =>
+      p._meta?.inputSources?.some((s) => s.path === inputSourcePath),
+    ),
+  )
+}
+
+module.exports = {
+  findInIndex,
+  findPageByPermalink,
+  findPageByInputPath,
+  findPageByIdAndLang,
+  findPageByDerivative,
+  findParentByPermalink,
+  findPageByInputSource,
+}