bun-plugin-civet-sourcemaps 0.0.0-stage → 0.0.1

package/LICENSE

@@ -0,0 +1,17 @@
+MIT License
+
+Copyright (c) 2026 Contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.

package/README.md

@@ -1,3 +1,84 @@
-# Temporary Holding Version
+# bun-plugin-civet-sourcemaps
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+**Unpublished 0.0.1 prototype** — a Bun bundler plugin that preserves original `.civet` locations after Civet → TypeScript/TSX → JavaScript compilation.
+
+This is a real Bun `BunPlugin`, not a wrapper around `Bun.build()`. It uses `onLoad` to compile Civet with an inline source map and `onEnd` to compose Bun's output source maps with each Civet input map. No separate postbuild command or source-map library is needed.
+
+## Quick start
+
+Install the package (from the accompanying local tarball until it is published) and Civet:
+
+```sh
+bun add -d @danielx/civet ./bun-plugin-civet-sourcemaps-0.0.1.tgz
+```
+
+Create `build.mjs`:
+
+```js
+import { civetSourceMaps } from 'bun-plugin-civet-sourcemaps';
+
+const result = await Bun.build({
+  entrypoints: ['./src/main.civet'],
+  outdir: './dist',
+  target: 'browser',
+  sourcemap: 'linked',
+  plugins: [civetSourceMaps()],
+});
+
+if (!result.success) throw new AggregateError(result.logs, 'Build failed');
+```
+
+Then run `bun build.mjs`. With `sourcemap: 'linked'`, DevTools receives `dist/main.js` and `dist/main.js.map`; mapped generated positions should refer to the original `.civet` source. `sourcemap: 'external'` works too, but doesn't append a sourceMappingURL comment.
+
+## API
+
+```ts
+civetSourceMaps({
+  // optional; default: lazy import of @danielx/civet's `compile`
+  compile?: (source: string, options: object) => string | Promise<string>,
+  // options passed through to Civet, e.g. { comptime: true }
+  compileOptions?: object,
+  // default: /\.civet$/
+  filter?: RegExp,
+})
+```
+
+`filename`, `outputFilename`, `inlineMap: true`, and `sourceMap: false` are forced to ensure reliable map generation and a string compiler return. `filter` must not have global or sticky flags.
+
+This plugin **replaces Civet's existing Bun `onLoad` handler** when used with `Bun.build()`. Do not install two `.civet` handlers for the same files. It is **not** a preload-time source-map solution for `bun file.civet`.
+
+## What it does
+
+1. `onStart`: clear the per-build registry.
+2. `onLoad`: read `.civet`, invoke Civet, record its inline Source Map v3, return TSX source to Bun.
+3. Bun bundles and creates its usual intermediate-TSX-based source map.
+4. `onEnd`: match input files individually, compose each final map, and rewrite the emitted `.js.map` files **before `Bun.build()` resolves**.
+5. Unmodified TypeScript/JavaScript input mappings remain untouched; `sourcesContent` embeds original Civet text.
+
+A missing compiler map, unsupported source map, absent output map, or disabled sourcemap causes an error rather than silently publishing incorrect debug positions.
+
+## Testing
+
+The supplied Bun Linux x64 **1.4.4** binary was used for integration tests:
+
+```sh
+bun test
+bun tests/integration.test.mjs
+```
+
+Unit tests exercise Base64 VLQ and map composition. Integration tests exercise Bun's actual `onLoad`/`onEnd`, multiple `.civet` modules, `.ts` input, multiple entrypoints, minification, repeated builds, linked/external maps, and failures. The compiler used in tests is a **simulated Civet compiler** with legitimate Source Map v3 output.
+
+**Not yet verified with actual Civet:** `@danielx/civet` is not present in the offline test environment. This remains an experimental package pending real compiler fixture verification. Please provide the Civet tarball and dependencies to run this step.
+
+## Current constraints
+
+- Requires `Bun.build({ sourcemap: 'linked' | 'external' })` and disk writes. No `sourcemap: 'inline'` or `write: false` support.
+- Bun `onEnd` is a bundler lifecycle hook, so this does not add runtime `bun run` source-map support.
+- Source Map v3 standard (not indexed/sectioned maps); the small in-house composer supports common local filesystem sources but is not a production-hardened implementation of URL/Windows/indexed maps or source-map edge cases.
+- Compiled input paths are matched against Bun's output map paths; filename discrepancies and source map changes in real Civet need further validation.
+- Watch/incremental builds must trigger a new Bun build so `onEnd` runs again. Bun's own watcher may need special handling for plugin-loaded files.
+- Other plugins with overlapping `onLoad` filters can conflict. When chaining other transforms, each intermediate stage needs its own upstream mapping.
+
+## Package contents
+
+The npm tarball contains `src/`, `README.md`, and `LICENSE`, not internal handoff docs or tests. The separate development source ZIP contains tests, examples, and `HANDOFF.md`.

package/package.json

@@ -1,6 +1,27 @@
 {
   "name": "bun-plugin-civet-sourcemaps",
-  "version": "0.0.0-stage",
-  "stub": true,
-  "description": "Temporary package placeholder for staged publishing"
-}
+  "version": "0.0.1",
+  "description": "Bun bundler plugin that maps compiled Civet/TSX back to original .civet source files",
+  "type": "module",
+  "exports": {
+    ".": {
+      "types": "./src/index.d.mts",
+      "default": "./src/index.mjs"
+    }
+  },
+  "main": "./src/index.mjs",
+  "types": "./src/index.d.mts",
+  "files": ["src", "README.md", "LICENSE"],
+  "peerDependencies": {
+    "@danielx/civet": "^0.11.16"
+  },
+  "engines": {
+    "bun": ">=1.4.4"
+  },
+  "scripts": {
+    "test": "bun test",
+    "test:integration": "bun tests/integration.test.mjs"
+  },
+  "keywords": ["bun", "plugin", "civet", "sourcemap", "typescript"],
+  "license": "MIT"
+}

package/src/compose.mjs

@@ -0,0 +1,98 @@
+import { dirname, isAbsolute, relative, resolve, sep } from 'node:path';
+import { fileURLToPath } from 'node:url';
+import { decodeMappings, encodeMappings, tracePosition } from './vlq.mjs';
+
+function filePath(source, root, basedir) {
+  if (source.startsWith('file://')) return fileURLToPath(source);
+  if (/^[a-zA-Z]+:\/\//.test(source)) return source;
+  const combined = root ? `${root.replace(/\/$/, '')}/${source}` : source;
+  return isAbsolute(combined) ? resolve(combined) : resolve(basedir, combined);
+}
+function relPath(file, basedir) {
+  if (/^[a-zA-Z]+:\/\//.test(file)) return file;
+  const rel = relative(basedir, file).split(sep).join('/');
+  return rel.startsWith('.') ? rel : `./${rel}`;
+}
+function normalizeRecord(record) {
+  if (record && record.map) return record;
+  throw new Error('Expected an upstream record with map and intermediatePath');
+}
+
+/**
+ * Compose Bun's final map with the onLoad compiler maps.
+ * upstreamMaps: Map<absolute plugin input path, {map: SourceMapV3, intermediatePath: absolute path}>
+ * Only plugin-covered Bun inputs are remapped; other JS/TS modules are preserved.
+ */
+export function composeMaps(bunMap, finalMapPath, upstreamMaps) {
+  if (bunMap.sections) throw new Error('Bun indexed/section maps are not yet supported');
+  if (bunMap.version !== 3) throw new Error('Expected Source Map v3');
+  const basedir = dirname(resolve(finalMapPath));
+  const decoded = decodeMappings(bunMap.mappings);
+  const sources = [], sourcesContent = [], names = [];
+  const sourceIndex = new Map(), nameIndex = new Map();
+  const cached = new Map();
+
+  const internSource = (p, content) => {
+    const label = relPath(p, basedir);
+    let n = sourceIndex.get(label);
+    if (n === undefined) {
+      n = sources.length;
+      sourceIndex.set(label, n);
+      sources.push(label);
+      sourcesContent.push(content ?? null);
+    } else if (sourcesContent[n] == null && content != null) {
+      sourcesContent[n] = content;
+    }
+    return n;
+  };
+  const internName = name => {
+    if (name == null) return null;
+    if (!nameIndex.has(name)) { nameIndex.set(name, names.length); names.push(name); }
+    return nameIndex.get(name);
+  };
+  const getUpstream = source => {
+    if (cached.has(source)) return cached.get(source);
+    const path = filePath(source, bunMap.sourceRoot ?? '', basedir);
+    const record = upstreamMaps.get(path);
+    if (!record) { cached.set(source, null); return null; }
+    const {map, intermediatePath} = normalizeRecord(record);
+    if (map.version !== 3 || map.sections) throw new Error(`Invalid/unsupported upstream map for ${path}`);
+    const value = {map, decoded: decodeMappings(map.mappings), dirname: dirname(resolve(intermediatePath))};
+    cached.set(source, value);
+    return value;
+  };
+
+  const composed = decoded.map((line) => line.map(seg => {
+    if (seg.length === 1) return [...seg];
+    const [genCol, bSource, bLine, bCol, bName] = seg;
+    const bunSource = bunMap.sources[bSource];
+    const upstream = getUpstream(bunSource);
+    if (!upstream) {
+      const p = filePath(bunSource, bunMap.sourceRoot ?? '', basedir);
+      const idx = internSource(p, bunMap.sourcesContent?.[bSource]);
+      const name = bName == null ? null : internName(bunMap.names[bName]);
+      return name == null ? [genCol, idx, bLine, bCol] : [genCol, idx, bLine, bCol, name];
+    }
+    const pos = tracePosition(upstream.decoded, bLine, bCol);
+    if (!pos) return [genCol]; // Never silently map generated-only code to unrelated original lines.
+    const [, upSource, origLine, origCol, upName] = pos;
+    const p = filePath(upstream.map.sources[upSource], upstream.map.sourceRoot ?? '', upstream.dirname);
+    const idx = internSource(p, upstream.map.sourcesContent?.[upSource]);
+    const name = upName == null ? (bName == null ? null : internName(bunMap.names[bName])) : internName(upstream.map.names[upName]);
+    return name == null ? [genCol, idx, origLine, origCol] : [genCol, idx, origLine, origCol, name];
+  }));
+
+  return {
+    version: 3, file: bunMap.file, sources, sourcesContent, names,
+    mappings: encodeMappings(composed),
+    ...(bunMap.debugId ? {debugId: bunMap.debugId} : {})
+  };
+}
+
+export function extractInlineMap(code) {
+  const match = /(?:\/\/#|\/\/@)\s*sourceMappingURL=data:application\/json(?:;charset=[^;,]+)?;base64,([A-Za-z0-9+/=]+)/g;
+  let found;
+  for (const next of code.matchAll(match)) found = next[1];
+  if (!found) throw new Error('Compiler did not return an inline data: sourcemap (inlineMap: true)');
+  return JSON.parse(Buffer.from(found, 'base64').toString('utf8'));
+}

package/src/index.d.mts

@@ -0,0 +1,36 @@
+import type { BunPlugin } from 'bun';
+
+export interface SourceMapV3 {
+  version: 3;
+  file?: string;
+  sources: string[];
+  sourcesContent?: (string | null)[];
+  sourceRoot?: string;
+  names: string[];
+  mappings: string;
+  debugId?: string;
+}
+
+export type CivetCompiler = (source: string, options: Record<string, unknown>) => string | Promise<string>;
+
+export interface CivetSourceMapOptions {
+  /** Defaults to dynamically importing @danielx/civet's `compile`. Useful for tests or alternate compilers. */
+  compile?: CivetCompiler;
+  /** Additional Civet options; filename and inlineMap are overwritten for correct map composition. */
+  compileOptions?: Record<string, unknown>;
+  /** Module filter, default /\.civet$/. */
+  filter?: RegExp;
+}
+
+/** Bun.build plugin. Requires sourcemap: 'linked' or 'external' and filesystem writes. */
+export declare function civetSourceMaps(options?: CivetSourceMapOptions): BunPlugin;
+
+/** Compose Bun's Source Map v3 with source maps for transformed module inputs. */
+export declare function composeMaps(
+  bunMap: SourceMapV3,
+  finalMapPath: string,
+  upstreamMaps: Map<string, { map: SourceMapV3; intermediatePath: string }>
+): SourceMapV3;
+
+/** Parse a base64-embedded inline Source Map v3 in compiler output. */
+export declare function extractInlineMap(code: string): SourceMapV3;

package/src/index.mjs

@@ -0,0 +1,91 @@
+import { readFile } from 'node:fs/promises';
+import { resolve } from 'node:path';
+import { composeMaps, extractInlineMap } from './compose.mjs';
+
+/**
+ * Bun.build plugin: compile .civet to TSX and compose Civet maps with Bun's maps in onEnd.
+ * No external source-map library is needed.
+ *
+ * @param {{compile?: Function, compileOptions?: object, filter?: RegExp}} options
+ * @returns {import('bun').BunPlugin}
+ */
+export function civetSourceMaps(options = {}) {
+  const {compile: suppliedCompile, compileOptions = {}, filter = /\.civet$/} = options;
+  if (suppliedCompile !== undefined && typeof suppliedCompile !== 'function') {
+    throw new TypeError('The compile option must be a function');
+  }
+  if (!(filter instanceof RegExp)) throw new TypeError('filter must be a RegExp');
+  if (filter.global || filter.sticky) throw new TypeError('filter must not use the g or y flags');
+
+  // Captured per plugin invocation/build; never shared across plugin instances.
+  const registry = new Map();
+  let compilePromise;
+  const loadCompiler = async () => {
+    if (suppliedCompile) return suppliedCompile;
+    compilePromise ??= import('@danielx/civet').then(mod => {
+      if (typeof mod.compile !== 'function') throw new Error('@danielx/civet does not export compile()');
+      return mod.compile;
+    });
+    return compilePromise;
+  };
+
+  return {
+    name: 'civet-original-sourcemaps',
+    setup(build) {
+      build.onStart(() => registry.clear());
+      build.onLoad({filter}, async ({path}) => {
+        const source = await readFile(path, 'utf8');
+        const compile = await loadCompiler();
+        // Force maps and a stable origin even when a caller passes conflicting options.
+        const contents = await compile(source, {
+          ...compileOptions,
+          filename: resolve(path),
+          outputFilename: `${resolve(path)}.tsx`,
+          sourceMap: false,
+          inlineMap: true,
+        });
+        if (typeof contents !== 'string') {
+          throw new Error(`Civet compile() must return a string for ${path}`);
+        }
+        let map;
+        try {
+          map = extractInlineMap(contents);
+        } catch (error) {
+          throw new Error(`Missing or invalid inline source map for ${path}: ${error.message}`, {cause: error});
+        }
+        // Fallback for compilers that only provide a single source but omit its text.
+        if (map.sources?.length === 1 && map.sourcesContent?.[0] == null) {
+          map.sourcesContent = [source];
+        } else if (map.sources?.length === 1 && !map.sourcesContent) {
+          map.sourcesContent = [source];
+        }
+        registry.set(resolve(path), {map, intermediatePath: `${resolve(path)}.tsx`});
+        return {contents, loader: 'tsx'};
+      });
+
+      build.onEnd(async result => {
+        if (!result.success || registry.size === 0) return;
+        if (build.config.write === false) {
+          throw new Error('civetSourceMaps requires Bun.build write !== false (the maps must be writable)');
+        }
+        if (build.config.sourcemap !== 'linked' && build.config.sourcemap !== 'external') {
+          throw new Error('civetSourceMaps requires Bun.build({ sourcemap: "linked" }) or "external"');
+        }
+        const maps = result.outputs.filter(output => output.kind === 'sourcemap');
+        if (maps.length === 0) {
+          throw new Error('Bun produced no sourcemap artifacts: cannot preserve Civet coordinates');
+        }
+        // Prepare/validate all maps before writing any of them.
+        const rewrites = [];
+        for (const output of maps) {
+          const original = JSON.parse(await output.text());
+          const composed = composeMaps(original, output.path, registry);
+          rewrites.push({output, contents: JSON.stringify(composed)});
+        }
+        for (const {output, contents} of rewrites) await Bun.write(output.path, contents);
+      });
+    },
+  };
+}
+
+export {composeMaps, extractInlineMap} from './compose.mjs';

package/src/vlq.mjs

@@ -0,0 +1,87 @@
+// Source Map v3 Base64-VLQ codec. No runtime dependencies.
+const alphabet = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/';
+const dictionary = Object.fromEntries([...alphabet].map((c, i) => [c, i]));
+
+export function decodeVLQ(text, offset) {
+  let value = 0, shift = 0, digit;
+  do {
+    if (offset >= text.length) throw new Error('Truncated VLQ segment');
+    digit = dictionary[text[offset++]];
+    if (digit === undefined) throw new Error('Invalid VLQ symbol');
+    value += (digit & 31) * 2 ** shift;
+    shift += 5;
+  } while (digit & 32);
+  return { value: value & 1 ? -(Math.floor(value / 2)) : Math.floor(value / 2), offset };
+}
+
+export function encodeVLQ(value) {
+  let unsigned = value < 0 ? -2 * value + 1 : 2 * value;
+  let out = '';
+  do {
+    let digit = unsigned % 32;
+    unsigned = Math.floor(unsigned / 32);
+    if (unsigned) digit |= 32;
+    out += alphabet[digit];
+  } while (unsigned);
+  return out;
+}
+
+// Each segment: [generatedColumn, sourceIndex?, originalLine?, originalColumn?, nameIndex?]
+export function decodeMappings(mappings) {
+  let prevSource = 0, prevLine = 0, prevOriginalCol = 0, prevName = 0;
+  return mappings.split(';').map((line) => {
+    let prevGenCol = 0;
+    if (!line) return [];
+    return line.split(',').map((part) => {
+      let offset = 0;
+      const nums = [];
+      while (offset < part.length) {
+        const res = decodeVLQ(part, offset);
+        nums.push(res.value);
+        offset = res.offset;
+      }
+      if (nums.length !== 1 && nums.length !== 4 && nums.length !== 5) {
+        throw new Error(`Unsupported sourcemap segment with ${nums.length} fields`);
+      }
+      const segment = [prevGenCol += nums[0]];
+      if (nums.length > 1) {
+        segment.push(prevSource += nums[1], prevLine += nums[2], prevOriginalCol += nums[3]);
+        if (nums.length > 4) segment.push(prevName += nums[4]);
+      }
+      return segment;
+    });
+  });
+}
+
+export function encodeMappings(lines) {
+  let prevSource = 0, prevLine = 0, prevOrigCol = 0, prevName = 0;
+  return lines.map(line => {
+    let prevGenCol = 0;
+    return line.map(seg => {
+      let out = encodeVLQ(seg[0] - prevGenCol);
+      prevGenCol = seg[0];
+      if (seg.length > 1) {
+        out += encodeVLQ(seg[1] - prevSource) + encodeVLQ(seg[2] - prevLine) + encodeVLQ(seg[3] - prevOrigCol);
+        prevSource = seg[1]; prevLine = seg[2]; prevOrigCol = seg[3];
+        if (seg.length > 4) {
+          out += encodeVLQ(seg[4] - prevName);
+          prevName = seg[4];
+        }
+      }
+      return out;
+    }).join(',');
+  }).join(';');
+}
+
+export function tracePosition(decodedLines, line, column) {
+  const segments = decodedLines[line] ?? [];
+  // Greatest lower bound, as specified by Source Map v3 tracing.
+  let left = 0, right = segments.length;
+  while (left < right) {
+    const mid = (left + right) >>> 1;
+    if (segments[mid][0] <= column) left = mid + 1;
+    else right = mid;
+  }
+  const segment = segments[left - 1];
+  return segment?.length > 1 ? segment : null;
+}