@coldsmirk/inkstone-core 0.20.1 → 0.22.0

package/README.md

@@ -1,13 +1,14 @@
 # @coldsmirk/inkstone-core
 
-The shared [Shiki](https://shiki.style/) highlighter service behind inkstone: one fine-grained, lazily-created singleton (catppuccin latte / macchiato, JS regex engine) that serves the editors, static code rendering, and markdown surfaces alike — one instance, consistent colors everywhere.
+The shared [Shiki](https://shiki.style/) highlighter service behind inkstone: one fine-grained, lazily-created singleton (catppuccin latte / macchiato, the JS regex engine by default or Oniguruma on request) that serves the editors, static code rendering, and markdown surfaces alike — one instance, consistent colors everywhere.
 
 Part of [inkstone](https://github.com/coldsmirk/inkstone). The editor layers ([`@coldsmirk/inkstone-monaco`](https://www.npmjs.com/package/@coldsmirk/inkstone-monaco), [`@coldsmirk/inkstone-codemirror`](https://www.npmjs.com/package/@coldsmirk/inkstone-codemirror)) and the React components ([`@coldsmirk/inkstone-react`](https://www.npmjs.com/package/@coldsmirk/inkstone-react)) consume this package; use it directly when a non-editor surface needs the same highlighting.
 
 ## What you get
 
 - **`getHighlighter()`** — the process-wide `HighlighterCore` promise. JavaScript / TypeScript / JSON / SQL grammars and the catppuccin theme pair load by default.
-- **`configureHighlighter(config)`** — extend or replace the default grammar / theme set, and choose the regex engine (`engine`: the strict JavaScript engine by default; `createJavaScriptRegexEngine({ forgiving: true })` for grammars the strict one refuses, or the Oniguruma WASM engine), once, before any editor mounts. `DEFAULT_HIGHLIGHTER_LANGS` is exported so extension is additive.
+- **`configureHighlighter(config)`** — extend or replace the default grammar / theme set, and choose the regex engine, once, before any editor mounts. `DEFAULT_HIGHLIGHTER_LANGS` is exported so extension is additive. `engine` takes the engine itself, a promise of one, or a factory called once when the highlighter is first built — configuration stays synchronous while the engine may be async to make: the strict JavaScript engine by default, `createJavaScriptRegexEngine({ forgiving: true })` for grammars the strict one refuses, or `onigurumaEngine`.
+- **`onigurumaEngine`** — the Oniguruma engine on its WASM, inlined into a chunk of your own bundle (about 600 KB, nothing fetched from anywhere else) and instantiated once, in about 20 ms. Which engine: the JavaScript one translates each grammar's patterns into JavaScript at the first tokenization, on the main thread, and pays for it on whole files — a TypeScript file's first sixty lines cost it about 170 ms, against Oniguruma's 20. Colouring fenced snippets, keep the default; reading whole files in an editor (`@coldsmirk/inkstone-monaco`'s Shiki bridge, `<MonacoEditor>`), pass `onigurumaEngine` as the factory. `onigurumaEngine(wasm)` takes the WASM from elsewhere in any form Shiki's `createOnigurumaEngine` accepts, such as a `fetch("/onig.wasm")`.
 - **`shikiThemeId(dark)`** plus the `INKSTONE_LIGHT_THEME` / `INKSTONE_DARK_THEME` constants — the theme names to pass wherever a Shiki theme id is expected.
 - **`configureEditorSurface(surfaces)`** — re-light the editors' own surfaces in your design language, once, before any editor mounts: per-scheme CSS color literals for `background` (content and gutter), `lineHighlight`, `lineNumber`, `widget` (completion, hover, parameter hints, the find widget and search panels), `widgetBorder`, and `widgetSelection`. The catppuccin pair keeps its syntax palette and gets these baked into its theme colors — Monaco, static rendering, and CodeMirror's chrome all read the same values — so an editor sits inside the app instead of bringing a second dark. A role or scheme left out keeps the flavor's own color, a later call replaces the schemes it names, and values must be hex colors (`#rrggbb`, `#rrggbbaa`) — Monaco parses theme colors as hex and paints red for anything else, so a `var()` reference or an `rgb(…)` is refused up front. Keep `lineHighlight` translucent: CodeMirror draws selections behind the content, so an opaque active line would hide them. `getEditorSurface(dark)` reads what was configured.
 - **`CODE_FONT_FAMILY`** — the shared code font stack both editor engines render with: the page's `--inkstone-font-family-code` CSS token when defined, else the platform's native monospace.
@@ -24,11 +25,12 @@ pnpm add @coldsmirk/inkstone-core
 ## Quick start
 
 ```ts
-import { configureEditorSurface, configureHighlighter, DEFAULT_HIGHLIGHTER_LANGS, getHighlighter, shikiThemeId } from "@coldsmirk/inkstone-core";
+import { configureEditorSurface, configureHighlighter, DEFAULT_HIGHLIGHTER_LANGS, getHighlighter, onigurumaEngine, shikiThemeId } from "@coldsmirk/inkstone-core";
 import yaml from "@shikijs/langs/yaml";
 
-// Optional: add grammars/themes beyond the defaults — once, before any editor mounts.
-configureHighlighter({ langs: [...DEFAULT_HIGHLIGHTER_LANGS, yaml] });
+// Optional: add grammars/themes beyond the defaults — once, before any editor mounts — and,
+// for an app that reads whole files in an editor, the Oniguruma engine (see above).
+configureHighlighter({ langs: [...DEFAULT_HIGHLIGHTER_LANGS, yaml], engine: onigurumaEngine });
 
 // Optional: paint the editors' surfaces in your own tokens (syntax colors stay catppuccin's).
 configureEditorSurface({

package/dist/index.d.ts

@@ -1,4 +1,4 @@
-import { HighlighterCore, HighlighterCore as HighlighterCore$1, LanguageInput, RegexEngine, ThemeInput, ThemeRegistration } from "shiki/core";
+import { Awaitable, HighlighterCore, HighlighterCore as HighlighterCore$1, LanguageInput, LoadWasmOptions, RegexEngine, ThemeInput, ThemeRegistration } from "shiki/core";
 //#region src/code-font.d.ts
 /**
  * The code font stack every inkstone editor host renders with: the page's
@@ -111,12 +111,36 @@ interface HighlighterConfig {
   themes?: ThemeInput[];
   /**
    * The regex engine the highlighter tokenizes with, replacing the default JavaScript engine
-   * in its strict form — for an app that loads grammars the strict engine refuses
-   * (`createJavaScriptRegexEngine({ forgiving: true })`), or one that ships the Oniguruma
-   * WASM and wants every grammar exact.
+   * in its strict form: the engine itself, a promise of one, or a factory called once when the
+   * highlighter is first built — so configuration stays synchronous (it must precede the first
+   * {@link getHighlighter}) while the engine may be async to make, as the Oniguruma one is.
+   *
+   * The JavaScript engine translates each grammar's patterns into JavaScript at the first
+   * tokenization, on the main thread, and pays for it on whole files: a TypeScript file's
+   * first sixty lines cost it about 170 ms, against Oniguruma's 20. An app that colors fenced
+   * snippets keeps the default; one that reads whole files in an editor configures
+   * {@link onigurumaEngine}; one whose grammars the strict engine refuses configures
+   * `createJavaScriptRegexEngine({ forgiving: true })`.
    */
-  engine?: RegexEngine;
+  engine?: Awaitable<RegexEngine> | (() => Awaitable<RegexEngine>);
 }
+/**
+ * The Oniguruma engine — the one every TextMate grammar was written against, so none is
+ * refused and each compiles in a few milliseconds — on its WASM, inlined by `shiki/wasm` into
+ * a chunk of the app's own bundle (about 600 KB; nothing is fetched from anywhere else), so
+ * it suits an offline or embedded deployment as the JavaScript engine does. Instantiated once,
+ * in about 20 ms. Pass the function itself as the engine factory, so the chunk is fetched and
+ * the WASM instantiated only when the highlighter is first built:
+ *
+ * ```ts
+ * configureHighlighter({ engine: onigurumaEngine });
+ * ```
+ *
+ * `wasm` takes the WASM from elsewhere in any form `createOnigurumaEngine` accepts — a
+ * `fetch("/onig.wasm")` for a deployment that would rather stream-compile the binary than
+ * decode it out of a script.
+ */
+declare function onigurumaEngine(wasm?: LoadWasmOptions): Promise<RegexEngine>;
 /**
  * Configure the shared highlighter before its first use. The highlighter is a process-wide
  * singleton (editors, static rendering, and markdown surfaces all share it for consistent
@@ -147,8 +171,9 @@ declare function getEditorSurface(dark: boolean): EditorSurface | undefined;
  *
  * Fine-grained build: only the configured grammars and the two catppuccin themes are bundled,
  * whereas the convenient `createHighlighter` generates a lazy chunk for *every* Shiki theme and
- * language. The JS regex engine avoids the Oniguruma WASM — smaller, no extra network request,
- * ideal for offline deployments (Tauri webviews, rust-embed consoles).
+ * language. The JavaScript regex engine is the default — nothing to instantiate, no WASM in the
+ * bundle — and right for fenced snippets; an app reading whole files in an editor configures
+ * {@link onigurumaEngine} instead (see {@link HighlighterConfig.engine}).
  *
  * A failed build is not cached: the slot resets so a later call retries (and
  * {@link configureHighlighter} becomes callable again), instead of replaying one transient
@@ -161,4 +186,4 @@ declare function getHighlighter(): Promise<HighlighterCore$1>;
  */
 declare function shikiThemeId(dark: boolean): string;
 //#endregion
-export { CODE_FONT_FAMILY, DEFAULT_HIGHLIGHTER_LANGS, type EditorSurface, type EditorSurfaces, type HighlighterConfig, type HighlighterCore, INKSTONE_DARK_THEME, INKSTONE_LIGHT_THEME, type RadiusSize, type ShadowSize, configureEditorSurface, configureHighlighter, getEditorSurface, getHighlighter, inkstoneRadius, inkstoneShadow, shikiThemeId, surfacedTheme };
+export { CODE_FONT_FAMILY, DEFAULT_HIGHLIGHTER_LANGS, type EditorSurface, type EditorSurfaces, type HighlighterConfig, type HighlighterCore, INKSTONE_DARK_THEME, INKSTONE_LIGHT_THEME, type RadiusSize, type ShadowSize, configureEditorSurface, configureHighlighter, getEditorSurface, getHighlighter, inkstoneRadius, inkstoneShadow, onigurumaEngine, shikiThemeId, surfacedTheme };

package/dist/index.js

@@ -6,6 +6,7 @@ import catppuccinLatte from "@shikijs/themes/catppuccin-latte";
 import catppuccinMacchiato from "@shikijs/themes/catppuccin-macchiato";
 import { createHighlighterCore } from "shiki/core";
 import { createJavaScriptRegexEngine } from "shiki/engine/javascript";
+import { createOnigurumaEngine } from "shiki/engine/oniguruma";
 //#region src/code-font.ts
 const CODE_FONT_FAMILY = "var(--inkstone-font-family-code, ui-monospace, SFMono-Regular, Menlo, Consolas, monospace)";
 //#endregion
@@ -75,6 +76,14 @@ const DEFAULT_HIGHLIGHTER_LANGS = [
 	langJson,
 	langSql
 ];
+function onigurumaEngine(wasm) {
+	return createOnigurumaEngine(wasm ?? import("shiki/wasm"));
+}
+function engineOf(config) {
+	const { engine } = config;
+	if (typeof engine === "function") return Promise.try(() => engine());
+	return engine ?? createJavaScriptRegexEngine();
+}
 const HIGHLIGHTER_REGISTRY_KEY = Symbol.for("coldsmirk.inkstone.shiki.registry");
 function getHighlighterRegistry() {
 	const host = globalThis;
@@ -129,7 +138,7 @@ function getHighlighter() {
 			...registry.config.themes ?? []
 		],
 		langs: registry.config.langs ?? [...DEFAULT_HIGHLIGHTER_LANGS],
-		engine: registry.config.engine ?? createJavaScriptRegexEngine()
+		engine: engineOf(registry.config)
 	});
 	registry.promise = attempt;
 	attempt.catch(() => {
@@ -141,4 +150,4 @@ function shikiThemeId(dark) {
 	return dark ? INKSTONE_DARK_THEME : INKSTONE_LIGHT_THEME;
 }
 //#endregion
-export { CODE_FONT_FAMILY, DEFAULT_HIGHLIGHTER_LANGS, INKSTONE_DARK_THEME, INKSTONE_LIGHT_THEME, configureEditorSurface, configureHighlighter, getEditorSurface, getHighlighter, inkstoneRadius, inkstoneShadow, shikiThemeId, surfacedTheme };
+export { CODE_FONT_FAMILY, DEFAULT_HIGHLIGHTER_LANGS, INKSTONE_DARK_THEME, INKSTONE_LIGHT_THEME, configureEditorSurface, configureHighlighter, getEditorSurface, getHighlighter, inkstoneRadius, inkstoneShadow, onigurumaEngine, shikiThemeId, surfacedTheme };

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@coldsmirk/inkstone-core",
-  "version": "0.20.1",
-  "description": "Shared Shiki highlighter service for the inkstone editor toolkit: one fine-grained singleton (catppuccin themes, JS regex engine) reused by Monaco, static rendering, and markdown surfaces.",
+  "version": "0.22.0",
+  "description": "Shared Shiki highlighter service for the inkstone editor toolkit: one fine-grained singleton (catppuccin themes, the JS regex engine by default or Oniguruma on request) reused by Monaco, static rendering, and markdown surfaces.",
   "keywords": [
     "shiki",
     "syntax-highlighting",