@coldsmirk/inkstone-core 0.29.0 → 0.30.0

package/README.md

@@ -10,6 +10,7 @@ Part of [inkstone](https://github.com/coldsmirk/inkstone). The editor layers ([`
 - **`configureHighlighter(config)`** — extend or replace the default grammar / theme set, and choose the regex engine, once, before any editor mounts. `DEFAULT_HIGHLIGHTER_LANGS` is exported so extension is additive. `engine` takes the engine itself, a promise of one, or a factory called once when the highlighter is first built — configuration stays synchronous while the engine may be async to make: the strict JavaScript engine by default, `createJavaScriptRegexEngine({ forgiving: true })` for grammars the strict one refuses, or `onigurumaEngine`.
 - **`onigurumaEngine`** — the Oniguruma engine on its WASM, inlined into a chunk of your own bundle (about 600 KB, nothing fetched from anywhere else) and instantiated once, in about 20 ms. Which engine: the JavaScript one translates each grammar's patterns into JavaScript at the first tokenization, on the main thread, and pays for it on whole files — a TypeScript file's first sixty lines cost it about 170 ms, against Oniguruma's 20. Colouring fenced snippets, keep the default; reading whole files in an editor (`@coldsmirk/inkstone-monaco`'s Shiki bridge, `<MonacoEditor>`), pass `onigurumaEngine` as the factory. `onigurumaEngine(wasm)` takes the WASM from elsewhere in any form Shiki's `createOnigurumaEngine` accepts, such as a `fetch("/onig.wasm")`.
 - **`shikiThemeId(dark)`** plus the `INKSTONE_LIGHT_THEME` / `INKSTONE_DARK_THEME` constants — the theme names to pass wherever a Shiki theme id is expected.
+- **`highlighterSources(highlighter)`** — what the shared highlighter was built from and has loaded since: every grammar registration in the order it was given them (the configured `langs` and every later `loadLanguage` / `loadLanguageSync`, which the highlighter wraps to keep them) and the kind of its engine — `oniguruma` wherever an engine was configured, `javascript` where none was. `@coldsmirk/inkstone-monaco`'s tokenization worker builds a highlighter of the same grammars from it; undefined for a highlighter inkstone did not build.
 - **`configureEditorColors(config)`** — set every color the editors paint that is not syntax (a VS Code theme's `colors`, beside its `tokenColors`) in your design language, once, before any editor mounts. Per scheme, CSS color literals for the editor (`background` — content, gutter and the fields of its chrome — `lineHighlight`, `lineNumber`, `lineNumberActive`, `cursor`, `selection`, `scrollbar`), its floating chrome — completion, hover, parameter hints, the find widget and search panels, the context menu — (`widget`, `widgetBorder`, `widgetSelection`, `widgetForeground`, `widgetForegroundMuted`), the `accent` that rings a focused field and marks what a completion matched, and the diagnostic severities `error`, `warning`, `info` and `hint`, named as CodeMirror's `Diagnostic.severity` names them. The catppuccin pair keeps its syntax palette — token colors, the editor's default text, the completion-kind icons that mirror them — and gets these baked into its theme colors — Monaco, static rendering, and CodeMirror's chrome all read the same values — so an editor sits inside the app instead of bringing a second dark. A role or scheme left out keeps the flavor's own color, a later call replaces the schemes it names, and values must be hex colors (`#rrggbb`, `#rrggbbaa`) — Monaco parses theme colors as hex and paints red for anything else, so a `var()` reference or an `rgb(…)` is refused up front. Keep `lineHighlight` translucent: CodeMirror draws selections behind the content, so an opaque active line would hide them. `selection` shows at half its alpha while the editor is unfocused, in both engines; `scrollbar` has no hover state (CodeMirror's `scrollbar-color` has none, so Monaco's slider keeps it too). `getEditorColors(dark)` reads what was configured, and `withEditorColors(theme, colors)` applies a set of them to a Shiki theme of your own.
 - **`CODE_FONT_FAMILY`** — the shared code font stack both editor engines render with: the page's `--inkstone-font-family-code` CSS token when defined, else the platform's native monospace.
 - **`inkstoneRadius(size, fallback)` / `inkstoneShadow(size, fallback)`** — `var()` references to the host's `--inkstone-radius-{small,medium,large,xlarge}` and `--inkstone-shadow-{medium,large,xlarge}` tokens. Set them once on your root and both engines' chrome follows: the Monaco host maps them onto VS Code's `--vscode-cornerRadius-*` / `--vscode-shadow-*` design tokens (which the standalone editor never defines), CodeMirror's polish reads them directly; unset, each engine keeps the look it ships with.

package/dist/index.d.ts

@@ -1,4 +1,4 @@
-import { Awaitable, HighlighterCore, HighlighterCore as HighlighterCore$1, LanguageInput, LoadWasmOptions, RegexEngine, ThemeInput, ThemeRegistration } from "shiki/core";
+import { Awaitable, HighlighterCore, HighlighterCore as HighlighterCore$1, LanguageInput, LanguageRegistration, LoadWasmOptions, RegexEngine, ThemeInput, ThemeRegistration } from "shiki/core";
 //#region src/code-font.d.ts
 /**
  * The code font stack every inkstone editor host renders with: the page's
@@ -197,6 +197,34 @@ interface HighlighterConfig {
  * decode it out of a script.
  */
 declare function onigurumaEngine(wasm?: LoadWasmOptions): Promise<RegexEngine>;
+/**
+ * The regex engine a highlighter built elsewhere from the shared one's grammars tokenizes
+ * with, to tokenize as the shared one does: `javascript`, the strict JavaScript engine, where
+ * the app configured none; `oniguruma` wherever it configured one. An engine is an object of
+ * the page's that cannot be sent anywhere, and Oniguruma is the engine every grammar was
+ * written against — the one {@link onigurumaEngine} is, and the one an engine configured for
+ * the grammars the strict engine refuses stands in for.
+ */
+type HighlighterEngineKind = "javascript" | "oniguruma";
+/**
+ * What the shared highlighter was built from, for a highlighter of the same grammars built
+ * elsewhere — Monaco's tokenization worker (`@coldsmirk/inkstone-monaco`) is one: every grammar
+ * registration it was given, in the order it was given them (shiki builds a grammar again when
+ * a language it embeds lazily arrives after it, so the order is part of what was built), and
+ * the kind of its engine.
+ */
+interface HighlighterSources {
+  readonly engine: HighlighterEngineKind;
+  /**
+   * Grows as the app loads grammars: read it when a copy is due, never once for good.
+   */
+  readonly languages: readonly LanguageRegistration[];
+}
+/**
+ * What the shared highlighter was built from and has loaded since (see
+ * {@link HighlighterSources}); undefined for a highlighter inkstone did not build.
+ */
+declare function highlighterSources(highlighter: HighlighterCore$1): HighlighterSources | undefined;
 /**
  * Configure the shared highlighter before its first use. The highlighter is a process-wide
  * singleton (editors, static rendering, and markdown surfaces all share it for consistent
@@ -234,6 +262,9 @@ declare function getEditorColors(dark: boolean): EditorColors | undefined;
  * A failed build is not cached: the slot resets so a later call retries (and
  * {@link configureHighlighter} becomes callable again), instead of replaying one transient
  * failure — a chunk 404 after a redeploy, a flaky network — for the rest of the session.
+ *
+ * The highlighter keeps what it was built from and every grammar loaded into it since, for a
+ * highlighter of the same grammars built elsewhere ({@link highlighterSources}).
  */
 declare function getHighlighter(): Promise<HighlighterCore$1>;
 /**
@@ -242,4 +273,4 @@ declare function getHighlighter(): Promise<HighlighterCore$1>;
  */
 declare function shikiThemeId(dark: boolean): string;
 //#endregion
-export { CODE_FONT_FAMILY, DEFAULT_HIGHLIGHTER_LANGS, type EditorColors, type EditorColorsConfig, type HighlighterConfig, type HighlighterCore, INKSTONE_DARK_THEME, INKSTONE_LIGHT_THEME, type RadiusSize, type ShadowSize, configureEditorColors, configureHighlighter, getEditorColors, getHighlighter, inkstoneRadius, inkstoneShadow, onigurumaEngine, shikiThemeId, withEditorColors };
+export { CODE_FONT_FAMILY, DEFAULT_HIGHLIGHTER_LANGS, type EditorColors, type EditorColorsConfig, type HighlighterConfig, type HighlighterCore, type HighlighterEngineKind, type HighlighterSources, INKSTONE_DARK_THEME, INKSTONE_LIGHT_THEME, type RadiusSize, type ShadowSize, configureEditorColors, configureHighlighter, getEditorColors, getHighlighter, highlighterSources, inkstoneRadius, inkstoneShadow, onigurumaEngine, shikiThemeId, withEditorColors };

package/dist/index.js

@@ -4,7 +4,7 @@ import langSql from "@shikijs/langs/sql";
 import langTypescript from "@shikijs/langs/typescript";
 import catppuccinLatte from "@shikijs/themes/catppuccin-latte";
 import catppuccinMacchiato from "@shikijs/themes/catppuccin-macchiato";
-import { createHighlighterCore } from "shiki/core";
+import { createHighlighterCore, isSpecialLang, normalizeGetter } from "shiki/core";
 import { createJavaScriptRegexEngine } from "shiki/engine/javascript";
 import { createOnigurumaEngine } from "shiki/engine/oniguruma";
 //#region src/code-font.ts
@@ -163,13 +163,47 @@ function getHighlighterRegistry() {
 		registry = {
 			config: {},
 			surfaces: {},
-			promise: null
+			promise: null,
+			sources: /* @__PURE__ */ new WeakMap()
 		};
 		host[HIGHLIGHTER_REGISTRY_KEY] = registry;
 	}
 	registry.surfaces ??= {};
+	registry.sources ??= /* @__PURE__ */ new WeakMap();
 	return registry;
 }
+async function resolveLanguages(langs) {
+	const loaded = await Promise.all(langs.filter((lang) => !isSpecialLang(lang)).map(async (lang) => {
+		const registrations = await normalizeGetter(lang);
+		return Array.isArray(registrations) ? registrations : [registrations];
+	}));
+	return [...new Set(loaded.flat())];
+}
+function keepingSources(highlighter, languages, engine, registry) {
+	const kept = [...languages];
+	const keep = (registrations) => {
+		for (const registration of registrations) if (!kept.includes(registration)) kept.push(registration);
+	};
+	const { loadLanguage, loadLanguageSync } = highlighter;
+	highlighter.loadLanguage = async (...langs) => {
+		const registrations = await resolveLanguages(langs);
+		await loadLanguage(...registrations);
+		keep(registrations);
+	};
+	highlighter.loadLanguageSync = (...langs) => {
+		const registrations = [...new Set(langs.flat())];
+		loadLanguageSync(...registrations);
+		keep(registrations);
+	};
+	registry.sources.set(highlighter, {
+		engine,
+		languages: kept
+	});
+	return highlighter;
+}
+function highlighterSources(highlighter) {
+	return getHighlighterRegistry().sources.get(highlighter);
+}
 function assertConfigurable(registry, caller) {
 	if (registry.promise) throw new Error(`${caller} must be called before the first getHighlighter()`);
 }
@@ -202,15 +236,19 @@ function getHighlighter() {
 	const registry = getHighlighterRegistry();
 	const existing = registry.promise;
 	if (existing) return existing;
-	const attempt = createHighlighterCore({
-		themes: [
-			withEditorColors(catppuccinLatte, registry.surfaces.light),
-			withEditorColors(catppuccinMacchiato, registry.surfaces.dark),
-			...registry.config.themes ?? []
-		],
-		langs: registry.config.langs ?? [...DEFAULT_HIGHLIGHTER_LANGS],
-		engine: engineOf(registry.config)
-	});
+	const { config, surfaces } = registry;
+	const attempt = (async () => {
+		const langs = await resolveLanguages(config.langs ?? DEFAULT_HIGHLIGHTER_LANGS);
+		return keepingSources(await createHighlighterCore({
+			themes: [
+				withEditorColors(catppuccinLatte, surfaces.light),
+				withEditorColors(catppuccinMacchiato, surfaces.dark),
+				...config.themes ?? []
+			],
+			langs,
+			engine: engineOf(config)
+		}), langs, config.engine === void 0 ? "javascript" : "oniguruma", registry);
+	})();
 	registry.promise = attempt;
 	attempt.catch(() => {
 		if (registry.promise === attempt) registry.promise = null;
@@ -221,4 +259,4 @@ function shikiThemeId(dark) {
 	return dark ? INKSTONE_DARK_THEME : INKSTONE_LIGHT_THEME;
 }
 //#endregion
-export { CODE_FONT_FAMILY, DEFAULT_HIGHLIGHTER_LANGS, INKSTONE_DARK_THEME, INKSTONE_LIGHT_THEME, configureEditorColors, configureHighlighter, getEditorColors, getHighlighter, inkstoneRadius, inkstoneShadow, onigurumaEngine, shikiThemeId, withEditorColors };
+export { CODE_FONT_FAMILY, DEFAULT_HIGHLIGHTER_LANGS, INKSTONE_DARK_THEME, INKSTONE_LIGHT_THEME, configureEditorColors, configureHighlighter, getEditorColors, getHighlighter, highlighterSources, inkstoneRadius, inkstoneShadow, onigurumaEngine, shikiThemeId, withEditorColors };

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@coldsmirk/inkstone-core",
-  "version": "0.29.0",
+  "version": "0.30.0",
   "description": "Shared Shiki highlighter service for the inkstone editor toolkit: one fine-grained singleton (catppuccin themes, the JS regex engine by default or Oniguruma on request) reused by Monaco, static rendering, and markdown surfaces.",
   "keywords": [
     "shiki",