@nvl/sveltex-math-language-server 0.2.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 N. V. Lang
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,111 @@
+# `@nvl/sveltex-math-language-server`
+
+> [!WARNING]
+> **This package is in alpha.** It is brand new and under active development.
+> Its API, behaviour, and configuration may change at any time, and breaking
+> changes should be expected before version `1.0.0`.
+
+A small, standalone Language Server Protocol (LSP) implementation that provides
+editor assistance for **TeX math** — the math written inside the `$…$` /
+`$$…$$` / `\(…\)` / `\[…\]` regions of a [SvelTeX](https://sveltex.dev)
+document.
+
+It is spawned by [`@nvl/sveltex-language-server`](../sveltex-language-server),
+which feeds it each math region of a `.sveltex` file as its own tiny virtual
+TeX document, but it is an ordinary LSP server and can be launched directly by
+any editor over stdio.
+
+## Features
+
+- **Command completion** — triggered on `\`. As you type `\fra…` the server
+  offers `\frac`, `\frak`, … Inside `\begin{…}` / `\end{…}` it offers
+  environment names instead.
+- **Hover** — hovering a command shows its signature when it takes arguments
+  (`\sqrt[index]{radicand}`), the Unicode glyph it stands for when it is a
+  symbol (`∮ (contour integral)`), a one-line description, and the package and
+  backend that provide it.
+
+## Two backends, two command sets
+
+The server runs in one of two modes, chosen by the LSP `initialize` request's
+`initializationOptions`:
+
+```jsonc
+{ "backend": "mathjax" } // or "katex"
+```
+
+This matters because **KaTeX and MathJax support different sets of commands**.
+A command list that mixed them would suggest commands that silently fail to
+render. So the server ships one accurate list per backend and offers only the
+commands the selected backend actually understands.
+
+If `initializationOptions.backend` is absent or unrecognised the server falls
+back to `mathjax` (SvelTeX's default math backend).
+
+## How the command lists are sourced
+
+The lists are **not** transcribed from prose documentation — that drifts. They
+are extracted directly from each backend's own package source by
+[`scripts/generate-commands.ts`](./scripts/generate-commands.ts), which writes
+[`src/data/commands.generated.ts`](./src/data/commands.generated.ts):
+
+- **KaTeX** declares its commands in four side-effect-populated tables —
+  `functions`, `symbols`, `macros` and `environments` (the default exports of
+  the matching files under `katex/src/`). KaTeX supports a finite, enumerable
+  set; those four tables _are_ that set.
+- **MathJax** registers its TeX macros/symbols/environments through a global
+  `MapHandler`. The generator patches `MapHandler.register`, imports the
+  packages the default `input/tex` configuration loads — plus the ones its
+  `autoload` extension pulls in on demand (`mhchem`, `cancel`, `braket`, …) —
+  and reads the registered token keys back out. Packages that need an explicit
+  `\require{}` (`physics`, `mathtools`, …) are intentionally excluded: they are
+  not available out of the box, so offering them would be a false promise.
+
+On top of the bare command set, the generator enriches each command with
+documentation metadata — a usage signature, the providing package and a
+one-line description — curated from each engine's reference docs and kept in
+`scripts/data/{katex,mathjax}-docs.json`. Symbol commands additionally carry
+the Unicode glyph they render (read from the engines' own symbol tables) and
+that glyph's Unicode standard name (looked up in the Unicode Character
+Database). All of it is baked into the generated file, so hover stays a pure,
+offline lookup.
+
+`katex` and `@mathjax/src` are **devDependencies only** — the published package
+ships the generated static data and has no runtime dependency on either. Run
+
+```sh
+pnpm --filter @nvl/sveltex-math-language-server generate
+```
+
+to regenerate the lists after a `katex` / `@mathjax/src` version bump.
+
+## Architecture
+
+```
+src/data/commands.generated.ts   ← generated, per-backend command tables
+src/core/commands.ts             ← CommandTable: indexed prefix/exact lookup
+src/core/context.ts              ← TeX caret-context lexing (\cmd, \begin{…})
+src/core/describe.ts             ← human-readable command descriptions
+src/core/features.ts             ← pure completion + hover (LSP payloads)
+src/core/server.ts               ← createServer(connection): the orchestrator
+src/index.ts                     ← startServer() (stdio convenience wrapper)
+bin/server.js                    ← #!/usr/bin/env node → startServer()
+```
+
+`createServer(connection)` is **transport-agnostic** — it never imports
+`vscode` and never touches `process.stdin`. The same core therefore backs the
+stdio `bin/server.js` and any in-process host, exactly like
+`@nvl/sveltex-language-server`.
+
+## Development
+
+```sh
+pnpm --filter @nvl/sveltex-math-language-server generate   # refresh command data
+pnpm --filter @nvl/sveltex-math-language-server build      # type-check + emit dist/
+pnpm --filter @nvl/sveltex-math-language-server test       # run unit/integration tests
+pnpm --filter @nvl/sveltex-math-language-server lint       # eslint + tsc --noEmit
+```
+
+The test suite covers the per-backend command tables, the caret-context lexer,
+completion filtering and hover, and an end-to-end check that spawns
+`bin/server.js` and drives it over stdio.

package/bin/server.js

@@ -0,0 +1,11 @@
+#!/usr/bin/env node
+// Executable entry point for the SvelTeX math language server.
+//
+// This file is intentionally tiny and dependency-free: editors and host
+// servers (notably `@nvl/sveltex-language-server`, which spawns one of these
+// per math backend) launch it directly. All real work lives in the compiled
+// core under `../dist`.
+
+import { startServer } from '../dist/index.js';
+
+startServer();

package/dist/core/commands.d.ts

@@ -0,0 +1,104 @@
+/**
+ * The category a TeX command falls into.
+ *
+ * - `function`: a command that consumes arguments (`\frac`, `\sqrt`, `\text`).
+ * - `symbol`: a no-argument command standing for a glyph (`\alpha`, `\sum`).
+ * - `macro`: a command defined in terms of others (`\TeX`, `\operatorname`).
+ * - `environment`: a name used with `\begin{...}` / `\end{...}` (`align`,
+ *   `matrix`). Completing `\begin{` offers these.
+ */
+export type CommandCategory = 'function' | 'symbol' | 'macro' | 'environment';
+/**
+ * One TeX command supported by a backend.
+ *
+ * @remarks
+ * `name` never includes the leading backslash — it is the bare control word
+ * (e.g. `frac`). The backslash is added by the completion/hover layer so the
+ * data is uniform regardless of how the user typed the trigger.
+ *
+ * Only `name` and `category` are always present: they come from the backend's
+ * own package source and so are exhaustive and exact. The remaining fields are
+ * documentation metadata, merged in from each engine's reference docs by
+ * `scripts/generate-commands.ts`; any of them may be absent for a given
+ * command, and the hover/completion layer degrades gracefully when they are.
+ */
+export interface MathCommand {
+    /** The command name without its leading backslash (e.g. `frac`). */
+    name: string;
+    /** Which {@link CommandCategory} the command belongs to. */
+    category: CommandCategory;
+    /**
+     * A usage signature spelling out the command's arguments, e.g.
+     * `\sqrt[degree]{radicand}`. Present only for commands that take
+     * arguments — a no-argument command (`\alpha`, `\sin`) has none.
+     */
+    signature?: string;
+    /**
+     * The single Unicode character the command stands for, e.g. `α` for
+     * `\alpha` or `∮` for `\oint`. Present only for commands that denote one
+     * Unicode glyph.
+     */
+    unicode?: string;
+    /**
+     * The Unicode standard name of {@link MathCommand.unicode}, lower-cased
+     * (e.g. `greek small letter alpha`, `contour integral`). Present only
+     * alongside `unicode`, and only when the name could be resolved.
+     */
+    unicodeName?: string;
+    /**
+     * The backend package/extension the command belongs to — a MathJax
+     * extension name (`base`, `ams`, `physics`, …) or, for KaTeX, the section
+     * of its support table the command is documented under.
+     */
+    package?: string;
+    /** A one-line, human-readable description of what the command does. */
+    description?: string;
+}
+/** The math backends this server can emulate. */
+export type MathLspBackend = 'katex' | 'mathjax';
+/**
+ * An indexed, queryable view of one backend's command set.
+ *
+ * Built once per backend by {@link createCommandTable} and cached, so repeated
+ * completion requests do no repeated work.
+ */
+export declare class CommandTable {
+    #private;
+    private constructor();
+    /**
+     * Builds a {@link CommandTable} from a raw command list.
+     *
+     * @param commands - The backend's commands (typically one of the generated
+     * arrays). Order does not matter; the table sorts internally.
+     */
+    static create(commands: readonly MathCommand[]): CommandTable;
+    /** The total number of commands in the table. */
+    get size(): number;
+    /** Every command in the table, sorted by name. */
+    get all(): readonly MathCommand[];
+    /**
+     * Looks a command up by its exact (backslash-free) name.
+     *
+     * @param name - The bare command name, e.g. `frac`.
+     * @returns The command, or `undefined` if the backend does not support it.
+     */
+    get(name: string): MathCommand | undefined;
+    /**
+     * Returns the commands whose name starts with `prefix`.
+     *
+     * @param prefix - A bare (backslash-free) prefix. An empty prefix matches
+     * every command.
+     * @param environmentsOnly - When `true`, only `environment` commands are
+     * considered (for `\begin{...}` completion); when `false`, environments are
+     * excluded (for ordinary `\command` completion).
+     * @returns The matching commands, sorted by name.
+     */
+    withPrefix(prefix: string, environmentsOnly: boolean): readonly MathCommand[];
+}
+/**
+ * Returns the {@link CommandTable} for a backend, building it on first use.
+ *
+ * @param backend - `'katex'` or `'mathjax'`.
+ * @returns The backend's command table.
+ */
+export declare function createCommandTable(backend: MathLspBackend): CommandTable;

package/dist/core/commands.js

@@ -0,0 +1,94 @@
+// File description: The TeX-command model plus the per-backend lookup API.
+//
+// `@nvl/sveltex-math-language-server` answers completion and hover requests for
+// TeX math written in SvelTeX `$...$` / `$$...$$` regions. The set of commands
+// it offers must be EXACTLY what the selected backend understands — KaTeX and
+// MathJax support overlapping but distinct sets. The raw lists are extracted
+// from each backend's own package source by `scripts/generate-commands.ts`
+// (see `../data/commands.generated.ts`); this module turns those flat lists
+// into an indexed `CommandTable` for fast prefix and exact lookup.
+import { KATEX_COMMANDS, MATHJAX_COMMANDS, } from '../data/commands.generated.js';
+/**
+ * An indexed, queryable view of one backend's command set.
+ *
+ * Built once per backend by {@link createCommandTable} and cached, so repeated
+ * completion requests do no repeated work.
+ */
+export class CommandTable {
+    /** All commands, sorted by name — the basis for completion lists. */
+    #all;
+    /** Exact-name index, for hover. */
+    #byName;
+    /** Only the `environment` commands, for `\begin{...}` completion. */
+    #environments;
+    /** Every command that is NOT an environment, for `\command` completion. */
+    #nonEnvironments;
+    constructor(commands) {
+        const sorted = [...commands].sort((a, b) => a.name.localeCompare(b.name));
+        this.#all = sorted;
+        this.#byName = new Map(sorted.map((c) => [c.name, c]));
+        this.#environments = sorted.filter((c) => c.category === 'environment');
+        this.#nonEnvironments = sorted.filter((c) => c.category !== 'environment');
+    }
+    /**
+     * Builds a {@link CommandTable} from a raw command list.
+     *
+     * @param commands - The backend's commands (typically one of the generated
+     * arrays). Order does not matter; the table sorts internally.
+     */
+    static create(commands) {
+        return new CommandTable(commands);
+    }
+    /** The total number of commands in the table. */
+    get size() {
+        return this.#all.length;
+    }
+    /** Every command in the table, sorted by name. */
+    get all() {
+        return this.#all;
+    }
+    /**
+     * Looks a command up by its exact (backslash-free) name.
+     *
+     * @param name - The bare command name, e.g. `frac`.
+     * @returns The command, or `undefined` if the backend does not support it.
+     */
+    get(name) {
+        return this.#byName.get(name);
+    }
+    /**
+     * Returns the commands whose name starts with `prefix`.
+     *
+     * @param prefix - A bare (backslash-free) prefix. An empty prefix matches
+     * every command.
+     * @param environmentsOnly - When `true`, only `environment` commands are
+     * considered (for `\begin{...}` completion); when `false`, environments are
+     * excluded (for ordinary `\command` completion).
+     * @returns The matching commands, sorted by name.
+     */
+    withPrefix(prefix, environmentsOnly) {
+        const pool = environmentsOnly
+            ? this.#environments
+            : this.#nonEnvironments;
+        if (prefix.length === 0)
+            return pool;
+        // Case-sensitive: TeX commands are case-sensitive (`\Pi` ≠ `\pi`).
+        return pool.filter((c) => c.name.startsWith(prefix));
+    }
+}
+/** Lazily-built, cached {@link CommandTable}s keyed by backend. */
+const tableCache = new Map();
+/**
+ * Returns the {@link CommandTable} for a backend, building it on first use.
+ *
+ * @param backend - `'katex'` or `'mathjax'`.
+ * @returns The backend's command table.
+ */
+export function createCommandTable(backend) {
+    const cached = tableCache.get(backend);
+    if (cached)
+        return cached;
+    const table = CommandTable.create(backend === 'katex' ? KATEX_COMMANDS : MATHJAX_COMMANDS);
+    tableCache.set(backend, table);
+    return table;
+}

package/dist/core/context.d.ts

@@ -0,0 +1,66 @@
+/**
+ * The command-typing context at a caret, as needed for completion.
+ */
+export interface CompletionContext {
+    /**
+     * The command prefix already typed after the backslash, WITHOUT the
+     * backslash itself (e.g. for `\fra|` the prefix is `fra`). Empty when the
+     * caret sits immediately after a lone backslash.
+     */
+    prefix: string;
+    /**
+     * Offset of the backslash that opens the command being typed. The
+     * completion's replace range runs from here to the caret.
+     */
+    backslashOffset: number;
+    /**
+     * `true` when the command is the environment-name argument of a `\begin{`
+     * or `\end{` — i.e. the caret is inside the braces of `\begin{...}`. In
+     * that case there is no backslash; completion offers environment names.
+     */
+    isEnvironmentName: boolean;
+}
+/**
+ * The command found under (or immediately after) a caret, as needed for hover.
+ */
+export interface CommandAtCaret {
+    /** The command name without its leading backslash (e.g. `frac`). */
+    name: string;
+    /** Offset of the opening backslash. */
+    start: number;
+    /** Offset one past the last character of the command. */
+    end: number;
+}
+/**
+ * Analyses the command-typing context at `offset` within `text`.
+ *
+ * Two shapes are recognised:
+ *
+ *  1. **Ordinary command** — the caret follows `\` and zero or more letters
+ *     (`\`, `\al`, `\alpha`). The nearest preceding backslash that is not
+ *     itself escaped opens the command.
+ *  2. **Environment name** — the caret is inside the braces of a `\begin{...}`
+ *     or `\end{...}`; the partial environment name is returned with
+ *     `isEnvironmentName: true`.
+ *
+ * @param text - The full TeX (math) document text.
+ * @param offset - The caret offset.
+ * @returns The {@link CompletionContext}, or `undefined` if the caret is not in
+ * a position where command completion makes sense.
+ */
+export declare function completionContextAt(text: string, offset: number): CompletionContext | undefined;
+/**
+ * Finds the TeX command that the caret at `offset` sits within or directly
+ * after — used to answer hover.
+ *
+ * A command is a backslash followed by either a run of letters (`\alpha`) or a
+ * single non-letter character (`\,`, `\#`). The caret matches if it lies
+ * anywhere from the backslash up to and including the position just past the
+ * command's last character.
+ *
+ * @param text - The full TeX (math) document text.
+ * @param offset - The caret offset.
+ * @returns The {@link CommandAtCaret}, or `undefined` if the caret is not on a
+ * command.
+ */
+export declare function commandAtCaret(text: string, offset: number): CommandAtCaret | undefined;

package/dist/core/context.js

@@ -0,0 +1,152 @@
+// File description: TeX-aware caret-context analysis.
+//
+// Completion and hover both need to know what TeX construct the caret sits in:
+// the command being typed (for completion) or the command under the cursor
+// (for hover), and whether that command is the environment name slot of a
+// `\begin{...}` / `\end{...}`. This module isolates that small bit of TeX
+// lexing so the completion and hover handlers stay declarative.
+/** Whether `ch` may appear in a multi-letter TeX control word. */
+function isControlWordChar(ch) {
+    return /^[a-zA-Z]$/u.test(ch);
+}
+/**
+ * Analyses the command-typing context at `offset` within `text`.
+ *
+ * Two shapes are recognised:
+ *
+ *  1. **Ordinary command** — the caret follows `\` and zero or more letters
+ *     (`\`, `\al`, `\alpha`). The nearest preceding backslash that is not
+ *     itself escaped opens the command.
+ *  2. **Environment name** — the caret is inside the braces of a `\begin{...}`
+ *     or `\end{...}`; the partial environment name is returned with
+ *     `isEnvironmentName: true`.
+ *
+ * @param text - The full TeX (math) document text.
+ * @param offset - The caret offset.
+ * @returns The {@link CompletionContext}, or `undefined` if the caret is not in
+ * a position where command completion makes sense.
+ */
+export function completionContextAt(text, offset) {
+    if (offset < 0 || offset > text.length)
+        return undefined;
+    // --- environment-name slot: `\begin{<here>` or `\end{<here>` -----------
+    const beforeCaret = text.slice(0, offset);
+    const envMatch = /\\(?:begin|end)\{([a-zA-Z*]*)$/u.exec(beforeCaret);
+    if (envMatch) {
+        return {
+            prefix: envMatch[1] ?? '',
+            backslashOffset: offset - (envMatch[1] ?? '').length,
+            isEnvironmentName: true,
+        };
+    }
+    // --- ordinary command: scan back over letters to a backslash -----------
+    let i = offset;
+    while (i > 0 && isControlWordChar(text.charAt(i - 1))) {
+        i -= 1;
+    }
+    // `i` now points just after the run of letters; the char before must be a
+    // backslash for this to be a command.
+    if (i === 0 || text.charAt(i - 1) !== '\\')
+        return undefined;
+    const backslashOffset = i - 1;
+    // A backslash is only a command opener if it is not itself escaped: count
+    // the unbroken run of backslashes ending here; an even count means the
+    // last one is escaped (`\\`), so it is line-break/literal, not a command.
+    let backslashes = 0;
+    let j = backslashOffset;
+    while (j >= 0 && text.charAt(j) === '\\') {
+        backslashes += 1;
+        j -= 1;
+    }
+    if (backslashes % 2 === 0)
+        return undefined;
+    return {
+        prefix: text.slice(i, offset),
+        backslashOffset,
+        isEnvironmentName: false,
+    };
+}
+/**
+ * Finds the TeX command that the caret at `offset` sits within or directly
+ * after — used to answer hover.
+ *
+ * A command is a backslash followed by either a run of letters (`\alpha`) or a
+ * single non-letter character (`\,`, `\#`). The caret matches if it lies
+ * anywhere from the backslash up to and including the position just past the
+ * command's last character.
+ *
+ * @param text - The full TeX (math) document text.
+ * @param offset - The caret offset.
+ * @returns The {@link CommandAtCaret}, or `undefined` if the caret is not on a
+ * command.
+ */
+export function commandAtCaret(text, offset) {
+    if (offset < 0 || offset > text.length)
+        return undefined;
+    // Find the backslash at or before the caret that could open the command.
+    // Scan left over letters first (the caret may be in the middle of a word),
+    // then expect a backslash.
+    let left = offset;
+    while (left > 0 && isControlWordChar(text.charAt(left - 1))) {
+        left -= 1;
+    }
+    let backslashOffset;
+    if (left > 0 && text.charAt(left - 1) === '\\') {
+        backslashOffset = left - 1;
+    }
+    else if (
+    // Single-character command: caret right after `\` + one non-letter,
+    // e.g. hovering `\,`.
+    offset >= 2 &&
+        text.charAt(offset - 2) === '\\' &&
+        !isControlWordChar(text.charAt(offset - 1))) {
+        backslashOffset = offset - 2;
+    }
+    else if (
+    // Caret sits on the backslash itself or just before a single-char cmd.
+    offset >= 1 &&
+        text.charAt(offset - 1) === '\\') {
+        backslashOffset = offset - 1;
+    }
+    else if (offset < text.length && text.charAt(offset) === '\\') {
+        backslashOffset = offset;
+    }
+    if (backslashOffset === undefined)
+        return undefined;
+    // Reject an escaped backslash (`\\`).
+    let backslashes = 0;
+    let j = backslashOffset;
+    while (j >= 0 && text.charAt(j) === '\\') {
+        backslashes += 1;
+        j -= 1;
+    }
+    if (backslashes % 2 === 0)
+        return undefined;
+    // Read the command name after the backslash.
+    let end = backslashOffset + 1;
+    if (end < text.length && isControlWordChar(text.charAt(end))) {
+        while (end < text.length && isControlWordChar(text.charAt(end))) {
+            end += 1;
+        }
+        // An optional trailing `*` is part of starred command/environment names
+        // (it is not, however, part of a control word the caret is hovering —
+        // keep it simple and exclude it here; starred names are matched by the
+        // environment-name path of completion instead).
+    }
+    else if (end < text.length &&
+        !isControlWordChar(text.charAt(end)) &&
+        !/\s/u.test(text.charAt(end)) &&
+        // A backslash here would make the pair `\\` — an escaped backslash,
+        // not a `\<char>` command — so it is not a single-character command.
+        text.charAt(end) !== '\\') {
+        // Single-character command (`\,`).
+        end += 1;
+    }
+    const name = text.slice(backslashOffset + 1, end);
+    if (name.length === 0)
+        return undefined;
+    // The caret must actually be within `[backslashOffset, end]`.
+    if (offset < backslashOffset || offset > end)
+        return undefined;
+    return { name, start: backslashOffset, end };
+}

package/dist/core/describe.d.ts

@@ -0,0 +1,24 @@
+import type { MathCommand, MathLspBackend } from './commands.js';
+/**
+ * Returns the one-line description of a command: the documentation-sourced
+ * description when the generator merged one in, else a curated gloss, else the
+ * generic category sentence (which is always available).
+ *
+ * @param command - The command to describe.
+ */
+export declare function describeCommand(command: MathCommand): string;
+/**
+ * Builds the Markdown body shown on hover for a command.
+ *
+ * The body is, in order: a fenced `latex` block with the command's signature
+ * (its arguments spelled out when it takes any); for a command that stands for
+ * a Unicode glyph, that glyph and its Unicode standard name; the one-line
+ * description; and a dimmed footer naming the backend, the providing package
+ * (when known) and the category.
+ *
+ * @param command - The command being hovered.
+ * @param backend - The active backend, named in the footer so the user knows
+ * which engine's support they are seeing.
+ * @returns A Markdown string.
+ */
+export declare function hoverMarkdown(command: MathCommand, backend: MathLspBackend): string;