@forming/tools 0.0.0-stage → 0.1.0-dev.2.1

package/LICENSE

@@ -0,0 +1,86 @@
+Forming Source Available License 1.0
+
+Copyright (c) 2026 Forming contributors.
+
+1. Scope and definitions
+
+This license covers the Forming implementation, tools, examples and other
+original material supplied with this notice (the Software). Each copyright
+holder grants rights only to material they own or are authorized to license.
+Third-party components and material expressly offered under another license
+remain governed by those terms. See NOTICE.md and the attribution notices.
+
+An Internal Use is use by your employees or contractors solely for your own
+organization's operations. Customers and other external users are not internal
+users. A Commercial Product is a product or service sold, licensed for a fee,
+monetized through advertising, or offered to support a paid business offering,
+including its free tiers, trials and bundled features.
+
+2. Permission
+
+Subject to the conditions below, you may use, inspect, copy and modify the
+Software for personal, educational, noncommercial and Internal Use. Internal
+Use by a commercial business is permitted without payment.
+
+You may also use the Software on the backend of a Commercial Product to compile
+definitions, validate data or process results, provided that you do not offer
+the Software's capabilities as a service or use the Software in that product's
+external user interface as described in section 3.
+
+You may redistribute unmodified or modified copies without charge for uses
+permitted by this license. You must include this license and all applicable
+copyright, attribution and third-party notices. Identify changes you make.
+Recipients receive their rights from the applicable copyright holders under
+these same terms; you may not grant broader rights to the Software.
+
+3. Uses requiring separate written permission
+
+Without a separate written agreement from the applicable copyright holders,
+you may not, directly or through another party:
+
+(a) sell, rent, sublicense for a fee, or otherwise charge for copies of or
+access to the Software, including modified, rebranded or bundled versions;
+
+(b) offer the Software or its form authoring, rendering, compilation, validation
+or runtime capabilities to external users as a hosted, managed, API or other
+commercial service, including through a wrapper or a free tier of that service;
+
+(c) embed, distribute or otherwise use the Software to provide any part of a
+Commercial Product's external user interface, including ordinary rendered
+forms, widgets, editors, visual builders and white-label interfaces.
+
+These restrictions apply even if the Software is only one part of an offering
+or is not separately priced. They do not prohibit Internal Use or the backend
+integration expressly allowed in section 2. Charging for consulting, installation
+or support is allowed when the resulting use itself complies with this license.
+
+4. Your material and other rights
+
+You retain ownership of your independently authored form definitions, submitted
+data and other original output. This license does not require you to disclose
+your application's source code. Output that contains Software code remains
+subject to this license to the extent it contains that code. No trademark rights
+are granted. Rights not expressly granted are reserved.
+
+5. Termination
+
+Your permissions end if you violate this license. If you correct your first
+violation within 30 days of receiving written notice from a copyright holder,
+your permissions are restored. Further violations require express written
+reinstatement. Termination does not remove obligations that apply to copies
+already distributed.
+
+6. Warranty and liability
+
+TO THE EXTENT PERMITTED BY LAW, THE SOFTWARE IS PROVIDED "AS IS", WITHOUT
+WARRANTIES, INCLUDING MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. COPYRIGHT HOLDERS AND CONTRIBUTORS ARE NOT LIABLE FOR CLAIMS,
+DAMAGES OR OTHER LIABILITY ARISING FROM THE SOFTWARE OR ITS USE, WHETHER IN
+CONTRACT, TORT OR OTHERWISE.
+
+7. Additional permission
+
+Contact the project maintainers through the repository from which you obtained
+the Software to request a commercial agreement. An unanswered request does not
+grant permission. A separate written agreement controls only to the extent it
+expressly grants different rights.

package/NOTICE.md

@@ -0,0 +1,18 @@
+# Licensing notices
+
+The Forming implementation is covered by the [Forming Source Available License](LICENSE).
+This license permits internal business use and commercial backend integration,
+but requires separate written permission for resale, commercial hosted Forming
+services and external UI use in commercial products. It is not an open-source
+license. The license text controls over this summary.
+
+The specifications and adapted specification text retain their existing
+[CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) terms and attribution.
+See `core/ATTRIBUTION.md` and `website/ATTRIBUTION.md` in the source repository.
+The implementation license does not replace those rights or revoke rights
+granted under earlier licenses.
+
+Dependencies retain their own licenses. Rust dependency notices are collected
+under `tools/skills/notices/` in the source repository and included with packaged
+WASM distributions. JavaScript and Go dependency notices are supplied by their
+respective packages. Preserve applicable notices when redistributing bundles.

package/README.md

@@ -1,3 +1,78 @@
-# Temporary Holding Version
+# Forming tools
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+JavaScript tooling for writing Forming: autocomplete, editor support and the language server. The lint rules, `forming check` and `forming fmt` live in the [Rust core](../core/spec/lint.md), so every tool reports the same errors and warnings.
+
+| Folder | Package | What it is |
+| --- | --- | --- |
+| `.` (this folder) | `@forming/tools` | Completion engine and tokenizer shared by the playground and editors |
+| [`language-server/`](language-server) | `@forming/language-server` | Errors, lint warnings, autocomplete and formatting over the Language Server Protocol: VS Code, Neovim, Zed, Helix and other LSP clients |
+| [`vscode/`](vscode) | VS Code extension | `.forming` files, syntax highlighting, comments and indentation, and the language server |
+| [`skills/`](skills) | `@forming/skills` (local package prototype) | Installable Codex/Claude skill with an offline WASM validator and extension references; not yet published |
+
+## @forming/tools
+
+```js
+import {complete, importKinds} from '@forming/tools/completions';
+
+const kinds = importKinds(source, packageManifests); // which imports are uploads, options, annotations...
+const result = complete(source, offset, {kinds});   // {from, to, options: [{label, type, apply}]}
+```
+
+- Suggests keywords, types, settings, rules, field paths, pages and functions for the line being edited.
+- With package manifests, imports are offered only where their kind fits: uploads after `upload with`, option lists after `options from`, validators after `validate with`, actions after `invoke`, and annotations beneath fields and fragment uses. Without manifests every import is offered.
+- `formingCompletions(context, {kinds})` is a CodeMirror completion source; `complete(text, offset)` works on plain text for any editor.
+- `@forming/tools/tokens` exports the tokenizer and its word lists (`WORDS`), from which the VS Code grammar is generated.
+
+The website vendors a copy (`npm run sync:runtime` in `website/`) so it builds on its own.
+
+## Editors
+
+Both the server and the extension read package manifests from the nearest `forming.packages.json` beside your forms, so imports resolve and completions know what each import is:
+
+```json
+[{"profile": "forming.packages/1", "name": "hidden", "version": "1.0.0", "exports": {"visibleTo": {"kind": "annotation", "valueSchema": {"type": "array", "items": {"type": "string"}}}}}]
+```
+
+Copy each plugin's `manifest.json` (for example `extensions/hidden/manifest.json`) into that array.
+
+### VS Code
+
+```sh
+cd tools/language-server && npm ci   # needs js-sdk built (npm run build:wasm && npm run build)
+cd ../vscode && npm ci && npm run package   # bundles, then: code --install-extension forming-*.vsix
+```
+
+`npm run package` bundles the extension and server into `dist/` with the core's WebAssembly in `wasm/`, so the `.vsix` carries no `node_modules`. The extension is not yet on the Marketplace.
+
+### Other editors
+
+Run the server with `--stdio`:
+
+```lua
+-- Neovim
+vim.lsp.start({name = 'forming', cmd = {'forming-language-server', '--stdio'}, root_dir = vim.fn.getcwd()})
+```
+
+## Development
+
+```sh
+npm test                            # @forming/tools
+npm --prefix language-server test   # diagnostics, completion, formatting and a real stdio LSP session
+npm --prefix vscode test            # grammar, activation, and the bundled server answering LSP
+npm --prefix vscode run grammar     # regenerate syntaxes/forming.tmLanguage.json
+```
+
+## Automatic npm publishing
+
+Publishes `@forming/tools` from `.`.
+
+`.github/workflows/publish.yml` publishes the JavaScript package on pushes to
+`dev` (base version plus `-dev.<run>.<attempt>`, npm tag `dev`) and `main`
+(stable package.json version, npm tag `latest`). Bump package.json and its lockfile
+before another stable release. Manual runs support only these two branches.
+The workflow creates no commits, Git tags or GitHub releases.
+
+Set Actions variable `NPM_ACCESS` to `public` or `restricted`. Configure secret
+`NPM_TOKEN`, or npm trusted publishing for this repository and workflow
+`publish.yml` after the first publish. Private repositories publish with provenance
+disabled. Packages exclude tests, source directories, build scripts and source maps.

package/dist/completions.mjs

@@ -0,0 +1,419 @@
+import {scriptTokens} from './tokens.mjs';
+
+const NAME = '[A-Za-z_][A-Za-z0-9_]*';
+const FIELD = new RegExp(`^(${NAME})\\s*:\\s*(.+)?$`);
+const TYPES = [
+  'string',
+  'integer',
+  'number',
+  'boolean',
+  'email',
+  'url',
+  'phone',
+  'date',
+  'time',
+  'datetime',
+  'file',
+  'object',
+  'enum(',
+];
+const VIEWS = ['section', 'group', 'stack', 'columns', 'grid', 'repeat', 'display', 'use'];
+const RULES = ['show when ', 'enable when ', 'readonly when '];
+const FUNCTIONS = {
+  has: 'has(path): is an answer present?',
+  default: 'default(value, fallback)',
+  size: 'size(text | list)',
+  startsWith: 'startsWith(text, prefix)',
+  endsWith: 'endsWith(text, suffix)',
+  contains: 'contains(text, part)',
+  sum: 'sum(list, key?)',
+  count: 'count(list, key)',
+  every: 'every(list, key)',
+  some: 'some(list, key)',
+};
+const OPERATORS = ['and', 'or', 'in', '??', '==', '!=', '<', '<=', '>', '>=', '+', '-', '*', '/', '%', '?'];
+const numeric = ['min ', 'max ', 'multipleOf '];
+const strings = ['minLength ', 'maxLength ', 'pattern '];
+const arrays = ['minItems ', 'maxItems ', 'uniqueItems'];
+const constraints = type => [
+  'required',
+  'default ',
+  'id ',
+  ...(type.includes('[]')
+    ? arrays
+    : /^(number|integer)\b/.test(type)
+      ? numeric
+      : /^(string|email|url|phone|date|time|datetime|enum)\b/.test(type)
+        ? strings
+        : []),
+  ...(!type.includes('[]') && !/^(object|file)\b/.test(type) ? ['mustBe '] : []),
+  ...(type.startsWith('phone') ? ['region '] : []),
+  ...(type.startsWith('file') ? ['accept ', 'maxSize ', ...(type.includes('[]') ? ['maxTotalSize '] : [])] : []),
+];
+const controls = type =>
+  type.includes('[]')
+    ? ['multichoice', 'hidden']
+    : /^(string|email|url|phone)\b/.test(type)
+      ? ['textarea', 'password', 'choice', 'hidden']
+      : /^(date|time|datetime)\b/.test(type)
+        ? [type.match(/^\w+/)[0], 'hidden']
+        : ['choice', 'hidden'];
+const choices = (labels, type = 'keyword') =>
+  labels.map(label => (typeof label === 'string' ? {label, type} : {...label, type}));
+const clean = text =>
+  scriptTokens(text)
+    .filter(token => token.type !== 'lineComment')
+    .map(token => token.text)
+    .join('');
+const masked = text =>
+  scriptTokens(text)
+    .map(token =>
+      token.type === 'string' || token.type === 'lineComment' ? ' '.repeat(token.text.length) : token.text
+    )
+    .join('');
+
+// Recover indentation scopes from incomplete source. Blank/comment lines don't
+// close blocks. Keep following declarations too, for forward references and
+// duplicate settings while editing in the middle of a document.
+function documentLines(text) {
+  const stack = [],
+    records = [];
+  let offset = 0;
+  for (const raw of text.split('\n')) {
+    const code = clean(raw).trim();
+    const indent = raw.match(/^ */)[0].length;
+    if (code) {
+      while (stack.length && stack.at(-1).indent >= indent) stack.pop();
+      const field = code.match(FIELD);
+      const record = {
+        code,
+        indent,
+        from: offset,
+        parents: [...stack],
+        field,
+        kind: field ? 'field' : code.split(/\s/)[0],
+      };
+      records.push(record);
+      stack.push(record);
+    }
+    offset += raw.length + 1;
+  }
+  return records;
+}
+
+function symbols(records, parents) {
+  const fragment = parents.find(record => record.kind === 'fragment');
+  const fields = [],
+    inputs = [],
+    computed = [];
+  for (const record of records) {
+    const owner = record.parents.find(parent => parent.kind === 'fragment');
+    if (owner !== fragment) continue;
+    if (record.field) {
+      const prefix = record.parents.flatMap(parent =>
+        parent.field ? [parent.field[1]] : parent.kind === 'repeat' ? [parent.code.split(/\s+/)[1]] : []
+      );
+      fields.push([...prefix, record.field[1]].join('.'));
+    }
+    const input = record.code.match(/^input\s+(\w+)\s*:/);
+    if (input) inputs.push(input[1]);
+    const calc = record.code.match(/^(\w+)\s*=/);
+    if (calc && record.parents.at(-1)?.kind === 'computed') computed.push(calc[1]);
+  }
+  return {fields, inputs, computed};
+}
+
+/**
+ * CodeMirror-compatible completion source. `kinds` maps imported names to
+ * their export kind ("upload", "options", "annotation", ...) when manifests
+ * are known; see importKinds. Without it every import is offered everywhere.
+ */
+export function formingCompletions(context, {kinds} = {}) {
+  const doc = context.state.doc;
+  const line = doc.lineAt(context.pos);
+  const column = context.pos - line.from;
+  const before = line.text.slice(0, column);
+  const tokens = scriptTokens(line.text);
+  // No suggestions in comments or quoted labels, even on explicit Ctrl+Space.
+  if (
+    tokens.some(
+      token =>
+        token.from < column &&
+        (token.type === 'lineComment' ||
+          (token.type === 'string' &&
+            (column < token.to || (column === token.to && !/^"(?:\\.|[^"\\])*"$/.test(token.text)))))
+    )
+  )
+    return null;
+  const word = before.match(/[A-Za-z_][A-Za-z0-9_-]*$/)?.[0] ?? '';
+  const from = context.pos - word.length;
+  const suffix = line.text.slice(column).match(/^[A-Za-z0-9_-]*/)[0];
+  const to = context.pos + suffix.length;
+  const head = before.slice(0, before.length - word.length);
+  const after = line.text.slice(to - line.from);
+  if (!word && !context.explicit && !/[\s:.]$/.test(before)) return null;
+  const done = list => {
+    const unique = [...new Map(list.map(option => [option.label, option])).values()];
+    return unique.length
+      ? {
+          from,
+          to,
+          options: unique.map(option => {
+            let apply = option.apply ?? option.label;
+            if (typeof apply === 'string') {
+              if (/\s$/.test(apply) && /^\s/.test(after)) apply = apply.trimEnd();
+              if (apply.endsWith('(') && /^\s*\(/.test(after)) apply = apply.slice(0, -1);
+            }
+            return {...option, apply};
+          }),
+          validFor: /^[A-Za-z0-9_-]*$/,
+        }
+      : null;
+  };
+  const text = doc.toString();
+  const records = documentLines(text);
+  const indent = line.text.match(/^ */)[0].length;
+  const current = records.find(record => record.from === line.from);
+  const previous = records.filter(record => record.from < line.from).at(-1);
+  const parents = previous ? [...previous.parents, previous].filter(record => record.indent < indent) : [];
+  const parent = parents.at(-1);
+  const names = symbols(records, parents);
+  const imported = records
+    .filter(record => record.kind === 'import')
+    .flatMap(record =>
+      (record.code.match(/\{([^}]*)\}/)?.[1] ?? '')
+        .split(',')
+        .map(name =>
+          name
+            .trim()
+            .split(/\s+as\s+/)
+            .at(-1)
+        )
+        .filter(Boolean)
+    );
+  const kindOf = name => kinds?.get?.(name) ?? kinds?.[name];
+  // Imports of the wanted kinds; a name whose kind is unknown stays on offer.
+  const importedOf = (...wanted) =>
+    imported.filter(name => kindOf(name) === undefined || wanted.includes(kindOf(name)));
+  const annotations = imported.filter(name => kindOf(name) === 'annotation');
+  const declared = kind =>
+    records
+      .filter(record => record.kind === kind && record.indent === 0)
+      .map(record => record.code.split(/\s+/)[1])
+      .filter(Boolean);
+  const field = current?.field ? current : [...parents].reverse().find(record => record.field);
+  const type = field?.field[2]?.split(/\s+"/)[0] ?? 'string';
+  const inData = parents.some(record => record.kind === 'data');
+  const inActions = parent?.kind === 'actions';
+  const inFragment = parents.some(record => record.kind === 'fragment');
+  const repeats = parents.filter(record => record.kind === 'repeat');
+  const namespaces = [
+    'data',
+    'computed',
+    ...(inFragment ? ['inputs', 'local'] : []),
+    ...(repeats.length ? ['item'] : []),
+    ...(repeats.length > 1 ? ['parent'] : []),
+  ];
+  const target = labels => done(choices(labels, 'variable'));
+  const words = labels => done(choices(labels));
+  const unused = (labels, owner = field) => {
+    // Include both sides of the cursor and settings elsewhere in this block.
+    const related = owner ? records.filter(record => record === owner || record.parents.at(-1) === owner) : [];
+    const occupied = related
+      .map(record => (record.from === line.from ? head + after : record.code))
+      .map(masked)
+      .join('\n');
+    return labels.filter(label => !new RegExp(`\\b${label.trim().split(/\s/)[0]}\\b`).test(occupied));
+  };
+
+  if (/^\s*(?:input\s+)?\w+\s*:\s*(?:\(\s*)?$/.test(head)) return done(choices(TYPES, 'type'));
+  if (/^\s*forming\s+$/.test(head)) return words(['1']);
+  if (/^\s*import\s+\{[^}]*\}\s*$/.test(head)) return words(['from ']);
+  if (/^\s*options\s+from\s+$/.test(head)) return target([...declared('options'), ...importedOf('options')]);
+  if (/^\s*options\s+$/.test(head) && field) return words(['from ']);
+  if (/^\s*data\s+$/.test(head)) return words(['from ']);
+  if (/^\s*data\s+from\s+$/.test(head)) return target(importedOf('schema'));
+  if (/^\s*use\s+$/.test(head)) return target([...declared('fragment'), ...importedOf('fragment')]);
+  if (/^\s*use\s+\w+\s+$/.test(head)) return words(['at ']);
+  if (/^\s*(?:use\s+\w+\s+at|repeat|set|clear)\s+$/.test(head)) return target(names.fields);
+  if (/^\s*next\s+$/.test(head) && inActions) return target(declared('page'));
+  if (/^\s*validate\s+with\s+$/.test(head)) return target(importedOf('validator'));
+  if (/^\s*upload\s+with\s+$/.test(head)) return target(importedOf('upload'));
+  if (/^\s*invoke\s+$/.test(head)) return target(importedOf('action'));
+  if (/^\s*upload\s+$/.test(head)) return words(['with ']);
+  if (/\bas\s+$/.test(masked(head)) && field) return done(choices(controls(type), 'enum'));
+  if (/^\s*(?:show|enable|readonly|required)\s+$/.test(head)) return words(['when ']);
+  if (/^\s*with\s+$/.test(head) && parent?.kind === 'set') return words(['value = ']);
+  if (/^\s*with\s+\w+\s*$/.test(head)) return words(['= ']);
+
+  const expressionMatch = masked(head).match(
+    /(?:\bwhen\s+|^\s*validate\s+(?!with\b)|^\s*computed\s+|(?<![=!<>])=(?!=)\s*)(.*)$/
+  );
+  if (expressionMatch && !(inActions && /^\s*validate\s/.test(head))) {
+    const expression = head.slice(expressionMatch.index + expressionMatch[0].length - expressionMatch[1].length);
+    const member = expression.match(/((?:[A-Za-z_][A-Za-z0-9_]*\.)+)$/)?.[1];
+    if (member) {
+      const [namespace, ...path] = member.slice(0, -1).split('.');
+      let paths = names.fields;
+      if (namespace === 'computed') paths = names.computed;
+      else if (namespace === 'inputs') paths = names.inputs;
+      else if (namespace === 'item' || namespace === 'parent') {
+        const repeat = repeats.at(namespace === 'item' ? -1 : -2);
+        if (!repeat) return null;
+        const prefix = repeat.code.split(/\s+/)[1] + '.';
+        paths = paths.filter(name => name.startsWith(prefix)).map(name => name.slice(prefix.length));
+      } else if (!['data', 'local'].includes(namespace)) path.unshift(namespace);
+      const prefix = path.length ? path.join('.') + '.' : '';
+      return target(paths.filter(name => name.startsWith(prefix)).map(name => name.slice(prefix.length).split('.')[0]));
+    }
+    const significant = scriptTokens(expression).filter(token => token.text.trim());
+    const last = significant.at(-1);
+    const operand =
+      !last ||
+      /^(and|or|not|in)$/.test(last.text) ||
+      (last.type === 'operator' && last.text !== '.') ||
+      /^[([,:]$/.test(last.text);
+    if (!operand) return words(OPERATORS);
+    return done([
+      ...choices(
+        names.fields.filter(name => !name.includes('.')),
+        'variable'
+      ),
+      ...choices(namespaces, 'namespace'),
+      ...choices(
+        Object.entries(FUNCTIONS).map(([label, detail]) => ({label, detail, apply: label + '('})),
+        'function'
+      ),
+      ...choices(['true', 'false', 'null', 'not', ...(/^\s*validate\s+$/.test(head) ? ['with '] : [])]),
+    ]);
+  }
+
+  // Value slots have their own vocabulary; structural keywords cannot fill them.
+  const valueSlot = head.match(
+    /\b(default|mustBe|min|max|multipleOf|minLength|maxLength|minItems|maxItems|pattern|region|accept|maxSize|maxTotalSize|description|placeholder|message|id)\s*$/
+  )?.[1];
+  if (valueSlot)
+    return ['default', 'mustBe'].includes(valueSlot)
+      ? words(
+          ['true', 'false', 'null'].filter(value =>
+            value === 'null' ? type.includes('?') : type.startsWith('boolean')
+          )
+        )
+      : null;
+  if (/^\s*next\s+\S+\s+"(?:\\.|[^"\\])*"\s+$/.test(head)) return words(['when ']);
+  if (current?.field && /"(?:\\.|[^"\\])*"\s/.test(head))
+    return words(unused([...constraints(type), ...(!inData ? ['as '] : [])]));
+  if (head.trim()) return null;
+
+  if (inActions)
+    return words([
+      'submit ',
+      'save ',
+      'reset ',
+      'validate ',
+      'set ',
+      'clear ',
+      'invoke ',
+      ...(parents.some(record => record.kind === 'page') ? ['next ', 'back '] : []),
+    ]);
+  if (parent?.kind === 'options') {
+    const provider = parent.code.match(/^options\s+from\s+(\w+)/)?.[1];
+    return provider ? (imported.includes(provider) ? words(['with ']) : null) : words(['option ']);
+  }
+  if (parent?.field) {
+    return words(
+      unused([
+        ...annotations,
+        ...constraints(type),
+        'description ',
+        'schema ',
+        ...(!inData
+          ? [
+              'as ',
+              'placeholder ',
+              'options',
+              'validate ',
+              ...RULES,
+              'required when ',
+              ...(type.startsWith('file') ? ['upload with '] : []),
+              ...(!/^(object|file)\b/.test(type) && !type.includes('[]') ? ['computed '] : []),
+            ]
+          : []),
+        ...(type.startsWith('object') && !inData ? VIEWS : []),
+      ])
+    );
+  }
+  if (parent?.kind === 'data' || parent?.kind === 'computed') return null;
+  if (parent?.kind === 'set') return words(['with value = ']);
+  if (parent && ['invoke', 'upload', 'as'].includes(parent.kind)) return words(['with ']);
+  if (parent?.kind === 'validate')
+    return words(unused(['message ', 'when ', ...(parent.code.includes(' with ') ? ['with '] : [])], parent));
+  if (parent && ['use', 'display'].includes(parent.kind))
+    return words([
+      'with ',
+      ...RULES,
+      ...(parent.kind === 'use' ? ['required when '] : []),
+      ...unused(annotations, parent),
+    ]);
+  if (
+    parent &&
+    ['page', ...VIEWS.filter(name => name !== 'use' && name !== 'display'), 'fragment'].includes(parent.kind)
+  ) {
+    return words([
+      ...VIEWS,
+      ...(!inFragment ? RULES : []),
+      ...(parent.kind === 'page' ? ['actions'] : []),
+      ...(parent.kind === 'repeat' ? ['computed', 'required when ', 'id '] : []),
+      ...(parent.kind === 'fragment' ? ['input '] : []),
+    ]);
+  }
+  if (indent > 0) return null;
+  const earlier = records.filter(record => record.from < line.from);
+  if (!earlier.some(record => record.kind === 'forming')) return words(['forming ']);
+  if (!earlier.some(record => record.kind === 'form'))
+    return words([
+      'import',
+      'options',
+      'fragment',
+      ...(!records.some(record => record !== current && record.kind === 'form') ? ['form'] : []),
+    ]);
+  const existing = records.filter(record => record !== current && record.indent === 0).map(record => record.kind);
+  return done([
+    ...choices(['page', ...VIEWS, ...['data', 'computed', 'actions'].filter(kind => !existing.includes(kind))]),
+    ...choices(names.fields, 'variable'),
+  ]);
+}
+
+/** Map each imported name to its export kind, using the supplied package manifests. */
+export function importKinds(source, packages = []) {
+  const manifests = new Map(
+    (typeof packages === 'string' ? JSON.parse(packages) : packages).map(p => [`${p.name}@${p.version}`, p])
+  );
+  const kinds = new Map();
+  for (const match of source.matchAll(/^import\s*\{([^}]*)\}\s*from\s*"([^"]+)"/gm)) {
+    const manifest = manifests.get(match[2]);
+    for (const part of match[1].split(',')) {
+      const [name, alias = name] = part.trim().split(/\s+as\s+/);
+      const kind = manifest?.exports?.[name]?.kind;
+      if (name && kind) kinds.set(alias, kind);
+    }
+  }
+  return kinds;
+}
+
+/** Completions for plain text at a UTF-16 offset, for editors and language servers. */
+export function complete(text, offset, {explicit = true, kinds} = {}) {
+  const starts = [0];
+  for (let i = 0; i < text.length; i++) if (text[i] === '\n') starts.push(i + 1);
+  const doc = {
+    toString: () => text,
+    lineAt(pos) {
+      let index = starts.length - 1;
+      while (starts[index] > pos) index--;
+      const from = starts[index];
+      const end = text.indexOf('\n', from);
+      return {from, text: text.slice(from, end < 0 ? text.length : end), number: index + 1};
+    },
+  };
+  return formingCompletions({state: {doc}, pos: offset, explicit}, {kinds});
+}

package/dist/tokens.mjs

@@ -0,0 +1,216 @@
+// Structural words and namespaces reserved by the grammar (spec/grammar.md).
+const STRUCTURE = new Set([
+  'forming',
+  'form',
+  'page',
+  'fragment',
+  'input',
+  'use',
+  'actions',
+  'options',
+  'option',
+  'schema',
+  'import',
+  'from',
+  'with',
+  'section',
+  'group',
+  'stack',
+  'columns',
+  'grid',
+  'repeat',
+  'display',
+  'data',
+  'computed',
+  'upload',
+]);
+const NAMESPACES = new Set(['data', 'context', 'item', 'parent', 'computed', 'inputs', 'local', 'node']);
+const ACTIONS = new Set(['next', 'back', 'save', 'submit', 'reset', 'set', 'clear', 'invoke', 'validate']);
+const MODIFIERS = new Set([
+  'required',
+  'show',
+  'enable',
+  'readonly',
+  'when',
+  'message',
+  'default',
+  'description',
+  'placeholder',
+  'as',
+  'at',
+  'id',
+  'region',
+  'accept',
+  'maxSize',
+  'maxTotalSize',
+  'min',
+  'max',
+  'minLength',
+  'maxLength',
+  'minItems',
+  'maxItems',
+  'multipleOf',
+  'mustBe',
+  'uniqueItems',
+  'pattern',
+  'format',
+  'hide',
+  'disable',
+]);
+const TYPES = new Set([
+  'string',
+  'integer',
+  'number',
+  'boolean',
+  'object',
+  'email',
+  'url',
+  'phone',
+  'date',
+  'time',
+  'datetime',
+  'file',
+  'enum',
+]);
+const FUNCTIONS = new Set([
+  'has',
+  'size',
+  'sum',
+  'default',
+  'startsWith',
+  'endsWith',
+  'contains',
+  'count',
+  'every',
+  'some',
+]);
+const LOGIC = new Set(['and', 'or', 'not', 'in']);
+const LITERALS = new Set(['true', 'false', 'null']);
+
+const NAME = /[A-Za-z_][A-Za-z0-9_-]*/;
+const NUMBER = /-?(?:0|[1-9][0-9]*)(?:\.[0-9]+)?(?:[eE][+-]?[0-9]+)?/;
+
+export const scriptTokenizer = {
+  name: 'forming',
+  startState: () => ({expectType: false}),
+  token(stream, state) {
+    if (stream.sol()) state.expectType = false;
+    if (stream.eatSpace()) return null;
+    if (stream.peek() === '#') {
+      stream.skipToEnd();
+      return 'lineComment';
+    }
+    if (stream.peek() === '"') {
+      stream.next();
+      while (!stream.eol()) {
+        const ch = stream.next();
+        if (ch === '\\') stream.next();
+        else if (ch === '"') break;
+      }
+      state.expectType = false;
+      return 'string';
+    }
+    if (stream.match(NUMBER)) {
+      state.expectType = false;
+      return 'number';
+    }
+    if (stream.match(NAME)) {
+      const word = stream.current();
+      const expectType = state.expectType;
+      state.expectType = false;
+      if (stream.match(/^\s*:/, false)) {
+        state.expectType = true;
+        return 'variableName.definition';
+      }
+      if (expectType && TYPES.has(word)) return 'typeName';
+      if (stream.match(/^\(/, false) && FUNCTIONS.has(word)) return 'variableName.function';
+      if (LITERALS.has(word)) return word === 'null' ? 'null' : 'bool';
+      if (LOGIC.has(word)) return 'logicOperator';
+      if (STRUCTURE.has(word)) return 'keyword';
+      if (ACTIONS.has(word)) return 'keyword';
+      if (NAMESPACES.has(word)) return 'namespace';
+      if (MODIFIERS.has(word)) return 'modifier';
+      if (TYPES.has(word)) return 'typeName';
+      return 'variableName';
+    }
+    const ch = stream.next();
+    if (ch === ':') {
+      state.expectType = true;
+      return 'punctuation';
+    }
+    if (ch === '?' && stream.peek() === '?') {
+      stream.next();
+      return 'operator';
+    }
+    if ('=!<>'.includes(ch) && stream.peek() === '=') {
+      stream.next();
+      return 'operator';
+    }
+    if ('=!<>+-*/%?.'.includes(ch)) return 'operator';
+    if ('()[],'.includes(ch)) return 'punctuation';
+    return null;
+  },
+};
+
+// Minimal stream interface for static snippets. No editor or browser is needed.
+export function scriptTokens(source) {
+  const result = [];
+  const state = scriptTokenizer.startState();
+  let offset = 0;
+  for (const line of source.split('\n')) {
+    const stream = {
+      pos: 0,
+      start: 0,
+      sol() {
+        return this.pos === 0;
+      },
+      eol() {
+        return this.pos >= line.length;
+      },
+      peek() {
+        return line[this.pos];
+      },
+      next() {
+        return line[this.pos++];
+      },
+      current() {
+        return line.slice(this.start, this.pos);
+      },
+      skipToEnd() {
+        this.pos = line.length;
+      },
+      eatSpace() {
+        return this.match(/^\s+/);
+      },
+      match(pattern, consume = true) {
+        const match = line.slice(this.pos).match(pattern);
+        if (!match || match.index !== 0) return false;
+        if (consume) this.pos += match[0].length;
+        return match;
+      },
+    };
+    while (!stream.eol()) {
+      stream.start = stream.pos;
+      const type = scriptTokenizer.token(stream, state);
+      // An unfinished escape can advance past the final character.
+      stream.pos = Math.min(stream.pos, line.length);
+      result.push({from: offset + stream.start, to: offset + stream.pos, type, text: stream.current()});
+    }
+    if (offset + line.length < source.length)
+      result.push({from: offset + line.length, to: offset + line.length + 1, type: null, text: '\n'});
+    offset += line.length + 1;
+  }
+  return result;
+}
+
+/** The tokenizer's word lists, for editors that build their own highlighting. */
+export const WORDS = Object.freeze({
+  structure: [...STRUCTURE],
+  namespaces: [...NAMESPACES],
+  actions: [...ACTIONS],
+  modifiers: [...MODIFIERS],
+  types: [...TYPES],
+  functions: [...FUNCTIONS],
+  logic: [...LOGIC],
+  literals: [...LITERALS],
+});

package/package.json

@@ -1,6 +1,28 @@
 {
   "name": "@forming/tools",
-  "version": "0.0.0-stage",
-  "stub": true,
-  "description": "Temporary package placeholder for staged publishing"
-}
+  "version": "0.1.0-dev.2.1",
+  "type": "module",
+  "exports": {
+    "./completions": "./dist/completions.mjs",
+    "./tokens": "./dist/tokens.mjs"
+  },
+  "files": [
+    "dist",
+    "README.md",
+    "NOTICE.md",
+    "LICENSE"
+  ],
+  "scripts": {
+    "test": "node --test test/*.test.mjs",
+    "build": "node build-package.mjs",
+    "prepack": "npm run build"
+  },
+  "license": "SEE LICENSE IN LICENSE",
+  "publishConfig": {
+    "registry": "https://registry.npmjs.org/"
+  },
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/forming-info/tools.git"
+  }
+}