hclang 0.5.14 → 0.15.1

package/README.md

@@ -1,50 +1,271 @@
-# Homiconic C
-version: 0.5.11
+# Homoiconic C
 
-[![Codeship Status for TheSwanFactory/hclang](https://app.codeship.com/projects/b1198f30-aee9-0137-5fdc-4a1003a17a1c/status?branch=master)](https://app.codeship.com/projects/362584)
+[![JSR](https://jsr.io/badges/@swanfactory/hclang)](https://jsr.io/@swanfactory/hclang)
+[![JSR Score](https://jsr.io/badges/@swanfactory/hclang/score)](https://jsr.io/@swanfactory/hclang)
+
+A universal language for code and data.
 
 ## Introduction
 
-[Homoiconic C](https://theswanfactory.wordpress.com/2016/12/20/homoiconic-c-a-universal-language-for-code-and-data/) (HC) is a single universal language for code and data.  This repository contains the first implementation of HC as an interpreter written in TypeScript running on `nodejs`.  It also contains a sample application called MAML, the [*Multipurpose Abstract Markup Language*](https://theswanfactory.wordpress.com/2016/11/08/introducing-maml-a-draft-proposal-for-html6/).  MAML is a radically simple proposal for replacing all the existing web technologies (HMTL, CSS, JavaScript, SVG, etc.) with a single format based on HC. It can also be used to define easily-parseable variants of other common file formats (e.g., HCSV, HCSON). 
+[Homoiconic C](https://theswanfactory.wordpress.com/2016/12/20/homoiconic-c-a-universal-language-for-code-and-data/)
+(HC) is a single universal language for code and data. This monorepo contains
+the first implementation of HC as an interpreter written in TypeScript running
+on [Deno](https://deno.land/).
 
-## Usage
+The repository includes several packages:
+
+- **[@swanfactory/hclang](https://jsr.io/@swanfactory/hclang)** - Core library
+  published to JSR
+- **CLI** - Interactive REPL and command-line tools
+- **MAML** -
+  [_Multipurpose Abstract Markup Language_](https://theswanfactory.wordpress.com/2016/11/08/introducing-maml-a-draft-proposal-for-html6/),
+  a radically simple proposal for replacing web technologies (HTML, CSS,
+  JavaScript, SVG) with a single format based on HC
+- **Web** - Web components and Fresh framework integration
+
+HC can also be used to define easily-parseable variants of other common file
+formats (e.g., HCSV, HCSON).
+
+## Quick Start
+
+### Try it Online
+
+Visit [hclang.deno.dev](https://hclang.deno.dev) to try HC directly in your
+browser with no installation required.
+
+### Use as a Library
+
+```bash
+# Deno
+deno add jsr:@swanfactory/hclang
 
+# Node.js/npm
+npx jsr add @swanfactory/hclang
 ```
-$ npm install
-$ export DEBUG=true # optional
-$ npm run hc
+
+Then in your code:
+
+```typescript
+import { execute } from "@swanfactory/hclang";
+
+const result = execute("1 + 1");
+console.log(result); // Output: '2'
 ```
-This will launch the interpreter.
 
-## BitScheme
+See the [library documentation](https://jsr.io/@swanfactory/hclang/doc) for more
+details.
+
+### Run the CLI
 
-To generate and run the BitScheme documentation, type:
+With Node.js 24 or newer, run the current CLI directly from npm. Deno is not
+required:
+
+```bash
+# Interactive REPL
+npx hclang@latest
+
+# Evaluate a file
+npx hclang@latest path/to/program.hc
+
+# Verify executable examples in an HC document
+npx hclang@latest path/to/examples.hc --testdoc
 ```
-$ npm run bs:all
+
+From source:
+
+```bash
+git clone https://github.com/TheSwanFactory/hclang.git
+cd hclang
+deno task hc
+```
+
+This will launch the interactive HC interpreter.
+
+### Run Tests
+
+```bash
+deno task test
+```
+
+## Usage
+
+### Available Tasks
+
+The root `deno.json` provides several tasks:
+
+| Task                   | Description                                       |
+| ---------------------- | ------------------------------------------------- |
+| `deno task hc`         | Launch the HC REPL                                |
+| `deno task setup`      | Install pre-commit hooks                          |
+| `deno task test`       | Run all tests (setup, format, lint, and tests)    |
+| `deno task test:setup` | Run all pre-commit hooks on all files             |
+| `deno task test:cli`   | Test CLI package                                  |
+| `deno task test:lib`   | Test library package                              |
+| `deno task test:maml`  | Test MAML package                                 |
+| `deno task test:web`   | Test web package                                  |
+| `deno task test:bs`    | Test BitScheme documentation                      |
+| `deno task test:doc`   | Test HC documentation examples                    |
+| `deno task build`      | Build native, npm, and web artifacts              |
+| `deno task build:npm`  | Generate the Node.js CLI package in `dist/npm`    |
+| `deno task test:npm`   | Pack and smoke-test the CLI package through `npx` |
+
+### BitScheme
+
+[BitScheme](cli/hc/BitScheme.hc) is a specialized scripting language for
+declaratively parsing, manipulating, and generating binary data. It includes
+executable documentation that can be tested:
+
+```bash
+deno task test:bs
+```
+
+This runs the BitScheme interpreter on its own documentation, executing embedded
+code examples and verifying outputs.
+
+### Debug Mode
+
+Enable verbose debugging output:
+
+```bash
+export DEBUG=true
+deno task hc
 ```
-This will open the BitScheme.html file, and also run the documentation through the testdoc evaluator which will generate pass/fail messages.
 
 ## Development
 
-1. Install the [Atom editor](http://flight-manual.atom.io/getting-started/sections/installing-atom/).
+### Setup
+
+1. Clone the [GitHub repository](https://github.com/TheSwanFactory/hclang.git):
+
+   ```bash
+   git clone https://github.com/TheSwanFactory/hclang.git
+   cd hclang
+   ```
+
+2. Install [Deno](https://github.com/denoland/deno):
+
+   ```bash
+   # macOS
+   brew install deno
+
+   # Other platforms: see https://deno.land/
+   ```
+
+3. (Optional) Install [pre-commit](https://pre-commit.com/) for automatic code
+   quality checks:
+
+   ```bash
+   # macOS
+   brew install pre-commit
+
+   # Other platforms: pip install pre-commit
+   ```
+
+   Pre-commit hooks are automatically installed when you run `deno task test` or
+   manually with `deno task setup`.
+
+4. Run tests:
+
+   ```bash
+   deno task test
+   ```
+
+   This will automatically set up pre-commit hooks, format code, run linters,
+   and execute the test suite.
+
+### Project Structure
+
+This is a Deno workspace (monorepo) with the following packages:
+
+```text
+hclang/
+├── cli/          # HC command-line interface & REPL
+├── lib/          # Core library (published to JSR)
+├── maml/         # MAML markup language
+├── web/          # Web components
+├── doc/          # Documentation and papers
+└── deno.json     # Workspace root configuration
+```
+
+### VS Code Extension
+
+The VS Code extension for HC is now included in this repository at
+[vscode-extension/](vscode-extension/). It provides comprehensive syntax
+highlighting for all HC language features based on the complete grammar
+specification.
+
+**Installation:**
+
+```bash
+# Install from source
+deno task vscode:install
+
+# Or package for distribution
+deno task vscode:package
+
+# Or publish to marketplace
+deno task vscode:publish
+```
+
+The extension recognizes `.hc` files and provides:
+
+- Syntax highlighting for all HC elements (numbers, strings, identifiers,
+  operators, etc.)
+- Smart bracket matching for `{}`, `[]`, `()`
+- Auto-closing pairs
+- Comment toggling with `#`
+
+**Note:** The old standalone
+[language-hclang](https://github.com/TheSwanFactory/language-hclang) repository
+is now deprecated in favor of this integrated extension.
+
+## Publishing
+
+For maintainers publishing new versions:
+
+1. Ensure all tests pass:
+
+   ```bash
+   deno task test
+   ```
 
-2. Install the [language-maml](https://github.com/TheSwanFactory/language-maml) Atom package.
+2. Update version everywhere:
 
-3. Clone the [github repository](https://github.com/TheSwanFactory/hclang.git).
+   ```bash
+   deno task bump
+   ```
 
-4. Install [node.js](https://nodejs.org/).
-  * e.g., `brew install node` on macOS.
+   This updates `deno.json`, `lib/version.ts`, and all workspace package
+   versions, then commits the changes.
 
-5. Run `npm test`.
+3. Merge the PR to the `master` branch.
 
-## Publishing 
+   GitHub Actions compares the version in `deno.json` with the version before
+   the merge. When the version changed, it publishes the libraries to
+   [JSR](https://jsr.io/@swanfactory/hclang), publishes the exact `hclang`
+   tarball exercised through `npx`, and creates a GitHub Release tagged
+   `v<version>` with generated notes. Each registry is checked independently, so
+   a rerun resumes a partial release instead of retrying versions that are
+   already immutable. Merges without a version bump do not publish or create a
+   release; no manual tag is required.
 
-From the feature branch:
+   npm publishing runs in the protected `npm-publish` GitHub environment using a
+   trusted publisher for the public `hclang` package. Its npm settings must
+   authorize the `TheSwanFactory/hclang` repository,
+   `.github/workflows/deno.js.yml`, the `npm-publish` environment, and direct
+   `npm publish` before merging a release bump. The build/PR job has no OIDC
+   token permission.
 
-1. Update CHANGELOG.md
+## Links
 
-1. Run `npm run done` to bump version and tag
+- **JSR Package**: [@swanfactory/hclang](https://jsr.io/@swanfactory/hclang)
+- **Documentation**: [JSR Docs](https://jsr.io/@swanfactory/hclang/doc)
+- **GitHub**: [TheSwanFactory/hclang](https://github.com/TheSwanFactory/hclang)
+- **Blog**:
+  [Homoiconic C Introduction](https://theswanfactory.wordpress.com/2016/12/20/homoiconic-c-a-universal-language-for-code-and-data/)
+- **MAML**:
+  [HTML6 Proposal](https://theswanfactory.wordpress.com/2016/11/08/introducing-maml-a-draft-proposal-for-html6/)
 
-1. Merge PR
+## License
 
-Should automatically publish to [npmjs](https://www.npmjs.com/package/hclang).
+MIT

package/esm/_dnt.polyfills.js

@@ -0,0 +1,144 @@
+/**
+ * Based on [import-meta-ponyfill](https://github.com/gaubee/import-meta-ponyfill),
+ * but instead of using npm to install additional dependencies,
+ * this approach manually consolidates cjs/mjs/d.ts into a single file.
+ *
+ * Note that this code might be imported multiple times
+ * (for example, both dnt.test.polyfills.ts and dnt.polyfills.ts contain this code;
+ *  or Node.js might dynamically clear the cache and then force a require).
+ * Therefore, it's important to avoid redundant writes to global objects.
+ * Additionally, consider that commonjs is used alongside esm,
+ * so the two ponyfill functions are stored independently in two separate global objects.
+ */
+//@ts-ignore
+import { createRequire } from "node:module";
+//@ts-ignore
+import { fileURLToPath, pathToFileURL } from "node:url";
+//@ts-ignore
+import { dirname } from "node:path";
+const defineGlobalPonyfill = (symbolFor, fn) => {
+    if (!Reflect.has(globalThis, Symbol.for(symbolFor))) {
+        Object.defineProperty(globalThis, Symbol.for(symbolFor), {
+            configurable: true,
+            get() {
+                return fn;
+            },
+        });
+    }
+};
+export let import_meta_ponyfill_commonjs = (Reflect.get(globalThis, Symbol.for("import-meta-ponyfill-commonjs")) ??
+    (() => {
+        const moduleImportMetaWM = new WeakMap();
+        return (require, module) => {
+            let importMetaCache = moduleImportMetaWM.get(module);
+            if (importMetaCache == null) {
+                const importMeta = Object.assign(Object.create(null), {
+                    url: pathToFileURL(module.filename).href,
+                    main: require.main == module,
+                    resolve: (specifier, parentURL = importMeta.url) => {
+                        return pathToFileURL((importMeta.url === parentURL
+                            ? require
+                            : createRequire(parentURL))
+                            .resolve(specifier)).href;
+                    },
+                    filename: module.filename,
+                    dirname: module.path,
+                });
+                moduleImportMetaWM.set(module, importMeta);
+                importMetaCache = importMeta;
+            }
+            return importMetaCache;
+        };
+    })());
+defineGlobalPonyfill("import-meta-ponyfill-commonjs", import_meta_ponyfill_commonjs);
+export let import_meta_ponyfill_esmodule = (Reflect.get(globalThis, Symbol.for("import-meta-ponyfill-esmodule")) ??
+    ((importMeta) => {
+        const resolveFunStr = String(importMeta.resolve);
+        const shimWs = new WeakSet();
+        //@ts-ignore
+        const mainUrl = ("file:///" + process.argv[1].replace(/\\/g, "/"))
+            .replace(/\/{3,}/, "///");
+        const commonShim = (importMeta) => {
+            if (typeof importMeta.main !== "boolean") {
+                importMeta.main = importMeta.url === mainUrl;
+            }
+            if (typeof importMeta.filename !== "string") {
+                importMeta.filename = fileURLToPath(importMeta.url);
+                importMeta.dirname = dirname(importMeta.filename);
+            }
+        };
+        if (
+        // v16.2.0+, v14.18.0+: Add support for WHATWG URL object to parentURL parameter.
+        resolveFunStr === "undefined" ||
+            // v20.0.0+, v18.19.0+"" This API now returns a string synchronously instead of a Promise.
+            resolveFunStr.startsWith("async")
+        // enable by --experimental-import-meta-resolve flag
+        ) {
+            import_meta_ponyfill_esmodule = (importMeta) => {
+                if (!shimWs.has(importMeta)) {
+                    shimWs.add(importMeta);
+                    const importMetaUrlRequire = {
+                        url: importMeta.url,
+                        require: createRequire(importMeta.url),
+                    };
+                    // note: the parameter types must be explicitly annotated because
+                    // `@types/node` also declares `ImportMeta.resolve`, which makes
+                    // the contextual type an overload set that provides no inference
+                    importMeta.resolve = function resolve(specifier, parentURL = importMeta.url) {
+                        return pathToFileURL((importMetaUrlRequire.url === parentURL
+                            ? importMetaUrlRequire.require
+                            : createRequire(parentURL)).resolve(specifier)).href;
+                    };
+                    commonShim(importMeta);
+                }
+                return importMeta;
+            };
+        }
+        else {
+            /// native support
+            import_meta_ponyfill_esmodule = (importMeta) => {
+                if (!shimWs.has(importMeta)) {
+                    shimWs.add(importMeta);
+                    commonShim(importMeta);
+                }
+                return importMeta;
+            };
+        }
+        return import_meta_ponyfill_esmodule(importMeta);
+    }));
+defineGlobalPonyfill("import-meta-ponyfill-esmodule", import_meta_ponyfill_esmodule);
+export let import_meta_ponyfill = ((...args) => {
+    const _MODULE = (() => {
+        if (typeof require === "function" && typeof module === "object") {
+            return "commonjs";
+        }
+        else {
+            // eval("typeof import.meta");
+            return "esmodule";
+        }
+    })();
+    if (_MODULE === "commonjs") {
+        //@ts-ignore
+        import_meta_ponyfill = (r, m) => import_meta_ponyfill_commonjs(r, m);
+    }
+    else {
+        //@ts-ignore
+        import_meta_ponyfill = (im) => import_meta_ponyfill_esmodule(im);
+    }
+    //@ts-ignore
+    return import_meta_ponyfill(...args);
+});
+// https://github.com/tc39/proposal-accessible-object-hasownproperty/blob/main/polyfill.js
+if (!Object.hasOwn) {
+    Object.defineProperty(Object, "hasOwn", {
+        value: function (object, property) {
+            if (object == null) {
+                throw new TypeError("Cannot convert undefined or null to object");
+            }
+            return Object.prototype.hasOwnProperty.call(Object(object), property);
+        },
+        configurable: true,
+        enumerable: false,
+        writable: true,
+    });
+}

package/esm/_dnt.shims.js

@@ -0,0 +1,61 @@
+import { Deno } from "@deno/shim-deno";
+export { Deno } from "@deno/shim-deno";
+const dntGlobals = {
+    Deno,
+};
+export const dntGlobalThis = createMergeProxy(globalThis, dntGlobals);
+function createMergeProxy(baseObj, extObj) {
+    return new Proxy(baseObj, {
+        get(_target, prop, _receiver) {
+            if (prop in extObj) {
+                return extObj[prop];
+            }
+            else {
+                return baseObj[prop];
+            }
+        },
+        set(_target, prop, value) {
+            if (prop in extObj) {
+                delete extObj[prop];
+            }
+            baseObj[prop] = value;
+            return true;
+        },
+        deleteProperty(_target, prop) {
+            let success = false;
+            if (prop in extObj) {
+                delete extObj[prop];
+                success = true;
+            }
+            if (prop in baseObj) {
+                delete baseObj[prop];
+                success = true;
+            }
+            return success;
+        },
+        ownKeys(_target) {
+            const baseKeys = Reflect.ownKeys(baseObj);
+            const extKeys = Reflect.ownKeys(extObj);
+            const extKeysSet = new Set(extKeys);
+            return [...baseKeys.filter((k) => !extKeysSet.has(k)), ...extKeys];
+        },
+        defineProperty(_target, prop, desc) {
+            if (prop in extObj) {
+                delete extObj[prop];
+            }
+            Reflect.defineProperty(baseObj, prop, desc);
+            return true;
+        },
+        getOwnPropertyDescriptor(_target, prop) {
+            if (prop in extObj) {
+                return Reflect.getOwnPropertyDescriptor(extObj, prop);
+            }
+            else {
+                return Reflect.getOwnPropertyDescriptor(baseObj, prop);
+            }
+        },
+        has(_target, prop) {
+            return prop in extObj || prop in baseObj;
+        },
+    });
+}

package/esm/cli/hc.js

@@ -0,0 +1,159 @@
+#!/usr/bin/env node
+import "../_dnt.polyfills.js";
+import * as dntShim from "../_dnt.shims.js";
+import { HCEval, make_context } from "../lib/execute/hc-eval.js";
+import { HCLog } from "../lib/execute/hc-log.js";
+import { HCTest } from "../lib/execute/hc-test.js";
+import { parseArgs } from "../deps/jsr.io/@std/cli/1.0.32/parse_args.js";
+import { runfile } from "./runfile.js";
+import { Prompt } from "./prompt.js";
+import { DenoFileStore } from "./resource-store.js";
+import { Frame, FrameResource, RESOURCE_ROOT_KEY, } from "../lib/frames.js";
+/**
+ * @module hc
+ *
+ * This is the main entry point for the hc command-line interface (CLI).
+ */
+/**
+ * aliases to short form of command-line interface (CLI) options.
+ */
+const aliases = {
+    e: "evaluate",
+    h: "help",
+    i: "interactive",
+    t: "testdoc",
+    v: "verbose",
+    V: "version",
+};
+/**
+ * Parses command-line arguments and returns the options.
+ *
+ * @param args - The array of command-line arguments (only).
+ * @returns The parsed options object.
+ */
+export function getOptions(args) {
+    return parseArgs(args, {
+        alias: aliases,
+        boolean: ["help", "interactive", "testdoc", "verbose", "version"],
+        string: ["evaluate"],
+    });
+}
+/**
+ * Creates and returns an instance of `HCEval` initialized with the provided environment variables.
+ *
+ * The host namespace also carries this harness's root binding, so a resource
+ * identifier resolves against a fresh temp directory rather than against the
+ * working directory. That is a narrowing: the process still holds whatever the
+ * harness granted it, and the root binding is the ceiling the language enforces
+ * inside it.
+ *
+ * @param env - An object containing key-value pairs of environment variables.
+ * @param out - Optional output sink. Tests inject a capture frame; the CLI uses HCLog.
+ * @returns An instance of `HCEval` configured with the provided environment variables.
+ */
+export function getEval(env, out) {
+    const context = make_context(env);
+    const output = out ?? new HCLog(context);
+    return new HCEval(output, new Frame(), getHost(context));
+}
+/**
+ * Builds this harness's host namespace, root binding included.
+ *
+ * Exported so a test can run a source unit under the authority the CLI actually
+ * grants, rather than under an empty namespace that would make the resource
+ * primitive invisible.
+ *
+ * @param context - Host bindings, which are copied rather than adopted.
+ * @returns A frame reachable from HC source through `$$`.
+ */
+export function getHost(context = {}) {
+    // Copied, because a Frame adopts the Context object it is handed and the
+    // logger holds the same one: the root binding belongs to the host namespace
+    // and nowhere else.
+    const host = new Frame({ ...context });
+    host.set(RESOURCE_ROOT_KEY, FrameResource.root(new DenoFileStore()));
+    return host;
+}
+/**
+ * The main function for the CLI application.
+ *
+ * @param hc_eval - An instance of HCEval.
+ * @param options - The options object returned by the getOptions function.
+ *
+ * The function performs the following tasks:
+ * - Logs the options if the verbose flag is set.
+ * - Creates a context and output logger.
+ * - Initializes the HCEval instance.
+ * - If the testdoc option is set, initializes the HCTest instance and updates the HCEval instance.
+ * - If the evaluate option is set, evaluates the provided code.
+ * - Iterates over the files provided in the options and runs each file.
+ * - If the interactive option is set or no evaluation has been performed, starts the REPL.
+ */
+export async function main(hcEval, options) {
+    let evaluated = false;
+    let sourceStarted = false;
+    let lexicalComplete = true;
+    let test;
+    if (options.verbose) {
+        console.error("options", options);
+    }
+    if (options.testdoc) {
+        test = new HCTest(hcEval.out);
+        hcEval = hcEval.withOutput(test);
+        evaluated = true;
+    }
+    const finishSource = (evaluator) => {
+        const complete = evaluator.finish();
+        if (!complete) {
+            const reason = evaluator.error() ?? "incomplete lexical input";
+            console.error(`HCEval.finish.failed: ${reason}`);
+        }
+        return complete;
+    };
+    if (options.evaluate) {
+        sourceStarted = true;
+        hcEval.call(options.evaluate.toString());
+        evaluated = true;
+        lexicalComplete = finishSource(hcEval);
+    }
+    for (const file of options._) {
+        if (!lexicalComplete)
+            break;
+        if (typeof file !== "string") {
+            console.error("Invalid file argument", file);
+            continue;
+        }
+        if (sourceStarted)
+            hcEval = hcEval.nextSourceUnit();
+        sourceStarted = true;
+        evaluated = await runfile(hcEval, file);
+        lexicalComplete = finishSource(hcEval);
+    }
+    if (lexicalComplete && (options.interactive || !evaluated)) {
+        if (sourceStarted)
+            hcEval = hcEval.nextSourceUnit();
+        sourceStarted = true;
+        if (hcEval.out instanceof HCLog)
+            hcEval.out.prompt = true;
+        await new Prompt(hcEval).repl();
+        lexicalComplete = finishSource(hcEval);
+    }
+    if (!sourceStarted) {
+        lexicalComplete = finishSource(hcEval);
+    }
+    if (test && lexicalComplete) {
+        test.finish();
+    }
+    return lexicalComplete ? test?.exitCode ?? 0 : 1;
+}
+if (globalThis[Symbol.for("import-meta-ponyfill-esmodule")](import.meta).main) {
+    const env = dntShim.Deno.env.toObject();
+    const options = getOptions(dntShim.Deno.args);
+    const hc_eval = getEval(env);
+    main(hc_eval, options).then((exitCode) => {
+        dntShim.Deno.exit(exitCode);
+    }).catch((err) => {
+        console.error(err);
+        dntShim.Deno.exit(1);
+    });
+}