shell-dsl 0.0.1 → 0.0.3

package/README.md

@@ -1,45 +1,589 @@
 # shell-dsl
 
-## ⚠️ IMPORTANT NOTICE ⚠️
+A sandboxed shell-style DSL for running scriptable command pipelines where all commands are explicitly registered and executed in-process, without access to the host OS.
 
-**This package is created solely for the purpose of setting up OIDC (OpenID Connect) trusted publishing with npm.**
+```ts
+import { createShellDSL, createVirtualFS } from "shell-dsl";
+import { createFsFromVolume, Volume } from "memfs";
+import { builtinCommands } from "shell-dsl/commands";
 
-This is **NOT** a functional package and contains **NO** code or functionality beyond the OIDC setup configuration.
+const vol = new Volume();
+vol.fromJSON({ "/data.txt": "foo\nbar\nbaz\n" });
 
-## Purpose
+const sh = createShellDSL({
+  fs: createVirtualFS(createFsFromVolume(vol)),
+  cwd: "/",
+  env: { USER: "alice" },
+  commands: builtinCommands,
+});
 
-This package exists to:
-1. Configure OIDC trusted publishing for the package name `shell-dsl`
-2. Enable secure, token-less publishing from CI/CD workflows
-3. Establish provenance for packages published under this name
+const count = await sh`cat /data.txt | grep foo | wc -l`.text();
+console.log(count.trim()); // "1"
+```
 
-## What is OIDC Trusted Publishing?
+## Installation
 
-OIDC trusted publishing allows package maintainers to publish packages directly from their CI/CD workflows without needing to manage npm access tokens. Instead, it uses OpenID Connect to establish trust between the CI/CD provider (like GitHub Actions) and npm.
+```bash
+bun add shell-dsl memfs
+```
 
-## Setup Instructions
+## Features
 
-To properly configure OIDC trusted publishing for this package:
+- **Sandboxed execution** — No host OS access; all commands run in-process
+- **Virtual filesystem** — Uses memfs for complete isolation from the real filesystem
+- **Explicit command registry** — Only registered commands can execute
+- **Automatic escaping** — Interpolated values are escaped by default for safety
+- **POSIX-inspired syntax** — Pipes, redirects, control flow operators, and more
+- **Streaming pipelines** — Commands communicate via async iteration
+- **TypeScript-first** — Full type definitions included
 
-1. Go to [npmjs.com](https://www.npmjs.com/) and navigate to your package settings
-2. Configure the trusted publisher (e.g., GitHub Actions)
-3. Specify the repository and workflow that should be allowed to publish
-4. Use the configured workflow to publish your actual package
+## Getting Started
 
-## DO NOT USE THIS PACKAGE
+Create a `ShellDSL` instance by providing a virtual filesystem, working directory, environment variables, and a command registry:
 
-This package is a placeholder for OIDC configuration only. It:
-- Contains no executable code
-- Provides no functionality
-- Should not be installed as a dependency
-- Exists only for administrative purposes
+```ts
+import { createShellDSL, createVirtualFS } from "shell-dsl";
+import { createFsFromVolume, Volume } from "memfs";
+import { builtinCommands } from "shell-dsl/commands";
 
-## More Information
+const vol = new Volume();
+const sh = createShellDSL({
+  fs: createVirtualFS(createFsFromVolume(vol)),
+  cwd: "/",
+  env: { USER: "alice", HOME: "/home/alice" },
+  commands: builtinCommands,
+});
 
-For more details about npm's trusted publishing feature, see:
-- [npm Trusted Publishing Documentation](https://docs.npmjs.com/generating-provenance-statements)
-- [GitHub Actions OIDC Documentation](https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/about-security-hardening-with-openid-connect)
+const greeting = await sh`echo "Hello, $USER"`.text();
+console.log(greeting); // "Hello, alice\n"
+```
 
----
+## Output Methods
 
-**Maintained for OIDC setup purposes only**
+Every shell command returns a `ShellPromise` that can be consumed in different formats:
+
+```ts
+// String output
+await sh`echo hello`.text();           // "hello\n"
+
+// Parsed JSON
+await sh`cat config.json`.json();      // { key: "value" }
+
+// Async line iterator
+for await (const line of sh`cat data.txt`.lines()) {
+  console.log(line);
+}
+
+// Raw Buffer
+await sh`cat binary.dat`.buffer();     // Buffer
+
+// Blob
+await sh`cat image.png`.blob();        // Blob
+```
+
+## Error Handling
+
+By default, commands with non-zero exit codes throw a `ShellError`:
+
+```ts
+import { ShellError } from "shell-dsl";
+
+try {
+  await sh`cat /nonexistent`;
+} catch (err) {
+  if (err instanceof ShellError) {
+    console.log(err.exitCode);          // 1
+    console.log(err.stderr.toString()); // "cat: /nonexistent: ..."
+    console.log(err.stdout.toString()); // ""
+  }
+}
+```
+
+### Disabling Throws
+
+Use `.nothrow()` to suppress throwing for a single command:
+
+```ts
+const result = await sh`cat /nonexistent`.nothrow();
+console.log(result.exitCode); // 1
+```
+
+Use `.throws(boolean)` for explicit control:
+
+```ts
+const result = await sh`cat /nonexistent`.throws(false);
+```
+
+### Global Throw Setting
+
+Disable throwing globally with `sh.throws(false)`:
+
+```ts
+sh.throws(false);
+const result = await sh`cat /nonexistent`;
+console.log(result.exitCode); // 1
+
+// Per-command override still works
+await sh`cat /nonexistent`.throws(true); // This throws
+```
+
+## Piping
+
+Use `|` to connect commands. Data flows between commands via async streams:
+
+```ts
+const result = await sh`cat /data.txt | grep pattern | wc -l`.text();
+```
+
+Each command in the pipeline receives the previous command's stdout as its stdin.
+
+## Control Flow Operators
+
+### Sequential Execution (`;`)
+
+Run commands one after another, regardless of exit codes:
+
+```ts
+await sh`echo one; echo two; echo three`.text();
+// "one\ntwo\nthree\n"
+```
+
+### AND Operator (`&&`)
+
+Run the next command only if the previous one succeeds (exit code 0):
+
+```ts
+await sh`test -f /config.json && cat /config.json`;
+```
+
+### OR Operator (`||`)
+
+Run the next command only if the previous one fails (non-zero exit code):
+
+```ts
+await sh`cat /config.json || echo "default config"`;
+```
+
+### Combined Operators
+
+```ts
+await sh`mkdir -p /out && echo "created" || echo "failed"`;
+```
+
+## Redirection
+
+### Input Redirection (`<`)
+
+Read stdin from a file:
+
+```ts
+await sh`cat < /input.txt`.text();
+```
+
+### Output Redirection (`>`, `>>`)
+
+Write stdout to a file:
+
+```ts
+// Overwrite
+await sh`echo "content" > /output.txt`;
+
+// Append
+await sh`echo "more" >> /output.txt`;
+```
+
+### Stderr Redirection (`2>`, `2>>`)
+
+```ts
+await sh`cmd 2> /errors.txt`;    // stderr to file
+await sh`cmd 2>> /errors.txt`;   // append stderr
+```
+
+### File Descriptor Redirects
+
+| Redirect | Effect |
+|----------|--------|
+| `2>&1` | Redirect stderr to stdout |
+| `1>&2` | Redirect stdout to stderr |
+| `&>` | Redirect both stdout and stderr to file |
+| `&>>` | Append both stdout and stderr to file |
+
+```ts
+// Capture both stdout and stderr
+const result = await sh`cmd 2>&1`.text();
+
+// Write both to file
+await sh`cmd &> /all-output.txt`;
+```
+
+## Environment Variables
+
+### Variable Expansion
+
+Variables are expanded with `$VAR` or `${VAR}` syntax:
+
+```ts
+const sh = createShellDSL({
+  // ...
+  env: { USER: "alice", HOME: "/home/alice" },
+});
+
+await sh`echo $USER`.text();        // "alice\n"
+await sh`echo "Home: $HOME"`.text(); // "Home: /home/alice\n"
+```
+
+### Quoting Semantics
+
+| Quote | Behavior |
+|-------|----------|
+| `"..."` | Variables expanded, special chars preserved |
+| `'...'` | Literal string, no expansion |
+
+```ts
+await sh`echo "Hello $USER"`.text();  // "Hello alice\n"
+await sh`echo 'Hello $USER'`.text();  // "Hello $USER\n"
+```
+
+### Inline Assignment
+
+Assign variables for subsequent commands:
+
+```ts
+await sh`FOO=bar && echo $FOO`.text();  // "bar\n"
+```
+
+Assign variables for a single command (scoped):
+
+```ts
+await sh`FOO=bar echo $FOO`.text();     // "bar\n"
+// FOO is not set after this command
+```
+
+### Per-Command Environment
+
+Override environment for a single command:
+
+```ts
+await sh`echo $CUSTOM`.env({ CUSTOM: "value" }).text();
+```
+
+### Global Environment
+
+Set environment variables globally:
+
+```ts
+sh.env({ API_KEY: "secret" });
+await sh`echo $API_KEY`.text();  // "secret\n"
+
+sh.resetEnv();  // Restore initial environment
+```
+
+## Glob Expansion
+
+Globs are expanded by the interpreter before command execution:
+
+```ts
+await sh`ls *.txt`;           // Matches: a.txt, b.txt, ...
+await sh`cat src/**/*.ts`;    // Recursive glob
+await sh`echo file[123].txt`; // Character classes
+await sh`echo {a,b,c}.txt`;   // Brace expansion: a.txt b.txt c.txt
+```
+
+## Command Substitution
+
+Use `$(command)` to capture command output:
+
+```ts
+await sh`echo "Current dir: $(pwd)"`.text();
+```
+
+Nested substitution is supported:
+
+```ts
+await sh`echo "Files: $(ls $(pwd))"`.text();
+```
+
+## Defining Custom Commands
+
+Commands are async functions that receive a `CommandContext` and return an exit code (0 = success):
+
+```ts
+import type { Command } from "shell-dsl";
+
+const hello: Command = async (ctx) => {
+  const name = ctx.args[0] ?? "World";
+  await ctx.stdout.writeText(`Hello, ${name}!\n`);
+  return 0;
+};
+
+const sh = createShellDSL({
+  // ...
+  commands: { ...builtinCommands, hello },
+});
+
+await sh`hello Alice`.text();  // "Hello, Alice!\n"
+```
+
+### CommandContext Interface
+
+```ts
+interface CommandContext {
+  args: string[];                    // Command arguments
+  stdin: Stdin;                      // Input stream
+  stdout: Stdout;                    // Output stream
+  stderr: Stderr;                    // Error stream
+  fs: VirtualFS;                     // Virtual filesystem
+  cwd: string;                       // Current working directory
+  env: Record<string, string>;       // Environment variables
+}
+```
+
+### Stdin Interface
+
+```ts
+interface Stdin {
+  stream(): AsyncIterable<Uint8Array>;  // Raw byte stream
+  buffer(): Promise<Buffer>;             // All input as Buffer
+  text(): Promise<string>;               // All input as string
+  lines(): AsyncIterable<string>;        // Line-by-line iterator
+}
+```
+
+### Stdout/Stderr Interface
+
+```ts
+interface Stdout {
+  write(chunk: Uint8Array): Promise<void>;  // Write bytes
+  writeText(str: string): Promise<void>;    // Write UTF-8 string
+}
+```
+
+### Example: echo
+
+```ts
+const echo: Command = async (ctx) => {
+  await ctx.stdout.writeText(ctx.args.join(" ") + "\n");
+  return 0;
+};
+```
+
+### Example: cat
+
+Read from stdin or files:
+
+```ts
+const cat: Command = async (ctx) => {
+  if (ctx.args.length === 0) {
+    // Read from stdin
+    for await (const chunk of ctx.stdin.stream()) {
+      await ctx.stdout.write(chunk);
+    }
+  } else {
+    // Read from files
+    for (const file of ctx.args) {
+      const path = ctx.fs.resolve(ctx.cwd, file);
+      const content = await ctx.fs.readFile(path);
+      await ctx.stdout.write(new Uint8Array(content));
+    }
+  }
+  return 0;
+};
+```
+
+### Example: grep
+
+Pattern matching with stdin:
+
+```ts
+const grep: Command = async (ctx) => {
+  const pattern = ctx.args[0];
+  if (!pattern) {
+    await ctx.stderr.writeText("grep: missing pattern\n");
+    return 1;
+  }
+
+  const regex = new RegExp(pattern);
+  let found = false;
+
+  for await (const line of ctx.stdin.lines()) {
+    if (regex.test(line)) {
+      await ctx.stdout.writeText(line + "\n");
+      found = true;
+    }
+  }
+
+  return found ? 0 : 1;
+};
+```
+
+### Example: Custom uppercase command
+
+```ts
+const upper: Command = async (ctx) => {
+  const text = await ctx.stdin.text();
+  await ctx.stdout.writeText(text.toUpperCase());
+  return 0;
+};
+
+// Usage
+await sh`echo "hello" | upper`.text();  // "HELLO\n"
+```
+
+## Built-in Commands
+
+Import all built-in commands:
+
+```ts
+import { builtinCommands } from "shell-dsl/commands";
+```
+
+Or import individually:
+
+```ts
+import { echo, cat, grep, wc, cp, mv, touch, tee } from "shell-dsl/commands";
+```
+
+| Command | Description |
+|---------|-------------|
+| `echo` | Print arguments to stdout |
+| `cat` | Concatenate files or stdin to stdout |
+| `grep` | Linux-compatible pattern search |
+| `wc` | Count lines, words, or characters (`-l`, `-w`, `-c`) |
+| `head` | Output first lines (`-n`) |
+| `tail` | Output last lines (`-n`) |
+| `sort` | Sort lines (`-r` reverse, `-n` numeric) |
+| `uniq` | Remove duplicate adjacent lines (`-c` count) |
+| `pwd` | Print working directory |
+| `ls` | List directory contents |
+| `mkdir` | Create directories (`-p` parents) |
+| `rm` | Remove files/directories (`-r` recursive, `-f` force) |
+| `cp` | Copy files/directories (`-r` recursive, `-n` no-clobber) |
+| `mv` | Move/rename files/directories (`-n` no-clobber) |
+| `touch` | Create empty files or update timestamps (`-c` no-create) |
+| `tee` | Duplicate stdin to stdout and files (`-a` append) |
+| `test` / `[` | File and string tests (`-f`, `-d`, `-e`, `-z`, `-n`, `=`, `!=`) |
+| `true` | Exit with code 0 |
+| `false` | Exit with code 1 |
+
+## Virtual Filesystem
+
+The `VirtualFS` interface wraps memfs for sandboxed file operations:
+
+```ts
+import { createVirtualFS } from "shell-dsl";
+import { createFsFromVolume, Volume } from "memfs";
+
+const vol = new Volume();
+vol.fromJSON({
+  "/data.txt": "file content",
+  "/config.json": '{"key": "value"}',
+});
+
+const fs = createVirtualFS(createFsFromVolume(vol));
+```
+
+### VirtualFS Interface
+
+```ts
+interface VirtualFS {
+  // Reading
+  readFile(path: string): Promise<Buffer>;
+  readdir(path: string): Promise<string[]>;
+  stat(path: string): Promise<FileStat>;
+  exists(path: string): Promise<boolean>;
+
+  // Writing
+  writeFile(path: string, data: Buffer | string): Promise<void>;
+  appendFile(path: string, data: Buffer | string): Promise<void>;
+  mkdir(path: string, opts?: { recursive?: boolean }): Promise<void>;
+
+  // Deletion
+  rm(path: string, opts?: { recursive?: boolean; force?: boolean }): Promise<void>;
+
+  // Utilities
+  resolve(...paths: string[]): string;
+  dirname(path: string): string;
+  basename(path: string): string;
+  glob(pattern: string, opts?: { cwd?: string }): Promise<string[]>;
+}
+```
+
+## Low-Level API
+
+For advanced use cases (custom tooling, AST inspection):
+
+```ts
+// Tokenize shell source
+const tokens = sh.lex("cat foo | grep bar");
+
+// Parse tokens into AST
+const ast = sh.parse(tokens);
+
+// Compile AST to executable program
+const program = sh.compile(ast);
+
+// Execute a compiled program
+const result = await sh.run(program);
+```
+
+### Manual Escaping
+
+```ts
+sh.escape("hello world");    // "'hello world'"
+sh.escape("$(rm -rf /)");    // "'$(rm -rf /)'"
+sh.escape("safe");           // "safe"
+```
+
+### Raw Escape Hatch
+
+Bypass escaping for trusted input:
+
+```ts
+await sh`echo ${{ raw: "$(date)" }}`.text();
+```
+
+**Warning:** Use `{ raw: ... }` with extreme caution when handling untrusted input.
+
+## Safety & Security
+
+1. **No host access** — All commands run in-process against a virtual filesystem
+2. **Automatic escaping** — Interpolated values are escaped by default
+3. **Explicit command registry** — Only registered commands can execute
+4. **No shell spawning** — Never invokes `/bin/sh` or similar
+
+The `{ raw: ... }` escape hatch exists for advanced use cases but should be used with extreme caution.
+
+## TypeScript Types
+
+Key exported types:
+
+```ts
+import type {
+  Command,
+  CommandContext,
+  Stdin,
+  Stdout,
+  Stderr,
+  VirtualFS,
+  FileStat,
+  ExecResult,
+  ShellConfig,
+  RawValue,
+} from "shell-dsl";
+```
+
+## Running Tests
+
+```bash
+bun test
+```
+
+## Typecheck
+
+```bash
+bun run typecheck
+```
+
+## License
+
+MIT

package/dist/cjs/index.cjs

@@ -0,0 +1,72 @@
+var __defProp = Object.defineProperty;
+var __getOwnPropNames = Object.getOwnPropertyNames;
+var __getOwnPropDesc = Object.getOwnPropertyDescriptor;
+var __hasOwnProp = Object.prototype.hasOwnProperty;
+var __reExport = (target, mod, secondTarget) => {
+  for (let key of __getOwnPropNames(mod))
+    if (!__hasOwnProp.call(target, key) && key !== "default")
+      __defProp(target, key, {
+        get: () => mod[key],
+        enumerable: true
+      });
+  if (secondTarget) {
+    for (let key of __getOwnPropNames(mod))
+      if (!__hasOwnProp.call(secondTarget, key) && key !== "default")
+        __defProp(secondTarget, key, {
+          get: () => mod[key],
+          enumerable: true
+        });
+    return secondTarget;
+  }
+};
+var __moduleCache = /* @__PURE__ */ new WeakMap;
+var __toCommonJS = (from) => {
+  var entry = __moduleCache.get(from), desc;
+  if (entry)
+    return entry;
+  entry = __defProp({}, "__esModule", { value: true });
+  if (from && typeof from === "object" || typeof from === "function")
+    __getOwnPropNames(from).map((key) => !__hasOwnProp.call(entry, key) && __defProp(entry, key, {
+      get: () => from[key],
+      enumerable: !(desc = __getOwnPropDesc(from, key)) || desc.enumerable
+    }));
+  __moduleCache.set(from, entry);
+  return entry;
+};
+var __export = (target, all) => {
+  for (var name in all)
+    __defProp(target, name, {
+      get: all[name],
+      enumerable: true,
+      configurable: true,
+      set: (newValue) => all[name] = () => newValue
+    });
+};
+
+// index.ts
+var exports_shell_dsl = {};
+__export(exports_shell_dsl, {
+  wc: () => import_commands2.wc,
+  uniq: () => import_commands2.uniq,
+  trueCmd: () => import_commands2.trueCmd,
+  test: () => import_commands2.test,
+  tail: () => import_commands2.tail,
+  sort: () => import_commands2.sort,
+  rm: () => import_commands2.rm,
+  pwd: () => import_commands2.pwd,
+  mkdir: () => import_commands2.mkdir,
+  ls: () => import_commands2.ls,
+  head: () => import_commands2.head,
+  grep: () => import_commands2.grep,
+  falseCmd: () => import_commands2.falseCmd,
+  echo: () => import_commands2.echo,
+  cat: () => import_commands2.cat,
+  builtinCommands: () => import_commands.builtinCommands,
+  bracket: () => import_commands2.bracket
+});
+module.exports = __toCommonJS(exports_shell_dsl);
+__reExport(exports_shell_dsl, require("./src/index.cjs"), module.exports);
+var import_commands = require("./commands/index.cjs");
+var import_commands2 = require("./commands/index.cjs");
+
+//# debugId=01F96ADDEEC727E964756E2164756E21

package/dist/cjs/index.cjs.map

@@ -0,0 +1,10 @@
+{
+  "version": 3,
+  "sources": ["../../index.ts"],
+  "sourcesContent": [
+    "// Re-export everything from src\nexport * from \"./src/index.cjs\";\n\n// Re-export built-in commands\nexport { builtinCommands } from \"./commands/index.cjs\";\nexport {\n  echo,\n  cat,\n  grep,\n  wc,\n  head,\n  tail,\n  sort,\n  uniq,\n  pwd,\n  ls,\n  mkdir,\n  rm,\n  test,\n  bracket,\n  trueCmd,\n  falseCmd,\n} from \"./commands/index.cjs\";\n"
+  ],
+  "mappings": ";;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;AACA;AAGgC,IAAhC;AAkBO,IAjBP;",
+  "debugId": "01F96ADDEEC727E964756E2164756E21",
+  "names": []
+}

package/dist/cjs/package.json

@@ -0,0 +1,5 @@
+{
+  "name": "shell-dsl",
+  "version": "0.0.3",
+  "type": "commonjs"
+}