@b9g/shovel 0.2.0-beta.10 → 0.2.0-beta.11

package/CHANGELOG.md

@@ -0,0 +1,160 @@
+# Changelog
+
+All notable changes to Shovel will be documented in this file.
+
+## [0.2.0-beta.11] - 2026-01-10
+
+### Changes since beta.10
+- **ESBuild configuration support (#18)** - Custom esbuild options in shovel.json
+- **Config expression syntax** - `$PORT || 3000`, `$DATABASE_URL ?? null`
+- **Null fallback fix** - Allow intentional null fallbacks in config expressions
+- **DatabaseStorage API redesign** - New open/get pattern with IndexedDB-style migrations
+- **Migrated from Drizzle to @b9g/zen** - Simpler, more portable database layer
+- **Logging DX improvements** - Better defaults, consolidated categories
+- **`impl` key unification** - Simplified resource configuration
+- **CI/lint enforcement** - ESLint and Prettier standardized
+- **Documentation** - Comprehensive docs for all APIs
+
+### Package Updates
+- `@b9g/router` → 0.2.0-beta.1 (CORS middleware, trailingSlash moved)
+- `@b9g/node-webworker` → 0.2.0-beta.1 (CloseEvent, onclose, env option)
+- `@b9g/cache-redis` → 0.2.0-beta.1 (logger category fix)
+- `@b9g/assets` → 0.2.0-beta.0
+- `@b9g/async-context` → 0.2.0-beta.0
+- `@b9g/cache` → 0.2.0-beta.0
+- `@b9g/http-errors` → 0.2.0-beta.0
+- `@b9g/match-pattern` → 0.2.0-beta.0
+
+---
+
+## [0.2.0-beta.10] - Previous Beta
+
+This is a major release that establishes Shovel as a complete ServiceWorker-based meta-framework. The 0.2.0 beta introduces locked-down APIs for core packages, a unified configuration system, and comprehensive platform support.
+
+### Breaking Changes
+
+- **`self.buckets` renamed to `self.directories`** - The file system API now uses `directories` to align with web standards terminology
+- **`@b9g/router` middleware moved** - `trailingSlash` middleware moved from main export to `@b9g/router/middleware`
+- **`self.loggers.get()` signature changed** - Now takes an array: `self.loggers.get(["app", "db"])` instead of dot notation
+- **Config `module`/`export` unified to `impl`** - Resource configurations now use a single `impl` key for reified implementations
+
+### New Features
+
+#### Consistent Worker Execution Model (#17)
+ServiceWorker code now ALWAYS runs in a worker thread, never the main thread. This ensures:
+- Same globals/environment in dev and prod (eliminates mode-only bugs)
+- Worker isolation preserved
+- Simplified mental model
+
+#### ESBuild Configuration Support (#18)
+Custom ESBuild options can now be specified in `shovel.json`:
+```json
+{
+  "esbuild": {
+    "external": ["lightningcss"],
+    "define": { "DEBUG": "true" }
+  }
+}
+```
+
+#### Config Expression Syntax
+Environment variables and expressions in `shovel.json`:
+```json
+{
+  "port": "$PORT || 3000",
+  "databases": {
+    "main": {
+      "url": "$DATABASE_URL"
+    }
+  }
+}
+```
+
+#### Comprehensive Logging System
+- Built-in LogTape integration with console sink by default
+- Configurable sinks (file, OpenTelemetry, Sentry, etc.)
+- Category-based log levels
+- `shovel` category logs at `info` level by default
+
+#### Database Storage API
+New `self.databases` API with IndexedDB-style migrations:
+```typescript
+self.addEventListener("activate", (event) => {
+  event.waitUntil(
+    databases.open("main", 2, (e) => {
+      e.waitUntil(e.db.exec`CREATE TABLE users (...)`);
+    })
+  );
+});
+
+// In fetch handlers
+const db = databases.get("main");
+const users = await db.all`SELECT * FROM users`;
+```
+
+#### CORS Middleware
+New CORS middleware in `@b9g/router/middleware`:
+```typescript
+import { cors } from "@b9g/router/middleware";
+router.use(cors({ origin: "https://example.com" }));
+```
+
+### Package Updates
+
+#### Locked at 0.2.0-beta.0 (API stable)
+- `@b9g/async-context` - AsyncContext polyfill
+- `@b9g/match-pattern` - URLPattern implementation
+- `@b9g/assets` - Static asset pipeline with content hashing
+- `@b9g/cache` - Cache API implementation (memory, postmessage)
+- `@b9g/http-errors` - HTTP error classes
+
+#### Updated to 0.2.0-beta.1
+- `@b9g/router` - Added CORS middleware, moved trailingSlash to /middleware
+- `@b9g/node-webworker` - Added CloseEvent, onclose handler, env option
+- `@b9g/cache-redis` - Logger category updates
+
+#### Still Evolving (0.1.x)
+- `@b9g/platform` - Core runtime and platform abstraction
+- `@b9g/platform-bun` - Bun platform adapter
+- `@b9g/platform-node` - Node.js platform adapter
+- `@b9g/platform-cloudflare` - Cloudflare Workers adapter
+- `@b9g/filesystem` - File System Access API implementation
+- `@b9g/filesystem-s3` - S3 filesystem adapter
+
+### CLI Changes
+
+- `shovel develop` - Development server with hot reload (note: `dev` alias removed)
+- `shovel build` - Production build
+- `shovel activate` - Static site generation
+- Removed `--verbose` flags (use logging config instead)
+
+### Infrastructure
+
+- Migrated from Drizzle ORM to `@b9g/zen` for database operations
+- Added GitHub Actions CI with parallel test execution
+- ESLint and Prettier configuration standardized
+- Comprehensive test suites with `bun:test`
+
+### Documentation
+
+New documentation pages:
+- Getting Started guide
+- CLI reference
+- Configuration (shovel.json) reference
+- Deployment guide
+- ServiceWorker lifecycle
+- Routing and middleware
+- Storage APIs (databases, caches, directories)
+- Cookies and AsyncContext
+
+---
+
+## [0.1.x] - Previous Releases
+
+Initial development releases establishing core architecture:
+- ServiceWorker-based request handling
+- Platform abstraction layer
+- Router with generator-based middleware
+- Cache API implementations
+- File System Access API
+- Hot reload in development

package/README.md

@@ -1,49 +1,23 @@
-# Shovel
+# Shovel.js 🪏
 
-**The ServiceWorker platform for server-side JavaScript.**
+**The portable meta-framework built on web standards.**
 
-Same code. Any runtime. Node.js, Bun, Cloudflare Workers.
+Shovel is a CLI platform for developing and deploying service workers as application servers.
 
 ```javascript
-// app.js
-self.addEventListener("fetch", (event) => {
-  event.respondWith(new Response("Hello World"));
+import {Router} from "@b9g/router";
+const router = new Router();
+
+router.route("/").get(() => new Response("Hello world"));
+
+self.addEventListener("fetch", (ev) => {
+  ev.respondWith(router.handle(ev.request));
 });
 ```
 
 ```bash
-npx @b9g/shovel develop app.js
+shovel develop app.js
 ```
-
-## Why Shovel?
-
-Browsers have ServiceWorker. Cloudflare has Workers. Node.js and Bun have... Express?
-
-Shovel brings the ServiceWorker programming model to server-side JavaScript. Write your app once using web standards, deploy it anywhere.
-
-## Web Standards
-
-Shovel implements web platform APIs that server-side JavaScript is missing:
-
-| API | Standard | What it does |
-|-----|----------|--------------|
-| `fetch` event | [Service Workers](https://w3c.github.io/ServiceWorker/) | Request handling |
-| `self.caches` | [Cache API](https://w3c.github.io/ServiceWorker/#cache-interface) | Response caching |
-| `self.directories` | [FileSystem API](https://fs.spec.whatwg.org/) | Storage (local, S3, R2) |
-| `self.cookieStore` | [Cookie Store API](https://wicg.github.io/cookie-store/) | Cookie management |
-| `URLPattern` | [URLPattern](https://urlpattern.spec.whatwg.org/) | Route matching (100% WPT) |
-| `AsyncContext.Variable` | [TC39 Stage 2](https://github.com/tc39/proposal-async-context) | Request-scoped state |
-
-Your code uses standards. Shovel makes them work everywhere.
-
-## True Portability
-
-Shovel is a complete meta-framework. Same code, any runtime, any rendering strategy:
-
-- **Server runtimes**: Node.js, Bun, Cloudflare Workers for development and production
-- **Browser ServiceWorkers**: The same app can run as a PWA service worker
-- **Universal rendering**: Dynamic, static, or client-side - link and deploy assets automatically
-
 ## Quick Start
 
 ```javascript
@@ -54,12 +28,12 @@ const router = new Router();
 
 router.route("/").get(() => new Response("Hello World"));
 
-router.route("/users/:id").get((request, {params}) => {
-  return Response.json({id: params.id});
+router.route("/greet/:name").get((request, {params}) => {
+  return new Response(`Hello ${params.name}`);
 });
 
 self.addEventListener("fetch", (event) => {
-  event.respondWith(router.handler(event.request));
+  event.respondWith(router.handle(event.request));
 });
 ```
 
@@ -76,6 +50,51 @@ npx @b9g/shovel build app.js --platform=bun
 npx @b9g/shovel build app.js --platform=cloudflare
 ```
 
+
+## Web Standards
+Shovel is obsessively standards-first. All Shovel APIs use web standards, and Shovel implements/shims useful standards when they're missing.
+
+  | API | Standard | Purpose |
+  |-----|----------|--------------|
+  | `fetch()` | [Fetch](https://fetch.spec.whatwg.org) | Networking |
+  | `"install"`, `"activate"`, `"fetch"` events | [Service Workers](https://w3c.github.io/ServiceWorker/) | Server lifecycle |
+  | `AsyncContext.Variable` | [TC39 Stage 2](https://github.com/tc39/proposal-async-context) | Request-scoped state |
+  | `self.caches` | [Cache API](https://w3c.github.io/ServiceWorker/#cache-interface) | Response caching |
+  | `self.directories` | [FileSystem API](https://fs.spec.whatwg.org/) | Storage (local, S3, R2) |
+  | `self.cookieStore` | [CookieStore API](https://cookiestore.spec.whatwg.org) | Cookie management |
+  | `URLPattern` | [URLPattern](https://urlpattern.spec.whatwg.org/) | Route matching |
+
+Your code uses standards. Shovel makes them work everywhere.
+
+## Meta-Framework
+
+Shovel is a meta-framework: it provides and implements primitives rather than opinions. Instead of dictating how you build, it gives you portable building blocks that work everywhere.
+
+## True Portability
+
+Same code, any runtime, any rendering strategy:
+
+- **Server runtimes**: Node.js, Bun, Cloudflare Workers
+- **Browser ServiceWorkers**: The same app can run as a PWA
+- **Universal rendering**: Dynamic, static, or client-side
+
+The core abstraction is the **ServiceWorker-style storage pattern**. Globals provide a consistent API for common web concerns:
+
+```javascript
+const cache  = await self.caches.open("sessions");     // Cache API
+const dir    = await self.directories.open("uploads"); // FileSystem API
+const db     = self.databases.get("main");             // Zen DB (opened on activate)
+const logger = self.loggers.get(["app", "requests"]); // LogTape
+```
+
+Each storage type is:
+- **Lazy** - connections created on first `open()`, cached thereafter
+- **Configured uniformly** - all are configured by `shovel.json`
+- **Platform-aware** - sensible defaults per platform, override what you need
+
+This pattern means your app logic stays clean. Swap Redis for memory cache, S3 for local filesystem, Postgres for SQLite - change the config, not the code.
+
+
 ## Platform APIs
 
 ```javascript
@@ -121,6 +140,247 @@ Assets are served via the platform's best option:
 - **Cloudflare**: Workers Assets (edge-cached, zero config)
 - **Node/Bun**: Static file middleware or directory storage
 
+## Configuration
+
+Configure Shovel using `shovel.json` in your project root.
+
+### Philosophy
+
+Shovel's configuration follows these principles:
+
+1. **Platform Defaults, User Overrides** - Each platform provides sensible defaults. You only configure what you want to change.
+
+2. **Uniform Interface** - Caches, directories, databases, and loggers all use the same `{ module, export, ...options }` pattern. No magic strings or builtin aliases.
+
+3. **Layered Resolution** - For any cache or directory name:
+   - If config specifies `module`/`export` → use that
+   - Otherwise → use platform default
+
+4. **Platform Re-exports** - Each platform exports `DefaultCache` representing what makes sense for that environment:
+   - Cloudflare: Native Cache API
+   - Bun/Node: MemoryCache
+
+5. **Transparency** - Config is what you see. Every backend is an explicit module path, making it easy to debug and trace.
+
+### Basic Config
+
+```json
+{
+  "port": "PORT || 3000",
+  "host": "HOST || localhost",
+  "workers": "WORKERS ?? 1",
+  "caches": {
+    "sessions": {
+      "module": "@b9g/cache-redis",
+      "export": "RedisCache",
+      "url": "REDIS_URL"
+    }
+  },
+  "directories": {
+    "uploads": {
+      "module": "@b9g/filesystem-s3",
+      "export": "S3Directory",
+      "bucket": "S3_BUCKET"
+    }
+  },
+  "databases": {
+    "main": {
+      "module": "@b9g/zen/bun",
+      "url": "DATABASE_URL"
+    }
+  },
+  "logging": {
+    "loggers": [
+      {"category": ["app"], "level": "info", "sinks": ["console"]}
+    ]
+  }
+}
+```
+
+### Caches
+
+Configure cache backends using `module` and `export`:
+
+```json
+{
+  "caches": {
+    "api-responses": {
+      "module": "@b9g/cache/memory",
+      "export": "MemoryCache"
+    },
+    "sessions": {
+      "module": "@b9g/cache-redis",
+      "export": "RedisCache",
+      "url": "REDIS_URL"
+    }
+  }
+}
+```
+
+- **Default**: Platform's `DefaultCache` when no config specified (MemoryCache on Bun/Node, native on Cloudflare)
+- **Pattern matching**: Use wildcards like `"api-*"` to match multiple cache names
+- **Empty config**: `"my-cache": {}` uses platform default explicitly
+
+### Directories
+
+Configure directory backends. Platforms provide defaults for well-known directories (`server`, `public`, `tmp`):
+
+```json
+{
+  "directories": {
+    "uploads": {
+      "module": "@b9g/filesystem-s3",
+      "export": "S3Directory",
+      "bucket": "MY_BUCKET",
+      "region": "us-east-1"
+    },
+    "data": {
+      "module": "@b9g/filesystem/node-fs",
+      "export": "NodeFSDirectory",
+      "path": "./data"
+    }
+  }
+}
+```
+
+- **Well-known defaults**: `server` (dist/server), `public` (dist/public), `tmp` (OS temp)
+- **Custom directories**: Must be explicitly configured
+
+### Logging
+
+Shovel uses [LogTape](https://logtape.org/) for logging:
+
+```typescript
+const logger = self.loggers.get(["shovel", "myapp"]);
+logger.info`Request received: ${request.url}`;
+```
+
+**Zero-config logging**: Use the `["shovel", ...]` category hierarchy to inherit Shovel's default logging (info level to console). No configuration needed.
+
+For custom configuration, use `shovel.json`:
+
+```json
+{
+  "logging": {
+    "sinks": {
+      "file": {
+        "module": "@logtape/logtape",
+        "export": "getFileSink",
+        "path": "./logs/app.log"
+      }
+    },
+    "loggers": [
+      {"category": ["myapp"], "level": "info", "sinks": ["console"]},
+      {"category": ["myapp", "db"], "level": "debug", "sinks": ["file"]}
+    ]
+  }
+}
+```
+
+- **Console sink is implicit** - always available as `"console"`
+- **Category hierarchy** - `["myapp", "db"]` inherits from `["myapp"]`
+- **parentSinks** - use `"override"` to replace parent sinks instead of inheriting
+
+### Databases
+
+Configure database drivers using the same `module`/`export` pattern:
+
+```json
+{
+  "databases": {
+    "main": {
+      "module": "@b9g/zen/bun",
+      "url": "DATABASE_URL"
+    }
+  }
+}
+```
+
+Open databases in `activate` (for migrations), then use `get()` in requests:
+
+```javascript
+self.addEventListener("activate", (event) => {
+  event.waitUntil(self.databases.open("main", 1, (e) => {
+    e.waitUntil(runMigrations(e));
+  }));
+});
+
+self.addEventListener("fetch", (event) => {
+  const db = self.databases.get("main");
+});
+```
+
+### Expression Syntax
+
+Configuration values support a domain-specific expression language that generates JavaScript code evaluated at runtime.
+
+#### Environment Variables
+
+```
+$VAR                    → process.env.VAR
+$VAR || fallback        → process.env.VAR || "fallback"
+$VAR ?? fallback        → process.env.VAR ?? "fallback"
+```
+
+#### Bracket Placeholders
+
+| Placeholder | Description | Resolution |
+|-------------|-------------|------------|
+| `[outdir]` | Build output directory | Build time |
+| `[tmpdir]` | OS temp directory | Runtime |
+| `[git]` | Git commit SHA | Build time |
+
+The bracket syntax mirrors esbuild/webpack output filename templating (`[name]`, `[hash]`).
+
+#### Operators
+
+| Operator | Example | Description |
+|----------|---------|-------------|
+| `\|\|` | `$VAR \|\| default` | Logical OR (falsy fallback) |
+| `??` | `$VAR ?? default` | Nullish coalescing |
+| `&&` | `$A && $B` | Logical AND |
+| `? :` | `$ENV === prod ? a : b` | Ternary conditional |
+| `===`, `!==` | `$ENV === production` | Strict equality |
+| `!` | `!$DISABLED` | Logical NOT |
+
+#### Path Expressions
+
+Path expressions support path segments and relative resolution:
+
+```
+$DATADIR/uploads        → joins env var with path segment
+[outdir]/server         → joins build output with path segment
+./data                  → resolved to absolute path at build time
+```
+
+#### Example
+
+```json
+{
+  "port": "$PORT || 3000",
+  "host": "$HOST || 0.0.0.0",
+  "directories": {
+    "server": { "path": "[outdir]/server" },
+    "public": { "path": "[outdir]/public" },
+    "tmp": { "path": "[tmpdir]" },
+    "data": { "path": "./data" },
+    "cache": { "path": "($CACHE_DIR || [tmpdir])/myapp" }
+  },
+  "cache": {
+    "provider": "$NODE_ENV === production ? redis : memory"
+  }
+}
+```
+
+Dynamic values (containing `$VAR` or `[tmpdir]`) use getters to ensure evaluation at access time, not module load time.
+
+### Access in Code
+
+```javascript
+import {config} from "shovel:config";
+console.log(config.port); // Resolved value
+```
+
 ## Packages
 
 | Package | Description |
@@ -128,7 +388,7 @@ Assets are served via the platform's best option:
 | `@b9g/shovel` | CLI for development and deployment |
 | `@b9g/platform` | Core runtime and platform APIs |
 | `@b9g/platform-node` | Node.js adapter |
-| `@b9g/platform-bun` | Bun adapter |
+| `@b9g/platform-bun` | Bun.js adapter |
 | `@b9g/platform-cloudflare` | Cloudflare Workers adapter |
 | `@b9g/router` | URLPattern-based routing with middleware |
 | `@b9g/cache` | Cache API implementation |
@@ -136,7 +396,6 @@ Assets are served via the platform's best option:
 | `@b9g/match-pattern` | URLPattern with extensions (100% WPT) |
 | `@b9g/async-context` | AsyncContext.Variable implementation |
 | `@b9g/http-errors` | Standard HTTP error classes |
-| `@b9g/auth` | OAuth2/PKCE and CORS middleware |
 | `@b9g/assets` | Static asset handling |
 
 ## License

package/bin/cli.js

@@ -5,41 +5,61 @@ import {
   DEFAULTS,
   findProjectRoot,
   loadConfig
-} from "../chunk-CSH7M4MK.js";
+} from "../src/_chunks/chunk-GRAFMTEH.js";
 
 // bin/cli.ts
+import { resolve } from "path";
 import { configureLogging } from "@b9g/platform/runtime";
 import { Command } from "commander";
 import pkg from "../package.json" with { type: "json" };
 var projectRoot = findProjectRoot();
 var config = loadConfig(projectRoot);
-await configureLogging(config.logging);
+async function reifySinks(sinks, baseDir) {
+  const reified = {};
+  for (const [name, sinkConfig] of Object.entries(sinks ?? {})) {
+    const { module: modulePath, export: exportName, ...rest } = sinkConfig;
+    if (modulePath) {
+      const resolvedPath = modulePath.startsWith("./") || modulePath.startsWith("../") ? resolve(baseDir, modulePath) : modulePath;
+      const mod = await import(resolvedPath);
+      const impl = exportName ? mod[exportName] : mod.default;
+      reified[name] = { ...rest, impl };
+    } else if (sinkConfig.impl) {
+      reified[name] = sinkConfig;
+    }
+  }
+  return reified;
+}
+var reifiedSinks = await reifySinks(config.logging?.sinks, projectRoot);
+await configureLogging({
+  sinks: reifiedSinks,
+  loggers: config.logging?.loggers
+});
 var program = new Command();
 program.name("shovel").description("Shovel CLI").version(pkg.version);
 program.command("develop <entrypoint>").description("Start development server with hot reload").option("-p, --port <port>", "Port to listen on", DEFAULTS.SERVER.PORT).option("-h, --host <host>", "Host to bind to", DEFAULTS.SERVER.HOST).option(
   "-w, --workers <count>",
   "Number of workers (default: CPU cores)",
   DEFAULTS.WORKERS
-).option("-v, --verbose", "Verbose logging", false).option("--platform <name>", "Runtime platform (node, cloudflare, bun)").action(async (entrypoint, options) => {
-  const { developCommand } = await import("../develop-5ORIPB7M.js");
+).option("--platform <name>", "Runtime platform (node, cloudflare, bun)").action(async (entrypoint, options) => {
+  const { developCommand } = await import("../src/_chunks/develop-A7EU2ZDY.js");
   await developCommand(entrypoint, options, config);
 });
-program.command("build <entrypoint>").description("Build app for production").option("-w, --workers <count>", "Worker count (defaults to 1)", void 0).option("-v, --verbose", "Verbose logging", false).option("--platform <name>", "Runtime platform (node, cloudflare, bun)").action(async (entrypoint, options) => {
-  const { buildCommand } = await import("../build-NDUV2F2Z.js");
+program.command("build <entrypoint>").description("Build app for production").option("-w, --workers <count>", "Worker count (defaults to 1)", void 0).option("--platform <name>", "Runtime platform (node, cloudflare, bun)").action(async (entrypoint, options) => {
+  const { buildCommand } = await import("../src/_chunks/build-V3IPZGKC.js");
   await buildCommand(entrypoint, options, config);
 });
 program.command("activate <entrypoint>").description(
   "Activate ServiceWorker (for static site generation in activate event)"
-).option("-v, --verbose", "Verbose logging", false).option("--platform <name>", "Runtime platform (node, cloudflare, bun)").option(
+).option("--platform <name>", "Runtime platform (node, cloudflare, bun)").option(
   "-w, --workers <count>",
   "Number of workers",
   DEFAULTS.WORKERS.toString()
 ).action(async (entrypoint, options) => {
-  const { activateCommand } = await import("../activate-5LWUTBLL.js");
+  const { activateCommand } = await import("../src/_chunks/activate-TP6RQP47.js");
   await activateCommand(entrypoint, options, config);
 });
 program.command("info").description("Display platform and runtime information").action(async () => {
-  const { infoCommand } = await import("../info-PRYEMZS4.js");
+  const { infoCommand } = await import("../src/_chunks/info-TDUY3FZN.js");
   await infoCommand();
 });
 program.parse();