luca 2.0.0 → 3.0.0

package/docs/tutorials/00-bootstrap.md

@@ -0,0 +1,166 @@
+---
+title: "Bootstrap: Learning the Container at Runtime"
+tags:
+  - bootstrap
+  - introspection
+  - repl
+  - agent
+  - discovery
+  - quickstart
+---
+# Bootstrap: Learning the Container at Runtime
+
+You don't need to memorize the Luca API. The container tells you everything it can do — at runtime. This tutorial teaches you the discovery pattern so you can explore any feature, client, server, or command without reading docs.
+
+## Start with `luca eval`
+
+The `eval` command runs JavaScript with the container in scope. All features are available as top-level variables.
+
+```bash
+# What features are available?
+luca eval "container.features.available"
+# => ['fs', 'git', 'proc', 'vm', 'networking', 'os', 'grep', ...]
+
+# What clients?
+luca eval "container.clients.available"
+
+# What servers?
+luca eval "container.servers.available"
+
+# What commands?
+luca eval "container.commands.available"
+```
+
+## Describe Anything
+
+The `luca describe` command generates API docs for any helper. It reads JSDoc, Zod schemas, and method signatures to produce markdown documentation.
+
+```bash
+# Describe the container itself
+luca describe
+
+# Describe a feature
+luca describe fs
+
+# Describe multiple at once
+luca describe git fs proc
+
+# Show only specific sections
+luca describe fs --methods --examples
+
+# Describe all features
+luca describe features
+```
+
+In code, the same works via registries:
+
+```js
+container.features.describe('fs')       // markdown docs for fs
+container.features.describeAll()        // condensed overview of all features
+container.clients.describe('rest')      // docs for the rest client
+```
+
+## The Discovery Pattern
+
+Every registry follows the same shape. Once you know the pattern, you can explore anything:
+
+```js
+// List what's available
+container.features.available
+container.clients.available
+container.servers.available
+container.commands.available
+
+// Get docs for a specific helper
+container.features.describe('fs')
+container.clients.describe('rest')
+container.servers.describe('express')
+
+// Check if something exists
+container.features.has('fs')           // => true
+
+// Get a helper instance
+const fs = container.feature('fs')
+const rest = container.client('rest')
+```
+
+## Instance Introspection
+
+Once you have a helper instance, it can describe itself:
+
+```js
+const fs = container.feature('fs')
+
+// Structured introspection (object with methods, getters, events, state, options)
+fs.introspect()
+
+// Human-readable markdown
+fs.introspectAsText()
+```
+
+The container itself is introspectable:
+
+```js
+container.introspect()          // structured object with all registries, state, events
+container.introspectAsText()    // full markdown overview
+```
+
+## The REPL
+
+For interactive exploration, use `luca console`. It gives you a persistent REPL with the container and all features in scope:
+
+```bash
+luca console
+```
+
+Inside the REPL, you can tab-complete, call methods, and explore interactively. Variables survive across lines.
+
+## Feature Shortcuts
+
+In eval and REPL contexts, core features are available as top-level variables — no need to call `container.feature()`:
+
+```bash
+luca eval "fs.readFile('package.json')"
+luca eval "git.branch"
+luca eval "proc.exec('ls')"
+luca eval "grep.search('.', 'TODO')"
+```
+
+## Quick Reference
+
+| Want to know...                | Ask                                    |
+|--------------------------------|----------------------------------------|
+| What registries exist?         | `container.registries`                 |
+| What features are available?   | `container.features.available`         |
+| Full docs for a feature?       | `container.features.describe('fs')`    |
+| All features at a glance?      | `container.features.describeAll()`     |
+| Structured introspection?      | `feature.introspect()`                 |
+| What state does it have?       | `feature.state.current`               |
+| What events does it emit?      | `feature.introspect().events`          |
+| Full container overview?       | `container.introspectAsText()`            |
+| CLI docs for a helper?         | `luca describe <name>`                 |
+
+## Gotchas
+
+### `paths.join()` vs `paths.resolve()`
+
+`container.paths.join()` and `container.paths.resolve()` are Node's `path.join` and `path.resolve` curried with `container.cwd`. This means `paths.join()` always prepends `cwd` — even if you pass an absolute path as the first argument.
+
+```js
+// WRONG — paths.join will prepend cwd to the absolute tmpdir path
+const bad = container.paths.join(os.tmpdir, 'mydir')
+// => "/your/project/tmp/mydir" (not what you want)
+
+// RIGHT — paths.resolve respects absolute first args
+const good = container.paths.resolve(os.tmpdir, 'mydir')
+// => "/tmp/mydir"
+```
+
+**Rule of thumb:** Use `paths.join()` for project-relative paths, `paths.resolve()` when the base is already absolute.
+
+## What's Next
+
+- [The Container](./02-container.md) — deep dive into state, events, and lifecycle
+- [Scripts](./03-scripts.md) — run scripts and executable markdown notebooks
+- [Features Overview](./04-features-overview.md) — explore built-in features
+- [Writing Commands](./08-commands.md) — add CLI commands to your project

package/docs/tutorials/01-getting-started.md

@@ -0,0 +1,106 @@
+---
+title: Getting Started with Luca
+tags: [setup, quickstart, project, init]
+---
+
+# Getting Started with Luca
+
+## Prerequisites
+
+- [Bun](https://bun.sh) installed (Luca's runtime)
+- A new or existing bun project
+
+## Create a New Project
+
+```bash
+mkdir my-app && cd my-app
+bun init -y
+bun add @soederpop/luca 
+```
+
+## Project Structure
+
+A typical Luca project looks like this:
+
+```
+my-app/
+├── package.json
+├── endpoints/          # File-based HTTP routes (auto-discovered by `luca serve`)
+│   ├── health.ts
+│   └── users.ts
+├── commands/           # Project-local CLI commands (auto-discovered by `luca`)
+│   └── seed.ts
+├── assistants/         # AI assistants (file-based convention)
+│   └── my-helper/
+│       ├── CORE.md
+│       ├── tools.ts
+│       ├── hooks.ts
+│       └── docs/
+├── public/             # Static files served by `luca serve`
+│   └── index.html
+└── scripts/            # Standalone scripts that use the container
+    └── migrate.ts
+```
+
+## The Container
+
+Everything in Luca revolves around the **container**. It is a per-process singleton that acts as your dependency injector, event bus, and state machine.
+
+In scripts, you create one directly:
+
+```typescript
+import container from '@soederpop/luca/node'
+
+// Now you have access to all features
+const fs = container.fs           // File system operations
+const git = container.git         // Git utilities (branch, sha, lsFiles, etc.)
+const ui = container.ui           // Terminal UI (colors, prompts, figlet)
+const proc = container.feature('proc')  // Process execution
+```
+
+In endpoints and commands, the container is provided for you via context:
+
+```typescript
+// endpoints/health.ts
+export const path = '/health'
+
+export async function get(_params: any, ctx: EndpointContext) {
+  const { container } = ctx
+  return { status: 'ok', uptime: process.uptime() }
+}
+```
+
+## Running Your Project
+
+### Start the API server
+
+```bash
+luca serve
+# or with options:
+luca serve --port 4000 --endpointsDir src/endpoints
+```
+
+This auto-discovers your `endpoints/` directory, mounts all routes, and generates an OpenAPI spec at `/openapi.json`.
+
+### Run a CLI command
+
+```bash
+luca seed --count 10
+```
+
+This auto-discovers `commands/seed.ts` from your project and runs it.
+
+### Run a script
+
+```bash
+luca run scripts/migrate.ts
+```
+
+## What's Next
+
+- [The Container](./02-container.md) -- deep dive into the container
+- [Scripts and Markdown Notebooks](./03-scripts.md) -- run scripts and executable markdown
+- [Using Features](./04-features-overview.md) -- explore built-in features
+- [Servers](./06-servers.md) -- set up Express and WebSocket servers
+- [Writing Endpoints](./07-endpoints.md) -- build your API routes
+- [Writing Commands](./08-commands.md) -- add CLI commands to your project

package/docs/tutorials/02-container.md

@@ -0,0 +1,210 @@
+---
+title: The Container
+tags: [container, singleton, state, events, registries, dependency-injection]
+---
+
+# The Container
+
+The container is the heart of every Luca application. It is a per-process singleton that provides:
+
+- **Dependency injection** via factory methods and registries
+- **Observable state** that you can watch for changes
+- **Event bus** for decoupled communication
+- **Registries** for discovering available helpers
+
+## Getting the Container
+
+```typescript
+import container from '@soederpop/luca'
+```
+
+The import resolves automatically based on environment -- `@soederpop/luca` gives you a `NodeContainer` on the server and a `WebContainer` in browser builds. You can also be explicit:
+
+```typescript
+import container from '@soederpop/luca/node'  // Always NodeContainer
+import container from '@soederpop/luca/web'   // Always WebContainer
+```
+
+The NodeContainer comes pre-loaded with registries for features, clients, servers, commands, and endpoints. Core features like `fs`, `git`, `proc`, `os`, `networking`, `ui`, and `vm` are auto-enabled.
+
+## Registries
+
+Every helper type has a registry. Registries let you discover what's available and create instances:
+
+```typescript
+// What features are available?
+container.features.available
+// => ['fs', 'git', 'proc', 'vm', 'ui', 'networking', 'os', 'diskCache', 'contentDb', ...]
+
+// Get documentation for a feature
+container.features.describe('fs')
+
+// Get documentation for all features
+container.features.describeAll()
+
+// Check if something is registered
+container.features.has('diskCache')
+
+// Same pattern for all helper types:
+container.servers.available    // ['express', 'websocket']
+container.clients.available    // ['rest', 'graph', 'websocket']
+container.commands.available   // ['serve', 'run', ...]
+```
+
+## Factory Methods
+
+Create helper instances through the container's factory methods:
+
+```typescript
+// Features (cached by id + options hash)
+const fs = container.feature('fs')
+const cache = container.feature('diskCache', { path: './cache' })
+
+// Servers
+const server = container.server('express', { port: 3000, cors: true })
+
+// Clients
+const api = container.client('rest', { baseURL: 'https://api.example.com' })
+```
+
+Factory results are **cached**. Calling `container.feature('fs')` twice returns the same instance. Different options produce different instances.
+
+## Enabled Features (Shortcuts)
+
+Some features are "enabled" on the container, giving them shortcut access:
+
+```typescript
+// These are equivalent:
+container.feature('fs')
+container.fs
+
+// Auto-enabled features:
+container.fs           // File system
+container.git          // Git operations
+container.proc         // Process execution
+container.vm           // JavaScript VM
+container.ui           // Terminal UI
+container.os           // OS info
+container.networking   // Port finding, availability
+```
+
+To enable your own feature:
+
+```typescript
+const myFeature = container.feature('myFeature', { enable: true })
+// Now accessible as container.myFeature
+```
+
+## Observable State
+
+The container (and every helper) has observable state:
+
+```typescript
+// Set state
+container.state.set('ready', true)
+
+// Get state
+container.state.get('ready') // true
+
+// Get a snapshot of all state
+container.state.current
+
+// Observe all changes (changeType is 'add' | 'update' | 'delete')
+container.state.observe((changeType, key, value) => {
+  console.log(`${key} ${changeType}:`, value)
+})
+
+// State has a version counter
+container.state.version // increments on every change
+```
+
+## Event Bus
+
+The container has a built-in event bus for decoupled communication:
+
+```typescript
+// Listen for events
+container.on('featureEnabled', (featureName) => {
+  console.log(`${featureName} was enabled`)
+})
+
+// Emit events
+container.emit('myCustomEvent', { some: 'data' })
+
+// One-time listener
+container.once('ready', () => console.log('Container is ready'))
+
+// Wait for an event (promise-based)
+await container.waitFor('ready')
+```
+
+## Plugins and `.use()`
+
+Extend the container with the `.use()` method:
+
+```typescript
+// Enable a feature by name
+container.use('diskCache')
+
+// Attach a plugin
+container.use(MyPlugin)
+```
+
+A plugin is any class with a static `attach(container)` method:
+
+```typescript
+class MyPlugin {
+  static attach(container) {
+    // Add registries, factories, whatever you need
+    container.myThing = new MyThing(container)
+    return container
+  }
+}
+```
+
+## Utilities
+
+The container provides common utilities so you don't need extra dependencies:
+
+```typescript
+container.utils.uuid()                          // Generate a v4 UUID
+container.utils.hashObject({ foo: 'bar' })      // Deterministic hash
+container.utils.stringUtils.camelCase('my-var')  // 'myVar'
+container.utils.stringUtils.kebabCase('MyVar')   // 'my-var'
+container.utils.stringUtils.pluralize('feature') // 'features'
+
+// Lodash utilities
+const { uniq, groupBy, keyBy, debounce, throttle } = container.utils.lodash
+```
+
+## Path Utilities
+
+```typescript
+container.paths.resolve('relative/path')    // Resolve from cwd
+container.paths.join('a', 'b', 'c')         // Join path segments
+container.paths.relative('/absolute/path')  // Make relative to cwd
+```
+
+## Package Manifest
+
+Access the project's package.json:
+
+```typescript
+container.manifest.name        // "my-app"
+container.manifest.version     // "0.1.0"
+container.manifest.dependencies
+```
+
+## Introspection
+
+Discover everything about the container at runtime:
+
+```typescript
+// Structured introspection data
+const info = container.introspect()
+
+// Human-readable markdown
+const docs = container.introspectAsText()
+```
+
+This is what makes Luca especially powerful for AI agents -- they can discover the entire API surface at runtime without reading documentation.

package/docs/tutorials/03-scripts.md

@@ -0,0 +1,194 @@
+---
+title: Running Scripts and Markdown Notebooks
+tags: [scripts, luca-run, automation, bun, standalone, markdown, codeblocks, notebook]
+---
+
+# Running Scripts and Markdown Notebooks
+
+`luca run` executes TypeScript/JavaScript files and markdown files. This is often the fastest way to try out Luca features, automate tasks, or build runnable documentation.
+
+## Running a TypeScript Script
+
+```bash
+luca run scripts/hello.ts
+```
+
+```typescript
+// scripts/hello.ts
+import container from '@soederpop/luca'
+
+console.log('Available features:', container.features.available)
+console.log('Git branch:', container.git.branch)
+console.log('OS:', container.os.platform, container.os.arch)
+```
+
+The extension is optional -- `luca run scripts/hello` tries `.ts`, `.js`, and `.md` automatically.
+
+## Running Markdown Files
+
+This is one of Luca's most useful features. `luca run` can execute markdown files as runnable notebooks. It walks through the document, renders the prose to the terminal, and executes each `ts` or `js` fenced codeblock in sequence. All blocks share the same VM context, so variables defined in one block are available in the next.
+
+```bash
+luca run docs/tutorial.md
+```
+
+### How It Works
+
+Given a markdown file like this:
+
+````markdown
+# Setup Tutorial
+
+First, let's see what's available (container is provided automatically):
+
+```ts
+console.log(container.features.available)
+```
+
+Now let's use the file system feature:
+
+```ts
+const { files } = container.fs.walk('./src', { include: ['*.ts'] })
+console.log(`Found ${files.length} TypeScript files`)
+```
+
+This block won't run because it's Python:
+
+```python
+print("I'm skipped -- only ts and js blocks run")
+```
+````
+
+When you run `luca run docs/tutorial.md`, it:
+
+1. Renders "# Setup Tutorial" and the prose as formatted markdown in your terminal
+2. Displays the first codeblock, then executes it
+3. Renders the next paragraph
+4. Displays and executes the second codeblock (which can reference `container` from block 1)
+5. Skips the Python block entirely (only `ts` and `js` blocks execute)
+
+### Skipping Blocks
+
+Add `skip` in the code fence meta to prevent a block from running:
+
+````markdown
+```ts skip
+// This block is shown but NOT executed
+dangerousOperation()
+```
+````
+
+### Safe Mode
+
+Use `--safe` to require manual approval before each block runs:
+
+```bash
+luca run docs/tutorial.md --safe
+```
+
+The runner will prompt "Run this block? (y/n)" before executing each codeblock. Great for walkthroughs where you want to pause and observe.
+
+### Shared Context
+
+All codeblocks in a markdown file share a VM context. The context includes `console` and the full container context, so you can use container features without importing:
+
+````markdown
+```ts
+// Block 1: container is already available in the context
+const { files } = container.fs.walk('./src')
+```
+
+```ts
+// Block 2: `files` from block 1 is still in scope
+console.log(`Found ${files.length} files in src/`)
+```
+````
+
+### Use Cases for Markdown Scripts
+
+- **Runnable tutorials** -- documentation that actually executes
+- **Onboarding guides** -- new developers run the guide and see real output
+- **Demo scripts** -- explain and execute in the same document
+- **Literate DevOps** -- annotated operational runbooks
+
+## TypeScript Script Examples
+
+### File Processor
+
+```typescript
+// scripts/process-images.ts
+import container from '@soederpop/luca'
+
+const { fs, proc } = container
+
+const { files: images } = fs.walk('./uploads', { include: ['*.png', '*.jpg'] })
+console.log(`Processing ${images.length} images...`)
+
+for (const image of images) {
+  console.log(`  Optimizing: ${image}`)
+  proc.exec(`optipng ${image}`)
+}
+
+console.log('Done.')
+```
+
+### Data Migration
+
+```typescript
+// scripts/migrate-data.ts
+import container from '@soederpop/luca'
+
+const { fs } = container
+
+const api = container.client('rest', {
+  baseURL: 'https://api.example.com',
+})
+await api.connect()
+
+const oldData = fs.readJson('./data/legacy-users.json')
+console.log(`Migrating ${oldData.length} users...`)
+
+for (const user of oldData) {
+  await api.post('/users', {
+    name: user.full_name,
+    email: user.email_address,
+    role: 'user',
+  })
+  console.log(`  Migrated: ${user.full_name}`)
+}
+
+console.log('Migration complete.')
+```
+
+### Generate Report
+
+```typescript
+// scripts/weekly-report.ts
+import container from '@soederpop/luca'
+
+const { git, fs } = container
+
+const branch = git.branch       // getter, not a method
+const sha = git.sha             // getter, not a method
+const files = await git.lsFiles()
+const { files: srcFiles } = fs.walk('./src', { include: ['*.ts'] })
+
+const report = `# Weekly Report
+
+- Branch: ${branch}
+- Commit: ${sha}
+- Tracked files: ${files.length}
+- Source files: ${srcFiles.length}
+
+Generated: ${new Date().toISOString()}
+`
+
+await fs.writeFile('./reports/weekly.md', report)
+console.log('Report generated: reports/weekly.md')
+```
+
+## Tips
+
+- **Use the container** -- don't import `fs` from Node directly. `container.fs` gives you the same operations with the benefit of working within the container ecosystem.
+- **Markdown scripts are great for prototyping** -- write a markdown file, mix explanation with code, run it, iterate.
+- **Use `--safe` for unfamiliar scripts** -- review each block before it runs.