luca 1.1.2 → 3.0.0

package/docs/scaffolds/feature.md

@@ -0,0 +1,158 @@
+# Building a Feature
+
+A feature is a container-managed capability — something your application needs that lives on the machine (file I/O, caching, encryption, etc). Features are lazy-loaded, observable, and self-documenting.
+
+When to build a feature:
+- You need a reusable local capability (not a network call — that's a client)
+- You want state management, events, and introspection for free
+- You're wrapping a library so the rest of the codebase uses a uniform interface
+
+## Imports
+
+```ts
+import { z } from 'zod'
+import { FeatureStateSchema, FeatureOptionsSchema, FeatureEventsSchema } from '@soederpop/luca'
+import { Feature } from '@soederpop/luca'
+```
+
+These are the only imports your feature file needs from luca. If your feature wraps a third-party library, import it here too — feature implementations are the ONE place where direct library imports are allowed.
+
+The use of dynamic imports is encouraged here, only import libraries you need when the feature is used, and only when necessary in the lifecycle of the feature if it can be done.
+
+feature's have built in ways to check if their requirements are supported and can be enabled cautiously.
+
+## Schemas
+
+Define the shape of your feature's state, options, and events using Zod. Every field must have a `.describe()` — this becomes the documentation.
+
+```ts
+export const {{PascalName}}StateSchema = FeatureStateSchema.extend({
+  // Add your state fields here. These are observable — changes emit events.
+  // Example: itemCount: z.number().default(0).describe('Number of items stored'),
+})
+export type {{PascalName}}State = z.infer<typeof {{PascalName}}StateSchema>
+
+export const {{PascalName}}OptionsSchema = FeatureOptionsSchema.extend({
+  // Add constructor options here. Validated when the feature is created.
+  // Example: directory: z.string().optional().describe('Storage directory path'),
+})
+export type {{PascalName}}Options = z.infer<typeof {{PascalName}}OptionsSchema>
+
+export const {{PascalName}}EventsSchema = FeatureEventsSchema.extend({
+  // Each key is an event name. Value is z.tuple() of listener arguments.
+  // Example: itemAdded: z.tuple([z.string().describe('Item key')]).describe('Emitted when an item is added'),
+})
+```
+
+## Class
+
+The class extends `Feature` with your state and options types. Static properties drive registration and introspection. Every public method needs a JSDoc block with `@param`, `@returns`, and `@example`.
+
+Running `luca introspect` captures JSDoc blocks and Zod schemas and includes them in the description whenever somebody calls `container.features.describe('{{camelName}}')` or `luca describe {{camelName}}`.
+
+```ts
+/**
+ * {{description}}
+ * ```typescript
+ * const {{camelName}} = container.feature('{{camelName}}')
+ * ```
+ *
+ * @extends Feature
+ */
+export class {{PascalName}} extends Feature<{{PascalName}}State, {{PascalName}}Options> {
+  static override shortcut = 'features.{{camelName}}' as const
+  static override stateSchema = {{PascalName}}StateSchema
+  static override optionsSchema = {{PascalName}}OptionsSchema
+  static override eventsSchema = {{PascalName}}EventsSchema
+
+  static { Feature.register(this, '{{camelName}}') }
+
+  /**
+   * Called after the feature is initialized. Use this for any setup logic
+   * instead of overriding the constructor.
+   */
+  async afterInitialize() {
+    // Set up initial state, start background tasks, etc.
+  }
+}
+```
+
+**Important**: You almost never need to override the constructor. Use `afterInitialize()` for any setup logic — it runs after the feature is fully wired into the container and has access to `this.container`, `this.options`, `this.state`, etc.
+
+## Module Augmentation
+
+This is what gives `container.feature('yourName')` TypeScript autocomplete. Without it, the feature works but TypeScript won't know about it.
+
+```ts
+declare module '@soederpop/luca' {
+  interface AvailableFeatures {
+    {{camelName}}: typeof {{PascalName}}
+  }
+}
+```
+
+## Registration
+
+Registration happens inside the class body using a static block. The default export is just the class itself.
+
+```ts
+// Inside the class:
+static { Feature.register(this, '{{camelName}}') }
+
+// At module level:
+export default {{PascalName}}
+```
+
+## Complete Example
+
+Here's a minimal but complete feature. This is what a real feature file looks like:
+
+```ts
+import { z } from 'zod'
+import { FeatureStateSchema, FeatureOptionsSchema } from '@soederpop/luca'
+import { Feature } from '@soederpop/luca'
+
+declare module '@soederpop/luca' {
+  interface AvailableFeatures {
+    {{camelName}}: typeof {{PascalName}}
+  }
+}
+
+export const {{PascalName}}StateSchema = FeatureStateSchema.extend({})
+export type {{PascalName}}State = z.infer<typeof {{PascalName}}StateSchema>
+
+export const {{PascalName}}OptionsSchema = FeatureOptionsSchema.extend({})
+export type {{PascalName}}Options = z.infer<typeof {{PascalName}}OptionsSchema>
+
+/**
+ * {{description}}
+ *
+ * @example
+ * ```typescript
+ * const {{camelName}} = container.feature('{{camelName}}')
+ * ```
+ *
+ * @extends Feature
+ */
+export class {{PascalName}} extends Feature<{{PascalName}}State, {{PascalName}}Options> {
+  static override shortcut = 'features.{{camelName}}' as const
+  static override stateSchema = {{PascalName}}StateSchema
+  static override optionsSchema = {{PascalName}}OptionsSchema
+  static { Feature.register(this, '{{camelName}}') }
+
+  async afterInitialize() {
+    // Setup logic goes here — not in the constructor
+  }
+}
+
+export default {{PascalName}}
+```
+
+## Conventions
+
+- **Naming**: PascalCase for class, camelCase for registration ID. The file name should be kebab-case (e.g. `disk-cache.ts`).
+- **JSDoc**: Every public method, getter, and the class itself needs a JSDoc block. Include `@example` with working code.
+- **Describe everything**: Every Zod field needs `.describe()`. Every event tuple argument needs `.describe()`. This IS the documentation.
+- **No Node builtins in consumer code**: If your feature wraps `fs` or `crypto`, that's fine inside the feature. But code that USES your feature should never import those directly.
+- **State is observable**: Use `this.state.set()` and `this.state.get()`. Don't use plain instance properties for data that should be reactive.
+- **Events for lifecycle**: Emit events for significant state changes so consumers can react.

package/docs/scaffolds/selector.md

@@ -0,0 +1,91 @@
+# Building a Selector
+
+A selector returns data. Where commands perform actions, selectors query and return structured results with built-in caching. Selectors live in a project's `selectors/` folder and are automatically discovered.
+
+When to build a selector:
+- You need to query project data (package info, file listings, config values)
+- The result benefits from caching (keyed by git SHA or custom key)
+- You want the data available via `container.select('name')` or `luca select name`
+
+## Imports
+
+```ts
+import { z } from 'zod'
+import type { ContainerContext } from '@soederpop/luca'
+```
+
+## Args Schema
+
+Define the selector's input arguments with Zod.
+
+```ts
+export const argsSchema = z.object({
+  // Add your input arguments here.
+  // Example: field: z.string().optional().describe('Specific field to return'),
+})
+```
+
+## Description
+
+Export a description string for discoverability:
+
+```ts
+export const description = '{{description}}'
+```
+
+## Caching
+
+Selectors cache by default. The default cache key is `hashObject({ selectorName, args, gitSha })` — same args + same commit = cache hit.
+
+To customize the cache key:
+
+```ts
+export function cacheKey(args: z.infer<typeof argsSchema>, context: ContainerContext) {
+  return context.container.git.currentCommitSha
+}
+```
+
+To disable caching:
+
+```ts
+export const cacheable = false
+```
+
+## Handler
+
+Export a `run` function that returns data. It receives parsed args and the container context.
+
+```ts
+export async function run(args: z.infer<typeof argsSchema>, context: ContainerContext) {
+  const { container } = context
+  // Query and return your data
+  return { /* your data */ }
+}
+```
+
+## Complete Example
+
+```ts
+import { z } from 'zod'
+import type { ContainerContext } from '@soederpop/luca'
+
+export const description = '{{description}}'
+
+export const argsSchema = z.object({})
+
+export async function run(args: z.infer<typeof argsSchema>, context: ContainerContext) {
+  const { container } = context
+
+  // Return your data here
+  return {}
+}
+```
+
+## Conventions
+
+- **File location**: `selectors/{{kebabName}}.ts` in the project root. Discovered automatically.
+- **Naming**: kebab-case for filename. `luca select {{kebabName}}` maps to `selectors/{{kebabName}}.ts`.
+- **Use the container**: Never import `fs`, `path` directly. Use `container.feature('fs')`, `container.paths`.
+- **Return data**: The `run` function must return the data. It gets wrapped in `{ data, cached, cacheKey }` by the framework.
+- **Caching**: On by default. Override `cacheKey()` for custom invalidation, or set `cacheable = false` to skip.
+- **CLI**: `luca select {{kebabName}}` runs the selector and prints JSON. Use `--json` for data only, `--no-cache` to force fresh.

package/docs/scaffolds/server.md

@@ -0,0 +1,196 @@
+# Building a Server
+
+A server is a container-managed listener — something that accepts connections and handles requests. Servers manage their own lifecycle (configure, start, stop) and expose observable state.
+
+When to build a server:
+- You need to accept incoming connections (HTTP, WebSocket, custom protocol)
+- You want lifecycle management, port handling, and observability for free
+- You're wrapping a server library so the codebase uses `container.server('name')`
+
+## Imports
+
+```ts
+import { z } from 'zod'
+import { Server } from '@soederpop/luca'
+import { ServerStateSchema, ServerOptionsSchema, ServerEventsSchema } from '@soederpop/luca'
+import type { NodeContainer } from '@soederpop/luca'
+import type { ServersInterface } from '@soederpop/luca'
+```
+
+## Schemas
+
+```ts
+export const {{PascalName}}StateSchema = ServerStateSchema.extend({
+  // Add your state fields here.
+  // Example: connectionCount: z.number().default(0).describe('Active connections'),
+})
+export type {{PascalName}}State = z.infer<typeof {{PascalName}}StateSchema>
+
+export const {{PascalName}}OptionsSchema = ServerOptionsSchema.extend({
+  // Add constructor options here. port and host come from ServerOptionsSchema.
+  // Example: cors: z.boolean().default(true).describe('Enable CORS'),
+})
+export type {{PascalName}}Options = z.infer<typeof {{PascalName}}OptionsSchema>
+
+export const {{PascalName}}EventsSchema = ServerEventsSchema.extend({
+  // Add your events here.
+  // Example: connection: z.tuple([z.string().describe('Client ID')]).describe('New client connected'),
+})
+```
+
+## Class
+
+Running `luca introspect` captures JSDoc blocks and Zod schemas and includes them in the description whenever somebody calls `container.servers.describe('{{camelName}}')` or `luca describe {{camelName}}`.
+
+```ts
+/**
+ * {{description}}
+ *
+ * @example
+ * ```typescript
+ * const {{camelName}} = container.server('{{camelName}}', { port: 3000 })
+ * await {{camelName}}.start()
+ * ```
+ *
+ * @extends Server
+ */
+export class {{PascalName}} extends Server<{{PascalName}}State, {{PascalName}}Options> {
+  static override shortcut = 'servers.{{camelName}}' as const
+  static override stateSchema = {{PascalName}}StateSchema
+  static override optionsSchema = {{PascalName}}OptionsSchema
+  static override eventsSchema = {{PascalName}}EventsSchema
+  static { Server.register(this, '{{camelName}}') }
+
+  static override attach(container: NodeContainer & ServersInterface) {
+    return container
+  }
+
+  override async configure() {
+    if (this.isConfigured) return this
+    // Set up the underlying server here
+    this.state.set('configured', true)
+    return this
+  }
+
+  override async start(options?: { port?: number }) {
+    if (this.isListening) return this
+    if (!this.isConfigured) await this.configure()
+
+    const port = options?.port || this.options.port || 3000
+    // Start listening here
+    this.state.set('port', port)
+    this.state.set('listening', true)
+    return this
+  }
+
+  override async stop() {
+    if (this.isStopped) return this
+    // Clean up connections here
+    this.state.set('listening', false)
+    this.state.set('stopped', true)
+    return this
+  }
+}
+```
+
+## Module Augmentation
+
+```ts
+declare module '@soederpop/luca' {
+  interface AvailableServers {
+    {{camelName}}: typeof {{PascalName}}
+  }
+}
+```
+
+## Registration
+
+Registration happens inside the class body using a static block. The default export is just the class itself.
+
+```ts
+// Inside the class:
+static { Server.register(this, '{{camelName}}') }
+
+// At module level:
+export default {{PascalName}}
+```
+
+## Complete Example
+
+```ts
+import { z } from 'zod'
+import { Server } from '@soederpop/luca'
+import { ServerStateSchema, ServerOptionsSchema, ServerEventsSchema } from '@soederpop/luca'
+import type { NodeContainer } from '@soederpop/luca'
+import type { ServersInterface } from '@soederpop/luca'
+
+declare module '@soederpop/luca' {
+  interface AvailableServers {
+    {{camelName}}: typeof {{PascalName}}
+  }
+}
+
+export const {{PascalName}}StateSchema = ServerStateSchema.extend({})
+export type {{PascalName}}State = z.infer<typeof {{PascalName}}StateSchema>
+
+export const {{PascalName}}OptionsSchema = ServerOptionsSchema.extend({})
+export type {{PascalName}}Options = z.infer<typeof {{PascalName}}OptionsSchema>
+
+export const {{PascalName}}EventsSchema = ServerEventsSchema.extend({})
+
+/**
+ * {{description}}
+ *
+ * @example
+ * ```typescript
+ * const {{camelName}} = container.server('{{camelName}}', { port: 3000 })
+ * await {{camelName}}.start()
+ * ```
+ *
+ * @extends Server
+ */
+export class {{PascalName}} extends Server<{{PascalName}}State, {{PascalName}}Options> {
+  static override shortcut = 'servers.{{camelName}}' as const
+  static override stateSchema = {{PascalName}}StateSchema
+  static override optionsSchema = {{PascalName}}OptionsSchema
+  static override eventsSchema = {{PascalName}}EventsSchema
+  static { Server.register(this, '{{camelName}}') }
+
+  static override attach(container: NodeContainer & ServersInterface) {
+    return container
+  }
+
+  override async configure() {
+    if (this.isConfigured) return this
+    this.state.set('configured', true)
+    return this
+  }
+
+  override async start(options?: { port?: number }) {
+    if (this.isListening) return this
+    if (!this.isConfigured) await this.configure()
+    const port = options?.port || this.options.port || 3000
+    this.state.set('port', port)
+    this.state.set('listening', true)
+    return this
+  }
+
+  override async stop() {
+    if (this.isStopped) return this
+    this.state.set('listening', false)
+    this.state.set('stopped', true)
+    return this
+  }
+}
+
+export default {{PascalName}}
+```
+
+## Conventions
+
+- **Lifecycle**: Implement `configure()`, `start()`, and `stop()`. Check guards (`isConfigured`, `isListening`, `isStopped`) at the top of each.
+- **Use `afterInitialize()`**: For any setup logic instead of overriding the constructor. Lifecycle methods (`configure`, `start`, `stop`) handle the server's runtime phases.
+- **State tracking**: Set `configured`, `listening`, `stopped`, and `port` on the state. This powers the introspection system.
+- **attach() is static**: It runs when the container first loads the server class. Use it for container-level setup if needed.
+- **Port from options**: Accept port via options schema and respect it in `start()`. Allow override via start options.
+- **JSDoc everything**: Every public method needs `@param`, `@returns`, `@example`. Run `luca introspect` after changes to update generated docs.

package/docs/selectors.md

@@ -0,0 +1,115 @@
+# Selector System
+
+## Vision
+
+Commands perform actions. Selectors return data. Together they form a complete agent tool interface — pick which commands and selectors you need and you get free assistant tools. Agents only need to know the external APIs of features, servers, and clients already in the container (which can be learned from `luca describe`).
+
+## What Is a Selector?
+
+A Selector is a new helper type (like Feature, Client, Server, Command) that:
+
+- Extends `Helper` with a `select(options)` method that **returns data**
+- Lives in a `SelectorsRegistry`, queryable via `container.selectors`
+- Is instantiated via `container.selector('name', options)` factory
+- Supports **caching** via `diskCache` — many selector values don't change if git SHAs don't change, so SHA makes a great cache key
+- Is discoverable from a `selectors/` folder in projects, just like `commands/`
+- Supports both **class-based** and **module-based** (export a `select` function) patterns
+
+## Architecture — How It Maps to Existing Patterns
+
+The implementation follows the exact same pattern as `Command`:
+
+### Files to Create/Modify
+
+1. **`src/schemas/base.ts`** — Add `SelectorStateSchema`, `SelectorOptionsSchema`, `SelectorEventsSchema`
+2. **`src/selector.ts`** — New file mirroring `src/command.ts`:
+   - `Selector` class extending `Helper` with `select()` method
+   - `SelectorsRegistry` extending `Registry` with `discover()` support
+   - `selectors` singleton, `helperCache`, `Selector.register()`, `Selector.attach()`
+   - `AvailableSelectors` interface, `SelectorsInterface`, `SelectorFactory` type
+   - Module-based pattern: `SimpleSelector` type for graft compatibility
+3. **`src/node/container.ts`** — Import `Selector` + `SelectorsInterface`, add to `ClientsAndServersInterface`, wire `this.use(Selector)`
+4. **`src/node/features/helpers.ts`** — Add `selectors` to `registryMap` and `RegistryType`, add to `discoverAll()` iteration
+
+### Selector Base Class Shape
+
+```typescript
+class Selector<T, K> extends Helper<T, K> {
+  static shortcut = 'selectors.base'
+  static argsSchema: z.ZodType  // schema for select() input
+  static cacheable: boolean = true  // opt-out of caching
+
+  // The core method — override in subclass or graft from module export
+  async select(args, context): Promise<any> {}
+
+  // Dispatch normalizer (like Command.execute)
+  async resolve(args?, source?): Promise<SelectorResult> {}
+}
+```
+
+### Module-Based Pattern (selectors/ folder)
+
+```typescript
+// selectors/package-info.ts
+export const description = 'Returns parsed package.json data'
+export const argsSchema = z.object({ field: z.string().optional() })
+export const cacheable = true
+
+export function cacheKey(args, context) {
+  // return a string key — if unchanged, cached value is returned
+  return context.container.git.sha
+}
+
+export async function select(args, context) {
+  const manifest = context.container.manifest
+  return args.field ? manifest[args.field] : manifest
+}
+```
+
+### Caching Strategy
+
+- Built into the `Selector` base class, powered by `diskCache` feature
+- Subclasses/modules can export a `cacheKey(args, context)` function
+- Default cache key strategy: `hashObject({ selectorName, args, gitSha })`
+- `cacheable: false` opts out entirely
+- Cache is key-invalidated (key changes = miss), no TTL by default
+- The `resolve()` method checks cache before calling `select()`
+
+## Open Questions (Needs Decision)
+
+### 1. Return Shape
+Should `select()` return data directly, or a wrapped result like `{ data, metadata, cached }`?
+
+**Leaning toward:** data directly from `select()`, but `resolve()` (the dispatch method) returns `{ data, cached, cacheKey }` so callers know if it was a cache hit.
+
+### 2. CLI Integration
+Should there be a `luca select <name>` CLI command? Would make selectors usable from the terminal, printing JSON output.
+
+### 3. Cache TTL
+Is pure key-invalidation enough, or do some selectors need time-based expiry?
+
+### 4. Scope of First Delivery
+
+Proposed skateboard:
+- `Selector` base class with `select()` and `resolve()` (cached dispatch)
+- `SelectorsRegistry` with `discover()`
+- Container wiring (`container.selectors`, `container.selector()`)
+- Schemas in `schemas/base.ts`
+- `selectors/` folder discovery via `Helpers` feature
+- Caching integration with `diskCache`
+- Module-based pattern support (export `select` + optional `cacheKey`)
+- `Selector` + `selectors` exported from `@soederpop/luca` barrel + seeded in VM virtual modules
+
+## Codebase Exploration Notes
+
+These are the key files that were studied to inform this design:
+
+- `src/command.ts` — The primary pattern to mirror. `Command` extends `Helper`, has `CommandsRegistry`, `commands` singleton, `Command.register()`, `Command.attach()`, `execute()`/`run()` split, headless capture, `graftModule` support.
+- `src/helper.ts` — Base class providing state, events, options (Zod-validated), introspection, `afterInitialize()` hook.
+- `src/registry.ts` — Abstract `Registry<T>` with `register()`, `lookup()`, `has()`, `available`, `describe()`, `describeAll()`, event bus.
+- `src/graft.ts` — `graftModule()` synthesizes a class from plain module exports. `RESERVED_EXPORTS` list determines what becomes static vs prototype methods.
+- `src/schemas/base.ts` — All Zod schemas. Each helper type has State/Options/Events schemas.
+- `src/node/container.ts` — `NodeContainer` wires everything: side-effect imports, `NodeFeatures` interface, `ClientsAndServersInterface`, `this.use(Command)` pattern.
+- `src/node/features/helpers.ts` — `Helpers` feature: `registryMap`, `RegistryType`, `discoverAll()`, class-based vs config-based discovery, VM module seeding.
+- `src/node/features/disk-cache.ts` — `DiskCache` feature: `get/set/has/rm`, `cacache`-backed, per-project cache dir, `container.utils.hashObject()` for keys.
+- `src/container.ts` — Base `Container`: `createHelperInstance()` with `helperCache` Map, `use()` plugin system, `registerHelperType()`.