@soederpop/luca 0.0.2

package/docs/ideas/class-registration-refactor-possibilities.md

@@ -0,0 +1,197 @@
+---
+tags: [dx, registration, metaprogramming]
+status: spark
+---
+
+# Class Registration Refactor Possibilities
+
+Exploring how static initialization blocks and class-level hooks could simplify helper registration, inspired by Ruby's `self.inherited(subclass)` pattern and how ActiveRecord::Base uses it.
+
+## The Ruby Inspiration
+
+In Ruby, when you write `class Post < ActiveRecord::Base`, the `inherited` hook fires automatically:
+
+```ruby
+class ActiveRecord::Base
+  def self.inherited(subclass)
+    subclass.table_name = subclass.name.tableize
+    subclass.establish_connection
+    # The parent configures the child at definition time
+  end
+end
+
+class Post < ActiveRecord::Base
+end
+# That's it. Full ORM-backed model. No registration call needed.
+```
+
+The magic: `inherited` fires at **class definition time**, receives the subclass, and lets the parent **reach into the child and set it up**. The empty class that does everything.
+
+## Current Luca Registration Pattern
+
+Today, every helper requires explicit registration at the bottom of the file:
+
+```typescript
+// src/node/features/fs.ts
+import { features, Feature } from "../feature.js"
+
+export class FS extends Feature {
+  static override shortcut = "features.fs" as const
+  static override stateSchema = FeatureStateSchema
+  static override optionsSchema = FeatureOptionsSchema
+
+  // ... methods ...
+}
+
+export default features.register("fs", FS)
+```
+
+And for extension features (AGI, etc), there's the `attach` pattern:
+
+```typescript
+export class Assistant extends Feature<AssistantState, AssistantOptions> {
+  static attach(container: Container<AvailableFeatures, any>) {
+    features.register('assistant', Assistant)
+    return container
+  }
+}
+
+export default features.register('assistant', Assistant)
+```
+
+The registration is always a separate, imperative call disconnected from the class definition itself. The class doesn't know it's been registered. The registry doesn't know until that line runs.
+
+## The Static Block Approach
+
+ES2022 static initialization blocks let code run at **class definition time**, inside the class body. This is JavaScript's closest equivalent to Ruby's `inherited`:
+
+```typescript
+class Feature {
+  static registry: Registry
+
+  // This is our "inherited" hook
+  static __initSubclass(SubClass: typeof Feature, id: string) {
+    this.registry.register(id, SubClass)
+  }
+}
+```
+
+### What it could look like for authors
+
+```typescript
+// The dream: define, register, and export in one declaration
+export default class FS extends Feature {
+  static {
+    Feature.register(this, "fs")
+  }
+
+  static override shortcut = "features.fs" as const
+  static override stateSchema = FeatureStateSchema
+  static override optionsSchema = FeatureOptionsSchema
+
+  async readFile(path: string) { /* ... */ }
+}
+```
+
+No trailing `features.register()` call. No disconnect between the class and its registration. The class **declares itself** as a registered member of the system.
+
+### The id could be derived from convention
+
+If we adopt a naming convention (like ActiveRecord derives table names from class names), we could eliminate the explicit id entirely:
+
+```typescript
+export default class DiskCache extends Feature {
+  static {
+    Feature.register(this) // id inferred as "diskCache" from class name
+  }
+}
+```
+
+The `register` implementation:
+
+```typescript
+class Feature {
+  static register(SubClass: typeof Feature, id?: string) {
+    // Convention: PascalCase class name -> camelCase registry id
+    const registryId = id ?? SubClass.name[0].toLowerCase() + SubClass.name.slice(1)
+    features.register(registryId, SubClass)
+  }
+}
+```
+
+## Going Further: The Base Class Does All The Work
+
+What if the base classes (`Feature`, `Client`, `Server`) provided a static `register` that also handled module augmentation hints and the `attach` pattern?
+
+```typescript
+// Before: 15+ lines of boilerplate per feature
+import { z } from 'zod'
+import { features, Feature } from '@soederpop/luca/feature'
+import type { AvailableFeatures } from '@soederpop/luca/feature'
+
+declare module '@soederpop/luca/feature' {
+  interface AvailableFeatures {
+    conversation: typeof Conversation
+  }
+}
+
+export class Conversation extends Feature<ConversationState, ConversationOptions> {
+  static override shortcut = 'features.conversation' as const
+  static attach(container: Container<AvailableFeatures, any>) {
+    features.register('conversation', Conversation)
+    return container
+  }
+  // ...
+}
+
+export default features.register('conversation', Conversation)
+```
+
+```typescript
+// After: the class IS the registration
+import { Feature } from '@soederpop/luca/feature'
+
+export default class Conversation extends Feature<ConversationState, ConversationOptions> {
+  static {
+    Feature.register(this, 'conversation')
+  }
+
+  // shortcut derived automatically from registry + id
+  // attach() provided by base class using the registered id
+  // ...
+}
+```
+
+The base `Feature.register()` could:
+1. Call `features.register(id, SubClass)`
+2. Set `SubClass.shortcut` automatically (`features.${id}`)
+3. Generate a default `attach()` that registers and returns the container
+4. Emit a `"subclass:registered"` event on the registry for any other wiring
+
+## Comparison With Other Approaches
+
+| Approach | When it runs | Cooperation needed | Ceremony |
+|----------|-------------|-------------------|----------|
+| Ruby `inherited` | Class definition | None | Zero — just `extends` |
+| Static block + `register` | Class definition | One line in static block | Minimal — inside the class |
+| Trailing `registry.register()` | Module evaluation | Separate call after class | Moderate — disconnected |
+| `defineFeature()` factory | Module evaluation | Wrap entire definition | Different paradigm entirely |
+| Decorators (`@tracked`) | Class definition | One line above class | Minimal — but external |
+
+The static block approach is the sweet spot for Luca: it's **standard JavaScript**, runs at **definition time**, lives **inside the class body**, and keeps the class as the primary authoring unit (unlike `defineFeature()` which replaces the class with a factory call).
+
+## The `defineFeature` Relationship
+
+This is complementary to the [`defineFeature()` idea](./luca-feature-authoring-dx.md), not a replacement. They solve different problems:
+
+- **`defineFeature()`** reduces boilerplate for simple features that are mostly a bag of methods — you skip writing a class entirely
+- **Static block registration** reduces boilerplate for full class-based helpers — you still write the class, but registration is self-contained
+
+For complex features that need class inheritance, lifecycle hooks, and full OOP — the static block pattern is cleaner. For simple features that are essentially a named collection of functions — `defineFeature()` is cleaner.
+
+## Open Questions
+
+- **Registry detection**: Can `Feature.register(this)` figure out which registry to use automatically? The base class (`Feature` vs `Client` vs `Server`) already implies it. A shared `Helper.register(this)` could dispatch to the right registry.
+- **Module augmentation**: The `declare module` blocks for TypeScript are the remaining boilerplate that can't be eliminated by runtime mechanics. Could a codegen step or a TS plugin handle this?
+- **Import side effects**: Today, importing a feature file triggers registration. With static blocks, this is still true — but the registration is more visible and intentional. Is there value in making registration lazy or explicit via `container.use()`?
+- **`attach()` consolidation**: Many `attach()` methods just call `register()` and return the container. If `Feature.register()` generates a default `attach()`, do we still need custom ones? The ones that do more (like wiring up other features) would still override.

package/docs/ideas/container-use-api.md

@@ -0,0 +1,9 @@
+# COntainer `.use()` API
+
+Can we find a way to support using dynamic imports
+
+```ts
+container.use(import("whatever))
+```
+
+these are promises, but the expectation is the default has an attach method.  if not, any function or class they export which has an attach property

package/docs/ideas/feature-stacks.md

@@ -0,0 +1,22 @@
+# Feature Stacks
+
+The number of features in the node container is getting a little absurd.
+
+We have the `container.use` API.  We can bundle / group features like the google related ones into their own folder's and start to build on our plugin / layers concept
+
+The startup time for the luca cli is getting slower and I'm wondering if it is all of these features causing it.
+
+## Group features into folders
+
+- don't autoload all of them, let's make it trivial like to load these features into scripts or commands 
+
+```ts
+import container from '@soederpop/luca/agi'
+import GoogleFeatures from '@soederpop/luca/features/google'
+
+container.use(GoogleFeatures)
+```
+
+## Feature Marketplace ( long term )
+
+Long term as I'm the only one using this shit, but the ability to publish a server that other containers could load features from is cool.  We have all the pieces on the frontend ( asset loader ) to go from a blankpage to a fully functional SPA 

package/docs/ideas/luca-cli-self-sufficiency-demo.md

@@ -0,0 +1,23 @@
+# Luca CLI Self Sufficiency Demo
+
+With the `luca` cli you can have a project folder like:
+
+```
+- public/
+	- index.html
+- commands/
+	- setup-db.ts
+- endpoints/
+	- health.ts
+- docs/
+	- models.ts
+- luca.cli.ts
+```
+
+This should work without there necessarily being a `node_modules` folder here, because each of these things are loaded by luca, transpiled, and run through the VM.
+
+Even the `contentDb` feature which depends on contentbase should work here.
+
+This is what I mean by luca being a dependency injection framework.
+
+The only downside of this is - how do we provide type support without them installing '@soederpop/luca'

package/docs/ideas/mcp-design.md

@@ -0,0 +1,9 @@
+# MCP Design
+
+I have a hunch that the best way to get coding assistants to be aware of our MCP is to have as verbose of tool descriptions as the spec allows, as this seems to be the primary form of guidance.
+
+One neat trick I learned from the excalidraw and other mcps is to have a tool_call called read_me, which we can probably really tune the tool description for, to make sure it gets called as early on as possible by the MCP, and as a way to further refine how it understands to use tool calls.
+
+## Current State
+
+We've got a `luca sandbox-mcp` command in this project which can be used to eval snippets of code inside the luca vm, as well as some other tools.  Claude code consistently fails to use them, however.

package/docs/ideas/web-container-debugging-feature.md

@@ -0,0 +1,13 @@
+# Web Container Debugging Feature
+
+How cool would it be if, from an application that uses WebContainer, we had a client side feature called `serverPlug` 
+
+The `serverPlug` would `connect` to a remote server over an API endpoint, to register itself, with info about the url it is on, its container's uuid.  The server would respond with a websocket url and other necessary info.
+
+On the server side `NodeContainer` there's server side `serverPlug` feature which can act as a host, that accepts registrations.  It spawns its own websocket server to handle communication.
+
+From server side code, I can connect to any individual web container instance, and eval code inside its vm.
+
+We need to think about auth, disconnection events, heartbeats, etc
+
+

package/docs/introspection-audit.md

@@ -0,0 +1,49 @@
+# Introspection Audit
+
+This audit is a periodic test, does the introspectAsText output actually help an LLM use that
+particular feature.
+
+## Files
+
+- `src/node/features/*.ts` NodeContainer features
+- `src/agi/features/*.ts` AGIContainer features
+
+## Your Task
+
+For each feature, you can run the following shell command:
+
+```shell
+luca describe feature-id
+```
+
+This command will accept diskCache, disk-cache.  Generally you want to go with the shortcut that is defined on the feature class.
+
+Your job is to take what you learn from that command, and build an idea of 3-4 example usecases of that feature.
+
+Compile all of this into a document in docs/reports/introspection-audit-tasks.md.
+
+I'd like the following format
+
+```
+# Introspection Audit Results
+
+## Features
+
+### [diskCache](src/node/features/disk-cache.ts)
+
+The purpose of this feature is to provided a consistent disk backed key-value store for large blobs and json objects.  It includes the ability to have an encrypted store that can only be read with a secret.
+
+- securely set / get a key across processes
+- query information about available keys
+- check if a key is present, if not write to it
+```
+
+Do that for every feature.
+
+## Next Steps
+
+Once we have this report written, I will review the use cases and descriptions to see how accurate they are and if I like them.
+
+Then we will have you attempt to eval your code and see if it works. If it doesn't, sugggest how the documentation could be improved or how the API might be improved.
+
+

package/docs/introspection.md

@@ -0,0 +1,154 @@
+# Introspection
+
+Luca's introspection system lets you discover everything about a container and its helpers at runtime. This document is a runnable demo — every code block works in the Luca REPL or as a script.
+
+## Container Introspection
+
+The container knows what it is, what registries it has, and what's available in each one.
+
+```ts
+const info = container.inspect()
+console.log(info.className)
+console.log(info.registries.map(r => `${r.name}: ${r.available.length} available`))
+console.log('factories:', info.factoryNames)
+console.log('enabled features:', info.enabledFeatures)
+```
+
+You can get the full introspection as markdown:
+
+```ts
+console.log(container.inspectAsText())
+```
+
+Or request a single section:
+
+```ts
+console.log(container.inspectAsText('methods'))
+```
+
+```ts
+console.log(container.inspectAsText('getters'))
+```
+
+## Querying Registries
+
+Every registry exposes what's available and can describe its members.
+
+```ts
+console.log('features:', container.features.available)
+```
+
+```ts
+console.log('clients:', container.clients.available)
+```
+
+```ts
+console.log('servers:', container.servers.available)
+```
+
+Describe a single member — returns markdown documentation derived from the code:
+
+```ts
+console.log(container.features.describe('git'))
+```
+
+Describe everything in a registry at once:
+
+```ts
+const allFeatureDocs = container.features.describeAll()
+console.log(`${allFeatureDocs.length} features documented`)
+console.log(allFeatureDocs[0].slice(0, 200) + '...')
+```
+
+## Helper Introspection — Structured Data
+
+Every helper instance (feature, client, server, etc.) can introspect itself. The result is a typed object you can traverse programmatically.
+
+```ts
+const git = container.feature('git')
+const data = git.introspect()
+console.log(data.id)
+console.log(data.description.slice(0, 100) + '...')
+console.log('methods:', Object.keys(data.methods))
+console.log('getters:', Object.keys(data.getters))
+```
+
+### Filtering by Section
+
+Pass a section name to get just that part:
+
+```ts
+const git = container.feature('git')
+const methodsOnly = git.introspect('methods')
+console.log(Object.keys(methodsOnly.methods))
+console.log(Object.keys(methodsOnly.getters)) // empty — filtered out
+```
+
+Valid sections: `'methods'`, `'getters'`, `'events'`, `'state'`, `'options'`.
+
+### Expanded Type Properties
+
+When a method parameter uses a custom type, introspection resolves it to show the type's members:
+
+```ts
+const git = container.feature('git')
+const lsFiles = git.introspect('methods').methods.lsFiles
+console.log(lsFiles.parameters.options.type) // "LsFilesOptions"
+console.log(lsFiles.parameters.options.properties)
+// { cached: { type: 'boolean', description: 'Show cached/staged files' }, ... }
+```
+
+## Helper Introspection — As Text
+
+Get the full markdown documentation for a helper:
+
+```ts
+const git = container.feature('git')
+console.log(git.introspectAsText())
+```
+
+Or just one section:
+
+```ts
+const git = container.feature('git')
+console.log(git.introspectAsText('methods'))
+```
+
+```ts
+const git = container.feature('git')
+console.log(git.introspectAsText('getters'))
+```
+
+The heading depth can be controlled — useful when embedding in larger documents:
+
+```ts
+const git = container.feature('git')
+console.log(git.introspectAsText('methods', 3)) // headings start at ###
+```
+
+## Inspecting State and Options
+
+Features that declare Zod schemas for state and options expose them through introspection:
+
+```ts
+const git = container.feature('git')
+const stateInfo = git.introspect('state')
+console.log(stateInfo.state)
+```
+
+```ts
+const ui = container.feature('ui')
+const stateInfo = ui.introspect('state')
+console.log(stateInfo.state)
+```
+
+## Putting It Together
+
+A quick way to survey everything the container offers:
+
+```ts
+for (const name of container.registryNames) {
+  const registry = container[name]
+  console.log(`\n${name}: ${registry.available.join(', ')}`)
+}
+```

package/docs/mcp/readme.md

@@ -0,0 +1,162 @@
+# Luca Development Guide
+
+You are working in a **luca project**. The luca container provides all capabilities your code needs. Do not install npm packages or import Node.js builtins directly.
+
+## The Contract
+
+Every capability goes through the container. If you need something that doesn't exist, build it as a feature, client, or server. If it wraps a third-party library, the helper IS the interface — consumer code never imports the library directly.
+
+## Import Rule
+
+All consumer code imports from `@soederpop/luca` only:
+
+```ts
+import { Feature, features, z, FeatureStateSchema, FeatureOptionsSchema } from '@soederpop/luca'
+import { Client, clients, RestClient, ClientStateSchema } from '@soederpop/luca/client'
+import { Server, servers, ServerStateSchema } from '@soederpop/luca'
+import { commands, CommandOptionsSchema } from '@soederpop/luca'
+```
+
+Never import from `fs`, `path`, `crypto`, or other Node builtins. Never import third-party packages in consumer code. The only exception is inside helper implementations themselves — a feature that wraps a library may import it.
+
+## Zod v4
+
+This project uses **Zod v4** — import `z` from `@soederpop/luca`, never from `'zod'` directly. All option, state, and event schemas use Zod v4 syntax. Key patterns:
+
+```ts
+// Extending base schemas (options, state, events)
+export const MyStateSchema = FeatureStateSchema.extend({
+  count: z.number().default(0).describe('Number of items'),
+  label: z.string().optional().describe('Display label'),
+})
+
+// Events use z.tuple() for listener arguments
+export const MyEventsSchema = FeatureEventsSchema.extend({
+  itemAdded: z.tuple([z.string().describe('key'), z.number().describe('index')]),
+})
+
+// Type inference
+export type MyState = z.infer<typeof MyStateSchema>
+```
+
+Zod v4 differences from v3 that matter:
+- `z.string().check(...)` replaces some v3 refinement patterns
+- `.toJSONSchema()` is built-in on any schema — no external library needed
+- Error customization uses `z.string({ error: "message" })` not `.refine()` for simple cases
+- `z.interface()` exists for recursive/lazy object types
+- Do NOT use `z.nativeEnum()` — use `z.enum()` instead
+
+## Dependencies
+
+If the project has `node_modules` and a package manager, helper implementations can import third-party libraries internally. If not (e.g. running via the `luca` binary's VM), all code must import only from `@soederpop/luca`.
+
+## Discovering Capabilities
+
+The container has registries for features, clients, servers, commands, and endpoints. **Do not guess** what is available — use your MCP tools to discover it:
+
+1. **`find_capability`** — Overview of all features, clients, and servers with descriptions. Start here.
+2. **`list_registry`** — List all names in a specific registry (features, clients, servers, commands, endpoints).
+3. **`describe_helper`** — Full API docs for a specific helper (methods, options, state, events). Call this before writing code that uses a helper.
+4. **`eval`** — Once you know what you need, prototype calls in the live sandbox before writing them into files.
+
+## Mini Examples
+
+### Feature with composition
+
+Features access other features via `this.container.feature(...)`:
+
+```ts
+import { z } from 'zod'
+import { FeatureStateSchema, FeatureOptionsSchema, FeatureEventsSchema } from '@soederpop/luca'
+import { Feature, features } from '@soederpop/luca'
+import type { ContainerContext } from '@soederpop/luca'
+
+export const ConfigStateSchema = FeatureStateSchema.extend({
+  loaded: z.boolean().default(false).describe('Whether config has been loaded'),
+})
+
+export const ConfigOptionsSchema = FeatureOptionsSchema.extend({
+  filePath: z.string().default('config.json').describe('Path to config file'),
+})
+
+export const ConfigEventsSchema = FeatureEventsSchema.extend({
+  configLoaded: z.tuple([z.record(z.unknown()).describe('parsed config')]),
+})
+
+export class Config extends Feature<z.infer<typeof ConfigStateSchema>, z.infer<typeof ConfigOptionsSchema>> {
+  static override shortcut = 'features.config' as const
+  static override stateSchema = ConfigStateSchema
+  static override optionsSchema = ConfigOptionsSchema
+  static override eventsSchema = ConfigEventsSchema
+  static override description = 'Loads and manages project configuration'
+
+  /** Load and parse the config file */
+  async load() {
+    const fs = this.container.feature('fs')
+    const raw = await fs.readFile(this.options.filePath)
+    const data = JSON.parse(raw)
+    this.state.set('loaded', true)
+    this.emit('configLoaded', data)
+    return data
+  }
+}
+
+declare module '@soederpop/luca' {
+  interface AvailableFeatures { config: typeof Config }
+}
+export default features.register('config', Config)
+```
+
+### Client with composition
+
+Clients access features and other clients via `this.container`:
+
+```ts
+import { z } from 'zod'
+import { Client, clients, ClientStateSchema, ClientOptionsSchema, ClientEventsSchema } from '@soederpop/luca'
+import type { ContainerContext } from '@soederpop/luca'
+
+export const GithubOptionsSchema = ClientOptionsSchema.extend({
+  token: z.string().describe('GitHub personal access token'),
+  owner: z.string().describe('Repository owner'),
+  repo: z.string().describe('Repository name'),
+})
+
+export const GithubEventsSchema = ClientEventsSchema.extend({
+  issuesFetched: z.tuple([z.number().describe('count')]),
+})
+
+export class GithubClient extends Client<z.infer<typeof ClientStateSchema>, z.infer<typeof GithubOptionsSchema>> {
+  static override shortcut = 'clients.github' as const
+  static override optionsSchema = GithubOptionsSchema
+  static override eventsSchema = GithubEventsSchema
+  static override description = 'GitHub API client using container REST client'
+
+  /** Fetch open issues */
+  async issues() {
+    const rest = this.container.client('rest')
+    const res = await rest.get(`https://api.github.com/repos/${this.options.owner}/${this.options.repo}/issues`, {
+      headers: { Authorization: `token ${this.options.token}` },
+    })
+    this.emit('issuesFetched', res.data.length)
+    return res.data
+  }
+}
+
+declare module '@soederpop/luca' {
+  interface AvailableClients { github: typeof GithubClient }
+}
+export default clients.register('github', GithubClient)
+```
+
+## Workflow
+
+1. **`find_capability`** — Search what already exists before writing anything
+2. **`describe_helper`** — Read the full API docs for the helper you need
+3. **`eval`** — Prototype and test container API calls in the sandbox
+4. **`scaffold`** — Generate correct boilerplate when building something new
+5. **Write the file** — Using the patterns from the scaffold
+
+## Portability
+
+Code that only imports from `@soederpop/luca` can be copied between any luca project. That's the goal. Features, clients, servers, and commands written this way are portable building blocks.

package/docs/models.ts

@@ -0,0 +1,38 @@
+import {
+  defineModel,
+  section,
+  hasMany,
+  belongsTo,
+  AstQuery,
+  z,
+} from "contentbase";
+
+export const Idea = defineModel("Idea", {
+	prefix: "ideas",
+	meta: z.object({
+		goal: z.string().optional().describe("Slug of the goal this idea is aligned to"),
+		tags: z.array(z.string()).default([]).describe("Arbitrary tags for categorizing the idea"),
+		status: z.enum(["spark", "exploring", "parked", "promoted"]).default("spark").describe("spark is a new raw idea, exploring means actively thinking about it, parked means on hold, promoted means it became a plan"),
+	}),
+});
+
+export const Tutorial = defineModel("Tutorial", {
+	prefix: "tutorials",
+	meta: z.object({
+		tags: z.array(z.string()).default([]).describe("Arbitrary tags for categorizing the tutorial"),
+	}),
+})
+
+export const Report = defineModel("Report", {
+	prefix: "reports",
+	meta: z.object({
+		tags: z.array(z.string()).default([]).describe("Arbitrary tags for categorizing the report"),
+	}),
+})
+
+export const Example = defineModel("Example", {
+	prefix: "examples",
+	meta: z.object({
+		tags: z.array(z.string()).default([]).describe("Arbitrary tags for categorizing the example"),
+	}),
+})