stream_weaver 0.3.0

data/docs/canvas-roadmap.md

@@ -0,0 +1,275 @@
+# StreamWeaver Canvas Roadmap
+
+## What is Canvas?
+
+**The Problem:** Agentic AI CLIs like Claude Code are powerful but constrained to text-based terminal interaction. Complex choices, data visualization, code review, and multi-step workflows become walls of text that are hard to parse and interact with.
+
+**The Solution:** A persistent browser canvas that AI agents can push rich HTML/CSS/JS UI to, enabling:
+- Visual data display (tables, charts, diffs)
+- Interactive forms and wizards
+- Clickable choices instead of typing responses
+- Real-time progress visualization
+
+**Why Ruby/Phlex?** Token efficiency. Instead of generating verbose HTML, the AI generates concise Ruby DSL:
+```ruby
+# 45 tokens of Ruby
+table headers: ["File", "Size"], rows: data, striped: true
+button "Apply Changes", variant: :primary
+
+# vs 200+ tokens of equivalent HTML/CSS
+```
+
+### Primary Use Case
+
+**Claude Code Companion** - Side-by-side experience:
+- Terminal on left: Claude Code CLI running
+- Canvas on right: Rich UI for interaction
+
+When Claude needs user input, instead of:
+```
+Please choose:
+1. PostgreSQL
+2. SQLite
+3. MySQL
+Enter number:
+```
+
+It pushes a visual choice UI, user clicks, Claude receives JSON response.
+
+### StreamWeaver Modes: When to Use What
+
+StreamWeaver has multiple modes. Canvas (live sessions) is one of them:
+
+| Mode | Command | Use When |
+|------|---------|----------|
+| **Standalone** | `streamweaver run app.rb` | Building a complete web app with predefined UI |
+| **Server** | `streamweaver serve` | Hosting multiple apps, production deployment |
+| **Agentic** | `streamweaver llm` | AI generates UI on-the-fly during conversation |
+| **Canvas** | `streamweaver live SESSION` | AI pushes UI during external workflow (e.g., Claude Code) |
+
+**Standalone** - You write the app, user interacts with fixed UI
+```ruby
+# my_app.rb - predefined screens and flow
+text_field :name
+button "Submit" do |state|
+  # handle submission
+end
+```
+
+**Agentic** - AI writes the app dynamically based on conversation
+```bash
+streamweaver llm  # AI generates UI based on chat
+```
+
+**Canvas** - External AI agent pushes UI as needed during its workflow
+```bash
+# Claude Code (or other agent) runs separately
+# Pushes to canvas when it needs rich interaction
+./exe/streamweaver template wizard mysession '{"steps":[...]}'
+```
+
+### When to Use Canvas vs Other Modes
+
+| Scenario | Best Mode | Why |
+|----------|-----------|-----|
+| Building a dashboard app | Standalone | Fixed UI, known requirements |
+| Deploying apps for team | Server | Multi-app hosting, production-ready |
+| Chatbot builds custom UI | Agentic | AI controls the whole experience |
+| Claude Code needs user input | **Canvas** | AI is running elsewhere, pushes when needed |
+| CI/CD wants to show results | **Canvas** | External process pushes status |
+| Debug session visualization | **Canvas** | Debugger pushes state as it runs |
+
+### Canvas-Specific Use Cases
+
+| Use Case | Description |
+|----------|-------------|
+| **Any agentic CLI** | Not Claude-specific - any AI agent can push to canvas |
+| **Build dashboards** | Show test results, coverage, build status visually |
+| **Code review UI** | Rich diff display with inline comments |
+| **Onboarding wizards** | Guide users through project setup |
+| **Debug visualizers** | Display data structures, call stacks graphically |
+| **Documentation preview** | Live preview of generated docs |
+| **Database explorers** | Query results in proper tables |
+| **Log viewers** | Filterable, searchable log display |
+| **Deployment dashboards** | Multi-environment status at a glance |
+| **LLM prompt builders** | Visual prompt construction with previews |
+
+### Design Principles
+
+1. **Token efficient** - Ruby DSL, not verbose HTML
+2. **Bidirectional** - Push UI, receive user actions as JSON
+3. **Stateless templates** - Each interaction is self-contained
+4. **Progressive enhancement** - Works with polling, can upgrade to WebSocket
+5. **Browser-native** - No Electron needed, any browser works
+
+## Current State (January 2025)
+
+### Working
+- Live sessions with 300ms polling
+- 7 templates: wizard, choices, confirm, info, table, code, diff
+- Native Phlex table component with styling options
+- Smooth CSS transitions between content updates
+- Theme support (dashboard, minimal, dark)
+- Form submission capture and retrieval
+
+**Key Files:**
+| File | Purpose |
+|------|---------|
+| `lib/stream_weaver/templates/*.rb` | 7 template classes (wizard, choices, confirm, info, table, code, diff) |
+| `lib/stream_weaver/cli.rb` | `template` command dispatch |
+| `lib/stream_weaver/service.rb` | Live session endpoints (`/live/:session`, `/push`, `/submissions`) |
+| `lib/stream_weaver/components.rb` | Native Table component |
+| `lib/stream_weaver/adapter/alpinejs.rb` | `render_table` and component rendering |
+| `lib/stream_weaver/css.rb` | Animations and table CSS classes |
+
+### Limitations
+- No real-time streaming (polling only)
+- No terminal embedding yet
+- Templates require user to have browser open
+- No progress updates during long-running Claude tasks
+
+## Future Directions
+
+### 1. Terminal Integration
+
+**Goal:** Side-by-side terminal + canvas experience
+
+**Options explored:**
+| Approach | Pros | Cons |
+|----------|------|------|
+| xterm.js | Full terminal emulator | Complex PTY setup, WebSocket needed |
+| tmux capture | Simple, read pane buffer | Polling delay, not true integration |
+| ttyd/gotty | Existing tools | External dependency, iframe embed |
+
+**Recommendation:** Start with tmux capture for simplicity, evolve to xterm.js if needed.
+
+### 2. Real-time Progress Updates
+
+**Problem:** Claude can't push updates during thinking/tool execution.
+
+**Explored solutions:**
+| Approach | How | Status |
+|----------|-----|--------|
+| Incremental pushes | Claude pushes between steps | Works, but adds tool calls |
+| Status file | Claude writes file, canvas polls | Not implemented |
+| Hooks | Claude Code hooks for output events | Unknown feasibility |
+
+**Recommendation:** For now, use incremental pushes. Status file approach is cleaner but requires infrastructure.
+
+### 3. New Templates
+
+**High value:**
+- `progress` - Progress bar with status messages (for multi-step tasks)
+- `log` - Scrolling log viewer (for build output, test results)
+- `file-tree` - Interactive file browser
+- `form` - Generic form builder (more flexible than wizard)
+
+**Nice to have:**
+- `chart` - Simple charts (bar, line)
+- `timeline` - Event timeline display
+- `kanban` - Task board view
+
+### 4. WebSocket Support
+
+**Current:** 300ms polling works but isn't true real-time.
+
+**Trade-off:** WebSocket adds complexity (Sinatra needs extra gems like Faye or switching to a different server).
+
+**Recommendation:** Polling is sufficient for current use cases. Revisit if latency becomes an issue.
+
+### 5. Electron/Desktop App
+
+**Idea:** Native app with terminal + canvas side-by-side.
+
+**Benefits:**
+- True terminal integration
+- No browser tab needed
+- Could hook into Claude Code process directly
+
+**Complexity:** High - separate app to maintain.
+
+**Recommendation:** Defer until web-based approach hits limitations.
+
+### 6. VS Code Extension
+
+**Idea:** Webview panel in VS Code alongside terminal.
+
+**Benefits:**
+- Developers already in VS Code
+- Native integration with editor
+- Could leverage VS Code's terminal API
+
+**Complexity:** Medium - VS Code extension API learning curve.
+
+### 7. Better Code Display
+
+**Current:** Basic dark theme code block.
+
+**Improvements:**
+- Syntax highlighting (via Prism.js or highlight.js)
+- Line number linking
+- Diff with inline comments
+- Copy button
+- Expand/collapse for long code
+
+### 8. Session Persistence
+
+**Current:** Sessions are in-memory, lost on restart.
+
+**Improvement:** Store session state to disk for:
+- Resume after service restart
+- History of interactions
+- Audit trail
+
+### 9. Multi-canvas Support
+
+**Idea:** Multiple named canvases for different purposes:
+- `main` - Primary interaction
+- `logs` - Continuous log output
+- `status` - Persistent status bar
+
+**Implementation:** Already supported via session names, just needs patterns/conventions.
+
+## Architecture Notes
+
+### Why Polling vs WebSocket
+
+Polling at 300ms provides:
+- Simpler implementation (plain HTTP)
+- Works through proxies/firewalls
+- No connection management
+- Good enough latency for human interaction
+
+WebSocket would provide:
+- True real-time updates
+- Lower server load for many clients
+- Better for streaming output
+
+### Why Phlex
+
+- Ruby-native templating
+- Type-safe HTML generation
+- Composable components
+- No separate template files
+
+### Template Design Principles
+
+1. **Single command** - Full interaction in one bash call
+2. **JSON in, JSON out** - Easy to parse programmatically
+3. **Clear state** - Each run starts fresh
+4. **Timeout** - Don't hang forever
+5. **Sensible defaults** - Work out of the box
+
+## Next Steps (Prioritized)
+
+1. **Dogfood** - Use templates in real agentic workflows
+2. **Progress template** - Most requested for long tasks
+3. **Syntax highlighting** - Improve code/diff display
+4. **Documentation** - More examples, best practices
+
+## Non-Goals (For Now)
+
+- Full IDE replacement
+- Complex state management
+- Multi-user collaboration
+- Mobile support

data/docs/case-studies/2026-09-03-didx-canvas-read-shelf.md

@@ -0,0 +1,147 @@
+# Case study: the Org + canvas-read document shelf (DiDX proposal session)
+
+Date: 2026-09-03. Environment: **Codex (ChatGPT desktop app) on macOS** — notably *not* the
+designed-for environment (Claude Code + iTerm2). The workflow held up anyway, which is the
+headline: the Org-source + canvas-read loop is agent-agnostic.
+
+**The winning loop:**
+
+```text
+Discuss → save a small Org document → reload canvas-read → compare versions → revise → export when ready
+```
+
+> Agent-friendly source, human-friendly reading, version-friendly files, and very low
+> friction between writing and reviewing.
+
+Three proposal documents (a neutral showcase, a personal version with several rounds of copy
+revision, and a technical starter) lived as 12-16KB Org files sharing one image directory,
+read side-by-side in canvas-read's shelf (left rail, per-doc TOC, Previous/Next). Preserving
+alternative versions felt cheap rather than burdensome — that is the feature.
+
+What follows is the session's own context write-up, verbatim.
+
+---
+
+## What happened
+
+In one Codex session, we developed a visual DiDX proposal through several rounds of discussion.
+
+Codex created and saved three StreamWeaver Org documents:
+
+1. `didx_whats_possible.org` — a neutral visual showcase.
+2. `didx_let_me_expand_the_picture.org` — a more personal version written from me to the recipient, with several rounds of copy revision.
+3. `didx_blueprint_theme_model_starter.org` — a local implementation starter for the component, blueprint, theme, color scheme, page, and preset hierarchy.
+
+The Org files remained small, around 12 to 16 KB each. The visual weight lived in shared image assets rather than bloated document source.
+
+## Why canvas-read felt so good
+
+`canvas-read` turned the files into a small local document shelf. The left rail showed all three documents. I could:
+
+- Move between versions immediately.
+- Compare the neutral and personal versions without opening separate tools.
+- Keep the technical starter beside the presentation documents.
+- Use the generated table of contents inside each document.
+- See images, callouts, tables, diagrams, and polished formatting.
+- Reload after an edit and immediately review the result.
+- Keep the canonical source as Org while getting a much better reading experience.
+
+The important experience was that these stopped feeling like loose files. They felt like a small, navigable local knowledge collection.
+
+This was especially useful while revising the message. We could preserve the previous version, create another version with a different voice, and compare them from the same shelf.
+
+## Actual command used
+
+The executable was not on `PATH`, so the working command was run from the repository:
+
+```bash
+SW_NO_OPEN=1 bundle exec ruby -Ilib exe/streamweaver canvas-read \
+  /path/to/didx_whats_possible.org \
+  /path/to/didx_let_me_expand_the_picture.org \
+  /path/to/didx_blueprint_theme_model_starter.org \
+  --theme=doc
+```
+
+`canvas-read` selected an available port and printed the URL:
+
+```text
+canvas-read 3 file(s) → http://127.0.0.1:4800/?file=0
+```
+
+Earlier runs used port 4801. Documentation should explicitly say not to assume a fixed port.
+
+Passing the files explicitly produced the controlled comparison shelf we wanted.
+
+## Org format used
+
+Each source begins with:
+
+```org
+#+STREAMWEAVER_DSL: 1
+#+TITLE: Document Title
+```
+
+Numbered headings with `CUSTOM_ID` properties become document sections and generate the sidebar table of contents. Normal prose, lists, tables, quotes, and source blocks remain readable as Org.
+
+Components not represented natively in the Org dialect can use the existing passthrough:
+
+```org
+#+begin_src ruby :streamweaver-raw t
+image_block File.join(__dir__, "assets/example.png"), base64: true,
+  alt: "Description"
+#+end_src
+```
+
+This let the document remain primarily Org while still using StreamWeaver image components and limited custom styling.
+
+## Images and shared assets
+
+The documents referenced one shared image directory; generated images were not duplicated per document. `image_block` with `base64: true` made the images work in `canvas-read`, where ordinary relative image paths do not have a backing asset route.
+
+```text
+Small Org source + shared high-quality images = rich canvas-read document
+```
+
+## Standalone delivery
+
+We also produced standalone HTML and an email-ready ZIP (~11 MB):
+
+```text
+Proposal/
+├── Proposal.html
+└── images/ (11 PNG files)
+```
+
+Recipient instructions: unzip, keep the HTML and `images` together, double-click the HTML.
+
+## Important export gap
+
+`canvas-read` handled the StreamWeaver Org files correctly. The normal `streamweaver export file.org` command did not — it attempted to evaluate the raw Org text as Ruby and failed. The session worked around it by reading the Org, converting through `StreamWeaver::Org::Reader.to_dsl`, evaluating in a `StreamWeaver::App`, and passing that to `StreamWeaver::Export::HtmlExporter`. This is a current gap; a first-class Org export path would remove the custom step.
+
+## Another known gap: links between documents
+
+Same-document Org anchors work. Native relative links from one Org document to another are not rewritten into `canvas-read` navigation URLs. The shelf made cross-document links unnecessary here, but a connected corpus will eventually want that. MarkyMark has prior art: it rewrites relative Markdown/Org links into internal reader routes preserving fragments and searches.
+
+## What was actually tested
+
+Directly tested: macOS; Codex Desktop and its in-app browser; local StreamWeaver source; Ruby and Bundler already installed; canvas-read with explicit Org file arguments; the `:doc` theme; multiple documents in one shelf; desktop and phone-width rendering; Org-to-DSL conversion; shared image assets; standalone HTML generation; ZIP packaging with relative images.
+
+Not tested: Linux; WSL; native Windows Ruby; browsers outside the Codex in-app browser for canvas-read; a packaged StreamWeaver executable; a clean machine without the development repository; `streamweaver panel` without iTerm.
+
+## Platform boundaries to document clearly
+
+`streamweaver panel` is the iTerm-specific convenience path — macOS + iTerm functionality. `canvas-read` is a local web server and far less platform-specific; the degraded path is: start without auto-open, print the URL, open it in any browser. That should be viable on macOS without iTerm, Linux, WSL, and possibly Windows, subject to Ruby/browser-launch testing. The Chrome plugin and `gh` CLI are workflow enhancers, not requirements for reading a local shelf.
+
+Documentation should distinguish: tested and supported / expected to work but not tested / known degraded path / future platform support.
+
+## Suggested degraded demo
+
+```bash
+SW_NO_OPEN=1 streamweaver canvas-read docs/
+```
+
+Expected: URL printed; user opens it; shelf lists Org and Ruby documents; navigation, TOC, images, diagrams, reading mode all work; no iTerm required. Test independently on macOS-plain-terminal, Linux, WSL, Windows.
+
+## Suggested documentation structure
+
+1. What canvas-read is. 2. The tested macOS happy path. 3. Saving or authoring StreamWeaver Org documents. 4. Opening one document. 5. Opening a shelf. 6. Comparing versions with the left rail and Previous/Next. 7. Working with images. 8. Exporting or packaging. 9. Platform support matrix. 10. No-iTerm degraded path. 11. Known gaps. 12. This case study.