@plures/runebook 0.4.0

package/MEMORY.md

@@ -0,0 +1,253 @@
+# RuneBook Cognitive Memory
+
+Local-first "cognitive memory" storage for terminal events, commands, outputs, errors, insights, and suggestions.
+
+## Overview
+
+The cognitive memory system provides persistent storage for terminal activity, enabling pattern analysis, error tracking, and intelligent suggestions. It uses PluresDB as the storage backend with a Rust API layer for performance and type safety.
+
+## Architecture
+
+### Storage Schema
+
+The memory system organizes data into the following collections:
+
+- **sessions**: Terminal session metadata (start time, shell type, working directory)
+- **commands**: Normalized command records (command, args, exit code, duration)
+- **outputs**: Chunked stdout/stderr output (optionally compressed)
+- **errors**: Classified error records (type, severity, context)
+- **insights**: AI/heuristic annotations (patterns, optimizations, warnings)
+- **suggestions**: Ranked suggestions (command, optimization, shortcut, warning, tip)
+- **provenance**: Source tracking (confidence, model/tool used)
+
+### Data Flow
+
+```
+Terminal Event → MemoryEvent → append_event() → PluresDB
+                                    ↓
+                            Schema-specific storage
+                                    ↓
+                    (sessions, commands, outputs, errors, etc.)
+```
+
+## API Reference
+
+### Rust API
+
+The main API is provided by the `MemoryStore` struct:
+
+```rust
+use runebook_lib::memory::*;
+
+// Initialize
+let store = init_memory_store("localhost", 34567, "./pluresdb-data").await?;
+
+// Append event
+let event = MemoryEvent { /* ... */ };
+store.append_event(event).await?;
+
+// List sessions
+let sessions = store.list_sessions().await?;
+
+// Query recent errors
+let errors = store.query_recent_errors(Some(10), None, None).await?;
+
+// Get context window
+let context = store.get_context(&session_id, ChronoDuration::hours(1)).await?;
+
+// Persist suggestion
+store.persist_suggestion(suggestion).await?;
+```
+
+### CLI Commands
+
+```bash
+# Inspect memory storage
+runebook memory inspect
+```
+
+## Memory Model
+
+### Sessions
+
+A session represents a terminal session with:
+- Unique session ID
+- Start/end timestamps
+- Shell type (bash, zsh, nushell, etc.)
+- Initial working directory
+- Hostname and user (optional)
+
+### Commands
+
+Commands are normalized and stored with:
+- Command name (normalized, e.g., "git" not "/usr/bin/git")
+- Arguments array
+- Environment summary (sanitized)
+- Working directory
+- Start/end timestamps
+- Exit code and success status
+- Duration in milliseconds
+- Process ID (if available)
+
+### Outputs
+
+Output chunks are stored separately to enable:
+- Streaming output handling
+- Optional compression (gzip)
+- Efficient retrieval of large outputs
+- Chunk indexing for reconstruction
+
+### Errors
+
+Errors are classified by:
+- Type (exit_code, stderr, timeout, permission, etc.)
+- Severity (low, medium, high, critical)
+- Message and context
+- Associated command and session
+
+### Insights
+
+Insights are AI/heuristic annotations with:
+- Type (pattern, optimization, warning, tip, correlation)
+- Confidence score (0.0 to 1.0)
+- Source (heuristic, ai, rule, etc.)
+- Optional links to commands or sessions
+
+### Suggestions
+
+Suggestions are ranked recommendations with:
+- Type (command, optimization, shortcut, warning, tip)
+- Priority (low, medium, high)
+- Rank score (higher = more relevant)
+- Dismissed/applied status
+
+### Provenance
+
+Provenance tracks the source of data:
+- Entity type and ID
+- Source (terminal, ai, heuristic, user, etc.)
+- Confidence score (if applicable)
+- Model/tool name (if applicable)
+
+## Retention Policy
+
+By default, memory data is retained indefinitely. You can configure retention policies:
+
+- **Max events**: Limit total number of events stored
+- **Retention days**: Automatically delete data older than N days
+- **Manual cleanup**: Use `wipe_all()` for testing/cleanup
+
+### Wiping Memory
+
+To completely wipe all memory data (useful for testing or privacy):
+
+```rust
+store.wipe_all().await?;
+```
+
+**Warning**: This permanently deletes all stored data. Use with caution.
+
+## Encryption
+
+The memory system provides encryption hooks for sensitive data:
+
+- **No-op encryption**: Default (no encryption)
+- **AES-256-GCM**: Application-level encryption (TODO)
+- **PluresDB native**: If PluresDB supports encryption (TODO)
+
+Encryption is optional and can be configured per deployment.
+
+## Migration and Versioning
+
+The schema includes a migration system for schema evolution:
+
+- Current schema version: 1
+- Automatic migration on initialization
+- Version tracking in PluresDB
+
+To add a new migration:
+
+1. Increment `CURRENT_SCHEMA_VERSION` in `migration.rs`
+2. Add migration logic in `migrate_to_version()`
+3. Test migration with existing data
+
+## Performance Considerations
+
+- **Streaming output**: Outputs are chunked to handle large streams
+- **Compression**: Optional gzip compression for outputs
+- **Indexing**: Key prefixes enable efficient queries
+- **Async operations**: All operations are async for non-blocking I/O
+
+## Testing
+
+### Integration Tests
+
+Tests require a running PluresDB server:
+
+```bash
+# Start PluresDB server
+pluresdb --port 34567 --data-dir ./test-pluresdb-data
+
+# Run tests
+cargo test --package runebook
+```
+
+### Property Tests
+
+Schema roundtrip tests verify data integrity:
+
+- Session roundtrip
+- Command roundtrip
+- Suggestion roundtrip
+
+## Configuration
+
+### PluresDB Connection
+
+Default configuration:
+- Host: `localhost`
+- Port: `34567`
+- Data directory: `./pluresdb-data`
+
+### Environment Variables
+
+- `PLURESDB_HOST`: Override host
+- `PLURESDB_PORT`: Override port
+- `PLURESDB_DATA_DIR`: Override data directory
+
+## Troubleshooting
+
+### PluresDB Not Available
+
+If you see "PluresDB server not available":
+
+1. Check if PluresDB is running: `curl http://localhost:34567/health`
+2. Start PluresDB: `pluresdb --port 34567`
+3. Check firewall/network settings
+
+### Migration Errors
+
+If migrations fail:
+
+1. Check PluresDB logs
+2. Verify schema version: `memory:schema:version` key
+3. Manually run migrations if needed
+
+### Performance Issues
+
+For large datasets:
+
+1. Enable output compression
+2. Implement retention policies
+3. Consider archiving old data
+
+## Future Enhancements
+
+- [ ] Full encryption implementation (AES-256-GCM)
+- [ ] PluresDB native encryption integration
+- [ ] Advanced querying (time ranges, filters)
+- [ ] Data export/import
+- [ ] Backup and restore
+- [ ] Cross-session pattern analysis
+- [ ] Real-time event streaming
+

package/NIXOS.md

@@ -0,0 +1,357 @@
+# NixOS Support for RuneBook
+
+RuneBook is fully integrated with NixOS and Nix Flakes for reproducible builds and development.
+
+## Quick Start
+
+### Development Environment
+
+Enter the development shell:
+
+```bash
+nix develop
+```
+
+This provides:
+- Node.js 20
+- Rust toolchain (stable, with rustfmt and clippy)
+- Tauri CLI
+- All system dependencies (webkitgtk, librsvg, etc.)
+- TypeScript, Vitest, and other dev tools
+
+### Building Packages
+
+Build the Tauri application:
+
+```bash
+nix build .#runebook
+```
+
+Build the headless agent CLI:
+
+```bash
+nix build .#runebook-agent
+```
+
+### Running Applications
+
+Run the Tauri app:
+
+```bash
+nix run .#runebook
+```
+
+Run the agent CLI:
+
+```bash
+nix run .#runebook-agent -- agent status
+nix run .#runebook-agent -- observer enable
+```
+
+## Package Outputs
+
+The flake provides the following outputs:
+
+- `packages.runebook` - The Tauri desktop application
+- `packages.runebook-agent` - Headless CLI agent wrapper
+- `devShells.default` - Development environment with all tools
+- `apps.runebook` - App launcher for the Tauri app
+- `apps.runebook-agent` - App launcher for the agent CLI
+- `nixosModules.default` - NixOS module for systemd service
+
+View all outputs:
+
+```bash
+nix flake show
+```
+
+## NixOS Module
+
+The flake includes a NixOS module for running `runebook-agent` as a systemd user service.
+
+### Basic Configuration
+
+Add to your `configuration.nix`:
+
+```nix
+{
+  imports = [
+    /path/to/runebook/nixos-module.nix
+  ];
+
+  services.runebook-agent = {
+    enable = true;
+    captureEvents = true;
+    analyzePatterns = true;
+    suggestImprovements = true;
+    dataDir = "/var/lib/runebook-agent/data";
+    maxEvents = 10000;
+    retentionDays = 30;
+  };
+}
+```
+
+### Using the Flake Module
+
+If using flakes in your NixOS configuration:
+
+```nix
+{
+  inputs.runebook.url = "github:plures/runebook";
+
+  outputs = { self, nixpkgs, runebook }: {
+    nixosConfigurations.your-host = nixpkgs.lib.nixosSystem {
+      modules = [
+        runebook.nixosModules.default
+        {
+          services.runebook-agent = {
+            enable = true;
+            captureEvents = true;
+            # ... other options
+          };
+        }
+      ];
+    };
+  };
+}
+```
+
+### Configuration Options
+
+- `enable` - Enable the runebook-agent service (default: `false`)
+- `captureEvents` - Enable event capture (default: `true`)
+- `analyzePatterns` - Enable pattern analysis (default: `true`)
+- `suggestImprovements` - Enable suggestion generation (default: `true`)
+- `dataDir` - Data directory for agent storage (default: `/var/lib/runebook-agent/data`)
+- `maxEvents` - Maximum number of events to store (default: `10000`)
+- `retentionDays` - Number of days to retain events (default: `30`)
+- `openaiApiKey` - OpenAI API key (default: `null`)
+
+**⚠️ Security Warning**: Setting `openaiApiKey` in the NixOS configuration will store it in the Nix store. Prefer using environment variables, agenix, or sops-nix instead.
+
+### Secure Secret Management
+
+#### Using Environment Variables
+
+Set secrets via systemd environment files:
+
+```nix
+{
+  services.runebook-agent = {
+    enable = true;
+    # Don't set openaiApiKey here
+  };
+
+  systemd.user.services.runebook-agent = {
+    serviceConfig.EnvironmentFile = "/etc/runebook/secrets.env";
+  };
+}
+```
+
+Create `/etc/runebook/secrets.env`:
+
+```bash
+OPENAI_API_KEY=sk-...
+```
+
+#### Using agenix
+
+If using [agenix](https://github.com/ryantm/agenix):
+
+```nix
+{
+  age.secrets.runebook-openai = {
+    file = ./secrets/runebook-openai.age;
+    owner = "your-user";
+  };
+
+  systemd.user.services.runebook-agent = {
+    serviceConfig.EnvironmentFile = config.age.secrets.runebook-openai.path;
+  };
+}
+```
+
+#### Using sops-nix
+
+If using [sops-nix](https://github.com/Mic92/sops-nix):
+
+```nix
+{
+  sops.secrets."runebook/openai_key" = {
+    owner = "your-user";
+    path = "/run/secrets/runebook-openai";
+  };
+
+  systemd.user.services.runebook-agent = {
+    serviceConfig.EnvironmentFile = config.sops.secrets."runebook/openai_key".path;
+  };
+}
+```
+
+### Disabling the Service
+
+To disable the service:
+
+```nix
+{
+  services.runebook-agent.enable = false;
+}
+```
+
+Or remove the service configuration entirely.
+
+### Manual Service Management
+
+If the service is enabled, manage it with systemd:
+
+```bash
+# Check status
+systemctl --user status runebook-agent
+
+# Start service
+systemctl --user start runebook-agent
+
+# Stop service
+systemctl --user stop runebook-agent
+
+# View logs
+journalctl --user -u runebook-agent -f
+```
+
+## CI/CD
+
+The project includes GitHub Actions workflows that:
+
+- Run `nix flake check` to validate the flake
+- Build packages on multiple platforms
+- Run tests (Rust and TypeScript)
+- Check formatting (rustfmt, TypeScript)
+
+See `.github/workflows/ci.yml` for details.
+
+## Reproducible Builds
+
+All dependencies are pinned via Nix, ensuring reproducible builds:
+
+- Rust toolchain version is fixed
+- Node.js version is fixed
+- System libraries are pinned to nixpkgs versions
+
+To update dependencies, update the `nixpkgs` input in `flake.nix`:
+
+```nix
+{
+  inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";  # or a specific commit
+}
+```
+
+Then update the lock file:
+
+```bash
+nix flake update
+```
+
+## Troubleshooting
+
+### Build Failures
+
+If builds fail, check:
+
+1. **Nix version**: Ensure you're using Nix 2.4+ with flakes enabled
+2. **System dependencies**: The flake should handle these automatically
+3. **Network access**: First build requires downloading dependencies
+
+### Service Not Starting
+
+If the systemd service fails to start:
+
+1. Check logs: `journalctl --user -u runebook-agent -n 50`
+2. Verify data directory permissions: `ls -la /var/lib/runebook-agent`
+3. Check configuration: `cat /var/lib/runebook-agent/agent-config.json`
+
+### Missing Dependencies
+
+If you encounter missing dependencies:
+
+1. Enter the dev shell: `nix develop`
+2. All dependencies should be available automatically
+3. If not, check `flake.nix` and add missing packages to `devShells.default.buildInputs`
+
+## Development
+
+### Adding New Dependencies
+
+**Rust dependencies**: Add to `src-tauri/Cargo.toml` as usual.
+
+**Node.js dependencies**: Add to `package.json` as usual, then update the `npmDepsHash` in `flake.nix`:
+
+```bash
+# After adding dependencies, get the new hash:
+nix build .#runebook-frontend 2>&1 | grep "got:" | head -1
+```
+
+Or manually calculate:
+```bash
+nix-prefetch-url --unpack --type sha256 $(nix eval --raw .#runebook-frontend.src 2>/dev/null || echo "file://$(pwd)")
+```
+
+**System dependencies**: Add to `nativeBuildInputs` or `buildInputs` in `flake.nix`.
+
+**Note**: The initial `npmDepsHash` in `flake.nix` is set to a placeholder. After the first successful build, Nix will provide the correct hash. You can then update it in the flake for faster subsequent builds.
+
+### Pre-commit Hooks (Optional)
+
+The dev shell includes `pre-commit` for optional git hooks. To enable:
+
+```bash
+nix develop
+pre-commit install
+```
+
+## Examples
+
+### Minimal NixOS Configuration
+
+```nix
+{
+  imports = [ ./nixos-module.nix ];
+
+  services.runebook-agent.enable = true;
+}
+```
+
+### Full NixOS Configuration with Secrets
+
+```nix
+{
+  imports = [ ./nixos-module.nix ];
+
+  services.runebook-agent = {
+    enable = true;
+    captureEvents = true;
+    analyzePatterns = true;
+    suggestImprovements = true;
+    dataDir = "/home/user/.local/share/runebook";
+    maxEvents = 50000;
+    retentionDays = 90;
+  };
+
+  # Use agenix for secrets
+  age.secrets.runebook-openai = {
+    file = ./secrets/openai.age;
+    owner = "user";
+  };
+
+  systemd.user.services.runebook-agent = {
+    serviceConfig.EnvironmentFile = config.age.secrets.runebook-openai.path;
+  };
+}
+```
+
+## Further Reading
+
+- [Nix Flakes Documentation](https://nixos.wiki/wiki/Flakes)
+- [NixOS Modules](https://nixos.wiki/wiki/NixOS_modules)
+- [Tauri Prerequisites](https://tauri.app/guides/prerequisites/)
+- [RuneBook Architecture](./ARCHITECTURE.md)
+