appydave-tools 0.84.0 → 0.85.0

data/docs/planning/query-apps-design.md

@@ -0,0 +1,344 @@
+# query_apps — App File Discovery Tool
+
+**Purpose**: A new CLI tool in appydave-tools that returns file paths from any registered application, using pre-computed named glob patterns stored in each project's `context.globs.json`.
+
+**Status**: Planning
+**Created**: 2026-04-05
+**Backlog**: B041
+
+---
+
+## Problem
+
+When an LLM agent (or David) needs files from a project, they currently have to:
+1. Know the project's directory structure
+2. Manually construct glob patterns
+3. Hope the patterns are still valid as the project evolves
+
+There's no way to say "give me the backend code from FliHub" or "get me all docs from AngelEye" without prior knowledge of each project's layout. `query_brain` solves this for brains, `query_omi` solves it for OMI transcripts — but there's nothing for applications.
+
+---
+
+## Design
+
+### Architecture — Same Pattern as BrainQuery and OmiQuery
+
+```
+bin/query_apps.rb          CLI entry point
+lib/appydave/tools/
+  app_context/
+    app_finder.rb           Core query engine (like BrainQuery, OmiQuery)
+    options.rb              CLI option parsing
+spec/appydave/tools/
+  app_context/
+    app_query_spec.rb       Unit tests
+    options_spec.rb
+```
+
+### Data Flow
+
+```
+locations.json          →  Find app path by key/alias/fuzzy match
+    ↓
+context.globs.json      →  Read named glob patterns from project root
+    ↓
+Glob expansion          →  Resolve patterns against actual filesystem
+    ↓
+File paths (stdout)     →  One per line, pipe to llm_context
+```
+
+### The Sidecar File — `context.globs.json`
+
+Generated by the `system-context` skill alongside `CONTEXT.md`. Lives in each project root.
+
+```json
+{
+  "generated": "2026-04-05",
+  "generator": "system-context",
+  "project": "flihub",
+  "pattern": "rvets",
+  "globs": {
+    "docs": ["docs/**/*.md", "BACKLOG.md"],
+    "types": ["shared/**/*.ts"],
+    "config": ["server/config.json", "*.config.*", ".env.example"],
+    "server": ["server/src/**/*.ts"],
+    "client": ["client/src/**/*.tsx"],
+    "services": ["server/src/services/**/*.ts"],
+    "routes": ["server/src/routes/**/*.ts"],
+    "components": ["client/src/components/**/*.tsx"],
+    "views": ["client/src/views/**/*.tsx"],
+    "tests": ["**/*.test.ts", "**/*.spec.ts"],
+    "styles": ["**/*.css"],
+    "mockups": [".mochaccino/designs/**/*.html"],
+    "planning": ["docs/prd/**/*.md", "docs/planning/**/*.md"],
+    "context": ["CLAUDE.md", "CONTEXT.md", "STEERING.md"]
+  },
+  "aliases": {
+    "backend": ["services", "routes"],
+    "frontend": ["components", "views", "styles"],
+    "api": ["routes", "types"],
+    "data-layer": ["types", "schema"],
+    "react": ["components", "views"],
+    "ui": ["components", "views", "styles"]
+  },
+  "composites": {
+    "understand": ["context", "docs", "types", "config"],
+    "codebase": ["services", "routes", "components", "views"],
+    "all-code": ["server", "client"],
+    "full": ["*"]
+  }
+}
+```
+
+**Key design choices:**
+- **Globs, not file lists** — durable as files are added/removed; only stale if project structure changes fundamentally
+- **Named categories** — "docs", "services", "types" are human-friendly handles
+- **Aliases** — "backend" → services + routes. Handles vague user intent
+- **Composites** — "understand" → context + docs + types + config. Pre-built bundles for common queries
+- **Pattern type** — "rvets", "nextjs", "ruby-gem", "python" enables cross-app queries by architecture
+
+### App Resolution — 4-Tier (Same as BrainQuery)
+
+```
+1. Exact key match:     "flihub"           → locations.json key
+2. Alias match:         "hub"              → locations.json aliases (future)
+3. Jump alias match:    "jfli-hub"         → locations.json jump field
+4. Substring/fuzzy:     "fli"              → partial match on key or description
+```
+
+Source: `~/.config/appydave/locations.json` (the `context` field tells us if `context.globs.json` exists).
+
+### Glob Resolution — 3-Tier
+
+When the user asks for `--glob backend`:
+
+```
+1. Direct glob name:    "services"         → globs.services
+2. Alias match:         "backend"          → aliases.backend → [services, routes]
+3. Composite match:     "understand"       → composites.understand → [context, docs, types, config]
+4. Fuzzy fallback:      "back"             → closest match to "backend"
+```
+
+### CLI Interface
+
+```bash
+# Basic: get docs from FliHub
+query_apps flihub --glob docs
+
+# Alias: "backend" resolves to services + routes
+query_apps flihub --glob backend
+
+# Composite: "understand" = context + docs + types + config
+query_apps angeleye --glob understand
+
+# Multiple globs (comma-separated)
+query_apps flihub --glob docs,types,config
+
+# Cross-app: all RVETS apps, backend code
+query_apps --pattern rvets --glob backend
+
+# List available globs for a project
+query_apps flihub --list
+
+# List all registered apps that have context.globs.json
+query_apps --list-apps
+
+# Meta output (JSON with file counts, not paths)
+query_apps flihub --glob backend --meta
+
+# Pipe to llm_context (the whole point)
+query_apps flihub --glob understand | llm_context --stdin -f content
+query_apps angeleye --glob services | llm_context --stdin -f tree
+```
+
+### Output
+
+Default: one file path per line (same as `query_brain` and `query_omi`).
+
+```
+/Users/davidcruwys/dev/ad/flivideo/flihub/server/src/services/watcher.service.ts
+/Users/davidcruwys/dev/ad/flivideo/flihub/server/src/services/naming.service.ts
+/Users/davidcruwys/dev/ad/flivideo/flihub/server/src/routes/index.ts
+```
+
+With `--meta`:
+```json
+{
+  "app": "flihub",
+  "path": "/Users/davidcruwys/dev/ad/flivideo/flihub",
+  "pattern": "rvets",
+  "matched_globs": ["services", "routes"],
+  "resolved_from": "backend (alias)",
+  "file_count": 14
+}
+```
+
+---
+
+## Standard Vocabulary
+
+These category names are conventions enforced by `system-context` when generating `context.globs.json`. Projects can add custom categories beyond these.
+
+| Category | Aliases | Description |
+|----------|---------|-------------|
+| `docs` | documentation, planning, specs | Markdown documentation |
+| `types` | models, schema, data-layer | Type definitions / data shapes |
+| `config` | settings, configuration | Config files |
+| `services` | backend, business-logic | Server-side logic |
+| `routes` | api, endpoints | API routes |
+| `components` | ui, frontend, react | UI components |
+| `views` | pages, screens | Page-level views |
+| `tests` | specs, test | Test files |
+| `styles` | css, styling | Stylesheets |
+| `context` | meta, about | CLAUDE.md, CONTEXT.md, STEERING.md |
+
+Standard composites:
+
+| Composite | Expands to |
+|-----------|-----------|
+| `understand` | context + docs + types + config |
+| `codebase` | services + routes + components + views |
+| `full` | all glob categories |
+
+### Pattern-Specific Categories
+
+These only appear in projects with the matching `pattern` type:
+
+| Pattern | Extra Categories |
+|---------|-----------------|
+| `rvets` | mockups (`.mochaccino/`), shared |
+| `nextjs` | actions, validation, auth, middleware, decisions (KDD) |
+| `ruby-gem` | lib, bin, gemspec |
+| `python` | src, requirements |
+| `bmad` | bmad-config, bmad-output, workflows |
+
+---
+
+## System-Context Integration
+
+The `system-context` skill (in appydave-plugins) needs a small addition:
+
+1. **After writing CONTEXT.md**, also generate `context.globs.json`
+2. **Detect project pattern** (rvets, nextjs, ruby-gem, python) from package.json / Gemfile / pyproject.toml
+3. **Scan the directory** and map discovered paths to standard category names
+4. **Add project-specific categories** for non-standard directories (e.g., `.mochaccino/` in RVETS apps)
+5. **Generate aliases and composites** using the standard vocabulary plus any project-specific additions
+
+The `context.globs.json` file should be listed in CONTEXT.md's `sources:` frontmatter.
+
+---
+
+## Locations.json Integration
+
+The `context` field in `locations.json` already points to `CONTEXT.md`. We can extend this:
+
+```json
+{
+  "key": "flihub",
+  "path": "/Users/davidcruwys/dev/ad/flivideo/flihub",
+  "context": "CONTEXT.md",
+  "context_globs": "context.globs.json",
+  ...
+}
+```
+
+Or, simpler: `query_apps` can just check for `context.globs.json` in the same directory as `CONTEXT.md` without needing a separate field.
+
+---
+
+## Provenance Chain
+
+```
+system-context skill (generates)
+    ↓
+context.globs.json (sidecar in each project)
+    ↓
+query_apps CLI tool (reads locations.json + globs file, resolves names, expands globs)
+    ↓
+file paths on stdout
+    ↓
+llm_context CLI tool (assembles content from file paths)
+    ↓
+LLM-ready payload
+```
+
+The full query ecosystem:
+
+```
+query_brain   →  brain files      ─┐
+query_omi     →  OMI transcripts   ├─→  llm_context  →  LLM payload
+query_apps    →  app files        ─┘
+```
+
+---
+
+## Skill Wrappers (Thin Shims)
+
+Each query tool should have a corresponding skill in `appydave-plugins/appydave/skills/`:
+
+| CLI Tool | Skill | Status |
+|----------|-------|--------|
+| `query_omi` | `omi-query` | Exists, correct pattern |
+| `query_brain` | (none) | **Needs creating** — thin shim to `query_brain` |
+| `query_apps` | (none) | **Create alongside CLI tool** |
+| `llm_context` | (none) | **Needs creating** — downstream assembler, not a query |
+
+The `focus` skill's `resolve_brain.py` should eventually be replaced by a call to `query_brain --find --meta` — the Ruby tool already has the same resolution logic with better test coverage.
+
+---
+
+## Implementation Phases
+
+### Phase 1: context.globs.json Generation
+- Update `system-context` skill to generate `context.globs.json` alongside `CONTEXT.md`
+- Generate for the 12 projects that already have `CONTEXT.md`
+- Validate standard vocabulary coverage
+
+### Phase 2: query_apps CLI Tool
+- `AppQuery` class following BrainQuery/OmiQuery pattern
+- 4-tier app resolution from locations.json
+- 3-tier glob resolution (direct → alias → composite → fuzzy)
+- `find` (file paths) and `find_meta` (structured JSON) methods
+- CLI entry point at `bin/query_apps.rb`
+- Unit tests with fixtures
+
+### Phase 3: Skill Wrappers
+- `app-query` skill in appydave-plugins (thin shim to `query_apps`)
+- `brain-query` skill (thin shim to `query_brain`)
+- `llm-context` skill (thin shim to `llm_context`)
+
+### Phase 4: Cross-App Queries
+- `--pattern rvets` to query across all apps of a pattern type
+- Aggregate results from multiple `context.globs.json` files
+
+---
+
+## Related Files
+
+| What | Where |
+|------|-------|
+| BrainQuery (reference impl) | `lib/appydave/tools/brain_context/brain_finder.rb` |
+| OmiQuery (reference impl) | `lib/appydave/tools/brain_context/omi_finder.rb` |
+| FileCollector (downstream) | `lib/appydave/tools/llm_context/file_collector.rb` |
+| Locations registry | `~/.config/appydave/locations.json` |
+| System-context skill | `appydave-plugins/appydave/skills/system-context/SKILL.md` |
+| omi-query skill (pattern) | `appydave-plugins/appydave/skills/omi-query/SKILL.md` |
+| Brain query tests | `spec/appydave/tools/brain_context/brain_query_spec.rb` |
+| OMI query tests | `spec/appydave/tools/brain_context/omi_query_spec.rb` |
+
+---
+
+## Open Questions
+
+1. Should `context.globs.json` live at project root or in `.claude/`?
+   - Root is simpler and visible; `.claude/` keeps it with other agent artifacts
+   - Leaning root (next to `CONTEXT.md`)
+
+2. Should fuzzy matching use Levenshtein distance or simpler substring?
+   - BrainQuery uses substring — probably sufficient here too
+
+3. Should `query_apps` live in `brain_context/` module or a new `app_context/` module?
+   - New module — apps are not brains, even though the query pattern is the same
+
+4. Should the `context` field in locations.json be extended to `context_globs`, or should query_apps just look for the file conventionally?
+   - Leaning conventional (just check for `context.globs.json` in the project dir)