appydave-tools 0.83.0 → 0.85.0

data/docs/planning/query-apps-design.md

@@ -0,0 +1,344 @@
+# query_apps — App File Discovery Tool
+
+**Purpose**: A new CLI tool in appydave-tools that returns file paths from any registered application, using pre-computed named glob patterns stored in each project's `context.globs.json`.
+
+**Status**: Planning
+**Created**: 2026-04-05
+**Backlog**: B041
+
+---
+
+## Problem
+
+When an LLM agent (or David) needs files from a project, they currently have to:
+1. Know the project's directory structure
+2. Manually construct glob patterns
+3. Hope the patterns are still valid as the project evolves
+
+There's no way to say "give me the backend code from FliHub" or "get me all docs from AngelEye" without prior knowledge of each project's layout. `query_brain` solves this for brains, `query_omi` solves it for OMI transcripts — but there's nothing for applications.
+
+---
+
+## Design
+
+### Architecture — Same Pattern as BrainQuery and OmiQuery
+
+```
+bin/query_apps.rb          CLI entry point
+lib/appydave/tools/
+  app_context/
+    app_finder.rb           Core query engine (like BrainQuery, OmiQuery)
+    options.rb              CLI option parsing
+spec/appydave/tools/
+  app_context/
+    app_query_spec.rb       Unit tests
+    options_spec.rb
+```
+
+### Data Flow
+
+```
+locations.json          →  Find app path by key/alias/fuzzy match
+    ↓
+context.globs.json      →  Read named glob patterns from project root
+    ↓
+Glob expansion          →  Resolve patterns against actual filesystem
+    ↓
+File paths (stdout)     →  One per line, pipe to llm_context
+```
+
+### The Sidecar File — `context.globs.json`
+
+Generated by the `system-context` skill alongside `CONTEXT.md`. Lives in each project root.
+
+```json
+{
+  "generated": "2026-04-05",
+  "generator": "system-context",
+  "project": "flihub",
+  "pattern": "rvets",
+  "globs": {
+    "docs": ["docs/**/*.md", "BACKLOG.md"],
+    "types": ["shared/**/*.ts"],
+    "config": ["server/config.json", "*.config.*", ".env.example"],
+    "server": ["server/src/**/*.ts"],
+    "client": ["client/src/**/*.tsx"],
+    "services": ["server/src/services/**/*.ts"],
+    "routes": ["server/src/routes/**/*.ts"],
+    "components": ["client/src/components/**/*.tsx"],
+    "views": ["client/src/views/**/*.tsx"],
+    "tests": ["**/*.test.ts", "**/*.spec.ts"],
+    "styles": ["**/*.css"],
+    "mockups": [".mochaccino/designs/**/*.html"],
+    "planning": ["docs/prd/**/*.md", "docs/planning/**/*.md"],
+    "context": ["CLAUDE.md", "CONTEXT.md", "STEERING.md"]
+  },
+  "aliases": {
+    "backend": ["services", "routes"],
+    "frontend": ["components", "views", "styles"],
+    "api": ["routes", "types"],
+    "data-layer": ["types", "schema"],
+    "react": ["components", "views"],
+    "ui": ["components", "views", "styles"]
+  },
+  "composites": {
+    "understand": ["context", "docs", "types", "config"],
+    "codebase": ["services", "routes", "components", "views"],
+    "all-code": ["server", "client"],
+    "full": ["*"]
+  }
+}
+```
+
+**Key design choices:**
+- **Globs, not file lists** — durable as files are added/removed; only stale if project structure changes fundamentally
+- **Named categories** — "docs", "services", "types" are human-friendly handles
+- **Aliases** — "backend" → services + routes. Handles vague user intent
+- **Composites** — "understand" → context + docs + types + config. Pre-built bundles for common queries
+- **Pattern type** — "rvets", "nextjs", "ruby-gem", "python" enables cross-app queries by architecture
+
+### App Resolution — 4-Tier (Same as BrainQuery)
+
+```
+1. Exact key match:     "flihub"           → locations.json key
+2. Alias match:         "hub"              → locations.json aliases (future)
+3. Jump alias match:    "jfli-hub"         → locations.json jump field
+4. Substring/fuzzy:     "fli"              → partial match on key or description
+```
+
+Source: `~/.config/appydave/locations.json` (the `context` field tells us if `context.globs.json` exists).
+
+### Glob Resolution — 3-Tier
+
+When the user asks for `--glob backend`:
+
+```
+1. Direct glob name:    "services"         → globs.services
+2. Alias match:         "backend"          → aliases.backend → [services, routes]
+3. Composite match:     "understand"       → composites.understand → [context, docs, types, config]
+4. Fuzzy fallback:      "back"             → closest match to "backend"
+```
+
+### CLI Interface
+
+```bash
+# Basic: get docs from FliHub
+query_apps flihub --glob docs
+
+# Alias: "backend" resolves to services + routes
+query_apps flihub --glob backend
+
+# Composite: "understand" = context + docs + types + config
+query_apps angeleye --glob understand
+
+# Multiple globs (comma-separated)
+query_apps flihub --glob docs,types,config
+
+# Cross-app: all RVETS apps, backend code
+query_apps --pattern rvets --glob backend
+
+# List available globs for a project
+query_apps flihub --list
+
+# List all registered apps that have context.globs.json
+query_apps --list-apps
+
+# Meta output (JSON with file counts, not paths)
+query_apps flihub --glob backend --meta
+
+# Pipe to llm_context (the whole point)
+query_apps flihub --glob understand | llm_context --stdin -f content
+query_apps angeleye --glob services | llm_context --stdin -f tree
+```
+
+### Output
+
+Default: one file path per line (same as `query_brain` and `query_omi`).
+
+```
+/Users/davidcruwys/dev/ad/flivideo/flihub/server/src/services/watcher.service.ts
+/Users/davidcruwys/dev/ad/flivideo/flihub/server/src/services/naming.service.ts
+/Users/davidcruwys/dev/ad/flivideo/flihub/server/src/routes/index.ts
+```
+
+With `--meta`:
+```json
+{
+  "app": "flihub",
+  "path": "/Users/davidcruwys/dev/ad/flivideo/flihub",
+  "pattern": "rvets",
+  "matched_globs": ["services", "routes"],
+  "resolved_from": "backend (alias)",
+  "file_count": 14
+}
+```
+
+---
+
+## Standard Vocabulary
+
+These category names are conventions enforced by `system-context` when generating `context.globs.json`. Projects can add custom categories beyond these.
+
+| Category | Aliases | Description |
+|----------|---------|-------------|
+| `docs` | documentation, planning, specs | Markdown documentation |
+| `types` | models, schema, data-layer | Type definitions / data shapes |
+| `config` | settings, configuration | Config files |
+| `services` | backend, business-logic | Server-side logic |
+| `routes` | api, endpoints | API routes |
+| `components` | ui, frontend, react | UI components |
+| `views` | pages, screens | Page-level views |
+| `tests` | specs, test | Test files |
+| `styles` | css, styling | Stylesheets |
+| `context` | meta, about | CLAUDE.md, CONTEXT.md, STEERING.md |
+
+Standard composites:
+
+| Composite | Expands to |
+|-----------|-----------|
+| `understand` | context + docs + types + config |
+| `codebase` | services + routes + components + views |
+| `full` | all glob categories |
+
+### Pattern-Specific Categories
+
+These only appear in projects with the matching `pattern` type:
+
+| Pattern | Extra Categories |
+|---------|-----------------|
+| `rvets` | mockups (`.mochaccino/`), shared |
+| `nextjs` | actions, validation, auth, middleware, decisions (KDD) |
+| `ruby-gem` | lib, bin, gemspec |
+| `python` | src, requirements |
+| `bmad` | bmad-config, bmad-output, workflows |
+
+---
+
+## System-Context Integration
+
+The `system-context` skill (in appydave-plugins) needs a small addition:
+
+1. **After writing CONTEXT.md**, also generate `context.globs.json`
+2. **Detect project pattern** (rvets, nextjs, ruby-gem, python) from package.json / Gemfile / pyproject.toml
+3. **Scan the directory** and map discovered paths to standard category names
+4. **Add project-specific categories** for non-standard directories (e.g., `.mochaccino/` in RVETS apps)
+5. **Generate aliases and composites** using the standard vocabulary plus any project-specific additions
+
+The `context.globs.json` file should be listed in CONTEXT.md's `sources:` frontmatter.
+
+---
+
+## Locations.json Integration
+
+The `context` field in `locations.json` already points to `CONTEXT.md`. We can extend this:
+
+```json
+{
+  "key": "flihub",
+  "path": "/Users/davidcruwys/dev/ad/flivideo/flihub",
+  "context": "CONTEXT.md",
+  "context_globs": "context.globs.json",
+  ...
+}
+```
+
+Or, simpler: `query_apps` can just check for `context.globs.json` in the same directory as `CONTEXT.md` without needing a separate field.
+
+---
+
+## Provenance Chain
+
+```
+system-context skill (generates)
+    ↓
+context.globs.json (sidecar in each project)
+    ↓
+query_apps CLI tool (reads locations.json + globs file, resolves names, expands globs)
+    ↓
+file paths on stdout
+    ↓
+llm_context CLI tool (assembles content from file paths)
+    ↓
+LLM-ready payload
+```
+
+The full query ecosystem:
+
+```
+query_brain   →  brain files      ─┐
+query_omi     →  OMI transcripts   ├─→  llm_context  →  LLM payload
+query_apps    →  app files        ─┘
+```
+
+---
+
+## Skill Wrappers (Thin Shims)
+
+Each query tool should have a corresponding skill in `appydave-plugins/appydave/skills/`:
+
+| CLI Tool | Skill | Status |
+|----------|-------|--------|
+| `query_omi` | `omi-query` | Exists, correct pattern |
+| `query_brain` | (none) | **Needs creating** — thin shim to `query_brain` |
+| `query_apps` | (none) | **Create alongside CLI tool** |
+| `llm_context` | (none) | **Needs creating** — downstream assembler, not a query |
+
+The `focus` skill's `resolve_brain.py` should eventually be replaced by a call to `query_brain --find --meta` — the Ruby tool already has the same resolution logic with better test coverage.
+
+---
+
+## Implementation Phases
+
+### Phase 1: context.globs.json Generation
+- Update `system-context` skill to generate `context.globs.json` alongside `CONTEXT.md`
+- Generate for the 12 projects that already have `CONTEXT.md`
+- Validate standard vocabulary coverage
+
+### Phase 2: query_apps CLI Tool
+- `AppQuery` class following BrainQuery/OmiQuery pattern
+- 4-tier app resolution from locations.json
+- 3-tier glob resolution (direct → alias → composite → fuzzy)
+- `find` (file paths) and `find_meta` (structured JSON) methods
+- CLI entry point at `bin/query_apps.rb`
+- Unit tests with fixtures
+
+### Phase 3: Skill Wrappers
+- `app-query` skill in appydave-plugins (thin shim to `query_apps`)
+- `brain-query` skill (thin shim to `query_brain`)
+- `llm-context` skill (thin shim to `llm_context`)
+
+### Phase 4: Cross-App Queries
+- `--pattern rvets` to query across all apps of a pattern type
+- Aggregate results from multiple `context.globs.json` files
+
+---
+
+## Related Files
+
+| What | Where |
+|------|-------|
+| BrainQuery (reference impl) | `lib/appydave/tools/brain_context/brain_finder.rb` |
+| OmiQuery (reference impl) | `lib/appydave/tools/brain_context/omi_finder.rb` |
+| FileCollector (downstream) | `lib/appydave/tools/llm_context/file_collector.rb` |
+| Locations registry | `~/.config/appydave/locations.json` |
+| System-context skill | `appydave-plugins/appydave/skills/system-context/SKILL.md` |
+| omi-query skill (pattern) | `appydave-plugins/appydave/skills/omi-query/SKILL.md` |
+| Brain query tests | `spec/appydave/tools/brain_context/brain_query_spec.rb` |
+| OMI query tests | `spec/appydave/tools/brain_context/omi_query_spec.rb` |
+
+---
+
+## Open Questions
+
+1. Should `context.globs.json` live at project root or in `.claude/`?
+   - Root is simpler and visible; `.claude/` keeps it with other agent artifacts
+   - Leaning root (next to `CONTEXT.md`)
+
+2. Should fuzzy matching use Levenshtein distance or simpler substring?
+   - BrainQuery uses substring — probably sufficient here too
+
+3. Should `query_apps` live in `brain_context/` module or a new `app_context/` module?
+   - New module — apps are not brains, even though the query pattern is the same
+
+4. Should the `context` field in locations.json be extended to `context_globs`, or should query_apps just look for the file conventionally?
+   - Leaning conventional (just check for `context.globs.json` in the project dir)

data/docs/planning/query-location-feature/IMPLEMENTATION_PLAN.md

@@ -0,0 +1,142 @@
+# FR-NEW: Query Location & App Registry Integration
+
+**Status**: Proposed
+**Priority**: Medium
+**Created**: 2026-04-05
+**Relates to**: `query_brain`, `llm_context` pipeline; `locations.json`, `apps.json`
+
+---
+
+## Problem
+
+`query_brain` and `llm_context` work well for brain files but have no way to resolve project paths from the existing registries. When a user says "package the FliVideo project for LLM research", there is no single command that:
+1. Looks up the project path from `locations.json` or `apps.json`
+2. Knows which file groupings are relevant (docs, code, prompts, tests)
+3. Feeds those paths directly into `llm_context`
+
+Instead, the user must manually look up the path and write the `llm_context` invocation by hand every time.
+
+---
+
+## Proposed Solution
+
+### Option A — `query_location` CLI (preferred)
+
+New binary `bin/query_location.rb` that queries `~/.config/appydave/locations.json` and `~/.config/appydave/apps.json` by key, name, type, or brand.
+
+```bash
+# Find a project by key or name
+query_location --find flivideo
+query_location --find ss-prompt
+
+# List all products
+query_location --type product
+
+# Get path only (for piping)
+query_location --find flivideo --path-only
+
+# Get JSON metadata
+query_location --find flivideo --meta
+
+# List all apps from apps.json
+query_location --apps
+query_location --apps --status active
+```
+
+Output modes:
+- Default: path on a single line (pipeable to `llm_context`)
+- `--meta`: JSON with key, path, description, type, status
+- `--path-only`: bare path string
+
+### Option B — Extend `jump.rb` with a query subcommand
+
+Add `jump.rb query --find flivideo` rather than a new binary. Lower overhead, shares existing location-loading code.
+
+**Recommendation**: Option B — reuse `jump.rb` infrastructure, add a `query` subcommand that outputs path/meta. Less surface area, same capability.
+
+---
+
+## File Groupings (Phase 2)
+
+Once location lookup works, the second gap is "which files within a project are relevant for LLM research?" Today the user must specify patterns manually.
+
+**Proposed**: A `.llm-groups.yaml` sidecar file at the project root declaring named groupings:
+
+```yaml
+# .llm-groups.yaml
+groups:
+  docs:
+    - "docs/**/*.md"
+    - "README.md"
+    - "CLAUDE.md"
+  code:
+    - "lib/**/*.rb"
+    - "bin/**/*.rb"
+  tests:
+    - "spec/**/*.rb"
+  prompts:
+    - "poem/**/*.json"
+    - "poem/**/*.yaml"
+```
+
+Then:
+```bash
+# Package docs group for LLM
+query_location --find appydave-tools --path-only | xargs -I{} llm_context -b {} --groups docs
+
+# Or inline
+llm_context -b ~/dev/ad/appydave-tools --groups docs,code
+```
+
+`llm_context` would read `.llm-groups.yaml` if present and expand the named groupings into `-i` patterns.
+
+---
+
+## Pipeline Vision (Full)
+
+```bash
+# Today (manual, fragile)
+llm_context -b ~/dev/ad/flivideo -i 'lib/**/*.rb' -i 'docs/**/*.md' -f content -o clipboard
+
+# After this feature (location-aware)
+query_location --find flivideo --path-only | xargs -I{} llm_context -b {} --groups docs,code -f content -o clipboard
+
+# Or shorthand once groups file exists
+llm_context --project flivideo --groups docs -o clipboard
+```
+
+---
+
+## Acceptance Criteria
+
+- [ ] `jump.rb query --find <term>` returns matching location path(s)
+- [ ] `jump.rb query --find <term> --meta` returns JSON with key, path, description, type
+- [ ] `jump.rb query --type product` lists all product locations
+- [ ] Output is pipeable to `llm_context -b`
+- [ ] Apps from `apps.json` are also queryable (or a separate `--apps` flag)
+- [ ] Spec coverage for new query subcommand
+
+### Phase 2 (separate backlog item)
+- [ ] `llm_context` reads `.llm-groups.yaml` and expands `--groups` flag into `-i` patterns
+- [ ] `.llm-groups.yaml` standard defined and documented
+- [ ] At least 3 projects have `.llm-groups.yaml` files (appydave-tools, flivideo, ss-prompt)
+
+---
+
+## Related Systems
+
+- `~/.config/appydave/locations.json` — 70+ project locations (source of truth for `jump`)
+- `~/.config/appydave/apps.json` — app registry with ports, start scripts, status
+- `query_brain` — parallel tool for brain files; this is the project equivalent
+- `llm_context` — the consumer; needs paths to operate
+- `system-comprehension-pattern.md` — the pattern that produces the mental model; needs groupings to be useful at scale
+- `appydave:system-context` skill — MISSING skill that generates `CONTEXT.md` per project; relates to Phase 2 groupings
+
+---
+
+## Notes
+
+- Do NOT create a separate JSON registry — `locations.json` is already the source of truth
+- The `jump` skill already knows about `locations.json`; extending `jump.rb` keeps things consistent
+- Phase 2 (`.llm-groups.yaml`) is the real value unlock — Phase 1 is just plumbing
+- This feature is what enables the "package project X for LLM research" workflow without manual path lookup every time

data/docs/planning/query-location-feature/system-context-gap-analysis.md

@@ -0,0 +1,107 @@
+# Gap Analysis: system-context vs system-comprehension-pattern
+
+**Context**: The `CONTEXT.md` file in this repo was generated by `/appydave:system-context` — a skill
+that doesn't yet exist in appydave-plugins. This note compares what that skill *produced* (the output)
+against the `system-comprehension-pattern.md` in the prompt-patterns brain to assess the gap.
+
+**Created**: 2026-04-05
+
+---
+
+## What system-context Produced (CONTEXT.md output)
+
+Looking at `CONTEXT.md` (generated 2026-04-03), the skill produced:
+
+```yaml
+# Frontmatter
+generated: 2026-04-03
+generator: system-context
+status: snapshot
+sources: [README.md, gemspec, lib/tools.rb, brand_resolver.rb, project_resolver.rb, jump/commands/generate.rb, docs/dam/batch-s3-listing-requirements.md, CHANGELOG.md (versions 0.76.0-0.77.7)]
+regenerate: "Run /appydave:system-context in the repo root"
+```
+
+Sections produced:
+1. **Purpose** — one sentence
+2. **Domain Concepts** — key vocabulary (Brand, Project, DAM, Project Naming, Configuration, Multi-channel)
+3. **Design Decisions** — 6 decisions with rationale
+4. **Scope Limits** — 6 explicit "does NOT" statements
+
+---
+
+## What system-comprehension-pattern Produces (Level 1)
+
+The 8-dimension Level 1 prompt produces:
+
+1. REAL PURPOSE — pain solved, world without it
+2. MENTAL MODEL — central metaphor or abstraction
+3. CORE ABSTRACTIONS — 3-5 building blocks and how they relate
+4. KEY WORKFLOWS — 2-4 end-to-end workflows (intent → outcome, not API calls)
+5. DESIGN DECISIONS — why it's built this way, alternatives considered
+6. NON-OBVIOUS CONSTRAINTS — things that look like they should work but don't
+7. EXPERT VS BEGINNER — mental shift from competent to expert
+8. SCOPE LIMITS — explicit non-scope + what handles those things instead
+
+---
+
+## Gap Analysis
+
+| Dimension | system-context | system-comprehension Level 1 |
+|-----------|---------------|------------------------------|
+| Purpose / Real problem | ✅ One sentence | ✅ Full explanation + "world without it" |
+| Domain concepts / Core abstractions | ✅ Reasonable depth | ✅ Richer — explains relationships between concepts |
+| Mental model / Central metaphor | ❌ Not present | ✅ The central "how does the system think?" question |
+| Key workflows | ❌ Not present | ✅ End-to-end workflows (intent → outcome) |
+| Design decisions | ✅ Good — 6 decisions with rationale | ✅ Same coverage + alternatives considered |
+| Non-obvious constraints | ❌ Not present | ✅ The "looks like it should work but doesn't" list |
+| Expert vs beginner mental shift | ❌ Not present | ✅ The hardest thing to get from docs |
+| Scope limits | ✅ Good — 6 explicit "does NOT" statements | ✅ Same + what handles those things instead |
+
+**Score: system-context covers ~4/8 dimensions well. Missing the 4 that are hardest to get from docs.**
+
+---
+
+## The Core Gap
+
+`system-context` is essentially an **automated documentation snapshot** — it reads source files and
+produces a structured summary. It does this well and efficiently (no LLM needed, or minimal).
+
+`system-comprehension-pattern` Level 1 is a **mental model prompt** — it produces reasoning about the
+system, not just facts about it. The 4 missing dimensions (mental model, key workflows, non-obvious
+constraints, expert vs beginner) are precisely the things that make a reasoning partner vs a fact retriever.
+
+**Practical implication**: `CONTEXT.md` is useful as orientation context for an LLM session. But an LLM
+given only `CONTEXT.md` would produce competent-but-generic answers. An LLM given the Level 1
+comprehension output would produce expert-level, system-aware answers.
+
+---
+
+## What This Means for the system-context Skill (when built)
+
+When `/appydave:system-context` is built as a proper skill, it should:
+
+1. **Keep what works**: automated extraction of purpose, domain concepts, design decisions, scope limits
+   (these come from README, gemspec, source files — no deep reasoning needed)
+
+2. **Add what's missing**: either
+   - Prompt the LLM to produce the missing 4 dimensions as part of the generation
+   - Or leave a `## Pending Comprehension` section flagging that Level 1 comprehension hasn't been run
+
+3. **Add a query_brain groupings block** — this is the new gap not in either system:
+   ```yaml
+   llm_groups:
+     docs: ["docs/**/*.md", "README.md", "CLAUDE.md"]
+     code: ["lib/**/*.rb", "bin/**/*.rb"]
+     tests: ["spec/**/*.rb"]
+   ```
+   So that `llm_context --project appydave-tools --groups docs` works without manual pattern entry.
+
+---
+
+## Recommendation
+
+- **`system-context` as-is**: Good for quick onboarding context. Use it.
+- **Before deep work**: Also run Level 1 comprehension from `system-comprehension-pattern.md` and append
+  the output to `CONTEXT.md` or keep it in a separate `COMPREHENSION.md`.
+- **Long-term**: Build the `system-context` skill to include LLM-driven comprehension + groupings block.
+  The skill that produces `CONTEXT.md` should be the same skill that makes the project queryable.