know-cli 0.3.0__tar.gz → 0.3.1__tar.gz

{know_cli-0.3.0 → know_cli-0.3.1}/.gitignore

@@ -5,3 +5,4 @@ dist/
 build/
 *.egg-info/
 .know/cache/
+.know/

{know_cli-0.3.0 → know_cli-0.3.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: know-cli
-Version: 0.3.0
+Version: 0.3.1
 Summary: Context Intelligence for AI Coding Agents — smart, token-budgeted code context
 Project-URL: Homepage, https://github.com/vic/know-cli
 Project-URL: Repository, https://github.com/vic/know-cli
@@ -127,17 +127,39 @@ know recall "how does auth work?"
 
 Memories are automatically included in `know context` results. Your agent gets smarter over time.
 
-### 🔍 Semantic Search — `know search`
+### 🔍 Search — `know search` (Hybrid like qmd)
 
-Real embeddings, not grep. Understands meaning, not just keywords.
+Hybrid retrieval by default:
+- **Lexical**: SQLite **FTS5** (BM25 ranking) over chunk text + metadata
+- **Vector**: embeddings via **fastembed** (when installed)
+- **Deterministic rerank**: **Reciprocal Rank Fusion (RRF)**
 
 ```bash
+# Default: hybrid (FTS + vectors when available)
 know search "error handling"
+
+# JSON output (stable schema)
+know --json search "database connection pooling" --top-k 10
+# (also supported after the command)
 know search "database connection pooling" --top-k 10 --json
-know search "authentication" --chunk  # Search at function level
+
+Notes:
+- Queries with punctuation (e.g. `click.option`) are normalized for lexical search.
+- Results downweight docs/tests so source code tends to rank higher.
+# Force engine
+know search "authentication" --engine lexical
+know search "authentication" --engine vector
+
+# Filters
+know search "jwt" --path src/auth/ --type py
+
+# Better snippets
+know search "retry" --line-numbers
 ```
 
-Uses [fastembed](https://github.com/qdrant/fastembed) (BAAI/bge-small-en-v1.5) — runs locally, no API calls.
+Notes:
+- The FTS index is stored in: `.know/cache/index.db` (table: `chunks_fts`).
+- If **FTS5** and/or **fastembed** are unavailable, `know search` gracefully falls back.
 
 ### 📊 Import Graph — `know graph`
 
@@ -173,6 +195,77 @@ know mcp config                   # Print Claude Desktop config
 
 **MCP Resources:** `codebase://digest`, `codebase://structure`, `codebase://memories`
 
+### 🛠 Tool Discovery — `know tools`
+
+Find runnable project tools so agents stop guessing commands.
+
+```bash
+know tools list
+know tools search "database"
+know tools suggest "deploy"
+know --output-format markdown tools list
+```
+
+Scans common sources: `scripts/`, `bin/`, `tools/`, `package.json` scripts, `Makefile` targets, Python CLIs, `docker-compose` services, and GitHub Actions workflows.
+
+### 🧭 Agent State — `know state`
+
+Track active task progress and create checkpoints you can restore later.
+
+```bash
+know state set --task "fix auth bug" --progress "analyzed middleware"
+know state get
+know state checkpoint --name "before refactor"
+know state restore cp_1739078400
+```
+
+### 🗂 Execution History — `know history`
+
+Log what you tried and whether it worked.
+
+```bash
+know history log --command "npm test" --result failed --reason "missing env var"
+know history search "test" --result failed
+know history success-rate "deploy"
+```
+
+### 💸 Cost Analytics — `know cost`
+
+Analyze token usage and estimated cost from local telemetry.
+If exact pricing/baseline data is unavailable, know reports fallback estimates and `unknown_fields` explicitly.
+
+```bash
+know cost summary
+know cost by-query --limit 5
+know cost savings
+know cost budget --limit 100000
+```
+
+### ⚡ Quick File Context — `know quick`
+
+Get a minimal context bundle for one file (default 500-token budget).
+
+```bash
+know quick src/auth.py
+know quick src/auth.py --budget 800 --json
+```
+
+### 🔗 Dependency Neighbors — `know related`
+
+See what a file imports and what imports it.
+
+```bash
+know related src/auth.py
+```
+
+### 📊 Module Summaries — `know summarize`
+
+One-sentence summary for each module in the codebase.
+
+```bash
+know summarize --limit 50
+```
+
 ### 📈 Usage Stats — `know stats`
 
 Track your ROI. See how much context you're serving and how efficiently.
@@ -267,9 +360,18 @@ know status --json
 | `know recall "query"` | Recall memories |
 | `know forget <id>` | Delete a memory |
 | `know graph <file>` | Show import dependencies |
+| `know quick <file>` | Minimal context for one file |
+| `know related <file>` | Show import/dependent neighbors for a file |
+| `know summarize` | One-line module summaries |
 | `know explain -c <name>` | AI-explain a component |
 | `know stats` | Usage statistics |
 | `know status` | Project health check |
+| `know tools list` | List discovered project tools |
+| `know tools search "query"` | Search for tools |
+| `know tools suggest "intent"` | Suggest relevant tools for a task |
+| `know state set/get/checkpoint/restore` | Manage agent task state |
+| `know history log/search/success-rate` | Track command outcomes |
+| `know cost summary/by-query/savings/budget` | Token/cost analytics + budget |
 | `know reindex` | Rebuild search index |
 | `know mcp serve` | Start MCP server |
 | `know mcp config` | Print MCP client config |
@@ -281,6 +383,7 @@ know status --json
 | Flag | Description |
 |------|-------------|
 | `--json` | Machine-readable JSON output |
+| `--output-format markdown` | Human-readable markdown output for supported commands |
 | `--quiet` | Minimal output |
 | `--verbose` | Detailed output |
 | `--time` | Show execution time |

{know_cli-0.3.0 → know_cli-0.3.1}/README.md

@@ -83,17 +83,39 @@ know recall "how does auth work?"
 
 Memories are automatically included in `know context` results. Your agent gets smarter over time.
 
-### 🔍 Semantic Search — `know search`
+### 🔍 Search — `know search` (Hybrid like qmd)
 
-Real embeddings, not grep. Understands meaning, not just keywords.
+Hybrid retrieval by default:
+- **Lexical**: SQLite **FTS5** (BM25 ranking) over chunk text + metadata
+- **Vector**: embeddings via **fastembed** (when installed)
+- **Deterministic rerank**: **Reciprocal Rank Fusion (RRF)**
 
 ```bash
+# Default: hybrid (FTS + vectors when available)
 know search "error handling"
+
+# JSON output (stable schema)
+know --json search "database connection pooling" --top-k 10
+# (also supported after the command)
 know search "database connection pooling" --top-k 10 --json
-know search "authentication" --chunk  # Search at function level
+
+Notes:
+- Queries with punctuation (e.g. `click.option`) are normalized for lexical search.
+- Results downweight docs/tests so source code tends to rank higher.
+# Force engine
+know search "authentication" --engine lexical
+know search "authentication" --engine vector
+
+# Filters
+know search "jwt" --path src/auth/ --type py
+
+# Better snippets
+know search "retry" --line-numbers
 ```
 
-Uses [fastembed](https://github.com/qdrant/fastembed) (BAAI/bge-small-en-v1.5) — runs locally, no API calls.
+Notes:
+- The FTS index is stored in: `.know/cache/index.db` (table: `chunks_fts`).
+- If **FTS5** and/or **fastembed** are unavailable, `know search` gracefully falls back.
 
 ### 📊 Import Graph — `know graph`
 
@@ -129,6 +151,77 @@ know mcp config                   # Print Claude Desktop config
 
 **MCP Resources:** `codebase://digest`, `codebase://structure`, `codebase://memories`
 
+### 🛠 Tool Discovery — `know tools`
+
+Find runnable project tools so agents stop guessing commands.
+
+```bash
+know tools list
+know tools search "database"
+know tools suggest "deploy"
+know --output-format markdown tools list
+```
+
+Scans common sources: `scripts/`, `bin/`, `tools/`, `package.json` scripts, `Makefile` targets, Python CLIs, `docker-compose` services, and GitHub Actions workflows.
+
+### 🧭 Agent State — `know state`
+
+Track active task progress and create checkpoints you can restore later.
+
+```bash
+know state set --task "fix auth bug" --progress "analyzed middleware"
+know state get
+know state checkpoint --name "before refactor"
+know state restore cp_1739078400
+```
+
+### 🗂 Execution History — `know history`
+
+Log what you tried and whether it worked.
+
+```bash
+know history log --command "npm test" --result failed --reason "missing env var"
+know history search "test" --result failed
+know history success-rate "deploy"
+```
+
+### 💸 Cost Analytics — `know cost`
+
+Analyze token usage and estimated cost from local telemetry.
+If exact pricing/baseline data is unavailable, know reports fallback estimates and `unknown_fields` explicitly.
+
+```bash
+know cost summary
+know cost by-query --limit 5
+know cost savings
+know cost budget --limit 100000
+```
+
+### ⚡ Quick File Context — `know quick`
+
+Get a minimal context bundle for one file (default 500-token budget).
+
+```bash
+know quick src/auth.py
+know quick src/auth.py --budget 800 --json
+```
+
+### 🔗 Dependency Neighbors — `know related`
+
+See what a file imports and what imports it.
+
+```bash
+know related src/auth.py
+```
+
+### 📊 Module Summaries — `know summarize`
+
+One-sentence summary for each module in the codebase.
+
+```bash
+know summarize --limit 50
+```
+
 ### 📈 Usage Stats — `know stats`
 
 Track your ROI. See how much context you're serving and how efficiently.
@@ -223,9 +316,18 @@ know status --json
 | `know recall "query"` | Recall memories |
 | `know forget <id>` | Delete a memory |
 | `know graph <file>` | Show import dependencies |
+| `know quick <file>` | Minimal context for one file |
+| `know related <file>` | Show import/dependent neighbors for a file |
+| `know summarize` | One-line module summaries |
 | `know explain -c <name>` | AI-explain a component |
 | `know stats` | Usage statistics |
 | `know status` | Project health check |
+| `know tools list` | List discovered project tools |
+| `know tools search "query"` | Search for tools |
+| `know tools suggest "intent"` | Suggest relevant tools for a task |
+| `know state set/get/checkpoint/restore` | Manage agent task state |
+| `know history log/search/success-rate` | Track command outcomes |
+| `know cost summary/by-query/savings/budget` | Token/cost analytics + budget |
 | `know reindex` | Rebuild search index |
 | `know mcp serve` | Start MCP server |
 | `know mcp config` | Print MCP client config |
@@ -237,6 +339,7 @@ know status --json
 | Flag | Description |
 |------|-------------|
 | `--json` | Machine-readable JSON output |
+| `--output-format markdown` | Human-readable markdown output for supported commands |
 | `--quiet` | Minimal output |
 | `--verbose` | Detailed output |
 | `--time` | Show execution time |

know_cli-0.3.1/docs/IMPLEMENTATION_PLAN.md

@@ -0,0 +1,305 @@
+# know-cli Implementation Plan
+**Based on:** STRATEGIC_AUDIT.md  
+**Start Date:** Feb 8, 2026  
+**Timeline:** 16 weeks to first revenue
+
+---
+
+## Sprint 1: Critical Bug Fixes (Week 1-2)
+
+### 🔴 P0 - Must Fix Before Any Marketing
+
+| Task | Effort | Status | Owner | Notes |
+|------|--------|--------|-------|-------|
+| Fix cross-project contamination | 4h | 🟡 In Progress | Vic | Verify EmbeddingCache isolation |
+| Implement hybrid search (BM25+vector) | 8h | ⬜ Not Started | Vic | Like QMD does |
+| Deduplicate search results | 2h | ⬜ Not Started | Vic | Filter by file path |
+| Better chunk previews | 2h | ⬜ Not Started | Vic | Show actual function body |
+| Add performance benchmarks | 2h | ⬜ Not Started | Vic | README stats |
+| Integration tests (MCP) | 6h | ⬜ Not Started | Vic | Automated |
+
+**Total: 24h**  
+**Target: Feb 15, 2026**
+
+**Success Criteria:**
+- ✅ Zero cross-project results
+- ✅ Zero duplicates
+- ✅ 95%+ precision@5
+- ✅ <500ms search latency
+
+---
+
+## Sprint 2: Power Features (Week 3-4)
+
+### 🟡 P1 - Quality Improvements
+
+| Task | Effort | Status | Notes |
+|------|--------|--------|-------|
+| Code graph traversal ("find callers") | 12h | ⬜ Not Started | High impact |
+| Query expansion (synonyms) | 6h | ⬜ Not Started | Better recall |
+| Reranking with cross-encoder | 8h | ⬜ Not Started | Better precision |
+| TypeScript/JavaScript AST support | 16h | ⬜ Not Started | Broader adoption |
+| Caching improvements | 6h | ⬜ Not Started | Faster UX |
+| Documentation overhaul | 4h | ⬜ Not Started | Clear value prop |
+
+**Total: 52h**  
+**Target: Feb 28, 2026**
+
+**Success Criteria:**
+- ✅ Works on TS/JS codebases
+- ✅ <200ms cached search
+- ✅ Graph queries supported
+
+---
+
+## Sprint 3: Distribution (Week 5-8)
+
+### 🟢 Marketing & Growth
+
+| Task | Effort | Status | Notes |
+|------|--------|--------|-------|
+| **Content Creation** | | | |
+| - Show HN post draft | 2h | ⬜ Not Started | Get feedback first |
+| - Demo video (Twitter, 30s) | 2h | ⬜ Not Started | Screen record |
+| - Tutorial videos (5x 5min) | 16h | ⬜ Not Started | YouTube |
+| - Dev.to articles (3x) | 12h | ⬜ Not Started | SEO keywords |
+| **Distribution** | | | |
+| - SEO-optimized docs site | 12h | ⬜ Not Started | Docusaurus/Nextra |
+| - ProductHunt launch prep | 4h | ⬜ Not Started | Images, copy |
+| - Twitter demo clips (10x) | 8h | ⬜ Not Started | Viral format |
+| **Community** | | | |
+| - Discord server setup | 2h | ⬜ Not Started | Invite early users |
+| - GitHub Discussions enable | 1h | ⬜ Not Started | Q&A |
+| - Email list (ConvertKit) | 2h | ⬜ Not Started | Collect signups |
+
+**Total: 61h**  
+**Target: Mar 31, 2026**
+
+**Success Criteria:**
+- ✅ 1K GitHub stars
+- ✅ 500 MAU
+- ✅ Show HN front page
+- ✅ 1K newsletter subscribers
+
+---
+
+## Sprint 4: Monetization (Week 9-12)
+
+### 💰 Launch Pro Tier
+
+| Task | Effort | Status | Notes |
+|------|--------|--------|-------|
+| **Backend Infrastructure** | | | |
+| - Cloud sync (Supabase) | 16h | ⬜ Not Started | Postgres + auth |
+| - Stripe integration | 8h | ⬜ Not Started | Subscriptions |
+| - User dashboard (Next.js) | 12h | ⬜ Not Started | Manage account |
+| - Billing + webhooks | 8h | ⬜ Not Started | Handle events |
+| **Product** | | | |
+| - Pro features (cloud sync, reranking) | 12h | ⬜ Not Started | Value differentiation |
+| - Pricing page | 4h | ⬜ Not Started | Clear tiers |
+| - Onboarding emails (5-part) | 6h | ⬜ Not Started | Activate users |
+| **Launch** | | | |
+| - Beta user recruitment | 4h | ⬜ Not Started | 20 early adopters |
+| - Pricing validation interviews | 6h | ⬜ Not Started | Willingness to pay |
+| - Launch announcement | 2h | ⬜ Not Started | Twitter, email |
+
+**Total: 78h**  
+**Target: Apr 30, 2026**
+
+**Success Criteria:**
+- ✅ $1K MRR
+- ✅ 50 paying users
+- ✅ <5% churn
+- ✅ 10% free→paid conversion
+
+---
+
+## Sprint 5-8: Scale (Week 13-16+)
+
+### 🚀 Growth & Optimization
+
+**Week 13-14: Team Tier**
+- Shared knowledge base
+- Team analytics
+- SSO/SAML basics
+- Admin dashboard
+
+**Week 15-16: Enterprise Pilot**
+- Self-hosted docs
+- Security audit
+- First enterprise customer
+- Custom contract
+
+**Ongoing:**
+- Content marketing (weekly)
+- Community management (daily)
+- Customer support (daily)
+- Feature iteration (bi-weekly)
+- Hiring prep (CTO/eng)
+
+**Success Criteria (Month 6):**
+- ✅ $5K-10K MRR
+- ✅ 200 paid users
+- ✅ 2-3 team accounts
+- ✅ 1 enterprise pilot
+
+---
+
+## Key Milestones
+
+| Date | Milestone | Metric |
+|------|-----------|--------|
+| **Feb 15** | v0.4.0 shipped | Zero critical bugs |
+| **Feb 28** | Multi-language support | Works on TS/JS |
+| **Mar 15** | Show HN launch | Front page |
+| **Mar 31** | 1K stars | GitHub |
+| **Apr 15** | Pro tier beta | 10 users |
+| **Apr 30** | Public launch | $1K MRR |
+| **May 31** | Team tier | $3K MRR |
+| **Jun 30** | Enterprise pilot | $5K MRR |
+
+---
+
+## Resource Allocation
+
+### Time Budget (Per Week)
+
+**Sushil (40h/week):**
+- 50% Engineering (20h) - Features, bugs
+- 25% Marketing (10h) - Content, community
+- 15% Product (6h) - Roadmap, interviews
+- 10% Admin (4h) - Ops, support
+
+**Vic (100h/week):**
+- 70% Engineering (70h) - Implementation
+- 20% QA (20h) - Testing, debugging
+- 10% Docs (10h) - Guides, examples
+
+### Monthly Burn Rate
+
+**Current: $0/mo** (bootstrap)
+
+**With revenue ($1K MRR):**
+- $0 - Infrastructure (Supabase free tier)
+- $0 - Tools (open source)
+- $0 - Marketing (organic only)
+- **Net: $1K profit → reinvest**
+
+**With scale ($5K MRR):**
+- $100 - Infrastructure (Supabase Pro)
+- $200 - Tools (Stripe, SendGrid, etc.)
+- $500 - Marketing (ads, sponsorships)
+- **Net: $4.2K profit → hire 1st employee**
+
+---
+
+## Decision Points
+
+### After Sprint 1 (Week 2)
+
+**Question:** Is search quality world-class?
+
+- ✅ YES → Proceed to Sprint 2 (distribution)
+- ❌ NO → Iterate 2 more weeks
+
+### After Sprint 3 (Week 8)
+
+**Question:** Do we have traction (1K stars, 500 MAU)?
+
+- ✅ YES → Build monetization (Sprint 4)
+- ❌ NO → Double down on content, delay Pro launch
+
+### After Sprint 4 (Week 12)
+
+**Question:** Are people paying ($1K MRR)?
+
+- ✅ YES → Scale (hire, enterprise)
+- ❌ NO → Reassess pricing/value prop, pivot if needed
+
+---
+
+## Risk Mitigation
+
+### If Search Quality Doesn't Improve
+
+**Pivot:** Position as "LLM context formatter" not "search tool"  
+**Focus:** Token budgeting, MCP integration, memory  
+**Market:** Still valuable even without best-in-class search
+
+### If No Traction (Low Stars/MAU)
+
+**Diagnosis:** Wrong audience? Wrong messaging? Wrong timing?  
+**Action:** 20 user interviews → find PMF  
+**Pivot:** Maybe B2B (teams) first, not B2C (individuals)
+
+### If Free→Paid Conversion Fails
+
+**Diagnosis:** Value prop unclear? Pricing wrong? Features weak?  
+**Action:** A/B test pricing ($15 vs $25), add killer Pro feature  
+**Pivot:** Open source everything, monetize via consulting/support
+
+---
+
+## Success Tracking
+
+### Weekly Review (Every Friday)
+
+**Metrics to track:**
+- GitHub stars (week-over-week growth)
+- PyPI downloads
+- MAU (active users)
+- Searches/day
+- Conversion funnel (signups → actives → paid)
+
+**Questions to answer:**
+- What worked this week?
+- What didn't work?
+- What are users asking for?
+- Any red flags (bugs, churn)?
+
+### Monthly Review (1st of month)
+
+**Deep dive:**
+- MRR growth
+- Churn analysis
+- Feature usage (what's popular?)
+- Competitor moves
+- Roadmap adjustments
+
+**Board update (if funded):**
+- Financials (revenue, burn)
+- Product (shipped features)
+- Metrics (users, engagement)
+- Risks + mitigation
+
+---
+
+## Next Actions (This Week)
+
+### Monday Feb 8
+- [x] Strategic audit complete
+- [x] Implementation plan written
+- [ ] Fix cross-project bug (4h)
+- [ ] Start hybrid search (4h)
+
+### Tuesday Feb 9
+- [ ] Finish hybrid search (4h)
+- [ ] Deduplicate results (2h)
+- [ ] Better previews (2h)
+
+### Wednesday Feb 10
+- [ ] Add benchmarks (2h)
+- [ ] Integration tests (6h)
+
+### Thursday Feb 11
+- [ ] Ship v0.4.0
+- [ ] Write Show HN draft
+- [ ] Create demo video
+
+### Friday Feb 12
+- [ ] Weekly review
+- [ ] Plan Sprint 2
+
+---
+
+**Let's ship. 🦉**

know_cli-0.3.1/docs/IMPROVEMENTS.md

@@ -0,0 +1,72 @@
+# know-cli Search Improvements
+
+## Issues Found (Feb 8, 2026)
+
+### 1. Duplicate Results
+**Problem:** Same file appears multiple times in search results  
+**Root cause:** TBD - need to investigate semantic_search.py  
+**Fix:** Deduplicate by file path before returning results
+
+### 2. Cross-Project Contamination
+**Problem:** Searching in know-cli returns allconvert files  
+**Root cause:** Project scoping exists but may not be working correctly  
+**Fix:** Verify project_root isolation in EmbeddingCache
+
+### 3. Missing --chunk Flag in CLI
+**Problem:** Backend supports function-level search but CLI doesn't expose it  
+**Root cause:** cli.py search command missing the --chunk option  
+**Fix:** Add `--chunk` flag to `@cli.command() search`
+
+### 4. Poor Preview Quality
+**Problem:** Previews show first 200 chars of file, not the matching chunk  
+**Root cause:** search_code() truncates file start instead of matching section  
+**Fix:** For chunk search, show the actual function/class body
+
+### 5. No Hybrid Search
+**Problem:** Only uses vector embeddings, no keyword matching  
+**Missing:** BM25 + vector combination like QMD does  
+**Fix:** Implement hybrid_search() combining both approaches
+
+---
+
+## Phase 1: Quick Wins (Today)
+
+- [x] Document issues
+- [ ] Fix #1: Deduplicate results
+- [ ] Fix #2: Verify project scoping
+- [ ] Fix #3: Expose --chunk in CLI
+- [ ] Fix #4: Better previews for chunk search
+- [ ] Test all fixes on real codebases
+
+## Phase 2: Power Features (Next)
+
+- [ ] Hybrid search (BM25 + vector)
+- [ ] Reranking with cross-encoder
+- [ ] Query expansion (synonyms, related terms)
+- [ ] Code-aware filters (complexity, recency, test vs prod)
+- [ ] Caching improvements (faster repeat searches)
+
+## Phase 3: Advanced (Later)
+
+- [ ] Multi-modal search (code + docs + comments)
+- [ ] Interactive refinement
+- [ ] Code graph traversal ("find all callers of X")
+- [ ] Diff-aware search ("what changed related to auth?")
+- [ ] Natural language understanding ("how to validate JWT tokens")
+
+---
+
+## Testing Strategy
+
+1. **Use on know-cli itself** - dogfood our own tool
+2. **Test on video-pipeline** - real production codebase
+3. **Test on remotion code** - TypeScript/React
+4. **Benchmark** - compare speed before/after
+
+## Success Metrics
+
+- Search returns 0 duplicates
+- Cross-project results = 0
+- Function-level search available via CLI
+- Previews show relevant code (not file start)
+- Search time < 500ms for 100+ files