@hasna/search 0.0.7 → 0.0.9

package/README.md

@@ -1,6 +1,6 @@
 # @hasna/search
 
-Unified search aggregator — 12 providers (Google, SerpAPI, Exa, Perplexity, Twitter, Reddit, YouTube, Brave, Bing, Hacker News, GitHub, arXiv) + YouTube transcription. CLI + MCP + REST API + Dashboard.
+Unified search for machines and agents — a **local file index** (find files by name, path, or content in milliseconds) plus **12 web providers** (Google, SerpAPI, Exa, Perplexity, Twitter, Reddit, YouTube, Brave, Bing, Hacker News, GitHub, arXiv) and YouTube transcription. CLI + MCP + REST API + Dashboard.
 
 [![npm](https://img.shields.io/npm/v/@hasna/search)](https://www.npmjs.com/package/@hasna/search)
 [![License](https://img.shields.io/badge/license-Apache--2.0-blue)](LICENSE)
@@ -11,6 +11,60 @@ Unified search aggregator — 12 providers (Google, SerpAPI, Exa, Perplexity, Tw
 npm install -g @hasna/search
 ```
 
+## Local Search
+
+Index a workspace once, then find anything in one call:
+
+```bash
+search index add ~/workspace          # register + index (gitignore-aware, incremental)
+search find storage-config            # by file name/path
+search find "deduplicate results"     # by content, with line numbers
+search find dedup -k file -e ts       # filters: kind, extension, dir, root
+search index status                   # roots, file counts, staleness
+search index update                   # incremental reindex (auto-runs when stale)
+```
+
+The index lives in SQLite (`~/.hasna/search/index.db`) using trigram FTS5 for substring
+matching. Re-indexing only touches changed files; stale roots refresh automatically
+before serving results. `node_modules`, `.git`, build output, binaries, and anything
+in `.gitignore` are excluded.
+
+Local results also join unified search as the `files` and `content` providers
+(`search query dedup --profile local`, or blended with web providers via `--profile all`).
+
+Regex (grep-style) search works too, Cursor-style — required literals from the
+pattern prefilter candidates through the trigram index, then the real regex runs
+only on those files:
+
+```bash
+search find "export (function|const) handle\w+" -x          # regex, line-based
+search find "storage-(config|sync)\.ts$" -x -k file         # regex over paths
+```
+
+### Performance
+
+Measured on a 158,140-file workspace index (79GB tree, 1.8GB index), warm cache,
+20-core Linux box — `search find -k content` vs `ripgrep -l`, wall clock per query:
+
+| Query | ripgrep | search find |
+|---|---|---|
+| `contentless_delete` | 0.65s | 0.06s |
+| `registerStorageCommands` | 0.20s | 0.07s |
+| `IgnoreMatcher` | 0.20s | 0.06s |
+
+CPU per query: rg ~2.2s across cores; find ~0.08s on one. Initial indexing of the
+same tree: 166s; incremental re-index when nothing changed: ~2.7s (runs
+automatically at most once per `indexStaleMinutes`). Results come back ranked,
+deduplicated across file-name and content matches, and scoped to all indexed
+roots regardless of the caller's working directory.
+
+## Web Search
+
+```bash
+search query "bun sqlite fts5" --profile research
+search exa "semantic search"          # any provider directly
+```
+
 ## CLI Usage
 
 ```bash
@@ -23,7 +77,9 @@ search --help
 search-mcp
 ```
 
-31 tools available.
+Agent-facing tools include `find` (one-call local file lookup), `index_add` /
+`index_update` / `index_status` / `index_remove`, unified `search`, per-provider
+`search_*` tools, history, saved searches, profiles, and export.
 
 ## HTTP mode
 
@@ -42,16 +98,21 @@ MCP_HTTP=1 search-mcp
 search-serve
 ```
 
-## Cloud Sync
+`/api/find`, `/api/index`, `/api/search`, and the dashboard (with a Local tab) on port 19800.
 
-This package supports cloud sync via `@hasna/cloud`:
+## Storage Sync
+
+Storage sync is optional. By default search uses local SQLite at `~/.hasna/search/`.
 
 ```bash
-cloud setup
-cloud sync push --service search
-cloud sync pull --service search
+search storage status
+search storage push
+search storage pull
+search storage sync
 ```
 
+Set `HASNA_SEARCH_DATABASE_URL` or `SEARCH_DATABASE_URL` to run in hybrid/remote mode with PostgreSQL. RDS host settings can be configured in `~/.hasna/search/storage/config.json`. Programmatic storage helpers are available from `@hasna/search/storage`.
+
 ## Data Directory
 
 Data is stored in `~/.hasna/search/`.