@liendev/lien 0.12.0 → 0.14.0

package/CURSOR_RULES_TEMPLATE.md

@@ -79,7 +79,7 @@ find_similar({
 - Consistency: ensure new code matches existing patterns
 - Duplication detection
 
-### `list_functions` ⚡ NEW in v0.5.0
+### `list_functions` ⚡
 **Fast symbol-based search for functions, classes, and interfaces by name.**
 
 ```typescript
@@ -104,7 +104,6 @@ list_functions({
 **Best practices:**
 - Use regex patterns that match naming conventions: `.*Controller.*`, `handle.*`, `get.*`
 - Combine with language filter for large codebases: `language: "typescript"`
-- For best results: run `lien reindex` after upgrading to v0.5.0
 
 **When to use `list_functions` vs `semantic_search`:**
 - ✅ Use `list_functions` when you know the naming pattern (e.g., "all Controllers")
@@ -114,6 +113,182 @@ list_functions({
 
 ---
 
+## Enhanced Metadata (AST-Based) ⚡ NEW in v0.13.0
+
+Lien now uses **Abstract Syntax Tree (AST) parsing** for TypeScript/JavaScript files to provide rich code metadata:
+
+### Metadata Fields in Search Results
+
+All search results (`semantic_search`, `get_file_context`, `find_similar`, `list_functions`) now include enhanced metadata when available:
+
+```typescript
+{
+  content: "function validateEmail(email: string): boolean { ... }",
+  metadata: {
+    file: "src/validators.ts",
+    startLine: 45,
+    endLine: 60,
+    type: "function",  // 'function' | 'class' | 'block'
+    language: "typescript",
+    
+    // AST-derived metadata (NEW in v0.13.0):
+    symbolName: "validateEmail",              // Function/class name
+    symbolType: "function",                   // 'function' | 'method' | 'class' | 'interface'
+    parentClass: undefined,                   // For methods: parent class name
+    complexity: 3,                            // Cyclomatic complexity
+    parameters: ["email: string"],            // Function parameters
+    signature: "function validateEmail(email: string): boolean",  // Full signature
+    imports: ["@/utils/regex"]               // File imports (for context)
+  },
+  score: 0.85,
+  relevance: "highly_relevant"
+}
+```
+
+### AST Metadata Benefits
+
+1. **Never splits functions** - Chunks respect semantic boundaries (no mid-function splits)
+2. **Function context** - Know exactly which function you're looking at
+3. **Complexity metrics** - Identify complex functions that may need refactoring
+4. **Signature awareness** - See parameters and return types at a glance
+5. **Better AI context** - AI assistants get structured code information
+
+### Using AST Metadata
+
+**Find complex functions:**
+```typescript
+// Search for authentication logic
+const results = await semantic_search({ query: "authentication logic" });
+
+// Filter by complexity
+const complexFunctions = results.filter(r => (r.metadata.complexity || 0) > 5);
+```
+
+**Identify methods in a class:**
+```typescript
+// Get file context
+const context = await get_file_context({ filepath: "src/auth/AuthService.ts" });
+
+// Find all methods
+const methods = context.results.filter(r => r.metadata.symbolType === 'method');
+```
+
+**List functions with specific parameters:**
+```typescript
+const functions = await list_functions({ pattern: ".*validate.*", language: "typescript" });
+
+// Filter by parameter count
+const simpleValidators = functions.filter(r => (r.metadata.parameters?.length || 0) <= 2);
+```
+
+### AST Support
+
+**Currently supported:**
+- ✅ TypeScript (`.ts`, `.tsx`)
+- ✅ JavaScript (`.js`, `.jsx`, `.mjs`, `.cjs`)
+- ✅ Shopify Liquid (`.liquid`) - **Special regex-based chunking**
+
+**Coming soon:**
+- 🔜 Python, Go, Rust, Java, PHP, and more
+
+**Fallback behavior:**
+- For unsupported languages, Lien automatically falls back to line-based chunking
+- No disruption to existing workflows
+
+### Shopify Liquid Support ⚡ NEW
+
+Lien provides specialized chunking for Shopify themes with **complete dependency tracking**:
+
+**Liquid template handling:**
+- `{% schema %}` blocks - Kept as single chunks, section names extracted
+- `{% style %}` blocks - Preserved together for scoped CSS
+- `{% javascript %}` blocks - Kept intact
+- Oversized blocks (>225 lines) - Intelligently split to prevent token limits
+
+**JSON template handling (Shopify 2.0+):**
+- `templates/**/*.json` - Template definition files
+- Section references extracted from JSON structure
+- Template names extracted from filepath
+
+**Complete dependency tracking (tracked in `metadata.imports`):**
+- `{% render 'snippet-name' %}` - Snippet dependencies
+- `{% include 'snippet-name' %}` - Legacy includes
+- `{% section 'section-name' %}` - Section usage in layouts
+- JSON template sections - Section type references
+
+**Example metadata:**
+```typescript
+// JSON Template
+{
+  content: "{\"sections\": {\"main\": {\"type\": \"main-product\"}}}",
+  metadata: {
+    file: "templates/product.json",
+    type: "template",
+    language: "json",
+    symbolName: "product",              // Template name
+    symbolType: "template",
+    imports: ["main-product"]           // Sections used by this template
+  }
+}
+
+// Liquid Section Schema
+{
+  content: "{% schema %}\n{\"name\": \"Hero Section\", ...}\n{% endschema %}",
+  metadata: {
+    file: "sections/hero.liquid",
+    type: "block",
+    language: "liquid",
+    symbolName: "Hero Section",        // Extracted from schema JSON
+    symbolType: "schema",
+    imports: undefined                  // No render/include/section tags found in this block
+  }
+}
+
+// Liquid Template Content
+{
+  content: "<div>{% render 'logo' %}{% render 'nav' %}</div>",
+  metadata: {
+    file: "sections/header.liquid",
+    type: "template",
+    language: "liquid",
+    imports: ["logo", "nav"]            // Tracked dependencies
+  }
+}
+```
+
+**Benefits:**
+- **Complete dependency graph** - JSON templates → sections → snippets
+- **Schema preservation** - Never splits section configuration across chunks
+- **Better context** - AI knows full theme structure and all dependencies
+
+### Known Limitations
+
+**Very large files (1000+ lines):**
+- Tree-sitter may fail with "Invalid argument" error on extremely large files
+- When this occurs, Lien automatically falls back to line-based chunking
+- This is a known Tree-sitter limitation with very large syntax trees
+- Fallback behavior is configurable via `astFallback` setting
+
+**Resilient parsing:**
+- Tree-sitter is designed to produce best-effort ASTs even for invalid syntax
+- Parse errors are rare; most malformed code still produces usable chunks
+- The `astFallback: 'error'` option mainly catches edge cases like large file errors
+
+### Configuration
+
+Control AST behavior in `.lien.config.json`:
+
+```json
+{
+  "chunking": {
+    "useAST": true,              // Enable AST-based chunking (default: true)
+    "astFallback": "line-based"  // Fallback strategy: 'line-based' | 'error'
+  }
+}
+```
+
+---
+
 ## Input Validation & Error Handling
 
 Lien uses Zod schemas for runtime type-safe validation of all tool inputs. This provides:
@@ -168,7 +343,7 @@ Lien uses structured error codes for programmatic error handling:
 - `INVALID_INPUT` - Parameter validation failed
 - `FILE_NOT_FOUND` - Requested file doesn't exist in index
 - `INDEX_NOT_FOUND` - No index found (run `lien index`)
-- `INDEX_CORRUPTED` - Index is corrupted (run `lien reindex`)
+- `INDEX_CORRUPTED` - Index is corrupted (run `lien index` to rebuild)
 - `EMBEDDING_GENERATION_FAILED` - Embedding model failed (retryable)
 - `INTERNAL_ERROR` - Unexpected internal error
 
@@ -232,7 +407,7 @@ Lien uses structured error codes for programmatic error handling:
 4. Analyze and suggest improvements
 ```
 
-### Pattern 7: Finding All Classes/Functions by Name Pattern ⚡ NEW
+### Pattern 7: Finding All Classes/Functions by Name Pattern ⚡
 ```
 1. list_functions({ pattern: ".*Controller.*", language: "php" })
 2. Review the list of matching classes
@@ -245,6 +420,39 @@ Lien uses structured error codes for programmatic error handling:
 - "What Services exist?" → `list_functions({ pattern: ".*Service.*" })`
 - "Find all API handlers" → `list_functions({ pattern: "handle.*" })`
 
+### Pattern 8: Working with Shopify Themes (Liquid + JSON) ⚡
+```
+1. semantic_search({ query: "product template configuration" })
+   → Finds JSON template with section references
+2. Check metadata.imports to see which sections are used
+3. semantic_search({ query: "main-product section schema" })
+   → Find section definition
+4. Review section's metadata.imports to see which snippets it renders
+   → Complete dependency chain visible!
+```
+
+**Example queries:**
+- "Find the product template sections" → Returns `templates/product.json` with section imports
+- "Which sections are on the collection page?" → Check JSON template imports
+- "What sections use the product-card snippet?" → Reverse lookup via imports
+- "Show the hero section schema" → Returns complete `{% schema %}` block with name
+- "What snippets does the footer render?" → See `metadata.imports: ["logo", "nav", ...]`
+
+**Complete dependency graph:**
+```
+templates/product.json
+  → imports: ["main-product", "recommendations"]
+    → sections/main-product.liquid
+      → imports: ["product-card", "price-tag"]
+        → snippets/product-card.liquid
+        → snippets/price-tag.liquid
+```
+
+**Dependency tracking:**
+- **JSON templates** - `metadata.imports` contains section type references
+- **Liquid templates** - `metadata.imports` contains `{% render %}`, `{% include %}`, `{% section %}` references
+- Full theme architecture visible through imports metadata
+
 ---
 
 ## Decision Tree: Lien vs Other Tools
@@ -256,7 +464,7 @@ Lien uses structured error codes for programmatic error handling:
 ✅ Exploring unfamiliar parts of codebase
 ✅ Searching by what code **does** (behavior, functionality)
 
-### Use `list_functions` when: ⚡ NEW
+### Use `list_functions` when: ⚡
 ✅ User asks "show me all Controllers" or similar structural queries
 ✅ Looking for classes/functions matching a **naming pattern**
 ✅ Getting architectural overview (all Services, all Handlers, etc.)
@@ -297,6 +505,9 @@ Lien uses structured error codes for programmatic error handling:
 - "React components with form state"
 - "database migration scripts"
 - "API endpoints for user data"
+- "Shopify section schema for hero banner" (Liquid)
+- "files that render product-card snippet" (Liquid)
+- "layout file with header and footer sections" (Liquid)
 
 ### Bad Queries (DON'T DO THIS):
 - "auth" (too vague)
@@ -317,9 +528,10 @@ Lien uses structured error codes for programmatic error handling:
 - First query loads embeddings (~1-2s), subsequent queries are fast (<500ms)
 - Increase `limit` to 10-15 for broad exploration
 - Results are ranked by semantic relevance (trust the ranking)
-- User can re-index with `lien reindex` if results seem stale
+- User can re-index with `lien index` if results seem stale
 - **Relevance categories**: All search results include a `relevance` field (`highly_relevant`, `relevant`, `loosely_related`, `not_relevant`) to help interpret search quality at a glance
 - **Test associations**: Lien automatically detects test-source relationships across 12 languages using convention-based patterns and import analysis
+- **Shopify Liquid themes**: Semantic chunking reduces chunk count by ~60% (schema/style/javascript blocks preserved), improving search quality and performance
 
 ---
 
@@ -341,10 +553,12 @@ Create a `lien.mdc` file in your `.cursor/rules/` directory:
 ```bash
 # From your project directory
 mkdir -p .cursor/rules
-cp /path/to/lien/CURSOR_RULES_TEMPLATE.md .cursor/rules/lien.mdc
+cp node_modules/@liendev/lien/CURSOR_RULES_TEMPLATE.md .cursor/rules/lien.mdc
 ```
 
 The `alwaysApply: true` frontmatter ensures Cursor uses Lien for all files in your project.
 
 This approach allows you to have multiple rule files in `.cursor/rules/` without conflicts.
 
+**Note:** The template is automatically copied during `lien init` to `.cursor/rules/lien.mdc`.
+

package/README.md

@@ -72,7 +72,15 @@ Contributions welcome! See **[CONTRIBUTING.md](./CONTRIBUTING.md)** for guidelin
 
 ## License
 
-MIT © [Alf Henderson](https://github.com/alfhen)
+AGPL-3.0 © [Alf Henderson](https://github.com/alfhen)
+
+**Lien is free forever for local use.** The AGPL license ensures that:
+- ✅ You can use Lien locally without restrictions
+- ✅ You can modify and distribute Lien freely
+- ✅ Improvements get contributed back to the community
+- ✅ We can sustain long-term development
+
+For questions about licensing, contact us at alf@lien.dev
 
 ---