sf-compact-cli 0.3.3 → 0.4.1

package/README.md

@@ -1,312 +1,77 @@
 # sf-compact
 
-Convert Salesforce metadata XML to AI-friendly compact formats. Semantically lossless roundtrip for Salesforce metadata.
+Cut Salesforce metadata tokens in half for AI coding agents.
 
-Salesforce metadata XML is extremely verbose — profiles, permission sets, flows, and objects can be 20,000–50,000+ lines of XML with 70–85% structural overhead. This burns tokens and money when AI tools (Claude Code, Codex, Cursor, etc.) read or edit your metadata.
+Salesforce metadata XML is extremely verbose — 70-85% structural overhead. **sf-compact** converts it to compact YAML/JSON. In controlled benchmarks, this reduced Claude Code costs by **11.5%** — and by **33%** with a custom exploration agent.
 
-**sf-compact** converts it to compact YAML or JSON, saving 42–54% of tokens depending on format.
+## Quick Start
 
-## Output Formats
-
-| Format | Preserves order | Human-readable | Token savings |
-|--------|:-:|:-:|:-:|
-| `yaml` | No | Yes | ~49% |
-| `yaml-ordered` | Yes | Yes | ~42% |
-| `json` | Yes | Less | ~54% |
-
-- **yaml** — groups repeated elements into arrays. Most compact YAML, but sibling order may change. Best for order-insensitive types (Profile, PermissionSet).
-- **yaml-ordered** — uses `_children` sequences to preserve exact element order. Best for order-sensitive types (Flow, FlexiPage, Layout).
-- **json** — compact single-line JSON with arrays. Preserves order, fewest tokens, less human-readable.
-
-## Before / After
-
-**XML (848 tokens):**
-```xml
-<?xml version="1.0" encoding="UTF-8"?>
-<Profile xmlns="http://soap.sforce.com/2006/04/metadata">
-    <custom>false</custom>
-    <userLicense>Salesforce</userLicense>
-    <fieldPermissions>
-        <editable>true</editable>
-        <field>Account.AnnualRevenue</field>
-        <readable>true</readable>
-    </fieldPermissions>
-    <fieldPermissions>
-        <editable>false</editable>
-        <field>Account.BillingCity</field>
-        <readable>true</readable>
-    </fieldPermissions>
-    ...
-</Profile>
-```
-
-**YAML (432 tokens — 49% reduction):**
-```yaml
-_tag: Profile
-_ns: http://soap.sforce.com/2006/04/metadata
-custom: false
-userLicense: Salesforce
-fieldPermissions:
-- editable: true
-  field: Account.AnnualRevenue
-  readable: true
-- editable: false
-  field: Account.BillingCity
-  readable: true
-...
-```
-
-**JSON (389 tokens — 54% reduction):**
-```json
-{"_tag":"Profile","_ns":"http://soap.sforce.com/2006/04/metadata","custom":"false","userLicense":"Salesforce","fieldPermissions":[{"editable":"true","field":"Account.AnnualRevenue","readable":"true"},{"editable":"false","field":"Account.BillingCity","readable":"true"}]}
-```
-
-## Install
-
-### From source (Rust required)
 ```bash
-cargo install --path .
-```
-
-### From crates.io
-```bash
-cargo install sf-compact
-```
-
-## Usage
-
-### Pack (XML → compact format)
-```bash
-sf-compact pack [source...] [-o output] [--format yaml|yaml-ordered|json] [--include pattern]
+npm install -g sf-compact-cli
+sf-compact pack
+sf-compact init instructions
 ```
 
-```bash
-# Pack entire project (default: YAML format)
-sf-compact pack force-app -o .sf-compact
+`init instructions` injects a directive into your CLAUDE.md telling the AI to read from `.sf-compact/` instead of `force-app/`. In 17 benchmark variants, this was the simplest approach that reliably worked.
 
-# Pack as JSON for maximum token savings
-sf-compact pack force-app --format json
+## Why YAML is the Default
 
-# Pack specific directories
-sf-compact pack force-app/main/default/profiles force-app/main/default/classes
+Token savings vary by metadata type. JSON saves 31-34% on large types (profiles, objects) but is *worse* for small types — fields (+0.4%), list views (+1.1%), compact layouts (+5.5%). YAML avoids the JSON `_children` overhead and works well across all sizes.
 
-# Pack only profiles
-sf-compact pack force-app --include "*.profile-meta.xml"
-```
+## Install
 
-### Unpack (compact format → XML)
+### npm (recommended)
 ```bash
-sf-compact unpack [source...] [-o output] [--include pattern]
+npm install -g sf-compact-cli
 ```
 
-Auto-detects format by file extension (`.yaml` or `.json`).
-
+### Homebrew (macOS / Linux)
 ```bash
-sf-compact unpack .sf-compact -o force-app
+brew install vradko/tap/sf-compact
 ```
 
-### Stats (preview savings)
+### From crates.io
 ```bash
-sf-compact stats [source...] [--include pattern] [--files]
+cargo install sf-compact
 ```
 
-Analyze metadata and preview token/byte savings without writing files.
+## Commands
 
 ```bash
-$ sf-compact stats force-app
-
-Preview: what sf-compact pack would produce
-Tokenizer: cl100k_base (GPT-4 / Claude)
-
-                                               XML (now)    YAML (after)     savings
-  --------------------------------------------------------------------------------
-                                     Bytes          7313          3418       53.3%
-                                    Tokens          1719           925       46.2%
-
-  Would save 794 tokens across 5 files
-
-  By metadata type:
-  type                 files         now →    after tokens     saved
-  ----------------------------------------------------------------------
-  profile                  1         848 →      432 tokens     49.1%
-  flow                     1         464 →      268 tokens     42.2%
-  field                    1         232 →      126 tokens     45.7%
-  js                       1         116 →       66 tokens     43.1%
-  cls                      1          59 →       33 tokens     44.1%
+sf-compact pack                                     # XML -> compact (force-app -> .sf-compact)
+sf-compact unpack .sf-compact -o force-app          # compact -> XML
+sf-compact watch                                    # auto-pack on changes
+sf-compact stats force-app                          # preview savings
+sf-compact diff                                     # detect unpacked changes
+sf-compact lint                                     # CI validation (exit 1 if stale)
+sf-compact changes --since-deploy                   # track modified files
 ```
 
-Use `--files` for per-file breakdown, `--include` to filter by glob pattern.
-
 ### Configuration
-
-sf-compact uses a `.sfcompact.yaml` config file for per-type format control.
-
-```bash
-# Create config with smart defaults (json default, yaml for order-insensitive types)
-sf-compact config init
-
-# Set format for specific types (batch — multiple types in one call)
-sf-compact config set flow json profile yaml flexipage yaml-ordered
-
-# Change default format for all types
-sf-compact config set default json
-
-# Skip a metadata type from conversion
-sf-compact config skip customMetadata
-
-# View current configuration
-sf-compact config show
-```
-
-Default config after `config init`:
-
-```yaml
-default_format: json
-formats:
-  Profile: yaml
-  PermissionSet: yaml
-  PermissionSetGroup: yaml
-  # ... other order-insensitive types get yaml for readability
-skip: []
-```
-
-When `pack` runs, it reads `.sfcompact.yaml` and applies the format per metadata type. The `--format` CLI flag overrides the config for a single run.
-
-### Watch (auto-pack on changes)
-```bash
-sf-compact watch [source...] [-o output] [--format yaml|yaml-ordered|json] [--include pattern]
-```
-
-Watches source directories for XML changes and automatically repacks. Runs an initial pack, then monitors for file changes.
-
-```bash
-# Watch default force-app directory
-sf-compact watch
-
-# Watch with JSON format
-sf-compact watch force-app --format json
-```
-
-### Diff (detect unpacked changes)
-```bash
-sf-compact diff [source...] [-o packed-dir] [--include pattern]
-```
-
-Compare current XML metadata against the last packed output. Shows new, modified, and deleted files.
-
-```bash
-$ sf-compact diff
-
-  + force-app/main/default/profiles/NewProfile.profile-meta.xml  (new — not yet packed)
-  ~ force-app/main/default/flows/Case_Assignment.flow-meta.xml  (modified since last pack)
-
-1 new, 1 modified, 0 deleted, 3 unchanged
-Run `sf-compact pack` to update.
-```
-
-### Lint (CI validation)
-```bash
-sf-compact lint [source...] [-o packed-dir] [--include pattern]
-```
-
-Check that compact files are up-to-date. Exits with code 1 if any files are stale, new, or orphaned. Use in CI pipelines.
-
-### Changes (track modified compact files)
 ```bash
-sf-compact changes [-o compact-dir]                    # show all modified files (global)
-sf-compact changes --since-deploy                      # show changes since last deploy reset
-sf-compact changes --json                              # machine-readable JSON output
-sf-compact changes reset --global                      # clear all tracking
-sf-compact changes reset --since-deploy                # clear deployment tracking only
+sf-compact config init                              # create .sfcompact.yaml with smart defaults
+sf-compact config set flow json profile yaml        # set format per type
+sf-compact config show                              # view current config
 ```
 
-Tracks which compact files were modified (by AI or human) since last `pack`. Per-branch tracking with two scopes: global (all changes) and deployment (delta since last deploy reset).
-
-### MCP Server
-
-sf-compact includes a built-in [MCP](https://modelcontextprotocol.io/) server for direct AI tool integration.
-
+### AI Tool Integration
 ```bash
-# Add to your project's .mcp.json
-sf-compact init mcp
-
-# Or start manually
-sf-compact mcp-serve
+sf-compact init instructions                        # CLAUDE.md / .cursorrules / etc. (recommended)
+sf-compact init agent                               # sf-explorer agent for Claude Code (-33% cost)
+sf-compact init hook                                # PreToolUse hook (optional)
+sf-compact init mcp                                 # MCP server
 ```
 
-This exposes `sf_compact_pack`, `sf_compact_unpack`, `sf_compact_stats`, `sf_compact_lint`, and `sf_compact_changes` as MCP tools that Claude Code, Cursor, and other MCP-compatible tools can discover and use automatically.
-
-### AI Instructions
-
-Inject a compact directive block into your AI tool's instruction file. Auto-detects which AI tools are configured in the project and writes to all of them.
-
-```bash
-# Auto-detect AI tools and inject into all found instruction files
-sf-compact init instructions
-
-# Inject only into a specific tool's file
-sf-compact init instructions --target claude      # CLAUDE.md
-sf-compact init instructions --target cursor      # .cursorrules
-sf-compact init instructions --target copilot     # .github/copilot-instructions.md
-sf-compact init instructions --target codex       # AGENTS.md
-
-# Remove sf-compact blocks from all AI instruction files
-sf-compact init instructions --remove
-
-# Legacy: create standalone reference file
-sf-compact init instructions --name SF_COMPACT.md
-```
-
-### Manifest
-
-Output supported metadata types in JSON (includes format support and order-sensitivity flags):
-
-```bash
-sf-compact manifest
-```
-
-## Supported Metadata Types
-
-76 file extensions mapping to Salesforce metadata types across 10 categories:
-
-| Category | Types |
-|----------|-------|
-| **Security** | Profile, PermissionSet, PermissionSetGroup, RemoteSiteSetting, CspTrustedSite, ConnectedApp, SharingRules, CustomPermission, Role, Group, AuthProvider, SamlSsoConfig, Certificate |
-| **Schema** | CustomObject, CustomField, ValidationRule, CustomMetadata, GlobalValueSet, StandardValueSet, RecordType, MatchingRule, DuplicateRule, CustomIndex, FieldSet |
-| **Code** | ApexClass, ApexTrigger, ApexComponent, ApexPage, LightningComponentBundle (js/css/html/xml), AuraDefinitionBundle (cmp/evt), StaticResource |
-| **Automation** | Flow*, Workflow, WorkflowRule, AssignmentRules, AutoResponseRules, EscalationRules |
-| **UI** | Layout*, CustomLabels, CustomApplication, CustomTab, FlexiPage*, CustomSite, QuickAction, PathAssistant, ListView, CompactLayout, WebLink, HomePageLayout, AppMenu, Community, Letterhead |
-| **Analytics** | ReportType, Report, Dashboard |
-| **Integration** | ExternalServiceRegistration, NamedCredential, ExternalCredential |
-| **Config** | Settings, InstalledPackage, TopicsForObjects, CustomNotificationType, CleanDataService, NotificationTypeConfig, PlatformEventChannelMember |
-| **Translation** | CustomObjectTranslation, CustomFieldTranslation |
-| **Content** | EmailTemplate, ManagedContentType, IframeWhiteListUrlSettings, LightningMessageChannel |
-
-\* Order-sensitive types — `config init` defaults these to `yaml-ordered` to preserve element order.
-
 ## Workflow
 
-1. **Configure** (once): `sf-compact config init` — creates `.sfcompact.yaml` with smart defaults
-2. **Pull metadata** from Salesforce (`sf project retrieve`)
-3. **Pack**: `sf-compact pack` — creates `.sf-compact/` with compact files
-4. **Work with compact files** — let AI tools read/edit the YAML/JSON format
-5. **Unpack**: `sf-compact unpack` — restores XML for deployment
-6. **Deploy** to Salesforce (`sf project deploy`)
-
-> Use `sf-compact watch` during development to auto-pack on changes, and `sf-compact diff` to check if a repack is needed.
-
-> Tip: Add `.sf-compact/` to `.gitignore` if you treat it as a build artifact, or commit it for AI-friendly diffs.
-
-## How it works
-
-- Parses Salesforce metadata XML into a tree structure
-- Groups repeated elements (e.g., `<fieldPermissions>`) into arrays (YAML) or `_children` sequences (yaml-ordered, JSON)
-- Coerces booleans: `"true"` → `true`, `"false"` → `false`. All other values (including numeric strings like `"59.0"`, `"0012"`) are preserved as-is
-- Flattens simple key-value containers into inline mappings
-- Preserves namespaces, attributes, and all structural information for semantically lossless roundtrip
-- Order-sensitive types (Flow, FlexiPage, Layout) default to `yaml-ordered` format, which preserves exact element order via `_children` sequences
+1. `sf-compact config init` (once)
+2. `sf project retrieve start`
+3. `sf-compact pack`
+4. `sf-compact init instructions` (once)
+5. AI reads compact files via CLAUDE.md directive
+6. `sf-compact unpack` -> `sf project deploy start`
 
-Token counting uses the `cl100k_base` tokenizer (same family used by GPT-4 and Claude).
+Full documentation: [github.com/vradko/sf-compact](https://github.com/vradko/sf-compact)
 
 ## License
 

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "sf-compact-cli",
-  "version": "0.3.3",
-  "description": "Convert Salesforce metadata XML to compact AI-friendly formats (YAML/JSON). Semantically lossless roundtrip.",
+  "version": "0.4.1",
+  "description": "Cut Salesforce metadata tokens in half for AI coding agents. Converts XML to compact YAML/JSON with AI instruction file integration.",
   "license": "MIT",
   "repository": {
     "type": "git",