resplite 1.2.6 → 1.2.10

package/skills/resplite-command-vertical-slice/SKILL.md

@@ -0,0 +1,134 @@
+---
+name: resplite-command-vertical-slice
+description: Implements or extends a Redis-like command in RESPLite from spec to docs and tests. Use when the user says "add a command", "support a Redis option", "fix command compatibility", "implement ZRANGE behavior", or "update the compatibility matrix". Do not use for migration-only or FT-only work unless the change also affects the general command surface.
+license: MIT
+metadata:
+  author: Cursor Agent
+  version: 1.0.0
+  category: workflow-automation
+  tags: [resplite, redis-compatibility, commands, tests]
+---
+
+# RESPLite Command Vertical Slice
+
+Use this skill when a task requires changing RESPLite's public command behavior end to end rather than only editing one isolated function.
+
+## Use cases
+
+**Use Case: Add a missing command**
+Trigger: "implement command X", "add Redis compatibility for X", "support command Y"
+Steps: inspect spec and current support, wire the handler, update engine or storage if needed, add tests, update docs
+Result: a verified command slice that works through RESP and is reflected in the public docs
+
+**Use Case: Extend a partially supported command**
+Trigger: "support another ZRANGE option", "fix TTL edge case", "match Redis error behavior"
+Steps: compare current implementation with intended semantics, patch the smallest layer that owns the behavior, strengthen tests, update compatibility notes
+Result: improved compatibility without widening scope unnecessarily
+
+**Use Case: Audit a command regression**
+Trigger: "this command broke", "redis-cli behaves differently", "wrongtype or ERR output is wrong"
+Steps: reproduce through tests, isolate whether the issue is parser, handler, engine, or storage, fix the owning layer, verify client-facing behavior
+Result: the bug is fixed with regression coverage
+
+## Instructions
+
+### Step 1: Lock the compatibility target
+
+Start by defining the exact command surface you are changing:
+
+- command name and sub-options
+- expected success reply shape
+- expected error strings
+- whether the change is Redis-standard behavior or RESPLite-specific behavior
+
+Favor the smallest compatibility increment that solves the user's request. RESPLite is intentionally a practical subset, not a full Redis clone.
+
+### Step 2: Review the canonical sources first
+
+Read these files before editing:
+
+- `README.md` for the public compatibility matrix and examples
+- `src/commands/registry.js` for command dispatch and handler registration
+
+Then inspect the command-specific implementation:
+
+- `src/commands/*.js` for argument parsing and command-level behavior
+- `src/engine/*.js` when semantics belong to the engine
+- `src/storage/sqlite/*.js` when persistence or query behavior changes
+- relevant tests in `test/unit`, `test/integration`, and `test/contract`
+
+### Step 3: Choose the owning layer
+
+Use this decision rule:
+
+- parser or command syntax issue: fix the command handler
+- business semantics issue: fix the engine layer
+- persistence or query issue: fix SQLite storage helpers
+- public support surface changed: also update `README.md` and, if needed, the spec
+
+Avoid scattering logic across layers when one layer can own it cleanly.
+
+### Step 4: Implement the smallest vertical slice
+
+Typical command workflow:
+
+1. Add or update the handler in `src/commands`
+2. Register it in `src/commands/registry.js` if needed
+3. Patch engine or storage behavior only where required
+4. Preserve existing error style such as `ERR ...` and wrong-type behavior
+5. Keep binary-safety expectations in mind when keys or values pass through buffers and SQLite
+
+If a new command expands publicly supported behavior, update the compatibility matrix in `README.md`.
+
+### Step 5: Prove it through the right test level
+
+Pick the smallest test mix that proves the change:
+
+- `test/unit/*.test.js` for pure logic, engine rules, parser behavior, or storage helpers
+- `test/integration/*.test.js` for end-to-end RESP behavior through the server
+- `test/contract/*.test.js` when compatibility with `redis` client or `redis-cli` is part of the user-facing promise
+
+Prefer adding a regression test that would fail before the fix and pass after it.
+
+### Step 6: Report the outcome clearly
+
+When you finish, summarize:
+
+- what compatibility surface changed
+- which layers were touched
+- which tests were run
+- any remaining intentional gaps versus Redis
+
+## RESPLite-specific checklist
+
+- Keep scope aligned with "practical Redis compatibility", not total parity
+- Preserve RESP2-facing behavior
+- Use the public docs to reflect newly supported behavior
+- Do not claim support in docs until tests back it up
+- If the command is intentionally unsupported, return the project's clear unsupported-command behavior instead of faking partial support
+
+## Examples
+
+**Example 1: Add a missing command**
+User says: "Implement a minimal `ZRANK` command."
+Actions: review `spec/SPEC_A.md`, inspect zset handlers and storage helpers, add `src/commands/zrank.js`, register it, add unit and integration coverage, update `README.md` only if support becomes public
+Result: one new command works end to end with verified semantics
+
+**Example 2: Fix a compatibility edge case**
+User says: "Make `TTL` return the right value for missing keys."
+Actions: inspect existing TTL tests, patch the owning logic, add a regression test, verify no unrelated TTL behavior changed
+Result: client-visible compatibility improves with minimal code churn
+
+## Troubleshooting
+
+**Problem: The handler exists but the command still returns unsupported**
+Cause: `src/commands/registry.js` was not updated, or the command name casing does not match dispatch behavior.
+Solution: register the handler in uppercase form and verify the RESP command path through an integration test.
+
+**Problem: The command works in unit tests but not through clients**
+Cause: only internal logic was tested; RESP parsing, argument shape, or connection handling may differ.
+Solution: add an integration test, and add a contract test if `redis-cli` or the official `redis` client is part of the promise.
+
+**Problem: The change feels larger than one command**
+Cause: the request may actually be a search feature or migration flow change rather than a generic command extension.
+Solution: switch to the more specific RESPLite skill for migration or `FT.*` work instead of forcing everything into one command slice.

package/skills/resplite-ft-search-workbench/SKILL.md

@@ -0,0 +1,138 @@
+---
+name: resplite-ft-search-workbench
+description: Builds or refines RESPLite `FT.*` behavior and RediSearch migration mapping on top of SQLite FTS5. Use when the user says "add FT command support", "fix FT.SEARCH", "adjust SQLite FTS5 behavior", "migrate RediSearch indices", or "work on FT.CREATE or FT.ADD semantics". Do not use for unrelated command work outside the search surface.
+license: MIT
+metadata:
+  author: Cursor Agent
+  version: 1.0.0
+  category: workflow-automation
+  tags: [resplite, search, redisearch, sqlite, fts5]
+---
+
+# RESPLite FT Search Workbench
+
+Use this skill for search-specific work where RediSearch-inspired behavior must stay grounded in what maps naturally to SQLite FTS5.
+
+## Use cases
+
+**Use Case: Add or refine an `FT.*` command**
+Trigger: "implement FT command", "fix FT.SEARCH output", "support another FT option"
+Steps: inspect the FT spec, trace command handler to SQLite search helpers, patch the narrowest layer, add unit and integration coverage, update docs if public behavior changed
+Result: search behavior improves without pretending to support the full RediSearch surface
+
+**Use Case: Work on index storage or ranking**
+Trigger: "change FTS5 schema", "fix BM25 ordering", "debug search index behavior"
+Steps: inspect per-index tables and search helpers, patch FTS mapping or metadata behavior, verify with search-focused tests
+Result: SQLite-backed search behavior remains internally consistent and externally predictable
+
+**Use Case: Improve RediSearch migration**
+Trigger: "migrate FT indices", "map FT.INFO attributes", "handle TAG or NUMERIC fields"
+Steps: inspect search migration mapping, keep support boundaries explicit, warn on downgraded or skipped fields, verify with unit tests
+Result: migration is useful and honest about current limits
+
+## Instructions
+
+### Step 1: Start with the explicit search scope
+
+Read these first:
+
+- `spec/SPEC_D.md`
+- `src/storage/sqlite/search.js`
+- `src/commands/ft-create.js`
+- `src/commands/ft-add.js`
+- `src/commands/ft-search.js`
+- `src/migration/migrate-search.js` when migration is involved
+
+Then inspect the relevant tests:
+
+- `test/unit/search.test.js`
+- `test/integration/search.test.js`
+- `test/unit/ft-parser.test.js`
+- `test/unit/migrate-search.test.js`
+
+### Step 2: Preserve RESPLite's search philosophy
+
+Keep these design constraints visible while you work:
+
+- search is RediSearch-inspired, not RediSearch-complete
+- the implementation should feel natural for SQLite FTS5
+- unsupported behavior should be explicit rather than approximated badly
+- public docs should not overstate feature parity
+
+If a requested feature requires excessive gymnastics or breaks the current storage model, prefer a scoped implementation or a clear unsupported response.
+
+### Step 3: Change the correct layer
+
+Use this split:
+
+- syntax and RESP command shape: `src/commands/ft-*.js`
+- index metadata, tables, search queries, suggestions: `src/storage/sqlite/search.js`
+- migration from a source RediSearch instance: `src/migration/migrate-search.js`
+
+Try not to duplicate validation rules across layers unless that duplication protects a stable public contract.
+
+### Step 4: Keep storage and public behavior aligned
+
+When you modify index or query behavior, verify these invariants:
+
+- index names and field names stay validated
+- supported field types are still explicit
+- document insert or replace behavior remains atomic
+- search result ordering still matches the chosen ranking rules
+- docs and migration warnings reflect the real support level
+
+If you widen support, also update `README.md` where the FT surface is described.
+
+### Step 5: Test the exact path you changed
+
+Use the narrowest relevant set:
+
+- parser tests for argument and syntax behavior
+- search unit tests for index helpers, query validation, ranking, and suggestion storage
+- integration tests for RESP-facing `FT.*` behavior
+- `migrate-search` tests for RediSearch compatibility and downgrade warnings
+
+Prefer regression tests that capture the exact failure mode instead of broad smoke coverage only.
+
+### Step 6: Close with support boundaries
+
+Report:
+
+- what `FT.*` behavior changed
+- whether the change affects runtime search, migration, or both
+- what remains intentionally unsupported
+- which tests prove the new behavior
+
+## RESPLite-specific checklist
+
+- Keep the search surface consistent with `spec/SPEC_D.md`
+- Do not imply full RediSearch parity
+- Preserve the SQLite-first data model
+- Keep migration warnings informative when mapping unsupported field types
+- Update docs when public support changes
+
+## Examples
+
+**Example 1: Fix `FT.SEARCH` behavior**
+User says: "Make `FT.SEARCH` reject invalid query characters consistently."
+Actions: inspect query validation in `src/storage/sqlite/search.js`, add a regression test in `test/unit/search.test.js`, run search integration coverage if the RESP result changes
+Result: search input validation is stricter and reproducible
+
+**Example 2: Improve RediSearch migration mapping**
+User says: "Map `NUMERIC` fields into the current schema as text and emit warnings."
+Actions: inspect `src/migration/migrate-search.js` and `test/unit/migrate-search.test.js`, preserve current support boundaries, update tests to prove the downgrade behavior
+Result: migration becomes more practical without claiming unsupported range-query semantics
+
+## Troubleshooting
+
+**Problem: The feature request sounds like full RediSearch parity**
+Cause: the requested behavior exceeds the current SQLite-first scope.
+Solution: narrow the request to the subset that maps naturally to FTS5, document the unsupported parts, and avoid shipping misleading compatibility claims.
+
+**Problem: Search tests pass but migration still breaks**
+Cause: runtime `FT.*` behavior and migration mapping are separate code paths.
+Solution: verify both `src/storage/sqlite/search.js` and `src/migration/migrate-search.js`, then run both search and migrate-search tests.
+
+**Problem: Delete behavior or row mapping becomes inconsistent**
+Cause: index storage invariants were changed without preserving the doc to row mapping assumptions.
+Solution: review the per-index tables and their relationships in `src/storage/sqlite/search.js`, then add a focused regression test before expanding the change.

package/skills/resplite-migration-cutover-assistant/SKILL.md

@@ -0,0 +1,138 @@
+---
+name: resplite-migration-cutover-assistant
+description: Guides Redis to RESPLite migration work using the programmatic migration API, dirty-key tracking, cutover, and verification. Use when the user says "migrate Redis", "dirty tracker", "cutover", "resume bulk import", "verify migration", or "move RediSearch data during migration". Do not use for generic command work that does not touch the migration flow.
+license: MIT
+metadata:
+  author: Cursor Agent
+  version: 1.0.0
+  category: workflow-automation
+  tags: [resplite, migration, redis, cutover, verification]
+---
+
+# RESPLite Migration Cutover Assistant
+
+Use this skill for migration work where correctness, resumability, and cutover sequencing matter more than raw implementation speed.
+
+## Use cases
+
+**Use Case: Drive a Redis to RESPLite migration**
+Trigger: "help me migrate Redis", "create migration script", "plan cutover"
+Steps: inspect the migration API, build or adjust a single-script flow, verify preflight, bulk, dirty tracking, cutover, and verification
+Result: a reproducible migration workflow with minimal downtime
+
+**Use Case: Fix dirty-key tracking or resumability**
+Trigger: "dirty tracker missed keys", "resume is broken", "migration status is wrong"
+Steps: inspect registry and tracker logic, reproduce through tests, fix the persistent state transitions, verify cutover semantics
+Result: the migration state machine behaves predictably across interruptions
+
+**Use Case: Add or change RediSearch migration behavior**
+Trigger: "migrate FT indices", "search migration", "map RediSearch fields"
+Steps: review `migrate-search`, map fields to current FT support, verify warnings and skipped cases, test the result with fake Redis or integration coverage
+Result: search migration stays aligned with the current FT implementation
+
+## Instructions
+
+### Step 1: Prefer the programmatic API
+
+Start from `resplite/migration`, not from deleted or legacy CLI paths. The current intended entry point is `createMigration()` in `src/migration/index.js`.
+
+Read these sources first:
+
+- `README.md` migration section
+- `spec/SPEC_F.md`
+- `src/migration/index.js`
+- `src/migration/tracker.js`
+- `src/migration/migrate-search.js` when `FT.*` is involved
+
+If the requested change conflicts with the documented sequence, treat the documented cutover flow as the source of truth unless the task explicitly updates docs and behavior together.
+
+### Step 2: Keep the real cutover model intact
+
+Model the migration as these phases:
+
+1. `preflight()`
+2. `enableKeyspaceNotifications()`
+3. `startDirtyTracker()`
+4. `bulk({ resume: true })`
+5. pause and freeze writes to Redis
+6. `applyDirty()`
+7. `stopDirtyTracker()`
+8. `migrateSearch()` if the source uses `FT.*`
+9. `verify()`
+10. `close()`
+
+Do not collapse cutover into "bulk then switch" if dirty tracking or write freeze is part of the scenario.
+
+### Step 3: Preserve resumability and persistence semantics
+
+When editing migration internals, make sure these properties stay true:
+
+- bulk progress checkpoints can resume after interruption
+- dirty keys are persisted in SQLite
+- final cutover uses the last dirty set after writes are frozen
+- status can be read from SQLite without Redis being connected
+
+If the source Redis has a renamed `CONFIG` command, keep `configCommand` support working across both preflight and notification setup.
+
+### Step 4: Distinguish KV migration from search migration
+
+Keep these boundaries clear:
+
+- `bulk` and `applyDirty` handle strings, hashes, sets, lists, and zsets
+- `migrateSearch()` handles RediSearch schema and document migration separately
+- search migration is not a substitute for the dirty-key delta flow
+
+This separation keeps the code and operator workflow predictable.
+
+### Step 5: Verify the exact failure mode
+
+Use the narrowest useful test set:
+
+- `test/integration/migration-dirty-tracker.test.js` for dirty tracking and cutover behavior
+- `test/unit/migration-registry.test.js` for persistent registry semantics
+- `test/unit/migrate-search.test.js` for RediSearch mapping and migration edge cases
+
+If you change the public script examples or workflow guidance, update `README.md` so the docs and implementation stay in sync.
+
+### Step 6: Report operationally, not just structurally
+
+When closing the task, summarize:
+
+- what phase of the migration flow changed
+- whether the change affects operator cutover steps
+- what verification was run
+- any remaining assumptions about Redis configuration, notifications, or FT availability
+
+## RESPLite-specific checklist
+
+- Use the single-script programmatic flow as the happy path
+- Treat the final dirty apply after write freeze as authoritative
+- Keep `resume: true` behavior simple and restart-safe
+- Preserve the distinction between key migration and search migration
+- Update docs when operator-facing behavior changes
+
+## Examples
+
+**Example 1: Fix cutover sequencing**
+User says: "The migration docs and code should make the final dirty apply happen only after writes are frozen."
+Actions: review `README.md` and `spec/SPEC_F.md`, patch the sequencing if needed, adjust tests, verify the user-facing migration script matches the intended cutover flow
+Result: docs, code, and tests describe the same operational sequence
+
+**Example 2: Improve search migration**
+User says: "Map RediSearch TAG fields into the current FT implementation with warnings."
+Actions: inspect `src/migration/migrate-search.js`, preserve the current FT scope, update unit tests around field mapping, report unsupported field types clearly
+Result: migration becomes more helpful without pretending to support unsupported search behavior
+
+## Troubleshooting
+
+**Problem: Dirty tracking appears to miss updates**
+Cause: keyspace notifications are not enabled, the tracker was not running during bulk, or the scenario expects durable CDC semantics that Redis notifications do not provide.
+Solution: verify `enableKeyspaceNotifications()`, keep the tracker active during bulk, and rely on the final write freeze plus `applyDirty()` for authoritative cutover.
+
+**Problem: Bulk resume logic is confusing**
+Cause: code or docs treat first run and resumed run as separate flows.
+Solution: keep `resume: true` as the default simple path and make sure the same script works for both start and resume.
+
+**Problem: Search migration is mixed into the key delta**
+Cause: the implementation is blurring `migrateSearch()` with key replication semantics.
+Solution: keep the keyspace migration and RediSearch migration as separate steps, then verify each with the appropriate tests.

package/spec/00-INDEX.md

@@ -0,0 +1,37 @@
+# RESPLite Specifications — Index
+
+Specifications are split by topic. Names are specific and ordered by dependency.
+
+## Core (RESP server v1)
+
+| File | Content |
+|------|--------|
+| [01-overview-and-goals.md](01-overview-and-goals.md) | Overview, product goals, non-goals, positioning |
+| [02-protocol-and-commands.md](02-protocol-and-commands.md) | Protocol (RESP2), command scope, semantics, SET scope |
+| [03-data-model-ttl-transactions.md](03-data-model-ttl-transactions.md) | Data model (SQLite schema), TTL/expiration, transaction rules |
+| [04-cache-architecture.md](04-cache-architecture.md) | Hot cache, architecture layers, internal API shape |
+| [05-scan-admin-implementation.md](05-scan-admin-implementation.md) | SCAN, admin commands (SQLITE.INFO, CACHE.INFO), project structure, testing, acceptance, implementation order |
+| [06-migration-strategy-core.md](06-migration-strategy-core.md) | Migration strategy from Redis (programmatic), compatibility matrix, final design rule |
+
+## Data types (beyond base v1)
+
+| File | Content |
+|------|--------|
+| [07-type-lists.md](07-type-lists.md) | Lists (LPUSH, RPUSH, LPOP, RPOP, LLEN, LRANGE, LINDEX, …) — data model and behavior |
+| [08-type-sorted-sets.md](08-type-sorted-sets.md) | Sorted sets (ZADD, ZREM, ZCARD, ZSCORE, ZRANGE, ZRANGEBYSCORE) — data model and behavior |
+| [09-search-ft-commands.md](09-search-ft-commands.md) | Search (FT.CREATE, FT.ADD, FT.SEARCH, FT.SUG*, …) — FTS5/BM25, schema, parser grammar |
+| [10-blocking-commands.md](10-blocking-commands.md) | Blocking list commands (BLPOP, BRPOP) — wait model, wakeup, tests |
+
+## Migration (dirty key registry)
+
+| File | Content |
+|------|--------|
+| [11-migration-dirty-registry.md](11-migration-dirty-registry.md) | Bulk import, dirty key tracker, delta apply, search index migration, operational guidance |
+
+---
+
+## Implementation status (as of review)
+
+- **Implemented:** All commands in §02 (connection, strings, TTL, hashes, sets, SCAN, admin), plus Lists (§07), Sorted Sets (§08), FT.* (§09), BLPOP/BRPOP (§10). Migration API with bulk, dirty tracker, apply-dirty, migrateSearch (§11).
+- **Schema:** Matches §03 and extends to list/zset/search tables per §07, §08, §09.
+- **Specs:** SPEC_A–F have been split into the files above; content is unchanged except in Appendix F: the duplicate “F.10 Search Index Migration” was renumbered to **F.12**, and “F.12 Operational Guidance” to **F.13**.

package/spec/01-overview-and-goals.md

@@ -0,0 +1,125 @@
+# RESPLite Specification v1 — Overview and Goals
+
+## 1. Overview
+
+RESPLite is a Redis-compatible subset server implemented in Node.js and backed by SQLite.
+It exposes a TCP port that speaks RESP2 so existing Redis clients can connect to it.
+The system is designed for single-node deployments where persistence, low operational overhead, and practical Redis compatibility matter more than full Redis feature parity.
+
+This project is not a full Redis clone.
+It is a RESP server with Redis-like semantics for a carefully selected subset of commands that map naturally and efficiently to SQLite.
+
+Core principles:
+
+- RESP-first product design
+- SQLite as the persistent source of truth
+- Hot in-memory cache for frequently accessed data
+- Strictly scoped compatibility
+- Strong correctness and test coverage before feature expansion
+- No features that require unnatural or fragile implementation on SQLite
+
+---
+
+## 2. Product Goals
+
+Version 1 focuses on the following goals:
+
+- Expose a TCP server that speaks RESP2
+- Support a useful subset of Redis commands
+- Persist all data in SQLite
+- Maintain practical compatibility with existing Redis clients
+- Support strings, hashes, sets, and TTLs
+- Provide SCAN and TYPE for basic introspection
+- Keep the architecture ready for future FT.* commands powered by SQLite FTS5 with BM25 ranking
+
+This project is intended to be packaged and distributed as an npm module, but v1 starts with RESP server mode only.
+There is no embedded direct JavaScript API in the initial scope.
+
+---
+
+## 3. Non-Goals
+
+The following are explicitly out of scope for v1:
+
+- Pub/Sub
+- Streams
+- Lua scripting
+- Redis Cluster
+- Replication
+- MULTI / EXEC / WATCH
+- Blocking commands such as BLPOP
+- Multiple logical databases via SELECT
+- Full edge-case parity with Redis for every command and protocol nuance
+- Matching Redis performance for high-concurrency, memory-first workloads
+
+The project should return a clear error for unsupported commands:
+
+- `ERR command not supported yet`
+
+---
+
+## 4. Positioning
+
+RESPLite should be described as:
+
+> A RESP2 server with practical Redis compatibility, backed by SQLite for persistent single-node workloads.
+
+It should not be described as:
+
+- a full Redis replacement for all workloads
+- a Redis Cluster alternative
+- a low-latency in-memory data store competitor for large-scale write-heavy systems
+
+The intended sweet spot is:
+
+- bots
+- internal tools
+- small to medium web applications
+- persistent caches
+- local development environments
+- low-ops VPS deployments
+- metadata stores
+- feature flags
+- counters
+- session-like state where persistence matters
+
+---
+
+## 5. Protocol
+
+### 5.1 Supported protocol version
+
+v1 supports:
+
+- RESP2 only
+
+RESP3 is out of scope for v1.
+
+### 5.2 Wire compatibility target
+
+The server should be compatible enough for practical use with:
+
+- `redis-cli`
+- the official `redis` npm client
+
+The wire-level contract must correctly support:
+
+- Simple Strings
+- Bulk Strings
+- Null Bulk Strings
+- Integers
+- Arrays
+- Errors
+
+### 5.3 Binary safety
+
+The implementation must treat the following as binary-safe values:
+
+- keys
+- string values
+- hash fields
+- hash values
+- set members
+
+Internal command processing should use `Buffer` objects, not UTF-8 strings, as the default representation.
+SQLite storage should use `BLOB` columns where appropriate.