resplite 1.2.6 → 1.2.10

package/spec/02-protocol-and-commands.md

@@ -0,0 +1,174 @@
+# RESPLite Specification v1 — Protocol and Commands
+
+## 6. Command Scope for v1
+
+### 6.1 Connection and basic commands
+
+Supported:
+
+- `PING`
+- `ECHO`
+- `QUIT`
+
+### 6.2 String commands
+
+Supported:
+
+- `GET`
+- `SET`
+- `MGET`
+- `MSET`
+- `DEL`
+- `EXISTS`
+- `INCR`
+- `DECR`
+- `INCRBY`
+- `DECRBY`
+
+### 6.3 TTL commands
+
+Supported:
+
+- `EXPIRE`
+- `PEXPIRE`
+- `TTL`
+- `PTTL`
+- `PERSIST`
+
+### 6.4 Hash commands
+
+Supported:
+
+- `HSET`
+- `HGET`
+- `HMGET`
+- `HGETALL`
+- `HDEL`
+- `HEXISTS`
+- `HINCRBY`
+
+### 6.5 Set commands
+
+Supported:
+
+- `SADD`
+- `SREM`
+- `SMEMBERS`
+- `SISMEMBER`
+- `SCARD`
+
+### 6.6 Introspection and navigation
+
+Supported:
+
+- `TYPE`
+- `OBJECT IDLETIME` (seconds since last write; uses `updated_at`; missing key returns nil)
+- `SCAN`
+
+### 6.7 Administrative extension commands
+
+Supported as project-specific commands:
+
+- `SQLITE.INFO`
+- `CACHE.INFO`
+
+These are not Redis-standard commands.
+They exist for observability and operational insight.
+
+---
+
+## 7. Commands Explicitly Not Supported in v1
+
+The following commands are out of scope in v1 and should return a clear unsupported-command error:
+
+- `SUBSCRIBE`
+- `PUBLISH`
+- `PSUBSCRIBE`
+- `MULTI`
+- `EXEC`
+- `WATCH`
+- `EVAL`
+- `EVALSHA`
+- `XADD`
+- `XRANGE`
+- `XREAD`
+- `ZADD`
+- `ZRANGE`
+- `LPUSH`
+- `RPUSH`
+- `BLPOP`
+- `SELECT`
+
+Future support may be considered only if the implementation maps cleanly to SQLite.
+
+---
+
+## 8. Semantic Rules
+
+### 8.1 Type ownership
+
+A key has exactly one logical type at a time.
+Supported types in v1:
+
+- `string`
+- `hash`
+- `set`
+
+If a command targets a key of the wrong type, the server must return:
+
+- `WRONGTYPE Operation against a key holding the wrong kind of value`
+
+### 8.2 Missing keys
+
+Behavior should follow Redis-like semantics where reasonable.
+Examples:
+
+- `GET missing` returns null bulk string
+- `TTL missing` returns `-2`
+- `PTTL missing` returns `-2`
+- `TYPE missing` returns `none`
+
+### 8.3 Keys without expiration
+
+For existing keys without expiration:
+
+- `TTL key` returns `-1`
+- `PTTL key` returns `-1`
+
+### 8.4 DEL and EXISTS
+
+- `DEL` returns the count of removed keys
+- `EXISTS` returns the count of keys that exist
+
+### 8.5 Numeric string commands
+
+`INCR`, `DECR`, `INCRBY`, and `DECRBY` operate on string values interpreted as integers.
+Rules:
+
+- missing key behaves like zero, then the operation is applied
+- non-integer content returns an error
+- result is persisted as a string-compatible integer representation
+
+### 8.6 Empty container behavior
+
+For hashes and sets, when the last field or member is removed and the structure becomes empty, the logical key should be deleted as well.
+This keeps the logical keyspace clean and avoids stale empty types.
+
+---
+
+## 9. SET Command v1 Scope
+
+Supported forms in v1:
+
+- `SET key value`
+- `SET key value EX seconds`
+- `SET key value PX milliseconds`
+
+Not supported in v1:
+
+- `NX`
+- `XX`
+- `GET`
+- `KEEPTTL`
+
+Invalid syntax should produce a Redis-style syntax error.

package/spec/03-data-model-ttl-transactions.md

@@ -0,0 +1,157 @@
+# RESPLite Specification v1 — Data Model, TTL, and Transactions
+
+## 10. Data Model
+
+SQLite is the persistent source of truth.
+Data is stored in separate tables by logical type.
+This avoids an overly generic storage model and keeps operations natural and efficient.
+
+### 10.1 Key metadata table
+
+```sql
+CREATE TABLE redis_keys (
+  key BLOB PRIMARY KEY,
+  type INTEGER NOT NULL,
+  expires_at INTEGER,
+  version INTEGER NOT NULL DEFAULT 1,
+  updated_at INTEGER NOT NULL
+);
+
+CREATE INDEX redis_keys_expires_at_idx ON redis_keys(expires_at);
+CREATE INDEX redis_keys_type_idx ON redis_keys(type);
+```
+
+Recommended type enum values:
+
+- `1` = string
+- `2` = hash
+- `3` = set
+
+Notes:
+
+- `key` is stored as `BLOB`
+- `expires_at` is an absolute timestamp in milliseconds
+- `version` supports cache invalidation
+- `updated_at` supports observability and future maintenance tasks; also used by `OBJECT IDLETIME` (time since last write)
+
+### 10.2 String storage
+
+```sql
+CREATE TABLE redis_strings (
+  key BLOB PRIMARY KEY,
+  value BLOB NOT NULL,
+  FOREIGN KEY(key) REFERENCES redis_keys(key) ON DELETE CASCADE
+);
+```
+
+### 10.3 Hash storage
+
+```sql
+CREATE TABLE redis_hashes (
+  key BLOB NOT NULL,
+  field BLOB NOT NULL,
+  value BLOB NOT NULL,
+  PRIMARY KEY (key, field),
+  FOREIGN KEY(key) REFERENCES redis_keys(key) ON DELETE CASCADE
+);
+```
+
+### 10.4 Set storage
+
+```sql
+CREATE TABLE redis_sets (
+  key BLOB NOT NULL,
+  member BLOB NOT NULL,
+  PRIMARY KEY (key, member),
+  FOREIGN KEY(key) REFERENCES redis_keys(key) ON DELETE CASCADE
+);
+```
+
+---
+
+## 11. TTL and Expiration
+
+Expiration must be implemented in two complementary ways.
+
+### 11.1 Lazy expiration
+
+Before any command operates on a key, the engine should verify whether the key has expired.
+If it has expired:
+
+- the key must be removed from SQLite
+- any cache entry must be invalidated
+- the command should proceed as if the key does not exist
+
+### 11.2 Active expiration
+
+A background sweeper should periodically delete expired keys in batches.
+Suggested configuration:
+
+```js
+{
+  expiration: {
+    sweepIntervalMs: 1000,
+    maxKeysPerSweep: 500
+  }
+}
+```
+
+This does not need to guarantee exact expiration timing to the millisecond.
+It must guarantee that expired keys behave as non-existent from the client's point of view.
+
+---
+
+## 15. SQLite Behavior and Pragmas
+
+The storage layer should use pragmatic defaults tuned for this workload.
+Suggested initial pragmas:
+
+```sql
+PRAGMA journal_mode=WAL;
+PRAGMA synchronous=NORMAL;
+PRAGMA foreign_keys=ON;
+PRAGMA temp_store=MEMORY;
+PRAGMA cache_size=-20000;
+PRAGMA mmap_size=268435456;
+```
+
+These settings may later become configurable.
+
+---
+
+## 16. Transaction Rules
+
+Every state-changing operation that spans multiple storage steps must run inside a SQLite transaction.
+
+Examples:
+
+### 16.1 SET
+
+A SET operation may require:
+
+- writing or updating metadata in `redis_keys`
+- deleting rows from other type tables if type replacement is allowed for string-over-string writes only
+- writing to `redis_strings`
+- updating version and timestamp
+- updating or invalidating cache
+
+All logical storage changes must be atomic.
+
+### 16.2 HSET
+
+An HSET operation may require:
+
+- creating key metadata if the key does not yet exist
+- validating type if the key already exists
+- inserting or updating one or more fields
+- updating version and timestamp
+- updating or invalidating cache
+
+### 16.3 SADD / SREM
+
+Set modifications must update:
+
+- membership rows
+- metadata timestamps and versions
+- key existence if the set becomes empty
+- cache state

package/spec/04-cache-architecture.md

@@ -0,0 +1,171 @@
+# RESPLite Specification v1 — Cache and Architecture
+
+## 12. Hot Cache
+
+The cache is an optimization layer only.
+SQLite remains the source of truth.
+
+### 12.1 Purpose
+
+The cache should reduce repeated reads from SQLite for hot keys.
+It should not change logical behavior.
+
+### 12.2 Cache candidates
+
+Good initial cache candidates:
+
+- strings
+- small hashes
+- small sets
+- key metadata such as type and expiration
+
+The cache should not aggressively optimize large result sets in v1.
+
+### 12.3 Cache model
+
+Suggested internal cache entry structure:
+
+```js
+{
+  kind: "string" | "hash" | "set",
+  version: number,
+  expiresAt: number | null,
+  value: Buffer | Map | Array
+}
+```
+
+### 12.4 Invalidation strategy
+
+Writes should update or invalidate cache entries immediately after a successful SQLite transaction.
+Version values stored in `redis_keys` should be used to prevent stale cache reads.
+
+### 12.5 Policy
+
+The cache should start with an LRU policy and support limits such as:
+
+```js
+{
+  cache: {
+    maxEntries: 50000,
+    maxBytes: 64 * 1024 * 1024
+  }
+}
+```
+
+---
+
+## 13. Architecture
+
+The implementation should be layered and protocol-independent at the core.
+
+### 13.1 Core layers
+
+1. TCP server layer
+2. RESP parser and encoder layer
+3. Command dispatcher layer
+4. Engine layer with Redis-like semantics
+5. SQLite storage layer
+6. Cache layer
+7. Expiration subsystem
+
+### 13.2 Layer responsibilities
+
+#### TCP server layer
+
+Responsible for:
+
+- accepting TCP connections
+- reading data from sockets
+- writing encoded RESP replies
+- handling connection lifecycle
+
+#### RESP layer
+
+Responsible for:
+
+- parsing incoming RESP2 frames
+- handling fragmented packets and multiple commands per chunk
+- encoding valid RESP2 responses
+
+#### Command dispatcher
+
+Responsible for:
+
+- normalizing command names to uppercase
+- routing commands to handlers
+- returning supported or unsupported command results
+
+#### Engine
+
+Responsible for:
+
+- key existence checks
+- expiration handling
+- type validation
+- semantic correctness
+- numeric operations
+- cleanup of empty structures
+- cache coordination
+
+#### SQLite storage layer
+
+Responsible for:
+
+- schema creation and migration
+- prepared statements
+- transactions
+- efficient per-type operations
+- SQLite pragmas
+
+#### Cache layer
+
+Responsible for:
+
+- storing hot results
+- enforcing limits
+- evicting entries
+- exposing metrics
+
+#### Expiration subsystem
+
+Responsible for:
+
+- lazy expiration checks
+- active sweeps
+- deletion batch limits
+
+---
+
+## 14. Internal API Shape
+
+Even though v1 is RESP-only, the internal engine should expose clear semantic operations.
+This keeps the system testable and prepares the codebase for a future embedded API if desired.
+
+Suggested engine methods:
+
+```js
+engine.get(key)
+engine.set(key, value, options)
+engine.del(keys)
+engine.exists(keys)
+engine.expire(key, ttlMs)
+engine.pttl(key)
+engine.persist(key)
+
+engine.hset(key, pairs)
+engine.hget(key, field)
+engine.hmget(key, fields)
+engine.hgetall(key)
+engine.hdel(key, fields)
+engine.hexists(key, field)
+engine.hincrby(key, field, amount)
+
+engine.sadd(key, members)
+engine.srem(key, members)
+engine.smembers(key)
+engine.sismember(key, member)
+engine.scard(key)
+
+engine.type(key)
+engine.scan(cursor, options)
+```