resplite 1.2.6 → 1.2.10

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (36) hide show
  1. package/README.md +168 -275
  2. package/package.json +1 -6
  3. package/scripts/create-interface-smoke.js +32 -0
  4. package/skills/README.md +22 -0
  5. package/skills/resplite-command-vertical-slice/SKILL.md +134 -0
  6. package/skills/resplite-ft-search-workbench/SKILL.md +138 -0
  7. package/skills/resplite-migration-cutover-assistant/SKILL.md +138 -0
  8. package/spec/00-INDEX.md +37 -0
  9. package/spec/01-overview-and-goals.md +125 -0
  10. package/spec/02-protocol-and-commands.md +174 -0
  11. package/spec/03-data-model-ttl-transactions.md +157 -0
  12. package/spec/04-cache-architecture.md +171 -0
  13. package/spec/05-scan-admin-implementation.md +379 -0
  14. package/spec/06-migration-strategy-core.md +79 -0
  15. package/spec/07-type-lists.md +202 -0
  16. package/spec/08-type-sorted-sets.md +220 -0
  17. package/spec/{SPEC_D.md → 09-search-ft-commands.md} +3 -1
  18. package/spec/{SPEC_E.md → 10-blocking-commands.md} +3 -1
  19. package/spec/{SPEC_F.md → 11-migration-dirty-registry.md} +61 -147
  20. package/src/commands/object.js +17 -0
  21. package/src/commands/registry.js +4 -0
  22. package/src/commands/zrevrange.js +27 -0
  23. package/src/engine/engine.js +19 -0
  24. package/src/migration/apply-dirty.js +8 -1
  25. package/src/migration/index.js +5 -4
  26. package/src/migration/migrate-search.js +25 -6
  27. package/src/storage/sqlite/zsets.js +34 -0
  28. package/test/integration/object-idletime.test.js +51 -0
  29. package/test/integration/zsets.test.js +18 -0
  30. package/test/unit/migrate-search.test.js +50 -2
  31. package/spec/SPEC_A.md +0 -1171
  32. package/spec/SPEC_B.md +0 -426
  33. package/src/cli/import-from-redis.js +0 -194
  34. package/src/cli/resplite-dirty-tracker.js +0 -92
  35. package/src/cli/resplite-import.js +0 -296
  36. package/test/contract/import-from-redis.test.js +0 -83
package/spec/SPEC_A.md DELETED
@@ -1,1171 +0,0 @@
1
- # RESPLite Specification v1
2
-
3
- ## 1. Overview
4
-
5
- RESPLite is a Redis-compatible subset server implemented in Node.js and backed by SQLite.
6
- It exposes a TCP port that speaks RESP2 so existing Redis clients can connect to it.
7
- The system is designed for single-node deployments where persistence, low operational overhead, and practical Redis compatibility matter more than full Redis feature parity.
8
-
9
- This project is not a full Redis clone.
10
- It is a RESP server with Redis-like semantics for a carefully selected subset of commands that map naturally and efficiently to SQLite.
11
-
12
- Core principles:
13
-
14
- - RESP-first product design
15
- - SQLite as the persistent source of truth
16
- - Hot in-memory cache for frequently accessed data
17
- - Strictly scoped compatibility
18
- - Strong correctness and test coverage before feature expansion
19
- - No features that require unnatural or fragile implementation on SQLite
20
-
21
- ---
22
-
23
- ## 2. Product Goals
24
-
25
- Version 1 focuses on the following goals:
26
-
27
- - Expose a TCP server that speaks RESP2
28
- - Support a useful subset of Redis commands
29
- - Persist all data in SQLite
30
- - Maintain practical compatibility with existing Redis clients
31
- - Support strings, hashes, sets, and TTLs
32
- - Provide SCAN and TYPE for basic introspection
33
- - Keep the architecture ready for future FT.* commands powered by SQLite FTS5 with BM25 ranking
34
-
35
- This project is intended to be packaged and distributed as an npm module, but v1 starts with RESP server mode only.
36
- There is no embedded direct JavaScript API in the initial scope.
37
-
38
- ---
39
-
40
- ## 3. Non-Goals
41
-
42
- The following are explicitly out of scope for v1:
43
-
44
- - Pub/Sub
45
- - Streams
46
- - Lua scripting
47
- - Redis Cluster
48
- - Replication
49
- - MULTI / EXEC / WATCH
50
- - Blocking commands such as BLPOP
51
- - Multiple logical databases via SELECT
52
- - Full edge-case parity with Redis for every command and protocol nuance
53
- - Matching Redis performance for high-concurrency, memory-first workloads
54
-
55
- The project should return a clear error for unsupported commands:
56
-
57
- - `ERR command not supported yet`
58
-
59
- ---
60
-
61
- ## 4. Positioning
62
-
63
- RESPLite should be described as:
64
-
65
- > A RESP2 server with practical Redis compatibility, backed by SQLite for persistent single-node workloads.
66
-
67
- It should not be described as:
68
-
69
- - a full Redis replacement for all workloads
70
- - a Redis Cluster alternative
71
- - a low-latency in-memory data store competitor for large-scale write-heavy systems
72
-
73
- The intended sweet spot is:
74
-
75
- - bots
76
- - internal tools
77
- - small to medium web applications
78
- - persistent caches
79
- - local development environments
80
- - low-ops VPS deployments
81
- - metadata stores
82
- - feature flags
83
- - counters
84
- - session-like state where persistence matters
85
-
86
- ---
87
-
88
- ## 5. Protocol
89
-
90
- ### 5.1 Supported protocol version
91
-
92
- v1 supports:
93
-
94
- - RESP2 only
95
-
96
- RESP3 is out of scope for v1.
97
-
98
- ### 5.2 Wire compatibility target
99
-
100
- The server should be compatible enough for practical use with:
101
-
102
- - `redis-cli`
103
- - the official `redis` npm client
104
-
105
- The wire-level contract must correctly support:
106
-
107
- - Simple Strings
108
- - Bulk Strings
109
- - Null Bulk Strings
110
- - Integers
111
- - Arrays
112
- - Errors
113
-
114
- ### 5.3 Binary safety
115
-
116
- The implementation must treat the following as binary-safe values:
117
-
118
- - keys
119
- - string values
120
- - hash fields
121
- - hash values
122
- - set members
123
-
124
- Internal command processing should use `Buffer` objects, not UTF-8 strings, as the default representation.
125
- SQLite storage should use `BLOB` columns where appropriate.
126
-
127
- ---
128
-
129
- ## 6. Command Scope for v1
130
-
131
- ### 6.1 Connection and basic commands
132
-
133
- Supported:
134
-
135
- - `PING`
136
- - `ECHO`
137
- - `QUIT`
138
-
139
- ### 6.2 String commands
140
-
141
- Supported:
142
-
143
- - `GET`
144
- - `SET`
145
- - `MGET`
146
- - `MSET`
147
- - `DEL`
148
- - `EXISTS`
149
- - `INCR`
150
- - `DECR`
151
- - `INCRBY`
152
- - `DECRBY`
153
-
154
- ### 6.3 TTL commands
155
-
156
- Supported:
157
-
158
- - `EXPIRE`
159
- - `PEXPIRE`
160
- - `TTL`
161
- - `PTTL`
162
- - `PERSIST`
163
-
164
- ### 6.4 Hash commands
165
-
166
- Supported:
167
-
168
- - `HSET`
169
- - `HGET`
170
- - `HMGET`
171
- - `HGETALL`
172
- - `HDEL`
173
- - `HEXISTS`
174
- - `HINCRBY`
175
-
176
- ### 6.5 Set commands
177
-
178
- Supported:
179
-
180
- - `SADD`
181
- - `SREM`
182
- - `SMEMBERS`
183
- - `SISMEMBER`
184
- - `SCARD`
185
-
186
- ### 6.6 Introspection and navigation
187
-
188
- Supported:
189
-
190
- - `TYPE`
191
- - `SCAN`
192
-
193
- ### 6.7 Administrative extension commands
194
-
195
- Supported as project-specific commands:
196
-
197
- - `SQLITE.INFO`
198
- - `CACHE.INFO`
199
-
200
- These are not Redis-standard commands.
201
- They exist for observability and operational insight.
202
-
203
- ---
204
-
205
- ## 7. Commands Explicitly Not Supported in v1
206
-
207
- The following commands are out of scope in v1 and should return a clear unsupported-command error:
208
-
209
- - `SUBSCRIBE`
210
- - `PUBLISH`
211
- - `PSUBSCRIBE`
212
- - `MULTI`
213
- - `EXEC`
214
- - `WATCH`
215
- - `EVAL`
216
- - `EVALSHA`
217
- - `XADD`
218
- - `XRANGE`
219
- - `XREAD`
220
- - `ZADD`
221
- - `ZRANGE`
222
- - `LPUSH`
223
- - `RPUSH`
224
- - `BLPOP`
225
- - `SELECT`
226
-
227
- Future support may be considered only if the implementation maps cleanly to SQLite.
228
-
229
- ---
230
-
231
- ## 8. Semantic Rules
232
-
233
- ### 8.1 Type ownership
234
-
235
- A key has exactly one logical type at a time.
236
- Supported types in v1:
237
-
238
- - `string`
239
- - `hash`
240
- - `set`
241
-
242
- If a command targets a key of the wrong type, the server must return:
243
-
244
- - `WRONGTYPE Operation against a key holding the wrong kind of value`
245
-
246
- ### 8.2 Missing keys
247
-
248
- Behavior should follow Redis-like semantics where reasonable.
249
- Examples:
250
-
251
- - `GET missing` returns null bulk string
252
- - `TTL missing` returns `-2`
253
- - `PTTL missing` returns `-2`
254
- - `TYPE missing` returns `none`
255
-
256
- ### 8.3 Keys without expiration
257
-
258
- For existing keys without expiration:
259
-
260
- - `TTL key` returns `-1`
261
- - `PTTL key` returns `-1`
262
-
263
- ### 8.4 DEL and EXISTS
264
-
265
- - `DEL` returns the count of removed keys
266
- - `EXISTS` returns the count of keys that exist
267
-
268
- ### 8.5 Numeric string commands
269
-
270
- `INCR`, `DECR`, `INCRBY`, and `DECRBY` operate on string values interpreted as integers.
271
- Rules:
272
-
273
- - missing key behaves like zero, then the operation is applied
274
- - non-integer content returns an error
275
- - result is persisted as a string-compatible integer representation
276
-
277
- ### 8.6 Empty container behavior
278
-
279
- For hashes and sets, when the last field or member is removed and the structure becomes empty, the logical key should be deleted as well.
280
- This keeps the logical keyspace clean and avoids stale empty types.
281
-
282
- ---
283
-
284
- ## 9. SET Command v1 Scope
285
-
286
- Supported forms in v1:
287
-
288
- - `SET key value`
289
- - `SET key value EX seconds`
290
- - `SET key value PX milliseconds`
291
-
292
- Not supported in v1:
293
-
294
- - `NX`
295
- - `XX`
296
- - `GET`
297
- - `KEEPTTL`
298
-
299
- Invalid syntax should produce a Redis-style syntax error.
300
-
301
- ---
302
-
303
- ## 10. Data Model
304
-
305
- SQLite is the persistent source of truth.
306
- Data is stored in separate tables by logical type.
307
- This avoids an overly generic storage model and keeps operations natural and efficient.
308
-
309
- ### 10.1 Key metadata table
310
-
311
- ```sql
312
- CREATE TABLE redis_keys (
313
- key BLOB PRIMARY KEY,
314
- type INTEGER NOT NULL,
315
- expires_at INTEGER,
316
- version INTEGER NOT NULL DEFAULT 1,
317
- updated_at INTEGER NOT NULL
318
- );
319
-
320
- CREATE INDEX redis_keys_expires_at_idx ON redis_keys(expires_at);
321
- CREATE INDEX redis_keys_type_idx ON redis_keys(type);
322
- ```
323
-
324
- Recommended type enum values:
325
-
326
- - `1` = string
327
- - `2` = hash
328
- - `3` = set
329
-
330
- Notes:
331
-
332
- - `key` is stored as `BLOB`
333
- - `expires_at` is an absolute timestamp in milliseconds
334
- - `version` supports cache invalidation
335
- - `updated_at` supports observability and future maintenance tasks
336
-
337
- ### 10.2 String storage
338
-
339
- ```sql
340
- CREATE TABLE redis_strings (
341
- key BLOB PRIMARY KEY,
342
- value BLOB NOT NULL,
343
- FOREIGN KEY(key) REFERENCES redis_keys(key) ON DELETE CASCADE
344
- );
345
- ```
346
-
347
- ### 10.3 Hash storage
348
-
349
- ```sql
350
- CREATE TABLE redis_hashes (
351
- key BLOB NOT NULL,
352
- field BLOB NOT NULL,
353
- value BLOB NOT NULL,
354
- PRIMARY KEY (key, field),
355
- FOREIGN KEY(key) REFERENCES redis_keys(key) ON DELETE CASCADE
356
- );
357
- ```
358
-
359
- ### 10.4 Set storage
360
-
361
- ```sql
362
- CREATE TABLE redis_sets (
363
- key BLOB NOT NULL,
364
- member BLOB NOT NULL,
365
- PRIMARY KEY (key, member),
366
- FOREIGN KEY(key) REFERENCES redis_keys(key) ON DELETE CASCADE
367
- );
368
- ```
369
-
370
- ---
371
-
372
- ## 11. TTL and Expiration
373
-
374
- Expiration must be implemented in two complementary ways.
375
-
376
- ### 11.1 Lazy expiration
377
-
378
- Before any command operates on a key, the engine should verify whether the key has expired.
379
- If it has expired:
380
-
381
- - the key must be removed from SQLite
382
- - any cache entry must be invalidated
383
- - the command should proceed as if the key does not exist
384
-
385
- ### 11.2 Active expiration
386
-
387
- A background sweeper should periodically delete expired keys in batches.
388
- Suggested configuration:
389
-
390
- ```js
391
- {
392
- expiration: {
393
- sweepIntervalMs: 1000,
394
- maxKeysPerSweep: 500
395
- }
396
- }
397
- ```
398
-
399
- This does not need to guarantee exact expiration timing to the millisecond.
400
- It must guarantee that expired keys behave as non-existent from the client's point of view.
401
-
402
- ---
403
-
404
- ## 12. Hot Cache
405
-
406
- The cache is an optimization layer only.
407
- SQLite remains the source of truth.
408
-
409
- ### 12.1 Purpose
410
-
411
- The cache should reduce repeated reads from SQLite for hot keys.
412
- It should not change logical behavior.
413
-
414
- ### 12.2 Cache candidates
415
-
416
- Good initial cache candidates:
417
-
418
- - strings
419
- - small hashes
420
- - small sets
421
- - key metadata such as type and expiration
422
-
423
- The cache should not aggressively optimize large result sets in v1.
424
-
425
- ### 12.3 Cache model
426
-
427
- Suggested internal cache entry structure:
428
-
429
- ```js
430
- {
431
- kind: "string" | "hash" | "set",
432
- version: number,
433
- expiresAt: number | null,
434
- value: Buffer | Map | Array
435
- }
436
- ```
437
-
438
- ### 12.4 Invalidation strategy
439
-
440
- Writes should update or invalidate cache entries immediately after a successful SQLite transaction.
441
- Version values stored in `redis_keys` should be used to prevent stale cache reads.
442
-
443
- ### 12.5 Policy
444
-
445
- The cache should start with an LRU policy and support limits such as:
446
-
447
- ```js
448
- {
449
- cache: {
450
- maxEntries: 50000,
451
- maxBytes: 64 * 1024 * 1024
452
- }
453
- }
454
- ```
455
-
456
- ---
457
-
458
- ## 13. Architecture
459
-
460
- The implementation should be layered and protocol-independent at the core.
461
-
462
- ### 13.1 Core layers
463
-
464
- 1. TCP server layer
465
- 2. RESP parser and encoder layer
466
- 3. Command dispatcher layer
467
- 4. Engine layer with Redis-like semantics
468
- 5. SQLite storage layer
469
- 6. Cache layer
470
- 7. Expiration subsystem
471
-
472
- ### 13.2 Layer responsibilities
473
-
474
- #### TCP server layer
475
-
476
- Responsible for:
477
-
478
- - accepting TCP connections
479
- - reading data from sockets
480
- - writing encoded RESP replies
481
- - handling connection lifecycle
482
-
483
- #### RESP layer
484
-
485
- Responsible for:
486
-
487
- - parsing incoming RESP2 frames
488
- - handling fragmented packets and multiple commands per chunk
489
- - encoding valid RESP2 responses
490
-
491
- #### Command dispatcher
492
-
493
- Responsible for:
494
-
495
- - normalizing command names to uppercase
496
- - routing commands to handlers
497
- - returning supported or unsupported command results
498
-
499
- #### Engine
500
-
501
- Responsible for:
502
-
503
- - key existence checks
504
- - expiration handling
505
- - type validation
506
- - semantic correctness
507
- - numeric operations
508
- - cleanup of empty structures
509
- - cache coordination
510
-
511
- #### SQLite storage layer
512
-
513
- Responsible for:
514
-
515
- - schema creation and migration
516
- - prepared statements
517
- - transactions
518
- - efficient per-type operations
519
- - SQLite pragmas
520
-
521
- #### Cache layer
522
-
523
- Responsible for:
524
-
525
- - storing hot results
526
- - enforcing limits
527
- - evicting entries
528
- - exposing metrics
529
-
530
- #### Expiration subsystem
531
-
532
- Responsible for:
533
-
534
- - lazy expiration checks
535
- - active sweeps
536
- - deletion batch limits
537
-
538
- ---
539
-
540
- ## 14. Internal API Shape
541
-
542
- Even though v1 is RESP-only, the internal engine should expose clear semantic operations.
543
- This keeps the system testable and prepares the codebase for a future embedded API if desired.
544
-
545
- Suggested engine methods:
546
-
547
- ```js
548
- engine.get(key)
549
- engine.set(key, value, options)
550
- engine.del(keys)
551
- engine.exists(keys)
552
- engine.expire(key, ttlMs)
553
- engine.pttl(key)
554
- engine.persist(key)
555
-
556
- engine.hset(key, pairs)
557
- engine.hget(key, field)
558
- engine.hmget(key, fields)
559
- engine.hgetall(key)
560
- engine.hdel(key, fields)
561
- engine.hexists(key, field)
562
- engine.hincrby(key, field, amount)
563
-
564
- engine.sadd(key, members)
565
- engine.srem(key, members)
566
- engine.smembers(key)
567
- engine.sismember(key, member)
568
- engine.scard(key)
569
-
570
- engine.type(key)
571
- engine.scan(cursor, options)
572
- ```
573
-
574
- ---
575
-
576
- ## 15. SQLite Behavior and Pragmas
577
-
578
- The storage layer should use pragmatic defaults tuned for this workload.
579
- Suggested initial pragmas:
580
-
581
- ```sql
582
- PRAGMA journal_mode=WAL;
583
- PRAGMA synchronous=NORMAL;
584
- PRAGMA foreign_keys=ON;
585
- PRAGMA temp_store=MEMORY;
586
- PRAGMA cache_size=-20000;
587
- PRAGMA mmap_size=268435456;
588
- ```
589
-
590
- These settings may later become configurable.
591
-
592
- ---
593
-
594
- ## 16. Transaction Rules
595
-
596
- Every state-changing operation that spans multiple storage steps must run inside a SQLite transaction.
597
-
598
- Examples:
599
-
600
- ### 16.1 SET
601
-
602
- A SET operation may require:
603
-
604
- - writing or updating metadata in `redis_keys`
605
- - deleting rows from other type tables if type replacement is allowed for string-over-string writes only
606
- - writing to `redis_strings`
607
- - updating version and timestamp
608
- - updating or invalidating cache
609
-
610
- All logical storage changes must be atomic.
611
-
612
- ### 16.2 HSET
613
-
614
- An HSET operation may require:
615
-
616
- - creating key metadata if the key does not yet exist
617
- - validating type if the key already exists
618
- - inserting or updating one or more fields
619
- - updating version and timestamp
620
- - updating or invalidating cache
621
-
622
- ### 16.3 SADD / SREM
623
-
624
- Set modifications must update:
625
-
626
- - membership rows
627
- - metadata timestamps and versions
628
- - key existence if the set becomes empty
629
- - cache state
630
-
631
- ---
632
-
633
- ## 17. SCAN Behavior
634
-
635
- SCAN is part of the v1 scope.
636
-
637
- ### 17.1 Minimum supported form
638
-
639
- Required in v1:
640
-
641
- - `SCAN cursor`
642
-
643
- ### 17.2 Response shape
644
-
645
- The response should follow Redis-like shape:
646
-
647
- - next cursor
648
- - array of keys
649
-
650
- ### 17.3 Implementation note
651
-
652
- A simple initial strategy is acceptable, such as lexicographic traversal over `redis_keys.key`.
653
-
654
- ### 17.4 Future extensions
655
-
656
- Possible later additions:
657
-
658
- - `MATCH pattern`
659
- - `COUNT n`
660
-
661
- These are not required in the first implementation.
662
-
663
- ---
664
-
665
- ## 18. Administrative Commands
666
-
667
- ### 18.1 SQLITE.INFO
668
-
669
- This command should expose useful operational information such as:
670
-
671
- - database path
672
- - SQLite version
673
- - counts by type
674
- - total logical key count
675
- - WAL mode status if available
676
-
677
- ### 18.2 CACHE.INFO
678
-
679
- This command should expose cache information such as:
680
-
681
- - cache enabled or disabled
682
- - entry count
683
- - estimated memory use
684
- - hit count
685
- - miss count
686
- - hit ratio if available
687
-
688
- These commands are intended for observability and debugging.
689
-
690
- ---
691
-
692
- ## 19. Future Search Scope
693
-
694
- Search is not part of the base v1 implementation, but the architecture must remain compatible with a near-term v1.1 search layer.
695
-
696
- ### 19.1 Search vision
697
-
698
- The future search subsystem should provide RediSearch-inspired commands powered by SQLite FTS5 with BM25 ranking.
699
-
700
- Candidate commands:
701
-
702
- - `FT.CREATE`
703
- - `FT.SEARCH`
704
- - `FT.INFO`
705
- - `FT.DROPINDEX`
706
-
707
- ### 19.2 Recommended document model
708
-
709
- The preferred future approach is to index hashes as documents.
710
- Example logical model:
711
-
712
- - `article:1` stored as a hash
713
- - an index with prefix `article:`
714
- - fields such as `title`, `body`, and `category`
715
-
716
- ### 19.3 Backend strategy
717
-
718
- - SQLite FTS5 virtual tables
719
- - BM25 ranking
720
- - auxiliary metadata tables for filtering and sorting
721
-
722
- The project should not attempt to fully clone RediSearch grammar or behavior at the start.
723
- A clean, reduced, Redis-like search surface is preferable.
724
-
725
- ---
726
-
727
- ## 20. Project Structure
728
-
729
- Recommended Node.js JavaScript ESM project layout:
730
-
731
- ```text
732
- resplite/
733
- package.json
734
- README.md
735
- SPEC.md
736
- src/
737
- index.js
738
- server/
739
- tcp-server.js
740
- connection.js
741
- resp/
742
- parser.js
743
- encoder.js
744
- types.js
745
- commands/
746
- registry.js
747
- ping.js
748
- echo.js
749
- quit.js
750
- get.js
751
- set.js
752
- del.js
753
- exists.js
754
- expire.js
755
- pexpire.js
756
- ttl.js
757
- pttl.js
758
- persist.js
759
- incr.js
760
- decr.js
761
- incrby.js
762
- decrby.js
763
- type.js
764
- scan.js
765
- hset.js
766
- hget.js
767
- hmget.js
768
- hgetall.js
769
- hdel.js
770
- hexists.js
771
- hincrby.js
772
- sadd.js
773
- srem.js
774
- smembers.js
775
- sismember.js
776
- scard.js
777
- sqlite-info.js
778
- cache-info.js
779
- engine/
780
- engine.js
781
- errors.js
782
- expiration.js
783
- validate.js
784
- storage/
785
- sqlite/
786
- db.js
787
- schema.js
788
- pragmas.js
789
- keys.js
790
- strings.js
791
- hashes.js
792
- sets.js
793
- tx.js
794
- cache/
795
- lru.js
796
- cache.js
797
- util/
798
- buffers.js
799
- patterns.js
800
- clock.js
801
- logger.js
802
- test/
803
- helpers/
804
- server.js
805
- client.js
806
- tmp.js
807
- clock.js
808
- fixtures.js
809
- unit/
810
- resp-parser.test.js
811
- resp-encoder.test.js
812
- engine-strings.test.js
813
- engine-hashes.test.js
814
- engine-sets.test.js
815
- expiration.test.js
816
- cache.test.js
817
- integration/
818
- ping.test.js
819
- strings.test.js
820
- ttl.test.js
821
- hashes.test.js
822
- sets.test.js
823
- type.test.js
824
- scan.test.js
825
- restart-persistence.test.js
826
- wrongtype.test.js
827
- binary-safe.test.js
828
- multi-client.test.js
829
- contract/
830
- redis-cli-smoke.test.js
831
- redis-client.test.js
832
- stress/
833
- concurrency.test.js
834
- sweep-load.test.js
835
- ```
836
-
837
- ---
838
-
839
- ## 21. Implementation Stack
840
-
841
- Recommended initial stack:
842
-
843
- - Node.js
844
- - JavaScript ESM
845
- - `better-sqlite3`
846
- - built-in `node:test`
847
- - `assert/strict`
848
- - `redis` npm package for contract tests
849
-
850
- The v1 codebase should prioritize correctness, testability, and operational simplicity over TypeScript or extensive framework usage.
851
-
852
- ---
853
-
854
- ## 22. Testing Strategy
855
-
856
- Testing is a first-class requirement.
857
- A command is not considered implemented until it has meaningful coverage.
858
-
859
- ### 22.1 Testing categories
860
-
861
- The project must include the following categories of tests:
862
-
863
- - unit tests
864
- - integration tests
865
- - contract tests
866
- - persistence tests
867
- - expiration tests
868
- - consistency tests
869
- - concurrency tests
870
- - protocol framing tests
871
- - binary-safety tests
872
-
873
- ### 22.2 Per-command testing rule
874
-
875
- Each supported command should have tests for:
876
-
877
- - normal behavior
878
- - missing-key behavior where applicable
879
- - wrong-type behavior where applicable
880
- - TTL interaction where applicable
881
- - persistence across restart where applicable
882
- - binary-safe behavior where applicable
883
-
884
- ### 22.3 RESP protocol tests
885
-
886
- The RESP layer must be tested for:
887
-
888
- - valid parsing of RESP2 types
889
- - partial frames split across TCP chunks
890
- - multiple commands arriving in a single TCP chunk
891
- - correct encoding of all response types
892
-
893
- ### 22.4 Contract tests with real clients
894
-
895
- v1 contract tests should use:
896
-
897
- - `redis-cli`
898
- - the official `redis` npm client
899
-
900
- `ioredis` is not required in v1.
901
- It may be added later as an additional compatibility suite.
902
-
903
- ### 22.5 Persistence tests
904
-
905
- The project must verify:
906
-
907
- - data survives server restart
908
- - TTL metadata survives restart
909
- - expired keys behave as missing after restart
910
- - mixed key types remain valid after restart
911
-
912
- ### 22.6 Consistency tests
913
-
914
- The project must verify internal invariants such as:
915
-
916
- - rows in type tables must correspond to a valid row in `redis_keys`
917
- - logical type must match actual storage table placement
918
- - expired keys must not appear through command results
919
- - empty hashes and sets are removed logically
920
-
921
- ### 22.7 Concurrency tests
922
-
923
- The project must verify behavior under multiple clients, including:
924
-
925
- - many concurrent INCR operations on the same key
926
- - reads and writes overlapping across connections
927
- - expiration sweeps occurring during active traffic
928
- - restart safety under recent write activity
929
-
930
- ### 22.8 Performance sanity tests
931
-
932
- The project does not need benchmark-grade tests in v1, but it should include sanity checks such as:
933
-
934
- - repeated SET and GET operations at modest scale
935
- - many TTL expirations without protocol failure
936
- - SCAN on a non-trivial keyspace
937
-
938
- ---
939
-
940
- ## 23. Package Scripts
941
-
942
- Suggested package scripts:
943
-
944
- ```json
945
- {
946
- "type": "module",
947
- "scripts": {
948
- "test": "node --test",
949
- "test:unit": "node --test test/unit",
950
- "test:integration": "node --test test/integration",
951
- "test:contract": "node --test test/contract",
952
- "test:stress": "node --test test/stress",
953
- "test:all": "node --test test"
954
- }
955
- }
956
- ```
957
-
958
- ---
959
-
960
- ## 24. Acceptance Criteria for Initial Milestone
961
-
962
- The first meaningful milestone is achieved when the server can be exercised successfully through `redis-cli` and passes automated tests for the following sequence:
963
-
964
- ```text
965
- PING
966
- SET foo bar
967
- GET foo
968
- EXPIRE foo 10
969
- TTL foo
970
- DEL foo
971
- HSET user:1 name Martin age 42
972
- HGET user:1 name
973
- HGETALL user:1
974
- SADD tags a b c
975
- SMEMBERS tags
976
- TYPE foo
977
- SCAN 0
978
- ```
979
-
980
- In addition, the system must satisfy these conditions:
981
-
982
- - persistence survives restart
983
- - wrong-type errors are returned correctly
984
- - expired keys behave as missing keys
985
- - binary-safe values do not break command semantics
986
- - multiple clients can connect and issue commands without corrupting state
987
-
988
- ---
989
-
990
- ## 25. Implementation Order
991
-
992
- Recommended phased implementation order:
993
-
994
- ### Phase 1
995
-
996
- Infrastructure:
997
-
998
- - TCP server
999
- - RESP parser
1000
- - RESP encoder
1001
- - command registry
1002
- - SQLite initialization and schema creation
1003
- - error model
1004
- - basic protocol tests
1005
-
1006
- ### Phase 2
1007
-
1008
- Strings and core behavior:
1009
-
1010
- - `PING`
1011
- - `ECHO`
1012
- - `GET`
1013
- - `SET`
1014
- - `DEL`
1015
- - `EXISTS`
1016
- - `TYPE`
1017
-
1018
- ### Phase 3
1019
-
1020
- Expiration:
1021
-
1022
- - `EXPIRE`
1023
- - `PEXPIRE`
1024
- - `TTL`
1025
- - `PTTL`
1026
- - `PERSIST`
1027
- - lazy expiration
1028
- - active sweeper
1029
-
1030
- ### Phase 4
1031
-
1032
- Numeric string operations:
1033
-
1034
- - `INCR`
1035
- - `DECR`
1036
- - `INCRBY`
1037
- - `DECRBY`
1038
-
1039
- ### Phase 5
1040
-
1041
- Hashes:
1042
-
1043
- - `HSET`
1044
- - `HGET`
1045
- - `HMGET`
1046
- - `HGETALL`
1047
- - `HDEL`
1048
- - `HEXISTS`
1049
- - `HINCRBY`
1050
-
1051
- ### Phase 6
1052
-
1053
- Sets:
1054
-
1055
- - `SADD`
1056
- - `SREM`
1057
- - `SMEMBERS`
1058
- - `SISMEMBER`
1059
- - `SCARD`
1060
-
1061
- ### Phase 7
1062
-
1063
- Introspection and observability:
1064
-
1065
- - `SCAN`
1066
- - `SQLITE.INFO`
1067
- - `CACHE.INFO`
1068
-
1069
- ### Phase 8
1070
-
1071
- Migration tooling:
1072
-
1073
- - external Redis import CLI using `SCAN`
1074
- - import verification and reporting
1075
-
1076
- ### Phase 9
1077
-
1078
- Search extension:
1079
-
1080
- - `FT.CREATE`
1081
- - `FT.SEARCH`
1082
- - `FT.INFO`
1083
- - `FT.DROPINDEX`
1084
-
1085
- Search should only begin after the core command, TTL, persistence, and concurrency suites are stable.
1086
-
1087
- ---
1088
-
1089
- ## 26. Migration Strategy from Redis
1090
-
1091
- Migration is not part of the RESP protocol surface in v1.
1092
- It should be implemented as an external CLI tool.
1093
-
1094
- ### 26.1 Initial migration method
1095
-
1096
- Recommended initial approach:
1097
-
1098
- - connect to a real Redis instance
1099
- - iterate keys using `SCAN`
1100
- - discover key type with `TYPE`
1101
- - fetch values according to type
1102
- - fetch expiration using `PTTL`
1103
- - write translated data into SQLite
1104
-
1105
- ### 26.2 Initial migratable subset
1106
-
1107
- The initial import tool should support:
1108
-
1109
- - strings
1110
- - hashes
1111
- - sets
1112
- - TTL metadata
1113
-
1114
- ### 26.3 Not required initially
1115
-
1116
- - RDB parsing
1117
- - AOF parsing
1118
- - mirror mode
1119
- - dual-write migration
1120
-
1121
- These can be considered later if adoption demands them.
1122
-
1123
- ---
1124
-
1125
- ## 27. Compatibility Matrix Guidance
1126
-
1127
- The README should include a compatibility matrix with at least three groups:
1128
-
1129
- ### Supported
1130
-
1131
- All commands implemented in v1.
1132
-
1133
- ### Planned
1134
-
1135
- Future near-term commands such as:
1136
-
1137
- - Redis import CLI
1138
- - `FT.CREATE`
1139
- - `FT.SEARCH`
1140
- - `FT.INFO`
1141
- - `FT.DROPINDEX`
1142
-
1143
- ### Not Supported
1144
-
1145
- Features explicitly excluded from the roadmap for now, such as:
1146
-
1147
- - Pub/Sub
1148
- - Streams
1149
- - Lua
1150
- - Cluster
1151
- - Replication
1152
- - Blocking commands
1153
-
1154
- This matrix is essential for setting correct user expectations.
1155
-
1156
- ---
1157
-
1158
- ## 28. Final Design Rule
1159
-
1160
- A command should only be added if its implementation is:
1161
-
1162
- - natural on SQLite
1163
- - semantically clear
1164
- - testable
1165
- - operationally safe
1166
- - maintainable without excessive special handling
1167
-
1168
- If a command requires too much implementation gymnastics to preserve Redis-like behavior, it should not be part of the supported surface until a clean design exists.
1169
-
1170
- This rule is central to the project.
1171
- It protects correctness, keeps the scope honest, and preserves the identity of RESPLite as a practical Redis-compatible server built on top of SQLite rather than a fragile imitation.