redis_exchange 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/README.md +542 -0
- data/docs/architecture.md +326 -0
- data/lib/redis_exchange/depth_stream_consumer.rb +39 -0
- data/lib/redis_exchange/ledger_stream_consumer.rb +14 -0
- data/lib/redis_exchange/maintenance.rb +641 -0
- data/lib/redis_exchange/order_stream_consumer.rb +17 -0
- data/lib/redis_exchange/stream_consumer.rb +163 -0
- data/lib/redis_exchange/trade_stream_consumer.rb +17 -0
- data/lib/redis_exchange/version.rb +5 -0
- data/lib/redis_exchange.rb +1544 -0
- data/redis_scripts/audit_user_balance.lua +150 -0
- data/redis_scripts/balance_op.lua +148 -0
- data/redis_scripts/cancel_order.lua +277 -0
- data/redis_scripts/finalize_precision_update.lua +88 -0
- data/redis_scripts/force_cancel_order.lua +222 -0
- data/redis_scripts/match_order.lua +1026 -0
- data/redis_scripts/migrate_orders_batch.lua +204 -0
- data/redis_scripts/update_market_precision.lua +260 -0
- data/script/export_rdb_data.sh +234 -0
- data/script/export_snapshot.rb +370 -0
- data/script/maintenance/adjust_precision.rb +176 -0
- data/script/rename_asset.rb +314 -0
- data/script/rename_market.rb +285 -0
- metadata +149 -0
checksums.yaml
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
SHA256:
|
|
3
|
+
metadata.gz: '0980e24813156bfcaf5dde6b38f18f3f24c2cd21952f353e1a5c20abb59ddf23'
|
|
4
|
+
data.tar.gz: b558893e90dd709a75cca8216a6ba5b65485af18d12a6d6bf1ee4571b08cde52
|
|
5
|
+
SHA512:
|
|
6
|
+
metadata.gz: 14168bdedf6ad8d050b245fc4396667f7db83622e5f0becee53ba2cbcfe79be0af46d11322a52417c8c1da05651dc202cab0eee46109253c8e9af01102101470
|
|
7
|
+
data.tar.gz: d49f3793ca6300ff6f0a22cf5c3d4fa94d0b688c15d2cf354aae4b64e69304ee88af37b5f0ff5408cd61ad2c0253f107e46fa5b00e6dd560d3e293c2281dd721
|
data/README.md
ADDED
|
@@ -0,0 +1,542 @@
|
|
|
1
|
+
# Redis Exchange
|
|
2
|
+
|
|
3
|
+
Redis Exchange is a Ruby gem that bundles a Redis-backed central limit order book. All state lives in Redis hashes, sorted sets, and lists, while a Lua script performs atomic matching for both limit and market orders.
|
|
4
|
+
|
|
5
|
+
## Features
|
|
6
|
+
|
|
7
|
+
- Lua matching engine stored in `redis_scripts/match_order.lua`
|
|
8
|
+
- Unified atomic balance operations (`redis_scripts/balance_op.lua`) supporting LOCK, UNLOCK, ADD, SUB
|
|
9
|
+
- Atomic cancellation scripts (`redis_scripts/cancel_order.lua`) and bulk cancellations (`redis_scripts/cancel_all_orders.lua`)
|
|
10
|
+
- Ruby client (`RedisExchange::Client`) for seeding markets, managing balances, submitting orders, and querying tickers
|
|
11
|
+
- Comprehensive maintenance tools: balance audit, ledger replay, market precision updates
|
|
12
|
+
- Per-market ticker hash exposing last trade plus top-of-book bid/ask
|
|
13
|
+
- RSpec suite covering core behaviours (matching, balance ops, audits)
|
|
14
|
+
- Documentation of Redis data layout in [`docs/architecture.md`](docs/architecture.md)
|
|
15
|
+
|
|
16
|
+
## Installation
|
|
17
|
+
|
|
18
|
+
**Requirements:** Ruby >= 3.0 (CI verifies 3.2, 3.4 and 4.0), Redis >= 6.
|
|
19
|
+
|
|
20
|
+
Add this line to your application's `Gemfile`:
|
|
21
|
+
|
|
22
|
+
```ruby
|
|
23
|
+
gem 'redis_exchange', path: 'path/to/redis-exchange'
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
Then run `bundle install`. Alternatively, build and install locally:
|
|
27
|
+
|
|
28
|
+
```bash
|
|
29
|
+
gem build redis_exchange.gemspec
|
|
30
|
+
gem install redis_exchange-0.1.0.gem
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
### TypeScript / Node.js
|
|
34
|
+
|
|
35
|
+
A TypeScript client (`redis-exchange` on npm, same Lua scripts and Redis key contract) lives in this repository under `src/`:
|
|
36
|
+
|
|
37
|
+
```bash
|
|
38
|
+
npm install # install deps at the repo root
|
|
39
|
+
npm run build # emit dist/
|
|
40
|
+
npm test # vitest smoke suite against a disposable Redis DB
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
```ts
|
|
44
|
+
import { Client } from 'redis-exchange';
|
|
45
|
+
|
|
46
|
+
const client = new Client({ redisUrl: 'redis://127.0.0.1:6379/0' });
|
|
47
|
+
await client.seedMarket({ symbol: 'BTCUSDT', baseAsset: 'BTC', quoteAsset: 'USDT', priceScale: 100, quantityScale: 100_000_000 });
|
|
48
|
+
await client.addFunds({ userId: 'alice', asset: 'USDT', amount: 1_000_000 });
|
|
49
|
+
const result = await client.placeLimitOrder({
|
|
50
|
+
symbol: 'BTCUSDT',
|
|
51
|
+
side: 'BUY',
|
|
52
|
+
orderId: 'ord-1',
|
|
53
|
+
userId: 'alice',
|
|
54
|
+
price: 30_000 * 100,
|
|
55
|
+
quantity: 100_000_000,
|
|
56
|
+
});
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
Numeric conventions match the Ruby gem exactly: amounts are integer minor units in the asset's `balance_scale`, prices/quantities use the market's `price_scale`/`quantity_scale` multipliers, and all values must stay below 9e15 to fit Lua doubles.
|
|
60
|
+
|
|
61
|
+
Stream consumers are included as well, mirroring the Ruby gem's `StreamConsumer` base plus the order/trade/depth/ledger specializations:
|
|
62
|
+
|
|
63
|
+
```ts
|
|
64
|
+
import { Client } from 'redis-exchange';
|
|
65
|
+
|
|
66
|
+
const client = new Client({ redisUrl: 'redis://127.0.0.1:6379/0' });
|
|
67
|
+
const trades = await client.tradeConsumer({ symbol: 'BTCUSDT', group: 'trades-writer', consumer: `worker-${process.pid}` });
|
|
68
|
+
await trades.each(
|
|
69
|
+
async (entry) => {
|
|
70
|
+
console.log(entry.id, entry.fields);
|
|
71
|
+
},
|
|
72
|
+
{ autoAck: true },
|
|
73
|
+
);
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
## Usage
|
|
77
|
+
|
|
78
|
+
```ruby
|
|
79
|
+
require 'redis'
|
|
80
|
+
require 'redis_exchange'
|
|
81
|
+
|
|
82
|
+
redis = Redis.new(host: '127.0.0.1', port: 6379)
|
|
83
|
+
exchange = RedisExchange::Client.new(redis: redis)
|
|
84
|
+
|
|
85
|
+
exchange.seed_market(
|
|
86
|
+
symbol: 'BTCUSDT',
|
|
87
|
+
base_asset: 'BTC',
|
|
88
|
+
quote_asset: 'USDT',
|
|
89
|
+
price_scale: 100,
|
|
90
|
+
quantity_scale: 100_000_000,
|
|
91
|
+
maker_fee_bps: 5,
|
|
92
|
+
taker_fee_bps: 10,
|
|
93
|
+
min_qty: 100_000,
|
|
94
|
+
min_notional: 1_000
|
|
95
|
+
)
|
|
96
|
+
|
|
97
|
+
exchange.set_balance(user_id: 'alice', asset: 'USDT', available: 1_000_000)
|
|
98
|
+
exchange.set_balance(user_id: 'bob', asset: 'BTC', available: 100_000_000)
|
|
99
|
+
|
|
100
|
+
# Or use the new atomic balance operations:
|
|
101
|
+
exchange.operate_balance(user_id: 'alice', asset: 'USDT', action: 'ADD', amount: 500_000)
|
|
102
|
+
exchange.lock_funds(user_id: 'bob', asset: 'BTC', amount: 10_000_000)
|
|
103
|
+
exchange.unlock_funds(user_id: 'bob', asset: 'BTC', amount: 5_000_000)
|
|
104
|
+
|
|
105
|
+
# Bob posts a sell limit order.
|
|
106
|
+
exchange.place_limit_order(
|
|
107
|
+
symbol: 'BTCUSDT',
|
|
108
|
+
side: 'SELL',
|
|
109
|
+
order_id: 'sell-1',
|
|
110
|
+
user_id: 'bob',
|
|
111
|
+
price: 3_000_000,
|
|
112
|
+
quantity: 100_000_000
|
|
113
|
+
)
|
|
114
|
+
|
|
115
|
+
# Later Bob can cancel the resting order and release his BTC:
|
|
116
|
+
exchange.cancel_order('sell-1')
|
|
117
|
+
|
|
118
|
+
# Cancel every open order for Bob in one shot:
|
|
119
|
+
cancellations = exchange.cancel_all_orders('bob')
|
|
120
|
+
# => array of hashes describing each cancelled order
|
|
121
|
+
|
|
122
|
+
# Inspect the current order book snapshot (bids and asks arrays):
|
|
123
|
+
book = exchange.order_book('BTCUSDT', limit: 5)
|
|
124
|
+
pp book
|
|
125
|
+
|
|
126
|
+
# Alice crosses the book with a market buy capped at 31,000 USDT.
|
|
127
|
+
response = exchange.place_market_order(
|
|
128
|
+
symbol: 'BTCUSDT',
|
|
129
|
+
side: 'BUY',
|
|
130
|
+
order_id: 'buy-1',
|
|
131
|
+
user_id: 'alice',
|
|
132
|
+
quantity: 50_000_000,
|
|
133
|
+
price_cap: 3_100_000
|
|
134
|
+
)
|
|
135
|
+
|
|
136
|
+
puts response # JSON parsed into a Ruby hash
|
|
137
|
+
|
|
138
|
+
# Retrieve ticker information (last trade, best bid/ask).
|
|
139
|
+
p exchange.ticker('BTCUSDT')
|
|
140
|
+
|
|
141
|
+
# Retrieve an order snapshot by id with typed fields
|
|
142
|
+
pp exchange.order('sell-1')
|
|
143
|
+
|
|
144
|
+
# Monitor real-time order book changes
|
|
145
|
+
# Note: This method blocks execution. Run in a separate thread.
|
|
146
|
+
exchange.monitor_order_book(symbol: 'BTCUSDT') do |event, side|
|
|
147
|
+
# event: 'zadd' (new order/price level), 'zrem' (level removed), etc.
|
|
148
|
+
# side: 'BUY' or 'SELL'
|
|
149
|
+
puts "Book update: #{side} #{event}"
|
|
150
|
+
end
|
|
151
|
+
```
|
|
152
|
+
|
|
153
|
+
### Auto-connect without providing `redis`
|
|
154
|
+
|
|
155
|
+
You can omit the `redis` parameter and the client will auto-connect using `ENV['REDIS_URL']` or fall back to `redis://127.0.0.1:6379/0`.
|
|
156
|
+
|
|
157
|
+
```ruby
|
|
158
|
+
# Uses ENV['REDIS_URL'] if set; otherwise connects to local Redis db 0
|
|
159
|
+
exchange = RedisExchange::Client.new
|
|
160
|
+
```
|
|
161
|
+
|
|
162
|
+
## Consuming Trade Stream
|
|
163
|
+
|
|
164
|
+
Each market emits executed trades to a Redis Stream `trades:{symbol}` inside the Lua script. You can use the built-in consumer helper to process and persist trades reliably:
|
|
165
|
+
|
|
166
|
+
```ruby
|
|
167
|
+
consumer = exchange.trade_consumer(symbol: 'BTCUSDT', group: 'trades-consumers', consumer: 'worker-1')
|
|
168
|
+
|
|
169
|
+
# Simple blocking loop with explicit read + ack
|
|
170
|
+
loop do
|
|
171
|
+
entries = consumer.read(count: 100, block_ms: 2000)
|
|
172
|
+
entries.each do |e|
|
|
173
|
+
# e.id is the Redis Stream entry id (e.g., "1707389234567-0")
|
|
174
|
+
fields = e.fields # Hash of strings: { 'taker_order_id' => '...', 'price' => '3000000', ... }
|
|
175
|
+
|
|
176
|
+
# Persist with idempotency: use e.id as unique key in your DB
|
|
177
|
+
# Convert numeric fields to integers before saving
|
|
178
|
+
# On success:
|
|
179
|
+
consumer.ack(e.id)
|
|
180
|
+
end
|
|
181
|
+
end
|
|
182
|
+
|
|
183
|
+
# Or use the convenience each to drive the loop
|
|
184
|
+
consumer.each(count: 100, block_ms: 2000, auto_ack: true) do |e|
|
|
185
|
+
# fields = e.fields
|
|
186
|
+
# Persist with idempotency and type conversions as needed
|
|
187
|
+
# Return :halt to break the loop if desired
|
|
188
|
+
# return :halt if shutdown?
|
|
189
|
+
end
|
|
190
|
+
```
|
|
191
|
+
|
|
192
|
+
Recommended persistence design:
|
|
193
|
+
- Use `stream_entry_id` unique constraint for idempotency
|
|
194
|
+
- Store scaled integer fields as `bigint` (price, quantity, quote, fees)
|
|
195
|
+
- Save `executed_ts_ms` from `fields['ts']` as bigint
|
|
196
|
+
|
|
197
|
+
`ticker` returns a hash of integers keyed by `:last_price`, `:last_quantity`, `:best_bid_price`, `:best_bid_volume`, `:best_ask_price`, `:best_ask_volume`, and more for quick UI snapshots.
|
|
198
|
+
|
|
199
|
+
## Consuming Order Stream
|
|
200
|
+
|
|
201
|
+
Each market emits order lifecycle events to `orders:{symbol}` (OPEN, PARTIALLY_FILLED, FILLED, CANCELLED, NO_LIQUIDITY). Use the built-in helper to consume reliably:
|
|
202
|
+
|
|
203
|
+
```ruby
|
|
204
|
+
consumer = exchange.order_consumer(symbol: 'BTCUSDT', group: 'orders-consumers', consumer: 'worker-1')
|
|
205
|
+
|
|
206
|
+
# Explicit read + ack style
|
|
207
|
+
loop do
|
|
208
|
+
entries = consumer.read(count: 100, block_ms: 2000)
|
|
209
|
+
entries.each do |e|
|
|
210
|
+
fields = e.fields # { 'order_id' => '...', 'status' => 'CANCELLED', 'price' => '3000000', ... }
|
|
211
|
+
|
|
212
|
+
# Persist with idempotency: use e.id as unique key in your DB
|
|
213
|
+
# Convert numeric fields to integers before saving
|
|
214
|
+
consumer.ack(e.id)
|
|
215
|
+
end
|
|
216
|
+
end
|
|
217
|
+
|
|
218
|
+
# Or use the convenience each to drive the loop
|
|
219
|
+
consumer.each(count: 100, block_ms: 2000, auto_ack: true) do |e|
|
|
220
|
+
# fields = e.fields
|
|
221
|
+
# Persist then optional halt
|
|
222
|
+
# return :halt if shutdown?
|
|
223
|
+
end
|
|
224
|
+
```
|
|
225
|
+
|
|
226
|
+
See `script/orders_demo.rb` for a runnable example.
|
|
227
|
+
|
|
228
|
+
## Consuming Account Ledger
|
|
229
|
+
|
|
230
|
+
Append-only account changes are published to the `account_ledger` Redis Stream. Use the built-in consumer to reliably process changes with consumer groups and idempotent persistence:
|
|
231
|
+
|
|
232
|
+
```ruby
|
|
233
|
+
consumer = exchange.ledger_consumer(group: 'ledger-consumers', consumer: 'worker-1')
|
|
234
|
+
|
|
235
|
+
# Explicit read + ack style
|
|
236
|
+
loop do
|
|
237
|
+
entries = consumer.read(count: 100, block_ms: 2000)
|
|
238
|
+
entries.each do |e|
|
|
239
|
+
fields = e.fields # { 'user_id' => 'u1', 'asset' => 'USDT', 'action' => 'ADD', 'amount' => '1000', 'reason' => 'DEPOSIT', 'ts' => '...', 'sequence' => '1', 'biz_id' => '...' }
|
|
240
|
+
|
|
241
|
+
# Persist with idempotency: use e.id as unique key in your DB
|
|
242
|
+
# Convert numeric fields to integers before saving
|
|
243
|
+
consumer.ack(e.id)
|
|
244
|
+
end
|
|
245
|
+
end
|
|
246
|
+
|
|
247
|
+
# Or use the convenience each to drive the loop
|
|
248
|
+
consumer.each(count: 100, block_ms: 2000, auto_ack: true) do |e|
|
|
249
|
+
# fields = e.fields
|
|
250
|
+
# Persist then optional halt
|
|
251
|
+
# return :halt if shutdown?
|
|
252
|
+
end
|
|
253
|
+
```
|
|
254
|
+
|
|
255
|
+
Fields of each ledger entry:
|
|
256
|
+
- `user_id` (String)
|
|
257
|
+
- `asset` (String)
|
|
258
|
+
- `action` (String: `LOCK`, `UNLOCK`, `ADD`, `SUB`, `SUB_LOCKED`, `ADD_LOCKED`)
|
|
259
|
+
- `amount` (String integer; scaled; always positive)
|
|
260
|
+
- `reason` (String)
|
|
261
|
+
- `ts` (String integer; milliseconds)
|
|
262
|
+
- `sequence` (String integer; per user+asset incremental)
|
|
263
|
+
- `biz_id` (String; idempotency key; auto-generated if not provided)
|
|
264
|
+
|
|
265
|
+
## Development
|
|
266
|
+
|
|
267
|
+
Ensure Redis 6+ is running locally. Install gem dependencies and run specs:
|
|
268
|
+
|
|
269
|
+
```bash
|
|
270
|
+
bundle install
|
|
271
|
+
bundle exec rspec
|
|
272
|
+
```
|
|
273
|
+
|
|
274
|
+
## Account Ledger & Balance Helpers
|
|
275
|
+
|
|
276
|
+
Convenience helpers are provided for managing funds and direct balance operations. All changes are executed atomically via Lua and emit events to the `account_ledger` stream.
|
|
277
|
+
|
|
278
|
+
```ruby
|
|
279
|
+
exchange = RedisExchange::Client.new(redis: Redis.new)
|
|
280
|
+
|
|
281
|
+
# Add 100 USDT (scaled integer) - internally calls operate_balance with action=ADD
|
|
282
|
+
exchange.add_funds(user_id: 'u1', asset: 'USDT', amount: 100_000)
|
|
283
|
+
|
|
284
|
+
# Subtract 50 USDT - internally calls operate_balance with action=SUB
|
|
285
|
+
exchange.sub_funds(user_id: 'u1', asset: 'USDT', amount: 50_000)
|
|
286
|
+
|
|
287
|
+
# Subtract 10 USDT from locked (reserved) balance - internally calls operate_balance with action=SUB_LOCKED
|
|
288
|
+
exchange.sub_locked_funds(user_id: 'u1', asset: 'USDT', amount: 10_000)
|
|
289
|
+
|
|
290
|
+
# Lock funds (move from available to reserved)
|
|
291
|
+
exchange.lock_funds(user_id: 'u1', asset: 'USDT', amount: 10_000, reason: 'ORDER_RESERVE')
|
|
292
|
+
|
|
293
|
+
# Unlock funds (move from reserved to available)
|
|
294
|
+
exchange.unlock_funds(user_id: 'u1', asset: 'USDT', amount: 5_000, reason: 'CANCEL_REFUND')
|
|
295
|
+
```
|
|
296
|
+
|
|
297
|
+
Each ledger entry includes: `user_id`, `asset`, `action`, `amount`, `reason`, `ts`, `sequence`, `biz_id`.
|
|
298
|
+
|
|
299
|
+
Idempotency: `operate_balance` (and helpers like `add_funds`/`sub_funds`) accepts an optional `biz_id` to deduplicate requests. When provided, the Redis Lua script returns a cached response for the same `{user_id, asset, action, biz_id}` within 24 hours and does not append a duplicate ledger entry or re-apply the balance mutation.
|
|
300
|
+
|
|
301
|
+
```ruby
|
|
302
|
+
# Using biz_id to prevent duplicate submissions
|
|
303
|
+
biz_id = 'req-123'
|
|
304
|
+
exchange.operate_balance(user_id: 'u1', asset: 'USDT', action: 'ADD', amount: 50_000, reason: 'MANUAL_ADJUST', biz_id: biz_id)
|
|
305
|
+
# A repeated call with the same biz_id returns the same response and does not duplicate effects
|
|
306
|
+
exchange.operate_balance(user_id: 'u1', asset: 'USDT', action: 'ADD', amount: 50_000, reason: 'MANUAL_ADJUST', biz_id: biz_id)
|
|
307
|
+
```
|
|
308
|
+
|
|
309
|
+
If `biz_id` is omitted, the client auto-generates a UUID and includes it in the response as `:biz_id`; the ledger entry will also carry this generated `biz_id`. Internally generated events (e.g., matching/cancellation) attach an auto `biz_id` as well.
|
|
310
|
+
|
|
311
|
+
## Random Bot (Stress Test)
|
|
312
|
+
|
|
313
|
+
A random trading bot is included for stress testing and verification. It performs random balance operations (add/sub), limit orders, and market orders while validating ledger consistency.
|
|
314
|
+
|
|
315
|
+
```bash
|
|
316
|
+
# Run with default settings (100 iterations, 1 concurrent thread)
|
|
317
|
+
ruby script/random_bot.rb
|
|
318
|
+
|
|
319
|
+
# Run heavily concurrent load (5 threads, 1000 iterations each) with final audit
|
|
320
|
+
ruby script/random_bot.rb --concurrency 5 --iterations 1000 --audit
|
|
321
|
+
|
|
322
|
+
# CI shape: dense ops without sleeps, quiet logs; exits non-zero if any audit fails
|
|
323
|
+
ruby script/random_bot.rb --flush --concurrency 5 --iterations 1000 \
|
|
324
|
+
--sleep-min 0 --sleep-max 0 --quiet --audit
|
|
325
|
+
|
|
326
|
+
# Options:
|
|
327
|
+
# --redis-url URL Redis connection string
|
|
328
|
+
# --symbol SYMBOL Market symbol (default: BTCUSDT)
|
|
329
|
+
# --iterations N Number of operations per thread
|
|
330
|
+
# --concurrency N Number of concurrent worker threads
|
|
331
|
+
# --audit Run full balance audit and ledger replay check after completion;
|
|
332
|
+
# exits with status 1 if any check fails
|
|
333
|
+
# --audit-interval N (Single-thread only) Run partial replay audit every N operations
|
|
334
|
+
# --quiet Suppress per-operation output (INIT/AUDIT/ERROR only)
|
|
335
|
+
# --no-seed Skip initial asset/market seeding
|
|
336
|
+
# --flush Flush DB before starting
|
|
337
|
+
```
|
|
338
|
+
|
|
339
|
+
## Benchmarking
|
|
340
|
+
|
|
341
|
+
Two scripts are provided. Both need a real Redis instance and **flush the target database** on start; point them at a throwaway instance (`docker run -d -p 6390:6379 redis:8-alpine redis-server --save '' --appendonly no`).
|
|
342
|
+
|
|
343
|
+
### Full suite — `script/benchmark_suite.rb`
|
|
344
|
+
|
|
345
|
+
Covers the paths a real deployment exercises: book building across price levels, crossing limit orders, market orders that walk N levels, single and bulk cancellation, balance lock/unlock, book/ticker reads, a realistic mixed workload, and multi-client contention. Every scenario reports p50/p90/p99/max latency, client throughput, error codes, and the Redis command count per operation.
|
|
346
|
+
|
|
347
|
+
```bash
|
|
348
|
+
ruby script/benchmark_suite.rb --redis-url=redis://127.0.0.1:6390/0 \
|
|
349
|
+
--depth=10000 --batch=2000 --levels=100 --sweep-levels=5 --sweep-ops=100 \
|
|
350
|
+
--duration=30 --clients=4 --json=tmp/bench.json
|
|
351
|
+
```
|
|
352
|
+
|
|
353
|
+
Run `--only=book_build,concurrent` to select scenarios, or `--help` for all flags. The script also seeds `asset:*` keys — without them every order is rejected with `ASSET_NOT_CONFIGURED` before the Lua matcher runs, and a benchmark would silently measure the rejection path.
|
|
354
|
+
|
|
355
|
+
Latest measured results and analysis: [`docs/benchmarks/2026-10-23-benchmark-suite.md`](docs/benchmarks/2026-10-23-benchmark-suite.md).
|
|
356
|
+
|
|
357
|
+
### Single-scenario smoke test — `script/benchmark_matching.rb`
|
|
358
|
+
|
|
359
|
+
A minimal throughput probe: seeds a sell book at one price and fires BUY market orders at it.
|
|
360
|
+
|
|
361
|
+
```bash
|
|
362
|
+
BENCH_REDIS_URL=redis://127.0.0.1:6379/15 \
|
|
363
|
+
ruby script/benchmark_matching.rb --resting=10_000 --taker=10_000 --price=3_000_000
|
|
364
|
+
```
|
|
365
|
+
|
|
366
|
+
It prints total time and computed orders/sec, but it does **not** seed `asset:*` and does not check results, so on its own it measures the pre-flight rejection path rather than matching — use the full suite for numbers you intend to quote. Tune `--price`, `--quantity`, or `--symbol` to match your production scales.
|
|
367
|
+
|
|
368
|
+
### Performance notes
|
|
369
|
+
|
|
370
|
+
Orders do not re-read market/asset/user-fee configuration on every call: the client caches that bundle for `DEFAULT_CONFIG_CACHE_TTL` seconds (5 by default) and every mutator (`seed_market`, `set_market_status`, `set_market_fees`, `seed_asset`, `set_user_fees`, `delete_user_fees`) invalidates it. Set the TTL to `0` to disable caching, or call `clear_config_cache!` (Ruby) / `clearConfigCache()` (TypeScript) after changing configuration outside the client. Market status is always read by the Lua script, so a `HALTED` market rejects orders immediately regardless of the TTL.
|
|
371
|
+
|
|
372
|
+
Measured effect: placement +81%, matched takers +69%, `order_book` reads +300% (42 → 4 commands per snapshot). Details and methodology in [`docs/benchmarks/2026-10-23-order-path-optimization.md`](docs/benchmarks/2026-10-23-order-path-optimization.md).
|
|
373
|
+
|
|
374
|
+
## Numeric Precision
|
|
375
|
+
|
|
376
|
+
Redis executes Lua with 64-bit floating point numbers. Keep scaled prices, quantities, and their product (`price * quantity`) below roughly 9e15 to avoid rounding in `math.floor` and fee calculations. Adjust `price_scale` and `quantity_scale` to respect this bound.
|
|
377
|
+
|
|
378
|
+
## Lua Script Contract
|
|
379
|
+
|
|
380
|
+
`match_order.lua` expects eight keys (market metadata, sequence counter, book/volume hashes, taker balance hash, and the trade stream) and eight arguments (`symbol`, `side`, `order_type`, `order_id`, `user_id`, `price_limit`, `quantity`, `timestamp_ms`). `price_limit` represents the order price for limit orders or an optional cap (0 means uncapped) for market orders. The script returns a JSON blob describing execution outcome and taker cash flows; validation failures return an object with an `error` field.
|
|
381
|
+
|
|
382
|
+
## Testing Notes
|
|
383
|
+
|
|
384
|
+
- Extend the RSpec suite with additional edge cases (partial fills, insufficient balance branches, multi-level sweeps).
|
|
385
|
+
- Consider using `fakeredis` for unit tests that do not require a live server.
|
|
386
|
+
- For integration coverage, run concurrent clients to validate atomic behaviour under contention.
|
|
387
|
+
|
|
388
|
+
## Asset & Market Configuration
|
|
389
|
+
|
|
390
|
+
Before placing orders, configure asset precision and market parameters. Balances and ledger entries use per-asset minor units (`balance_scale`), while matching uses per-market scales (`price_scale`, `quantity_scale`).
|
|
391
|
+
|
|
392
|
+
```ruby
|
|
393
|
+
exchange = RedisExchange::Client.new(redis: Redis.new)
|
|
394
|
+
|
|
395
|
+
# Configure assets (balance scales and status)
|
|
396
|
+
exchange.seed_asset(asset: 'BTC', balance_scale: 100_000_000, status: 'ACTIVE')
|
|
397
|
+
exchange.seed_asset(asset: 'USDT', balance_scale: 100, status: 'ACTIVE')
|
|
398
|
+
|
|
399
|
+
# Configure market (price/quantity scales, fees, thresholds, and status)
|
|
400
|
+
exchange.seed_market(
|
|
401
|
+
symbol: 'BTCUSDT',
|
|
402
|
+
base_asset: 'BTC',
|
|
403
|
+
quote_asset: 'USDT',
|
|
404
|
+
price_scale: 100,
|
|
405
|
+
quantity_scale: 100_000_000,
|
|
406
|
+
maker_fee_bps: 5,
|
|
407
|
+
taker_fee_bps: 10,
|
|
408
|
+
min_qty: 100_000,
|
|
409
|
+
min_notional: 1_000,
|
|
410
|
+
status: 'OPEN' # OPEN | HALTED | CLOSED | MAINTENANCE
|
|
411
|
+
)
|
|
412
|
+
```
|
|
413
|
+
|
|
414
|
+
### Scale Compatibility Rules
|
|
415
|
+
|
|
416
|
+
To guarantee integer conversions without rounding:
|
|
417
|
+
- Base asset balance scale must be a multiple of `quantity_scale`.
|
|
418
|
+
- Quote asset balance scale must be a multiple of `price_scale`.
|
|
419
|
+
|
|
420
|
+
If violated, order placement returns:
|
|
421
|
+
- `ASSET_NOT_CONFIGURED` when the asset configuration is missing.
|
|
422
|
+
- `ASSET_SCALE_MISMATCH` when the asset balance scale does not evenly divide the market scale. Returned fields include `asset`, `balance_scale`, and `market_scale`.
|
|
423
|
+
|
|
424
|
+
### Market Status Semantics
|
|
425
|
+
|
|
426
|
+
- `OPEN`: normal operation; order placement and cancellation allowed.
|
|
427
|
+
- `HALTED`: new limit/market orders are rejected with `MARKET_HALTED`; cancellations are **allowed**.
|
|
428
|
+
- `CLOSED`: new limit/market orders are rejected with `MARKET_CLOSED`; cancellations are **allowed**.
|
|
429
|
+
- `MAINTENANCE`: all operations (including cancellation) are rejected with `MARKET_MAINTENANCE`. Used for atomic operations like snapshotting or precision updates.
|
|
430
|
+
|
|
431
|
+
### Force Cancellation (Maintenance Mode)
|
|
432
|
+
|
|
433
|
+
During maintenance (or for emergency cleanup), `force_cancel_order` can be used to remove orders regardless of market status.
|
|
434
|
+
|
|
435
|
+
```ruby
|
|
436
|
+
# Mode 1: Delete only (no balance changes)
|
|
437
|
+
exchange.maintenance.force_cancel_order(order_id: 'o-1', mode: :delete_only)
|
|
438
|
+
|
|
439
|
+
# Mode 2: Best-effort unlock (refunds reserved balance if available)
|
|
440
|
+
exchange.maintenance.force_cancel_order(order_id: 'o-1', mode: :best_effort_unlock)
|
|
441
|
+
```
|
|
442
|
+
|
|
443
|
+
### Balance Audit
|
|
444
|
+
|
|
445
|
+
To prevent accounting errors, you can audit and reconcile user reserved balances against active orders.
|
|
446
|
+
|
|
447
|
+
```bash
|
|
448
|
+
# Audit all users and automatically fix discrepancies (release surplus / lock deficit)
|
|
449
|
+
bundle exec rake audit:balances
|
|
450
|
+
|
|
451
|
+
# Dry-run mode (report only, no changes)
|
|
452
|
+
bundle exec rake audit:balances FIX=false
|
|
453
|
+
|
|
454
|
+
# Audit a specific user
|
|
455
|
+
bundle exec rake audit:balances USER_ID=user-123
|
|
456
|
+
```
|
|
457
|
+
|
|
458
|
+
The audit process:
|
|
459
|
+
1. Scans all active orders for the user(s).
|
|
460
|
+
2. Calculates the required reserved balance for each asset.
|
|
461
|
+
3. Compares with the actual `reserved` balance in Redis.
|
|
462
|
+
4. If `Required > Actual`: Locks available funds to cover the deficit. If available funds are insufficient, reports a deficit.
|
|
463
|
+
5. If `Required < Actual`: Releases the surplus reserved funds back to available.
|
|
464
|
+
6. Emits ledger entries for all adjustments (`AUDIT_FIX_LOCK` / `AUDIT_FIX_RELEASE`).
|
|
465
|
+
|
|
466
|
+
Example rejection when market is halted:
|
|
467
|
+
```ruby
|
|
468
|
+
exchange.seed_market(
|
|
469
|
+
symbol: 'BTCUSDT', base_asset: 'BTC', quote_asset: 'USDT',
|
|
470
|
+
price_scale: 100, quantity_scale: 100_000_000,
|
|
471
|
+
status: 'HALTED'
|
|
472
|
+
)
|
|
473
|
+
|
|
474
|
+
res = exchange.place_limit_order(
|
|
475
|
+
symbol: 'BTCUSDT', side: 'SELL', order_id: 'o-1', user_id: 'bob',
|
|
476
|
+
price: 3_000_000, quantity: 100_000_000
|
|
477
|
+
)
|
|
478
|
+
# => { error: 'MARKET_HALTED' }
|
|
479
|
+
```
|
|
480
|
+
|
|
481
|
+
## Error Codes
|
|
482
|
+
|
|
483
|
+
All error responses are returned as a Hash with an `error` key:
|
|
484
|
+
|
|
485
|
+
```ruby
|
|
486
|
+
{ error: 'ERROR_CODE' }
|
|
487
|
+
```
|
|
488
|
+
|
|
489
|
+
### System/Market Status Errors
|
|
490
|
+
|
|
491
|
+
| Error Code | Description |
|
|
492
|
+
|-----------|-------------|
|
|
493
|
+
| `SYSTEM_MAINTENANCE` | System-wide maintenance mode enabled |
|
|
494
|
+
| `MARKET_MAINTENANCE` | Market is in maintenance mode |
|
|
495
|
+
| `MARKET_HALTED` | Market is halted (orders rejected, cancellations allowed) |
|
|
496
|
+
| `MARKET_CLOSED` | Market is closed (orders rejected, cancellations allowed) |
|
|
497
|
+
|
|
498
|
+
### Argument Validation Errors
|
|
499
|
+
|
|
500
|
+
| Error Code | Description |
|
|
501
|
+
|-----------|-------------|
|
|
502
|
+
| `INVALID_ARGUMENTS` | Invalid number or type of arguments provided |
|
|
503
|
+
| `INVALID_ORDER_TYPE` | Invalid order type (must be LIMIT or MARKET) |
|
|
504
|
+
| `INVALID_SIDE` | Invalid order side (must be BUY or SELL) |
|
|
505
|
+
| `INVALID_PRICE_OR_QUANTITY` | Invalid price or quantity value |
|
|
506
|
+
| `INVALID_AMOUNT` | Invalid amount for balance operation |
|
|
507
|
+
| `INVALID_ACTION` | Invalid balance action type |
|
|
508
|
+
| `INVALID_ORDER` | Invalid order state or data |
|
|
509
|
+
|
|
510
|
+
### Order Errors
|
|
511
|
+
|
|
512
|
+
| Error Code | Description |
|
|
513
|
+
|-----------|-------------|
|
|
514
|
+
| `DUPLICATE_ORDER_ID` | Order ID already exists |
|
|
515
|
+
| `UNKNOWN_MARKET` | Market configuration not found |
|
|
516
|
+
|
|
517
|
+
### Asset Errors
|
|
518
|
+
|
|
519
|
+
| Error Code | Description |
|
|
520
|
+
|-----------|-------------|
|
|
521
|
+
| `ASSET_NOT_CONFIGURED` | Asset configuration not found |
|
|
522
|
+
| `ASSET_SCALE_MISMATCH` | Asset balance scale incompatible with market configuration |
|
|
523
|
+
| `ASSET_SCALE_MISMATCH_WITH_NEW_CONFIG` | Asset scale mismatch during precision update |
|
|
524
|
+
|
|
525
|
+
### Trading Rule Errors
|
|
526
|
+
|
|
527
|
+
| Error Code | Description |
|
|
528
|
+
|-----------|-------------|
|
|
529
|
+
| `MIN_QTY_NOT_SATISFIED` | Order quantity below minimum required |
|
|
530
|
+
| `MIN_NOTIONAL_NOT_SATISFIED` | Order notional value below minimum required |
|
|
531
|
+
| `PRECISION_OVERFLOW` | Precision overflow in calculation |
|
|
532
|
+
|
|
533
|
+
### Balance Errors
|
|
534
|
+
|
|
535
|
+
| Error Code | Description | Additional Fields |
|
|
536
|
+
|-----------|-------------|-------------------|
|
|
537
|
+
| `INSUFFICIENT_FUNDS` | Insufficient funds for operation | `asset`, `bucket` |
|
|
538
|
+
| `INSUFFICIENT_FUNDS_TO_REST` | Insufficient funds to place order | `asset` |
|
|
539
|
+
|
|
540
|
+
## License
|
|
541
|
+
|
|
542
|
+
MIT
|