omq-cli 0.13.0 → 0.14.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 0135d1ab507a4e82186cae115a477ccbd4b231e12a7450cb8989a5c0c1479e50
4
- data.tar.gz: efc832159dc103ec4b18ba5f65acb5976a09c7530b246e5e433795357ca21181
3
+ metadata.gz: 5352d1d8bdbab1b64aa1e7a36afcc1c70619eb0ae8373d78e95c77669d6a4fec
4
+ data.tar.gz: 38172d3bbff810e3be322e3b8aa4e6d9e09520976d76b34e6721606dd8a15c95
5
5
  SHA512:
6
- metadata.gz: a8093420e5209f9e4672dd15a343c60167959532d90fda0fce7f108042807a65d27f033eb39a3abfee80059e8832d2e90680a624f36575c8b608f2da5095865c
7
- data.tar.gz: 30dc24c50221b726130e638764d891d2736044b358b364f2bd987a21eceac85bea44ed0821ab601d1070d2e60bc10492a61ffd8baa8129588a0dce06b72ec192
6
+ metadata.gz: 158eb775b0ccea84275a69dc39e8279f41fab5f9bfeb94e8ca0c9e5dd7342b6472980e925850b7d82eceb509981942c42cd80abcb52a9e871b335075898a05b6
7
+ data.tar.gz: 8885731e2d99e87038bb6cffe9f4dc7edd480d285d4b2ac5b5f2244772433f70123e1ed49e9da7a9aa55944aab938bbbf10d7acc98975a04df6b45fd0a1d6b7c
data/CHANGELOG.md CHANGED
@@ -1,5 +1,123 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.14.1 — 2026-04-13
4
+
5
+ ### Changed
6
+
7
+ - **`-M` (Marshal) now carries raw Ruby objects, not array-wrapped
8
+ frames.** Under `-M`, each wire frame is one Marshal-dumped Ruby
9
+ object; inside `-e` / `-E`, `it` is that object directly (not
10
+ `[object]`). Enables natural scalar/hash/custom-class flows:
11
+
12
+ ```sh
13
+ omq push -b tcp://:5557 -ME '"foo"'
14
+ omq pull -c tcp://:5557 -M -e '{it => it.encoding}'
15
+ # => {"foo" => #<Encoding:UTF-8>}
16
+ ```
17
+
18
+ The previous one-element-Array wrap was cosmetic — it always
19
+ produced exactly one wire frame anyway — so no multipart
20
+ semantics are lost.
21
+
22
+ ### Fixed
23
+
24
+ - **`-vvv` trace lines now precede stdout side-effects from
25
+ `-e` / `-E`.** A `<<` / `>>` line is emitted from the app fiber
26
+ *before* the eval expression runs, so sequences like
27
+ `-e 'p it'` read strictly as `trace → eval output → body` on a
28
+ shared tty. Previous design emitted traces from the monitor
29
+ fiber and raced with stdout.
30
+ - **`-vvv` under `-M` now shows the app-level object, not wire
31
+ bytes.** Preview header switches to `(marshal) <inspect>` with
32
+ sanitization and 60-byte truncation, e.g.
33
+ `<< (marshal) [nil, :foo, "bar"]`.
34
+ - **`-vvv` trace preview sanitizes control characters.** Tabs,
35
+ newlines, CR, and backslash render as `\t`, `\n`, `\r`, `\\`;
36
+ other non-printables collapse to `.`. Previously raw LF inside
37
+ a binary frame could leak and break the single-line guarantee.
38
+ - Test suite runs cleanly without protocol-error stderr noise.
39
+
40
+ ## 0.14.0 — 2026-04-13
41
+
42
+ ### Added
43
+
44
+ - **Receive-capable sockets decompress by default.** All socket types
45
+ except pure senders (`push`, `pub`, `scatter`, `radio`) now advertise
46
+ the ZMTP-Zstd profile in **passive mode** at startup, so they accept
47
+ compressed frames from any active-sender peer without requiring
48
+ `-z` on the receive side. They never compress their own outgoing
49
+ frames in this mode — use `-z` / `-Z` / `--compress=LEVEL` on the
50
+ sender to opt it in. A `push` piped into a `pull` with no flags on
51
+ either side stays uncompressed; `omq push -z | omq pull` compresses
52
+ on the wire and the pull side decodes transparently. This is the
53
+ RFC Sec. 6.4 "Passive senders" mode; requires omq-rfc-zstd >= 0.1.0.
54
+ - **`-Z` flag for better-ratio compression (zstd level 3).** `-z`
55
+ remains the fast default (level -3) and `--compress=LEVEL` takes
56
+ a custom zstd level (e.g. `--compress=19`, `--compress=-1`). Short
57
+ bundling (`-zvvv`, `-Zvvv`) still works.
58
+ - **`-vvv` logs `ZDICT` exchange.** When the auto-trained dictionary
59
+ is shipped/received, the trace prints `>> ZDICT (NB)` on the sender
60
+ and `<< ZDICT (NB)` on the receiver.
61
+ - **`-vvv` wire-size annotation for compressed traces.** Message
62
+ previews on compressed sockets include the post-compression byte
63
+ count: `(280B wire=29B) ZZ…`. Plumbed from the ZMTP-Zstd wrapper
64
+ through the engine's verbose monitor.
65
+
66
+ ### Changed
67
+
68
+ - **Compression backend switched from `rlz4` to `omq-rfc-zstd`.**
69
+ Compression is now a ZMTP wire-protocol extension negotiated via
70
+ the `X-Compression` READY property and applied below the
71
+ application API. Auto-trained dictionaries are shipped over a
72
+ `ZDICT` command frame once the sender has enough samples. The
73
+ `Formatter` no longer compresses or decompresses anything — it
74
+ only encodes/decodes wire formats. Pipe `-z` is no longer modal
75
+ (`compress_in`/`compress_out` removed) since compression is a
76
+ per-socket, send-side property negotiated with each peer.
77
+ - **`-vvv` output ordering under compression.** At `-vvv`, the
78
+ monitor fiber now writes both the trace line and the plaintext
79
+ body, so trace-and-body pairs land on the tty in order instead of
80
+ interleaving between the recv pump and the app fiber.
81
+ - **TCP host normalization moved into `OMQ::Transport::TCP`.** `omq`
82
+ v0.19.0 now handles `tcp://*:PORT`, `tcp://:PORT`, and
83
+ `tcp://localhost:PORT` natively (including dual-stack `*` binding
84
+ both IPv4 and IPv6 wildcards), so `CliParser` no longer rewrites
85
+ these URLs before handing them off. Removed
86
+ `CliParser.loopback_bind_host` and the `normalize_bind`/
87
+ `normalize_connect`/`normalize_ep` block. Requires `omq ~> 0.19`.
88
+ - **Terminate on protocol errors instead of silent reconnect.** When
89
+ a peer sends a frame that violates the ZMTP wire protocol
90
+ (oversized, bad framing, zstd bytebomb, nonce exhaustion, …), the
91
+ library drops that one connection and reconnects — the libzmq
92
+ parity behavior. The CLI is a different audience: a persistent
93
+ protocol violation is almost always a misconfiguration the user
94
+ needs to see, not silently paper over. Every runner
95
+ (`BaseRunner`, `PipeRunner`, `ParallelWorker`, `PipeWorker`) now
96
+ attaches a monitor that watches for `:disconnected` events whose
97
+ `detail[:error]` is a `Protocol::ZMTP::Error`, prints
98
+ `omq: <reason>` to stderr, kills the socket, and exits with
99
+ status 1. Requires `omq ~> 0.19.2` for the new `:disconnected`
100
+ detail shape and `Socket#engine` accessor.
101
+ - **`-vvv` disconnect events render the reason in parentheses.**
102
+ `Term.format_event` now pretty-prints `:disconnected` details
103
+ that contain a `:reason` key, e.g.
104
+ `disconnected tcp://:5555 (frame size 1024 exceeds max_message_size 32)`,
105
+ instead of dumping the raw hash.
106
+
107
+ ### Fixed
108
+
109
+ - **`--recv-maxsz` is now actually applied in pipe and parallel
110
+ modes.** `PipeRunner`, `PipeWorker`, and `ParallelWorker` were
111
+ only calling `SocketSetup.apply_options` + `apply_compression`
112
+ and skipping `max_message_size` entirely — so the default 1 MiB
113
+ cap (and any `--recv-maxsz` override) silently had no effect on
114
+ `omq pipe` or `omq pull -P`. Extracted the logic into
115
+ `SocketSetup.apply_recv_maxsz` and wired it into all four setup
116
+ paths (sequential pull/rep, sequential pipe, parallel worker,
117
+ pipe worker). Oversized frames now drop the connection as
118
+ intended and — combined with the CLI termination policy above —
119
+ exit with a clear error instead of hanging in a reconnect loop.
120
+
3
121
  ## 0.13.0 — 2026-04-12
4
122
 
5
123
  ### Added
data/README.md CHANGED
@@ -312,7 +312,7 @@ OMQ.outgoing { |msg| [*msg, Time.now.iso8601] }
312
312
  | `--raw` | Raw ZMTP binary (pipe to `hexdump -C` for debugging) |
313
313
  | `-J` / `--jsonl` | JSON Lines — `["frame1","frame2"]` per line |
314
314
  | `--msgpack` | MessagePack arrays (binary stream) |
315
- | `-M` / `--marshal` | Ruby Marshal (binary stream of `Array<String>` objects) |
315
+ | `-M` / `--marshal` | Ruby Marshal one arbitrary Ruby object per message |
316
316
 
317
317
  Multipart messages: in ASCII/quoted mode, frames are tab-separated. In JSONL mode,
318
318
  each message is a JSON array.
@@ -326,6 +326,20 @@ echo '["key","value"]' | omq push -c tcp://localhost:5557 -J
326
326
  omq pull -b tcp://:5557 -J
327
327
  ```
328
328
 
329
+ Under `-M`, each wire frame is one Marshal-dumped Ruby object. Inside `-e` / `-E`,
330
+ `it` is that raw object — not an Array of frames — so scalars, hashes, custom
331
+ classes, or any Marshal-safe value flow through transparently:
332
+
333
+ ```sh
334
+ # send a bare String, receive a { string => encoding } Hash
335
+ omq push -b tcp://:5557 -ME '"foo"'
336
+ omq pull -c tcp://:5557 -M -e '{it => it.encoding}'
337
+ # => {"foo" => #<Encoding:UTF-8>}
338
+ ```
339
+
340
+ At `-vvv`, trace lines for `-M` render the app-level object instead of wire
341
+ bytes: `omq: >> (marshal) [nil, :foo, "bar"]`.
342
+
329
343
  ## Timing
330
344
 
331
345
  | Flag | Effect |
@@ -348,8 +362,16 @@ omq pull -b tcp://:5557 -t 5
348
362
 
349
363
  ## Compression
350
364
 
351
- Both sides must use `--compress` (`-z`). Uses LZ4 frame format, provided
352
- by the `rlz4` gem (Ractor-safe, Rust extension via `lz4_flex`).
365
+ Set `--compress` (`-z`) on either or both sides. The flag enables
366
+ ZMTP-Zstd (provided by `omq-rfc-zstd`), a wire-protocol extension
367
+ that negotiates Zstandard compression during the ZMTP handshake via
368
+ an `X-Compression` READY metadata field. If both peers advertise it,
369
+ each side compresses its outgoing frames; if only one side does, the
370
+ connection stays plaintext (no error). The extension uses the
371
+ auto-trained dictionary mode: the sender feeds the first messages
372
+ into a dictionary trainer, ships the trained dictionary over a
373
+ ZDICT command frame, then switches to dict-bound compression for
374
+ the rest of the connection.
353
375
 
354
376
  ```sh
355
377
  omq push -c tcp://remote:5557 -z < data.txt
@@ -15,7 +15,7 @@ module OMQ
15
15
  def initialize(config, socket_class)
16
16
  @config = config
17
17
  @klass = socket_class
18
- @fmt = Formatter.new(config.format, compress: config.compress)
18
+ @fmt = Formatter.new(config.format)
19
19
  end
20
20
 
21
21
 
@@ -26,13 +26,17 @@ module OMQ
26
26
  def call(task)
27
27
  set_process_title
28
28
  setup_socket
29
- start_event_monitor if config.verbose >= 2
29
+ start_event_monitor
30
30
  maybe_start_transient_monitor(task)
31
31
  sleep(config.delay) if config.delay && config.recv_only?
32
32
  wait_for_peer if needs_peer_wait?
33
33
  run_begin_blocks
34
34
  run_loop(task)
35
35
  run_end_blocks
36
+ rescue OMQ::SocketDeadError => error
37
+ reason = error.cause&.message || error.message
38
+ $stderr.write("omq: #{reason}\n")
39
+ exit 1
36
40
  ensure
37
41
  @sock&.close
38
42
  end
@@ -262,6 +266,7 @@ module OMQ
262
266
  loop do
263
267
  parts = recv_msg
264
268
  break if parts.nil?
269
+ trace_recv(parts)
265
270
  parts = eval_recv_expr(parts)
266
271
  output(parts)
267
272
  i += 1
@@ -288,12 +293,24 @@ module OMQ
288
293
  @recv_tick_eof = true
289
294
  return 0
290
295
  end
296
+ trace_recv(parts)
291
297
  parts = eval_recv_expr(parts)
292
298
  output(parts)
293
299
  1
294
300
  end
295
301
 
296
302
 
303
+ # At -vvv, log the received message *before* eval runs. Eval
304
+ # may write to stdout (e.g. `-e 'p it'`), and we want the
305
+ # trace line to precede any such output so the sequence on the
306
+ # terminal reads as: trace → eval side-effects → body.
307
+ def trace_recv(parts)
308
+ return unless config.verbose >= 3
309
+ $stderr.write("#{Term.log_prefix(config.timestamps)}omq: << #{Formatter.preview(parts, format: config.format)}\n")
310
+ $stderr.flush
311
+ end
312
+
313
+
297
314
  def wait_for_loops(receiver, sender)
298
315
  if config.data || config.file || config.send_expr || config.recv_expr || config.target
299
316
  sender.wait
@@ -312,42 +329,38 @@ module OMQ
312
329
 
313
330
 
314
331
  def send_msg(parts)
315
- return if parts.empty?
316
- log_parts = parts
317
- parts = [Marshal.dump(parts)] if config.format == :marshal
318
- if config.compress
319
- wire = @fmt.compress(parts)
320
- trace_msg(">>", log_parts, wire)
321
- @sock.send(wire)
332
+ case config.format
333
+ when :marshal
334
+ trace_send(parts)
335
+ @sock.send([Marshal.dump(parts)])
322
336
  else
337
+ return if parts.empty?
338
+ trace_send(parts)
323
339
  @sock.send(parts)
324
340
  end
325
341
  transient_ready!
326
342
  end
327
343
 
328
344
 
345
+ # Symmetric to #trace_recv — log the outgoing message *before*
346
+ # Marshal.dump runs, so -M traces show the app-level object
347
+ # (`[nil, :foo, "bar"]`) instead of the wire-side dump bytes.
348
+ def trace_send(parts)
349
+ return unless config.verbose >= 3
350
+ $stderr.write("#{Term.log_prefix(config.timestamps)}omq: >> #{Formatter.preview(parts, format: config.format)}\n")
351
+ $stderr.flush
352
+ end
353
+
354
+
329
355
  def recv_msg
330
- raw = @sock.receive
331
- return nil if raw.nil?
332
- parts = @fmt.decompress(raw)
333
- trace_msg("<<", parts, raw) if config.compress
356
+ parts = @sock.receive
357
+ return nil if parts.nil?
334
358
  parts = Marshal.load(parts.first) if config.format == :marshal
335
359
  transient_ready!
336
360
  parts
337
361
  end
338
362
 
339
363
 
340
- # Writes an explicit -vvv message trace line. Used when compression
341
- # is active: the engine-level monitor would see compressed bytes,
342
- # so we log post-decompression (recv) / pre-compression (send)
343
- # and annotate the compressed wire size.
344
- def trace_msg(marker, parts, wire)
345
- return unless config.verbose >= 3
346
- wire_size = wire.sum(&:bytesize)
347
- $stderr.write("#{Term.log_prefix(config.timestamps)}omq: #{marker} #{Formatter.preview(parts, wire_size: wire_size)}\n")
348
- end
349
-
350
-
351
364
  def recv_msg_raw
352
365
  msg = @sock.receive
353
366
  msg&.dup
@@ -480,17 +493,43 @@ module OMQ
480
493
  end
481
494
 
482
495
 
483
- # -vv: log connect/disconnect/retry/timeout events via Socket#monitor
484
- # -vvv: also log message sent/received traces
496
+ # Always attached so protocol-level disconnect events can kill
497
+ # the socket. Verbose gating lives inside the callback:
498
+ # -vv log connect/disconnect/retry/timeout events
499
+ # -vvv also log message sent/received traces
485
500
  # --timestamps[=s|ms|us]: prepend UTC timestamps to log lines
501
+ #
502
+ # :message_received and :message_sent are intentionally skipped
503
+ # here and traced from BaseRunner#trace_recv / #trace_send
504
+ # instead — same fiber as the body write, so trace-then-body
505
+ # ordering is strict on a shared tty. The monitor-fiber path
506
+ # suffered from $stderr/$stdout buffer races and from dumping
507
+ # wire-side bytes (pre-Marshal.load on recv, post-Marshal.dump
508
+ # on send) instead of app-level parts.
509
+ SKIP_MONITOR_EVENTS = %i[message_received message_sent].freeze
486
510
  def start_event_monitor
487
- # When compress is on, BaseRunner#trace_msg logs post-
488
- # decompression, so we suppress the engine-level message trace.
489
- trace = config.verbose >= 3 && !config.compress
511
+ trace = config.verbose >= 3
512
+ log_events = config.verbose >= 2
490
513
  @sock.monitor(verbose: trace) do |event|
491
- Term.write_event(event, config.timestamps)
514
+ Term.write_event(event, config.timestamps) if log_events && !SKIP_MONITOR_EVENTS.include?(event.type)
515
+ kill_on_protocol_error(event)
492
516
  end
493
517
  end
518
+
519
+
520
+ # omq-cli policy: a peer that commits a protocol-level violation
521
+ # (Protocol::ZMTP::Error — oversized frame, decompression
522
+ # bytebomb, bad framing, …) is almost certainly a
523
+ # misconfiguration the user needs to see. Mark the socket dead
524
+ # so the next receive raises SocketDeadError. The library
525
+ # itself just drops the connection and keeps serving the
526
+ # others; this stricter policy is CLI-only.
527
+ def kill_on_protocol_error(event)
528
+ return unless event.type == :disconnected
529
+ error = event.detail && event.detail[:error]
530
+ return unless error.is_a?(Protocol::ZMTP::Error)
531
+ @sock.engine.signal_fatal_error(error)
532
+ end
494
533
  end
495
534
  end
496
535
  end
@@ -123,10 +123,30 @@ module OMQ
123
123
  # multipart via tabs
124
124
  printf "routing-key\tpayload" | omq push --connect tcp://localhost:5557
125
125
 
126
+ -- Marshal (arbitrary Ruby objects) -------------------------
127
+
128
+ # With -M, each message on the wire is one Marshal-dumped
129
+ # Ruby object. Inside -e/-E, `it` is that raw object — not
130
+ # an Array of frames — so you can send/receive scalars,
131
+ # hashes, custom classes, whatever Marshal handles.
132
+
133
+ # send a bare string, receive a { string => encoding } hash
134
+ omq push -b tcp://:5557 -ME '"foo"'
135
+ omq pull -c tcp://:5557 -Mvvv -e '{it => it.encoding}'
136
+ # output: {"foo" => #<Encoding:UTF-8>}
137
+
138
+ # -vvv traces render the app object, not wire bytes
139
+ omq push -b tcp://:5557 -ME '{now: Time.now, pid: Process.pid}' -vvv
140
+ # >> (marshal) {now: 2026-04-13 ..., pid: 12345}
141
+
126
142
  -- Compression ----------------------------------------------
127
143
 
128
- # both sides must use --compress/-z
129
- omq pull --bind tcp://:5557 --compress &
144
+ # ZMTP-Zstd is negotiated transparently during the handshake.
145
+ # Receive-capable sockets (pull, sub, rep, ...) advertise the
146
+ # profile by default in passive mode: they decode compressed
147
+ # frames from an active sender but never compress their own
148
+ # outgoing frames. Use -z / -Z on the sender to opt it in.
149
+ omq pull --bind tcp://:5557 &
130
150
  echo "compressible data" | omq push --connect tcp://localhost:5557 -z
131
151
 
132
152
  -- CURVE Encryption -----------------------------------------
@@ -225,8 +245,7 @@ module OMQ
225
245
  rcvbuf: nil,
226
246
  conflate: false,
227
247
  compress: false,
228
- compress_in: false,
229
- compress_out: false,
248
+ compress_level: nil,
230
249
  send_expr: nil,
231
250
  recv_expr: nil,
232
251
  parallel: nil,
@@ -341,7 +360,7 @@ module OMQ
341
360
  o.on( "--raw", "Raw binary, no framing") { opts[:format] = :raw }
342
361
  o.on("-J", "--jsonl", "JSON Lines (array of strings per line)") { opts[:format] = :jsonl }
343
362
  o.on( "--msgpack", "MessagePack arrays (binary stream)") { require "msgpack"; opts[:format] = :msgpack }
344
- o.on("-M", "--marshal", "Ruby Marshal stream (binary, Array<String>)") { opts[:format] = :marshal }
363
+ o.on("-M", "--marshal", "Ruby Marshal stream (one arbitrary object per message)") { opts[:format] = :marshal }
345
364
 
346
365
  o.separator "\nSubscription/groups:"
347
366
  o.on("-s", "--subscribe PREFIX", "Subscribe prefix (SUB, default all)") { |v| opts[:subscribes] << v }
@@ -386,16 +405,17 @@ module OMQ
386
405
  o.on("--conflate", "Keep only last message per subscriber (PUB/RADIO)") { opts[:conflate] = true }
387
406
 
388
407
  o.separator "\nCompression:"
389
- o.on("-z", "--compress", "LZ4 compression per frame (modal with --in/--out)") do
390
- require "rlz4"
391
- case pipe_side
392
- when :in
393
- opts[:compress_in] = true
394
- when :out
395
- opts[:compress_out] = true
396
- else
397
- opts[:compress] = true
398
- end
408
+ o.on("-z", "Zstd compression (level -3, fast)") do
409
+ opts[:compress] = true
410
+ opts[:compress_level] = -3
411
+ end
412
+ o.on("-Z", "Zstd compression (level 3, better ratio)") do
413
+ opts[:compress] = true
414
+ opts[:compress_level] = 3
415
+ end
416
+ o.on("--compress=LEVEL", Integer, "Zstd compression with custom level (e.g. 19, -1)") do |v|
417
+ opts[:compress] = true
418
+ opts[:compress_level] = v
399
419
  end
400
420
 
401
421
  o.separator "\nProcessing (-e = incoming, -E = outgoing):"
@@ -470,26 +490,9 @@ module OMQ
470
490
  opts[:type_name] = type_name.downcase
471
491
  end
472
492
 
473
- # Normalize shorthand hostnames to concrete addresses.
474
- #
475
- # Binds: tcp://:PORT → loopback (::1 if IPv6 available, else 127.0.0.1)
476
- # tcp://*:PORT → 0.0.0.0 (all interfaces, IPv4)
477
- # tcp://0.0.0.0:PORT, tcp://[::]:PORT → pass through
478
- #
479
- # Connects: tcp://:PORT → localhost (Happy Eyeballs)
480
- # tcp://*:PORT → localhost
481
- #
482
- # The hang on macOS (IPv6 connect(2) not getting ECONNREFUSED via
483
- # kqueue) is fixed by the connect timeout in Engine::Reconnect.
484
- loopback = self.class.loopback_bind_host
485
- normalize_bind = ->(url) { url.sub(%r{\Atcp://\*:}, "tcp://0.0.0.0:").sub(%r{\Atcp://:}, "tcp://#{loopback}:") }
486
- normalize_connect = ->(url) { url.sub(%r{\Atcp://(\*|):}, "tcp://localhost:") }
487
- normalize_ep = ->(ep) { Endpoint.new(ep.bind? ? normalize_bind.call(ep.url) : normalize_connect.call(ep.url), ep.bind?) }
488
- opts[:binds].map!(&normalize_bind)
489
- opts[:connects].map!(&normalize_connect)
490
- opts[:endpoints].map!(&normalize_ep)
491
- opts[:in_endpoints].map!(&normalize_ep)
492
- opts[:out_endpoints].map!(&normalize_ep)
493
+ # Host shorthand (tcp://*:PORT, tcp://:PORT, tcp://localhost:PORT)
494
+ # is normalized inside OMQ::Transport::TCP — see its
495
+ # #normalize_bind_host / #normalize_connect_host / #loopback_host.
493
496
 
494
497
  opts
495
498
  end
@@ -592,19 +595,6 @@ module OMQ
592
595
  end
593
596
 
594
597
 
595
- # Returns the loopback address for bind normalization.
596
- # Prefers IPv6 loopback ([::1]) when the host has at least one
597
- # non-loopback, non-link-local IPv6 address, otherwise 127.0.0.1.
598
- #
599
- def self.loopback_bind_host
600
- @loopback_bind_host ||= begin
601
- has_ipv6 = ::Socket.getifaddrs.any? { |ifa|
602
- addr = ifa.addr
603
- addr&.ipv6? && !addr.ipv6_loopback? && !addr.ipv6_linklocal?
604
- }
605
- has_ipv6 ? "[::1]" : "127.0.0.1"
606
- end
607
- end
608
598
  end
609
599
  end
610
600
  end
@@ -25,8 +25,7 @@ module OMQ
25
25
  parts = recv_msg_raw
26
26
  break if parts.nil?
27
27
  routing_id = parts.shift
28
- body = @fmt.decompress(parts)
29
- break unless handle_server_request(routing_id, body)
28
+ break unless handle_server_request(routing_id, parts)
30
29
  i += 1
31
30
  break if n && n > 0 && i >= n
32
31
  end
@@ -37,15 +36,15 @@ module OMQ
37
36
  if config.recv_expr || @recv_eval_proc
38
37
  reply = eval_recv_expr(body)
39
38
  output([display_routing_id(routing_id), *(reply || [""])])
40
- @sock.send_to(routing_id, @fmt.compress(reply || [""]).first)
39
+ @sock.send_to(routing_id, (reply || [""]).first)
41
40
  elsif config.echo
42
41
  output([display_routing_id(routing_id), *body])
43
- @sock.send_to(routing_id, @fmt.compress(body).first || "")
42
+ @sock.send_to(routing_id, body.first || "")
44
43
  elsif config.data || config.file || !config.stdin_is_tty
45
44
  reply = read_next
46
45
  return false unless reply
47
46
  output([display_routing_id(routing_id), *body])
48
- @sock.send_to(routing_id, @fmt.compress(reply).first || "")
47
+ @sock.send_to(routing_id, reply.first || "")
49
48
  end
50
49
  true
51
50
  end
@@ -66,7 +65,6 @@ module OMQ
66
65
  parts = recv_msg_raw
67
66
  break if parts.nil?
68
67
  routing_id = parts.shift
69
- parts = @fmt.decompress(parts)
70
68
  result = eval_recv_expr([display_routing_id(routing_id), *parts])
71
69
  output(result)
72
70
  i += 1
@@ -77,7 +75,7 @@ module OMQ
77
75
 
78
76
 
79
77
  def send_to_peer(id, parts)
80
- @sock.send_to(id, @fmt.compress(parts).first || "")
78
+ @sock.send_to(id, parts.first || "")
81
79
  end
82
80
  end
83
81
  end
@@ -44,8 +44,7 @@ module OMQ
44
44
  :rcvbuf,
45
45
  :conflate,
46
46
  :compress,
47
- :compress_in,
48
- :compress_out,
47
+ :compress_level,
49
48
  :send_expr,
50
49
  :recv_expr,
51
50
  :parallel,
@@ -46,7 +46,7 @@ module OMQ
46
46
  result = context.instance_exec(parts, &@eval_proc)
47
47
  return nil if result.nil?
48
48
  return SENT if result.equal?(context)
49
- return [result] if @format == :marshal
49
+ return result if @format == :marshal
50
50
 
51
51
  result = result.is_a?(Array) ? result : [result]
52
52
  result.map(&:to_s)
@@ -56,14 +56,16 @@ module OMQ
56
56
  end
57
57
 
58
58
 
59
- # Normalises an eval result to nil (skip) or an Array.
59
+ # Normalises an eval result to nil (skip), an Array (text formats),
60
+ # or an arbitrary Ruby object (+:marshal+).
61
+ #
60
62
  # Used inside Ractor worker blocks where instance methods are unavailable.
61
- # When +format+ is :marshal, arbitrary objects are preserved (wrapped
62
- # in a one-element Array so the wire path can Marshal.dump them).
63
+ # When +format+ is :marshal, the raw result is passed through — the
64
+ # wire path will Marshal.dump it into a single frame.
63
65
  #
64
66
  def self.normalize_result(result, format: nil)
65
67
  return nil if result.nil?
66
- return [result] if format == :marshal
68
+ return result if format == :marshal
67
69
  result = result.is_a?(Array) ? result : [result]
68
70
  result.map(&:to_s)
69
71
  end
@@ -2,17 +2,14 @@
2
2
 
3
3
  module OMQ
4
4
  module CLI
5
- # Raised when LZ4 decompression fails.
6
- class DecompressError < RuntimeError; end
7
-
8
- # Handles encoding/decoding messages in the configured format,
9
- # plus optional LZ4 compression.
5
+ # Handles encoding/decoding messages in the configured format.
6
+ # Compression is handled below the application API by ZMTP-Zstd
7
+ # (omq-rfc-zstd) once enabled via +socket.compression=+; the
8
+ # formatter sees plaintext frames in both directions.
10
9
  class Formatter
11
10
  # @param format [Symbol] wire format (:ascii, :quoted, :raw, :jsonl, :msgpack, :marshal)
12
- # @param compress [Boolean] whether to apply LZ4 compression per frame
13
- def initialize(format, compress: false)
14
- @format = format
15
- @compress = compress
11
+ def initialize(format)
12
+ @format = format
16
13
  end
17
14
 
18
15
 
@@ -35,7 +32,8 @@ module OMQ
35
32
  when :msgpack
36
33
  MessagePack.pack(parts)
37
34
  when :marshal
38
- parts.map(&:inspect).join("\t") + "\n"
35
+ # Under -M, `parts` is a single Ruby object (not a frame array).
36
+ parts.inspect + "\n"
39
37
  end
40
38
  end
41
39
 
@@ -83,47 +81,74 @@ module OMQ
83
81
  end
84
82
 
85
83
 
86
- # Compresses each frame with LZ4 if compression is enabled.
87
- #
88
- # @param parts [Array<String>] message frames
89
- # @return [Array<String>] optionally compressed frames
90
- def compress(parts)
91
- @compress ? parts.map { |p| RLZ4.compress(p) if p } : parts
92
- end
93
-
94
-
95
- # Decompresses each frame with LZ4 if compression is enabled.
96
- # nil/empty frames pass through — they were nil before send coercion.
97
- #
98
- # @param parts [Array<String>] possibly compressed message frames
99
- # @return [Array<String>] decompressed frames
100
- def decompress(parts)
101
- @compress ? parts.map { |p| p && !p.empty? ? RLZ4.decompress(p) : p } : parts
102
- rescue RLZ4::DecompressError
103
- raise DecompressError, "decompression failed (did the sender use --compress?)"
104
- end
84
+ # Whitespace/backslash visible escape sequence used by
85
+ # {Formatter.sanitize}. Everything else outside printable ASCII
86
+ # collapses to '.' via a single String#tr call.
87
+ LINE_ESCAPES = {
88
+ "\t" => '\\t',
89
+ "\n" => '\\n',
90
+ "\r" => '\\r',
91
+ "\\" => '\\\\',
92
+ }.freeze
105
93
 
106
94
 
107
95
  # Formats message parts for human-readable preview (logging).
108
- # When +wire_size+ is given, the header also shows the
109
- # compressed on-the-wire size: "(29B wire=12B)".
96
+ # When +wire_size+ is given (ZMTP-Zstd negotiated), the header
97
+ # also shows the compressed on-the-wire size: "(29B wire=12B)".
98
+ # Accepts either wire-side Array<String> (monitor events) or
99
+ # post-decode app parts that may contain non-String objects
100
+ # (e.g. -M Marshal.load output).
101
+ #
102
+ # When +format+ is +:marshal+, +parts+ is the raw Ruby object
103
+ # itself (not an Array of frames); the preview inspects it so
104
+ # the reader sees the actual payload structure (e.g.
105
+ # `[nil, :foo, "bar"]`) instead of a meaningless "1obj" header.
110
106
  #
111
- # @param parts [Array<String>] message frames (decompressed)
107
+ # @param parts [Array<String, Object>, Object] message frames, or raw object when +format+ is :marshal
108
+ # @param format [Symbol, nil] active CLI format (:marshal enables object-inspect mode)
112
109
  # @param wire_size [Integer, nil] compressed bytes on the wire
113
110
  # @return [String] truncated preview of each frame joined by |
114
- def self.preview(parts, wire_size: nil)
115
- total = parts.sum(&:bytesize)
111
+ def self.preview(parts, format: nil, wire_size: nil)
112
+ if format == :marshal
113
+ inspected = parts.inspect
114
+ truncated = inspected.bytesize > 60
115
+ inspected = inspected.byteslice(0, 60) if truncated
116
+ out = +"(marshal) #{sanitize(inspected)}"
117
+ out << "…" if truncated
118
+ return out
119
+ end
120
+
116
121
  nparts = parts.size
117
122
  shown = parts.first(3).map { |p| preview_frame(p) }
118
123
  tail = nparts > 3 ? "|…" : ""
119
- size = wire_size ? "#{total}B wire=#{wire_size}B" : "#{total}B"
124
+ total = parts.all?(String) ? parts.sum(&:bytesize) : nil
125
+ size =
126
+ if wire_size && total
127
+ "#{total}B wire=#{wire_size}B"
128
+ elsif total
129
+ "#{total}B"
130
+ else
131
+ "#{nparts}obj"
132
+ end
120
133
  header = nparts > 1 ? "(#{size} #{nparts}F)" : "(#{size})"
121
134
 
122
135
  "#{header} #{shown.join("|")}#{tail}"
123
136
  end
124
137
 
125
138
 
139
+ # Renders one frame or decoded object for {Formatter.preview}.
140
+ # Strings are sanitized byte-wise (first 12 bytes); non-String
141
+ # objects fall back to #inspect (always single-line) truncated
142
+ # at 24 bytes.
143
+ #
144
+ # @param part [String, Object]
145
+ # @return [String]
126
146
  def self.preview_frame(part)
147
+ unless part.is_a?(String)
148
+ s = part.inspect
149
+ return s.bytesize > 24 ? "#{s.byteslice(0, 24)}…" : s
150
+ end
151
+
127
152
  bytes = part.b
128
153
  # Empty frames must render as a visible marker, not as the empty
129
154
  # string — otherwise joining with "|" would produce misleading
@@ -137,11 +162,24 @@ module OMQ
137
162
  if printable < sample.bytesize / 2
138
163
  "[#{bytes.bytesize}B]"
139
164
  elsif bytes.bytesize > 12
140
- "#{sample.gsub(/[^[:print:]]/, ".")}…"
165
+ "#{sanitize(sample)}…"
141
166
  else
142
- sample.gsub(/[^[:print:]]/, ".")
167
+ sanitize(sample)
143
168
  end
144
169
  end
170
+
171
+
172
+ # Escapes bytes so a preview/body line is guaranteed single-line
173
+ # on a shared tty. Tab/newline/CR/backslash render as literal
174
+ # \t/\n/\r/\\; other non-printables collapse to '.'. Forced to
175
+ # binary encoding first to prevent UTF-8 quirks from rendering
176
+ # raw LF bytes.
177
+ #
178
+ # @param bytes [String]
179
+ # @return [String]
180
+ def self.sanitize(bytes)
181
+ bytes.b.gsub(/[\t\n\r\\]/, LINE_ESCAPES).tr("^ -~", ".")
182
+ end
145
183
  end
146
184
  end
147
185
  end
@@ -29,8 +29,15 @@ module OMQ
29
29
  compile_expr
30
30
  run_loop
31
31
  run_end_block
32
- rescue DecompressError => e
33
- @error_port.send(e.message)
32
+ rescue OMQ::SocketDeadError => error
33
+ # Socket was killed by a protocol violation on the peer side
34
+ # (see Engine#signal_fatal_error). Surface the underlying
35
+ # cause via the log stream and exit cleanly -- the Ractor
36
+ # completes, consumer threads unblock.
37
+ reason = error.cause&.message || error.message
38
+ @log_port.send("omq: #{reason}")
39
+ rescue => error
40
+ @error_port.send("#{error.class}: #{error.message}")
34
41
  ensure
35
42
  @sock&.close
36
43
  end
@@ -43,6 +50,8 @@ module OMQ
43
50
  def setup_socket
44
51
  @sock = @config.ffi ? OMQ.const_get(@socket_sym).new(backend: :ffi) : OMQ.const_get(@socket_sym).new
45
52
  OMQ::CLI::SocketSetup.apply_options(@sock, @config)
53
+ OMQ::CLI::SocketSetup.apply_recv_maxsz(@sock, @config)
54
+ OMQ::CLI::SocketSetup.apply_compression(@sock, @config, @config.type_name)
46
55
  @sock.identity = @config.identity if @config.identity
47
56
  OMQ::CLI::SocketSetup.attach_endpoints(@sock, @endpoints, verbose: 0)
48
57
  end
@@ -57,26 +66,23 @@ module OMQ
57
66
 
58
67
 
59
68
  def start_monitors
60
- return unless @config.verbose >= 2
61
- # When compress is on, messages are traced explicitly post-
62
- # decompression; suppress engine-level message trace to avoid
63
- # showing compressed bytes.
64
- trace = @config.verbose >= 3 && !@config.compress
69
+ trace = @config.verbose >= 3
70
+ log_events = @config.verbose >= 2
65
71
  @sock.monitor(verbose: trace) do |event|
66
- @log_port.send(OMQ::CLI::Term.format_event(event, @config.timestamps))
72
+ @log_port.send(OMQ::CLI::Term.format_event(event, @config.timestamps)) if log_events
73
+ kill_on_protocol_error(event)
67
74
  end
68
75
  end
69
76
 
70
77
 
71
- def trace_in(parts, wire)
72
- return unless @config.verbose >= 3 && @config.compress
73
- @log_port.send("#{OMQ::CLI::Term.log_prefix(@config.timestamps)}omq: << #{OMQ::CLI::Formatter.preview(parts, wire_size: wire.sum(&:bytesize))}")
74
- end
75
-
76
-
77
- def trace_out(parts, wire)
78
- return unless @config.verbose >= 3 && @config.compress
79
- @log_port.send("#{OMQ::CLI::Term.log_prefix(@config.timestamps)}omq: >> #{OMQ::CLI::Formatter.preview(parts, wire_size: wire.sum(&:bytesize))}")
78
+ # Mirrors BaseRunner#kill_on_protocol_error: CLI-level policy
79
+ # that protocol-level disconnects kill the socket so the
80
+ # recv loop unblocks with SocketDeadError.
81
+ def kill_on_protocol_error(event)
82
+ return unless event.type == :disconnected
83
+ error = event.detail && event.detail[:error]
84
+ return unless error.is_a?(Protocol::ZMTP::Error)
85
+ @sock.engine.signal_fatal_error(error)
80
86
  end
81
87
 
82
88
 
@@ -96,7 +102,7 @@ module OMQ
96
102
  def compile_expr
97
103
  @begin_proc, @end_proc, @eval_proc =
98
104
  OMQ::CLI::ExpressionEvaluator.compile_inside_ractor(@config.recv_expr)
99
- @fmt = OMQ::CLI::Formatter.new(@config.format, compress: @config.compress)
105
+ @fmt = OMQ::CLI::Formatter.new(@config.format)
100
106
  @ctx = Object.new
101
107
  @ctx.instance_exec(&@begin_proc) if @begin_proc
102
108
  end
@@ -119,10 +125,8 @@ module OMQ
119
125
  n = @config.count
120
126
  i = 0
121
127
  loop do
122
- wire = @sock.receive
123
- break if wire.nil?
124
- parts = @fmt.decompress(wire)
125
- trace_in(parts, wire)
128
+ parts = @sock.receive
129
+ break if parts.nil?
126
130
  if @eval_proc
127
131
  parts = normalize(
128
132
  @ctx.instance_exec(parts, &@eval_proc)
@@ -145,15 +149,11 @@ module OMQ
145
149
  n = @config.count
146
150
  i = 0
147
151
  loop do
148
- wire = @sock.receive
149
- break if wire.nil?
150
- parts = @fmt.decompress(wire)
151
- trace_in(parts, wire)
152
+ parts = @sock.receive
153
+ break if parts.nil?
152
154
  reply = compute_reply(parts)
153
155
  output(reply)
154
- reply_wire = @fmt.compress(reply)
155
- trace_out(reply, reply_wire)
156
- @sock.send(reply_wire)
156
+ @sock.send(reply)
157
157
  i += 1
158
158
  break if n && n > 0 && i >= n
159
159
  end
data/lib/omq/cli/pipe.rb CHANGED
@@ -11,9 +11,8 @@ module OMQ
11
11
 
12
12
  # @param config [Config] frozen CLI configuration
13
13
  def initialize(config)
14
- @config = config
15
- @fmt_in = Formatter.new(config.format, compress: config.compress_in || config.compress)
16
- @fmt_out = Formatter.new(config.format, compress: config.compress_out || config.compress)
14
+ @config = config
15
+ @fmt = Formatter.new(config.format)
17
16
  end
18
17
 
19
18
 
@@ -51,12 +50,16 @@ module OMQ
51
50
  @pull, @push = build_pull_push(in_eps, out_eps)
52
51
  compile_expr
53
52
  @sock = @pull # for eval instance_exec
54
- start_event_monitors if config.verbose >= 2
53
+ start_event_monitors
55
54
  wait_for_peers_with_timeout if config.timeout
56
55
  setup_sequential_transient(task)
57
56
  @sock.instance_exec(&@recv_begin_proc) if @recv_begin_proc
58
57
  sequential_message_loop(fan_out: out_eps.size > 1)
59
58
  @sock.instance_exec(&@recv_end_proc) if @recv_end_proc
59
+ rescue OMQ::SocketDeadError => error
60
+ reason = error.cause&.message || error.message
61
+ $stderr.write("omq: #{reason}\n")
62
+ exit 1
60
63
  ensure
61
64
  @pull&.close
62
65
  @push&.close
@@ -85,6 +88,9 @@ module OMQ
85
88
  push = OMQ::PUSH.new(**kwargs)
86
89
  SocketSetup.apply_options(pull, config)
87
90
  SocketSetup.apply_options(push, config)
91
+ SocketSetup.apply_recv_maxsz(pull, config)
92
+ SocketSetup.apply_compression(pull, config, "pull")
93
+ SocketSetup.apply_compression(push, config, "push")
88
94
  SocketSetup.attach_endpoints(pull, in_eps, verbose: config.verbose, timestamps: config.timestamps)
89
95
  SocketSetup.attach_endpoints(push, out_eps, verbose: config.verbose, timestamps: config.timestamps)
90
96
  [pull, push]
@@ -107,10 +113,9 @@ module OMQ
107
113
  loop do
108
114
  parts = @pull.receive
109
115
  break if parts.nil?
110
- parts = @fmt_in.decompress(parts)
111
116
  parts = eval_recv_expr(parts)
112
117
  if parts && !parts.empty?
113
- @push.send(@fmt_out.compress(parts))
118
+ @push.send(parts)
114
119
  end
115
120
  # Yield after send so send-pump fibers can drain the queue
116
121
  # before the next message is enqueued. Without this, one pump
@@ -163,7 +168,7 @@ module OMQ
163
168
  def set_pipe_process_title
164
169
  in_eps, out_eps = resolve_endpoints
165
170
  title = ["omq pipe"]
166
- title << "-z" if config.compress || config.compress_in || config.compress_out
171
+ title << "-z" if config.compress
167
172
  title << "-P#{config.parallel}" if config.parallel
168
173
  title.concat(in_eps.map(&:url))
169
174
  title << "->"
@@ -193,13 +198,23 @@ module OMQ
193
198
 
194
199
 
195
200
  def start_event_monitors
196
- trace = config.verbose >= 3
201
+ trace = config.verbose >= 3
202
+ log_events = config.verbose >= 2
197
203
  [@pull, @push].each do |sock|
198
204
  sock.monitor(verbose: trace) do |event|
199
- Term.write_event(event, config.timestamps)
205
+ Term.write_event(event, config.timestamps) if log_events
206
+ kill_on_protocol_error(sock, event)
200
207
  end
201
208
  end
202
209
  end
210
+
211
+
212
+ def kill_on_protocol_error(sock, event)
213
+ return unless event.type == :disconnected
214
+ error = event.detail && event.detail[:error]
215
+ return unless error.is_a?(Protocol::ZMTP::Error)
216
+ sock.engine.signal_fatal_error(error)
217
+ end
203
218
  end
204
219
  end
205
220
  end
@@ -19,13 +19,14 @@ module OMQ
19
19
  Async do
20
20
  setup_sockets
21
21
  log_endpoints if @config.verbose >= 1
22
- start_monitors if @config.verbose >= 2
22
+ start_monitors
23
23
  wait_for_peers_with_timeout if @config.timeout
24
24
  compile_expr
25
25
  run_message_loop
26
26
  run_end_block
27
- rescue OMQ::CLI::DecompressError => e
28
- @error_port&.send(e.message)
27
+ rescue OMQ::SocketDeadError => error
28
+ reason = error.cause&.message || error.message
29
+ @log_port.send("omq: #{reason}")
29
30
  ensure
30
31
  @pull&.close
31
32
  @push&.close
@@ -42,6 +43,9 @@ module OMQ
42
43
  @push = OMQ::PUSH.new(**kwargs)
43
44
  OMQ::CLI::SocketSetup.apply_options(@pull, @config)
44
45
  OMQ::CLI::SocketSetup.apply_options(@push, @config)
46
+ OMQ::CLI::SocketSetup.apply_recv_maxsz(@pull, @config)
47
+ OMQ::CLI::SocketSetup.apply_compression(@pull, @config, "pull")
48
+ OMQ::CLI::SocketSetup.apply_compression(@push, @config, "push")
45
49
  OMQ::CLI::SocketSetup.attach_endpoints(@pull, @in_eps, verbose: 0)
46
50
  OMQ::CLI::SocketSetup.attach_endpoints(@push, @out_eps, verbose: 0)
47
51
  end
@@ -55,28 +59,22 @@ module OMQ
55
59
 
56
60
 
57
61
  def start_monitors
58
- # When compress is on, messages are traced explicitly in
59
- # run_message_loop post-decompression; keep engine-level
60
- # traces off to avoid showing compressed bytes.
61
- compress = @config.compress || @config.compress_in || @config.compress_out
62
- trace = @config.verbose >= 3 && !compress
62
+ trace = @config.verbose >= 3
63
+ log_events = @config.verbose >= 2
63
64
  [@pull, @push].each do |sock|
64
65
  sock.monitor(verbose: trace) do |event|
65
- @log_port.send(OMQ::CLI::Term.format_event(event, @config.timestamps))
66
+ @log_port.send(OMQ::CLI::Term.format_event(event, @config.timestamps)) if log_events
67
+ kill_on_protocol_error(sock, event)
66
68
  end
67
69
  end
68
70
  end
69
71
 
70
72
 
71
- def trace_in(parts, wire)
72
- return unless @config.verbose >= 3 && (@config.compress || @config.compress_in)
73
- @log_port.send("#{OMQ::CLI::Term.log_prefix(@config.timestamps)}omq: << #{OMQ::CLI::Formatter.preview(parts, wire_size: wire.sum(&:bytesize))}")
74
- end
75
-
76
-
77
- def trace_out(parts, wire)
78
- return unless @config.verbose >= 3 && (@config.compress || @config.compress_out)
79
- @log_port.send("#{OMQ::CLI::Term.log_prefix(@config.timestamps)}omq: >> #{OMQ::CLI::Formatter.preview(parts, wire_size: wire.sum(&:bytesize))}")
73
+ def kill_on_protocol_error(sock, event)
74
+ return unless event.type == :disconnected
75
+ error = event.detail && event.detail[:error]
76
+ return unless error.is_a?(Protocol::ZMTP::Error)
77
+ sock.engine.signal_fatal_error(error)
80
78
  end
81
79
 
82
80
 
@@ -96,8 +94,7 @@ module OMQ
96
94
  def compile_expr
97
95
  @begin_proc, @end_proc, @eval_proc =
98
96
  OMQ::CLI::ExpressionEvaluator.compile_inside_ractor(@config.recv_expr)
99
- @fmt_in = OMQ::CLI::Formatter.new(@config.format, compress: @config.compress_in || @config.compress)
100
- @fmt_out = OMQ::CLI::Formatter.new(@config.format, compress: @config.compress_out || @config.compress)
97
+ @fmt = OMQ::CLI::Formatter.new(@config.format)
101
98
  @ctx = Object.new
102
99
  @ctx.instance_exec(&@begin_proc) if @begin_proc
103
100
  end
@@ -124,29 +121,21 @@ module OMQ
124
121
 
125
122
 
126
123
  def process_one_eval
127
- wire_in = @pull.receive
128
- return false if wire_in.nil?
129
- parts_in = @fmt_in.decompress(wire_in)
130
- trace_in(parts_in, wire_in)
124
+ parts_in = @pull.receive
125
+ return false if parts_in.nil?
131
126
  parts_out = OMQ::CLI::ExpressionEvaluator.normalize_result(
132
127
  @ctx.instance_exec(parts_in, &@eval_proc), format: @config.format
133
128
  )
134
129
  return true if parts_out.nil? || parts_out.empty?
135
- wire_out = @fmt_out.compress(parts_out)
136
- trace_out(parts_out, wire_out)
137
- @push << wire_out
130
+ @push << parts_out
138
131
  true
139
132
  end
140
133
 
141
134
 
142
135
  def process_one_passthrough
143
- wire_in = @pull.receive
144
- return false if wire_in.nil?
145
- parts_in = @fmt_in.decompress(wire_in)
146
- trace_in(parts_in, wire_in)
147
- wire_out = @fmt_out.compress(parts_in)
148
- trace_out(parts_in, wire_out)
149
- @push << wire_out
136
+ parts_in = @pull.receive
137
+ return false if parts_in.nil?
138
+ @push << parts_in
150
139
  true
151
140
  end
152
141
 
@@ -156,7 +145,7 @@ module OMQ
156
145
  out = OMQ::CLI::ExpressionEvaluator.normalize_result(
157
146
  @ctx.instance_exec(&@end_proc), format: @config.format
158
147
  )
159
- @push << @fmt_out.compress(out) if out && !out.empty?
148
+ @push << out if out && !out.empty?
160
149
  end
161
150
  end
162
151
  end
@@ -11,11 +11,16 @@ module OMQ
11
11
 
12
12
 
13
13
  def send_msg(parts)
14
- return if parts.empty?
15
- parts = [Marshal.dump(parts)] if config.format == :marshal
16
- parts = @fmt.compress(parts)
17
- group = config.group || parts.shift
18
- @sock.publish(group, parts.first || "")
14
+ case config.format
15
+ when :marshal
16
+ trace_send(parts)
17
+ @sock.publish(config.group || "", Marshal.dump(parts))
18
+ else
19
+ return if parts.empty?
20
+ trace_send(parts)
21
+ group = config.group || parts.shift
22
+ @sock.publish(group, parts.first || "")
23
+ end
19
24
  transient_ready!
20
25
  end
21
26
  end
@@ -25,7 +25,6 @@ module OMQ
25
25
  break if parts.nil?
26
26
  identity = parts.shift
27
27
  parts.shift if parts.first == ""
28
- parts = @fmt.decompress(parts)
29
28
  result = eval_recv_expr([display_routing_id(identity), *parts])
30
29
  output(result)
31
30
  i += 1
@@ -36,7 +35,7 @@ module OMQ
36
35
 
37
36
 
38
37
  def send_to_peer(id, parts)
39
- @sock.send([id, "", *@fmt.compress(parts)])
38
+ @sock.send([id, "", *parts])
40
39
  end
41
40
  end
42
41
  end
@@ -22,6 +22,40 @@ module OMQ
22
22
  # it entirely with --recv-maxsz 0.
23
23
  DEFAULT_RECV_MAXSZ = 1 << 20
24
24
 
25
+ # Socket types that never receive application data. These opt
26
+ # out of the default passive-compression behavior -- they would
27
+ # otherwise advertise a profile and wrap every outgoing frame in
28
+ # a 4-byte uncompressed sentinel for no benefit, since there's
29
+ # nothing to decompress in return.
30
+ PURE_SEND_TYPES = %w[push pub scatter radio].freeze
31
+
32
+
33
+ # Installs ZMTP-Zstd compression on +sock+ based on +type_name+
34
+ # and the explicit flags in +config+. Three outcomes:
35
+ #
36
+ # * +config.compress+ is true --> active compression (auto-dict).
37
+ # Outgoing frames are compressed; incoming are decoded.
38
+ # * +config.compress+ is false and +type_name+ can receive -->
39
+ # passive compression (RFC Sec. 6.4). The socket advertises the
40
+ # profile so an active peer can compress on the wire and we can
41
+ # decode it, but we never compress our own outgoing frames.
42
+ # * +config.compress+ is false and +type_name+ is in
43
+ # +PURE_SEND_TYPES+ --> no compression. Pure senders have no
44
+ # incoming traffic to decompress, so passive mode is pure
45
+ # overhead on outgoing.
46
+ #
47
+ # Callers pass +type_name+ explicitly (rather than reading it off
48
+ # +config+) so the pipe runners can install different modes on
49
+ # their push/pull ends of a single pipe.
50
+ def self.apply_compression(sock, config, type_name)
51
+ if config.compress
52
+ sock.compression = OMQ::RFC::Zstd::Compression.auto(level: config.compress_level || -3)
53
+ elsif !PURE_SEND_TYPES.include?(type_name)
54
+ sock.compression = OMQ::RFC::Zstd::Compression.auto(passive: true)
55
+ end
56
+ end
57
+
58
+
25
59
  # Apply common socket options from +config+ to +sock+.
26
60
  #
27
61
  def self.apply_options(sock, config)
@@ -44,16 +78,22 @@ module OMQ
44
78
  sock = config.ffi ? klass.new(backend: :ffi) : klass.new
45
79
  sock.conflate = true if config.conflate && %w[pub radio].include?(config.type_name)
46
80
  apply_options(sock, config)
47
- # --recv-maxsz: nil → 1 MiB default; 0 → explicitly unlimited; else → as-is.
81
+ apply_recv_maxsz(sock, config)
82
+ sock.identity = config.identity if config.identity
83
+ sock.router_mandatory = true if config.type_name == "router"
84
+ apply_compression(sock, config, config.type_name)
85
+ sock
86
+ end
87
+
88
+
89
+ # --recv-maxsz: nil → 1 MiB default; 0 → explicitly unlimited; else → as-is.
90
+ def self.apply_recv_maxsz(sock, config)
48
91
  sock.max_message_size =
49
92
  case config.recv_maxsz
50
93
  when nil then DEFAULT_RECV_MAXSZ
51
94
  when 0 then nil
52
95
  else config.recv_maxsz
53
96
  end
54
- sock.identity = config.identity if config.identity
55
- sock.router_mandatory = true if config.type_name == "router"
56
- sock
57
97
  end
58
98
 
59
99
 
data/lib/omq/cli/term.rb CHANGED
@@ -40,12 +40,23 @@ module OMQ
40
40
  prefix = log_prefix(timestamps)
41
41
  case event.type
42
42
  when :message_sent
43
- "#{prefix}omq: >> #{Formatter.preview(event.detail[:parts])}"
43
+ "#{prefix}omq: >> #{Formatter.preview(event.detail[:parts], wire_size: event.detail[:wire_size])}"
44
44
  when :message_received
45
- "#{prefix}omq: << #{Formatter.preview(event.detail[:parts])}"
45
+ "#{prefix}omq: << #{Formatter.preview(event.detail[:parts], wire_size: event.detail[:wire_size])}"
46
+ when :zdict_sent
47
+ "#{prefix}omq: >> ZDICT (#{event.detail[:size]}B)"
48
+ when :zdict_received
49
+ "#{prefix}omq: << ZDICT (#{event.detail[:size]}B)"
46
50
  else
47
51
  ep = event.endpoint ? " #{event.endpoint}" : ""
48
- detail = event.detail ? " #{event.detail}" : ""
52
+ detail =
53
+ if event.detail.is_a?(Hash) && event.detail[:reason]
54
+ " (#{event.detail[:reason]})"
55
+ elsif event.detail
56
+ " #{event.detail}"
57
+ else
58
+ ""
59
+ end
49
60
  "#{prefix}omq: #{event.type}#{ep}#{detail}"
50
61
  end
51
62
  end
@@ -2,6 +2,6 @@
2
2
 
3
3
  module OMQ
4
4
  module CLI
5
- VERSION = "0.13.0"
5
+ VERSION = "0.14.1"
6
6
  end
7
7
  end
data/lib/omq/cli.rb CHANGED
@@ -197,6 +197,7 @@ module OMQ
197
197
  require "omq/rfc/scattergather"
198
198
  require "omq/rfc/channel"
199
199
  require "omq/rfc/p2p"
200
+ require "omq/rfc/zstd"
200
201
  require "async"
201
202
  require "json"
202
203
  require "console"
@@ -247,9 +248,6 @@ module OMQ
247
248
  runner_class.new(config)
248
249
  end
249
250
  runner.call(task)
250
- rescue DecompressError => e
251
- $stderr.puts "omq: #{e.message}"
252
- exit 1
253
251
  rescue IO::TimeoutError, Async::TimeoutError
254
252
  $stderr.puts "omq: timeout" unless config.quiet
255
253
  exit 2
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: omq-cli
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.13.0
4
+ version: 0.14.1
5
5
  platform: ruby
6
6
  authors:
7
7
  - Patrik Wenger
@@ -15,14 +15,20 @@ dependencies:
15
15
  requirements:
16
16
  - - "~>"
17
17
  - !ruby/object:Gem::Version
18
- version: '0.18'
18
+ version: '0.19'
19
+ - - ">="
20
+ - !ruby/object:Gem::Version
21
+ version: 0.19.2
19
22
  type: :runtime
20
23
  prerelease: false
21
24
  version_requirements: !ruby/object:Gem::Requirement
22
25
  requirements:
23
26
  - - "~>"
24
27
  - !ruby/object:Gem::Version
25
- version: '0.18'
28
+ version: '0.19'
29
+ - - ">="
30
+ - !ruby/object:Gem::Version
31
+ version: 0.19.2
26
32
  - !ruby/object:Gem::Dependency
27
33
  name: omq-ffi
28
34
  requirement: !ruby/object:Gem::Requirement
@@ -108,51 +114,51 @@ dependencies:
108
114
  - !ruby/object:Gem::Version
109
115
  version: '0.1'
110
116
  - !ruby/object:Gem::Dependency
111
- name: msgpack
117
+ name: omq-rfc-zstd
112
118
  requirement: !ruby/object:Gem::Requirement
113
119
  requirements:
114
- - - ">="
120
+ - - "~>"
115
121
  - !ruby/object:Gem::Version
116
- version: '0'
122
+ version: '0.1'
117
123
  type: :runtime
118
124
  prerelease: false
119
125
  version_requirements: !ruby/object:Gem::Requirement
120
126
  requirements:
121
- - - ">="
127
+ - - "~>"
122
128
  - !ruby/object:Gem::Version
123
- version: '0'
129
+ version: '0.1'
124
130
  - !ruby/object:Gem::Dependency
125
- name: rbnacl
131
+ name: msgpack
126
132
  requirement: !ruby/object:Gem::Requirement
127
133
  requirements:
128
- - - "~>"
134
+ - - ">="
129
135
  - !ruby/object:Gem::Version
130
- version: '7.0'
136
+ version: '0'
131
137
  type: :runtime
132
138
  prerelease: false
133
139
  version_requirements: !ruby/object:Gem::Requirement
134
140
  requirements:
135
- - - "~>"
141
+ - - ">="
136
142
  - !ruby/object:Gem::Version
137
- version: '7.0'
143
+ version: '0'
138
144
  - !ruby/object:Gem::Dependency
139
- name: rlz4
145
+ name: rbnacl
140
146
  requirement: !ruby/object:Gem::Requirement
141
147
  requirements:
142
148
  - - "~>"
143
149
  - !ruby/object:Gem::Version
144
- version: '0.1'
150
+ version: '7.0'
145
151
  type: :runtime
146
152
  prerelease: false
147
153
  version_requirements: !ruby/object:Gem::Requirement
148
154
  requirements:
149
155
  - - "~>"
150
156
  - !ruby/object:Gem::Version
151
- version: '0.1'
157
+ version: '7.0'
152
158
  description: Command-line tool for sending and receiving ZeroMQ messages on any socket
153
159
  type (REQ/REP, PUB/SUB, PUSH/PULL, DEALER/ROUTER, and all draft types). Supports
154
160
  Ruby eval (-e/-E), script handlers (-r), pipe virtual socket with Ractor parallelism,
155
- multiple formats (ASCII, JSON Lines, msgpack, Marshal), LZ4 compression, and CURVE
161
+ multiple formats (ASCII, JSON Lines, msgpack, Marshal), Zstd compression, and CURVE
156
162
  encryption. Like nngcat from libnng, but with Ruby superpowers.
157
163
  email:
158
164
  - paddor@gmail.com