anycable 0.6.1 → 0.6.2

Sign up to get free protection for your applications and to get access to all the features.
Files changed (60) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +43 -2
  3. data/bin/anycable +1 -1
  4. data/bin/anycabled +30 -0
  5. data/lib/anycable/cli.rb +1 -1
  6. data/lib/anycable/config.rb +4 -2
  7. data/lib/anycable/exceptions_handling.rb +9 -3
  8. data/lib/anycable/handler/capture_exceptions.rb +4 -4
  9. data/lib/anycable/server.rb +22 -6
  10. data/lib/anycable/version.rb +1 -1
  11. metadata +5 -53
  12. data/.github/ISSUE_TEMPLATE.md +0 -25
  13. data/.github/PULL_REQUEST_TEMPLATE.md +0 -31
  14. data/.gitignore +0 -40
  15. data/.hound.yml +0 -3
  16. data/.rubocop.yml +0 -71
  17. data/.travis.yml +0 -16
  18. data/Gemfile +0 -8
  19. data/Makefile +0 -5
  20. data/PITCHME.md +0 -139
  21. data/PITCHME.yaml +0 -1
  22. data/Rakefile +0 -8
  23. data/anycable.gemspec +0 -35
  24. data/assets/Memory3.png +0 -0
  25. data/assets/Memory5.png +0 -0
  26. data/assets/RTT3.png +0 -0
  27. data/assets/RTT5.png +0 -0
  28. data/assets/Scheme1.png +0 -0
  29. data/assets/Scheme2.png +0 -0
  30. data/assets/cpu_chart.gif +0 -0
  31. data/assets/cpu_chart2.gif +0 -0
  32. data/assets/evlms.png +0 -0
  33. data/benchmarks/.gitignore +0 -2
  34. data/benchmarks/2017-02-12.md +0 -308
  35. data/benchmarks/2018-03-04.md +0 -192
  36. data/benchmarks/2018-05-27-rpc-bench.md +0 -57
  37. data/benchmarks/2018-10-27.md +0 -181
  38. data/benchmarks/HowTo.md +0 -23
  39. data/benchmarks/ansible.cfg +0 -9
  40. data/benchmarks/assets/2018-10-27-action-cable-rss.png +0 -0
  41. data/benchmarks/assets/2018-10-27-action-cable-rtt.png +0 -0
  42. data/benchmarks/assets/2018-10-27-anycable-rss.png +0 -0
  43. data/benchmarks/assets/2018-10-27-anycable-rtt.png +0 -0
  44. data/benchmarks/assets/2018-10-27-async-rss.png +0 -0
  45. data/benchmarks/assets/2018-10-27-async-rtt.png +0 -0
  46. data/benchmarks/assets/2018-10-27-falcon-cable-rss.png +0 -0
  47. data/benchmarks/assets/2018-10-27-falcon-cable-rtt.png +0 -0
  48. data/benchmarks/assets/2018-10-27-iodine-cable-rss.png +0 -0
  49. data/benchmarks/assets/2018-10-27-iodine-cable-rtt.png +0 -0
  50. data/benchmarks/assets/2018-10-27-plezi-rss.png +0 -0
  51. data/benchmarks/assets/2018-10-27-plezi-rtt.png +0 -0
  52. data/benchmarks/bench.png +0 -0
  53. data/benchmarks/benchmark.yml +0 -69
  54. data/benchmarks/hosts +0 -5
  55. data/benchmarks/rtt_plot.py +0 -74
  56. data/benchmarks/rtt_plot_test.py +0 -16
  57. data/benchmarks/servers.yml +0 -58
  58. data/circle.yml +0 -8
  59. data/etc/bug_report_template.rb +0 -76
  60. data/protos/rpc.proto +0 -55
data/benchmarks/2018-03-04.md DELETED
@@ -1,192 +0,0 @@
1
- # WebSocket Shootout Benchmark (2018-03-04)
2
-
3
- Code: https://github.com/palkan/websocket-shootout
4
- Client/Server instances: c3.2xlarge (8 vCPU, 15 GiB RAM).
5
-
6
- Ruby: 2.5.0
7
- Rails: 5.1.4
8
-
9
- The benchmark measures broadcasting RTT for 10k clients.
10
-
11
- **NOTE**: memory and peak CPU usage was measured by looking at `htop` output; avg CPU usage is a 1 minute LA.
12
-
13
- ## Action Cable (8 workers)
14
-
15
- Command:
16
-
17
- ```sh
18
- bin/websocket-bench broadcast -l 172.31.26.145 -l 172.31.26.146 -l 172.31.26.147 --concurrent 4 --sample-size 100 --step-size 1000 --payload-padding 200 --total-steps 10 --origin http://0.0.0.0 ws://172.31.22.121:3334/cable --server-type=actioncable
19
-
20
- clients: 1000 95per-rtt: 847ms min-rtt: 1ms median-rtt: 174ms max-rtt: 930ms
21
- clients: 2000 95per-rtt: 1611ms min-rtt: 1ms median-rtt: 387ms max-rtt: 1768ms
22
- clients: 3000 95per-rtt: 2107ms min-rtt: 1ms median-rtt: 644ms max-rtt: 2601ms
23
- clients: 4000 95per-rtt: 2504ms min-rtt: 1ms median-rtt: 899ms max-rtt: 3118ms
24
- clients: 5000 95per-rtt: 3452ms min-rtt: 1ms median-rtt: 1070ms max-rtt: 4487ms
25
- clients: 6000 95per-rtt: 4044ms min-rtt: 1ms median-rtt: 1261ms max-rtt: 5018ms
26
- clients: 7000 95per-rtt: 3207ms min-rtt: 1ms median-rtt: 1647ms max-rtt: 3958ms
27
- clients: 8000 95per-rtt: 4115ms min-rtt: 1ms median-rtt: 1957ms max-rtt: 5059ms
28
- clients: 9000 95per-rtt: 4306ms min-rtt: 1ms median-rtt: 2273ms max-rtt: 7160ms
29
- clients: 10000 95per-rtt: 5147ms min-rtt: 1ms median-rtt: 2462ms max-rtt: 6734ms
30
-
31
- Missing received broadcasts: expected 6600000, got 6554972
32
- ```
33
-
34
- CPU (peak): ~100%
35
- CPU (avg): ~30%
36
-
37
- MEM: ~1.7 GiB
38
-
39
-
40
- ## AnyCable-Go 0.5.4
41
-
42
- ```
43
- bin/websocket-bench broadcast -l 172.31.26.145 -l 172.31.26.146 -l 172.31.26.147 --concurrent 4 --sample-size 100 --step-size 1000 --payload-padding 200 --total-steps 10 --origin http://0.0.0.0 ws://172.31.22.121:3334/cable --server-type=actioncable
44
-
45
- clients: 1000 95per-rtt: 61ms min-rtt: 2ms median-rtt: 18ms max-rtt: 178ms
46
- clients: 2000 95per-rtt: 214ms min-rtt: 2ms median-rtt: 26ms max-rtt: 568ms
47
- clients: 3000 95per-rtt: 275ms min-rtt: 2ms median-rtt: 32ms max-rtt: 645ms
48
- clients: 4000 95per-rtt: 430ms min-rtt: 2ms median-rtt: 40ms max-rtt: 1463ms
49
- clients: 5000 95per-rtt: 352ms min-rtt: 3ms median-rtt: 61ms max-rtt: 1017ms
50
- clients: 6000 95per-rtt: 619ms min-rtt: 3ms median-rtt: 60ms max-rtt: 1630ms
51
- clients: 7000 95per-rtt: 623ms min-rtt: 2ms median-rtt: 74ms max-rtt: 3099ms
52
- clients: 8000 95per-rtt: 846ms min-rtt: 2ms median-rtt: 89ms max-rtt: 3165ms
53
- clients: 9000 95per-rtt: 642ms min-rtt: 2ms median-rtt: 110ms max-rtt: 981ms
54
- clients: 10000 95per-rtt: 1019ms min-rtt: 2ms median-rtt: 114ms max-rtt: 4759ms
55
-
56
- Missing received broadcasts: expected 6600000, got 6580959
57
- ```
58
-
59
- CPU (peak): ~40%
60
- CPU (avg): ~3%
61
-
62
- MEM: ~500 MiB
63
-
64
- ## AnyCable-Go 0.6.0-preview1
65
-
66
- ```
67
- bin/websocket-bench broadcast -l 172.31.26.145 -l 172.31.26.146 -l 172.31.26.147 --concurrent 4 --sample-size 100 --step-size 1000 --payload-padding 200 --total-steps 10 --origin http://0.0.0.0 ws://172.31.22.121:3334/cable --server-type=actioncable
68
-
69
- clients: 1000 95per-rtt: 64ms min-rtt: 2ms median-rtt: 15ms max-rtt: 191ms
70
- clients: 2000 95per-rtt: 188ms min-rtt: 2ms median-rtt: 27ms max-rtt: 325ms
71
- clients: 3000 95per-rtt: 218ms min-rtt: 2ms median-rtt: 36ms max-rtt: 854ms
72
- clients: 4000 95per-rtt: 483ms min-rtt: 2ms median-rtt: 37ms max-rtt: 2353ms
73
- clients: 5000 95per-rtt: 388ms min-rtt: 2ms median-rtt: 63ms max-rtt: 659ms
74
- clients: 6000 95per-rtt: 500ms min-rtt: 2ms median-rtt: 59ms max-rtt: 1430ms
75
- clients: 7000 95per-rtt: 692ms min-rtt: 2ms median-rtt: 85ms max-rtt: 1333ms
76
- clients: 8000 95per-rtt: 591ms min-rtt: 2ms median-rtt: 90ms max-rtt: 2499ms
77
- clients: 9000 95per-rtt: 959ms min-rtt: 2ms median-rtt: 90ms max-rtt: 5961ms
78
- clients: 10000 95per-rtt: 674ms min-rtt: 2ms median-rtt: 106ms max-rtt: 2543ms
79
-
80
- Missing received broadcasts: expected 6600000, got 6599507
81
- ```
82
-
83
- CPU (peak): ~40%
84
- CPU (avg): ~3%
85
-
86
- MEM: ~450 MiB
87
-
88
- ## Plezi (0.15.0) + Iodine (0.4.16) (8 workers)
89
-
90
- ```
91
- bin/websocket-bench broadcast -l 172.31.26.145 -l 172.31.26.146 -l 172.31.26.147 --concurrent 4 --sample-size 100 --step-size 1000 --payload-padding 200 --total-steps 10 ws://172.31.22.121:3334/cable
92
-
93
- clients: 1000 95per-rtt: 48ms min-rtt: 1ms median-rtt: 18ms max-rtt: 208ms
94
- clients: 2000 95per-rtt: 236ms min-rtt: 0ms median-rtt: 24ms max-rtt: 612ms
95
- clients: 3000 95per-rtt: 224ms min-rtt: 0ms median-rtt: 38ms max-rtt: 662ms
96
- clients: 4000 95per-rtt: 402ms min-rtt: 0ms median-rtt: 39ms max-rtt: 3071ms
97
- clients: 5000 95per-rtt: 392ms min-rtt: 1ms median-rtt: 47ms max-rtt: 1850ms
98
- clients: 6000 95per-rtt: 582ms min-rtt: 1ms median-rtt: 64ms max-rtt: 1825ms
99
- clients: 7000 95per-rtt: 393ms min-rtt: 1ms median-rtt: 92ms max-rtt: 729ms
100
- clients: 8000 95per-rtt: 1181ms min-rtt: 1ms median-rtt: 42ms max-rtt: 5782ms
101
- clients: 9000 95per-rtt: 676ms min-rtt: 1ms median-rtt: 107ms max-rtt: 2220ms
102
- clients: 10000 95per-rtt: 603ms min-rtt: 1ms median-rtt: 131ms max-rtt: 5703ms
103
-
104
- Missing received broadcasts: expected 6600000, got 6586987
105
- ```
106
-
107
- CPU (peak): ~50%
108
- CPU (avg): ~5%
109
-
110
- MEM: ~700 MiB
111
-
112
- *NOTE*: Plezi app is not a Rails app, thus its memory is less than in Rails examples.
113
-
114
- # More Stress
115
-
116
- Peak connections: 20k.
117
- Concurrency: 8.
118
-
119
- ## Plezi (0.15.0) + Iodine (0.4.16)
120
-
121
- CPU (peak): ~90%
122
- CPU (avg): ~6%
123
- Mem: ~1100MB
124
-
125
-
126
- ```
127
- clients: 1000 95per-rtt: 216ms min-rtt: 1ms median-rtt: 17ms max-rtt: 364ms
128
- clients: 2000 95per-rtt: 446ms min-rtt: 0ms median-rtt: 24ms max-rtt: 738ms
129
- clients: 3000 95per-rtt: 348ms min-rtt: 0ms median-rtt: 46ms max-rtt: 925ms
130
- clients: 4000 95per-rtt: 654ms min-rtt: 2ms median-rtt: 65ms max-rtt: 1175ms
131
- clients: 5000 95per-rtt: 590ms min-rtt: 0ms median-rtt: 67ms max-rtt: 1842ms
132
- clients: 6000 95per-rtt: 469ms min-rtt: 1ms median-rtt: 121ms max-rtt: 1098ms
133
- clients: 7000 95per-rtt: 705ms min-rtt: 1ms median-rtt: 121ms max-rtt: 1617ms
134
- clients: 8000 95per-rtt: 1090ms min-rtt: 1ms median-rtt: 159ms max-rtt: 3253ms
135
- clients: 9000 95per-rtt: 869ms min-rtt: 1ms median-rtt: 163ms max-rtt: 3945ms
136
- clients: 10000 95per-rtt: 1171ms min-rtt: 1ms median-rtt: 247ms max-rtt: 2326ms
137
- clients: 11000 95per-rtt: 1634ms min-rtt: 1ms median-rtt: 191ms max-rtt: 4383ms
138
- clients: 12000 95per-rtt: 1138ms min-rtt: 2ms median-rtt: 283ms max-rtt: 8319ms
139
- clients: 13000 95per-rtt: 1234ms min-rtt: 1ms median-rtt: 306ms max-rtt: 5429ms
140
- clients: 14000 95per-rtt: 947ms min-rtt: 1ms median-rtt: 346ms max-rtt: 7804ms
141
- clients: 15000 95per-rtt: 1716ms min-rtt: 1ms median-rtt: 344ms max-rtt: 7372ms
142
- clients: 16000 95per-rtt: 1999ms min-rtt: 1ms median-rtt: 364ms max-rtt: 3047ms
143
- clients: 17000 95per-rtt: 1729ms min-rtt: 1ms median-rtt: 410ms max-rtt: 7231ms
144
- clients: 18000 95per-rtt: 1455ms min-rtt: 1ms median-rtt: 442ms max-rtt: 4725ms
145
- clients: 19000 95per-rtt: 2006ms min-rtt: 1ms median-rtt: 456ms max-rtt: 3811ms
146
- clients: 20000 95per-rtt: 2923ms min-rtt: 1ms median-rtt: 378ms max-rtt: 11954ms
147
-
148
- Missing received broadcasts: expected 23100000, got 23089387
149
- ```
150
-
151
- ## Anycable-Go 0.6.0-preview1
152
-
153
- CPU (peak): ~60%
154
- CPU (avg): ~4%
155
- Mem: ~1000MB
156
-
157
- ```
158
- clients: 1000 95per-rtt: 173ms min-rtt: 2ms median-rtt: 21ms max-rtt: 617ms
159
- clients: 2000 95per-rtt: 311ms min-rtt: 2ms median-rtt: 39ms max-rtt: 594ms
160
- clients: 3000 95per-rtt: 386ms min-rtt: 3ms median-rtt: 91ms max-rtt: 592ms
161
- clients: 4000 95per-rtt: 539ms min-rtt: 2ms median-rtt: 93ms max-rtt: 973ms
162
- clients: 5000 95per-rtt: 534ms min-rtt: 2ms median-rtt: 128ms max-rtt: 1486ms
163
- clients: 6000 95per-rtt: 1092ms min-rtt: 2ms median-rtt: 110ms max-rtt: 3073ms
164
- clients: 7000 95per-rtt: 726ms min-rtt: 3ms median-rtt: 185ms max-rtt: 1116ms
165
- clients: 8000 95per-rtt: 1078ms min-rtt: 2ms median-rtt: 174ms max-rtt: 3296ms
166
- clients: 9000 95per-rtt: 1401ms min-rtt: 8ms median-rtt: 195ms max-rtt: 6838ms
167
- clients: 10000 95per-rtt: 1157ms min-rtt: 3ms median-rtt: 282ms max-rtt: 3446ms
168
- clients: 11000 95per-rtt: 1804ms min-rtt: 4ms median-rtt: 271ms max-rtt: 5640ms
169
- clients: 12000 95per-rtt: 1339ms min-rtt: 2ms median-rtt: 344ms max-rtt: 2251ms
170
- clients: 13000 95per-rtt: 1638ms min-rtt: 3ms median-rtt: 336ms max-rtt: 3341ms
171
- clients: 14000 95per-rtt: 1811ms min-rtt: 3ms median-rtt: 339ms max-rtt: 2079ms
172
- clients: 15000 95per-rtt: 1407ms min-rtt: 2ms median-rtt: 435ms max-rtt: 6666ms
173
- clients: 16000 95per-rtt: 2796ms min-rtt: 2ms median-rtt: 398ms max-rtt: 4769ms
174
- clients: 17000 95per-rtt: 2494ms min-rtt: 6ms median-rtt: 338ms max-rtt: 36085ms
175
- clients: 18000 95per-rtt: 2844ms min-rtt: 2ms median-rtt: 374ms max-rtt: 7384ms
176
- clients: 19000 95per-rtt: 4010ms min-rtt: 17ms median-rtt: 370ms max-rtt: 5504ms
177
- clients: 20000 95per-rtt: 2819ms min-rtt: 2ms median-rtt: 470ms max-rtt: 8598ms
178
-
179
- Missing received broadcasts: expected 23100000, got 23065521
180
- ```
181
-
182
- # Idle Connections Benchmark
183
-
184
- Connection 20k idle clients at 200 connections per second rate; measuring final memory usage.
185
-
186
- Using [cable_bench](https://github.com/palkan/cable_bench).
187
-
188
- **Action Cable (8 workers)**: ~1500 MB
189
-
190
- **Anycable-Go 0.6.0-preview1**: ~700 MB
191
-
192
- **Plezi (0.15.0) + Iodine (0.4.16)**: ~800 MB (a lot of connection timeout errors and "The connection was lost" errors).
data/benchmarks/2018-05-27-rpc-bench.md DELETED
@@ -1,57 +0,0 @@
1
- # AnyCable RPC benchmarks
2
-
3
- AnyCable RPC server benchmarks.
4
-
5
- The benchmark is pretty simple:
6
- - Create N concurrent _clients_ (authenticate and subscribe to `BenchmarkChannel` within the RPC app)
7
- - Every client starts performing an action (`echo`) in a loop
8
- - Every call is collected (request time)
9
- - Wait for the specified amount of seconds and aggragate the results.
10
-
11
- Measuring **requests per seconds**.
12
-
13
- **NOTE:** this benchmark shows performance characteristics for both _clients_ and the server.
14
- The purpose of this benchmark is to measure the thoughout of RPC connection between AnyCable services.
15
-
16
- **NOTE 2:** "Noop RPC" is just a gRPC handling doing nothing except from responding with success result.
17
-
18
- Sources:
19
- - [Golang](https://github.com/anycable/anycable-go/blob/chore/benchmarking/cmd/rpc-bench/main.go)
20
- - [Erlang](https://github.com/anycable/simple-cable-app/tree/master/benchmarks)
21
-
22
- ## Erlang single connection
23
-
24
- RPC type \ Number of clients | 1 | 10 | 50
25
- --------------------------------|------|------|----
26
- AnyCable RPC | 100 | 110 | 130
27
- Noop RPC | 100 | 115 | 125
28
-
29
- Erlang client definitely has problems(
30
-
31
- Looks like HTTP2 streaming is working in _blocking mode_.
32
-
33
- Requires investigation.
34
-
35
- ## Erlycable connection pool (5-50)
36
-
37
- RPC type \ Number of clients | 1 | 10 | 50
38
- --------------------------------|------|------|----
39
- AnyCable RPC | 100 | 800 | 1700
40
- Noop RPC | 100 | 900 | 2700
41
-
42
- ## Golang single connection
43
-
44
- RPC type \ Number of clients | 1 | 10 | 50
45
- --------------------------------|------|------|----
46
- AnyCable RPC | 1600 | 2200 | 2400
47
- Noop RPC with connection object building\* | 2200 | 3500 | 4000
48
- Noop RPC | 3000 | 4600 | 5500
49
-
50
- \* Only build Action Cable connection object without performing an action.
51
-
52
- ## Golang connection pool (5-50)
53
-
54
- RPC type \ Number of clients | 1 | 10 | 50
55
- --------------------------------|------|------|----
56
- AnyCable RPC | 1400 | 1500 | 1700
57
- Noop RPC | 2100 | 3200 | 4000
data/benchmarks/2018-10-27.md DELETED
@@ -1,181 +0,0 @@
1
- # WebSocket Shootout Benchmark (2018-10-27)
2
-
3
- Code: https://github.com/palkan/websocket-shootout
4
-
5
- Client/Server instances: c3.2xlarge (8 vCPU, 15 GiB RAM).
6
-
7
- Ruby: 2.5.1p57 (2018-03-29 revision 63029) [x86_64-darwin16]
8
- Rails: 5.2.0 / 6.0.0.aplha (this [PR](https://github.com/rails/rails/pull/33295))
9
-
10
- The benchmark measures broadcasting RTT for 10k clients and the memory usage.
11
-
12
- Memory usage charts were captured with [psrecord](https://github.com/astrofrog/psrecord):
13
-
14
- ```sh
15
- psrecord <pid> --include-children --plot plot.png
16
- ```
17
-
18
- ## Action Cable + Puma (8 workers)
19
-
20
- Server: `WEB_CONCURRENCY=8 bundle exec rails s -p 3334 -e production`
21
-
22
- ```
23
- $ bin/websocket-bench broadcast --concurrent 8 --sample-size 100 --step-size 1000 --payload-padding 200 --total-steps 10 --origin http://0.0.0.0 ws://172.31.21.207:3334/cable --server-type=actioncable
24
-
25
- clients: 1000 95per-rtt: 2470ms min-rtt: 1ms median-rtt: 518ms max-rtt: 2893ms
26
- clients: 2000 95per-rtt: 3066ms min-rtt: 1ms median-rtt: 1045ms max-rtt: 3878ms
27
- clients: 3000 95per-rtt: 3785ms min-rtt: 1ms median-rtt: 2071ms max-rtt: 4211ms
28
- clients: 4000 95per-rtt: 5141ms min-rtt: 1ms median-rtt: 2363ms max-rtt: 6042ms
29
- clients: 5000 95per-rtt: 9477ms min-rtt: 1ms median-rtt: 3283ms max-rtt: 10000ms
30
- clients: 6000 95per-rtt: 7038ms min-rtt: 1ms median-rtt: 3898ms max-rtt: 8684ms
31
- clients: 7000 95per-rtt: 8517ms min-rtt: 2ms median-rtt: 4108ms max-rtt: 8436ms
32
- clients: 8000 95per-rtt: 9105ms min-rtt: 2ms median-rtt: 5387ms max-rtt: 9166ms
33
- clients: 9000 95per-rtt: 10064ms min-rtt: 1ms median-rtt: 5335ms max-rtt: 10108ms
34
- clients: 10000 95per-rtt: 15766ms min-rtt: 2ms median-rtt: 5921ms max-rtt: 17907ms
35
- ```
36
-
37
- ![](assets/2018-10-27-action-cable-rtt.png)
38
-
39
- ![](assets/2018-10-27-action-cable-rss.png)
40
-
41
- ## Action Cable + Iodine (8 workers & 16 threads)
42
-
43
- Server: `bundle exec iodine -p 3334 -w 8 -t 16`
44
-
45
- ```
46
- $ bin/websocket-bench broadcast --concurrent 8 --sample-size 100 --step-size 1000 --payload-padding 200 --total-steps 10 --origin http://0.0.0.0 ws://172.31.21.207:3334/cable --server-type=actioncable
47
-
48
- clients: 1000 95per-rtt: 2354ms min-rtt: 1ms median-rtt: 798ms max-rtt: 2944ms
49
- clients: 2000 95per-rtt: 3199ms min-rtt: 2ms median-rtt: 2068ms max-rtt: 3603ms
50
- clients: 3000 95per-rtt: 4808ms min-rtt: 2ms median-rtt: 3226ms max-rtt: 6016ms
51
- clients: 4000 95per-rtt: 6628ms min-rtt: 1ms median-rtt: 4511ms max-rtt: 7520ms
52
- clients: 5000 95per-rtt: 7761ms min-rtt: 1ms median-rtt: 5592ms max-rtt: 8828ms
53
- clients: 6000 95per-rtt: 17042ms min-rtt: 1ms median-rtt: 4557ms max-rtt: 18245ms
54
- clients: 7000 95per-rtt: 12331ms min-rtt: 1ms median-rtt: 8355ms max-rtt: 13525ms
55
- clients: 8000 95per-rtt: 16785ms min-rtt: 2ms median-rtt: 6217ms max-rtt: 17609ms
56
- clients: 9000 95per-rtt: 18438ms min-rtt: 1ms median-rtt: 1111ms max-rtt: 18848ms
57
- clients: 10000 95per-rtt: 19034ms min-rtt: 1ms median-rtt: 11235ms max-rtt: 24343ms
58
- ```
59
-
60
- ![](assets/2018-10-27-iodine-cable-rtt.png)
61
-
62
- ![](assets/2018-10-27-iodine-cable-rss.png)
63
-
64
-
65
- ## Action Cable + Falcon (8 workers)
66
-
67
- Server: `RAILS_ENV=production bundle exec falcon serve -b http://0.0.0.0:3334`
68
-
69
- ```
70
- $ bin/websocket-bench broadcast --concurrent 8 --sample-size 100 --step-size 1000 --payload-padding 200 --total-steps 10 --origin http://0.0.0.0 ws://172.31.21.207:3334/cable --server-type=actioncable
71
-
72
-
73
- clients: 1000 95per-rtt: 1957ms min-rtt: 1ms median-rtt: 766ms max-rtt: 2077ms
74
- clients: 2000 95per-rtt: 5225ms min-rtt: 1ms median-rtt: 624ms max-rtt: 5969ms
75
- clients: 3000 95per-rtt: 6358ms min-rtt: 2ms median-rtt: 1205ms max-rtt: 7116ms
76
- clients: 4000 95per-rtt: 8878ms min-rtt: 1ms median-rtt: 1152ms max-rtt: 10167ms
77
- clients: 5000 95per-rtt: 8829ms min-rtt: 1ms median-rtt: 2368ms max-rtt: 9792ms
78
- clients: 6000 95per-rtt: 10831ms min-rtt: 1ms median-rtt: 2892ms max-rtt: 12766ms
79
- clients: 7000 95per-rtt: 12760ms min-rtt: 1ms median-rtt: 5229ms max-rtt: 14713ms
80
- clients: 8000 95per-rtt: 15512ms min-rtt: 1ms median-rtt: 4843ms max-rtt: 16353ms
81
- clients: 9000 95per-rtt: 20176ms min-rtt: 1ms median-rtt: 4919ms max-rtt: 22529ms
82
- clients: 10000 95per-rtt: 19865ms min-rtt: 1ms median-rtt: 4733ms max-rtt: 20686ms
83
- ```
84
-
85
- ![](assets/2018-10-27-falcon-cable-rtt.png)
86
-
87
- ![](assets/2018-10-27-falcon-cable-rss.png)
88
-
89
- **NOTE**: for some reason `psrecord` reports negative RSS usage when Falcon starts (almost every run).
90
-
91
- ## Action Cable + AnyCable-Go 0.6.0.alpha
92
-
93
- **NOTE**: this benchmark has twice more steps (i.e. 20k instead of 10k).
94
-
95
- Server: `ANYCABLE_GO_BIN="anycable-go-0.6.0-alpha" ANYCABLE_PORT="3334" bundle exec bin/anycable`
96
-
97
- ```
98
- $ bin/websocket-bench broadcast --concurrent 8 --sample-size 100 --step-size 1000 --payload-padding 200 --total-steps 10 --origin http://0.0.0.0 ws://172.31.21.207:3334/cable --server-type=actioncable
99
-
100
- clients: 1000 95per-rtt: 200ms min-rtt: 4ms median-rtt: 30ms max-rtt: 261ms
101
- clients: 2000 95per-rtt: 284ms min-rtt: 2ms median-rtt: 51ms max-rtt: 380ms
102
- clients: 3000 95per-rtt: 490ms min-rtt: 5ms median-rtt: 80ms max-rtt: 651ms
103
- clients: 4000 95per-rtt: 601ms min-rtt: 3ms median-rtt: 63ms max-rtt: 2114ms
104
- clients: 5000 95per-rtt: 678ms min-rtt: 11ms median-rtt: 123ms max-rtt: 2946ms
105
- clients: 6000 95per-rtt: 933ms min-rtt: 12ms median-rtt: 146ms max-rtt: 1815ms
106
- clients: 7000 95per-rtt: 1034ms min-rtt: 17ms median-rtt: 163ms max-rtt: 4282ms
107
- clients: 8000 95per-rtt: 866ms min-rtt: 5ms median-rtt: 205ms max-rtt: 1631ms
108
- clients: 9000 95per-rtt: 1297ms min-rtt: 8ms median-rtt: 230ms max-rtt: 2380ms
109
- clients: 10000 95per-rtt: 1075ms min-rtt: 4ms median-rtt: 363ms max-rtt: 2718ms
110
- clients: 11000 95per-rtt: 1382ms min-rtt: 2ms median-rtt: 263ms max-rtt: 4310ms
111
- clients: 12000 95per-rtt: 1448ms min-rtt: 4ms median-rtt: 205ms max-rtt: 1686ms
112
- clients: 13000 95per-rtt: 1503ms min-rtt: 11ms median-rtt: 322ms max-rtt: 1879ms
113
- clients: 14000 95per-rtt: 2662ms min-rtt: 4ms median-rtt: 318ms max-rtt: 5604ms
114
- clients: 15000 95per-rtt: 2113ms min-rtt: 7ms median-rtt: 353ms max-rtt: 2484ms
115
- clients: 16000 95per-rtt: 2930ms min-rtt: 3ms median-rtt: 383ms max-rtt: 4021ms
116
- clients: 17000 95per-rtt: 3349ms min-rtt: 2ms median-rtt: 386ms max-rtt: 5844ms
117
- clients: 18000 95per-rtt: 2797ms min-rtt: 4ms median-rtt: 361ms max-rtt: 6973ms
118
- clients: 19000 95per-rtt: 3534ms min-rtt: 6ms median-rtt: 448ms max-rtt: 5482ms
119
- clients: 20000 95per-rtt: 2892ms min-rtt: 3ms median-rtt: 399ms max-rtt: 4639ms
120
- ```
121
-
122
- ![](assets/2018-10-27-anycable-rtt.png)
123
-
124
- ![](assets/2018-10-27-anycable-rss.png)
125
-
126
- **NOTE:** RSS were captured only for `anycable-go`
127
-
128
- ## Falcon + `async-websocket` (4 workers)
129
-
130
- ```
131
- $ bin/websocket-bench broadcast --concurrent 8 --sample-size 100 --step-size 1000 --payload-padding 200 --total-steps 10 ws://172.31.21.207:3334/cable
132
-
133
- clients: 1000 95per-rtt: 964ms min-rtt: 5ms median-rtt: 542ms max-rtt: 971ms
134
- clients: 2000 95per-rtt: 4912ms min-rtt: 1ms median-rtt: 2796ms max-rtt: 5247ms
135
- clients: 3000 95per-rtt: 3263ms min-rtt: 6ms median-rtt: 891ms max-rtt: 4166ms
136
- clients: 4000 95per-rtt: 1778ms min-rtt: 1ms median-rtt: 490ms max-rtt: 2115ms
137
- clients: 5000 95per-rtt: 2294ms min-rtt: 5ms median-rtt: 513ms max-rtt: 2807ms
138
- clients: 6000 95per-rtt: 2802ms min-rtt: 9ms median-rtt: 426ms max-rtt: 3263ms
139
- clients: 7000 95per-rtt: 4196ms min-rtt: 7ms median-rtt: 485ms max-rtt: 5377ms
140
- clients: 8000 95per-rtt: 4741ms min-rtt: 4ms median-rtt: 772ms max-rtt: 5005ms
141
- clients: 9000 95per-rtt: 6207ms min-rtt: 2ms median-rtt: 972ms max-rtt: 7296ms
142
- clients: 10000 95per-rtt: 5468ms min-rtt: 35ms median-rtt: 887ms max-rtt: 6420ms
143
- ```
144
-
145
- ![](assets/2018-10-27-async-rtt.png)
146
-
147
- ![](assets/2018-10-27-async-rss.png)
148
-
149
-
150
- ## Iodine + Plezi (4 workers & 16 threads)
151
-
152
- **NOTE**: this benchmark has twice more steps (i.e. 20k instead of 10k).
153
-
154
- ```
155
- $ bin/websocket-bench broadcast --concurrent 8 --sample-size 100 --step-size 1000 --payload-padding 200 --total-steps 10 ws://172.31.21.207:3334/cable
156
-
157
- clients: 1000 95per-rtt: 119ms min-rtt: 0ms median-rtt: 19ms max-rtt: 348ms
158
- clients: 2000 95per-rtt: 321ms min-rtt: 1ms median-rtt: 45ms max-rtt: 634ms
159
- clients: 3000 95per-rtt: 392ms min-rtt: 0ms median-rtt: 66ms max-rtt: 1543ms
160
- clients: 4000 95per-rtt: 424ms min-rtt: 1ms median-rtt: 128ms max-rtt: 617ms
161
- clients: 5000 95per-rtt: 747ms min-rtt: 1ms median-rtt: 126ms max-rtt: 1252ms
162
- clients: 6000 95per-rtt: 659ms min-rtt: 0ms median-rtt: 153ms max-rtt: 1621ms
163
- clients: 7000 95per-rtt: 839ms min-rtt: 0ms median-rtt: 205ms max-rtt: 1041ms
164
- clients: 8000 95per-rtt: 995ms min-rtt: 1ms median-rtt: 221ms max-rtt: 1534ms
165
- clients: 9000 95per-rtt: 1072ms min-rtt: 0ms median-rtt: 234ms max-rtt: 5679ms
166
- clients: 10000 95per-rtt: 1030ms min-rtt: 0ms median-rtt: 298ms max-rtt: 3998ms
167
- clients: 11000 95per-rtt: 1052ms min-rtt: 1ms median-rtt: 302ms max-rtt: 5539ms
168
- clients: 12000 95per-rtt: 1017ms min-rtt: 0ms median-rtt: 332ms max-rtt: 5342ms
169
- clients: 13000 95per-rtt: 1528ms min-rtt: 1ms median-rtt: 391ms max-rtt: 2488ms
170
- clients: 14000 95per-rtt: 1411ms min-rtt: 1ms median-rtt: 418ms max-rtt: 1810ms
171
- clients: 15000 95per-rtt: 1674ms min-rtt: 1ms median-rtt: 428ms max-rtt: 2154ms
172
- clients: 16000 95per-rtt: 1436ms min-rtt: 1ms median-rtt: 463ms max-rtt: 7788ms
173
- clients: 17000 95per-rtt: 1816ms min-rtt: 0ms median-rtt: 449ms max-rtt: 7299ms
174
- clients: 18000 95per-rtt: 2025ms min-rtt: 1ms median-rtt: 492ms max-rtt: 4009ms
175
- clients: 19000 95per-rtt: 1740ms min-rtt: 2ms median-rtt: 522ms max-rtt: 3834ms
176
- clients: 20000 95per-rtt: 2017ms min-rtt: 2ms median-rtt: 591ms max-rtt: 3579ms
177
- ```
178
-
179
- ![](assets/2018-10-27-plezi-rtt.png)
180
-
181
- ![](assets/2018-10-27-plezi-rss.png)
data/benchmarks/HowTo.md DELETED
@@ -1,23 +0,0 @@
1
- ## Hot to run benchmarks
2
-
3
- ### Installation
4
-
5
- - Install [Ansible](http://ansible.com) (`brew install ansible`)
6
-
7
- - Launch two EC2 instances from `anycable-benchmark-xxx` AMI
8
-
9
- - Attach more local network interfaces to the _client_ instance
10
-
11
- - Update `hosts` file and playbooks with required information (IPs, etc)
12
-
13
- - Prepare the _client_ instance: `ansible-playbook --tags prepare benchmark.yml`
14
-
15
- ### Running benchmarks
16
-
17
- To run a server, e.g. Action Cable: `ansible-playbook --tags action_cable servers.yml`
18
-
19
- To run a benchmark against it: `ansible-playbook --tags action_cable benchmark.yml`
20
-
21
- You can also specify benchmark parameters: `ansible-playbook --extra-vars "step_size=1000 steps=10 sample_size=10" benchmark.yml`.
22
-
23
- **NOTE**: Ansible doesn't support command output streaming, so we can only see the results at the end of the run.