bbperf 0.0.4__tar.gz → 0.0.5__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (27) hide show
  1. {bbperf-0.0.4 → bbperf-0.0.5}/PKG-INFO +25 -11
  2. {bbperf-0.0.4 → bbperf-0.0.5}/README.md +24 -10
  3. {bbperf-0.0.4 → bbperf-0.0.5}/pyproject.toml +1 -1
  4. {bbperf-0.0.4 → bbperf-0.0.5}/src/bbperf/output.py +4 -1
  5. {bbperf-0.0.4 → bbperf-0.0.5}/.gitignore +0 -0
  6. {bbperf-0.0.4 → bbperf-0.0.5}/DEPLOY.txt +0 -0
  7. {bbperf-0.0.4 → bbperf-0.0.5}/LICENSE +0 -0
  8. {bbperf-0.0.4 → bbperf-0.0.5}/src/bbperf/.gitignore +0 -0
  9. {bbperf-0.0.4 → bbperf-0.0.5}/src/bbperf/__init__.py +0 -0
  10. {bbperf-0.0.4 → bbperf-0.0.5}/src/bbperf/bbperf.py +0 -0
  11. {bbperf-0.0.4 → bbperf-0.0.5}/src/bbperf/calibration.py +0 -0
  12. {bbperf-0.0.4 → bbperf-0.0.5}/src/bbperf/client.py +0 -0
  13. {bbperf-0.0.4 → bbperf-0.0.5}/src/bbperf/const.py +0 -0
  14. {bbperf-0.0.4 → bbperf-0.0.5}/src/bbperf/control_receiver_thread.py +0 -0
  15. {bbperf-0.0.4 → bbperf-0.0.5}/src/bbperf/data_receiver_thread.py +0 -0
  16. {bbperf-0.0.4 → bbperf-0.0.5}/src/bbperf/data_sender_thread.py +0 -0
  17. {bbperf-0.0.4 → bbperf-0.0.5}/src/bbperf/data_udp_ping_sender_thread.py +0 -0
  18. {bbperf-0.0.4 → bbperf-0.0.5}/src/bbperf/exceptions.py +0 -0
  19. {bbperf-0.0.4 → bbperf-0.0.5}/src/bbperf/graph.py +0 -0
  20. {bbperf-0.0.4 → bbperf-0.0.5}/src/bbperf/json_output_class.py +0 -0
  21. {bbperf-0.0.4 → bbperf-0.0.5}/src/bbperf/server.py +0 -0
  22. {bbperf-0.0.4 → bbperf-0.0.5}/src/bbperf/tcp-graph.gp +0 -0
  23. {bbperf-0.0.4 → bbperf-0.0.5}/src/bbperf/tcp_control_connection_class.py +0 -0
  24. {bbperf-0.0.4 → bbperf-0.0.5}/src/bbperf/udp-graph.gp +0 -0
  25. {bbperf-0.0.4 → bbperf-0.0.5}/src/bbperf/udp_rate_manager_class.py +0 -0
  26. {bbperf-0.0.4 → bbperf-0.0.5}/src/bbperf/util.py +0 -0
  27. {bbperf-0.0.4 → bbperf-0.0.5}/tests/run_test.sh +0 -0
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: bbperf
3
- Version: 0.0.4
3
+ Version: 0.0.5
4
4
  Summary: An end-to-end performance and bufferbloat measurement tool
5
5
  Project-URL: Homepage, https://github.com/cloudflare/bbperf
6
6
  Project-URL: Issues, https://github.com/cloudflare/bbperf/issues
@@ -18,27 +18,41 @@ Description-Content-Type: text/markdown
18
18
 
19
19
  <p align="center"><strong>bbperf</strong> <em>- An end-to-end performance and bufferbloat measurement tool</em></p>
20
20
 
21
- `bbperf` measures the following for both TCP and UDP:
21
+ `bbperf` measures what matters most.
22
22
 
23
- * End-to-end latency, both unloaded and loaded
24
- * Throughput
25
- * Bandwidth Delay Product (BDP)
26
- * Usage of buffers between the endpoints
27
- * Bufferbloat (when the usage of buffers is excessive)
23
+ Traditional network performance measurement tools collect metrics such as latency and throughput regardless of the conditions that exist during the collection period. While valuable for many uses, that approach can miss reporting the actual performance that real user payloads experience on production networks. This tool only reports performance metrics when the flow is operating at "max buffer usage". Max buffer usage is when the active flow has filled any and all buffers that exist along the packet path between the endpoints.
24
+
25
+ User payload is used to measure latency and throughput. This accounts for the performance impact of transparent proxies, transparent tunnels, transparent firewalls, and all the other things that are not visible to the endpoints. It also simplifies the interpretation of retransmissions on user performance, which is non-intuitive at best. This is because some retransmissions are due to the real loss of user payload while many are not. In this tool, the loss of user payload will show up in the latency and throughput metrics, i.e. higher latencies and lower throughput.
28
26
 
29
- Features that distinguish this tool from existing tools include:
27
+ Features:
30
28
 
31
29
  * Latency, both unloaded and loaded, is measured by the same flow that is under test.
32
30
 
33
31
  Other tools will commonly measure latency using a different flow or different protocol. One of the reasons why using different protocols and/or different flows is not desirable is because fair queuing will cause the latency of those other flows to be much lower (better) than the flow that matters.
34
32
 
33
+ * Throughput
34
+
35
+ Both sender and receiver rates are collected, but the receiver rate (a.k.a. goodput) is the important one.
36
+
35
37
  * Bufferbloat is calculated
36
38
 
37
39
  It is often assumed that TCP receive buffers are the only source of bufferbloat. While that is common, it misses many other locations where bufferbloat may occur. This tool reports the effects of all sources of bufferbloat, not just TCP receive buffers.
38
40
 
39
- * Automatic generation of graphs
41
+ `bbperf` calculates both the BDP (bandwidth delay product) and the total amount of buffer actually used. The difference between those two is reported as "excess buffer usage". A small number for this metric is normal and expected, but a large number, relative to BDP, is bufferbloat. Bufferbloat also appears as a large difference between unloaded and loaded latency.
42
+
43
+ * Both TCP and UDP are supported
40
44
 
41
- * The UDP option will automatically adjust the sending rate to just above the goodput rate so that any bufferbloat issues are measured.
45
+ Both benchmark tests will wait until it has reached "max buffer usage" before collecting metrics data. For TCP, it will wait for the sending and receiving rates to match. For UDP, the sending rate will be automatically adjusted to be just above the maximum packet rate without dropping packets before starting its metrics collection.
46
+
47
+ * `bbperf` measures the performance of data flow in one direction only.
48
+
49
+ Network routing can be asymmetric, bottleneck links are asymmetric, bufferbloat is asymmetric, all of which means that performance is asymmetric. `bbperf` allows us to see the asymmetry.
50
+
51
+ Data flow in `bbperf` is one way. The direction of data flow is from the client host to the server host (unless the `-R` option is specified). That is the direction being measured, and is what is reported in the metrics.
52
+
53
+ Latency is measured round trip, but the return traffic (from the data receiver back to the data sender) is low-volume and should not contribute any bufferbloat-related latency to the measurement. This cannot be guaranteed, in the same way that it cannot be guaranteed that the unloaded latency measurement does not contain any bufferbloat-induced latency. But it does ensure that no bufferbloat-induced latency is cause by `bbperf`s own flow.
54
+
55
+ * Automatic generation of graphs
42
56
 
43
57
  ### Usage
44
58
 
@@ -78,7 +92,7 @@ options:
78
92
  -R, --reverse data flow in download direction (server to client)
79
93
  -t, --time SECONDS duration of run in seconds
80
94
  -u, --udp run in UDP mode (default: TCP mode)
81
- -g, --graph generate graph (requires gnuplot)
95
+ -g, --graph generate graph (requires gnuplot on the client host)
82
96
  -k, --keep keep data file
83
97
  -J, --json-file JSON_FILE
84
98
  JSON output file
@@ -1,26 +1,40 @@
1
1
  <p align="center"><strong>bbperf</strong> <em>- An end-to-end performance and bufferbloat measurement tool</em></p>
2
2
 
3
- `bbperf` measures the following for both TCP and UDP:
3
+ `bbperf` measures what matters most.
4
4
 
5
- * End-to-end latency, both unloaded and loaded
6
- * Throughput
7
- * Bandwidth Delay Product (BDP)
8
- * Usage of buffers between the endpoints
9
- * Bufferbloat (when the usage of buffers is excessive)
5
+ Traditional network performance measurement tools collect metrics such as latency and throughput regardless of the conditions that exist during the collection period. While valuable for many uses, that approach can miss reporting the actual performance that real user payloads experience on production networks. This tool only reports performance metrics when the flow is operating at "max buffer usage". Max buffer usage is when the active flow has filled any and all buffers that exist along the packet path between the endpoints.
6
+
7
+ User payload is used to measure latency and throughput. This accounts for the performance impact of transparent proxies, transparent tunnels, transparent firewalls, and all the other things that are not visible to the endpoints. It also simplifies the interpretation of retransmissions on user performance, which is non-intuitive at best. This is because some retransmissions are due to the real loss of user payload while many are not. In this tool, the loss of user payload will show up in the latency and throughput metrics, i.e. higher latencies and lower throughput.
10
8
 
11
- Features that distinguish this tool from existing tools include:
9
+ Features:
12
10
 
13
11
  * Latency, both unloaded and loaded, is measured by the same flow that is under test.
14
12
 
15
13
  Other tools will commonly measure latency using a different flow or different protocol. One of the reasons why using different protocols and/or different flows is not desirable is because fair queuing will cause the latency of those other flows to be much lower (better) than the flow that matters.
16
14
 
15
+ * Throughput
16
+
17
+ Both sender and receiver rates are collected, but the receiver rate (a.k.a. goodput) is the important one.
18
+
17
19
  * Bufferbloat is calculated
18
20
 
19
21
  It is often assumed that TCP receive buffers are the only source of bufferbloat. While that is common, it misses many other locations where bufferbloat may occur. This tool reports the effects of all sources of bufferbloat, not just TCP receive buffers.
20
22
 
21
- * Automatic generation of graphs
23
+ `bbperf` calculates both the BDP (bandwidth delay product) and the total amount of buffer actually used. The difference between those two is reported as "excess buffer usage". A small number for this metric is normal and expected, but a large number, relative to BDP, is bufferbloat. Bufferbloat also appears as a large difference between unloaded and loaded latency.
24
+
25
+ * Both TCP and UDP are supported
22
26
 
23
- * The UDP option will automatically adjust the sending rate to just above the goodput rate so that any bufferbloat issues are measured.
27
+ Both benchmark tests will wait until it has reached "max buffer usage" before collecting metrics data. For TCP, it will wait for the sending and receiving rates to match. For UDP, the sending rate will be automatically adjusted to be just above the maximum packet rate without dropping packets before starting its metrics collection.
28
+
29
+ * `bbperf` measures the performance of data flow in one direction only.
30
+
31
+ Network routing can be asymmetric, bottleneck links are asymmetric, bufferbloat is asymmetric, all of which means that performance is asymmetric. `bbperf` allows us to see the asymmetry.
32
+
33
+ Data flow in `bbperf` is one way. The direction of data flow is from the client host to the server host (unless the `-R` option is specified). That is the direction being measured, and is what is reported in the metrics.
34
+
35
+ Latency is measured round trip, but the return traffic (from the data receiver back to the data sender) is low-volume and should not contribute any bufferbloat-related latency to the measurement. This cannot be guaranteed, in the same way that it cannot be guaranteed that the unloaded latency measurement does not contain any bufferbloat-induced latency. But it does ensure that no bufferbloat-induced latency is cause by `bbperf`s own flow.
36
+
37
+ * Automatic generation of graphs
24
38
 
25
39
  ### Usage
26
40
 
@@ -60,7 +74,7 @@ options:
60
74
  -R, --reverse data flow in download direction (server to client)
61
75
  -t, --time SECONDS duration of run in seconds
62
76
  -u, --udp run in UDP mode (default: TCP mode)
63
- -g, --graph generate graph (requires gnuplot)
77
+ -g, --graph generate graph (requires gnuplot on the client host)
64
78
  -k, --keep keep data file
65
79
  -J, --json-file JSON_FILE
66
80
  JSON output file
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
4
4
 
5
5
  [project]
6
6
  name = "bbperf"
7
- version = "0.0.4"
7
+ version = "0.0.5"
8
8
  authors = [
9
9
  { name="Mike Freemon", email="mfreemon@cloudflare.com" },
10
10
  ]
@@ -156,12 +156,15 @@ def print_output(s1):
156
156
  write_graph_data_to_file(lineout)
157
157
 
158
158
  # add to JSON output
159
+ excess = r_record["buffered_bytes"] - bdp_bytes
160
+ if excess < 0:
161
+ excess = 0
159
162
  new_entry = {
160
163
  "sent_time_sec": r_record["r_pkt_sent_time_sec"],
161
164
  "loaded_rtt_ms": r_record["rtt_ms"],
162
165
  "sender_throughput_rate_mbps": r_record["sender_interval_rate_mbps"],
163
166
  "receiver_throughput_rate_mbps": r_record["receiver_interval_rate_mbps"],
164
- "excess_buffered_bytes": (r_record["buffered_bytes"] - bdp_bytes),
167
+ "excess_buffered_bytes": excess,
165
168
  "receiver_pps": r_record["receiver_pps"],
166
169
  "pkt_loss_percent": dropped_this_interval_percent
167
170
  }
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes