@blinklabs/dingo 0.27.1 → 0.27.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/RELEASE_NOTES.md +73 -0
- package/benchmark_results_bp_pi.md +50 -0
- package/benchmark_results_targeted.md +73 -38
- package/generate_benchmarks.sh +1 -0
- package/generate_bp_pi_report.sh +359 -0
- package/package.json +1 -1
package/RELEASE_NOTES.md
CHANGED
|
@@ -1,6 +1,79 @@
|
|
|
1
1
|
# Release Notes
|
|
2
2
|
|
|
3
3
|
|
|
4
|
+
## v0.27.2 (March 16, 2026)
|
|
5
|
+
|
|
6
|
+
**Title:** Snapshot events and safer services
|
|
7
|
+
|
|
8
|
+
**Date:** March 16, 2026
|
|
9
|
+
|
|
10
|
+
**Version:** v0.27.2
|
|
11
|
+
|
|
12
|
+
Hi folks! Here’s what we shipped in v0.27.2.
|
|
13
|
+
|
|
14
|
+
### ✨ What's New
|
|
15
|
+
|
|
16
|
+
- **Snapshot event publishing and clean shutdown:** Relay operation is more reliable because the relay now publishes events from snapshots and shuts down cleanly without dropping in-flight work.
|
|
17
|
+
|
|
18
|
+
### 💪 Improvements
|
|
19
|
+
|
|
20
|
+
- **HTTP timeouts for public APIs:** Network-facing services are more resilient under slow or stalled connections thanks to new write/read/idle timeouts on the Bark, Blockfrost, and UTxO RPC HTTP servers.
|
|
21
|
+
- **Streamlined peer and connection management:** Peer and connection management is faster and uses fewer resources on constrained machines thanks to quicker inbound host lookups, tighter Badger cache defaults, and expanded benchmarks and sizing guidance.
|
|
22
|
+
- **More consistent key-period handling:** Block production key period handling is more consistent across configurations thanks to improved key-period calculations with added validation and tests.
|
|
23
|
+
- **Race detection in CI:** Test runs catch concurrency bugs earlier because the Linux test job now runs with the Go race detector enabled.
|
|
24
|
+
|
|
25
|
+
### 🔧 Fixes
|
|
26
|
+
|
|
27
|
+
- **Safer concurrent chain reads:** Reads are more consistent under load because primary chain and protocol-parameter access are now protected with read locks.
|
|
28
|
+
- **Validated epoch nonce reuse:** Nonce reuse is safer because cached epoch nonce entries are now validated against the nonce provided for the current run before reuse.
|
|
29
|
+
- **Graceful invalid hash handling:** Malformed block metadata no longer crashes encoding because previous-hash length issues now return errors instead of panicking.
|
|
30
|
+
- **SQLite VACUUM actually runs:** Database maintenance now completes as intended because SQLite VACUUM is now executed rather than only prepared.
|
|
31
|
+
|
|
32
|
+
### 📋 What You Need to Know
|
|
33
|
+
|
|
34
|
+
- **Release notes alignment:** Release documentation was updated to reflect the final set of changes, including a detailed v0.27.1 section in `RELEASE_NOTES.md`.
|
|
35
|
+
- **Build provenance updates:** Supply-chain attestations are easier to verify because build provenance now uses `actions/attest` and updated Docker Hub image subjects.
|
|
36
|
+
|
|
37
|
+
### 🙏 Thank You
|
|
38
|
+
|
|
39
|
+
Thank you for trying!
|
|
40
|
+
|
|
41
|
+
---
|
|
42
|
+
|
|
43
|
+
## v0.27.1 (March 16, 2026)
|
|
44
|
+
|
|
45
|
+
**Title:** Smoother reconnects and safer chain-sync
|
|
46
|
+
|
|
47
|
+
**Date:** March 16, 2026
|
|
48
|
+
|
|
49
|
+
**Version:** v0.27.1
|
|
50
|
+
|
|
51
|
+
Hi folks! Here’s what we shipped in v0.27.1.
|
|
52
|
+
|
|
53
|
+
### ✨ What's New
|
|
54
|
+
|
|
55
|
+
- **Better chain-sync intersection:** Resuming sync is easier and faster because chain-sync uses a denser, wider set of intersect points to improve `ChainSync` intersection behavior.
|
|
56
|
+
|
|
57
|
+
### 💪 Improvements
|
|
58
|
+
|
|
59
|
+
- **Inbound connection reuse and `TxSubmission`:** Networking is more rock-solid on reconnect because peer reuse and governance now normalize exact peer addresses, require client-capable connections for reuse, and start `TxSubmission` on duplex inbound connections.
|
|
60
|
+
- **Stake snapshot and epoch summary upserts:** Data storage is more consistent across supported databases because write paths now upsert across DB backends and report errors more clearly.
|
|
61
|
+
- **More robust delegation parsing:** Delegation reads are more reliable because parsing now handles multiple account encodings with expanded tests.
|
|
62
|
+
|
|
63
|
+
### 🔧 Fixes
|
|
64
|
+
|
|
65
|
+
- **Dependency refresh:** Upgrades are less error-prone because Go modules were refreshed (including AWS SDK v2/S3, `golang.org/x/*`, `plutigo` v0.0.27, `go-ethereum` v1.17.1, and `google.golang.org/api` v0.271.0).
|
|
66
|
+
|
|
67
|
+
### 📋 What You Need to Know
|
|
68
|
+
|
|
69
|
+
- **Go module sync (some builds):** You’re all set for most setups, but if you vendor dependencies or run reproducible builds you may need to re-sync Go modules (update `go.mod`/`go.sum`) to pick up refreshed versions.
|
|
70
|
+
|
|
71
|
+
### 🙏 Thank You
|
|
72
|
+
|
|
73
|
+
Thank you for trying!
|
|
74
|
+
|
|
75
|
+
---
|
|
76
|
+
|
|
4
77
|
## v0.27.0 (March 15, 2026)
|
|
5
78
|
|
|
6
79
|
**Title:** S3-backed CI tests and embedded network configs
|
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
# Four-Thread Block Producer Sizing Report (Non-Pi Host)
|
|
2
|
+
|
|
3
|
+
Assumptions:
|
|
4
|
+
- `storageMode=core`
|
|
5
|
+
- `runMode=serve`, `blob=badger`, `metadata=sqlite`
|
|
6
|
+
- `GOMAXPROCS=4`
|
|
7
|
+
- immutable fixture: `database/immutable/testdata`
|
|
8
|
+
- measurements were collected on non-Pi hardware and are a sizing proxy, not Raspberry Pi device benchmarks
|
|
9
|
+
- block producer peer count remains small; this is not a relay profile
|
|
10
|
+
|
|
11
|
+
## Load Footprint
|
|
12
|
+
|
|
13
|
+
| Profile | Elapsed | Peak RSS | Notes |
|
|
14
|
+
| --- | --- | --- | --- |
|
|
15
|
+
| `core-default` | `100.35s` | `568.1 MB` | Current default `serve/core` cache profile |
|
|
16
|
+
|
|
17
|
+
## Recommendation
|
|
18
|
+
|
|
19
|
+
For Raspberry Pi-class sizing in `storageMode=core`, the current default `serve/core` profile is already in the measured low-memory range on this four-thread non-Pi host.
|
|
20
|
+
Explicit Badger cache settings still override these defaults if an operator wants to tune further.
|
|
21
|
+
Treat this as a sizing baseline rather than a direct Raspberry Pi hardware measurement.
|
|
22
|
+
|
|
23
|
+
# Four-Thread Block Producer Sizing Benchmark Results (Non-Pi Host)
|
|
24
|
+
|
|
25
|
+
## Latest Results
|
|
26
|
+
|
|
27
|
+
### Test Environment
|
|
28
|
+
- **Date**: March 15, 2026
|
|
29
|
+
- **Go Version**: 1.25.8
|
|
30
|
+
- **OS**: Linux
|
|
31
|
+
- **Architecture**: aarch64
|
|
32
|
+
- **CPU Cores**: 128
|
|
33
|
+
- **Data Source**: database/immutable/testdata plus local immutable load runs
|
|
34
|
+
- **Scope**: Four-thread block producer path (GOMAXPROCS=4) in core mode on a non-Pi host
|
|
35
|
+
|
|
36
|
+
### Benchmark Results
|
|
37
|
+
|
|
38
|
+
All benchmarks run with `-benchmem` flag. Iterations are Go benchmark iteration counts; benchmark-specific throughput metrics (for example `blocks/sec`) are reported separately.
|
|
39
|
+
|
|
40
|
+
| Benchmark | Iterations | Time/op | Extra Metrics | Memory/op | Allocs/op |
|
|
41
|
+
|-----------|------------|---------|---------------|-----------|-----------|
|
|
42
|
+
| ledger:Blockfetch Near Tip Queued Header Predecoded | 155935 | 36391ns | 27479 blocks/sec | 2KB | 51 |
|
|
43
|
+
| ledger:Blockfetch Near Tip Throughput | 181371 | 66974ns | 14931 blocks/sec | 25KB | 128 |
|
|
44
|
+
| ledger:Blockfetch Near Tip Throughput Predecoded | 328588 | 36882ns | 27113 blocks/sec | 2KB | 50 |
|
|
45
|
+
| ledger:Verify Block Header/direct | 5946 | 1005264ns | - | 2KB | 29 |
|
|
46
|
+
| ledger:Verify Block Header/ledger_state | 6008 | 997998ns | - | 2KB | 29 |
|
|
47
|
+
|
|
48
|
+
## Performance Changes
|
|
49
|
+
|
|
50
|
+
No previous results found. This is the first benchmark run.
|
|
@@ -3,13 +3,13 @@
|
|
|
3
3
|
## Latest Results
|
|
4
4
|
|
|
5
5
|
### Test Environment
|
|
6
|
-
- **Date**: March
|
|
6
|
+
- **Date**: March 15, 2026
|
|
7
7
|
- **Go Version**: 1.25.8
|
|
8
8
|
- **OS**: Linux
|
|
9
9
|
- **Architecture**: aarch64
|
|
10
10
|
- **CPU Cores**: 128
|
|
11
|
-
- **Data Source**: database/immutable/testdata
|
|
12
|
-
- **Scope**: BP runtime
|
|
11
|
+
- **Data Source**: database/immutable/testdata plus synthetic relay fanout/control-loop benchmarks
|
|
12
|
+
- **Scope**: BP and relay runtime paths
|
|
13
13
|
|
|
14
14
|
### Benchmark Results
|
|
15
15
|
|
|
@@ -17,51 +17,86 @@ All benchmarks run with `-benchmem` flag. Iterations are Go benchmark iteration
|
|
|
17
17
|
|
|
18
18
|
| Benchmark | Iterations | Time/op | Extra Metrics | Memory/op | Allocs/op |
|
|
19
19
|
|-----------|------------|---------|---------------|-----------|-----------|
|
|
20
|
-
|
|
|
21
|
-
|
|
|
22
|
-
|
|
|
23
|
-
|
|
|
24
|
-
|
|
|
25
|
-
|
|
|
26
|
-
|
|
|
27
|
-
|
|
|
28
|
-
|
|
|
29
|
-
|
|
|
20
|
+
| connmanager:Has Inbound From Host/10 | 81306320 | 62.31ns | - | 0B | 0 |
|
|
21
|
+
| connmanager:Has Inbound From Host/100 | 94509072 | 70.91ns | - | 0B | 0 |
|
|
22
|
+
| connmanager:Has Inbound From Host/1000 | 92533866 | 64.39ns | - | 0B | 0 |
|
|
23
|
+
| connmanager:Has Inbound From Host/500 | 99476421 | 59.57ns | - | 0B | 0 |
|
|
24
|
+
| connmanager:Try Reserve Inbound Slot Parallel/10 | 15721854 | 388.8ns | - | 0B | 0 |
|
|
25
|
+
| connmanager:Try Reserve Inbound Slot Parallel/100 | 7024470 | 823.6ns | - | 0B | 0 |
|
|
26
|
+
| connmanager:Try Reserve Inbound Slot Parallel/500 | 7173524 | 840.6ns | - | 0B | 0 |
|
|
27
|
+
| connmanager:Update Connection Metrics/10 | 199656918 | 29.24ns | - | 0B | 0 |
|
|
28
|
+
| connmanager:Update Connection Metrics/100 | 203453912 | 29.82ns | - | 0B | 0 |
|
|
29
|
+
| connmanager:Update Connection Metrics/1000 | 203063875 | 30.09ns | - | 0B | 0 |
|
|
30
|
+
| connmanager:Update Connection Metrics/500 | 204545601 | 29.69ns | - | 0B | 0 |
|
|
31
|
+
| event:Publish Subscribers/1 | 7974799 | 650.6ns | - | 0B | 0 |
|
|
32
|
+
| event:Publish Subscribers/10 | 746738 | 7520ns | - | 0B | 0 |
|
|
33
|
+
| event:Publish Subscribers/100 | 61120 | 94657ns | - | 0B | 0 |
|
|
34
|
+
| event:Publish Subscribers/500 | 12976 | 489401ns | - | 0B | 0 |
|
|
35
|
+
| ledger:Block Processing Throughput | 54142 | 95123ns | 10513 blocks/sec | 25KB | 133 |
|
|
36
|
+
| ledger:Block Processing Throughput Predecoded | 144867 | 40441ns | 24728 blocks/sec | 2KB | 51 |
|
|
37
|
+
| ledger:Blockfetch Near Tip Throughput | 172838 | 69818ns | 14323 blocks/sec | 25KB | 129 |
|
|
38
|
+
| ledger:Blockfetch Near Tip Flush Only Predecoded | 342222 | 37234ns | 26857 blocks/sec | 2KB | 50 |
|
|
39
|
+
| ledger:Blockfetch Near Tip Throughput Predecoded | 340428 | 36428ns | 27452 blocks/sec | 2KB | 50 |
|
|
40
|
+
| ledger:Blockfetch Verified Header Dispatch | 156298824 | 38.37ns | - | 0B | 0 |
|
|
41
|
+
| ledger:Verify Block Header/direct | 5907 | 1008310ns | - | 2KB | 29 |
|
|
42
|
+
| ledger:Verify Block Header/ledger_state | 5935 | 1006660ns | - | 2KB | 29 |
|
|
43
|
+
| peergov:Reconcile/100 | 64309 | 93859ns | - | 19KB | 124 |
|
|
44
|
+
| peergov:Reconcile/1000 | 3626 | 1632793ns | - | 250KB | 1597 |
|
|
45
|
+
| peergov:Reconcile/500 | 7881 | 733098ns | - | 124KB | 697 |
|
|
46
|
+
|
|
30
47
|
## Performance Changes
|
|
31
48
|
|
|
32
|
-
Changes
|
|
49
|
+
Changes relative to the earlier **March 15, 2026** baseline below:
|
|
33
50
|
|
|
34
51
|
### Summary
|
|
35
|
-
-
|
|
36
|
-
-
|
|
37
|
-
-
|
|
38
|
-
-
|
|
52
|
+
- Focused timed reruns were used for the connmanager and peergov rows in this update.
|
|
53
|
+
- `Update Connection Metrics` now uses incremental cached counts instead of rescanning all connections.
|
|
54
|
+
- The default `serve/core` Badger profile now uses the previously measured Pi-tight cache sizing.
|
|
55
|
+
- `peergov.reconcile` still benefits heavily from keeping per-peer churn logs below `Info`.
|
|
56
|
+
- `peergov.reconcile` now reuses one reconcile timestamp and caches under-valency for warm-promotion candidates.
|
|
57
|
+
- `peergov.enforceStateLimit` now sorts only removable peers and rebuilds the slice once instead of repeatedly deleting from it.
|
|
58
|
+
- `peergov.reconcile` now skips valency-status scans entirely unless debug logging is enabled.
|
|
39
59
|
|
|
40
60
|
### Top Improvements
|
|
41
|
-
-
|
|
42
|
-
-
|
|
43
|
-
-
|
|
44
|
-
-
|
|
45
|
-
- ledger:
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
-
|
|
49
|
-
-
|
|
61
|
+
- connmanager:Update Connection Metrics/1000: `291625ns -> 30.09ns`
|
|
62
|
+
- connmanager:Update Connection Metrics/500: `138551ns -> 29.69ns`
|
|
63
|
+
- connmanager:Update Connection Metrics/100: `24518ns -> 29.82ns`
|
|
64
|
+
- connmanager:Update Connection Metrics/10: `3123ns -> 29.24ns`
|
|
65
|
+
- ledger:Blockfetch Near Tip Throughput: `86320ns -> 69818ns`
|
|
66
|
+
- ledger:Blockfetch Near Tip Throughput Predecoded: `40031ns -> 36428ns`
|
|
67
|
+
- peergov:Reconcile/1000: `2587803ns -> 1632793ns`
|
|
68
|
+
- peergov:Reconcile/500: `1072577ns -> 733098ns`
|
|
69
|
+
- peergov:Reconcile/100: `130667ns -> 93859ns`
|
|
50
70
|
|
|
51
71
|
|
|
52
72
|
## Historical Results
|
|
53
73
|
|
|
54
|
-
### March
|
|
74
|
+
### Earlier March 15, 2026 Baseline
|
|
55
75
|
|
|
56
76
|
| Benchmark | Iterations | Time/op | Extra Metrics | Memory/op | Allocs/op |
|
|
57
77
|
|-----------|------------|---------|---------------|-----------|-----------|
|
|
58
|
-
|
|
|
59
|
-
|
|
|
60
|
-
|
|
|
61
|
-
|
|
|
62
|
-
|
|
|
63
|
-
|
|
|
64
|
-
|
|
|
65
|
-
|
|
|
66
|
-
|
|
|
67
|
-
|
|
|
78
|
+
| connmanager:Has Inbound From Host/10 | 102497121 | 63.18ns | - | 0B | 0 |
|
|
79
|
+
| connmanager:Has Inbound From Host/100 | 93197125 | 57.95ns | - | 0B | 0 |
|
|
80
|
+
| connmanager:Has Inbound From Host/1000 | 99058254 | 60.09ns | - | 0B | 0 |
|
|
81
|
+
| connmanager:Has Inbound From Host/500 | 91016371 | 65.75ns | - | 0B | 0 |
|
|
82
|
+
| connmanager:Try Reserve Inbound Slot Parallel/10 | 15616777 | 403.8ns | - | 0B | 0 |
|
|
83
|
+
| connmanager:Try Reserve Inbound Slot Parallel/100 | 7449546 | 779.0ns | - | 0B | 0 |
|
|
84
|
+
| connmanager:Try Reserve Inbound Slot Parallel/500 | 7706102 | 772.6ns | - | 0B | 0 |
|
|
85
|
+
| connmanager:Update Connection Metrics/10 | 1876092 | 3123ns | - | 456B | 3 |
|
|
86
|
+
| connmanager:Update Connection Metrics/100 | 239965 | 24518ns | - | 3KB | 3 |
|
|
87
|
+
| connmanager:Update Connection Metrics/1000 | 20446 | 291625ns | - | 54KB | 5 |
|
|
88
|
+
| connmanager:Update Connection Metrics/500 | 43100 | 138551ns | - | 27KB | 3 |
|
|
89
|
+
| event:Publish Subscribers/1 | 7974799 | 650.6ns | - | 0B | 0 |
|
|
90
|
+
| event:Publish Subscribers/10 | 746738 | 7520ns | - | 0B | 0 |
|
|
91
|
+
| event:Publish Subscribers/100 | 61120 | 94657ns | - | 0B | 0 |
|
|
92
|
+
| event:Publish Subscribers/500 | 12976 | 489401ns | - | 0B | 0 |
|
|
93
|
+
| ledger:Block Processing Throughput | 54142 | 95123ns | 10513 blocks/sec | 25KB | 133 |
|
|
94
|
+
| ledger:Block Processing Throughput Predecoded | 144867 | 40441ns | 24728 blocks/sec | 2KB | 51 |
|
|
95
|
+
| ledger:Blockfetch Near Tip Throughput | 76536 | 86320ns | 11585 blocks/sec | 25KB | 128 |
|
|
96
|
+
| ledger:Blockfetch Near Tip Throughput Predecoded | 160242 | 40031ns | 24981 blocks/sec | 2KB | 50 |
|
|
97
|
+
| ledger:Blockfetch Verified Header Dispatch | 156298824 | 38.37ns | - | 0B | 0 |
|
|
98
|
+
| ledger:Verify Block Header/direct | 5907 | 1008310ns | - | 2KB | 29 |
|
|
99
|
+
| ledger:Verify Block Header/ledger_state | 5935 | 1006660ns | - | 2KB | 29 |
|
|
100
|
+
| peergov:Reconcile/100 | 45027 | 130667ns | - | 16KB | 185 |
|
|
101
|
+
| peergov:Reconcile/1000 | 2343 | 2587803ns | - | 202KB | 1662 |
|
|
102
|
+
| peergov:Reconcile/500 | 5739 | 1072577ns | - | 100KB | 760 |
|
package/generate_benchmarks.sh
CHANGED
|
@@ -0,0 +1,359 @@
|
|
|
1
|
+
#!/usr/bin/env bash
|
|
2
|
+
|
|
3
|
+
set -euo pipefail
|
|
4
|
+
|
|
5
|
+
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
6
|
+
OUTPUT_FILE="${1:-benchmark_results_bp_pi.md}"
|
|
7
|
+
CACHE_DIR="${CACHE_DIR:-/tmp/dingo-bp-pi-go-cache}"
|
|
8
|
+
GOMAXPROCS_VALUE="${GOMAXPROCS_VALUE:-4}"
|
|
9
|
+
BENCH_TIME="${BENCH_TIME:-5s}"
|
|
10
|
+
IMMUTABLE_PATH="${IMMUTABLE_PATH:-database/immutable/testdata}"
|
|
11
|
+
RSS_LIMIT_MB="${RSS_LIMIT_MB:-1024}"
|
|
12
|
+
GOMEMLIMIT_VALUE="${GOMEMLIMIT_VALUE:-}"
|
|
13
|
+
BLOCK_CACHE_SIZE_VALUE="${BLOCK_CACHE_SIZE_VALUE:-}"
|
|
14
|
+
INDEX_CACHE_SIZE_VALUE="${INDEX_CACHE_SIZE_VALUE:-}"
|
|
15
|
+
MEMTABLE_SIZE_VALUE="${MEMTABLE_SIZE_VALUE:-}"
|
|
16
|
+
DEFAULT_DB_DIR="${DEFAULT_DB_DIR:-}"
|
|
17
|
+
DEFAULT_LOG_FILE="${DEFAULT_LOG_FILE:-}"
|
|
18
|
+
DB_DIR_BASE="${DB_DIR_BASE:-/tmp}"
|
|
19
|
+
TMP_BENCH_FILE="$(mktemp)"
|
|
20
|
+
TIME_FORMAT_STYLE=""
|
|
21
|
+
TIME_CMD=()
|
|
22
|
+
AUTO_DEFAULT_DB_DIR=0
|
|
23
|
+
AUTO_DEFAULT_LOG_FILE=0
|
|
24
|
+
DINGO_CACHE_ARGS=()
|
|
25
|
+
|
|
26
|
+
cleanup() {
|
|
27
|
+
local exit_status=$?
|
|
28
|
+
|
|
29
|
+
rm -f "$TMP_BENCH_FILE"
|
|
30
|
+
if [[ "$exit_status" -ne 0 ]]; then
|
|
31
|
+
if [[ "$AUTO_DEFAULT_LOG_FILE" -eq 1 ]]; then
|
|
32
|
+
printf 'preserving profile log for debugging: %s\n' \
|
|
33
|
+
"$DEFAULT_LOG_FILE" >&2
|
|
34
|
+
fi
|
|
35
|
+
if [[ "$AUTO_DEFAULT_DB_DIR" -eq 1 ]]; then
|
|
36
|
+
printf 'preserving profile database for debugging: %s\n' \
|
|
37
|
+
"$DEFAULT_DB_DIR" >&2
|
|
38
|
+
fi
|
|
39
|
+
return "$exit_status"
|
|
40
|
+
fi
|
|
41
|
+
|
|
42
|
+
if [[ "$AUTO_DEFAULT_LOG_FILE" -eq 1 ]]; then
|
|
43
|
+
rm -f "$DEFAULT_LOG_FILE"
|
|
44
|
+
fi
|
|
45
|
+
if [[ "$AUTO_DEFAULT_DB_DIR" -eq 1 ]]; then
|
|
46
|
+
rm -rf "$DEFAULT_DB_DIR"
|
|
47
|
+
fi
|
|
48
|
+
return "$exit_status"
|
|
49
|
+
}
|
|
50
|
+
trap cleanup EXIT
|
|
51
|
+
|
|
52
|
+
create_temp_dir() {
|
|
53
|
+
mktemp -d "${DB_DIR_BASE%/}/dingo-bp-pi-db-default.XXXXXX"
|
|
54
|
+
}
|
|
55
|
+
|
|
56
|
+
create_temp_file() {
|
|
57
|
+
mktemp "${DB_DIR_BASE%/}/dingo-bp-pi-default.XXXXXX.log"
|
|
58
|
+
}
|
|
59
|
+
|
|
60
|
+
resolve_path() {
|
|
61
|
+
local path="$1"
|
|
62
|
+
local current
|
|
63
|
+
local candidate
|
|
64
|
+
local part
|
|
65
|
+
local -a parts=()
|
|
66
|
+
|
|
67
|
+
if [[ -z "$path" ]]; then
|
|
68
|
+
return 1
|
|
69
|
+
fi
|
|
70
|
+
|
|
71
|
+
if [[ "$path" == /* ]]; then
|
|
72
|
+
current="/"
|
|
73
|
+
else
|
|
74
|
+
current="$(pwd -P)"
|
|
75
|
+
fi
|
|
76
|
+
|
|
77
|
+
local IFS='/'
|
|
78
|
+
read -r -a parts <<< "$path"
|
|
79
|
+
for part in "${parts[@]}"; do
|
|
80
|
+
case "$part" in
|
|
81
|
+
""|".")
|
|
82
|
+
continue
|
|
83
|
+
;;
|
|
84
|
+
"..")
|
|
85
|
+
if [[ "$current" != "/" ]]; then
|
|
86
|
+
current="${current%/*}"
|
|
87
|
+
if [[ -z "$current" ]]; then
|
|
88
|
+
current="/"
|
|
89
|
+
fi
|
|
90
|
+
fi
|
|
91
|
+
;;
|
|
92
|
+
*)
|
|
93
|
+
if [[ "$current" == "/" ]]; then
|
|
94
|
+
candidate="/$part"
|
|
95
|
+
else
|
|
96
|
+
candidate="$current/$part"
|
|
97
|
+
fi
|
|
98
|
+
if [[ -d "$candidate" ]]; then
|
|
99
|
+
current="$(cd "$candidate" && pwd -P)"
|
|
100
|
+
else
|
|
101
|
+
current="$candidate"
|
|
102
|
+
fi
|
|
103
|
+
;;
|
|
104
|
+
esac
|
|
105
|
+
done
|
|
106
|
+
|
|
107
|
+
printf '%s\n' "$current"
|
|
108
|
+
}
|
|
109
|
+
|
|
110
|
+
configure_time_cmd() {
|
|
111
|
+
if command -v gtime >/dev/null 2>&1 &&
|
|
112
|
+
gtime -f "elapsed=%e rss_kb=%M" true >/dev/null 2>&1; then
|
|
113
|
+
TIME_FORMAT_STYLE="gnu"
|
|
114
|
+
TIME_CMD=(gtime -f "elapsed=%e rss_kb=%M")
|
|
115
|
+
return 0
|
|
116
|
+
fi
|
|
117
|
+
|
|
118
|
+
if /usr/bin/time -f "elapsed=%e rss_kb=%M" true >/dev/null 2>&1; then
|
|
119
|
+
TIME_FORMAT_STYLE="gnu"
|
|
120
|
+
TIME_CMD=(/usr/bin/time -f "elapsed=%e rss_kb=%M")
|
|
121
|
+
return 0
|
|
122
|
+
fi
|
|
123
|
+
|
|
124
|
+
if /usr/bin/time -l true >/dev/null 2>&1; then
|
|
125
|
+
TIME_FORMAT_STYLE="bsd"
|
|
126
|
+
TIME_CMD=(/usr/bin/time -l)
|
|
127
|
+
return 0
|
|
128
|
+
fi
|
|
129
|
+
|
|
130
|
+
echo "error: failed to find a supported time command" >&2
|
|
131
|
+
return 1
|
|
132
|
+
}
|
|
133
|
+
|
|
134
|
+
validate_db_dir() {
|
|
135
|
+
local db_dir="$1"
|
|
136
|
+
local log_file="$2"
|
|
137
|
+
local resolved_db_dir
|
|
138
|
+
local resolved_base_dir
|
|
139
|
+
|
|
140
|
+
if [[ -z "$db_dir" || "$db_dir" == "/" || "$db_dir" == "." || "$db_dir" == "~" ]]; then
|
|
141
|
+
printf 'error: refusing to remove unsafe db_dir value %q\n' "$db_dir" >"$log_file"
|
|
142
|
+
return 1
|
|
143
|
+
fi
|
|
144
|
+
|
|
145
|
+
if ! resolved_db_dir="$(resolve_path "$db_dir")"; then
|
|
146
|
+
printf 'error: failed to resolve db_dir %q\n' "$db_dir" >"$log_file"
|
|
147
|
+
return 1
|
|
148
|
+
fi
|
|
149
|
+
if ! resolved_base_dir="$(resolve_path "$DB_DIR_BASE")"; then
|
|
150
|
+
printf 'error: failed to resolve DB_DIR_BASE %q\n' "$DB_DIR_BASE" >"$log_file"
|
|
151
|
+
return 1
|
|
152
|
+
fi
|
|
153
|
+
|
|
154
|
+
if [[ "$resolved_db_dir" == "$resolved_base_dir" ]]; then
|
|
155
|
+
printf 'error: refusing to remove db_dir %q because it resolves to the base directory %q\n' \
|
|
156
|
+
"$resolved_db_dir" "$resolved_base_dir" >"$log_file"
|
|
157
|
+
return 1
|
|
158
|
+
fi
|
|
159
|
+
|
|
160
|
+
case "$resolved_db_dir" in
|
|
161
|
+
"$resolved_base_dir"/*) ;;
|
|
162
|
+
*)
|
|
163
|
+
printf 'error: refusing to remove db_dir %q outside allowed base %q\n' \
|
|
164
|
+
"$resolved_db_dir" "$resolved_base_dir" >"$log_file"
|
|
165
|
+
return 1
|
|
166
|
+
;;
|
|
167
|
+
esac
|
|
168
|
+
}
|
|
169
|
+
|
|
170
|
+
extract_profile_line() {
|
|
171
|
+
local log_file="$1"
|
|
172
|
+
|
|
173
|
+
case "$TIME_FORMAT_STYLE" in
|
|
174
|
+
gnu)
|
|
175
|
+
awk '/elapsed=/{ line = $0 } END { print line }' "$log_file"
|
|
176
|
+
;;
|
|
177
|
+
bsd)
|
|
178
|
+
awk '
|
|
179
|
+
/^[[:space:]]*[0-9]+([.][0-9]+)?[[:space:]]+real([[:space:]]|$)/ {
|
|
180
|
+
elapsed = $1
|
|
181
|
+
}
|
|
182
|
+
/^[[:space:]]*[0-9]+[[:space:]]+maximum resident set size([[:space:]]|$)/ {
|
|
183
|
+
rss_kb = $1
|
|
184
|
+
}
|
|
185
|
+
END {
|
|
186
|
+
if (elapsed != "" || rss_kb != "") {
|
|
187
|
+
printf "elapsed=%s rss_kb=%s\n", elapsed, rss_kb
|
|
188
|
+
}
|
|
189
|
+
}
|
|
190
|
+
' "$log_file"
|
|
191
|
+
;;
|
|
192
|
+
*)
|
|
193
|
+
return 1
|
|
194
|
+
;;
|
|
195
|
+
esac
|
|
196
|
+
}
|
|
197
|
+
|
|
198
|
+
run_load_profile() {
|
|
199
|
+
local db_dir="$1"
|
|
200
|
+
local log_file="$2"
|
|
201
|
+
shift 2
|
|
202
|
+
|
|
203
|
+
validate_db_dir "$db_dir" "$log_file"
|
|
204
|
+
rm -rf "$db_dir"
|
|
205
|
+
: > "$log_file"
|
|
206
|
+
local -a env_args=(
|
|
207
|
+
PATH="$PATH"
|
|
208
|
+
HOME="${HOME:-/tmp}"
|
|
209
|
+
GOMAXPROCS="$GOMAXPROCS_VALUE"
|
|
210
|
+
DINGO_RUN_MODE=serve
|
|
211
|
+
DINGO_STORAGE_MODE=core
|
|
212
|
+
)
|
|
213
|
+
if [[ -n "$GOMEMLIMIT_VALUE" ]]; then
|
|
214
|
+
env_args+=(GOMEMLIMIT="$GOMEMLIMIT_VALUE")
|
|
215
|
+
fi
|
|
216
|
+
env -i \
|
|
217
|
+
"${env_args[@]}" \
|
|
218
|
+
"${TIME_CMD[@]}" \
|
|
219
|
+
./dingo \
|
|
220
|
+
--config /dev/null \
|
|
221
|
+
--blob badger \
|
|
222
|
+
--metadata sqlite \
|
|
223
|
+
--data-dir "$db_dir" \
|
|
224
|
+
"${DINGO_CACHE_ARGS[@]}" \
|
|
225
|
+
load "$IMMUTABLE_PATH" "$@" \
|
|
226
|
+
> "$log_file" 2>&1
|
|
227
|
+
|
|
228
|
+
extract_profile_line "$log_file"
|
|
229
|
+
}
|
|
230
|
+
|
|
231
|
+
parse_metric() {
|
|
232
|
+
local line="$1"
|
|
233
|
+
local key="$2"
|
|
234
|
+
echo "$line" | awk -v needle="$key" '{
|
|
235
|
+
for (i = 1; i <= NF; i++) {
|
|
236
|
+
split($i, kv, "=")
|
|
237
|
+
if (kv[1] == needle) {
|
|
238
|
+
print kv[2]
|
|
239
|
+
exit
|
|
240
|
+
}
|
|
241
|
+
}
|
|
242
|
+
}'
|
|
243
|
+
}
|
|
244
|
+
|
|
245
|
+
configure_time_cmd
|
|
246
|
+
|
|
247
|
+
if [[ -n "$BLOCK_CACHE_SIZE_VALUE" ]]; then
|
|
248
|
+
DINGO_CACHE_ARGS+=(--block-cache-size "$BLOCK_CACHE_SIZE_VALUE")
|
|
249
|
+
fi
|
|
250
|
+
if [[ -n "$INDEX_CACHE_SIZE_VALUE" ]]; then
|
|
251
|
+
DINGO_CACHE_ARGS+=(--index-cache-size "$INDEX_CACHE_SIZE_VALUE")
|
|
252
|
+
fi
|
|
253
|
+
if [[ -n "$MEMTABLE_SIZE_VALUE" ]]; then
|
|
254
|
+
DINGO_CACHE_ARGS+=(--memtable-size "$MEMTABLE_SIZE_VALUE")
|
|
255
|
+
fi
|
|
256
|
+
|
|
257
|
+
DB_DIR_BASE="$(resolve_path "$DB_DIR_BASE")"
|
|
258
|
+
|
|
259
|
+
if [[ -z "$DEFAULT_DB_DIR" ]]; then
|
|
260
|
+
DEFAULT_DB_DIR="$(create_temp_dir)"
|
|
261
|
+
AUTO_DEFAULT_DB_DIR=1
|
|
262
|
+
fi
|
|
263
|
+
|
|
264
|
+
if [[ -z "$DEFAULT_LOG_FILE" ]]; then
|
|
265
|
+
DEFAULT_LOG_FILE="$(create_temp_file)"
|
|
266
|
+
AUTO_DEFAULT_LOG_FILE=1
|
|
267
|
+
fi
|
|
268
|
+
|
|
269
|
+
pushd "$ROOT_DIR" >/dev/null
|
|
270
|
+
|
|
271
|
+
env \
|
|
272
|
+
GOCACHE="$CACHE_DIR" \
|
|
273
|
+
GOMAXPROCS="$GOMAXPROCS_VALUE" \
|
|
274
|
+
./generate_benchmarks.sh \
|
|
275
|
+
"$TMP_BENCH_FILE" \
|
|
276
|
+
--write \
|
|
277
|
+
--packages ./ledger \
|
|
278
|
+
--bench 'Benchmark(BlockfetchNearTipThroughput|BlockfetchNearTipThroughputPredecoded|BlockfetchNearTipQueuedHeaderPredecoded|BlockfetchNearTipFlushOnlyPredecoded|VerifyBlockHeader)$' \
|
|
279
|
+
--benchtime "$BENCH_TIME" \
|
|
280
|
+
--title "Four-Thread Block Producer Sizing Benchmark Results (Non-Pi Host)" \
|
|
281
|
+
--scope "Four-thread block producer path (GOMAXPROCS=${GOMAXPROCS_VALUE}) in core mode on a non-Pi host" \
|
|
282
|
+
--data-source "database/immutable/testdata plus local immutable load runs"
|
|
283
|
+
|
|
284
|
+
env \
|
|
285
|
+
GOCACHE="$CACHE_DIR" \
|
|
286
|
+
GOFLAGS=-buildvcs=false \
|
|
287
|
+
CGO_ENABLED=0 \
|
|
288
|
+
go build -o ./dingo ./cmd/dingo
|
|
289
|
+
|
|
290
|
+
default_line="$(run_load_profile "$DEFAULT_DB_DIR" "$DEFAULT_LOG_FILE")"
|
|
291
|
+
|
|
292
|
+
default_elapsed="$(parse_metric "$default_line" "elapsed")"
|
|
293
|
+
default_rss_kb="$(parse_metric "$default_line" "rss_kb")"
|
|
294
|
+
|
|
295
|
+
if [[ ! "$default_elapsed" =~ ^[0-9]+([.][0-9]+)?$ ]]; then
|
|
296
|
+
echo "error: failed to parse numeric elapsed value from $DEFAULT_LOG_FILE: ${default_elapsed:-<empty>}" >&2
|
|
297
|
+
exit 1
|
|
298
|
+
fi
|
|
299
|
+
if [[ ! "$default_rss_kb" =~ ^[0-9]+$ ]]; then
|
|
300
|
+
echo "error: failed to parse numeric rss_kb value from $DEFAULT_LOG_FILE: ${default_rss_kb:-<empty>}" >&2
|
|
301
|
+
exit 1
|
|
302
|
+
fi
|
|
303
|
+
|
|
304
|
+
default_rss_mb="$(awk "BEGIN { printf \"%.1f\", ${default_rss_kb} / 1024 }")"
|
|
305
|
+
rss_limit_kb=$((RSS_LIMIT_MB * 1024))
|
|
306
|
+
|
|
307
|
+
if (( default_rss_kb > rss_limit_kb )); then
|
|
308
|
+
echo "error: peak RSS ${default_rss_mb} MB exceeds limit ${RSS_LIMIT_MB} MB" >&2
|
|
309
|
+
exit 1
|
|
310
|
+
fi
|
|
311
|
+
|
|
312
|
+
{
|
|
313
|
+
echo "# Four-Thread Block Producer Sizing Report (Non-Pi Host)"
|
|
314
|
+
echo
|
|
315
|
+
echo "Assumptions:"
|
|
316
|
+
printf -- "- \`storageMode=core\`\n"
|
|
317
|
+
printf -- "- \`runMode=serve\`, \`blob=badger\`, \`metadata=sqlite\`\n"
|
|
318
|
+
printf -- "- \`GOMAXPROCS=%s\`\n" "$GOMAXPROCS_VALUE"
|
|
319
|
+
printf -- "- target peak RSS: \`<= %s MB\`\n" "$RSS_LIMIT_MB"
|
|
320
|
+
printf -- "- immutable fixture: \`%s\`\n" "$IMMUTABLE_PATH"
|
|
321
|
+
if [[ -n "$GOMEMLIMIT_VALUE" ]]; then
|
|
322
|
+
printf -- "- test-only \`GOMEMLIMIT=%s\`\n" "$GOMEMLIMIT_VALUE"
|
|
323
|
+
fi
|
|
324
|
+
if [[ -n "$BLOCK_CACHE_SIZE_VALUE" || -n "$INDEX_CACHE_SIZE_VALUE" || -n "$MEMTABLE_SIZE_VALUE" ]]; then
|
|
325
|
+
echo "- explicit Badger test-only overrides:"
|
|
326
|
+
if [[ -n "$BLOCK_CACHE_SIZE_VALUE" ]]; then
|
|
327
|
+
printf -- " - \`--block-cache-size=%s\`\n" "$BLOCK_CACHE_SIZE_VALUE"
|
|
328
|
+
fi
|
|
329
|
+
if [[ -n "$INDEX_CACHE_SIZE_VALUE" ]]; then
|
|
330
|
+
printf -- " - \`--index-cache-size=%s\`\n" "$INDEX_CACHE_SIZE_VALUE"
|
|
331
|
+
fi
|
|
332
|
+
if [[ -n "$MEMTABLE_SIZE_VALUE" ]]; then
|
|
333
|
+
printf -- " - \`--memtable-size=%s\`\n" "$MEMTABLE_SIZE_VALUE"
|
|
334
|
+
fi
|
|
335
|
+
fi
|
|
336
|
+
echo "- measurements were collected on non-Pi hardware and are a sizing proxy, not Raspberry Pi device benchmarks"
|
|
337
|
+
echo "- block producer peer count remains small; this is not a relay profile"
|
|
338
|
+
echo
|
|
339
|
+
echo "## Load Footprint"
|
|
340
|
+
echo
|
|
341
|
+
echo "| Profile | Elapsed | Peak RSS | Notes |"
|
|
342
|
+
echo "| --- | --- | --- | --- |"
|
|
343
|
+
if [[ -n "$BLOCK_CACHE_SIZE_VALUE" || -n "$INDEX_CACHE_SIZE_VALUE" || -n "$MEMTABLE_SIZE_VALUE" || -n "$GOMEMLIMIT_VALUE" ]]; then
|
|
344
|
+
printf -- "| \`core-constrained\` | \`%ss\` | \`%s MB\` | Explicit test-only memory/cache overrides |\n" "$default_elapsed" "$default_rss_mb"
|
|
345
|
+
else
|
|
346
|
+
printf -- "| \`core-default\` | \`%ss\` | \`%s MB\` | Current default \`serve/core\` cache profile |\n" "$default_elapsed" "$default_rss_mb"
|
|
347
|
+
fi
|
|
348
|
+
echo
|
|
349
|
+
echo "## Recommendation"
|
|
350
|
+
echo
|
|
351
|
+
printf -- "For Raspberry Pi-class sizing in \`storageMode=core\`, keep production defaults aligned with normal Badger behavior and use explicit memory/cache constraints only in this harness when validating low-memory targets.\n"
|
|
352
|
+
printf -- "This run stayed under the configured \`%s MB\` RSS limit.\n" "$RSS_LIMIT_MB"
|
|
353
|
+
echo "Explicit Badger cache settings still override these defaults if an operator wants to tune further."
|
|
354
|
+
echo "Treat this as a sizing baseline rather than a direct Raspberry Pi hardware measurement."
|
|
355
|
+
echo
|
|
356
|
+
cat "$TMP_BENCH_FILE"
|
|
357
|
+
} > "$OUTPUT_FILE"
|
|
358
|
+
|
|
359
|
+
popd >/dev/null
|