meshtastic 0.0.183 → 0.0.185
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.rubocop_todo.yml +1 -0
- data/Gemfile +2 -1
- data/documentation/README.md +6 -0
- data/documentation/admin-backup.md +369 -0
- data/documentation/admin.md +4 -0
- data/documentation/forwarder.md +149 -0
- data/documentation/mesh-interface.md +157 -1
- data/documentation/payload-formats.md +187 -0
- data/documentation/reticulum.md +89 -0
- data/lib/meshtastic/admin/backup.rb +560 -0
- data/lib/meshtastic/admin.rb +2 -0
- data/lib/meshtastic/atak.rb +98 -13
- data/lib/meshtastic/atak_pb.rb +3 -1
- data/lib/meshtastic/config_pb.rb +1 -1
- data/lib/meshtastic/field_metadata_pb.rb +1 -1
- data/lib/meshtastic/forwarder.rb +190 -0
- data/lib/meshtastic/forwarder_pb.rb +77 -0
- data/lib/meshtastic/mesh_beacon_pb.rb +1 -1
- data/lib/meshtastic/mesh_interface.rb +115 -65
- data/lib/meshtastic/mesh_pb.rb +2 -1
- data/lib/meshtastic/module_config_pb.rb +2 -1
- data/lib/meshtastic/mqtt.rb +4 -34
- data/lib/meshtastic/payload_compression.rb +90 -0
- data/lib/meshtastic/payload_formats.rb +248 -0
- data/lib/meshtastic/reticulum.rb +71 -0
- data/lib/meshtastic/serial.rb +1 -19
- data/lib/meshtastic/telemetry_pb.rb +2 -2
- data/lib/meshtastic/unishox2.rb +467 -0
- data/lib/meshtastic/version.rb +1 -1
- data/lib/meshtastic.rb +4 -0
- data/meshtastic.gemspec +2 -0
- data/spec/lib/meshtastic/admin/backup_spec.rb +704 -0
- data/spec/lib/meshtastic/atak_spec.rb +50 -0
- data/spec/lib/meshtastic/bluetooth_spec.rb +14 -0
- data/spec/lib/meshtastic/forwarder_pb_spec.rb +11 -0
- data/spec/lib/meshtastic/forwarder_spec.rb +97 -0
- data/spec/lib/meshtastic/mesh_interface_spec.rb +169 -0
- data/spec/lib/meshtastic/mqtt_spec.rb +56 -0
- data/spec/lib/meshtastic/payload_compression_spec.rb +43 -0
- data/spec/lib/meshtastic/payload_formats_spec.rb +160 -0
- data/spec/lib/meshtastic/reticulum_spec.rb +91 -0
- data/spec/lib/meshtastic/serial_spec.rb +46 -0
- data/spec/lib/meshtastic/tcp_spec.rb +15 -0
- data/spec/lib/meshtastic/unishox2_spec.rb +34 -0
- data/spec/support/payload_fixtures.rb +124 -0
- data/spec/support/reticulum_fixtures.json +12 -0
- data/spec/support/tak_codec_fixtures.rb +4 -0
- data/spec/support/unishox_fixtures.rb +4 -0
- metadata +39 -3
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: f78bc7d067491d26e216b42e75192c1609e8e9c70cd7193bb615a9f934052936
|
|
4
|
+
data.tar.gz: e61f34b1e8834fd08c17911b8223a36fced64f0cc9548dfbd2a61ca494ce614d
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: e7d9948952297de8db885f77df10dd5ab1b3a4db37d020ba4f43ad1ee73c68c5b13bbecc0eab0c8a35490d6c3107a04f53069ce9db3b9cd9db945bfc2fdbc074
|
|
7
|
+
data.tar.gz: 00c3afd0cd032883f2ca8d075d8e4ec7f8347b2cfa31b9e66a66ef5c7120af1ec21d8e354a69dd10e9427e92f7db0675a4b7a2fca21ecb79a48ded40186ee613
|
data/.rubocop_todo.yml
CHANGED
data/Gemfile
CHANGED
|
@@ -9,9 +9,10 @@ gemspec
|
|
|
9
9
|
gem 'bundler', '>=4.0.21'
|
|
10
10
|
gem 'bundler-audit', '>=0.9.3'
|
|
11
11
|
gem 'executable-hooks', '1.7.1'
|
|
12
|
+
gem 'fiddle', '1.1.8'
|
|
12
13
|
gem 'gem-wrappers', '1.4.0'
|
|
13
14
|
gem 'geocoder', '1.8.6'
|
|
14
|
-
gem 'google-protobuf', '4.36.
|
|
15
|
+
gem 'google-protobuf', '4.36.2'
|
|
15
16
|
gem 'grpc-tools', '1.84.0'
|
|
16
17
|
gem 'mqtt', '0.7.0'
|
|
17
18
|
gem 'rake', '13.4.2'
|
data/documentation/README.md
CHANGED
|
@@ -23,11 +23,14 @@ Examples use placeholder addresses only:
|
|
|
23
23
|
|
|
24
24
|
Feature modules send protobufs over Serial, Bluetooth, or TCP via `Meshtastic.deliver_data`. Pass `serial_obj:`, `bluetooth_obj:`, or `tcp_obj:`. MQTT encrypts on the host; the radio encrypts on the other three.
|
|
25
25
|
|
|
26
|
+
All four subscriptions share [payload decoding and the complete port inventory](mesh-interface.md#receive-payload-coverage), including empty proto3 messages and explicit raw/unsupported codec boundaries.
|
|
27
|
+
|
|
26
28
|
Do not open Serial and Bluetooth to the same radio at once. Always `disconnect` in `ensure`.
|
|
27
29
|
|
|
28
30
|
## Feature modules
|
|
29
31
|
|
|
30
32
|
- [Meshtastic::Admin](admin.md)
|
|
33
|
+
- [Meshtastic::Admin::Backup](admin-backup.md)
|
|
31
34
|
- [Meshtastic::Admin::Channel](admin-channel.md)
|
|
32
35
|
- [Meshtastic::Admin::Config](admin-config.md)
|
|
33
36
|
- [Meshtastic::Admin::Firmware](admin-firmware.md)
|
|
@@ -42,6 +45,9 @@ Do not open Serial and Bluetooth to the same radio at once. Always `disconnect`
|
|
|
42
45
|
- [Meshtastic::RemoteHardware](remote-hardware.md)
|
|
43
46
|
- [Meshtastic::Storeforward](storeforward.md)
|
|
44
47
|
- [Meshtastic::ATAK](atak.md)
|
|
48
|
+
- [Meshtastic::Forwarder](forwarder.md)
|
|
49
|
+
- [Meshtastic::Reticulum](reticulum.md)
|
|
50
|
+
- [Meshtastic::PayloadFormats](payload-formats.md)
|
|
45
51
|
- [Meshtastic::Paxcount](paxcount.md)
|
|
46
52
|
- [Meshtastic::Cannedmessages](cannedmessages.md)
|
|
47
53
|
- [Meshtastic::RTTTL](rtttl.md)
|
|
@@ -0,0 +1,369 @@
|
|
|
1
|
+
# Admin::Backup — native host-side configuration snapshots
|
|
2
|
+
|
|
3
|
+
`Meshtastic::Admin::Backup` exports a versioned JSON document (default) or binary
|
|
4
|
+
Meshtastic `DeviceProfile` (`format: :device_profile`) using fresh,
|
|
5
|
+
request-ID/source-correlated Admin requests. It imports validated protobuf
|
|
6
|
+
sections with bounded routing ACK waits. It does not invoke the firmware's
|
|
7
|
+
filesystem backup/restore commands or shell out to another client.
|
|
8
|
+
|
|
9
|
+
**Backups contain secrets by default**, including channel PSKs, Wi-Fi/MQTT
|
|
10
|
+
credentials, private security keys and UI PINs when firmware returns them in
|
|
11
|
+
selected sections. Keep the file and returned Hash private; do not print either
|
|
12
|
+
in logs. Firmware can redact secrets or return default/empty values. Such a
|
|
13
|
+
snapshot cannot recover values the device did not disclose. Importing those
|
|
14
|
+
values can clear existing settings. Review the selection before exporting and
|
|
15
|
+
before restoring, especially across devices or firmware versions.
|
|
16
|
+
|
|
17
|
+
## Export
|
|
18
|
+
|
|
19
|
+
```ruby
|
|
20
|
+
require 'meshtastic'
|
|
21
|
+
require 'meshtastic/admin/backup'
|
|
22
|
+
|
|
23
|
+
snapshot = Meshtastic::Admin::Backup.export(
|
|
24
|
+
transport_obj: connection,
|
|
25
|
+
path: '/private/directory/node-backup.json', # optional; MUST NOT exist
|
|
26
|
+
config_types: %i[DEVICE_CONFIG POSITION_CONFIG POWER_CONFIG DISPLAY_CONFIG LORA_CONFIG],
|
|
27
|
+
module_config_types: %i[MQTT_CONFIG TELEMETRY_CONFIG],
|
|
28
|
+
channel_indexes: [0, 1],
|
|
29
|
+
include_owner: true,
|
|
30
|
+
include_ui: false,
|
|
31
|
+
timeout: 10
|
|
32
|
+
)
|
|
33
|
+
# Safe to display status/count; NOT snapshot[:backup].
|
|
34
|
+
snapshot.slice(:status, :count)
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
`transport_obj:` is a connected Serial, Bluetooth or TCP handle with a receive
|
|
38
|
+
queue. **MQTT synchronous export/import is explicitly unsupported**, including
|
|
39
|
+
dry runs. `to:` optionally selects a unicast target; otherwise the connected
|
|
40
|
+
local node is used. `channel:` and `hop_limit:` are Admin delivery options.
|
|
41
|
+
Pause external queue consumers and do not interleave other Admin edits with a
|
|
42
|
+
backup/restore. The underlying Admin layer serializes individual requests,
|
|
43
|
+
not an entire multi-request snapshot or transaction.
|
|
44
|
+
|
|
45
|
+
JSON selection defaults (binary-specific differences are listed below):
|
|
46
|
+
|
|
47
|
+
- `include_owner: true`: only `long_name`, `short_name`, `is_licensed`.
|
|
48
|
+
Never clone User ID, MAC, hardware model, public key, role or messaging
|
|
49
|
+
capability metadata. Role belongs to device configuration; security keys
|
|
50
|
+
belong to explicitly selected security configuration.
|
|
51
|
+
- `config_types:` defaults to `DEVICE_CONFIG`, `POSITION_CONFIG`, `POWER_CONFIG`,
|
|
52
|
+
`NETWORK_CONFIG`, `DISPLAY_CONFIG`, `LORA_CONFIG`, `BLUETOOTH_CONFIG`,
|
|
53
|
+
`SECURITY_CONFIG`. Use a narrower list for firmware that lacks sections.
|
|
54
|
+
- `module_config_types: []`: opt in to supported module ConfigType symbols.
|
|
55
|
+
All 17 currently generated ModuleConfig sections are supported, including
|
|
56
|
+
`EXTNOTIF_CONFIG`, `STOREFORWARD_CONFIG`, `CANNEDMSG_CONFIG`,
|
|
57
|
+
`STATUSMESSAGE_CONFIG`, `TRAFFICMANAGEMENT_CONFIG`, `TAK_CONFIG`,
|
|
58
|
+
`MESHBEACON_CONFIG`. `Backup::MODULE_TYPES` lists the complete set.
|
|
59
|
+
- `channel_indexes: [0, 1, 2, 3, 4, 5, 6, 7]`: explicit zero-based slots,
|
|
60
|
+
including disabled channels. Wire getters use one-based indexes.
|
|
61
|
+
- `include_ui: false`: opt in to dedicated get/store UI operations.
|
|
62
|
+
- `timeout: 10`: positive finite seconds **per Admin request**, not a deadline
|
|
63
|
+
for the entire backup. Remote writes can first acquire an Admin session.
|
|
64
|
+
|
|
65
|
+
Unsupported selected sections fail the export; they are not silently skipped.
|
|
66
|
+
No file is created until every selected response has been collected and the
|
|
67
|
+
document validated. Empty JSON selections are allowed. A successful JSON return contains
|
|
68
|
+
`:status => :exported`, `:count`, `:backup` and `:warnings`. A read error raises.
|
|
69
|
+
Snapshots are sequential fresh reads, not an atomic device-wide snapshot.
|
|
70
|
+
|
|
71
|
+
Files are exclusively created with mode `0600`, `O_EXCL` and `O_NOFOLLOW`, then
|
|
72
|
+
flushed and fsynced. There is no overwrite flag. Existing files and final-path
|
|
73
|
+
symlinks fail. Use a trusted parent directory: parent-directory symlinks are not
|
|
74
|
+
resolved or pinned. A disk write/sync failure raises and can leave a partial
|
|
75
|
+
0600 file; it is never reported as a successful backup. File mode protection is
|
|
76
|
+
not encryption and does not protect against privileged host access.
|
|
77
|
+
|
|
78
|
+
## Validate, plan and import
|
|
79
|
+
|
|
80
|
+
```ruby
|
|
81
|
+
plan = Meshtastic::Admin::Backup.import(
|
|
82
|
+
transport_obj: connection,
|
|
83
|
+
path: '/private/directory/node-backup.json',
|
|
84
|
+
dry_run: true
|
|
85
|
+
)
|
|
86
|
+
|
|
87
|
+
result = Meshtastic::Admin::Backup.import(
|
|
88
|
+
transport_obj: connection,
|
|
89
|
+
backup: snapshot[:backup], # use exactly one of backup: or path:
|
|
90
|
+
edit_transaction: true, # ONLY when target firmware supports it
|
|
91
|
+
verify: true,
|
|
92
|
+
timeout: 10
|
|
93
|
+
)
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
`dry_run: true` performs **no radio requests or writes**, including no session
|
|
97
|
+
acquisition, edit begin, getters or readback. It validates the entire document
|
|
98
|
+
first. `dry_run`, `edit_transaction` and `verify` default to `false` and must be
|
|
99
|
+
booleans. Unknown options are rejected. Import paths must be regular files and
|
|
100
|
+
must not be final-path symlinks.
|
|
101
|
+
|
|
102
|
+
The importer rejects incompatible format/version, unknown document/record/
|
|
103
|
+
protobuf fields, unknown section selectors, duplicate section/slot records,
|
|
104
|
+
invalid channel indexes, section/slot mismatches, conflicting oneofs, malformed
|
|
105
|
+
protobuf scalar values, null values and noncanonical bytes/base64 **before any
|
|
106
|
+
radio request**. A version-1 document is a Hash with string keys:
|
|
107
|
+
|
|
108
|
+
```json
|
|
109
|
+
{
|
|
110
|
+
"format": "meshtastic-admin-backup",
|
|
111
|
+
"version": 1,
|
|
112
|
+
"warning": "A human-readable limitations notice",
|
|
113
|
+
"records": [
|
|
114
|
+
{
|
|
115
|
+
"section": "config",
|
|
116
|
+
"slot": "DEVICE_CONFIG",
|
|
117
|
+
"value": { "device": { "serial_enabled": false } }
|
|
118
|
+
}
|
|
119
|
+
]
|
|
120
|
+
}
|
|
121
|
+
```
|
|
122
|
+
|
|
123
|
+
Sections are `owner`, `config`, `module_config`, `channel`, `ui`. Owner/UI slots
|
|
124
|
+
are JSON null; config/module slots are uppercase enum names; channel slots are
|
|
125
|
+
integers. `value` is protobuf JSON with **snake_case protobuf field names**,
|
|
126
|
+
canonical base64 bytes, protobuf enum representations and emitted scalar
|
|
127
|
+
defaults. Known fields and protobuf presence/default semantics round-trip;
|
|
128
|
+
unknown binary wire fields are not archived. This is not an opaque protobuf
|
|
129
|
+
wire backup. Missing fields in a manually edited document mean protobuf
|
|
130
|
+
defaults, not a patch/merge. YAML and arbitrary object deserialization are never
|
|
131
|
+
used. Session envelope passkeys and `SESSIONKEY_CONFIG` are never exported or
|
|
132
|
+
accepted; automatic session state stays inside the existing Admin connection.
|
|
133
|
+
|
|
134
|
+
Writes preserve input order within priority groups: ordinary owner/module/UI/
|
|
135
|
+
core settings first, channels next, then LoRa, Bluetooth, network and security.
|
|
136
|
+
These latter settings can disrupt the connection. Other firmware-specific
|
|
137
|
+
settings can also disrupt it; ordering cannot guarantee connectivity.
|
|
138
|
+
|
|
139
|
+
## Binary DeviceProfile (`.cfg`) export and import
|
|
140
|
+
|
|
141
|
+
Select binary output explicitly; export accepts only `format: :json` (default)
|
|
142
|
+
or `format: :device_profile`, not `:auto`. The path extension does not choose
|
|
143
|
+
export format. JSON version-1 output and return keys are unchanged.
|
|
144
|
+
|
|
145
|
+
```ruby
|
|
146
|
+
snapshot = Meshtastic::Admin::Backup.export(
|
|
147
|
+
transport_obj: connection,
|
|
148
|
+
format: :device_profile,
|
|
149
|
+
path: '/private/directory/nodeConfig.cfg', # optional; MUST NOT exist
|
|
150
|
+
config_types: %i[DEVICE_CONFIG POSITION_CONFIG LORA_CONFIG],
|
|
151
|
+
module_config_types: %i[MQTT_CONFIG TELEMETRY_CONFIG],
|
|
152
|
+
channel_indexes: [0, 1], # actual contiguous PRIMARY/SECONDARY slots only
|
|
153
|
+
include_owner: true,
|
|
154
|
+
include_ui: false,
|
|
155
|
+
include_ringtone: true,
|
|
156
|
+
include_canned_messages: true,
|
|
157
|
+
timeout: 10
|
|
158
|
+
)
|
|
159
|
+
# Safe status fields only. NEVER print backup bytes or decoded profile.
|
|
160
|
+
snapshot.slice(:status, :format, :count)
|
|
161
|
+
# snapshot[:backup] is a raw ASCII-8BIT String, not JSON/base64 or a protobuf object.
|
|
162
|
+
# Pass it directly to Backup.import(transport_obj: connection,
|
|
163
|
+
# backup: snapshot[:backup], dry_run: true).
|
|
164
|
+
```
|
|
165
|
+
|
|
166
|
+
Binary returns `{ status: :exported, format: :device_profile, count: Integer,
|
|
167
|
+
backup: binary_string, warnings: [...] }`. `count` is the number of restorable
|
|
168
|
+
sections (owner counts once, each channel counts once, duplicate URL/config
|
|
169
|
+
LoRa counts once), not the byte count or number of protobuf fields. The `.cfg`
|
|
170
|
+
file contains exactly `snapshot[:backup]`. File protections are the same
|
|
171
|
+
exclusive/no-follow 0600 creation, flush and fsync as JSON. No file is created
|
|
172
|
+
until reads and strict binary validation succeed.
|
|
173
|
+
|
|
174
|
+
Binary selection uses the same core/module/owner/channel defaults as JSON,
|
|
175
|
+
with these format-specific rules:
|
|
176
|
+
|
|
177
|
+
- All eight core and 17 module selections are supported. Present empty nested
|
|
178
|
+
messages, byte fields, repeated values and scalar defaults round-trip. Local
|
|
179
|
+
container storage versions are not invented.
|
|
180
|
+
- Owner names and license flag are explicitly present, including empty names
|
|
181
|
+
and `false`. Optional `is_unmessagable` is copied only when present in the
|
|
182
|
+
fresh User response, including explicitly present `false`. ID, MAC, hardware,
|
|
183
|
+
public key and envelope session passkeys are not copied.
|
|
184
|
+
- `include_ui: true` is **rejected before any radio requests**: DeviceProfile
|
|
185
|
+
has no UI field. Use JSON for UI snapshots.
|
|
186
|
+
- ChannelSet URLs cannot preserve arbitrary indexes or disabled slots. Select
|
|
187
|
+
`channel_indexes: []` to omit channels, or an ordered contiguous prefix such
|
|
188
|
+
as `[0]` or `[0, 1]`. Sparse/reordered selections are rejected before requests.
|
|
189
|
+
Returned slot 0 must be PRIMARY, later selected slots SECONDARY, each with
|
|
190
|
+
settings; mismatched indexes, disabled slots and other roles fail export
|
|
191
|
+
without creating a file. No selected channel is silently skipped or moved.
|
|
192
|
+
**The default all-eight selection therefore fails if any slot is disabled**;
|
|
193
|
+
choose the actual enabled prefix or use JSON to preserve disabled/sparse slots.
|
|
194
|
+
Selected LoRa is included identically in both `config.lora` and the URL;
|
|
195
|
+
if LoRa is not selected, the URL does not invent or fetch it. Unselected
|
|
196
|
+
trailing channel slots are not disabled by a later binary import.
|
|
197
|
+
- `include_ringtone: true` and `include_canned_messages: true` request fresh
|
|
198
|
+
strings and preserve present empty strings. Both default to false and are
|
|
199
|
+
binary-only options; JSON rejects these keys rather than ignoring them.
|
|
200
|
+
- `fixed_position:` optionally accepts an explicit `Meshtastic::Position` or
|
|
201
|
+
field Hash, e.g. `{ latitude_i: 0, longitude_i: 0, altitude: 0 }`. It is
|
|
202
|
+
validated before any requests and copied with protobuf presence/defaults.
|
|
203
|
+
This is caller-supplied input, **not a fresh radio read**: no dedicated Admin
|
|
204
|
+
getter exists. It defaults to absent and is binary-only. No position is
|
|
205
|
+
inferred from cached GPS data or the position configuration.
|
|
206
|
+
- An entirely empty binary selection is rejected before requests because it
|
|
207
|
+
would not produce an importable profile. JSON still permits empty records.
|
|
208
|
+
|
|
209
|
+
The importer accepts the same binary `meshtastic.DeviceProfile` used by
|
|
210
|
+
Meshtastic clients, entirely in Ruby.
|
|
211
|
+
Use exactly one source and select `format: :device_profile` explicitly, or use
|
|
212
|
+
`format: :auto` (the default for import):
|
|
213
|
+
|
|
214
|
+
```ruby
|
|
215
|
+
plan = Meshtastic::Admin::Backup.import(
|
|
216
|
+
transport_obj: connection,
|
|
217
|
+
path: '/private/directory/nodeConfig.cfg',
|
|
218
|
+
format: :device_profile,
|
|
219
|
+
dry_run: true
|
|
220
|
+
)
|
|
221
|
+
# Only section/slot identities and counts are returned; no profile contents.
|
|
222
|
+
plan.slice(:status, :planned, :plan)
|
|
223
|
+
|
|
224
|
+
# Alternative in-memory source, still offline validation with no radio requests:
|
|
225
|
+
plan = Meshtastic::Admin::Backup.import(
|
|
226
|
+
transport_obj: connection,
|
|
227
|
+
backup: binary_profile_string,
|
|
228
|
+
format: :device_profile,
|
|
229
|
+
dry_run: true
|
|
230
|
+
)
|
|
231
|
+
```
|
|
232
|
+
|
|
233
|
+
`format:` accepts only `:auto`, `:json`, or `:device_profile`. In auto mode a
|
|
234
|
+
`.cfg` path (case-insensitive) selects binary, `.json` selects JSON, and a Hash
|
|
235
|
+
selects the existing JSON document API. Other paths and Strings are validated
|
|
236
|
+
as binary first, then as JSON when JSON-looking content fails binary validation.
|
|
237
|
+
This avoids confusing a valid binary name field beginning with a newline and
|
|
238
|
+
`{` with JSON. Explicit format overrides the extension. JSON Strings are also
|
|
239
|
+
accepted; versioned JSON Hash behavior and JSON export selections are unchanged.
|
|
240
|
+
No YAML, executable serialization, or implicit protobuf object input is used.
|
|
241
|
+
|
|
242
|
+
Presence-aware mapping:
|
|
243
|
+
|
|
244
|
+
- Present `long_name`, `short_name`, `is_licensed`, and `is_unmessagable` become
|
|
245
|
+
one `owner`/`set_owner` operation. Explicit `false` and empty names are not
|
|
246
|
+
treated as absent. User identity/MAC/hardware/public-key fields are never
|
|
247
|
+
synthesized. The Admin User schema has no presence for names/license, so
|
|
248
|
+
omitted members of a sent owner message carry protobuf defaults; this is not
|
|
249
|
+
a read/merge patch API. Firmware determines how those defaults are applied.
|
|
250
|
+
- Every present `LocalConfig` and `LocalModuleConfig` message becomes its own
|
|
251
|
+
writable Config/ModuleConfig oneof, including present empty submessages.
|
|
252
|
+
All eight core and 17 module sections are supported. Nested bytes, repeated
|
|
253
|
+
values, defaults and optional presence are retained. The local containers'
|
|
254
|
+
`version` fields are storage-schema metadata, not Admin settings; they are
|
|
255
|
+
validated but not written. A metadata-only/empty profile is rejected.
|
|
256
|
+
- `channel_url` must be an HTTPS `meshtastic.org/e/` or `/d/` URL with a valid
|
|
257
|
+
base64url ChannelSet and one through eight settings, no query or userinfo.
|
|
258
|
+
Settings map in order to primary slot 0 and secondary slots 1–7; unlisted
|
|
259
|
+
trailing slots are **not** disabled. PSK lengths and channel-name bounds are
|
|
260
|
+
checked. Embedded LoRa configuration is restored too. If both the profile and
|
|
261
|
+
URL carry LoRa, equal messages are deduplicated; conflicting messages reject
|
|
262
|
+
the entire profile rather than silently choosing one.
|
|
263
|
+
- Present `fixed_position`, `ringtone`, and `canned_messages` map respectively
|
|
264
|
+
to `set_fixed_position`, `set_ringtone_message`, and
|
|
265
|
+
`set_canned_message_module_messages`, with nil slots and matching section
|
|
266
|
+
names in the plan. Zero coordinates/altitude and present empty strings are
|
|
267
|
+
preserved. Absent fields are not turned into clears or removal operations.
|
|
268
|
+
|
|
269
|
+
The **entire** binary profile and nested ChannelSet are validated before any
|
|
270
|
+
Admin request, including session acquisition or begin-edit. A strict recursive
|
|
271
|
+
wire check rejects unknown tags (including sessionkey additions), unknown enum
|
|
272
|
+
values, duplicate singular fields/conflicting oneofs, invalid wire types,
|
|
273
|
+
truncation, oversized/noncanonical varints, invalid booleans and nonfinite
|
|
274
|
+
floats. Binary messages are limited to 1 MiB and 32 nested levels. Empty or
|
|
275
|
+
arbitrary bytes are not accepted as an empty restore. Strictness intentionally
|
|
276
|
+
rejects unsupported future schemas rather than dropping fields. This is format
|
|
277
|
+
validation, not authentication: protobuf has no file magic or signature, so a
|
|
278
|
+
valid nonempty DeviceProfile cannot be distinguished from another byte stream
|
|
279
|
+
that happens to encode the same schema. Session passkeys are never imported.
|
|
280
|
+
|
|
281
|
+
`verify: true` compares fresh owner/config/module/channel/ringtone/canned-message
|
|
282
|
+
responses. Fixed position has **no dedicated Admin getter**: its record reports
|
|
283
|
+
`readback: :unsupported`, making the overall status `:readback_incomplete` even
|
|
284
|
+
when all available comparisons match. No cached position or config flag is
|
|
285
|
+
misrepresented as fixed-position verification. All existing ACK, transaction,
|
|
286
|
+
no-replay, secret-handling and persistence limitations apply.
|
|
287
|
+
|
|
288
|
+
## Transactions and failure reporting
|
|
289
|
+
|
|
290
|
+
`edit_transaction: true` explicitly sends `begin_edit_settings`, waits for its
|
|
291
|
+
ACK, sends each section once with a correlated ACK wait (or matching timeout
|
|
292
|
+
readback as described below), then sends
|
|
293
|
+
`commit_edit_settings` and waits again. Current upstream
|
|
294
|
+
[AdminModule.cpp](https://github.com/meshtastic/firmware/blob/master/src/modules/AdminModule.cpp)
|
|
295
|
+
implements a begin flag and commit save of configuration/module/device/channel/
|
|
296
|
+
node database segments. Commit also disables Bluetooth. **This is not an atomic
|
|
297
|
+
rollback facility**: firmware may apply values before commit, UI has its own
|
|
298
|
+
storage operation, and the protocol has no supported cancel/rollback here.
|
|
299
|
+
There is no reliable capability probe, so enable this option only with firmware
|
|
300
|
+
you have verified supports it. Default `false` avoids claiming universal edit
|
|
301
|
+
transaction support.
|
|
302
|
+
|
|
303
|
+
Result fields:
|
|
304
|
+
|
|
305
|
+
- `status`: `dry_run`, `acknowledged`, `applied`, `partial_failure`,
|
|
306
|
+
`readback_matched` or `readback_incomplete`. `applied` means all sections
|
|
307
|
+
completed, but at least one required timeout readback instead of an ACK;
|
|
308
|
+
it does not claim every mutation was acknowledged or persisted.
|
|
309
|
+
- `planned`, `attempted`, `acknowledged`, `readback_confirmed`: section counts, excluding transaction
|
|
310
|
+
commands and automatic session requests. Attempted is an API attempt and
|
|
311
|
+
does not prove the write reached the wire.
|
|
312
|
+
- `plan`: ordered section/slot identities, including in dry runs; no secret values.
|
|
313
|
+
- `acknowledged` counts only actual successful setter ACKs. `readback_confirmed`
|
|
314
|
+
counts only setters whose ACK timed out but whose fresh getter matched.
|
|
315
|
+
These counts are separate, never double-counted, and retained on later failure.
|
|
316
|
+
- `records`: completed section/slot statuses (`acknowledged` or
|
|
317
|
+
`readback_confirmed`), without configuration values.
|
|
318
|
+
- `transaction`: `not_requested`, `begin_uncertain`, `open`, `commit_uncertain`
|
|
319
|
+
or `commit_acknowledged`.
|
|
320
|
+
- `failure`: on write/begin/commit failure, the operation, section/slot where
|
|
321
|
+
applicable, exception class and routing reason where available. Exception
|
|
322
|
+
contents and secrets are not included. Failed timeout recovery also includes
|
|
323
|
+
`readback: mismatch|unsupported|failed`; getter exceptions retain their class
|
|
324
|
+
and routing reason where available.
|
|
325
|
+
- `warnings`: secret/redaction/persistence limitations.
|
|
326
|
+
- `persistence_verified`: always `false`.
|
|
327
|
+
|
|
328
|
+
Only a section setter's `Timeout::Error` triggers recovery: if a supported
|
|
329
|
+
getter exists, import issues one **new**, request-ID/source-correlated Admin GET
|
|
330
|
+
with its own per-request timeout. This happens even with `verify: false`.
|
|
331
|
+
It continues only if the entire expected protobuf setting matches, using the
|
|
332
|
+
same presence-aware comparison as optional verification. Portable owner fields
|
|
333
|
+
are compared while generated owner identity/hardware metadata is excluded.
|
|
334
|
+
No cached value, unrelated response, late setter ACK, or transport submission
|
|
335
|
+
is substituted for that fresh read. The mutation is **never resent**.
|
|
336
|
+
|
|
337
|
+
Mismatch (including redacted/empty secrets in place of expected nonempty
|
|
338
|
+
secrets), absent getter, getter timeout/error, and correlated routing rejection
|
|
339
|
+
remain partial failures. Secret fields are not ignored or treated as wildcards;
|
|
340
|
+
the report never includes expected/actual values. Firmware normalization can
|
|
341
|
+
also prevent exact matching. A matching empty/default value cannot establish
|
|
342
|
+
that firmware disclosed a secret; existing backup completeness limitations
|
|
343
|
+
still apply. Fixed-position writes have no supported getter. Begin and commit
|
|
344
|
+
timeouts do not use this recovery, and routing errors do not trigger it.
|
|
345
|
+
|
|
346
|
+
After an unresolved failed write, no later sections, commit, rollback or replay are sent.
|
|
347
|
+
A transaction may remain open. A missing commit ACK is uncertain even when all
|
|
348
|
+
section ACKs arrived; the node may have committed and rebooted. Inspect the
|
|
349
|
+
node and establish a fresh connection before deciding on any manual recovery.
|
|
350
|
+
Do not automatically rerun the import.
|
|
351
|
+
|
|
352
|
+
`verify: true` performs another round of fresh correlated reads **after** all
|
|
353
|
+
sections complete and any requested commit ACK arrives. Timeout-recovery reads
|
|
354
|
+
do not replace this optional post-commit check.
|
|
355
|
+
It compares protobuf values (only portable owner fields), adds per-record
|
|
356
|
+
`readback: matched|mismatch|failed` and `readback_matched` count. Redaction,
|
|
357
|
+
normalization, link loss or reboot can prevent a match. Readback failures never
|
|
358
|
+
replay writes. Even a match proves only current reported state, not persistence
|
|
359
|
+
across reboot or successful firmware-side storage.
|
|
360
|
+
|
|
361
|
+
## Verification scope
|
|
362
|
+
|
|
363
|
+
The automated peer decodes production Serial framing/ToRadio/Admin protobufs,
|
|
364
|
+
maintains configuration state, and returns encoded correlated FromRadio Admin
|
|
365
|
+
or routing packets through the real Admin queue. Tests cover round-trip bytes
|
|
366
|
+
and defaults, dry runs, validation, secret handling, file permissions/symlinks,
|
|
367
|
+
ACK failures, timeouts, commit uncertainty, optional readback and help
|
|
368
|
+
conventions. No hardware, firmware reboot/persistence, radio-link reliability,
|
|
369
|
+
Bluetooth GATT or live TCP endpoint was exercised for this feature.
|
data/documentation/admin.md
CHANGED
|
@@ -24,6 +24,10 @@ Every generated payload field is available through `encode` and `send`, includin
|
|
|
24
24
|
|
|
25
25
|
## Operation catalog
|
|
26
26
|
|
|
27
|
+
For host-side configuration files, use [Admin::Backup](admin-backup.md).
|
|
28
|
+
The `backup_preferences` / `restore_preferences` operations below instead manage
|
|
29
|
+
the device's own FLASH/SD preference backup; they do not export a file to the host.
|
|
30
|
+
|
|
27
31
|
| Group | Methods and payload arguments |
|
|
28
32
|
| --- | --- |
|
|
29
33
|
| Identity | `get_owner`; `set_owner(owner:)` or `set_owner(long_name:, short_name:)`; `set_ham_mode(ham:)` or callsign/frequency/power/name fields |
|
|
@@ -0,0 +1,149 @@
|
|
|
1
|
+
# ATAK Forwarder (port 257)
|
|
2
|
+
|
|
3
|
+
`require 'meshtastic/forwarder'` loads a Ruby decoder independent of ATAK V1/V2.
|
|
4
|
+
The caller must preserve the original payload when decoding fails.
|
|
5
|
+
|
|
6
|
+
```ruby
|
|
7
|
+
# One actual port-257 packet, including the chunk header:
|
|
8
|
+
result = Meshtastic::Forwarder.decode_packet(payload: bytes)
|
|
9
|
+
|
|
10
|
+
# A complete explicitly grouped set of header-bearing chunks (any order):
|
|
11
|
+
result = Meshtastic::Forwarder.decode_chunks(chunks: chunks)
|
|
12
|
+
|
|
13
|
+
# Already reassembled libcotshrink bytes, without the chunk header:
|
|
14
|
+
result = Meshtastic::Forwarder.decode(payload: bytes)
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
## Implemented support and precise limits
|
|
18
|
+
|
|
19
|
+
* `:libcotshrink_protobuf` results contain `:protobuf`, a schema-level hash
|
|
20
|
+
covering the complete 43-file upstream CoT descriptor graph, including all
|
|
21
|
+
22 Detail fields, nested shapes, routes, GeoChat, video, sensors and medevac.
|
|
22
|
+
This is **not** Meshtastic TAKPacket or standard TAK Protocol CotEvent.
|
|
23
|
+
* `:event` additionally interprets UID, omitted PLI type, scaled latitude and
|
|
24
|
+
longitude, CE/LE, packed HAE, time offset and stale interval. `:extensions`
|
|
25
|
+
interprets all 16 customBytesExt fields, including nullable values and mapped
|
|
26
|
+
strings. These conversions follow libcotshrink's own converters.
|
|
27
|
+
* Nested Detail values remain **schema-level encoded values**: this module does
|
|
28
|
+
not implement all per-detail scaling, string substitutions, defaults or CoT
|
|
29
|
+
XML reconstruction. A decoded protobuf hash is not a full reconstructed CoT
|
|
30
|
+
event. The packed source values remain in `:protobuf`.
|
|
31
|
+
* Optional GZIP wrapping is detected by its magic bytes. Both compressed input
|
|
32
|
+
and decompressed output are bounded by `max_bytes` (default and hard ceiling
|
|
33
|
+
1,048,576). Malformed/truncated GZIP and trailing bytes are rejected. Plain
|
|
34
|
+
zlib/DEFLATE is not the libcotshrink wrapper.
|
|
35
|
+
* Year and timezone are **not transmitted**. Upstream derives its epoch using
|
|
36
|
+
local `Calendar`, January 1 of its startup year. Without context the decoder
|
|
37
|
+
returns `time_offset_seconds` and `stale_after_seconds`; it does not guess.
|
|
38
|
+
Supply `start_of_year: Time.new(2025, 1, 1, 0, 0, 0, '+00:00')` only when the
|
|
39
|
+
sender's epoch is known. Then `time`, `start`, and `stale` are emitted as UTC
|
|
40
|
+
ISO 8601 strings. This matters for archived messages and New Year boundaries.
|
|
41
|
+
* Invalid mapped values, malformed protobuf, and size violations raise
|
|
42
|
+
`ArgumentError`. EXI raises `Meshtastic::Forwarder::UnsupportedFormat`, a
|
|
43
|
+
subclass of `ArgumentError`. Empty reassembled bytes remain a valid default
|
|
44
|
+
proto3 message; an empty radio packet lacks its mandatory chunk header.
|
|
45
|
+
|
|
46
|
+
### Framing is not optional
|
|
47
|
+
|
|
48
|
+
Upstream MeshSender prepends **one byte**: high nibble is zero-based chunk
|
|
49
|
+
index, low nibble is total chunk count (1–15). Its normal chunk body is at most
|
|
50
|
+
200 bytes. Even an unfragmented message begins with `0x01`, **not `0x00`**.
|
|
51
|
+
`decode_packet` strips that byte and decodes a single-chunk message. For a
|
|
52
|
+
multi-chunk message it returns `format: :forwarder_fragment`, `index`, `count`,
|
|
53
|
+
and binary `body`; it never treats a fragment as a complete protobuf.
|
|
54
|
+
|
|
55
|
+
`decode_chunks` requires every index exactly once with a consistent count,
|
|
56
|
+
rejects missing/duplicate chunks, orders the chunks, bounds aggregate bytes,
|
|
57
|
+
and decodes only after reassembly. It is deliberately stateless. Integration
|
|
58
|
+
must group by connection/channel/sender and impose a short lifetime and a
|
|
59
|
+
bounded queue. The upstream header has **no message ID**, so concurrent
|
|
60
|
+
same-sender messages cannot be disambiguated reliably from payload bytes.
|
|
61
|
+
Do not silently merge unrelated generations. A Meshtastic packet ID identifies
|
|
62
|
+
one radio packet, not all chunks of an application message.
|
|
63
|
+
|
|
64
|
+
`ATAKBCAST,mesh-id,uid,callsign,initial` discovery broadcasts are returned as
|
|
65
|
+
`:forwarder_discovery`, not sent through a CoT decoder.
|
|
66
|
+
|
|
67
|
+
## EXI blocker — full Forwarder decoding is not complete
|
|
68
|
+
|
|
69
|
+
libcotshrink's lossless mode uses EXIficient `DefaultEXIFactory.newInstance()`;
|
|
70
|
+
its lossy mode falls back to that mode for unsupported CoT details. Either can
|
|
71
|
+
be GZIP wrapped. No XSD is supplied: the required format is schema-less EXI,
|
|
72
|
+
not a fixed CoT field table. Real support needs EXI header/options handling,
|
|
73
|
+
bit-level event codes, dynamically learned XML grammars, QName/string-table
|
|
74
|
+
partitions and EXI datatype decoding. A zlib inflater or protobuf schema cannot
|
|
75
|
+
replace this engine.
|
|
76
|
+
|
|
77
|
+
This implementation recognizes the EXI distinguishing bits/cookie and fails
|
|
78
|
+
explicitly. No working Ruby EXI engine or captured EXI wire fixture was found
|
|
79
|
+
in this research. That is an implementation/dependency gap, not a claim that
|
|
80
|
+
EXI is mathematically impossible in Ruby. The superficially named `xi_parser`
|
|
81
|
+
gem is a personal-wiki-to-HTML parser, not EXI. No Java or Python subprocess is
|
|
82
|
+
used; full EXI support remains unimplemented.
|
|
83
|
+
|
|
84
|
+
## Provenance and verification
|
|
85
|
+
|
|
86
|
+
The generated `forwarder_pb.rb` embeds FileDescriptorProto bytes from
|
|
87
|
+
`paulmandal/libcotshrink` revision
|
|
88
|
+
`7d818b0d119a8df291770923b463fbb734b7e1b0`. They were produced with `protoc
|
|
89
|
+
--include_imports --descriptor_set_out=... cotevent.proto` and registered in a
|
|
90
|
+
private Ruby DescriptorPool in dependency order. The upstream MIT license is
|
|
91
|
+
included in that generated file. Runtime requires neither protoc nor the
|
|
92
|
+
Android libraries.
|
|
93
|
+
|
|
94
|
+
Specs include an independent `protoc --encode` wire fixture, direct
|
|
95
|
+
hand-authored wire, packed-bit/null handling, nested schema decoding, actual
|
|
96
|
+
nibble framing, shuffled reassembly, missing/duplicate fragments, discovery,
|
|
97
|
+
GZIP and output limits. These are schema-conformance fixtures, **not captured
|
|
98
|
+
radio packets**. Upstream `HackyTests.java` supplies CoT XML test inputs and
|
|
99
|
+
Android runtime round-trip tests, not checked-in binary fixtures. The Android
|
|
100
|
+
implementation was inspected but not executed, and interoperability with live
|
|
101
|
+
hardware is not claimed.
|
|
102
|
+
|
|
103
|
+
Primary implementation sources:
|
|
104
|
+
|
|
105
|
+
* [CotShrinker format selection and GZIP](https://github.com/paulmandal/libcotshrink/blob/7d818b0d119a8df291770923b463fbb734b7e1b0/src/main/java/com/paulmandal/atak/libcotshrink/pub/api/CotShrinker.java)
|
|
106
|
+
* [CoT schema](https://github.com/paulmandal/libcotshrink/blob/7d818b0d119a8df291770923b463fbb734b7e1b0/src/main/proto/cotevent.proto)
|
|
107
|
+
* [Detail schema graph](https://github.com/paulmandal/libcotshrink/blob/7d818b0d119a8df291770923b463fbb734b7e1b0/src/main/proto/detail/detail.proto)
|
|
108
|
+
* [Packed time/height](https://github.com/paulmandal/libcotshrink/blob/7d818b0d119a8df291770923b463fbb734b7e1b0/src/main/java/com/paulmandal/atak/libcotshrink/protobuf/CustomBytesConverter.java)
|
|
109
|
+
* [Packed extensions/mappings](https://github.com/paulmandal/libcotshrink/blob/7d818b0d119a8df291770923b463fbb734b7e1b0/src/main/java/com/paulmandal/atak/libcotshrink/protobuf/CustomBytesExtConverter.java)
|
|
110
|
+
* [Nullable bit semantics](https://github.com/paulmandal/libcotshrink/blob/7d818b0d119a8df291770923b463fbb734b7e1b0/src/main/java/com/paulmandal/atak/libcotshrink/protobuf/utils/BitUtils.java)
|
|
111
|
+
* [Local-year epoch](https://github.com/paulmandal/libcotshrink/blob/7d818b0d119a8df291770923b463fbb734b7e1b0/src/main/java/com/paulmandal/atak/libcotshrink/protobuf/cotevent/CotEventProtobufConverterFactory.java)
|
|
112
|
+
* [EXI configuration](https://github.com/paulmandal/libcotshrink/blob/7d818b0d119a8df291770923b463fbb734b7e1b0/src/main/java/com/paulmandal/atak/libcotshrink/exi/ExiConverterFactory.java), [EXI standard](https://www.w3.org/TR/exi/)
|
|
113
|
+
* [Forwarder chunk construction](https://github.com/paulmandal/atak-forwarder/blob/ad47f2c69e45785eb6029945be232bc52acd695c/app/src/main/java/com/paulmandal/atak/forwarder/comm/meshtastic/MeshSender.java)
|
|
114
|
+
* [Forwarder chunk reception](https://github.com/paulmandal/atak-forwarder/blob/ad47f2c69e45785eb6029945be232bc52acd695c/app/src/main/java/com/paulmandal/atak/forwarder/comm/meshtastic/InboundMeshMessageHandler.java)
|
|
115
|
+
* [Upstream XML tests](https://github.com/paulmandal/libcotshrink/blob/7d818b0d119a8df291770923b463fbb734b7e1b0/src/main/java/com/paulmandal/atak/libcotshrink/hackytests/HackyTests.java)
|
|
116
|
+
|
|
117
|
+
## Read-only fragmentation assessment: ports 76 and 75
|
|
118
|
+
|
|
119
|
+
### Reticulum tunnel (76)
|
|
120
|
+
|
|
121
|
+
The inspected [RNS Meshtastic interface, revision e5eb5d23](https://github.com/landandair/RNS_Over_Meshtastic/blob/e5eb5d23a619958ae0556c88fcbc4bf1d867ab0a/Interface/Meshtastic_Interface.py)
|
|
122
|
+
uses a two-byte `Bb` header: unsigned message index and signed fragment
|
|
123
|
+
position. Positions start at 1; a negative position marks the final fragment
|
|
124
|
+
and its absolute value is the total count. `REQ` followed by the same two-byte
|
|
125
|
+
metadata requests retransmission; it is not RNS packet data. Reassembly must
|
|
126
|
+
key by sender and message index (plus connection/channel isolation), wait for
|
|
127
|
+
every position through the known last index, strip headers, and only then
|
|
128
|
+
parse the RNS packet. Out-of-order final-first delivery, duplicate conflicts,
|
|
129
|
+
index wrap/reuse, timeouts, maximum packet bytes and bounded state all need
|
|
130
|
+
coverage. An encrypted RNS inner payload cannot be made plaintext by tunnel
|
|
131
|
+
reassembly. This assessment does not assert every third-party port-76 sender
|
|
132
|
+
uses this identical protocol. [Reticulum](reticulum.md) implements this pinned
|
|
133
|
+
tunnel framing and explicitly grouped, bounded reassembly; the receive dispatcher
|
|
134
|
+
decodes individual frames only. Inner RNS parsing and decryption remain unsupported.
|
|
135
|
+
|
|
136
|
+
### LoRaWAN bridge (75)
|
|
137
|
+
|
|
138
|
+
The [pinned authoritative bridge protobuf](https://github.com/meshtastic/protobufs/blob/51028ca5a6945c76d3977c2bb803f9947d319ac5/meshtastic/lorawan_bridge.proto)
|
|
139
|
+
explicitly defines fragmentation: Uplink/Downlink `chunk_count == 0` is a full
|
|
140
|
+
PHY frame; `chunk_count == 2` means `payload` is its first part and `payload_id`
|
|
141
|
+
is 1–255. A separate `PayloadChunk` has the same ID, `chunk_index == 1` and the
|
|
142
|
+
remaining `payload_chunk`. Reassembly keys on sender and payload ID; retain
|
|
143
|
+
head RF metadata, append exactly one continuation and reject other counts or
|
|
144
|
+
indexes. Isolate connections/channels/directions, expire incomplete entries,
|
|
145
|
+
handle continuation-before-head, and reject conflicting duplicates. This is
|
|
146
|
+
not the RNS signed-position format. Only the reassembled LoRaWAN PHYPayload
|
|
147
|
+
should reach a LoRaWAN frame parser; application plaintext also needs the
|
|
148
|
+
correct LoRaWAN security context. TxResult correlates by `request_id`, not
|
|
149
|
+
`tmst`. No LoRaWAN transport/reassembly code was changed.
|