openloggernet 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- openloggernet-0.1.0/.gitignore +6 -0
- openloggernet-0.1.0/LICENSE +21 -0
- openloggernet-0.1.0/PKG-INFO +274 -0
- openloggernet-0.1.0/README.md +258 -0
- openloggernet-0.1.0/docs/lessons.md +44 -0
- openloggernet-0.1.0/docs/pakbus.md +275 -0
- openloggernet-0.1.0/pyproject.toml +58 -0
- openloggernet-0.1.0/src/openloggernet/__init__.py +0 -0
- openloggernet-0.1.0/src/openloggernet/collect_data.py +113 -0
- openloggernet-0.1.0/src/openloggernet/comusb_capture.py +121 -0
- openloggernet-0.1.0/src/openloggernet/cr6comp.py +102 -0
- openloggernet-0.1.0/src/openloggernet/flash.py +240 -0
- openloggernet-0.1.0/src/openloggernet/logger_clock.py +84 -0
- openloggernet-0.1.0/src/openloggernet/pakbus.py +1142 -0
- openloggernet-0.1.0/src/openloggernet/read_values.py +128 -0
- openloggernet-0.1.0/src/openloggernet/set_value.py +67 -0
- openloggernet-0.1.0/tests/test_cr6comp.py +91 -0
- openloggernet-0.1.0/tests/test_pakbus.py +186 -0
- openloggernet-0.1.0/tests/test_set_value.py +21 -0
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Utrecht University
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1,274 @@
|
|
|
1
|
+
Metadata-Version: 2.5
|
|
2
|
+
Name: openloggernet
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: Open replacement for parts of the Campbell Scientific LoggerNet toolchain: PakBus/BMP5 client, compiler runner and logger CLIs
|
|
5
|
+
Project-URL: Repository, https://github.com/UtrechtUniversity/open-loggernet
|
|
6
|
+
Author: Utrecht University
|
|
7
|
+
License-Expression: MIT
|
|
8
|
+
License-File: LICENSE
|
|
9
|
+
Classifier: Operating System :: OS Independent
|
|
10
|
+
Classifier: Programming Language :: Python :: 3
|
|
11
|
+
Classifier: Topic :: Scientific/Engineering :: Atmospheric Science
|
|
12
|
+
Classifier: Topic :: System :: Hardware
|
|
13
|
+
Requires-Python: >=3.15
|
|
14
|
+
Requires-Dist: pyserial>=3.5
|
|
15
|
+
Description-Content-Type: text/markdown
|
|
16
|
+
|
|
17
|
+
# openloggernet
|
|
18
|
+
|
|
19
|
+
An open replacement for the parts of the Campbell Scientific LoggerNet toolchain that the
|
|
20
|
+
[FirnMelt stations](https://github.com/UtrechtUniversity/firnmelt-station) use: a PakBus/BMP5
|
|
21
|
+
client, a runner for Campbell's CRBasic compilers, and command-line tools that flash programs to a
|
|
22
|
+
logger and read, set and collect its values. It is not affiliated with Campbell Scientific.
|
|
23
|
+
|
|
24
|
+
```bash
|
|
25
|
+
pip install openloggernet # or: uv tool install openloggernet
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
The commands are installed as console scripts (`flash`, `read_values`, ...) and as modules
|
|
29
|
+
(`python -m openloggernet.flash`). Python 3.15 or newer is required.
|
|
30
|
+
|
|
31
|
+
## Development
|
|
32
|
+
|
|
33
|
+
```bash
|
|
34
|
+
uv sync # creates .venv with the dev tools
|
|
35
|
+
uv run pre-commit install # ruff, pyrefly and file hygiene on every commit
|
|
36
|
+
uv run pre-commit run --all-files
|
|
37
|
+
uv run pytest
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
CI (`.github/workflows/ci.yml`) runs the same checks on every push and pull request. Agent and
|
|
41
|
+
contributor conventions are in [AGENTS.md](AGENTS.md).
|
|
42
|
+
|
|
43
|
+
## Releasing
|
|
44
|
+
|
|
45
|
+
Every push to `dev` that passes CI is published to [TestPyPI](https://test.pypi.org/p/openloggernet)
|
|
46
|
+
as `<version>.dev<run number>`. A push to `main` publishes to [PyPI](https://pypi.org/p/openloggernet)
|
|
47
|
+
when the `version` in `pyproject.toml` has not been released yet, so bump it (`uv version --bump
|
|
48
|
+
patch`) in the change that should be released. Publishing uses PyPI trusted publishing from the
|
|
49
|
+
`pypi` and `testpypi` GitHub environments; no API token is stored in the repository.
|
|
50
|
+
|
|
51
|
+
| Module | What it does |
|
|
52
|
+
|---|---|
|
|
53
|
+
| `flash` | Compile a CRBasic program and send it to a connected CR6 over USB. |
|
|
54
|
+
| `read_values` | Read `Public` variables, data table fields and `Status` fields off a running CR6. |
|
|
55
|
+
| `set_value` | Set `Public` variables on a running CR6; how testers are driven. |
|
|
56
|
+
| `collect_data` | Read stored data table records off a running CR6. |
|
|
57
|
+
| `logger_clock` | Read the CR6 clock, or set it to the host's UTC time. |
|
|
58
|
+
| `cr6comp` | Runs `compiler/CR6Comp.exe` (`.CR6`) or `compiler/cr1000xcomp.exe` (`.CR1X`) through `powershell.exe` (WSL) or Wine (macOS, Linux); used by `flash`. Not a script. |
|
|
59
|
+
| `pakbus` | PakBus/BMP5 client library used by `flash`, `read_values` and `logger_clock`. Not a script. |
|
|
60
|
+
| `comusb_capture` | Captures timestamped tester output and sends keys through a FIFO. |
|
|
61
|
+
|
|
62
|
+
For a long interactive tester session:
|
|
63
|
+
|
|
64
|
+
```bash
|
|
65
|
+
comusb_capture --log run.log --fifo /tmp/cr6.key
|
|
66
|
+
printf '1' > /tmp/cr6.key
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
---
|
|
70
|
+
|
|
71
|
+
## Compiling without Windows
|
|
72
|
+
|
|
73
|
+
`cr6comp` runs `CR6Comp.exe` through `powershell.exe` on WSL and through Wine on macOS and
|
|
74
|
+
Linux (on Ubuntu: `dpkg --add-architecture i386 && apt-get install wine32:i386 wine64 wine`).
|
|
75
|
+
The compilers are Campbell's and are not part of this package: `compiler/` is looked up in the
|
|
76
|
+
program's directory and then each parent, so a program under `crbasic/` finds `../compiler/`. Copy
|
|
77
|
+
`CR6Comp.exe` (and `cr1000xcomp.exe` for `.CR1X`) there from a CRBasic Editor or Device
|
|
78
|
+
Configuration Utility install.
|
|
79
|
+
`CR6COMP=opencrbasic` uses the native re-implementation from the
|
|
80
|
+
[opencrbasic](https://github.com/UtrechtUniversity/open-crbasic) package instead (`pip install
|
|
81
|
+
opencrbasic`); it reproduces CR6Comp's output for the FirnMelt programs but is not the reference
|
|
82
|
+
compiler, so keep the exe as the gate before flashing hardware.
|
|
83
|
+
|
|
84
|
+
## Compiling on macOS
|
|
85
|
+
|
|
86
|
+
`CR6Comp.exe` is a 32-bit Windows console program that imports only `KERNEL32.dll`, so it runs
|
|
87
|
+
under Wine; on Apple Silicon, Wine's x86-64 code runs through Rosetta 2. The Homebrew wine casks
|
|
88
|
+
were disabled on 2026-09-01 for failing Gatekeeper, so install the same Gcenx build they used,
|
|
89
|
+
without admin rights:
|
|
90
|
+
|
|
91
|
+
```bash
|
|
92
|
+
softwareupdate --install-rosetta --agree-to-license
|
|
93
|
+
mkdir -p ~/.local/wine
|
|
94
|
+
curl -L https://github.com/Gcenx/macOS_Wine_builds/releases/download/11.17/wine-devel-11.17-osx64.tar.xz | tar -xJ -C ~/.local/wine
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
`cr6comp` looks for Wine in `$CR6_WINE`, then at
|
|
98
|
+
`~/.local/wine/Wine Devel.app/Contents/Resources/wine/bin/wine`, then as `wine` on `PATH`. It
|
|
99
|
+
uses the prefix `~/.wine-cr6` unless `WINEPREFIX` is set; Wine creates it on first use, which takes
|
|
100
|
+
about 15 s. `flash` then compiles exactly as on WSL.
|
|
101
|
+
|
|
102
|
+
Checked on 2026-09-23 with Wine 11.17 on an arm64 Mac: every entry program compiled; a syntax error
|
|
103
|
+
in a `.CR6` or in an included `.crb` gave `Compile Failed!` with its line and exit status 1; and the
|
|
104
|
+
FirnMelt tree at commit `da10004` reproduced the PipelineMode results its bench captures record. The
|
|
105
|
+
compiler writes `CR6cmple.xxx` into `crbasic/`, which git ignores.
|
|
106
|
+
|
|
107
|
+
Checked on 2026-10-20 with Ubuntu's Wine 10.0 package (`10.0~repack-12ubuntu1`) on x86-64 Ubuntu
|
|
108
|
+
26.04, installed with the command under [Compiling without Windows](#compiling-without-windows):
|
|
109
|
+
the FirnMelt `scripts/verify.py` compiled all 19 entry programs, `FinAppAcceptance.CR1X` included, with six warnings, all
|
|
110
|
+
for tables declared but never called. The compile-failure path was not exercised on this host.
|
|
111
|
+
|
|
112
|
+
---
|
|
113
|
+
|
|
114
|
+
## Reading stored records: `collect_data`
|
|
115
|
+
|
|
116
|
+
`read_values` sees only the newest value of a field. This reads the tables themselves, which is
|
|
117
|
+
what makes a run's full-resolution data survive: the serial trace carries only what the program
|
|
118
|
+
chose to print, and flashing deletes the tables.
|
|
119
|
+
|
|
120
|
+
```bash
|
|
121
|
+
collect_data --list # the logger's tables, sizes and intervals
|
|
122
|
+
collect_data DailyPower # the power-profile decision history
|
|
123
|
+
collect_data Shunt_Data --count 60 --csv shunt.csv
|
|
124
|
+
collect_data TString_Data --count 5 # all 48 sensors per record
|
|
125
|
+
```
|
|
126
|
+
|
|
127
|
+
Table definitions come from the logger's own `.TDF`, so field names, types and array shapes are
|
|
128
|
+
whatever the running program actually declared. A reply holds about 1 KB of records, so a long table is walked page
|
|
129
|
+
by page, each page re-anchored on the logger's newest record because the table keeps growing during
|
|
130
|
+
collection; a page that would leave a gap raises instead. Ten days of the 60 s `StationStatus` table
|
|
131
|
+
takes a few minutes. Records whose size exceeds one packet -- `Status`,
|
|
132
|
+
`Public`, `DataTableInfo` -- cannot be collected; read those with `read_values`. Details and the
|
|
133
|
+
behaviours this depends on are in [docs/pakbus.md](docs/pakbus.md).
|
|
134
|
+
|
|
135
|
+
---
|
|
136
|
+
|
|
137
|
+
## Setting the clock: `logger_clock`
|
|
138
|
+
|
|
139
|
+
The FirnMelt `Main.CR6` derives every schedule index from the logger's own clock and requires
|
|
140
|
+
that clock to be UTC. The logger stores no time zone, so a logger left on local time
|
|
141
|
+
silently shifts the whole grid, the daily aggregation boundary and the SBD send unit of day, with
|
|
142
|
+
no symptom other than things happening at the wrong time.
|
|
143
|
+
|
|
144
|
+
```bash
|
|
145
|
+
logger_clock # report logger time and its offset from host UTC
|
|
146
|
+
logger_clock --set-utc # set it, then measure the result
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
The set is reported as an adjustment the logger applies to itself, so the transport delay is what
|
|
150
|
+
limits accuracy; the script measures the outcome afterwards rather than assuming it, and exits
|
|
151
|
+
non-zero if more than `--max-drift` seconds remain. The bench logger was found 7201 s fast on
|
|
152
|
+
2026-09-03, which is CEST, and was corrected to within 1 ms.
|
|
153
|
+
|
|
154
|
+
---
|
|
155
|
+
|
|
156
|
+
## Writing values to a running program: `set_value`
|
|
157
|
+
|
|
158
|
+
The counterpart to `read_values`, over `pakbus`'s Set Values transaction:
|
|
159
|
+
|
|
160
|
+
```bash
|
|
161
|
+
set_value Public.DoRead=true # true/false are Booleans (-1/0)
|
|
162
|
+
set_value Public.ProfileOverride=0 # whole numbers go as Long
|
|
163
|
+
set_value Public.ForcedBattV=11.8 # anything else as IEEE4
|
|
164
|
+
```
|
|
165
|
+
|
|
166
|
+
This is how testers are driven: `TDR_tester.CR6` has no key menu and acts on its `Do*` flags, and
|
|
167
|
+
the others are following. It is also how a long run's power profile is changed without
|
|
168
|
+
re-flashing, which would delete the run's data tables, so check the port before writing to a
|
|
169
|
+
logger that is not on the bench.
|
|
170
|
+
|
|
171
|
+
---
|
|
172
|
+
|
|
173
|
+
## Flashing a program: `flash`
|
|
174
|
+
|
|
175
|
+
Replaces the manual "send the program with Device Configuration Utility" step. It compiles with
|
|
176
|
+
the validated `compiler/CR6Comp.exe`, writes the program **and the libraries it includes** to the
|
|
177
|
+
logger's `CPU:` drive, tells the logger to compile and run it, and reads the logger's own compile
|
|
178
|
+
results back out.
|
|
179
|
+
|
|
180
|
+
```bash
|
|
181
|
+
flash ../crbasic/PowerProfile_tester.CR6 # compile, send, run
|
|
182
|
+
flash ../crbasic/PowerProfile_tester.CR6 --monitor 60 --send '\r'
|
|
183
|
+
flash ../crbasic/Main.CR6 --no-run # write the file only
|
|
184
|
+
```
|
|
185
|
+
|
|
186
|
+
The `--monitor` window prints whatever the running program writes to `ComUSB`, so a flash and the
|
|
187
|
+
start of a bench session are one command. The tester programs are menu driven and stay silent
|
|
188
|
+
until they get a keystroke, hence `--send '\r'`.
|
|
189
|
+
|
|
190
|
+
Exit status is 0 only when the logger reports a clean compile; a compile error on the logger is
|
|
191
|
+
printed and returns 1.
|
|
192
|
+
|
|
193
|
+
A `.CR1X` program is compiled with `cr1000xcomp.exe` and sent the same way to the bench CR1000X.
|
|
194
|
+
Files go in 256-byte packets; see [docs/lessons.md](docs/lessons.md#cr1000x-file-downloads-need-256-byte-packets).
|
|
195
|
+
|
|
196
|
+
### Requirements
|
|
197
|
+
|
|
198
|
+
* The CR6's USB port attached to WSL (`usbipd attach`), where it appears as
|
|
199
|
+
`/dev/serial/by-id/usb-Campbell_Scientific_Inc._CR6_*`. `flash` finds it there by itself;
|
|
200
|
+
`--port` overrides.
|
|
201
|
+
* Your user in the `dialout` group.
|
|
202
|
+
* `powershell.exe` (WSL) or Wine (macOS/Linux, see [Compiling on macOS](#compiling-on-macos)) for
|
|
203
|
+
`CR6Comp.exe`. Use `--no-compile` to skip that step.
|
|
204
|
+
|
|
205
|
+
### Useful options
|
|
206
|
+
|
|
207
|
+
| Option | Purpose |
|
|
208
|
+
|---|---|
|
|
209
|
+
| `--monitor SECONDS`, `--send TEXT` | Stream `ComUSB` output after flashing; write `TEXT` first. |
|
|
210
|
+
| `--no-run` | Write the files but leave the running program alone. |
|
|
211
|
+
| `--no-compile` | Skip `CR6Comp.exe` and send the file as-is. |
|
|
212
|
+
| `--no-includes` | Do not send the `IncludeSection` libraries alongside the program. |
|
|
213
|
+
| `--address`, `--my-address` | PakBus addresses (logger default 1, host default 4094). |
|
|
214
|
+
| `--debug` | Dump every PakBus packet as hex. |
|
|
215
|
+
|
|
216
|
+
---
|
|
217
|
+
|
|
218
|
+
## Reading state off a running logger: `read_values`
|
|
219
|
+
|
|
220
|
+
The counterpart to `flash`. `flash` puts a program on the logger and shows what it writes to
|
|
221
|
+
`ComUSB`; `read_values` shows the state of a program that writes nothing. **Most libraries here
|
|
222
|
+
are exercised from a slow sequence, which produces no serial output at all**, so without this their
|
|
223
|
+
variables are unobservable while `Main.CR6` runs.
|
|
224
|
+
|
|
225
|
+
```bash
|
|
226
|
+
read_values Public.PT1000_Temp_C Public.AmbientForDecision Status.SkippedScan
|
|
227
|
+
read_values --type long --swath 9 Public.PowerRTime # an array
|
|
228
|
+
read_values --watch 30 Public.PT1000Ok # poll, print on change
|
|
229
|
+
```
|
|
230
|
+
|
|
231
|
+
Each argument is `Table.Field`. The `Public` table, the `Status` table and the program's own data
|
|
232
|
+
tables are all readable, and an array element can be named directly (`Public.Acclima_VWC(2)`).
|
|
233
|
+
|
|
234
|
+
`--watch SECONDS` polls until interrupted and prints **only when a value changes**, which is how to
|
|
235
|
+
wait out a gated slow sequence without guessing when it fired. A bad field name inside a watch is
|
|
236
|
+
reported and the watch continues, so one typo does not end a long observation.
|
|
237
|
+
|
|
238
|
+
### Types
|
|
239
|
+
|
|
240
|
+
`--type` chooses the representation to ask the logger for. The default, `asciiz`, asks the logger
|
|
241
|
+
to format the value itself: it reads any type, renders a `Boolean` as `False` and a `NAN` as `NAN`,
|
|
242
|
+
and is the right choice unless you need exact binary or an array.
|
|
243
|
+
|
|
244
|
+
| `--type` | Use for | Note |
|
|
245
|
+
|---|---|---|
|
|
246
|
+
| `asciiz` | anything; the default | **ignores `--swath` and returns one value** |
|
|
247
|
+
| `ieee4` | `Float` / `IEEE4` | exact; honours `--swath` |
|
|
248
|
+
| `long` | `As Long` | exact; honours `--swath` |
|
|
249
|
+
| `ieee8` | extra precision | honours `--swath` |
|
|
250
|
+
|
|
251
|
+
A `Boolean` read as `long` comes back from the logger as *response code 0 with an empty body* rather
|
|
252
|
+
than as an error, so the library checks the returned byte count and raises instead of silently
|
|
253
|
+
returning nothing. Read Booleans as `asciiz`. The wire format, the type codes and how each was
|
|
254
|
+
established on the bench are in [docs/pakbus.md](docs/pakbus.md).
|
|
255
|
+
|
|
256
|
+
### Requirements
|
|
257
|
+
|
|
258
|
+
The same as `flash`: the CR6's USB port attached to WSL, and your user in `dialout`. It does not
|
|
259
|
+
need `powershell.exe`, since nothing is compiled.
|
|
260
|
+
|
|
261
|
+
---
|
|
262
|
+
|
|
263
|
+
There is deliberately **no** option for a PakBus security code. This is a bench tool for
|
|
264
|
+
validation work on an unsecured logger; it sends a security code of 0 and has no way to send
|
|
265
|
+
anything else, so it cannot be pointed at a secured station.
|
|
266
|
+
|
|
267
|
+
### What it does to the logger
|
|
268
|
+
|
|
269
|
+
Sending a program is the same operation Device Configuration Utility performs: the file is marked
|
|
270
|
+
**run on power up**, and the logger deletes the data tables of the program it replaces. The logger
|
|
271
|
+
disconnects and re-enumerates its USB device while it compiles — `flash` waits for it and
|
|
272
|
+
reopens the port, which may be a different `/dev/ttyACM*` node than before.
|
|
273
|
+
|
|
274
|
+
Protocol details and what has been verified on hardware: [docs/pakbus.md](docs/pakbus.md).
|
|
@@ -0,0 +1,258 @@
|
|
|
1
|
+
# openloggernet
|
|
2
|
+
|
|
3
|
+
An open replacement for the parts of the Campbell Scientific LoggerNet toolchain that the
|
|
4
|
+
[FirnMelt stations](https://github.com/UtrechtUniversity/firnmelt-station) use: a PakBus/BMP5
|
|
5
|
+
client, a runner for Campbell's CRBasic compilers, and command-line tools that flash programs to a
|
|
6
|
+
logger and read, set and collect its values. It is not affiliated with Campbell Scientific.
|
|
7
|
+
|
|
8
|
+
```bash
|
|
9
|
+
pip install openloggernet # or: uv tool install openloggernet
|
|
10
|
+
```
|
|
11
|
+
|
|
12
|
+
The commands are installed as console scripts (`flash`, `read_values`, ...) and as modules
|
|
13
|
+
(`python -m openloggernet.flash`). Python 3.15 or newer is required.
|
|
14
|
+
|
|
15
|
+
## Development
|
|
16
|
+
|
|
17
|
+
```bash
|
|
18
|
+
uv sync # creates .venv with the dev tools
|
|
19
|
+
uv run pre-commit install # ruff, pyrefly and file hygiene on every commit
|
|
20
|
+
uv run pre-commit run --all-files
|
|
21
|
+
uv run pytest
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
CI (`.github/workflows/ci.yml`) runs the same checks on every push and pull request. Agent and
|
|
25
|
+
contributor conventions are in [AGENTS.md](AGENTS.md).
|
|
26
|
+
|
|
27
|
+
## Releasing
|
|
28
|
+
|
|
29
|
+
Every push to `dev` that passes CI is published to [TestPyPI](https://test.pypi.org/p/openloggernet)
|
|
30
|
+
as `<version>.dev<run number>`. A push to `main` publishes to [PyPI](https://pypi.org/p/openloggernet)
|
|
31
|
+
when the `version` in `pyproject.toml` has not been released yet, so bump it (`uv version --bump
|
|
32
|
+
patch`) in the change that should be released. Publishing uses PyPI trusted publishing from the
|
|
33
|
+
`pypi` and `testpypi` GitHub environments; no API token is stored in the repository.
|
|
34
|
+
|
|
35
|
+
| Module | What it does |
|
|
36
|
+
|---|---|
|
|
37
|
+
| `flash` | Compile a CRBasic program and send it to a connected CR6 over USB. |
|
|
38
|
+
| `read_values` | Read `Public` variables, data table fields and `Status` fields off a running CR6. |
|
|
39
|
+
| `set_value` | Set `Public` variables on a running CR6; how testers are driven. |
|
|
40
|
+
| `collect_data` | Read stored data table records off a running CR6. |
|
|
41
|
+
| `logger_clock` | Read the CR6 clock, or set it to the host's UTC time. |
|
|
42
|
+
| `cr6comp` | Runs `compiler/CR6Comp.exe` (`.CR6`) or `compiler/cr1000xcomp.exe` (`.CR1X`) through `powershell.exe` (WSL) or Wine (macOS, Linux); used by `flash`. Not a script. |
|
|
43
|
+
| `pakbus` | PakBus/BMP5 client library used by `flash`, `read_values` and `logger_clock`. Not a script. |
|
|
44
|
+
| `comusb_capture` | Captures timestamped tester output and sends keys through a FIFO. |
|
|
45
|
+
|
|
46
|
+
For a long interactive tester session:
|
|
47
|
+
|
|
48
|
+
```bash
|
|
49
|
+
comusb_capture --log run.log --fifo /tmp/cr6.key
|
|
50
|
+
printf '1' > /tmp/cr6.key
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
---
|
|
54
|
+
|
|
55
|
+
## Compiling without Windows
|
|
56
|
+
|
|
57
|
+
`cr6comp` runs `CR6Comp.exe` through `powershell.exe` on WSL and through Wine on macOS and
|
|
58
|
+
Linux (on Ubuntu: `dpkg --add-architecture i386 && apt-get install wine32:i386 wine64 wine`).
|
|
59
|
+
The compilers are Campbell's and are not part of this package: `compiler/` is looked up in the
|
|
60
|
+
program's directory and then each parent, so a program under `crbasic/` finds `../compiler/`. Copy
|
|
61
|
+
`CR6Comp.exe` (and `cr1000xcomp.exe` for `.CR1X`) there from a CRBasic Editor or Device
|
|
62
|
+
Configuration Utility install.
|
|
63
|
+
`CR6COMP=opencrbasic` uses the native re-implementation from the
|
|
64
|
+
[opencrbasic](https://github.com/UtrechtUniversity/open-crbasic) package instead (`pip install
|
|
65
|
+
opencrbasic`); it reproduces CR6Comp's output for the FirnMelt programs but is not the reference
|
|
66
|
+
compiler, so keep the exe as the gate before flashing hardware.
|
|
67
|
+
|
|
68
|
+
## Compiling on macOS
|
|
69
|
+
|
|
70
|
+
`CR6Comp.exe` is a 32-bit Windows console program that imports only `KERNEL32.dll`, so it runs
|
|
71
|
+
under Wine; on Apple Silicon, Wine's x86-64 code runs through Rosetta 2. The Homebrew wine casks
|
|
72
|
+
were disabled on 2026-09-01 for failing Gatekeeper, so install the same Gcenx build they used,
|
|
73
|
+
without admin rights:
|
|
74
|
+
|
|
75
|
+
```bash
|
|
76
|
+
softwareupdate --install-rosetta --agree-to-license
|
|
77
|
+
mkdir -p ~/.local/wine
|
|
78
|
+
curl -L https://github.com/Gcenx/macOS_Wine_builds/releases/download/11.17/wine-devel-11.17-osx64.tar.xz | tar -xJ -C ~/.local/wine
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
`cr6comp` looks for Wine in `$CR6_WINE`, then at
|
|
82
|
+
`~/.local/wine/Wine Devel.app/Contents/Resources/wine/bin/wine`, then as `wine` on `PATH`. It
|
|
83
|
+
uses the prefix `~/.wine-cr6` unless `WINEPREFIX` is set; Wine creates it on first use, which takes
|
|
84
|
+
about 15 s. `flash` then compiles exactly as on WSL.
|
|
85
|
+
|
|
86
|
+
Checked on 2026-09-23 with Wine 11.17 on an arm64 Mac: every entry program compiled; a syntax error
|
|
87
|
+
in a `.CR6` or in an included `.crb` gave `Compile Failed!` with its line and exit status 1; and the
|
|
88
|
+
FirnMelt tree at commit `da10004` reproduced the PipelineMode results its bench captures record. The
|
|
89
|
+
compiler writes `CR6cmple.xxx` into `crbasic/`, which git ignores.
|
|
90
|
+
|
|
91
|
+
Checked on 2026-10-20 with Ubuntu's Wine 10.0 package (`10.0~repack-12ubuntu1`) on x86-64 Ubuntu
|
|
92
|
+
26.04, installed with the command under [Compiling without Windows](#compiling-without-windows):
|
|
93
|
+
the FirnMelt `scripts/verify.py` compiled all 19 entry programs, `FinAppAcceptance.CR1X` included, with six warnings, all
|
|
94
|
+
for tables declared but never called. The compile-failure path was not exercised on this host.
|
|
95
|
+
|
|
96
|
+
---
|
|
97
|
+
|
|
98
|
+
## Reading stored records: `collect_data`
|
|
99
|
+
|
|
100
|
+
`read_values` sees only the newest value of a field. This reads the tables themselves, which is
|
|
101
|
+
what makes a run's full-resolution data survive: the serial trace carries only what the program
|
|
102
|
+
chose to print, and flashing deletes the tables.
|
|
103
|
+
|
|
104
|
+
```bash
|
|
105
|
+
collect_data --list # the logger's tables, sizes and intervals
|
|
106
|
+
collect_data DailyPower # the power-profile decision history
|
|
107
|
+
collect_data Shunt_Data --count 60 --csv shunt.csv
|
|
108
|
+
collect_data TString_Data --count 5 # all 48 sensors per record
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
Table definitions come from the logger's own `.TDF`, so field names, types and array shapes are
|
|
112
|
+
whatever the running program actually declared. A reply holds about 1 KB of records, so a long table is walked page
|
|
113
|
+
by page, each page re-anchored on the logger's newest record because the table keeps growing during
|
|
114
|
+
collection; a page that would leave a gap raises instead. Ten days of the 60 s `StationStatus` table
|
|
115
|
+
takes a few minutes. Records whose size exceeds one packet -- `Status`,
|
|
116
|
+
`Public`, `DataTableInfo` -- cannot be collected; read those with `read_values`. Details and the
|
|
117
|
+
behaviours this depends on are in [docs/pakbus.md](docs/pakbus.md).
|
|
118
|
+
|
|
119
|
+
---
|
|
120
|
+
|
|
121
|
+
## Setting the clock: `logger_clock`
|
|
122
|
+
|
|
123
|
+
The FirnMelt `Main.CR6` derives every schedule index from the logger's own clock and requires
|
|
124
|
+
that clock to be UTC. The logger stores no time zone, so a logger left on local time
|
|
125
|
+
silently shifts the whole grid, the daily aggregation boundary and the SBD send unit of day, with
|
|
126
|
+
no symptom other than things happening at the wrong time.
|
|
127
|
+
|
|
128
|
+
```bash
|
|
129
|
+
logger_clock # report logger time and its offset from host UTC
|
|
130
|
+
logger_clock --set-utc # set it, then measure the result
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
The set is reported as an adjustment the logger applies to itself, so the transport delay is what
|
|
134
|
+
limits accuracy; the script measures the outcome afterwards rather than assuming it, and exits
|
|
135
|
+
non-zero if more than `--max-drift` seconds remain. The bench logger was found 7201 s fast on
|
|
136
|
+
2026-09-03, which is CEST, and was corrected to within 1 ms.
|
|
137
|
+
|
|
138
|
+
---
|
|
139
|
+
|
|
140
|
+
## Writing values to a running program: `set_value`
|
|
141
|
+
|
|
142
|
+
The counterpart to `read_values`, over `pakbus`'s Set Values transaction:
|
|
143
|
+
|
|
144
|
+
```bash
|
|
145
|
+
set_value Public.DoRead=true # true/false are Booleans (-1/0)
|
|
146
|
+
set_value Public.ProfileOverride=0 # whole numbers go as Long
|
|
147
|
+
set_value Public.ForcedBattV=11.8 # anything else as IEEE4
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
This is how testers are driven: `TDR_tester.CR6` has no key menu and acts on its `Do*` flags, and
|
|
151
|
+
the others are following. It is also how a long run's power profile is changed without
|
|
152
|
+
re-flashing, which would delete the run's data tables, so check the port before writing to a
|
|
153
|
+
logger that is not on the bench.
|
|
154
|
+
|
|
155
|
+
---
|
|
156
|
+
|
|
157
|
+
## Flashing a program: `flash`
|
|
158
|
+
|
|
159
|
+
Replaces the manual "send the program with Device Configuration Utility" step. It compiles with
|
|
160
|
+
the validated `compiler/CR6Comp.exe`, writes the program **and the libraries it includes** to the
|
|
161
|
+
logger's `CPU:` drive, tells the logger to compile and run it, and reads the logger's own compile
|
|
162
|
+
results back out.
|
|
163
|
+
|
|
164
|
+
```bash
|
|
165
|
+
flash ../crbasic/PowerProfile_tester.CR6 # compile, send, run
|
|
166
|
+
flash ../crbasic/PowerProfile_tester.CR6 --monitor 60 --send '\r'
|
|
167
|
+
flash ../crbasic/Main.CR6 --no-run # write the file only
|
|
168
|
+
```
|
|
169
|
+
|
|
170
|
+
The `--monitor` window prints whatever the running program writes to `ComUSB`, so a flash and the
|
|
171
|
+
start of a bench session are one command. The tester programs are menu driven and stay silent
|
|
172
|
+
until they get a keystroke, hence `--send '\r'`.
|
|
173
|
+
|
|
174
|
+
Exit status is 0 only when the logger reports a clean compile; a compile error on the logger is
|
|
175
|
+
printed and returns 1.
|
|
176
|
+
|
|
177
|
+
A `.CR1X` program is compiled with `cr1000xcomp.exe` and sent the same way to the bench CR1000X.
|
|
178
|
+
Files go in 256-byte packets; see [docs/lessons.md](docs/lessons.md#cr1000x-file-downloads-need-256-byte-packets).
|
|
179
|
+
|
|
180
|
+
### Requirements
|
|
181
|
+
|
|
182
|
+
* The CR6's USB port attached to WSL (`usbipd attach`), where it appears as
|
|
183
|
+
`/dev/serial/by-id/usb-Campbell_Scientific_Inc._CR6_*`. `flash` finds it there by itself;
|
|
184
|
+
`--port` overrides.
|
|
185
|
+
* Your user in the `dialout` group.
|
|
186
|
+
* `powershell.exe` (WSL) or Wine (macOS/Linux, see [Compiling on macOS](#compiling-on-macos)) for
|
|
187
|
+
`CR6Comp.exe`. Use `--no-compile` to skip that step.
|
|
188
|
+
|
|
189
|
+
### Useful options
|
|
190
|
+
|
|
191
|
+
| Option | Purpose |
|
|
192
|
+
|---|---|
|
|
193
|
+
| `--monitor SECONDS`, `--send TEXT` | Stream `ComUSB` output after flashing; write `TEXT` first. |
|
|
194
|
+
| `--no-run` | Write the files but leave the running program alone. |
|
|
195
|
+
| `--no-compile` | Skip `CR6Comp.exe` and send the file as-is. |
|
|
196
|
+
| `--no-includes` | Do not send the `IncludeSection` libraries alongside the program. |
|
|
197
|
+
| `--address`, `--my-address` | PakBus addresses (logger default 1, host default 4094). |
|
|
198
|
+
| `--debug` | Dump every PakBus packet as hex. |
|
|
199
|
+
|
|
200
|
+
---
|
|
201
|
+
|
|
202
|
+
## Reading state off a running logger: `read_values`
|
|
203
|
+
|
|
204
|
+
The counterpart to `flash`. `flash` puts a program on the logger and shows what it writes to
|
|
205
|
+
`ComUSB`; `read_values` shows the state of a program that writes nothing. **Most libraries here
|
|
206
|
+
are exercised from a slow sequence, which produces no serial output at all**, so without this their
|
|
207
|
+
variables are unobservable while `Main.CR6` runs.
|
|
208
|
+
|
|
209
|
+
```bash
|
|
210
|
+
read_values Public.PT1000_Temp_C Public.AmbientForDecision Status.SkippedScan
|
|
211
|
+
read_values --type long --swath 9 Public.PowerRTime # an array
|
|
212
|
+
read_values --watch 30 Public.PT1000Ok # poll, print on change
|
|
213
|
+
```
|
|
214
|
+
|
|
215
|
+
Each argument is `Table.Field`. The `Public` table, the `Status` table and the program's own data
|
|
216
|
+
tables are all readable, and an array element can be named directly (`Public.Acclima_VWC(2)`).
|
|
217
|
+
|
|
218
|
+
`--watch SECONDS` polls until interrupted and prints **only when a value changes**, which is how to
|
|
219
|
+
wait out a gated slow sequence without guessing when it fired. A bad field name inside a watch is
|
|
220
|
+
reported and the watch continues, so one typo does not end a long observation.
|
|
221
|
+
|
|
222
|
+
### Types
|
|
223
|
+
|
|
224
|
+
`--type` chooses the representation to ask the logger for. The default, `asciiz`, asks the logger
|
|
225
|
+
to format the value itself: it reads any type, renders a `Boolean` as `False` and a `NAN` as `NAN`,
|
|
226
|
+
and is the right choice unless you need exact binary or an array.
|
|
227
|
+
|
|
228
|
+
| `--type` | Use for | Note |
|
|
229
|
+
|---|---|---|
|
|
230
|
+
| `asciiz` | anything; the default | **ignores `--swath` and returns one value** |
|
|
231
|
+
| `ieee4` | `Float` / `IEEE4` | exact; honours `--swath` |
|
|
232
|
+
| `long` | `As Long` | exact; honours `--swath` |
|
|
233
|
+
| `ieee8` | extra precision | honours `--swath` |
|
|
234
|
+
|
|
235
|
+
A `Boolean` read as `long` comes back from the logger as *response code 0 with an empty body* rather
|
|
236
|
+
than as an error, so the library checks the returned byte count and raises instead of silently
|
|
237
|
+
returning nothing. Read Booleans as `asciiz`. The wire format, the type codes and how each was
|
|
238
|
+
established on the bench are in [docs/pakbus.md](docs/pakbus.md).
|
|
239
|
+
|
|
240
|
+
### Requirements
|
|
241
|
+
|
|
242
|
+
The same as `flash`: the CR6's USB port attached to WSL, and your user in `dialout`. It does not
|
|
243
|
+
need `powershell.exe`, since nothing is compiled.
|
|
244
|
+
|
|
245
|
+
---
|
|
246
|
+
|
|
247
|
+
There is deliberately **no** option for a PakBus security code. This is a bench tool for
|
|
248
|
+
validation work on an unsecured logger; it sends a security code of 0 and has no way to send
|
|
249
|
+
anything else, so it cannot be pointed at a secured station.
|
|
250
|
+
|
|
251
|
+
### What it does to the logger
|
|
252
|
+
|
|
253
|
+
Sending a program is the same operation Device Configuration Utility performs: the file is marked
|
|
254
|
+
**run on power up**, and the logger deletes the data tables of the program it replaces. The logger
|
|
255
|
+
disconnects and re-enumerates its USB device while it compiles — `flash` waits for it and
|
|
256
|
+
reopens the port, which may be a different `/dev/ttyACM*` node than before.
|
|
257
|
+
|
|
258
|
+
Protocol details and what has been verified on hardware: [docs/pakbus.md](docs/pakbus.md).
|
|
@@ -0,0 +1,44 @@
|
|
|
1
|
+
# Logger communication lessons
|
|
2
|
+
|
|
3
|
+
Behaviour of Campbell loggers observed while building this package. Each entry states the rule,
|
|
4
|
+
its evidence, and what the code does about it. Evidence paths under `knowledge/test_results/` are raw
|
|
5
|
+
bench captures in the [firnmelt-station](https://github.com/UtrechtUniversity/firnmelt-station)
|
|
6
|
+
repository, where these lessons were first recorded.
|
|
7
|
+
|
|
8
|
+
## PakBus interleaves with `ComUSB`
|
|
9
|
+
|
|
10
|
+
The CR6 continues sending `0xBD`-framed PakBus traffic while a program writes text to `ComUSB`.
|
|
11
|
+
Verified in `test_result_FinAppLib_25_08_2026_15_09.md`. Capture tools must validate and remove
|
|
12
|
+
frames with a bounded filter; use `src/openloggernet/pakbus.py:TextFilter`.
|
|
13
|
+
|
|
14
|
+
## USB re-enumeration during compile
|
|
15
|
+
|
|
16
|
+
The CR6 drops and recreates its USB device while compiling a downloaded program, possibly under a
|
|
17
|
+
different `/dev/ttyACM*` name. Observed on CR6 OS 6.14.5.1 on 2026-08-26. Resolve through
|
|
18
|
+
`/dev/serial/by-id/`, wait, and reopen as `src/openloggernet/flash.py` does.
|
|
19
|
+
|
|
20
|
+
## First PakBus transaction
|
|
21
|
+
|
|
22
|
+
The first packet after opening USB was lost once while the next packet succeeded. Observed once on
|
|
23
|
+
2026-08-26 and not reproduced. Retry the opening hello; do not treat one miss as proof of link
|
|
24
|
+
failure.
|
|
25
|
+
|
|
26
|
+
## CR1000X file downloads need 256-byte packets
|
|
27
|
+
|
|
28
|
+
The bench CR1000X (serial 59624, OS `CR1000X.8.5.0`) answered PakBus hello and programming
|
|
29
|
+
statistics from a Mac but never answered a 512-byte file-download packet, twice; 256-byte packets
|
|
30
|
+
transferred every file. Observed on 2026-09-24 in
|
|
31
|
+
`knowledge/test_results/test_result_FinAppLib_24_09_2026_13_34.md`. Whether the limit belongs to
|
|
32
|
+
the CR1000X or to macOS USB is unknown, so `src/openloggernet/pakbus.py` sends 256 bytes per packet to every
|
|
33
|
+
logger.
|
|
34
|
+
|
|
35
|
+
## A resent identical program keeps its tables
|
|
36
|
+
|
|
37
|
+
File control code 1, which `flash` sends, erased `FinApp_Poll` on the bench CR1000X when the
|
|
38
|
+
program had changed, although only a `Public` variable was added and the table was untouched
|
|
39
|
+
(2026-09-24), but kept all 3212 records when the byte-identical program was sent again
|
|
40
|
+
(2026-09-28). Observed in `knowledge/test_results/test_result_FinAppLib_24_09_2026_13_34.md` and
|
|
41
|
+
`knowledge/test_results/test_result_FinAppLib_28_09_2026_08_34.md`; not tried on a CR6. A flash
|
|
42
|
+
cannot be relied on to clear a table. The CHM documents code 13 as stopping the program and
|
|
43
|
+
deleting its tables (`extracted_chm/Info/webserverapicommands1.htm`), untested here; otherwise
|
|
44
|
+
tell the runs apart by time.
|