openc3 7.4.1 → 7.5.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/bin/pipinstall +18 -5
- data/bin/pipuninstall +1 -1
- data/bin/uvinstall +36 -8
- data/data/config/default_plugins.yaml +43 -0
- data/data/config/interface_modifiers.yaml +22 -7
- data/data/config/item_modifiers.yaml +14 -4
- data/data/config/plugins.yaml +1 -1
- data/data/config/widgets.yaml +1 -1
- data/lib/openc3/api/cmd_api.rb +10 -13
- data/lib/openc3/api/interface_api.rb +1 -2
- data/lib/openc3/api/limits_api.rb +149 -160
- data/lib/openc3/api/metrics_api.rb +26 -1
- data/lib/openc3/api/router_api.rb +1 -2
- data/lib/openc3/api/tlm_api.rb +225 -71
- data/lib/openc3/config/config_parser.rb +3 -1
- data/lib/openc3/core_ext/tempfile.rb +2 -0
- data/lib/openc3/interfaces/file_interface.rb +6 -1
- data/lib/openc3/interfaces/http_client_interface.rb +14 -6
- data/lib/openc3/interfaces/interface.rb +27 -3
- data/lib/openc3/interfaces/read_queue_interface.rb +331 -0
- data/lib/openc3/interfaces/stream_interface.rb +22 -3
- data/lib/openc3/interfaces/tcpip_server_interface.rb +13 -0
- data/lib/openc3/interfaces/udp_interface.rb +16 -2
- data/lib/openc3/interfaces.rb +1 -0
- data/lib/openc3/logs/log_writer.rb +14 -4
- data/lib/openc3/microservices/decom_microservice.rb +30 -5
- data/lib/openc3/microservices/interface_decom_common.rb +3 -1
- data/lib/openc3/microservices/interface_microservice.rb +146 -50
- data/lib/openc3/microservices/log_microservice.rb +1 -3
- data/lib/openc3/microservices/microservice.rb +25 -1
- data/lib/openc3/microservices/plugin_microservice.rb +32 -0
- data/lib/openc3/microservices/text_log_microservice.rb +1 -3
- data/lib/openc3/models/cvt_model.rb +7 -2
- data/lib/openc3/models/interface_model.rb +1 -1
- data/lib/openc3/models/interface_status_model.rb +6 -0
- data/lib/openc3/models/metric_model.rb +52 -4
- data/lib/openc3/models/microservice_model.rb +7 -3
- data/lib/openc3/models/plugin_model.rb +12 -12
- data/lib/openc3/models/python_package_model.rb +3 -13
- data/lib/openc3/models/target_model.rb +92 -15
- data/lib/openc3/operators/microservice_operator.rb +5 -0
- data/lib/openc3/operators/operator.rb +69 -0
- data/lib/openc3/packets/packet.rb +10 -4
- data/lib/openc3/packets/packet_config.rb +12 -1
- data/lib/openc3/packets/packet_item.rb +40 -22
- data/lib/openc3/packets/parsers/state_parser.rb +1 -1
- data/lib/openc3/packets/structure_item.rb +3 -2
- data/lib/openc3/script/api_shared.rb +62 -86
- data/lib/openc3/script/extract.rb +263 -20
- data/lib/openc3/script/plugins.rb +4 -11
- data/lib/openc3/script/script.rb +1 -1
- data/lib/openc3/script/storage.rb +81 -10
- data/lib/openc3/system/system.rb +9 -6
- data/lib/openc3/topics/command_decom_topic.rb +1 -1
- data/lib/openc3/topics/command_topic.rb +2 -1
- data/lib/openc3/topics/limits_event_topic.rb +90 -3
- data/lib/openc3/topics/router_topic.rb +7 -2
- data/lib/openc3/topics/system_events_topic.rb +9 -6
- data/lib/openc3/topics/telemetry_decom_topic.rb +1 -1
- data/lib/openc3/topics/telemetry_topic.rb +3 -2
- data/lib/openc3/topics/topic.rb +162 -3
- data/lib/openc3/utilities/bucket.rb +5 -0
- data/lib/openc3/utilities/cli_generator.rb +67 -15
- data/lib/openc3/utilities/config_overlay.rb +35 -17
- data/lib/openc3/utilities/default_plugins.rb +33 -0
- data/lib/openc3/utilities/local_mode.rb +82 -46
- data/lib/openc3/utilities/metric.rb +16 -0
- data/lib/openc3/utilities/process_stats.rb +160 -0
- data/lib/openc3/utilities/pypi_url.rb +65 -0
- data/lib/openc3/utilities/questdb_client.rb +140 -32
- data/lib/openc3/utilities/running_script.rb +22 -18
- data/lib/openc3/utilities/secrets.rb +5 -0
- data/lib/openc3/utilities/store_autoload.rb +23 -4
- data/lib/openc3/utilities/store_queued.rb +37 -9
- data/lib/openc3/utilities/target_file.rb +57 -12
- data/lib/openc3/utilities/topic_lag_monitor.rb +193 -0
- data/lib/openc3/version.rb +5 -5
- data/templates/conversion/conversion.py +6 -2
- data/templates/conversion/conversion.rb +5 -0
- data/templates/plugin/.github/workflows/playwright.yml +62 -0
- data/templates/plugin/.github/workflows/release.yml +45 -0
- data/templates/plugin/.github/workflows/unit-tests.yml +35 -0
- data/templates/plugin/.gitignore +6 -0
- data/templates/plugin/README.md +114 -2
- data/templates/plugin/lib/PACKAGE/__init__.py +4 -0
- data/templates/plugin/plugin.gemspec +10 -1
- data/templates/plugin/pyproject.toml +119 -0
- data/templates/tool_angular/package.json +2 -2
- data/templates/tool_angular/pnpm-workspace.yaml +16 -0
- data/templates/tool_react/package.json +1 -1
- data/templates/tool_react/pnpm-workspace.yaml +16 -0
- data/templates/tool_svelte/package.json +2 -1
- data/templates/tool_svelte/pnpm-workspace.yaml +20 -0
- data/templates/tool_vue/package.json +3 -3
- data/templates/tool_vue/pnpm-workspace.yaml +16 -0
- data/templates/widget/package.json +2 -2
- data/templates/widget/pnpm-workspace.yaml +16 -0
- metadata +16 -1
|
@@ -31,6 +31,15 @@ module OpenC3
|
|
|
31
31
|
name.sub(/\*$/, '')
|
|
32
32
|
end
|
|
33
33
|
|
|
34
|
+
# Reject names that could resolve outside the scope's targets_modified
|
|
35
|
+
# directory. The bucket normalizes '..' segments in keys just like the
|
|
36
|
+
# filesystem does, so this must be checked before any bucket call.
|
|
37
|
+
def self.validate_name(name)
|
|
38
|
+
unless OpenC3::LocalMode.safe_key?(name)
|
|
39
|
+
raise ArgumentError, "Invalid target file name: #{name.inspect}"
|
|
40
|
+
end
|
|
41
|
+
end
|
|
42
|
+
|
|
34
43
|
def self.all(scope, path_matchers, target: nil)
|
|
35
44
|
target = target.upcase if target
|
|
36
45
|
|
|
@@ -46,14 +55,26 @@ module OpenC3
|
|
|
46
55
|
prefix = "#{scope}/targets/"
|
|
47
56
|
modified_prefix = "#{scope}/targets_modified/"
|
|
48
57
|
end
|
|
49
|
-
|
|
50
|
-
|
|
58
|
+
if target and OpenC3::LocalMode.local_only_target?(target)
|
|
59
|
+
# Local only targets are never listed from the bucket
|
|
60
|
+
result = Set.new
|
|
61
|
+
modified = Set.new
|
|
62
|
+
temp = Set.new
|
|
63
|
+
else
|
|
64
|
+
result, _ = remote_target_files(bucket_client: bucket, prefix: prefix, include_temp: false, path_matchers: path_matchers)
|
|
65
|
+
modified, temp = remote_target_files(bucket_client: bucket, prefix: modified_prefix, include_temp: include_temp, path_matchers: path_matchers)
|
|
66
|
+
end
|
|
51
67
|
|
|
52
68
|
# Add in local targets_modified if present
|
|
53
|
-
if ENV['OPENC3_LOCAL_MODE']
|
|
69
|
+
if ENV['OPENC3_LOCAL_MODE'] or !OpenC3::LocalMode.local_only_targets.empty?
|
|
54
70
|
local_modified = OpenC3::LocalMode.local_target_files(scope: scope, target: target, path_matchers: path_matchers, include_temp: include_temp)
|
|
55
71
|
local_modified.each do |filename|
|
|
56
|
-
if
|
|
72
|
+
if OpenC3::LocalMode.local_only_name?(filename)
|
|
73
|
+
# The local file is the only copy so it isn't marked as modified
|
|
74
|
+
result << filename
|
|
75
|
+
elsif !ENV['OPENC3_LOCAL_MODE']
|
|
76
|
+
next
|
|
77
|
+
elsif include_temp and filename.include?(TEMP_FOLDER)
|
|
57
78
|
temp << filename
|
|
58
79
|
else
|
|
59
80
|
modified << filename unless modified.include?(filename)
|
|
@@ -100,16 +121,14 @@ module OpenC3
|
|
|
100
121
|
|
|
101
122
|
def self.body(scope, name)
|
|
102
123
|
name = strip_modified(name) # Remove '*' that indicates modified
|
|
124
|
+
# Local only targets are read from the local mode volume and never the bucket
|
|
125
|
+
if OpenC3::LocalMode.local_only_name?(name)
|
|
126
|
+
return local_body(scope, name)
|
|
127
|
+
end
|
|
103
128
|
# First try opening a potentially modified version by looking for the modified target
|
|
104
129
|
if ENV['OPENC3_LOCAL_MODE']
|
|
105
|
-
|
|
106
|
-
if
|
|
107
|
-
if File.extname(name) == ".bin"
|
|
108
|
-
return local_file.read
|
|
109
|
-
else
|
|
110
|
-
return local_file.read.force_encoding('UTF-8')
|
|
111
|
-
end
|
|
112
|
-
end
|
|
130
|
+
text = local_body(scope, name)
|
|
131
|
+
return text if text
|
|
113
132
|
end
|
|
114
133
|
|
|
115
134
|
bucket = Bucket.getClient()
|
|
@@ -136,6 +155,11 @@ module OpenC3
|
|
|
136
155
|
# file listing, not a name any caller means to write. Remove it to avoid
|
|
137
156
|
# creating a file with a '*' in the name.
|
|
138
157
|
name = strip_modified(name)
|
|
158
|
+
validate_name(name)
|
|
159
|
+
if OpenC3::LocalMode.local_only_name?(name)
|
|
160
|
+
OpenC3::LocalMode.put_target_file("#{scope}/targets_modified/#{name}", text, scope: scope)
|
|
161
|
+
return true
|
|
162
|
+
end
|
|
139
163
|
if ENV['OPENC3_LOCAL_MODE']
|
|
140
164
|
OpenC3::LocalMode.put_target_file("#{scope}/targets_modified/#{name}", text, scope: scope)
|
|
141
165
|
end
|
|
@@ -163,6 +187,11 @@ module OpenC3
|
|
|
163
187
|
# Match body/create so deleting a name taken straight from a listing
|
|
164
188
|
# removes the file the user actually picked
|
|
165
189
|
name = strip_modified(name)
|
|
190
|
+
validate_name(name)
|
|
191
|
+
if OpenC3::LocalMode.local_only_name?(name)
|
|
192
|
+
OpenC3::LocalMode.delete_local("#{scope}/targets_modified/#{name}")
|
|
193
|
+
return true
|
|
194
|
+
end
|
|
166
195
|
if ENV['OPENC3_LOCAL_MODE']
|
|
167
196
|
OpenC3::LocalMode.delete_local("#{scope}/targets_modified/#{name}")
|
|
168
197
|
end
|
|
@@ -177,6 +206,20 @@ module OpenC3
|
|
|
177
206
|
|
|
178
207
|
# protected
|
|
179
208
|
|
|
209
|
+
def self.local_body(scope, name)
|
|
210
|
+
local_file = OpenC3::LocalMode.open_local_file(name, scope: scope)
|
|
211
|
+
return nil unless local_file
|
|
212
|
+
begin
|
|
213
|
+
if File.extname(name) == ".bin"
|
|
214
|
+
local_file.read
|
|
215
|
+
else
|
|
216
|
+
local_file.read.force_encoding('UTF-8')
|
|
217
|
+
end
|
|
218
|
+
ensure
|
|
219
|
+
local_file.close
|
|
220
|
+
end
|
|
221
|
+
end
|
|
222
|
+
|
|
180
223
|
def self.remote_target_files(bucket_client:, prefix:, include_temp: false, path_matchers: nil)
|
|
181
224
|
result = Set.new
|
|
182
225
|
temp = Set.new
|
|
@@ -191,6 +234,8 @@ module OpenC3
|
|
|
191
234
|
temp << split_key[2..-1].join('/') if include_temp
|
|
192
235
|
next
|
|
193
236
|
end
|
|
237
|
+
# Local only targets are never read from the bucket
|
|
238
|
+
next if OpenC3::LocalMode.local_only_target?(split_key[2])
|
|
194
239
|
|
|
195
240
|
if path_matchers
|
|
196
241
|
found = false
|
|
@@ -0,0 +1,193 @@
|
|
|
1
|
+
# encoding: ascii-8bit
|
|
2
|
+
|
|
3
|
+
# Copyright 2026 OpenC3, Inc.
|
|
4
|
+
# All Rights Reserved.
|
|
5
|
+
#
|
|
6
|
+
# This program is distributed in the hope that it will be useful,
|
|
7
|
+
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
8
|
+
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
|
9
|
+
# See LICENSE.md for more details.
|
|
10
|
+
#
|
|
11
|
+
# This file may also be used under the terms of a commercial license
|
|
12
|
+
# if purchased from OpenC3, Inc.
|
|
13
|
+
|
|
14
|
+
require 'openc3/utilities/logger'
|
|
15
|
+
|
|
16
|
+
module OpenC3
|
|
17
|
+
# Tracks how far a microservice is behind the Redis streams it consumes.
|
|
18
|
+
#
|
|
19
|
+
# Every processed message updates the lag gauge metric with the maximum lag
|
|
20
|
+
# seen over the last metric update interval. When the lag crosses the warning
|
|
21
|
+
# or critical threshold a user notification is published, and a recovery
|
|
22
|
+
# notification is published once the microservice has caught back up (with
|
|
23
|
+
# hysteresis so a lag hovering around a threshold doesn't flap).
|
|
24
|
+
#
|
|
25
|
+
# Streams are trimmed by time once their data is logged, so a consumer can
|
|
26
|
+
# only skip trimmed data after lagging far past the critical threshold.
|
|
27
|
+
#
|
|
28
|
+
# Recovery assumes each monitor is fed by a single reader that reads every
|
|
29
|
+
# stream with pending entries on each XREAD (COUNT limits how many entries
|
|
30
|
+
# but not which streams). A backlogged topic then reports its high lag every
|
|
31
|
+
# read cycle, so fresh messages on other topics can't clear the alert early.
|
|
32
|
+
class TopicLagMonitor
|
|
33
|
+
OK = 0
|
|
34
|
+
WARN = 1
|
|
35
|
+
CRITICAL = 2
|
|
36
|
+
|
|
37
|
+
# Lag must drop below threshold * CLEAR_RATIO to count as recovering
|
|
38
|
+
CLEAR_RATIO = 0.5
|
|
39
|
+
# Matches Metric::UPDATE_INTERVAL so every lag spike is published
|
|
40
|
+
GAUGE_WINDOW_SECONDS = 5.0
|
|
41
|
+
|
|
42
|
+
attr_reader :state
|
|
43
|
+
|
|
44
|
+
# Lag in seconds that triggers the warning and critical notifications.
|
|
45
|
+
# A warning <= 0 disables notifications. Also used by Data Flows so its
|
|
46
|
+
# indicators match the notifications.
|
|
47
|
+
def self.thresholds
|
|
48
|
+
{
|
|
49
|
+
'warning' => env_float('OPENC3_LAG_WARN_SECONDS', 5.0),
|
|
50
|
+
'critical' => env_float('OPENC3_LAG_CRITICAL_SECONDS', 30.0),
|
|
51
|
+
}
|
|
52
|
+
end
|
|
53
|
+
|
|
54
|
+
def self.env_float(name, default)
|
|
55
|
+
value = ENV.fetch(name, '').strip
|
|
56
|
+
return default if value.empty?
|
|
57
|
+
Float(value)
|
|
58
|
+
rescue ArgumentError
|
|
59
|
+
default
|
|
60
|
+
end
|
|
61
|
+
|
|
62
|
+
def initialize(name:, logger:, metric: nil)
|
|
63
|
+
@name = name
|
|
64
|
+
@logger = logger
|
|
65
|
+
@metric = metric
|
|
66
|
+
thresholds = self.class.thresholds
|
|
67
|
+
@warn_seconds = thresholds['warning']
|
|
68
|
+
@critical_seconds = thresholds['critical']
|
|
69
|
+
@clear_seconds = self.class.env_float('OPENC3_LAG_CLEAR_SECONDS', 10.0)
|
|
70
|
+
@renotify_seconds = self.class.env_float('OPENC3_LAG_RENOTIFY_SECONDS', 300.0)
|
|
71
|
+
@state = OK
|
|
72
|
+
@max_lag = 0.0
|
|
73
|
+
@max_lag_topic = nil
|
|
74
|
+
@below_since = nil
|
|
75
|
+
@below_ok_since = nil
|
|
76
|
+
@last_notify = nil
|
|
77
|
+
@last_notify_level = OK
|
|
78
|
+
@notified = false
|
|
79
|
+
@gauges = {}
|
|
80
|
+
@mutex = Mutex.new
|
|
81
|
+
end
|
|
82
|
+
|
|
83
|
+
# Record that a message was read from a topic
|
|
84
|
+
#
|
|
85
|
+
# @param topic [String] Redis stream the message was read from
|
|
86
|
+
# @param msg_id [String] Redis stream id of the message
|
|
87
|
+
# @param metric_name [String] Gauge metric to update with the lag
|
|
88
|
+
# @param help [String] Help text for the gauge metric
|
|
89
|
+
# @return [Float] The lag in seconds
|
|
90
|
+
def record(topic, msg_id, metric_name: nil, help: nil, now: Time.now.to_f)
|
|
91
|
+
lag = now - (id_ms(msg_id) / 1000.0)
|
|
92
|
+
update_gauge(metric_name, help, lag, now) if @metric and metric_name
|
|
93
|
+
return lag if @warn_seconds <= 0 # Notifications disabled
|
|
94
|
+
|
|
95
|
+
# Fast path for the common case of keeping up: skip the mutex and state machine
|
|
96
|
+
return lag if @state == OK and lag < @warn_seconds
|
|
97
|
+
|
|
98
|
+
@mutex.synchronize { update_state(topic, lag, now) }
|
|
99
|
+
return lag
|
|
100
|
+
end
|
|
101
|
+
|
|
102
|
+
protected
|
|
103
|
+
|
|
104
|
+
# Topics read by one microservice can lag by very different amounts, so
|
|
105
|
+
# publish the maximum lag rather than whichever topic was read last. The
|
|
106
|
+
# previous window's maximum is held through the current window so a spike
|
|
107
|
+
# just before a window boundary is still published.
|
|
108
|
+
def update_gauge(metric_name, help, lag, now)
|
|
109
|
+
window = (now / GAUGE_WINDOW_SECONDS).to_i
|
|
110
|
+
gauge = @gauges[metric_name]
|
|
111
|
+
if gauge.nil?
|
|
112
|
+
gauge = @gauges[metric_name] = { window: window, max: lag, previous_max: 0.0 }
|
|
113
|
+
elsif window != gauge[:window]
|
|
114
|
+
gauge[:previous_max] = (window == gauge[:window] + 1) ? gauge[:max] : 0.0
|
|
115
|
+
gauge[:window] = window
|
|
116
|
+
gauge[:max] = lag
|
|
117
|
+
elsif lag > gauge[:max]
|
|
118
|
+
gauge[:max] = lag
|
|
119
|
+
else
|
|
120
|
+
return # Published value is unchanged
|
|
121
|
+
end
|
|
122
|
+
value = (gauge[:max] > gauge[:previous_max]) ? gauge[:max] : gauge[:previous_max]
|
|
123
|
+
@metric.set(name: metric_name, value: value, type: 'gauge', unit: 'seconds', help: help)
|
|
124
|
+
end
|
|
125
|
+
|
|
126
|
+
def update_state(topic, lag, now)
|
|
127
|
+
if lag < @warn_seconds * CLEAR_RATIO
|
|
128
|
+
@below_ok_since ||= now
|
|
129
|
+
else
|
|
130
|
+
@below_ok_since = nil
|
|
131
|
+
end
|
|
132
|
+
if lag > @max_lag
|
|
133
|
+
@max_lag = lag
|
|
134
|
+
@max_lag_topic = topic
|
|
135
|
+
end
|
|
136
|
+
level = if lag >= @critical_seconds
|
|
137
|
+
CRITICAL
|
|
138
|
+
elsif lag >= @warn_seconds
|
|
139
|
+
WARN
|
|
140
|
+
else
|
|
141
|
+
OK
|
|
142
|
+
end
|
|
143
|
+
|
|
144
|
+
if level > @state
|
|
145
|
+
@state = level
|
|
146
|
+
@below_since = nil
|
|
147
|
+
# Bursty load can make a microservice recover and fall behind over and
|
|
148
|
+
# over, so falling back to an already announced level soon after the
|
|
149
|
+
# last notification continues that episode rather than notifying again
|
|
150
|
+
unless @last_notify and level <= @last_notify_level and (now - @last_notify) < @renotify_seconds
|
|
151
|
+
notify_lag(topic, lag, now)
|
|
152
|
+
end
|
|
153
|
+
elsif @state != OK
|
|
154
|
+
threshold = (@state == CRITICAL) ? @critical_seconds : @warn_seconds
|
|
155
|
+
if lag < threshold * CLEAR_RATIO
|
|
156
|
+
@below_since ||= now
|
|
157
|
+
if (now - @below_since) >= @clear_seconds
|
|
158
|
+
@below_since = nil
|
|
159
|
+
if @state == CRITICAL and (@below_ok_since.nil? or (now - @below_ok_since) < @clear_seconds)
|
|
160
|
+
@state = WARN
|
|
161
|
+
@below_since = @below_ok_since
|
|
162
|
+
else
|
|
163
|
+
@state = OK
|
|
164
|
+
@logger.info("#{@name} has caught up processing (max lag #{@max_lag.round(1)}s on #{@max_lag_topic})", type: Logger::NOTIFICATION) if @notified
|
|
165
|
+
@notified = false
|
|
166
|
+
@max_lag = 0.0
|
|
167
|
+
@max_lag_topic = nil
|
|
168
|
+
end
|
|
169
|
+
end
|
|
170
|
+
else
|
|
171
|
+
@below_since = nil
|
|
172
|
+
notify_lag(topic, lag, now) if (now - @last_notify) >= @renotify_seconds
|
|
173
|
+
end
|
|
174
|
+
end
|
|
175
|
+
end
|
|
176
|
+
|
|
177
|
+
def notify_lag(topic, lag, now)
|
|
178
|
+
message = "#{@name} is falling behind processing: #{lag.round(1)}s behind on #{topic} (max #{@max_lag.round(1)}s on #{@max_lag_topic})"
|
|
179
|
+
if @state == CRITICAL
|
|
180
|
+
@logger.error(message, type: Logger::NOTIFICATION)
|
|
181
|
+
else
|
|
182
|
+
@logger.warn(message, type: Logger::NOTIFICATION)
|
|
183
|
+
end
|
|
184
|
+
@last_notify = now
|
|
185
|
+
@last_notify_level = @state
|
|
186
|
+
@notified = true
|
|
187
|
+
end
|
|
188
|
+
|
|
189
|
+
def id_ms(id)
|
|
190
|
+
id.to_s.split('-')[0].to_i
|
|
191
|
+
end
|
|
192
|
+
end
|
|
193
|
+
end
|
data/lib/openc3/version.rb
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
# encoding: ascii-8bit
|
|
2
2
|
|
|
3
|
-
OPENC3_VERSION = '7.
|
|
3
|
+
OPENC3_VERSION = '7.5.1'
|
|
4
4
|
module OpenC3
|
|
5
5
|
module Version
|
|
6
6
|
MAJOR = '7'
|
|
7
|
-
MINOR = '
|
|
7
|
+
MINOR = '5'
|
|
8
8
|
PATCH = '1'
|
|
9
9
|
OTHER = ''
|
|
10
|
-
BUILD = '
|
|
10
|
+
BUILD = 'e3c5693f00248aa4cb1833dad3aaeeaabb6fd581'
|
|
11
11
|
end
|
|
12
|
-
VERSION = '7.
|
|
13
|
-
GEM_VERSION = '7.
|
|
12
|
+
VERSION = '7.5.1'
|
|
13
|
+
GEM_VERSION = '7.5.1'
|
|
14
14
|
end
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
from openc3.conversions.conversion import Conversion
|
|
2
|
-
# Using
|
|
3
|
-
# from openc3.
|
|
2
|
+
# Using System.telemetry.packet() requires the following:
|
|
3
|
+
# from openc3.system.system import System
|
|
4
4
|
|
|
5
5
|
# Custom conversion class
|
|
6
6
|
# See https://docs.openc3.com/docs/configuration/conversions
|
|
@@ -20,4 +20,8 @@ class <%= conversion_class %>(Conversion):
|
|
|
20
20
|
# Used when conversion is applied to a regular (not DERIVED) item
|
|
21
21
|
# NOTE: You can also use packet.read("ITEM") to get additional values
|
|
22
22
|
# return value / 2 * packet.read("OTHER_ITEM")
|
|
23
|
+
#
|
|
24
|
+
# In a READ_CONVERSION, read the latest value from a different packet in the same target
|
|
25
|
+
# other = System.telemetry.packet(packet.target_name, "OTHER_PACKET")
|
|
26
|
+
# return value * other.read("OTHER_ITEM")
|
|
23
27
|
return value
|
|
@@ -20,6 +20,11 @@ module OpenC3
|
|
|
20
20
|
# Used when conversion is applied to a regular (not DERIVED) item
|
|
21
21
|
# NOTE: You can also use packet.read("ITEM") to get additional values
|
|
22
22
|
# return value / 2 * packet.read("OTHER_ITEM")
|
|
23
|
+
#
|
|
24
|
+
# In a READ_CONVERSION, read the latest value from a different packet in the same target
|
|
25
|
+
# other = System.telemetry.packet(packet.target_name, "OTHER_PACKET")
|
|
26
|
+
# return value * other.read("OTHER_ITEM")
|
|
27
|
+
return value
|
|
23
28
|
end
|
|
24
29
|
end
|
|
25
30
|
end
|
|
@@ -0,0 +1,62 @@
|
|
|
1
|
+
# Builds this plugin and installs it into a real COSMOS with Playwright.
|
|
2
|
+
#
|
|
3
|
+
# Runs the most recent COSMOS Core release in Docker, builds the plugin gem, and
|
|
4
|
+
# drives the Admin tool the way an operator would: Install From File, submit the
|
|
5
|
+
# variables dialog, and wait for the install to report Complete. That catches
|
|
6
|
+
# what unit tests can't - a plugin.txt that doesn't parse, a target that doesn't
|
|
7
|
+
# build, a microservice that won't start.
|
|
8
|
+
#
|
|
9
|
+
# Nothing here needs configuring as the plugin grows. Tool pages are discovered
|
|
10
|
+
# from the COSMOS API after install and checked for browser console errors, and
|
|
11
|
+
# a frontend is built with pnpm whenever the plugin has a package.json, so
|
|
12
|
+
# `cli generate tool` and `cli generate widget` are covered with no edit here.
|
|
13
|
+
#
|
|
14
|
+
# Runs on every commit and once a week. The weekly run is what catches a new
|
|
15
|
+
# COSMOS release breaking this plugin. Note that GitHub disables scheduled
|
|
16
|
+
# workflows in a repo with no activity for 60 days.
|
|
17
|
+
|
|
18
|
+
name: Playwright
|
|
19
|
+
|
|
20
|
+
on:
|
|
21
|
+
push:
|
|
22
|
+
branches: [main]
|
|
23
|
+
pull_request:
|
|
24
|
+
branches: ["**"]
|
|
25
|
+
# Off the hour because scheduled runs at :00 get queued behind everyone else's
|
|
26
|
+
schedule:
|
|
27
|
+
- cron: '27 7 * * 1'
|
|
28
|
+
workflow_dispatch:
|
|
29
|
+
inputs:
|
|
30
|
+
cosmos_version:
|
|
31
|
+
description: 'COSMOS version to run, or latest'
|
|
32
|
+
required: false
|
|
33
|
+
type: string
|
|
34
|
+
default: 'latest'
|
|
35
|
+
|
|
36
|
+
permissions:
|
|
37
|
+
contents: read
|
|
38
|
+
|
|
39
|
+
concurrency:
|
|
40
|
+
group: ${{ github.workflow }}-${{ github.ref }}
|
|
41
|
+
# Superseding a pull request run is fine, but a scheduled run and a push to
|
|
42
|
+
# main share a ref, so cancelling on those would have them kill each other
|
|
43
|
+
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
|
|
44
|
+
|
|
45
|
+
jobs:
|
|
46
|
+
playwright:
|
|
47
|
+
uses: OpenC3/.github/.github/workflows/plugin-playwright-reusable.yml@main
|
|
48
|
+
with:
|
|
49
|
+
plugin_directory: .
|
|
50
|
+
cosmos_version: ${{ inputs.cosmos_version || 'latest' }}
|
|
51
|
+
# Once you have added targets with `cli generate target`, list them here
|
|
52
|
+
# and the install is checked for them:
|
|
53
|
+
# expected_targets: MY_TARGET ANOTHER_TARGET
|
|
54
|
+
#
|
|
55
|
+
# install_demo: true # Uncomment if this plugin references INST/INST2
|
|
56
|
+
# is_tool: true # Uncomment to require that this plugin registers a tool
|
|
57
|
+
#
|
|
58
|
+
# Tool pages are checked for browser console errors automatically. Add
|
|
59
|
+
# regexes here for errors to allow:
|
|
60
|
+
# console_ignore: |
|
|
61
|
+
# some third party gripe
|
|
62
|
+
# another known message
|
|
@@ -0,0 +1,45 @@
|
|
|
1
|
+
# Builds the gem, tags the commit, and creates a GitHub release. Run it by hand
|
|
2
|
+
# from the Actions tab and give it a version number.
|
|
3
|
+
#
|
|
4
|
+
# Both publish steps are off by default so a first release works with no setup
|
|
5
|
+
# at all. Tick the box in the run dialog when you are ready to publish, after
|
|
6
|
+
# adding the matching secret to the repo:
|
|
7
|
+
#
|
|
8
|
+
# Deploy to OpenC3 App Store -> OPENC3_API_TOKEN (your store account token)
|
|
9
|
+
# Publish the gem to RubyGems -> RUBYGEMS_API_KEY (your RubyGems API key)
|
|
10
|
+
#
|
|
11
|
+
# A tool or widget frontend needs nothing set here: once the plugin has a
|
|
12
|
+
# package.json it is built with pnpm before the gem is packaged.
|
|
13
|
+
|
|
14
|
+
name: Release
|
|
15
|
+
|
|
16
|
+
on:
|
|
17
|
+
workflow_dispatch:
|
|
18
|
+
inputs:
|
|
19
|
+
version:
|
|
20
|
+
description: 'Version number'
|
|
21
|
+
required: true
|
|
22
|
+
type: string
|
|
23
|
+
deploy_to_store:
|
|
24
|
+
description: 'Deploy to OpenC3 App Store'
|
|
25
|
+
required: false
|
|
26
|
+
type: boolean
|
|
27
|
+
default: false
|
|
28
|
+
deploy_to_rubygems:
|
|
29
|
+
description: 'Publish the gem to RubyGems'
|
|
30
|
+
required: false
|
|
31
|
+
type: boolean
|
|
32
|
+
default: false
|
|
33
|
+
|
|
34
|
+
jobs:
|
|
35
|
+
release:
|
|
36
|
+
uses: OpenC3/.github/.github/workflows/release-cosmos-plugin-reusable.yml@main
|
|
37
|
+
with:
|
|
38
|
+
version: ${{ inputs.version }}
|
|
39
|
+
deploy_to_store: ${{ inputs.deploy_to_store }}
|
|
40
|
+
deploy_to_rubygems: ${{ inputs.deploy_to_rubygems }}
|
|
41
|
+
plugin_directory: .
|
|
42
|
+
# is_tool: true # Uncomment to fail the release if package.json is missing
|
|
43
|
+
secrets:
|
|
44
|
+
OPENC3_API_TOKEN: ${{ secrets.OPENC3_API_TOKEN }}
|
|
45
|
+
RUBYGEMS_API_KEY: ${{ secrets.RUBYGEMS_API_KEY }}
|
|
@@ -0,0 +1,35 @@
|
|
|
1
|
+
# Runs this plugin's unit tests on every push and pull request.
|
|
2
|
+
#
|
|
3
|
+
# Python tests are discovered in tests/ or test/, Ruby specs in spec/ or specs/.
|
|
4
|
+
# Whichever this plugin doesn't have is skipped, so there is nothing to set up:
|
|
5
|
+
# add a test file and it starts running. A plugin with no tests at all passes
|
|
6
|
+
# with a warning until you set require_tests below.
|
|
7
|
+
#
|
|
8
|
+
# Dependencies are picked up automatically - requirements.txt for the code under
|
|
9
|
+
# test, <test dir>/requirements.txt for test only ones, and a Gemfile for Ruby
|
|
10
|
+
# when there is one.
|
|
11
|
+
|
|
12
|
+
name: Unit Tests
|
|
13
|
+
|
|
14
|
+
on:
|
|
15
|
+
push:
|
|
16
|
+
branches: [main]
|
|
17
|
+
pull_request:
|
|
18
|
+
branches: ["**"]
|
|
19
|
+
|
|
20
|
+
permissions:
|
|
21
|
+
contents: read
|
|
22
|
+
|
|
23
|
+
concurrency:
|
|
24
|
+
group: ${{ github.workflow }}-${{ github.ref }}
|
|
25
|
+
cancel-in-progress: true
|
|
26
|
+
|
|
27
|
+
jobs:
|
|
28
|
+
unit-tests:
|
|
29
|
+
uses: OpenC3/.github/.github/workflows/plugin-unit-tests-reusable.yml@main
|
|
30
|
+
with:
|
|
31
|
+
plugin_directory: .
|
|
32
|
+
# python_version: '3.12' # Uncomment to pin a different Python
|
|
33
|
+
# ruby_version: '3.4' # Uncomment to pin a different Ruby
|
|
34
|
+
# require_tests: true # Uncomment to fail the build when there are no tests
|
|
35
|
+
# gemfile: Gemfile.dev # Uncomment if the specs need a different Gemfile than the plugin's own
|
data/templates/plugin/.gitignore
CHANGED
data/templates/plugin/README.md
CHANGED
|
@@ -8,6 +8,96 @@ Update this comment with your own description.
|
|
|
8
8
|
|
|
9
9
|
1. Edit the .gemspec file fields: name, summary, description, authors, email, and homepage
|
|
10
10
|
1. Update the LICENSE.md file with your company name
|
|
11
|
+
<% if @@language == 'py' -%>
|
|
12
|
+
1. Install [uv](https://docs.astral.sh/uv/getting-started/installation/) to manage the plugin's Python dependencies
|
|
13
|
+
|
|
14
|
+
## Python dependencies with uv
|
|
15
|
+
|
|
16
|
+
The plugin's Python dependencies are declared in `pyproject.toml` and locked in `uv.lock`.
|
|
17
|
+
Both files are packaged into the gem automatically, and COSMOS reads them when the plugin is installed.
|
|
18
|
+
|
|
19
|
+
### Adding dependencies
|
|
20
|
+
|
|
21
|
+
```sh
|
|
22
|
+
uv add requests # runtime dependency, installed into COSMOS with the plugin
|
|
23
|
+
uv add --group dev pytest-mock # development only, never installed into COSMOS
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
`uv add` updates both `pyproject.toml` and `uv.lock`. Commit both files.
|
|
27
|
+
|
|
28
|
+
`uv lock` resolves `pyproject.toml` and writes `uv.lock` without installing anything. `uv sync` does the
|
|
29
|
+
same when the lock is stale, then installs into `.venv`. Either works after editing `pyproject.toml` by hand;
|
|
30
|
+
commit the updated `uv.lock`.
|
|
31
|
+
|
|
32
|
+
COSMOS installs a locked plugin with `uv sync --frozen`, which installs exactly what `uv.lock` records and never
|
|
33
|
+
relocks, so a stale lock ships stale dependencies. `uv lock --check` fails if the lock is out of date.
|
|
34
|
+
|
|
35
|
+
### How COSMOS installs them
|
|
36
|
+
|
|
37
|
+
- **With `uv.lock`** (recommended): COSMOS runs `uv sync --frozen --no-dev`. Every install gets the same
|
|
38
|
+
versions, downloaded from the URLs recorded in the lock.
|
|
39
|
+
- **Without `uv.lock`**: COSMOS resolves the dependencies at install time against the PyPI URL set in
|
|
40
|
+
Admin Console -> Settings, so versions can change from one install to the next.
|
|
41
|
+
|
|
42
|
+
The `dev` dependency group is never installed into COSMOS.
|
|
43
|
+
|
|
44
|
+
### Developing locally
|
|
45
|
+
|
|
46
|
+
```sh
|
|
47
|
+
uv sync --group dev # create .venv with runtime and dev dependencies
|
|
48
|
+
uv run --group dev pytest # run tests; lib/ is on the import path
|
|
49
|
+
uv run --group dev ruff check # lint
|
|
50
|
+
uv run --group dev ruff format
|
|
51
|
+
uv run --group dev ty check # type check
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
A plain `uv sync` installs only the runtime dependencies, matching what COSMOS installs.
|
|
55
|
+
|
|
56
|
+
The plugin itself is not installed into `.venv`: COSMOS puts `lib/` on the Python path instead, and pytest and ty
|
|
57
|
+
are configured to do the same. Anything else needs `lib/` on the path to import `<%= package_name %>`:
|
|
58
|
+
|
|
59
|
+
```
|
|
60
|
+
PYTHONPATH=lib uv run python my_script.py
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
`cli generate` writes a `.python-version` file pinning the Python version of the COSMOS image it ran in, so
|
|
64
|
+
uv develops against the same Python COSMOS runs. If you upgrade to a COSMOS release with a newer Python, update
|
|
65
|
+
it with `uv python pin <version>`. Without that file uv picks any installed Python that satisfies
|
|
66
|
+
`requires-python`.
|
|
67
|
+
|
|
68
|
+
The `openc3` dev dependency provides the COSMOS Python API for your editor, tests and `ty`. Pin it to the
|
|
69
|
+
COSMOS version you deploy to so they see the same API, e.g. `uv add --group dev "openc3==X.Y.Z"`.
|
|
70
|
+
|
|
71
|
+
### Where to put Python code
|
|
72
|
+
|
|
73
|
+
Put Python code shared across the plugin in `lib/<%= package_name %>/`, never directly in `lib/`.
|
|
74
|
+
|
|
75
|
+
COSMOS adds the `lib/` directory of every installed plugin to the Python path. If two plugins both ship
|
|
76
|
+
`lib/helpers.py`, `import helpers` loads whichever one Python finds first, and the other can't be imported
|
|
77
|
+
at all. A module directly in `lib/` can also hide an installed package of the same name, such as `requests`.
|
|
78
|
+
The `<%= package_name %>` package is named after this plugin, so its modules can't collide.
|
|
79
|
+
|
|
80
|
+
Import from the package:
|
|
81
|
+
|
|
82
|
+
```python
|
|
83
|
+
from <%= package_name %>.helpers import Helper
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
Reference its files by path in plugin configuration, e.g. `INTERFACE MY_INT <%= package_name %>/my_interface.py`.
|
|
87
|
+
|
|
88
|
+
### Updating dependencies
|
|
89
|
+
|
|
90
|
+
```
|
|
91
|
+
uv lock --upgrade-package requests # upgrade one package
|
|
92
|
+
uv lock --upgrade # upgrade everything
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
### Private package indexes
|
|
96
|
+
|
|
97
|
+
See the comments in the `[tool.uv]` section of `pyproject.toml`. If the COSMOS PyPI URL setting points at
|
|
98
|
+
anything other than public PyPI, COSMOS ignores the plugin's own index settings when it resolves, so a
|
|
99
|
+
package published only to your private index must also be served by that index, or be locked in `uv.lock`.
|
|
100
|
+
<% end -%>
|
|
11
101
|
|
|
12
102
|
## Building non-tool / widget plugins
|
|
13
103
|
|
|
@@ -26,19 +116,41 @@ If you don’t have a local node environment, you can use our openc3-node contai
|
|
|
26
116
|
|
|
27
117
|
Mac / Linux:
|
|
28
118
|
|
|
29
|
-
```
|
|
119
|
+
```sh
|
|
30
120
|
docker run -it -v `pwd`:/openc3/local:z -w /openc3/local docker.io/openc3inc/openc3-node sh
|
|
31
121
|
```
|
|
32
122
|
|
|
33
123
|
Windows:
|
|
34
124
|
|
|
35
|
-
```
|
|
125
|
+
```sh
|
|
36
126
|
docker run -it -v %cd%:/openc3/local -w /openc3/local docker.io/openc3inc/openc3-node sh
|
|
37
127
|
```
|
|
38
128
|
|
|
39
129
|
1. pnpm install --frozen-lockfile --ignore-scripts
|
|
40
130
|
1. rake build VERSION=1.0.0
|
|
41
131
|
|
|
132
|
+
## Continuous integration
|
|
133
|
+
|
|
134
|
+
`.github/workflows/` ships three GitHub Actions workflows that work as soon as
|
|
135
|
+
you push this plugin to GitHub. They call shared workflows from `OpenC3/.github`,
|
|
136
|
+
so the steps stay current without you updating anything here.
|
|
137
|
+
|
|
138
|
+
- **Unit Tests** - on every push and pull request. Runs Python tests in `tests/`
|
|
139
|
+
or `test/` and Ruby specs in `spec/` or `specs/`. Whichever you don't have is
|
|
140
|
+
skipped, so just add a test file.
|
|
141
|
+
- **Playwright** - on every push and pull request, plus weekly. Builds the gem,
|
|
142
|
+
starts the latest COSMOS release in Docker, and installs the plugin through the
|
|
143
|
+
Admin tool, which catches a `plugin.txt` that doesn't parse, a target that
|
|
144
|
+
doesn't build, or a microservice that won't start. The weekly run is what tells
|
|
145
|
+
you a new COSMOS release broke this plugin. Once you add targets, list them in
|
|
146
|
+
`expected_targets` so the install is verified against them.
|
|
147
|
+
- **Release** - manual, from the Actions tab. Builds the gem, tags the commit and
|
|
148
|
+
creates a GitHub release. Publishing to the OpenC3 App Store and to RubyGems is
|
|
149
|
+
off until you add the matching secret and tick the box.
|
|
150
|
+
|
|
151
|
+
Adding a tool or a widget needs no change to any of these. All three build a
|
|
152
|
+
frontend with pnpm as soon as the plugin has a `package.json`.
|
|
153
|
+
|
|
42
154
|
## Installing into OpenC3 COSMOS
|
|
43
155
|
|
|
44
156
|
1. Go to the OpenC3 Admin Tool, Plugins Tab
|
|
@@ -0,0 +1,4 @@
|
|
|
1
|
+
# Python code shared across this plugin lives in this package. COSMOS puts
|
|
2
|
+
# every installed plugin's lib/ on sys.path, so a module placed directly in
|
|
3
|
+
# lib/ can collide with another plugin's module of the same name. Keeping it
|
|
4
|
+
# under this uniquely named package avoids that. See README.md.
|