smart_proxy_openbolt 0.1.1 → 1.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +297 -11
- data/lib/smart_proxy_openbolt/api.rb +20 -41
- data/lib/smart_proxy_openbolt/config/choria-client.conf +25 -0
- data/lib/smart_proxy_openbolt/error.rb +17 -27
- data/lib/smart_proxy_openbolt/executor.rb +32 -26
- data/lib/smart_proxy_openbolt/job.rb +17 -81
- data/lib/smart_proxy_openbolt/lru_cache.rb +43 -0
- data/lib/smart_proxy_openbolt/main.rb +293 -106
- data/lib/smart_proxy_openbolt/plugin.rb +7 -9
- data/lib/smart_proxy_openbolt/result.rb +20 -23
- data/lib/smart_proxy_openbolt/task_job.rb +36 -41
- data/lib/smart_proxy_openbolt/version.rb +1 -1
- data/settings.d/openbolt.yml +1 -0
- metadata +6 -3
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: b6a0f4f6e883f4a2077ea3afccd8f244a1dfd27aafa73c6ce69f29ee423f921b
|
|
4
|
+
data.tar.gz: f74f56b1108bc2d0570af5135c225194d046e40e5741541265eff68c4bc61359
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: cfcc15bcfa344fa5a92b323ec6c9e9a11a0408ac49f111e1bf45e1eafdb228c5a557909d489cadde2594b54e8fd063de944dfff352bca9fca2950b4ac839477f
|
|
7
|
+
data.tar.gz: 427492ec2136bf63d811c50edd8fa24ab2642e0b54180b97bc02f4c396e9188fa8963d35d845bdc59f06bd0a591b7bdbf2e3317dd84d166ce2522b4a199e774d
|
data/README.md
CHANGED
|
@@ -1,22 +1,308 @@
|
|
|
1
1
|
# Smart Proxy - OpenBolt
|
|
2
2
|
|
|
3
3
|
[](https://github.com/overlookinfra/smart_proxy_openbolt/blob/master/LICENSE)
|
|
4
|
-
[](https://github.com/overlookinfra/smart_proxy_openbolt/actions/workflows/ci.yml)
|
|
5
5
|
[](https://github.com/overlookinfra/smart_proxy_openbolt/actions/workflows/release.yml)
|
|
6
6
|
[](https://rubygems.org/gems/smart_proxy_openbolt)
|
|
7
7
|
[](https://rubygems.org/gems/smart_proxy_openbolt)
|
|
8
8
|
|
|
9
|
-
This
|
|
9
|
+
This plugin adds [OpenBolt](https://github.com/OpenVoxProject/openbolt) support to [Foreman's Smart Proxy](https://github.com/theforeman/smart-proxy).
|
|
10
|
+
It exposes an HTTP API that the [foreman_openbolt](https://github.com/overlookinfra/foreman_openbolt) plugin uses to run Tasks and Plans on remote targets via the OpenBolt CLI.
|
|
11
|
+
|
|
12
|
+
## Introduction
|
|
13
|
+
|
|
14
|
+
[OpenBolt](https://github.com/OpenVoxProject/openbolt) is the open source successor of [Bolt](https://github.com/puppetlabs/bolt) by [Perforce](https://www.perforce.com/).
|
|
15
|
+
It runs Tasks and Plans against remote targets over SSH, WinRM, Choria, or other transports.
|
|
16
|
+
|
|
17
|
+
This smart proxy plugin wraps the OpenBolt CLI and provides:
|
|
18
|
+
|
|
19
|
+
* A REST API for listing available tasks, launching task runs, and retrieving results
|
|
20
|
+
* Concurrent job execution via a configurable thread pool
|
|
21
|
+
* Disk-based result storage so results survive proxy restarts
|
|
22
|
+
* Transport option forwarding (SSH, WinRM, Choria) from the Foreman UI
|
|
23
|
+
|
|
24
|
+
The Foreman UI talks to this plugin; it does not invoke OpenBolt directly. See the [foreman_openbolt README](https://github.com/overlookinfra/foreman_openbolt) for screenshots and the full user-facing workflow.
|
|
25
|
+
|
|
26
|
+
## Installation
|
|
27
|
+
|
|
28
|
+
See [How to Install a Plugin](https://theforeman.org/plugins/#2.Installation) for general Foreman plugin installation instructions.
|
|
29
|
+
The [theforeman/foreman_proxy](https://github.com/theforeman/puppet-foreman_proxy/blob/master/manifests/plugin/openbolt.pp) Puppet module supports automated installation.
|
|
30
|
+
|
|
31
|
+
You need `bolt` in your `$PATH` on the Smart Proxy host.
|
|
32
|
+
OpenBolt packages are available at [yum.voxpupuli.org](https://yum.voxpupuli.org/) and [apt.voxpupuli.org](https://apt.voxpupuli.org/) in the openvox8 repo.
|
|
33
|
+
You can also use the legacy Bolt packages from Perforce from the `puppet-tools` repo on [apt.puppet.com](https://apt.puppet.com/) or [yum.puppet.com](https://yum.puppet.com/).
|
|
34
|
+
|
|
35
|
+
OpenBolt relies on Tasks and Plans distributed as Puppet modules.
|
|
36
|
+
Deploy your code with [r10k](https://github.com/puppetlabs/r10k) or [g10k](https://github.com/xorpaul/g10k), as you do on your compilers.
|
|
37
|
+
A handful of core Tasks and Plans are also included in the [OpenBolt packages](https://github.com/OpenVoxProject/openbolt/blob/main/Puppetfile).
|
|
38
|
+
|
|
39
|
+
The integration is supported on Foreman 3.17 and all following versions, including development/nightly builds.
|
|
40
|
+
|
|
41
|
+
## Things to be aware of
|
|
10
42
|
|
|
11
|
-
# Things to be aware of
|
|
12
43
|
* Any SSH keys to be used should be readable by the foreman-proxy user.
|
|
13
|
-
* Results are
|
|
44
|
+
* Results are stored on disk at `/var/log/foreman-proxy/openbolt` by default (configurable via `log_dir`). Fetching old results is possible as long as the files remain on disk.
|
|
45
|
+
|
|
46
|
+
## Configuration
|
|
47
|
+
|
|
48
|
+
The plugin is configured in `settings.d/openbolt.yml` on the Smart Proxy host. All settings have sensible defaults:
|
|
49
|
+
|
|
50
|
+
```yaml
|
|
51
|
+
---
|
|
52
|
+
:enabled: https
|
|
53
|
+
:environment_path: /etc/puppetlabs/code/environments/production
|
|
54
|
+
:workers: 20
|
|
55
|
+
:concurrency: 100
|
|
56
|
+
:connect_timeout: 30
|
|
57
|
+
:log_dir: /var/log/foreman-proxy/openbolt
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
| Setting | Default | Description |
|
|
61
|
+
|---------|---------|-------------|
|
|
62
|
+
| `enabled` | `https` | Enable the plugin (`https`, `http`, or `false`) |
|
|
63
|
+
| `environment_path` | `/etc/puppetlabs/code/environments/production` | Path to the Puppet environment containing Tasks and Plans |
|
|
64
|
+
| `workers` | `20` | Number of threads in the job executor pool |
|
|
65
|
+
| `concurrency` | `100` | Maximum number of concurrent target connections per job |
|
|
66
|
+
| `connect_timeout` | `30` | Connection timeout in seconds for target connections |
|
|
67
|
+
| `log_dir` | `/var/log/foreman-proxy/openbolt` | Directory for job result files (created automatically if missing) |
|
|
68
|
+
|
|
69
|
+
After installing the plugin and restarting the smart proxy, refresh the
|
|
70
|
+
proxy features in Foreman so it detects the OpenBolt capability:
|
|
71
|
+
|
|
72
|
+
```bash
|
|
73
|
+
foreman-rake openbolt:refresh_proxies
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
## Choria Transport
|
|
77
|
+
|
|
78
|
+
**Requires OpenBolt 5.5 or later.** The Choria transport is not available
|
|
79
|
+
in Puppet Bolt. SSH and WinRM transports work with any version.
|
|
80
|
+
|
|
81
|
+
The Choria transport works out of the box on HTTPS-enabled proxies. The
|
|
82
|
+
proxy ships a built-in MCollective client configuration and automatically
|
|
83
|
+
derives SSL paths and the MCollective certname from the proxy's own
|
|
84
|
+
OpenVox/Puppet certificates (configured via `ssl_certificate`,
|
|
85
|
+
`ssl_private_key`, and `ssl_ca_file` in `settings.yml`).
|
|
86
|
+
|
|
87
|
+
The `foreman-proxy` user must be in the `puppet` group to read the SSL
|
|
88
|
+
files. This is the default when the proxy is set up with
|
|
89
|
+
`foreman-installer` (the `puppet-foreman_proxy` module's
|
|
90
|
+
`manage_puppet_group` parameter handles this).
|
|
91
|
+
|
|
92
|
+
In common configurations where the Choria broker and Foreman are on the
|
|
93
|
+
same host, OpenVox/Puppet certificates are being used for authentication,
|
|
94
|
+
and Choria configuration is largely using default settings, the Choria
|
|
95
|
+
transport mostly just works out of the box with default settings. The one
|
|
96
|
+
setting you may need to evaluate is **Choria Brokers** (in the Foreman UI
|
|
97
|
+
under Administer > Settings > OpenBolt). If `puppet` resolves to the
|
|
98
|
+
broker host or SRV records are configured, it can be left blank. Otherwise,
|
|
99
|
+
set it to your broker's address (e.g. `primary.example.com:4222`). Other
|
|
100
|
+
Choria settings (SSL, a default config file we provide, certname) are
|
|
101
|
+
derived automatically and can be ignored unless your Choria configuration
|
|
102
|
+
requires customization.
|
|
103
|
+
|
|
104
|
+
### Custom Choria configuration file
|
|
105
|
+
|
|
106
|
+
The built-in default config is at
|
|
107
|
+
`lib/smart_proxy_openbolt/config/choria-client.conf` in this gem. You
|
|
108
|
+
can view it to see what settings are included. To use a custom
|
|
109
|
+
MCollective client configuration file instead, set the "Choria Config
|
|
110
|
+
File" setting under Administer > Settings > OpenBolt. When a custom
|
|
111
|
+
config file is provided, the proxy does not inject SSL defaults (the
|
|
112
|
+
config file is expected to handle SSL on its own). SSL settings from
|
|
113
|
+
the Foreman UI still override the config file if set.
|
|
114
|
+
|
|
115
|
+
If your custom config file includes its own SSL paths, you should also
|
|
116
|
+
set **Choria MCollective Certname** to the CN of the certificate in your
|
|
117
|
+
config file. Without it, the MCollective client defaults to
|
|
118
|
+
`<user>.mcollective` (e.g. `foreman-proxy.mcollective`), which will fail
|
|
119
|
+
authentication if the certificate's CN doesn't match that pattern.
|
|
120
|
+
|
|
121
|
+
### How it works
|
|
122
|
+
|
|
123
|
+
When the transport is Choria and no custom config file is provided:
|
|
124
|
+
|
|
125
|
+
1. The proxy uses its built-in `choria-client.conf` (MCollective boilerplate)
|
|
126
|
+
2. SSL cert, key, and CA default from the proxy's `settings.yml` SSL paths
|
|
127
|
+
3. The MCollective certname is read from the certificate's CN
|
|
128
|
+
4. All values are passed as OpenBolt CLI flags (`--choria-ssl-cert`,
|
|
129
|
+
`--choria-ssl-key`, `--choria-ssl-ca`, `--choria-mcollective-certname`,
|
|
130
|
+
`--choria-config-file`)
|
|
131
|
+
|
|
132
|
+
For full Choria infrastructure setup (broker, managed nodes, Puppet modules),
|
|
133
|
+
see the [Choria Testing](https://github.com/overlookinfra/foreman_openbolt/blob/main/docs/choria-testing.md) guide in the foreman_openbolt repo.
|
|
134
|
+
|
|
135
|
+
## API
|
|
136
|
+
|
|
137
|
+
The plugin mounts its API at `/openbolt` on the Smart Proxy. All endpoints are used by the Foreman plugin and are not intended for direct use, but can be useful for debugging.
|
|
138
|
+
|
|
139
|
+
| Method | Endpoint | Description |
|
|
140
|
+
|--------|----------|-------------|
|
|
141
|
+
| `GET` | `/openbolt/tasks` | List available tasks from the configured environment |
|
|
142
|
+
| `GET` | `/openbolt/tasks/reload` | Clear the task cache and reload from disk |
|
|
143
|
+
| `GET` | `/openbolt/tasks/options` | Get available transport options (SSH, WinRM, Choria) |
|
|
144
|
+
| `POST` | `/openbolt/launch/task` | Launch a task run against specified targets |
|
|
145
|
+
| `GET` | `/openbolt/job/:id/status` | Get the status of a running or completed job |
|
|
146
|
+
| `GET` | `/openbolt/job/:id/result` | Get the full result of a completed job |
|
|
147
|
+
| `DELETE` | `/openbolt/job/:id/artifacts` | Delete stored result files for a job |
|
|
148
|
+
|
|
149
|
+
## Authentication
|
|
150
|
+
|
|
151
|
+
The OpenBolt API is HTTPS-only and requires SSL client certificate authentication.
|
|
152
|
+
The smart proxy verifies that:
|
|
153
|
+
|
|
154
|
+
1. The client presents a valid SSL certificate signed by the same CA configured
|
|
155
|
+
in `ssl_ca_file` in `/etc/foreman-proxy/settings.yml`.
|
|
156
|
+
2. The certificate's CN appears in the `trusted_hosts` list in that same file.
|
|
157
|
+
|
|
158
|
+
In a standard Foreman installation, the Foreman server's certificate is already
|
|
159
|
+
trusted by the smart proxy, so no additional configuration is needed for
|
|
160
|
+
Foreman-to-proxy communication.
|
|
161
|
+
|
|
162
|
+
### Direct API access
|
|
163
|
+
|
|
164
|
+
If you want to hit the smart proxy OpenBolt API directly from the Foreman host
|
|
165
|
+
(for debugging, scripting, etc.), you can reuse Foreman's own client
|
|
166
|
+
certificates since they are already trusted by the proxy:
|
|
167
|
+
|
|
168
|
+
```bash
|
|
169
|
+
curl -s \
|
|
170
|
+
--cacert /etc/puppetlabs/puppet/ssl/certs/ca.pem \
|
|
171
|
+
--cert /etc/puppetlabs/puppet/ssl/certs/$(hostname -f).pem \
|
|
172
|
+
--key /etc/puppetlabs/puppet/ssl/private_keys/$(hostname -f).pem \
|
|
173
|
+
https://$(hostname -f):8443/openbolt/tasks
|
|
174
|
+
```
|
|
175
|
+
|
|
176
|
+
The Foreman server's CN should already be in `trusted_hosts` in the smart
|
|
177
|
+
proxy's `/etc/foreman-proxy/settings.yml`. If not, add it:
|
|
178
|
+
|
|
179
|
+
```yaml
|
|
180
|
+
:trusted_hosts:
|
|
181
|
+
- foreman.example.com
|
|
182
|
+
```
|
|
183
|
+
|
|
184
|
+
## Development
|
|
185
|
+
|
|
186
|
+
### Unit Tests
|
|
187
|
+
|
|
188
|
+
```bash
|
|
189
|
+
bundle exec rake test
|
|
190
|
+
```
|
|
191
|
+
|
|
192
|
+
### Acceptance Tests (SSH)
|
|
193
|
+
|
|
194
|
+
Acceptance tests run against Docker containers with a proxy and SSH targets.
|
|
195
|
+
|
|
196
|
+
```bash
|
|
197
|
+
bundle exec rake acceptance:ssh:up # Start containers
|
|
198
|
+
bundle exec rake acceptance:ssh # Run tests
|
|
199
|
+
bundle exec rake acceptance:ssh:down # Stop and clean up
|
|
200
|
+
```
|
|
201
|
+
|
|
202
|
+
### Building Packages
|
|
203
|
+
|
|
204
|
+
Build RPM or DEB packages locally using containers. The [foreman-packaging](https://github.com/theforeman/foreman-packaging) repo is cloned automatically:
|
|
205
|
+
|
|
206
|
+
```bash
|
|
207
|
+
bundle exec rake build:rpm # Build RPM
|
|
208
|
+
bundle exec rake build:deb # Build DEB
|
|
209
|
+
```
|
|
210
|
+
|
|
211
|
+
### Environment Variables
|
|
212
|
+
|
|
213
|
+
| Variable | Default | Description |
|
|
214
|
+
|----------|---------|-------------|
|
|
215
|
+
| `FOREMAN_PACKAGING_REPO` | `https://github.com/theforeman/foreman-packaging.git` | Git URL for foreman-packaging |
|
|
216
|
+
| `FOREMAN_VERSION` | `3.18` | Foreman version for package builds |
|
|
217
|
+
|
|
218
|
+
## Contributing and support
|
|
219
|
+
|
|
220
|
+
Fork and send a Pull Request. Thanks!
|
|
221
|
+
If you have questions or need professional support, please join the `#sig-orchestrator` channel on the [Vox Pupuli Slack](https://voxpupuli.org/connect/).
|
|
222
|
+
|
|
223
|
+
## Copyright
|
|
224
|
+
|
|
225
|
+
Copyright (c) 2025 Overlook InfraTech
|
|
226
|
+
|
|
227
|
+
Copyright (c) 2025 betadots GmbH
|
|
228
|
+
|
|
229
|
+
This program is free software: you can redistribute it and/or modify
|
|
230
|
+
it under the terms of the GNU General Public License as published by
|
|
231
|
+
the Free Software Foundation, either version 3 of the License, or
|
|
232
|
+
(at your option) any later version.
|
|
233
|
+
|
|
234
|
+
This program is distributed in the hope that it will be useful,
|
|
235
|
+
but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
236
|
+
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
237
|
+
GNU General Public License for more details.
|
|
238
|
+
|
|
239
|
+
You should have received a copy of the GNU General Public License
|
|
240
|
+
along with this program. If not, see <http://www.gnu.org/licenses/>.
|
|
241
|
+
|
|
242
|
+
## How to Release
|
|
243
|
+
|
|
244
|
+
### Release steps
|
|
245
|
+
|
|
246
|
+
1. Go to [Actions > Prepare Release](../../actions/workflows/prepare_release.yml) and run the workflow with the version to release (e.g. `1.2.0`)
|
|
247
|
+
2. The workflow bumps the version in `version.rb`, generates the changelog, and opens a PR with the `skip-changelog` label
|
|
248
|
+
3. Review and merge the PR
|
|
249
|
+
4. Go to [Actions > Release](../../actions/workflows/release.yml) and run the workflow with the same version
|
|
250
|
+
5. The release workflow:
|
|
251
|
+
- Verifies the version in `version.rb` matches the input
|
|
252
|
+
- Creates and pushes a git tag
|
|
253
|
+
- Builds the gem
|
|
254
|
+
- Creates a GitHub Release with auto-generated notes and the gem attached
|
|
255
|
+
- Publishes the gem to GitHub Packages
|
|
256
|
+
- Publishes the gem to RubyGems.org (requires the `release` environment)
|
|
257
|
+
- Verifies the gem is available on RubyGems.org
|
|
258
|
+
|
|
259
|
+
### RPM/DEB packaging
|
|
260
|
+
|
|
261
|
+
After the gem is published to RubyGems, both RPM and DEB packages need to be updated in [theforeman/foreman-packaging](https://github.com/theforeman/foreman-packaging).
|
|
262
|
+
|
|
263
|
+
A bot automatically creates PRs against the `rpm/develop` and `deb/develop` branches to pick up the new gem version. These PRs build packages for Foreman nightly.
|
|
264
|
+
|
|
265
|
+
For stable Foreman releases (currently 3.17 and 3.18), cherry-pick the packaging commits from the develop branches into the corresponding stable branches. For each stable version you want to support:
|
|
266
|
+
|
|
267
|
+
```bash
|
|
268
|
+
cd foreman-packaging
|
|
269
|
+
|
|
270
|
+
# RPM: cherry-pick from rpm/develop into a branch off the stable target
|
|
271
|
+
git checkout rpm/3.18
|
|
272
|
+
git checkout -b cherry-pick/rubygem-smart_proxy_openbolt-rpm-3.18
|
|
273
|
+
git cherry-pick <commit-from-rpm/develop>
|
|
274
|
+
# Push to your fork and open a PR targeting rpm/3.18
|
|
275
|
+
|
|
276
|
+
# DEB: same approach for the deb side
|
|
277
|
+
git checkout deb/3.18
|
|
278
|
+
git checkout -b cherry-pick/rubygem-smart_proxy_openbolt-deb-3.18
|
|
279
|
+
git cherry-pick <commit-from-deb/develop>
|
|
280
|
+
# Push to your fork and open a PR targeting deb/3.18
|
|
281
|
+
```
|
|
282
|
+
|
|
283
|
+
PRs against stable branches should be labeled "Stable branch".
|
|
284
|
+
|
|
285
|
+
**Alternative: manual version bump**
|
|
286
|
+
|
|
287
|
+
If the cherry-pick doesn't apply cleanly, you can bump the version manually on the stable branch instead.
|
|
288
|
+
|
|
289
|
+
*RPM:* Checkout the target branch and run `bump_rpm.sh`:
|
|
290
|
+
```bash
|
|
291
|
+
cd foreman-packaging
|
|
292
|
+
git checkout rpm/3.18
|
|
293
|
+
git checkout -b bump_rpm/rubygem-smart_proxy_openbolt
|
|
294
|
+
./bump_rpm.sh packages/plugins/rubygem-smart_proxy_openbolt
|
|
295
|
+
# Review changes, push to your fork, and open a PR targeting rpm/3.18
|
|
296
|
+
```
|
|
14
297
|
|
|
15
|
-
|
|
298
|
+
*DEB:* Checkout the target branch and update these files:
|
|
299
|
+
- `debian/gem.list` -- new gem filename
|
|
300
|
+
- `smart_proxy_openbolt.rb` -- new version
|
|
301
|
+
- `debian/control` -- dependency versions (if changed)
|
|
302
|
+
- `debian/changelog` -- add a new entry
|
|
16
303
|
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
* github actions will publish the tag
|
|
304
|
+
```bash
|
|
305
|
+
git checkout deb/3.18
|
|
306
|
+
git checkout -b bump_deb/ruby-smart-proxy-openbolt
|
|
307
|
+
# Make the changes above, push to your fork, and open a PR targeting deb/3.18
|
|
308
|
+
```
|
|
@@ -5,23 +5,23 @@ require 'smart_proxy_openbolt/main'
|
|
|
5
5
|
require 'smart_proxy_openbolt/error'
|
|
6
6
|
|
|
7
7
|
module Proxy::OpenBolt
|
|
8
|
-
|
|
9
8
|
class Api < ::Sinatra::Base
|
|
10
9
|
include ::Proxy::Log
|
|
11
|
-
helpers ::Proxy::Helpers
|
|
12
10
|
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
# the best way to open them to foreman-proxy if we want to use this, I think.
|
|
17
|
-
#authorize_with_trusted_hosts
|
|
18
|
-
#authorize_with_ssl_client
|
|
11
|
+
helpers ::Proxy::Helpers
|
|
12
|
+
authorize_with_trusted_hosts
|
|
13
|
+
authorize_with_ssl_client
|
|
19
14
|
|
|
20
15
|
# Call reload_tasks at class load so the first call to /tasks
|
|
21
16
|
# is potentially faster (if called after this finishes). Do it
|
|
22
17
|
# async so we don't block. The reload_tasks function uses a mutex
|
|
23
18
|
# so it will be safe to call /tasks before it completes.
|
|
24
|
-
Thread.new
|
|
19
|
+
Thread.new do
|
|
20
|
+
Proxy::OpenBolt.tasks
|
|
21
|
+
rescue StandardError => e
|
|
22
|
+
Proxy::OpenBolt.logger.error("Task prefetch failed (#{e.class}): #{e.message}")
|
|
23
|
+
Proxy::OpenBolt.logger.debug(e.backtrace.join("\n")) if e.backtrace
|
|
24
|
+
end
|
|
25
25
|
|
|
26
26
|
get '/tasks' do
|
|
27
27
|
catch_errors { Proxy::OpenBolt.tasks.to_json }
|
|
@@ -32,12 +32,16 @@ module Proxy::OpenBolt
|
|
|
32
32
|
end
|
|
33
33
|
|
|
34
34
|
get '/tasks/options' do
|
|
35
|
-
catch_errors { Proxy::OpenBolt.openbolt_options.to_json}
|
|
35
|
+
catch_errors { Proxy::OpenBolt.openbolt_options.to_json }
|
|
36
36
|
end
|
|
37
37
|
|
|
38
38
|
post '/launch/task' do
|
|
39
39
|
catch_errors do
|
|
40
|
-
|
|
40
|
+
begin
|
|
41
|
+
data = JSON.parse(request.body.read)
|
|
42
|
+
rescue JSON::ParserError => e
|
|
43
|
+
raise Error.new(message: "Invalid JSON in request body: #{e.message}")
|
|
44
|
+
end
|
|
41
45
|
Proxy::OpenBolt.launch_task(data)
|
|
42
46
|
end
|
|
43
47
|
end
|
|
@@ -51,40 +55,15 @@ module Proxy::OpenBolt
|
|
|
51
55
|
end
|
|
52
56
|
|
|
53
57
|
delete '/job/:id/artifacts' do |id|
|
|
54
|
-
catch_errors
|
|
55
|
-
# Validate the job ID format to prevent directory traversal
|
|
56
|
-
unless id =~ /\A[a-f0-9\-]+\z/i
|
|
57
|
-
raise Proxy::OpenBolt::Error.new(message: "Invalid job ID format")
|
|
58
|
-
end
|
|
59
|
-
|
|
60
|
-
file_path = File.join(Proxy::OpenBolt::Plugin.settings.log_dir, "#{id}.json")
|
|
61
|
-
|
|
62
|
-
if File.exist?(file_path)
|
|
63
|
-
real_path = File.realpath(file_path)
|
|
64
|
-
expected_dir = File.realpath(Proxy::OpenBolt::Plugin.settings.log_dir)
|
|
65
|
-
raise Proxy::OpenBolt::Error.new(message: "Invalid file path") unless real_path.start_with?(expected_dir)
|
|
66
|
-
|
|
67
|
-
File.delete(file_path)
|
|
68
|
-
logger.info("Deleted artifacts for job #{id}")
|
|
69
|
-
{ status: 'deleted', job_id: id, path: file_path }.to_json
|
|
70
|
-
else
|
|
71
|
-
logger.warning("Artifacts not found for job #{id}")
|
|
72
|
-
{ status: 'not_found', job_id: id }.to_json
|
|
73
|
-
end
|
|
74
|
-
end
|
|
58
|
+
catch_errors { Proxy::OpenBolt.delete_artifacts(id) }
|
|
75
59
|
end
|
|
76
60
|
|
|
77
61
|
private
|
|
78
62
|
|
|
79
|
-
def catch_errors
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
e.to_json
|
|
84
|
-
rescue Exception => e
|
|
85
|
-
raise e
|
|
86
|
-
#Proxy::OpenBolt::Error.new(message: "Unhandled exception", exception: e).to_json
|
|
87
|
-
end
|
|
63
|
+
def catch_errors
|
|
64
|
+
yield
|
|
65
|
+
rescue Error => e
|
|
66
|
+
e.to_json
|
|
88
67
|
end
|
|
89
68
|
end
|
|
90
69
|
end
|
|
@@ -0,0 +1,25 @@
|
|
|
1
|
+
collectives = mcollective
|
|
2
|
+
main_collective = mcollective
|
|
3
|
+
connector = nats
|
|
4
|
+
libdir = /opt/puppetlabs/mcollective/plugins
|
|
5
|
+
logger_type = console
|
|
6
|
+
loglevel = warn
|
|
7
|
+
securityprovider = choria
|
|
8
|
+
|
|
9
|
+
# This is the default Choria client configuration shipped with
|
|
10
|
+
# smart_proxy_openbolt. SSL paths, the MCollective certname, and
|
|
11
|
+
# Choria broker addresses are provided by the proxy via OpenBolt
|
|
12
|
+
# CLI flags and do not need to be configured here.
|
|
13
|
+
#
|
|
14
|
+
# To use a custom configuration file instead, set the "Choria Config
|
|
15
|
+
# File" option under Administer > Settings > OpenBolt. If your config
|
|
16
|
+
# file includes SSL
|
|
17
|
+
# paths, leave the Choria SSL settings in Foreman blank and they
|
|
18
|
+
# will be read from the config file.
|
|
19
|
+
#
|
|
20
|
+
# Example SSL configuration for a custom config file:
|
|
21
|
+
#
|
|
22
|
+
# plugin.security.provider = file
|
|
23
|
+
# plugin.security.file.certificate = /etc/puppetlabs/puppet/ssl/certs/<certname>.pem
|
|
24
|
+
# plugin.security.file.key = /etc/puppetlabs/puppet/ssl/private_keys/<certname>.pem
|
|
25
|
+
# plugin.security.file.ca = /etc/puppetlabs/puppet/ssl/certs/ca.pem
|
|
@@ -1,43 +1,33 @@
|
|
|
1
1
|
module Proxy::OpenBolt
|
|
2
2
|
class Error < StandardError
|
|
3
|
-
|
|
4
|
-
fields.each { |key, val| instance_variable_set("@#{key}", val) }
|
|
5
|
-
super(fields[:message])
|
|
6
|
-
end
|
|
7
|
-
|
|
8
|
-
def to_json
|
|
9
|
-
details = {}
|
|
10
|
-
instance_variables.each do |var|
|
|
11
|
-
name = var.to_s.delete("@").to_sym
|
|
12
|
-
val = instance_variable_get(var)
|
|
3
|
+
attr_reader :details
|
|
13
4
|
|
|
14
|
-
|
|
5
|
+
def initialize(message:, **details)
|
|
6
|
+
@details = details
|
|
7
|
+
super(message)
|
|
8
|
+
end
|
|
15
9
|
|
|
16
|
-
|
|
17
|
-
|
|
10
|
+
def to_json(*args)
|
|
11
|
+
result = { message: message }
|
|
12
|
+
details.each do |key, val|
|
|
13
|
+
if key == :exception && val.is_a?(Exception)
|
|
14
|
+
result[:exception] = {
|
|
18
15
|
class: val.class.to_s,
|
|
19
16
|
message: val.message,
|
|
20
|
-
backtrace: val.backtrace
|
|
17
|
+
backtrace: val.backtrace,
|
|
21
18
|
}
|
|
22
19
|
else
|
|
23
|
-
|
|
20
|
+
result[key] = val unless val.nil?
|
|
24
21
|
end
|
|
25
22
|
end
|
|
26
|
-
{ error:
|
|
23
|
+
{ error: result }.to_json(*args)
|
|
27
24
|
end
|
|
28
25
|
end
|
|
29
26
|
|
|
30
27
|
class CliError < Error
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
def
|
|
34
|
-
|
|
35
|
-
message: message,
|
|
36
|
-
exitcode: exitcode,
|
|
37
|
-
stdout: stdout,
|
|
38
|
-
stderr: stderr,
|
|
39
|
-
command: command,
|
|
40
|
-
)
|
|
41
|
-
end
|
|
28
|
+
def exitcode = details[:exitcode]
|
|
29
|
+
def stdout = details[:stdout]
|
|
30
|
+
def stderr = details[:stderr]
|
|
31
|
+
def command = details[:command]
|
|
42
32
|
end
|
|
43
33
|
end
|
|
@@ -2,6 +2,7 @@ require 'concurrent'
|
|
|
2
2
|
require 'securerandom'
|
|
3
3
|
require 'singleton'
|
|
4
4
|
require 'smart_proxy_openbolt/job'
|
|
5
|
+
require 'smart_proxy_openbolt/lru_cache'
|
|
5
6
|
require 'smart_proxy_openbolt/task_job'
|
|
6
7
|
|
|
7
8
|
module Proxy::OpenBolt
|
|
@@ -9,17 +10,18 @@ module Proxy::OpenBolt
|
|
|
9
10
|
include Singleton
|
|
10
11
|
|
|
11
12
|
SHUTDOWN_TIMEOUT = 30
|
|
13
|
+
MAX_CACHED_JOBS = 1000
|
|
12
14
|
|
|
13
15
|
def initialize
|
|
14
|
-
@pool = Concurrent::FixedThreadPool.new(
|
|
15
|
-
@jobs =
|
|
16
|
+
@pool = Concurrent::FixedThreadPool.new(Plugin.settings.workers.to_i)
|
|
17
|
+
@jobs = LruCache.new(MAX_CACHED_JOBS)
|
|
16
18
|
end
|
|
17
19
|
|
|
18
20
|
def add_job(job)
|
|
19
21
|
raise ArgumentError, "Only Job instances can be added" unless job.is_a?(Job)
|
|
20
22
|
id = SecureRandom.uuid
|
|
21
23
|
job.id = id
|
|
22
|
-
@jobs
|
|
24
|
+
@jobs.put(id, job)
|
|
23
25
|
@pool.post { job.process }
|
|
24
26
|
id
|
|
25
27
|
end
|
|
@@ -27,7 +29,7 @@ module Proxy::OpenBolt
|
|
|
27
29
|
def status(id)
|
|
28
30
|
job = get_job(id)
|
|
29
31
|
return :invalid unless job
|
|
30
|
-
job
|
|
32
|
+
job.status
|
|
31
33
|
end
|
|
32
34
|
|
|
33
35
|
def result(id)
|
|
@@ -36,6 +38,10 @@ module Proxy::OpenBolt
|
|
|
36
38
|
job.result
|
|
37
39
|
end
|
|
38
40
|
|
|
41
|
+
def remove_job(id)
|
|
42
|
+
@jobs.delete(id)
|
|
43
|
+
end
|
|
44
|
+
|
|
39
45
|
# How many workers are currently busy
|
|
40
46
|
def num_running
|
|
41
47
|
@pool.length
|
|
@@ -56,36 +62,36 @@ module Proxy::OpenBolt
|
|
|
56
62
|
@pool.running?
|
|
57
63
|
end
|
|
58
64
|
|
|
59
|
-
# Stop accepting tasks and wait
|
|
60
|
-
#
|
|
61
|
-
def shutdown(timeout)
|
|
65
|
+
# Stop accepting tasks and wait for in-flight jobs to finish.
|
|
66
|
+
# If timeout is nil, wait forever.
|
|
67
|
+
def shutdown(timeout = SHUTDOWN_TIMEOUT)
|
|
62
68
|
@pool.shutdown
|
|
63
|
-
@pool.wait_for_termination(
|
|
69
|
+
@pool.wait_for_termination(timeout)
|
|
64
70
|
end
|
|
65
71
|
|
|
66
72
|
private
|
|
67
73
|
|
|
68
74
|
def get_job(id)
|
|
69
|
-
|
|
75
|
+
cached = @jobs.get(id)
|
|
76
|
+
return cached if cached
|
|
77
|
+
|
|
70
78
|
# Look on disk for a past run that may have happened
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
end
|
|
79
|
+
file = Proxy::OpenBolt.result_file_path(id)
|
|
80
|
+
begin
|
|
81
|
+
data = JSON.parse(File.read(file))
|
|
82
|
+
return nil if data['schema'].nil? || data['schema'] != 1
|
|
83
|
+
return nil if data['status'].nil?
|
|
84
|
+
# This is only for reading back status and result. Don't try
|
|
85
|
+
# to fill in the other arguments correctly, and don't assume
|
|
86
|
+
# they are there after execution.
|
|
87
|
+
job = Job.new(nil, nil, nil)
|
|
88
|
+
job.id = id
|
|
89
|
+
job.update_status(data['status'].to_sym)
|
|
90
|
+
@jobs.put(id, job)
|
|
91
|
+
job
|
|
92
|
+
rescue Errno::ENOENT, JSON::ParserError
|
|
93
|
+
nil
|
|
87
94
|
end
|
|
88
|
-
job
|
|
89
95
|
end
|
|
90
96
|
end
|
|
91
97
|
end
|