foreman_openbolt 0.1.1 → 1.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 9c2932001d70466f491793accaaf126a6995e8278da33c771c8e74802dee996c
-  data.tar.gz: 7ce6c51f49107dfefe7e28ed9a62eb4b5f41e838ba0fdb057cecf97e67957cfc
+  metadata.gz: 19c30e74d249eacf814179dac70292c3e42182b39477cab06adf304885298587
+  data.tar.gz: bbc28cd01be2a7f100d1ec015c92aff471584e7445468184e7ef1bd97c485b27
 SHA512:
-  metadata.gz: f3667fe230f4417c35b7acf87411132be4122999bb858e6f753b214891809d33732ca3cd2b7f49ee23994f8a4d9d9e969266532989d4a54e1a86c3d0bfccf4cc
-  data.tar.gz: c1e2ade0752b6546fd5e3146128e8454fd89455fb291d5cedc774de6b340bc61b7b45683af3594fd9df381841c395b3156ce462b4f33fa89cbb345a075068a7e
+  metadata.gz: 2b40018885e22e7d3690b13c524787dadbf8ad2fe8fa5af2e82e5eb4fd99d65a56a7015cbd02717573c5cd12395d1a157130c16257fa94a5b3f28ef78470fd55
+  data.tar.gz: 753476abd475f3de15ad928874048c9772d4a55534129f35b9d4c032beb5c0f787e8ba6aed271763bb59d66383986360cbf8d0438618aefdb9f2175790b959c7

data/README.md

@@ -1,4 +1,4 @@
-# ForemanPluginTemplate
+# foreman-openbolt
 
 [![License](https://img.shields.io/github/license/overlookinfra/foreman_openbolt.svg)](https://github.com/overlookinfra/foreman_openbolt/blob/master/LICENSE)
 [![Test](https://github.com/overlookinfra/foreman_openbolt/actions/workflows/test.yml/badge.svg)](https://github.com/overlookinfra/foreman_openbolt/actions/workflows/test.yml)
@@ -6,28 +6,234 @@
 [![RubyGem Version](https://img.shields.io/gem/v/foreman_openbolt.svg)](https://rubygems.org/gems/foreman_openbolt)
 [![RubyGem Downloads](https://img.shields.io/gem/dt/foreman_openbolt.svg)](https://rubygems.org/gems/foreman_openbolt)
 
-*Introdction here*
+Bringing OpenBolt Task & Plans into Foreman!
+
+## Introduction
+
+[OpenBolt](https://github.com/OpenVoxProject/openbolt) is the open source successor of [Bolt](https://github.com/puppetlabs/bolt) by [Perforce](https://www.perforce.com/).
+OpenBolt supports running Tasks or Plans against various targets  via different transport protocols.
+OpenBolt and Bolt are CLI-only tools.
+They connect to the targets from a central location (usually a jumpnode or workstation).
+
+## Tasks
+
+Tasks are little executeable things, like binaries or scripts.
+They are enhanced with a metadata file, which describes input and output parameters.
+A task is copied to N targets and executed there.
+
+## Plans
+
+Plans provide complex logic options, written in Puppet DSL.
+Besides the usual Puppet DSL functions, it's also possible to execute tasks and evaluate their responses.
+
+## OpenBolt in Foreman!
+
+OpenBolt is the Ansible counterpart and OpenBolt is Puppet "native".
+OpenBolt and Puppet integrate very well together and OpenBolt can reuse your existing Puppet code.
+Since OpenBolt is a CLI only application, and most Puppet users run Foreman anyways, it made sense to integrate OpenBolt into Foreman, instead of writing another web UI.
 
 ## Installation
 
-See [How_to_Install_a_Plugin](http://projects.theforeman.org/projects/foreman/wiki/How_to_Install_a_Plugin)
-for how to install Foreman plugins
+The installation is split into four parts:
+
+* Foreman Plugin
+* Foreman Smartproxy Plugin
+* OpenBolt
+* Code Deployment
+
+See [How_to_Install_a_Plugin](https://theforeman.org/plugins/#2.Installation) for how to install Foreman plugins.
+The [theforeman/foreman](https://github.com/theforeman/puppet-foreman/blob/master/manifests/plugin/openbolt.pp) puppet module also supports the **Foreman plugin** installation.
+The [theforeman/foreman_proxy](https://github.com/theforeman/puppet-foreman_proxy/blob/master/manifests/plugin/openbolt.pp) puppet module also supports the **Foreman Smartproxy plugin** installation.
+
+
+The Foreman plugin provides UI elements to start Tasks on various nodes.
+Foreman then talks to a Smartproxy to run OpenBolt.
+The Smartproxy also establishes the connections to the various targets.
+This is usually a ssh or WinRM connection (and soon choria, see [the TODO section](#todo)).
+
+You need to have `bolt` in your `$PATH` on the Smartproxy.
+OpenBolt packages are available at [yum.voxpupuli.org](https://yum.voxpupuli.org/) & [apt.voxpupuli.org](https://apt.voxpupuli.org/) in the openvox8 repo.
+You can also use the legacy Bolt packages from Perforce from the `puppet-tools` repo on [apt.puppet.com](https://apt.puppet.com/) or [yum.puppet.com](https://yum.puppet.com/).
+
+The integration is supported on Foreman 3.17 and all following versions, including development/nightly builds.
+
+OpenBolt relies on Tasks & Plans. They are distributed as puppet modules.
+The plugin assumes that you deployed your code.
+We recommend to use [r10k](https://github.com/puppetlabs/r10k?tab=readme-ov-file#r10k) or [g10k](https://github.com/xorpaul/g10k?tab=readme-ov-file#g10k) to deploy code, as you do it on your compilers.
+
+A handful of core/default Tasks & Plans are also included in the [OpenBolt rpm/deb packages](https://github.com/OpenVoxProject/openbolt/blob/main/Puppetfile).
 
 ## Usage
 
-*Usage here*
+(all screenshots were taken on Foreman 3.17)
+
+After installation, you will see a new UI element
+
+![foreman UI menu screenshot](./ext/foreman-ui-menu.png)
+
+The "Launch Task" option allows you to select any smartproxy with the `openbolt` feature (which is available when the OpenBolt Smartproxy plugin is installed).
+Afterwards you can select N targets to run the task and select an available task from the selected Smartproxy.
+On the right side you can configure OpenBolt connection settings.
+
+![launch task detail view](./ext/foreman-launch-task.png)
+
+After selecting a task, the task metadata is fetched and shown.
+Additional input elements will appear, if the task support it.
+
+![service task metadata](./ext/task-metadata-minimal.png)
+
+The metadata can contains a description and datatypes for tasks.
+Those information can be shown as well.
+
+![service task detailed metadata](./ext/task-metadata.png)
+
+While the task is running, the UI polls the status from the smart proxy.
+
+![task loading screen](./ext/task-running.png)
+
+After the task finished, it will display a success for failure page.
+
+![failed task view](./ext/task-execution-details.png)
+
+You can also see the used parameters for a task.
+
+![task used parameters](./ext/task-task-details.png)
+
+We also display the used OpenBolt command line, in case you want to manually run it or debug it.
+
+![display used OpenBolt command](./ext/task-log-output.png)
+
+OpenBolt returns JSON for executed tasks.
+That's visible in the UI.
+For failed tasks but also for passed tasks.
+
+![failed task output](./ext/task-result.png)
+
+![service task passed on two nodes](./ext/task-successful-result.png)
+
+## Development
+
+### Linting
+
+```bash
+bundle exec rake lint        # Run all linters (rubocop, erb_lint, eslint)
+bundle exec rake lint:fix    # Auto-fix where possible
+```
+
+Ruby and ERB linters run directly. The JavaScript linter requires npm dependencies, so either install them locally (`npm install --legacy-peer-deps`) or run lint:js inside a container:
+
+```bash
+CONTAINER=1 bundle exec rake lint:js
+```
+
+### Unit Tests
+
+Unit tests run inside Docker containers with a full Foreman installation. Requires Docker with compose support.
+
+```bash
+bundle exec rake test:unit:up    # Build image, start containers, install deps
+bundle exec rake test:unit:ruby  # Run Ruby tests
+bundle exec rake test:unit:js    # Run JavaScript tests
+bundle exec rake test:unit:all   # Run all unit tests
+bundle exec rake test:unit:down  # Stop and remove containers
+bundle exec rake test              # Shortcut: up, test, down in one step
+```
+
+Set `FOREMAN_VERSION` to test against a specific Foreman version (default: `3.18`):
+
+```bash
+FOREMAN_VERSION=3.17 bundle exec rake test:unit:up
+```
+
+### Acceptance Tests
+
+Acceptance tests exercise the plugin through the browser using Capybara and Selenium. They build RPMs, start a multi-container environment (Foreman + OpenVox + SSH targets + Chromium), and run tests against the real UI.
+
+**Prerequisites:**
+
+```bash
+bundle install --with acceptance
+```
+
+The [smart_proxy_openbolt](https://github.com/overlookinfra/smart_proxy_openbolt) and [foreman-packaging](https://github.com/theforeman/foreman-packaging) repos are cloned automatically when needed.
+
+**Running:**
+
+```bash
+bundle exec rake acceptance         # Full cycle: up, run tests, down
+bundle exec rake acceptance:up      # Build RPMs, start Foreman, configure everything
+bundle exec rake acceptance:run     # Run tests (requires up first)
+bundle exec rake acceptance:down    # Stop containers
+bundle exec rake acceptance:clean   # Full reset: stop containers, remove images and artifacts
+```
+
+The `acceptance:up` task is idempotent and can be re-run to pick up new RPM changes. It caches the Foreman Docker image per version so subsequent runs are faster.
+
+**Watching tests in the browser:**
+
+Set `HEADFUL=1` to disable headless mode, then open `http://localhost:7900` (password: `secret`) to watch the tests via noVNC:
+
+```bash
+HEADFUL=1 bundle exec rake acceptance:run
+```
+
+**Running a subset of tests:**
+
+`acceptance:run` accepts `TEST=<path>` to limit which test files are loaded, and `TESTOPTS=<opts>` to forward options (e.g. `--name=/pattern/`) to the Test::Unit autorunner. Both can be combined.
+
+```bash
+# Run every test in one file
+bundle exec rake acceptance:run TEST=test/acceptance/tests/settings_test.rb
+
+# Run a single test by exact method name (any file)
+bundle exec rake acceptance:run TESTOPTS='--name=test_echo_task_succeeds_on_all_targets'
+
+# Run tests whose name matches a regex within one file
+bundle exec rake acceptance:run \
+  TEST=test/acceptance/tests/settings_test.rb \
+  TESTOPTS='--name=/host_key/'
+```
+
+**Environment variables:**
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `CHROMEDRIVER_URL` | `http://localhost:4444` | Selenium WebDriver endpoint |
+| `FOREMAN_BRANCH` | `<version>-stable` | Foreman git branch for unit test image (derived from `FOREMAN_VERSION`) |
+| `FOREMAN_PACKAGING_REPO` | `https://github.com/theforeman/foreman-packaging.git` | Git URL for foreman-packaging (cloned automatically for RPM builds) |
+| `FOREMAN_PASS` | `changeme` | Foreman login password |
+| `FOREMAN_URL` | `https://foreman` | Foreman URL as seen by Chrome. Override to run tests against a live instance |
+| `FOREMAN_USER` | `admin` | Foreman login username |
+| `FOREMAN_VERSION` | `3.18` | Foreman version to test against |
+| `HEADFUL` | unset | Set to `1` to show the browser in noVNC |
+| `SELENIUM_IMAGE` | auto-detected (ARM/x86) | Selenium container image (auto-selects `seleniarm/standalone-chromium` or `selenium/standalone-chrome`) |
+| `SMART_PROXY_OPENBOLT_REF` | `main` | Branch or tag to clone |
+| `SMART_PROXY_OPENBOLT_REPO` | `https://github.com/overlookinfra/smart_proxy_openbolt.git` | Git URL for smart_proxy_openbolt (cloned automatically for RPM builds) |
+
+### Building Packages
+
+Build RPM or DEB packages locally using containers. The [foreman-packaging](https://github.com/theforeman/foreman-packaging) repo is cloned automatically:
+
+```bash
+bundle exec rake build:rpm   # Build RPM
+bundle exec rake build:deb   # Build DEB
+```
 
 ## TODO
 
-*Todo list here*
+* Integrate plans into the web UI
+* Provide a choria transport plugin
 
-## Contributing
+## Contributing & support
 
 Fork and send a Pull Request. Thanks!
+If you have questions or need professional support, please join the `#sig-orchestrator` channel on the [Vox Pupuli slack](https://voxpupuli.org/connect/).
 
 ## Copyright
 
-Copyright (c) *year* *your name*
+Copyright (c) *2025* *Overlook InfraTech*
+
+Copyright (c) *2025* *betadots GmbH*
 
 This program is free software: you can redistribute it and/or modify
 it under the terms of the GNU General Public License as published by
@@ -42,14 +248,88 @@ GNU General Public License for more details.
 You should have received a copy of the GNU General Public License
 along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
-## how to release
+## How to Release
+
+### Version locations
+
+Update the version in these files:
+
+1. `lib/foreman_openbolt/version.rb` -- the gem version (authoritative source)
+2. `package.json` -- the npm package version (must match)
+
+If the minimum Foreman version changes, also update:
+
+3. `lib/foreman_openbolt/engine.rb` -- `requires_foreman '>= X.Y.Z'`
+4. `foreman_openbolt.spec.erb` -- `%global foreman_min_version X.Y.Z`
+5. `.github/workflows/build.yml` -- default `foreman_version` and `foreman_packaging_ref` inputs
+
+### Release steps
+
+1. Bump the version in the two files listed above
+2. Generate the changelog:
+   ```bash
+   CHANGELOG_GITHUB_TOKEN=github_pat_... bundle exec rake changelog
+   ```
+3. Create a PR with the version bump and changelog, get it reviewed and merged
+4. Create and push a tag matching the version:
+   ```bash
+   git tag 1.1.0
+   git push origin 1.1.0
+   ```
+5. The [release workflow](.github/workflows/release.yml) runs automatically on tag push and:
+   - Builds the gem
+   - Creates a GitHub Release with auto-generated notes and the gem attached
+   - Publishes the gem to GitHub Packages
+   - Publishes the gem to RubyGems.org (requires the `release` environment)
+   - Verifies the gem is available on RubyGems.org
+
+### RPM/DEB packaging
+
+After the gem is published to RubyGems, both RPM and DEB packages need to be updated in [theforeman/foreman-packaging](https://github.com/theforeman/foreman-packaging).
+
+A bot automatically creates PRs against the `rpm/develop` and `deb/develop` branches to pick up the new gem version. These PRs build packages for Foreman nightly.
+
+For stable Foreman releases (currently 3.17 and 3.18), cherry-pick the packaging commits from the develop branches into the corresponding stable branches. For each stable version you want to support:
+
+```bash
+cd foreman-packaging
+
+# RPM: cherry-pick from rpm/develop into a branch off the stable target
+git checkout rpm/3.18
+git checkout -b cherry-pick/rubygem-foreman_openbolt-rpm-3.18
+git cherry-pick <commit-from-rpm/develop>
+# Push to your fork and open a PR targeting rpm/3.18
+
+# DEB: same approach for the deb side
+git checkout deb/3.18
+git checkout -b cherry-pick/rubygem-foreman-openbolt-deb-3.18
+git cherry-pick <commit-from-deb/develop>
+# Push to your fork and open a PR targeting deb/3.18
+```
+
+PRs against stable branches should be labeled "Stable branch".
+
+**Alternative: manual version bump**
+
+If the cherry-pick doesn't apply cleanly, you can bump the version manually on the stable branch instead.
+
+*RPM:* Checkout the target branch and run `bump_rpm.sh`:
+```bash
+cd foreman-packaging
+git checkout rpm/3.18
+git checkout -b bump_rpm/rubygem-foreman_openbolt
+./bump_rpm.sh packages/plugins/rubygem-foreman_openbolt
+# Review changes, push to your fork, and open a PR targeting rpm/3.18
+```
 
-* bump version in `lib/foreman_openbolt/version.rb` and `package.json`
-* run `CHANGELOG_GITHUB_TOKEN=github_pat... bundle exec rake changelog`
-* create a PR
-* get a review & merge
-* create and push a tag
-* github actions will publish the tag
+*DEB:* Checkout the target branch and update these files:
+- `debian/gem.list` -- new gem filename
+- `foreman_openbolt.rb` -- new version
+- `debian/control` -- dependency versions (if changed)
+- `debian/changelog` -- add a new entry
 
-The Foreman team packages this gem as Debian package (deb) and as RedHat package (rpm).
-They have a bot that will automatically propose an rpm/deb update at [github.com/theforeman/foreman-packaging](https://github.com/theforeman/foreman-packaging/pulls).
+```bash
+git checkout deb/3.18
+git checkout -b bump_deb/ruby-foreman-openbolt
+# Make the changes above, push to your fork, and open a PR targeting deb/3.18
+```

data/Rakefile

@@ -1,99 +1,23 @@
 # frozen_string_literal: true
 
-# !/usr/bin/env rake
-
-begin
-  require 'rdoc/task'
-rescue LoadError
-  require 'rdoc/rdoc'
-  require 'rake/rdoctask'
-  RDoc::Task = Rake::RDocTask
-end
-
-RDoc::Task.new(:rdoc) do |rdoc|
-  rdoc.rdoc_dir = 'rdoc'
-  rdoc.title    = 'ForemanOpenbolt'
-  rdoc.options << '--line-numbers'
-  rdoc.rdoc_files.include('README.rdoc')
-  rdoc.rdoc_files.include('lib/**/*.rb')
-end
-
-require 'rake/testtask'
-
-Rake::TestTask.new(:test) do |t|
-  t.libs << 'lib'
-  t.libs << 'test'
-  t.pattern = 'test/**/*_test.rb'
-  t.verbose = false
-end
-
-begin
-  require 'rubocop/rake_task'
-  RuboCop::RakeTask.new
-rescue LoadError
-  puts 'Rubocop not loaded.'
-end
-
-# This is all kinda screwy. Fix it up later.
-LINTERS = {
-  ruby: { cmd: 'rubocop', fix: '--auto-correct' },
-  erb: { cmd: 'erb_lint', fix: '--autocorrect', glob: '**/*.erb' },
-  js: { image: 'registry.access.redhat.com/ubi9/nodejs-20:latest', cmd: 'npm run lint --', fix: '--fix' },
-}.freeze
-
-namespace :lint do
-  def fix?
-    !ENV['FIX'].nil?
-  end
-
-  def local?
-    !ENV['LOCAL'].nil?
-  end
-
-  def bin
-    ENV['CONTAINER_BIN'] || 'docker'
-  end
-
-  LINTERS.each do |name, cfg|
-    desc "Run #{name} linter#{' (fix)' if fix?}"
-    task name do
-      cmd = [cfg[:cmd]]
-      cmd << cfg[:fix] if fix?
-      cmd << cfg[:glob] unless cfg[:glob].nil? || cfg[:glob].empty? # rubocop:disable Rails/Blank
-      cmd = cmd.join(' ')
-      if cfg[:image] && !local?
-        cmd = "#{bin} run --rm -v #{Dir.pwd}:/code #{cfg[:image]} /bin/bash -c " +
-              "'cd /code && npm install --loglevel=error && #{cmd}'"
-      end
-      sh cmd
-    end
-  end
-
-  desc 'Run all linters'
-  task all: LINTERS.keys
-
-  desc 'Run all linters and apply fixes'
-  task :fix do
-    ENV['FIX'] = 'true'
-    Rake::Task['lint:all'].invoke
+require_relative 'rakelib/utils/shell'
+
+def latest_foreman_version
+  result = Shell.capture(
+    ['git', 'ls-remote', '--tags', 'https://github.com/theforeman/foreman.git'],
+    print_command: false, allowed_exit_codes: [0, 1]
+  )
+  tags = result.output.scan(%r{refs/tags/([^\s]+)$}).flatten
+  versions = tags.filter_map do |tag|
+    Gem::Version.new(tag)
+  rescue ArgumentError
+    nil
   end
+  latest = versions.reject(&:prerelease?).max
+  latest ? "#{latest.segments[0]}.#{latest.segments[1]}" : '3.18'
 end
 
-task default: ['lint:all', 'test']
-
-begin
-  require 'rubygems'
-  require 'github_changelog_generator/task'
+DEFAULT_FOREMAN_VERSION = latest_foreman_version
+FOREMAN_VERSION = ENV.fetch('FOREMAN_VERSION', DEFAULT_FOREMAN_VERSION)
 
-  GitHubChangelogGenerator::RakeTask.new :changelog do |config|
-    config.exclude_labels = %w[duplicate question invalid wontfix wont-fix skip-changelog github_actions]
-    config.user = 'overlookinfra'
-    config.project = 'foreman_openbolt'
-    gem_version = Gem::Specification.load("#{config.project}.gemspec").version
-    config.future_release = gem_version
-  end
-rescue LoadError
-  task :changelog do
-    abort("Run `bundle install --with release` to install the `github_changelog_generator` gem.")
-  end
-end
+task default: :lint