cem_acpt 0.11.2 → 0.12.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 68da70c79c3346b5ce515014b2202746db3f2ef2395c60aa4891ea15635f7e7b
-  data.tar.gz: 0273522771cc02caac4f07e7b7c150f7ab3b7bbc4fc3b9126309819a5f607608
+  metadata.gz: b82bc3c9709dc1486de6675c33508a532f8c036fde480e6fd967e5a2e7dfdae8
+  data.tar.gz: 4fad921e96175030f521ac21a898ee3488a57b109e309f696980e044efc28123
 SHA512:
-  metadata.gz: eff07c3af3f0692706e5863825bb324b6f9b1f821cebd54c0baad9b6b4ea30850d7307342188eaa9c81de6833650cb472f74b8bc378a333886117320033dfbd2
-  data.tar.gz: 5c5cf88592e47515349bde762ee85baf5f07000b024d20e6f5fa550780641049b5b9cf514ab3a74a804fbfb8685c73fd6227271a7d2ea2aa734e9abc819e26f6
+  metadata.gz: b711d5b4ee1cf1f4c712d38b12c68fd3e20939a69e161004b1b2ecc0256838c04473bf089b86e1c78097db848197b6218617cbc31fb53ba52a2bd7dc84910b07
+  data.tar.gz: bd425c7655b89ce251c32d239b0dd7ac1333d0d17f262e08a8c997b388a065a900f6849404e31fbdd969d3e8042aeb934b276b2b002611c72e190e4094245a83

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    cem_acpt (0.11.2)
+    cem_acpt (0.12.0)
       async-http (>= 0.60, < 0.70)
       bcrypt_pbkdf (>= 1.0, < 2.0)
       deep_merge (>= 1.2, < 2.0)

data/README.md

@@ -27,6 +27,7 @@ cem_acpt --config ./cem_acpt_config.yaml
 ```bash
 cem_acpt -h | --help
 cem_acpt_image -h | --help
+cem_acpt_scan -h | --help
 ```
 
 > If you do not delete Gemfile.lock before running `bundle install`, you may encounter dependency version errors. Just delete Gemfile.lock and run `bundle install` again to get past these.
@@ -297,6 +298,98 @@ Much like `name_pattern_vars`, specifying the `image_name_builder` top-level key
 * Destroy the test nodes
 * Report the results of the tests
 
+## Automated benchmark scans (`cem_acpt_scan`)
+
+`cem_acpt_scan` is a third binary that provisions test nodes the same way `cem_acpt` does, applies the acceptance test case's `manifest.pp`, then runs a real benchmark scan against the node — OpenSCAP for STIG profiles, CIS-CAT Pro for CIS profiles — instead of executing Goss assertions. It exists for AI-driven and human workflows that need to validate a manifest change against the actual published benchmark, not just developer-authored Goss tests.
+
+Linux only in this version. A Windows test case requested via `-t` causes the run to fail before any node is provisioned.
+
+### Quickstart
+
+From the root of the consuming module (e.g. `sce_linux`):
+
+```bash
+cem_acpt_scan --config ./cem_acpt_scan_config.yaml -t cis_rhel-8_firewalld_server_2
+```
+
+`cem_acpt_scan` discovers test cases the same way `cem_acpt` does, but only `manifest.pp` is required in each test directory — `goss.yaml` is optional.
+
+### CLI flags
+
+In addition to the shared flags (`-c/--config`, `-I/--CI`, `-q/--quiet`, `-v/--verbose`, `-L/--log-file`, `-Y`, `-X`, `-D/--no-destroy-nodes`, etc.), `cem_acpt_scan` adds:
+
+| Flag | Effect |
+| --- | --- |
+| `-m, --module-dir DIR` | Path to the consuming module root. Same semantics as `cem_acpt`. |
+| `-t, --tests TESTS` | Comma-separated list of acceptance-test case names to scan. |
+| `--scanner SCANNER` | Override the scanner. One of `openscap`, `ciscat`. Default: derived from the test-case framework (`stig_*` → openscap, `cis_*` → ciscat). |
+| `--threshold N` | Override the global pass/fail score threshold (float, 0–100). |
+| `--scan-output FILE` | Save the normalized JSON report to `FILE`. Multi-case runs fan out to `FILE.<test_case>.json` so reports don't collide. Without this flag, the JSON is printed to stdout only. |
+
+### Configuration
+
+Scan-mode settings live under the top-level `cem_acpt_scan` config key:
+
+```yaml
+cem_acpt_scan:
+  # Global pass/fail threshold (scanner-native pass-rate, 0-100).
+  threshold: 80.0
+
+  # Per-test-case threshold overrides. Optional.
+  test_thresholds:
+    cis_rhel-8_firewalld_server_2: 75.0
+
+  # Required for any test case that resolves to the ciscat scanner.
+  # Local file path or gs:// URI; gs:// URIs are downloaded to a per-run
+  # cache file before being passed to Terraform.
+  cis_cat_pro_source: gs://my-bucket/cis-cat-pro-4.x.tar.gz
+  # cis_cat_pro_source: /opt/cis-cat-pro/Assessor-CLI-4.5.0.tar.gz
+
+  # Scanner-native profile id mapping, keyed by full acceptance-test
+  # directory name. Required: a missing entry fails before provisioning.
+  profiles:
+    openscap:
+      stig_rhel-8_firewalld_public_3: xccdf_org.ssgproject.content_profile_stig
+    cis_cat:
+      cis_rhel-8_firewalld_server_2: xccdf_org.cisecurity.benchmarks_profile_Level_2_-_Server
+
+  # On-node scan daemon (a small WebRick service installed at provision time).
+  daemon:
+    port: 8084
+    ready_timeout: 60
+```
+
+### Output
+
+For each scanned test case, `cem_acpt_scan` prints a normalized JSON document to stdout:
+
+```json
+{
+  "test_case": "cis_rhel-8_firewalld_server_2",
+  "scanner": "ciscat",
+  "profile": "xccdf_org.cisecurity.benchmarks_profile_Level_2_-_Server",
+  "score": 87.4,
+  "threshold": 80.0,
+  "passed_count": 187,
+  "failed_count": 27,
+  "not_applicable_count": 14,
+  "error_count": 0,
+  "rules": [
+    { "id": "...", "title": "...", "severity": "high", "result": "pass" }
+  ],
+  "pass": true
+}
+```
+
+`--scan-output FILE` writes the same JSON to disk. The exit code is `0` if every test case scored at or above its threshold, `1` if any scored below.
+
+### How it differs from `cem_acpt`
+
+* The runner registers a single `:scan` action group instead of `:goss` and `:bolt`.
+* The acceptance-test discovery predicate flips from `goss.yaml` to `manifest.pp` — scan-only test directories don't need a Goss file.
+* The on-node Goss systemd services are skipped; in their place an on-node scan daemon is installed (mirroring the Goss server pattern but serving `/scan` and `/health` endpoints).
+* Pass/fail is decided by the configured threshold, not by per-rule assertions.
+
 ## Generating acceptance test node images
 
 CemAcpt provides the command `cem_acpt_image` that allows you to generate new acceptance test node images that are then used with the `cem_acpt` command. This is useful for when you want to test a new version of Puppet or a new OS.

data/docs/ARCHITECTURE.md

@@ -62,7 +62,6 @@ cem_acpt/
 │   │   ├── logging/          # log line formatters
 │   │   ├── platform.rb       # dynamic loader for platform implementations
 │   │   ├── platform/         # base + GCP platform module
-│   │   ├── provision.rb      # provisioner factory
 │   │   ├── provision/        # Terraform driver + OS-specific backends
 │   │   ├── test_runner.rb    # main TestRunner::Runner
 │   │   ├── test_runner/      # log_formatter/, test_results.rb
@@ -170,9 +169,9 @@ ordering visible in code (`base.rb:125-155`) is:
 6. **`add_static_options!`** runs last as a single phase; internally
    it does two things:
    - 6a. Sets the framework-owned keys `user_config.dir`,
-     `user_config.file`, `provisioner = 'terraform'`, and
-     `terraform.dir`. Only these four keys clobber anything earlier;
-     the rest of the merged config is untouched.
+     `user_config.file`, and `terraform.dir`. Only these three keys
+     clobber anything earlier; the rest of the merged config is
+     untouched.
    - 6b. Calls `set_third_party_env_vars!`: `RUNNER_DEBUG=1` forces
      `log_level=debug` and `verbose=true`; `GITHUB_ACTIONS=true` or
      `CI=true` forces `ci_mode=true`.
@@ -402,12 +401,11 @@ name, test name).
 
 ## 7. Provisioner: Terraform
 
-There is currently one provisioner. `Provision.new_provisioner`
-(`lib/cem_acpt/provision.rb`) returns
-`Provision::Terraform` for `provisioner == 'terraform'` and raises
-otherwise. `provisioner` is currently force-set to `'terraform'` in
-`Config::Base#add_static_options!`, so this dispatch is effectively
-fixed today.
+There is currently one provisioner. `TestRunner#provision_test_nodes`
+instantiates `Provision::Terraform` directly. A factory was removed
+in CEM-6720 because it signaled an extension surface that did not
+exist; re-introduce one when there is a concrete second
+implementation.
 
 ### `Provision::Terraform`
 
@@ -1017,17 +1015,13 @@ These are things the exploration surfaced that don't seem load-bearing
 in the current code but would warrant a quick decision before
 spec-driven refactors.
 
-1. **`provisioner` is statically forced to `'terraform'`** in
-   `Config::Base#add_static_options!`, even though the `Provision`
-   factory branches on it. If we want a non-Terraform path in the
-   future the static set would need to become a default.
-2. **Platform constant cache.** `Platform.platform_class` caches the
+1. **Platform constant cache.** `Platform.platform_class` caches the
    created class under `CemAcpt::Platform` (e.g.
    `CemAcpt::Platform::Gcp`) and uses `const_defined?(name, false)`,
    so unrelated same-named constants elsewhere in the graph cannot
    collide. Mixins live alongside under `CemAcpt::Platform::Mixin`.
    Resolved by [CEM-6717](rfcs/0008-namespace-platform-classes.md).
-3. **`lib/terraform/image/gcp/windows/` is an empty directory.** It
+2. **`lib/terraform/image/gcp/windows/` is an empty directory.** It
    contains only `.keep` — no `main.tf`. As of the fix for
    [CEM-6712](rfcs/0003-windows-image-builder-template.md),
    `TerraformBuilder#assert_template_present!` raises

data/exe/cem_acpt_scan

@@ -0,0 +1,16 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require 'dotenv/load'
+require 'cem_acpt'
+require 'cem_acpt/cli'
+
+command, options = CemAcpt::Cli.parse_opts_for(:cem_acpt_scan)
+# Set CLI defaults
+options[:module_dir] = Dir.pwd unless options[:module_dir]
+if (options[:log_level] == 'debug' || options[:verbose]) && !options[:quiet]
+  puts '############################### RUNNING ###############################'
+  puts "Command #{File.basename(__FILE__)}:#{command} using options from command line: #{options}"
+  puts '#######################################################################'
+end
+CemAcpt.run(command, File.basename(__FILE__).to_sym, options)

data/lib/cem_acpt/cli.rb

@@ -92,6 +92,30 @@ module CemAcpt
           opts.on('-F', '--filter FILTER', 'Filter images to build by name with regex. Example: -F "(alma|rhel)-8"') do |f|
             options[:image_name_filter] = Regexp.new(f)
           end
+        when :cem_acpt_scan
+          opts.on('-m', '--module-dir DIR', 'Set module directory. Example: -m "/tmp/module"') do |m|
+            options[:module_dir] = m
+          end
+
+          opts.on('-t', '--tests TESTS', 'Set tests. Example: -t "test1,test2"') do |t|
+            options[:tests] = t.split(',')
+          end
+
+          opts.on('--scanner SCANNER', %w[openscap ciscat],
+                  'Override scanner choice. One of: openscap, ciscat. Default: derived from test-case framework.') do |s|
+            options[:cem_acpt_scan] ||= {}
+            options[:cem_acpt_scan][:scanner] = s
+          end
+
+          opts.on('--threshold N', Float, 'Override the global pass/fail score threshold (0-100).') do |n|
+            options[:cem_acpt_scan] ||= {}
+            options[:cem_acpt_scan][:threshold] = n
+          end
+
+          opts.on('--scan-output FILE', 'Save normalized JSON report to FILE. Multi-case runs fan out to FILE.<test_case>.json.') do |f|
+            options[:cem_acpt_scan] ||= {}
+            options[:cem_acpt_scan][:scan_output] = f
+          end
         end
 
         opts.on('-D', '--debug', 'Enable debug logging') do

data/lib/cem_acpt/config/base.rb

@@ -52,7 +52,6 @@ module CemAcpt
         no_destroy_nodes
         no_ephemeral_ssh_key
         platform
-        provisioner
         puppet
         quiet
         secrets
@@ -190,6 +189,16 @@ module CemAcpt
       end
       alias ci? ci_mode?
 
+      # Whether this config represents a scan run (cem_acpt_scan).
+      # Subclasses that drive a scan workflow override to true.
+      def scan_mode?
+        false
+      end
+
+      def scan?
+        scan_mode?
+      end
+
       def debug_mode?
         get('log_level') == 'debug'
       end
@@ -262,8 +271,6 @@ module CemAcpt
         add_config_explanation('user_config.dir', 'static value')
         config.dset('user_config.file', user_config_file)
         add_config_explanation('user_config.file', 'static value')
-        config.dset('provisioner', 'terraform')
-        add_config_explanation('provisioner', 'static value')
         config.dset('terraform.dir', terraform_dir)
         add_config_explanation('terraform.dir', 'static value')
         set_third_party_env_vars!(config)

data/lib/cem_acpt/config/cem_acpt_scan.rb

@@ -0,0 +1,112 @@
+# frozen_string_literal: true
+
+require_relative 'base'
+
+module CemAcpt
+  module Config
+    # Holds the configuration for cem_acpt_scan.
+    #
+    # Mirrors the {Config::CemAcpt} pattern: a hardcoded VALID_KEYS list and a
+    # {#defaults} hash. The new top-level key {cem_acpt_scan} carries
+    # scan-specific settings (scanner choice, threshold, profile mapping,
+    # daemon port, CIS-CAT Pro source); the rest of the schema reuses keys
+    # already understood by other commands so node provisioning code does not
+    # have to special-case scan mode.
+    class CemAcptScan < Base
+      VALID_KEYS = %i[
+        cem_acpt_scan
+        actions
+        image_name_builder
+        module_dir
+        node_data
+        test_data
+        tests
+      ].freeze
+
+      def env_var_prefix
+        'CEM_ACPT_SCAN'
+      end
+
+      def scan_mode?
+        true
+      end
+
+      # The default configuration
+      def defaults
+        {
+          actions: {},
+          cem_acpt_scan: {
+            scanner: nil, # nil = derive from test-case framework (cis -> ciscat, stig -> openscap)
+            threshold: 80.0,
+            test_thresholds: {},
+            scan_output: nil, # nil = stdout only; otherwise a file path
+            cis_cat_pro_source: nil, # local path or gs:// URI; required when any test resolves to ciscat
+            cis_cat_pro_license: nil, # local path or gs:// URI to license bundle (.tar.gz or .zip); required when any test resolves to ciscat. Rotates independently of cis_cat_pro_source.
+            daemon: {
+              port: 8084,
+              ready_timeout: 60,
+            },
+            profiles: {
+              openscap: {},
+              cis_cat: {},
+            },
+            benchmarks: {
+              cis_cat: {},
+            },
+          },
+          ci_mode: false,
+          config_file: nil,
+          image_name_builder: {
+            character_substitutions: [['_', '-']],
+            parts: ['cem-acpt', '$image_fam', '$collection', '$firewall'],
+            join_with: '-',
+          },
+          log_level: 'info',
+          log_file: nil,
+          log_format: 'text',
+          module_dir: Dir.pwd,
+          node_data: {},
+          no_destroy_nodes: false,
+          no_ephemeral_ssh_key: false,
+          platform: {
+            name: 'gcp',
+            gcp: {
+              windows_bucket: 'win_cem_acpt',
+            },
+          },
+          quiet: false,
+          test_data: {
+            for_each: {
+              collection: %w[puppet8],
+            },
+            vars: {},
+            name_pattern_vars: %r{^(?<framework>[a-z]+)_(?<image_fam>[a-z0-9-]+)_(?<firewall>[a-z]+)_(?<framework_vars>[-_a-z0-9]+)$},
+            vars_post_processing: {
+              new_vars: [
+                {
+                  name: 'profile',
+                  string_split: {
+                    from: 'framework_vars',
+                    using: '_',
+                    part: 0,
+                  },
+                },
+                {
+                  name: 'level',
+                  string_split: {
+                    from: 'framework_vars',
+                    using: '_',
+                    part: 1,
+                  },
+                },
+              ],
+              delete_vars: %w[framework_vars],
+            },
+          },
+          tests: [],
+          verbose: false,
+        }
+      end
+    end
+  end
+end

data/lib/cem_acpt/config.rb

@@ -5,5 +5,6 @@ module CemAcpt
   module Config
     require_relative 'config/cem_acpt'
     require_relative 'config/cem_acpt_image'
+    require_relative 'config/cem_acpt_scan'
   end
 end

data/lib/cem_acpt/platform/gcp.rb

@@ -37,19 +37,16 @@ module CemAcpt
           @gcp_username = @config.get('platform.username')
           return @gcp_username if @gcp_username
 
-          stdout, stderr, status = Open3.capture3('gcloud compute os-login describe-profile --format json')
-          raise "gcloud os-login describe-profile failed (exit #{status.exitstatus}): #{stderr.chomp}" unless status.success?
+          stdout, stderr, status = Open3.capture3('gcloud auth list --format json')
+          raise "gcloud auth list failed (exit #{status.exitstatus}): #{stderr.chomp}" unless status.success?
 
           parsed = JSON.parse(stdout)
-          accounts = parsed['posixAccounts']
-          raise "gcloud os-login profile returned no posixAccounts. stderr: #{stderr.chomp}" if accounts.nil? || accounts.empty?
+          active_account = parsed.find { |account| account['status'] == 'ACTIVE' }
+          raise 'No active account found in gcloud auth list output' unless active_account
 
-          username = accounts.first['username']
-          raise "gcloud os-login posixAccounts entry has no username. stdout: #{stdout.chomp}" unless username
-
-          @gcp_username = username
+          @gcp_username = active_account['account'].split('@').first
         rescue JSON::ParserError => e
-          raise "Failed to parse gcloud os-login output: #{e.message}. stdout: #{stdout.chomp}"
+          raise "Failed to parse gcloud auth output: #{e.message}. stdout: #{stdout.chomp}"
         end
 
         def gcp_credentials_file

data/lib/cem_acpt/provision/terraform/linux.rb

@@ -47,31 +47,77 @@ module CemAcpt
         commands << apply_command
       end
 
+      # Commands to run on a scan-mode node. Skips Goss installation and the
+      # Goss systemd units (they are not used in scan mode), installs the
+      # scan daemon and its scanners (including a Java JRE because CIS-CAT
+      # Pro's Assessor-CLI is a Java app), then applies the puppet manifest.
+      # The CIS-CAT Pro tarball is uploaded *and* extracted by the
+      # cis_cat_pro_upload null_resource in main.tf, so this method does not
+      # touch /opt/cis-cat-pro/.
+      # @return [Array<String>] commands for `provisioner "remote-exec"`.
+      def scan_provision_commands
+        commands = [
+          "sudo /opt/puppetlabs/puppet/bin/puppet module install #{destination_provision_directory}/#{remote_module_package_name}",
+          'sudo /opt/puppetlabs/puppet/bin/gem install webrick',
+          'sudo chmod +x /opt/cem_acpt/log_service/log_service.rb',
+          'sudo /opt/cem_acpt/log_service/log_service.rb',
+          install_scanner_packages_command,
+          install_java_command,
+          'sudo chmod +x /opt/cem_acpt/scan/scan_service.rb',
+          'sudo cp /opt/cem_acpt/scan/scan_service.service /etc/systemd/system/scan_service.service',
+          'sudo systemctl daemon-reload',
+          'sudo systemctl start scan_service.service && sudo systemctl enable scan_service.service',
+        ]
+        commands << apply_command
+        commands
+      end
+
       # A wrapper around provision_commands that allows for extra commands to be added for a specific OS version(i.e EL 8)
       # @param image_name [String] The name of the OS image being provisioned.
+      # @param scan_mode [Boolean] When true, returns the scan-mode command list
+      #   (Goss-free, scan-daemon-installed) wrapped with the same OS-specific prefixes.
       # @return [Array<String>] An array of shell commands to be run on the provisioned nodes to set up the necessary
       #   environment for running the Puppet manifest, including any additional commands needed for specific OS
       #   versions.
-      def provision_commands_wrapper(image_name)
+      def provision_commands_wrapper(image_name, scan_mode: false)
+        base = scan_mode ? scan_provision_commands : provision_commands
         if ['rhel-8', 'oel-8', 'alma-8', 'rocky-8'].any? { |el8| image_name.include?(el8) }
           commands = [
             'sudo dnf upgrade --refresh -y rpm glibc',
             'sudo rm /var/lib/rpm/.rpm.lock',
             'sudo dnf upgrade -y dnf',
           ]
-          commands = (commands << provision_commands).flatten
-          commands
+          (commands << base).flatten
         elsif image_name.include?('ubuntu')
           commands = ['sudo apt purge -y unattended-upgrades', 'sudo apt-get update -y']
-          commands = (commands << provision_commands).flatten
-          commands
+          (commands << base).flatten
         else
-          provision_commands
+          base
         end
       end
 
       private
 
+      # Returns the OS-package install command for OpenSCAP. Both EL and Ubuntu
+      # ship `oscap`; the SCAP content package differs by distro family. The
+      # command is intentionally OR-chained so it succeeds on whichever package
+      # manager is present without needing fact-based dispatch.
+      # @return [String] shell command to install the OpenSCAP scanner.
+      def install_scanner_packages_command
+        # Try dnf first (RHEL/Alma/Rocky/Oracle), fall back to apt (Ubuntu).
+        'sudo dnf install -y openscap-scanner scap-security-guide || sudo apt-get install -y libopenscap8 ssg-base ssg-debderived'
+      end
+
+      # Returns the OS-package install command for a headless Java runtime.
+      # CIS-CAT Pro's Assessor-CLI.sh is a Java application, so the JRE must
+      # be present on every scan-mode node regardless of which scanner the
+      # test case resolves to (installing unconditionally is cheap, and
+      # threading scanner-resolution into the provisioner is not worth it).
+      # @return [String] shell command to install a headless JRE.
+      def install_java_command
+        'sudo dnf install -y java-11-openjdk-headless || sudo apt-get install -y default-jre-headless'
+      end
+
       # @return [String] The command to apply the Puppet manifest on the provisioned nodes, including any necessary
       #   options for logging and debugging. This command is constructed based on the configuration settings for the
       #   provisioner, and may include additional options for debug and verbose logging if those settings are enabled.