atex 0.1__tar.gz

atex-0.1/.editorconfig

@@ -0,0 +1,6 @@
+root = true
+[*]
+end_of_line = lf
+insert_final_newline = true
+indent_style = space
+indent_size = 4

atex-0.1/.gitignore

@@ -0,0 +1,9 @@
+__pycache__/
+dist/
+*.py[cod]
+*$py.class
+
+# vim swap files
+*~
+*.swp
+*.swo

atex-0.1/COPYING.txt

@@ -0,0 +1,14 @@
+Copyright (C) 2025  Red Hat, Inc. <https://redhat.com/>
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <https://www.gnu.org/licenses/>.

atex-0.1/PKG-INFO

@@ -0,0 +1,11 @@
+Metadata-Version: 2.4
+Name: atex
+Version: 0.1
+License-Expression: GPL-3.0-or-later
+License-File: COPYING.txt
+Classifier: Operating System :: POSIX :: Linux
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Software Development :: Testing
+Requires-Python: >=3.9
+Requires-Dist: fmf>=1.6
+Requires-Dist: urllib3<3,>=2

atex-0.1/README.md

@@ -0,0 +1,87 @@
+# ATEX = Ad-hoc Test EXecutor
+
+A collections of Python APIs to provision operating systems, collect
+and execute [FMF](https://github.com/teemtee/fmf/)-style tests, gather
+and organize their results and generate reports from those results.
+
+The name comes from a (fairly unique to FMF/TMT ecosystem) approach that
+allows provisioning a pool of systems and scheduling tests on them as one would
+on an ad-hoc pool of thread/process workers - once a worker becomes free,
+it receives a test to run.  
+This is in contrast to splitting a large list of N tests onto M workers
+like N/M, which yields significant time penalties due to tests having
+very varies runtimes.
+
+Above all, this project is meant to be a toolbox, not a silver-plate solution.
+Use its Python APIs to build a CLI tool for your specific use case.  
+The CLI tool provided here is just for demonstration / testing, not for serious
+use - we want to avoid huge modular CLIs for Every Possible Scenario. That's
+the job of the Python API. Any CLI should be simple by nature.
+
+---
+
+THIS PROJECT IS HEAVILY WIP, THINGS WILL MOVE AROUND, CHANGE AND OTHERWISE
+BREAK. DO NOT USE IT (for now).
+
+---
+
+## License
+
+Unless specified otherwise, any content within this repository is distributed
+under the GNU GPLv3 license, see the [COPYING.txt](COPYING.txt) file for more.
+
+## Unsorted notes
+
+```
+- this is not tmt, the goal is to make a python toolbox *for* making runcontest
+  style tools easily, not to replace those tools with tmt-style CLI syntax
+
+  - the whole point is to make usecase-targeted easy-to-use tools that don't
+    intimidate users with 1 KB long command line, and runcontest is a nice example
+
+  - TL;DR - use a modular pythonic approach, not a modular CLI like tmt
+
+
+- Orchestrator with
+  - add_provisioner(<class>, max_workers=1)   # will instantiate <class> at most max_workers at a time
+  - algo
+    - for all provisioner classes, spawns classes*max_workers as new Threads
+    - waits for any .reserve() to return
+    - creates a new Thread for minitmt, gives it p.get_ssh() details
+      - minitmt will
+        - establish a SSHConn
+        - install test deps, copy test repo over, prepare socket dir on SUT, etc.
+        - run the test in the background as
+            f=os.open('some/test/log', os.WRONLY); subprocess.Popen(..., stdout=f, stderr=f, stdin=subprocess.DEVNULL)
+        - read/process Unix sock results in the foreground, non-blocking,
+          probably calling some Orchestrator-provided function to store results persistently
+        - regularly check Popen proc status, re-accept UNIX sock connection, etc., etc.
+      - minitmt also has some Thread-independent way to .cancel(), killing the proc, closing SSHConn, etc.
+
+  - while waiting for minitmt Threads to finish, to re-assign existing Provisioner instances
+    to new minitmt Threads, .. Orchestrator uses some logic to select, which TestRun
+    would be ideal to run next
+    - TestRun probably has some "fitness" function that returns some priority number
+      when given a Provisioner instance (?) ...
+    - something from minitmt would also have access to the Provisioner instance
+    - the idea is to allow some logic to set "hey I set up nested VM snapshot on this thing"
+      on the Provisioner instance, and if another /hardening/oscap TestRun finds
+      a Provisioner instance like that, it would return high priority
+    - ...
+    - similar to "fitness" like function, we need some "applicability" function
+      - if TestRun is mixed to RHEL-9 && x86_64, we need it to return True
+        for a Provisioner instance that provides RHEL-9 and x86_64, but False otherwise
+
+- basically Orchestrator has
+  - .add_provisioner()
+  - .run_test()  # called with an exclusively-borrowed Provisioner instance
+    - if Provisioner is_alive()==False after .run_test(), instantiate a new one from the same inst.__class__
+    - if test failed and reruns > 0, try run_test() again (or maybe re-queue the test)
+  - .output_result()  # called by run_test() to persistently log a test result
+  - .applicable()  # return True if a passed TestRun is meant for a passed Platform (Provisioner?)
+    - if no TestRun returns True, the Provisioner is .release()d because we don't need it anymore
+  - .fitness()    # return -inf / 0 / +inf with how much should a passed TestRun run on a Provisioner
+  - MAYBE combine applicable() and fitness() into one function, next_test() ?
+    - given the free Provisioner and a list of TestRuns, select which should run next on the Provisioner
+      - if none is chosen, .release() the Provisioner without replacement, continue
+```

atex-0.1/atex/__init__.py

@@ -0,0 +1,35 @@
+"""
+Ad-hoc Test EXecutor
+
+Some documentation here.
+"""
+
+import importlib as _importlib
+import pkgutil as _pkgutil
+
+__all__ = [
+    info.name for info in _pkgutil.iter_modules(__spec__.submodule_search_locations)
+]
+
+
+def __dir__():
+    return __all__
+
+
+# lazily import submodules
+def __getattr__(attr):
+#    # from mod import *
+#    if attr == '__all__':
+#        print("importing all")
+#        for mod in __all__:
+#            _importlib.import_module(f'.{mod}', __name__)
+#        return __all__
+#    # accessing __all__, __getattr__, etc. directly
+#    elif attr in globals():
+#        print("importing globals")
+#        return globals()[attr]
+    # importing a module known to exist
+    if attr in __all__:
+        return _importlib.import_module(f'.{attr}', __name__)
+    else:
+        raise AttributeError(f'module {__name__} has no attribute {attr}')

atex-0.1/atex/cli/__init__.py

@@ -0,0 +1,83 @@
+r"""
+Command line interface to atex
+
+Submodules (subpackages) of this one must define a module-level dict with
+these keys:
+
+  - help
+    - short oneliner about what the submodule is about (for argparse --help)
+
+  - aliases (optional)
+    - tuple of aliases of the module name, for argument parsing
+
+  - args
+    - function (or other callable) for argument specification/parsing,
+      gets passed one non-kw argument: argparse-style parser
+
+  - main
+    - function (or other callable) that will be called when invoked by the user,
+      gets passed one non-kw argument: argparse-style Namespace
+
+This module-level dict must be named 'CLI_SPEC'.
+"""
+
+import sys
+import importlib
+import pkgutil
+import argparse
+import logging
+
+
+def setup_logging(level):
+    logging.basicConfig(
+        level=level,
+        stream=sys.stderr,
+        format='%(asctime)s %(name)s: %(message)s',
+        datefmt='%Y-%m-%d %H:%M:%S',
+    )
+
+
+def collect_modules():
+    for info in pkgutil.iter_modules(__spec__.submodule_search_locations):
+        mod = importlib.import_module(f'.{info.name}', __name__)
+        if not hasattr(mod, 'CLI_SPEC'):
+            raise ValueError(f"CLI submodule {info.name} does not define CLI_SPEC")
+        yield (info.name, mod.CLI_SPEC)
+
+
+def main():
+    parser = argparse.ArgumentParser()
+
+    log_grp = parser.add_mutually_exclusive_group()
+    log_grp.add_argument(
+        '--debug', '-d', action='store_const', dest='loglevel', const=logging.DEBUG,
+        help="enable extra debugging (logging.DEBUG)",
+    )
+    log_grp.add_argument(
+        '--quiet', '-q', action='store_const', dest='loglevel', const=logging.WARNING,
+        help="be quiet during normal operation (logging.WARNING)",
+    )
+    parser.set_defaults(loglevel=logging.INFO)
+
+    mains = {}
+    subparsers = parser.add_subparsers(dest='_module', metavar='<module>', required=True)
+    for name, spec in collect_modules():
+        aliases = spec['aliases'] if 'aliases' in spec else ()
+        subp = subparsers.add_parser(
+            name,
+            aliases=aliases,
+            help=spec['help'],
+        )
+        spec['args'](subp)
+        mains[name] = spec['main']
+        for alias in aliases:
+            mains[alias] = spec['main']
+
+    args = parser.parse_args()
+
+    setup_logging(args.loglevel)
+
+    try:
+        mains[args._module](args)
+    except KeyboardInterrupt:
+        raise SystemExit() from None

atex-0.1/atex/cli/testingfarm.py

@@ -0,0 +1,171 @@
+import sys
+#from datetime import datetime
+
+from .. import util
+from .. import testingfarm as tf
+
+
+def _get_api(args):
+    api_args = {}
+    if args.url:
+        api_args['url'] = args.url
+    if args.token:
+        api_args['token'] = args.token
+    return tf.TestingFarmAPI(**api_args)
+
+
+def composes(args):
+    api = _get_api(args)
+    comps = api.composes(ranch=args.ranch)
+    comps_list = comps['composes']
+    for comp in comps_list:
+        print(comp['name'])
+
+
+def get_request(args):
+    api = _get_api(args)
+    request = tf.Request(args.request_id, api=api)
+    request.update()
+    print(str(request))
+
+
+def search_requests(args):
+    api = _get_api(args)
+    reply = api.search_requests(
+        state=args.state,
+        mine=not args.all,
+        ranch=args.ranch,
+        created_before=args.before,
+        created_after=args.after,
+    )
+    if not reply:
+        return
+
+    for req in sorted(reply, key=lambda x: x['created']):
+        req_id = req['id']
+        #created_utc = req['created'].partition('.')[0]
+        #created_dt = datetime.fromisoformat(f'{created_utc}+00:00')
+        #created = created_dt.astimezone().isoformat().partition('.')[0]
+        created = req['created'].partition('.')[0]
+
+        envs = []
+        for env in req['environments_requested']:
+            if 'os' in env and env['os'] and 'compose' in env['os']:
+                compose = env['os']['compose']
+                arch = env['arch']
+                if compose and arch:
+                    envs.append(f'{compose}@{arch}')
+        envs_str = ', '.join(envs)
+
+        print(f'{created} {req_id} : {envs_str}')
+        #request = tf.Request(initial_data=req)
+        #print(str(request))
+    #request.update()
+    #print(str(request))
+
+
+def reserve(args):
+    util.info(f"Reserving {args.compose} on {args.arch} for {args.timeout} minutes")
+
+    api = _get_api(args)
+    res = tf.Reserve(
+        compose=args.compose,
+        arch=args.arch,
+        timeout=args.timeout,
+        api=api,
+    )
+    with res as m:
+        util.info(f"Got machine: {m}")
+        util.subprocess_run([
+            'ssh', '-q', '-i', m.ssh_key,
+            '-oStrictHostKeyChecking=no', '-oUserKnownHostsFile=/dev/null',
+            f'{m.user}@{m.host}',
+        ])
+
+
+def watch_pipeline(args):
+    api = _get_api(args)
+    request = tf.Request(id=args.request_id, api=api)
+
+    util.info(f"Waiting for {args.request_id} to be 'running'")
+    try:
+        request.wait_for_state('running')
+    except tf.GoneAwayError:
+        util.info(f"Request {args.request_id} already finished")
+        return
+
+    util.info("Querying pipeline.log")
+    try:
+        for line in tf.PipelineLogStreamer(request):
+            sys.stdout.write(line)
+            sys.stdout.write('\n')
+    except tf.GoneAwayError:
+        util.info(f"Request {args.request_id} finished, exiting")
+
+
+def parse_args(parser):
+    parser.add_argument('--url', help='Testing Farm API URL')
+    parser.add_argument('--token', help='Testing Farm API auth token')
+    cmds = parser.add_subparsers(
+        dest='_cmd', help="TF helper to run", metavar='<cmd>', required=True,
+    )
+
+    cmd = cmds.add_parser(
+        'composes',
+        help="list all composes available on a given ranch",
+    )
+    cmd.add_argument('ranch', nargs='?', help="Testing Farm ranch (autodetected if token)")
+
+    cmd = cmds.add_parser(
+        'get-request', aliases=('gr',),
+        help="retrieve and print JSON of a Testing Farm request",
+    )
+    cmd.add_argument('request_id', help="Testing Farm request UUID")
+
+    cmd = cmds.add_parser(
+        'search-requests', aliases=('sr',),
+        help="return a list of requests matching the criteria",
+    )
+    cmd.add_argument('--state', help="request state (running, etc.)", required=True)
+    cmd.add_argument('--all', help="all requests, not just owned by token", action='store_true')
+    cmd.add_argument('--ranch', help="Testing Farm ranch")
+    cmd.add_argument('--before', help="only requests created before ISO8601")
+    cmd.add_argument('--after', help="only requests created after ISO8601")
+
+    cmd = cmds.add_parser(
+        'reserve',
+        help="reserve a system and ssh into it",
+    )
+    cmd.add_argument('--compose', '-c', help="OS compose to install", required=True)
+    cmd.add_argument('--arch', '-a', help="system HW architecture", default='x86_64')
+    cmd.add_argument('--timeout', '-t', help="pipeline timeout (in minutes)", type=int, default=60)
+    cmd.add_argument('--ssh-key', help="path to a ssh private key file like 'id_rsa'")
+
+    cmd = cmds.add_parser(
+        'watch-pipeline', aliases=('wp',),
+        help="continuously output pipeline.log like 'tail -f'",
+    )
+    cmd.add_argument('request_id', help="Testing Farm request UUID")
+
+
+def main(args):
+    if args._cmd == 'composes':
+        composes(args)
+    elif args._cmd in ('get-request', 'gr'):
+        get_request(args)
+    elif args._cmd in ('search-requests', 'sr'):
+        search_requests(args)
+    elif args._cmd == 'reserve':
+        reserve(args)
+    elif args._cmd in ('watch-pipeline', 'wp'):
+        watch_pipeline(args)
+    else:
+        raise RuntimeError(f"unknown args: {args}")
+
+
+CLI_SPEC = {
+    'aliases': ('tf',),
+    'help': "various utils for Testing Farm",
+    'args': parse_args,
+    'main': main,
+}

atex-0.1/atex/fmf.py

@@ -0,0 +1,168 @@
+import re
+import collections
+from pathlib import Path
+
+# from system-wide sys.path
+import fmf
+
+# name: fmf path to the test as string, ie. /some/test
+# data: dict of the parsed fmf metadata (ie. {'tag': ... , 'environment': ...})
+# dir:  relative pathlib.Path of the test .fmf to repo root, ie. some/test
+#       (may be different from name for "virtual" tests that share the same dir)
+FMFTest = collections.namedtuple('FMFTest', ['name', 'data', 'dir'])
+
+
+class FMFData:
+    """
+    Helper class for reading and querying fmf metadata from the filesystem.
+    """
+    # TODO: usage example ^^^^
+
+    @staticmethod
+    def _listlike(data, key):
+        """
+        Get a piece of fmf metadata as an iterable regardless of whether it was
+        defined as a dict or a list.
+
+        This is needed because many fmf metadata keys can be used either as
+            some_key: 123
+        or as lists via YAML syntax
+            some_key:
+              - 123
+              - 456
+        and, for simplicity, we want to always deal with lists (iterables).
+        """
+        if value := data.get(key):
+            return value if isinstance(value, list) else (value,)
+        else:
+            return ()
+
+    def __init__(self, fmf_tree, plan_name, context=None):
+        """
+        'fmf_tree' is filesystem path somewhere inside fmf metadata tree,
+        or a root fmf.Tree instance.
+
+        'plan_name' is fmf identifier (like /some/thing) of a tmt plan
+        to use for discovering tests.
+
+        'context' is a dict like {'distro': 'rhel-9.6'} used for filtering
+        discovered tests.
+        """
+        self.prepare_pkgs = []
+        self.prepare_scripts = []
+        self.tests = []
+
+        tree = fmf_tree.copy() if isinstance(fmf_tree, fmf.Tree) else fmf.Tree(fmf_tree)
+        ctx = fmf.Context(**context) if context else fmf.Context()
+        tree.adjust(context=ctx)
+
+        self.fmf_root = tree.root
+
+        # lookup the plan first
+        plan = tree.find(plan_name)
+        if not plan:
+            raise ValueError(f"plan {plan_name} not found in {tree.root}")
+        if 'test' in plan.data:
+            raise ValueError(f"plan {plan_name} appears to be a test")
+
+        # gather all prepare scripts / packages
+        #
+        # prepare:
+        #   - how: install
+        #     package:
+        #       - some-rpm-name
+        #   - how: shell
+        #     script:
+        #       - some-command
+        for entry in self._listlike(plan.data, 'prepare'):
+            if 'how' not in entry:
+                continue
+            if entry['how'] == 'install':
+                self.prepare_pkgs += self._listlike(entry, 'package')
+            elif entry['how'] == 'shell':
+                self.prepare_scripts += self._listlike(entry, 'script')
+
+        # gather all tests selected by the plan
+        #
+        # discover:
+        #   - how: fmf
+        #     filter:
+        #       - tag:some_tag
+        #     test:
+        #       - some-test-regex
+        #     exclude:
+        #       - some-test-regex
+        if 'discover' in plan.data:
+            discover = plan.data['discover']
+            if not isinstance(discover, list):
+                discover = (discover,)
+
+            for entry in discover:
+                if entry.get('how') != 'fmf':
+                    continue
+
+                filtering = {}
+                for meta_name in ('filter', 'test', 'exclude'):
+                    if value := self._listlike(entry, meta_name):
+                        filtering[meta_name] = value
+
+                children = tree.prune(
+                    names=filtering.get('test'),
+                    filters=filtering.get('filter'),
+                )
+                for child in children:
+                    # excludes not supported by .prune(), we have to do it here
+                    excludes = filtering.get('exclude')
+                    if excludes and any(re.match(x, child.name) for x in excludes):
+                        continue
+                    # only enabled tests
+                    if 'enabled' in child.data and not child.data['enabled']:
+                        continue
+                    # no manual tests
+                    if child.data.get('manual'):
+                        continue
+                    # after adjusting above, any adjusts are useless, free some space
+                    if 'adjust' in child.data:
+                        del child.data['adjust']
+                    # ie. ['/abs/path/to/some.fmf', '/abs/path/to/some/node.fmf']
+                    source_dir = Path(child.sources[-1]).parent.relative_to(self.fmf_root)
+                    self.tests.append(
+                        FMFTest(name=child.name, data=child.data, dir=source_dir),
+                    )
+
+
+# Some extra notes for fmf.prune() arguments:
+#
+# Set 'names' to filter by a list of fmf node names, ie.
+#     ['/some/test', '/another/test']
+#
+# Set 'filters' to filter by a list of fmf-style filter expressions, see
+#     https://fmf.readthedocs.io/en/stable/modules.html#fmf.filter
+#
+# Set 'conditions' to filter by a list of python expressions whose namespace
+# locals() are set up to be a dictionary of the tree. When any of the
+# expressions returns True, the tree is returned, ie.
+#     ['environment["FOO"] == "BAR"']
+#     ['"enabled" not in locals() or enabled']
+# Note that KeyError is silently ignored and treated as False.
+#
+# Set 'context' to a dictionary to post-process the tree metadata with
+# adjust expressions (that may be present in a tree) using the specified
+# context. Any other filters are applied afterwards to allow modification
+# of tree metadata by the adjust expressions. Ie.
+#     {'distro': 'rhel-9.6.0', 'arch': 'x86_64'}
+
+Platform = collections.namedtuple('Platform', ['distro', 'arch'])
+
+
+def combine_platforms(fmf_path, plan_name, platforms):
+    # TODO: document
+    fmf_datas = {}
+    tree = fmf.Tree(fmf_path)
+    for platform in platforms:
+        context = {'distro': platform.distro, 'arch': platform.arch}
+        fmf_datas[platform] = FMFData(tree, plan_name, context=context)
+    return fmf_datas
+
+# TODO: in Orchestrator, when a Provisioner becomes free, have it pick a test
+#       from the appropriate tests[platform] per the Provisioner's platform