atex 0.7__py3-none-any.whl → 0.9__py3-none-any.whl

atex-0.9.dist-info/METADATA

@@ -0,0 +1,178 @@
+Metadata-Version: 2.4
+Name: atex
+Version: 0.9
+Summary: Ad-hoc Test EXecutor
+Project-URL: Homepage, https://github.com/RHSecurityCompliance/atex
+License-Expression: GPL-3.0-or-later
+License-File: COPYING.txt
+Classifier: Operating System :: POSIX :: Linux
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Software Development :: Testing
+Requires-Python: >=3.11
+Requires-Dist: fmf>=1.6
+Requires-Dist: pyyaml
+Requires-Dist: urllib3<3,>=2
+Description-Content-Type: text/markdown
+
+# ATEX = Ad-hoc Test EXecutor
+
+A collections of Python APIs to provision operating systems, collect
+and execute [FMF](https://github.com/teemtee/fmf/)-style tests, gather
+and organize their results and generate reports from those results.
+
+The name comes from a (fairly unique to FMF/TMT ecosystem) approach that
+allows provisioning a pool of systems and scheduling tests on them as one would
+on an ad-hoc pool of thread/process workers - once a worker becomes free,
+it receives a test to run.  
+This is in contrast to splitting a large list of N tests onto M workers
+like N/M, which yields significant time penalties due to tests having
+very varies runtimes.
+
+Above all, this project is meant to be a toolbox, not a silver-plate solution.
+Use its Python APIs to build a CLI tool for your specific use case.  
+The CLI tool provided here is just for demonstration / testing, not for serious
+use - we want to avoid huge modular CLIs for Every Possible Scenario. That's
+the job of the Python API. Any CLI should be simple by nature.
+
+---
+
+THIS PROJECT IS HEAVILY WIP, THINGS WILL MOVE AROUND, CHANGE AND OTHERWISE
+BREAK. DO NOT USE IT (for now).
+
+---
+
+## License
+
+Unless specified otherwise, any content within this repository is distributed
+under the GNU GPLv3 license, see the [COPYING.txt](COPYING.txt) file for more.
+
+## Testing this project
+
+There are some limited sanity tests provided via `pytest`, although:
+
+* Some require additional variables (ie. Testing Farm) and will ERROR
+  without them.
+* Some take a long time (ie. Testing Farm) due to system provisioning
+  taking a long time, so install `pytest-xdist` and run with a large `-n`.
+
+Currently, the recommended approach is to split the execution:
+
+```
+# synchronously, because podman CLI has concurrency issues
+pytest tests/provision/test_podman.py
+
+# in parallel, because provisioning takes a long time
+export TESTING_FARM_API_TOKEN=...
+export TESTING_FARM_COMPOSE=...
+pytest -n 20 tests/provision/test_podman.py
+
+# fast enough for synchronous execution
+pytest tests/fmf
+```
+
+## Parallelism and cleanup
+
+There are effectively 3 methods of running things in parallel in Python:
+
+- `threading.Thread` (and related `concurrent.futures` classes)
+- `multiprocessing.Process` (and related `concurrent.futures` classes)
+- `asyncio`
+
+and there is no clear winner (in terms of cleanup on `SIGTERM` or Ctrl-C):
+
+- `Thread` has signal handlers only in the main thread and is unable to
+  interrupt any running threads without super ugly workarounds like `sleep(1)`
+  in every thread, checking some "pls exit" variable
+- `Process` is too heavyweight and makes sharing native Python objects hard,
+  but it does handle signals in each process individually
+- `asyncio` handles interrupting perfectly (every `try`/`except`/`finally`
+  completes just fine, `KeyboardInterrupt` is raised in every async context),
+  but async python is still (3.14) too weird and unsupported
+  - `asyncio` effectively re-implements `subprocess` with a slightly different
+    API, same with `asyncio.Transport` and derivatives reimplementing `socket`
+  - 3rd party libraries like `requests` or `urllib3` don't support it, one needs
+    to resort to spawning these in separate threads anyway
+  - same with `os.*` functions and syscalls
+  - every thing exposed via API needs to have 2 copies - async and non-async,
+    making it unbearable
+  - other stdlib bugs, ie. "large" reads returning BlockingIOError sometimes
+
+The approach chosen by this project was to use `threading.Thread`, and
+implement thread safety for classes and their functions that need it.  
+For example:
+
+```python
+class MachineReserver:
+    def __init__(self):
+        self.lock = threading.RLock()
+        self.job = None
+        self.proc = None
+
+    def reserve(self, ...):
+        try:
+            ...
+            job = schedule_new_job_on_external_service()
+            with self.lock:
+                self.job = job
+            ...
+            while not reserved(self.job):
+                time.sleep(60)
+            ...
+            with self.lock:
+                self.proc = subprocess.Popen(["ssh", f"{user}@{host}", ...)
+            ...
+            return machine
+        except Exception:
+            self.abort()
+            raise
+
+    def abort(self):
+        with self.lock:
+            if self.job:
+                cancel_external_service(self.job)
+                self.job = None
+            if self.proc:
+                self.proc.kill()
+                self.proc = None
+```
+
+Here, it is expected for `.reserve()` to be called in a long-running thread that
+provisions a new machine on some external service, waits for it to be installed
+and reserved, connects an ssh session to it and returns it back.
+
+But equally, `.abort()` can be called from an external thread and clean up any
+non-pythonic resources (external jobs, processes, temporary files, etc.) at
+which point **we don't care what happens to .reserve()**, it will probably fail
+with some exception, but doesn't do any harm.
+
+Here is where `daemon=True` threads come in handy - we can simply call `.abort()`
+from a `KeyboardInterrupt` (or `SIGTERM`) handle in the main thread, and just
+exit, automatically killing any leftover threads that are uselessly sleeping.  
+(Realistically, we might want to spawn new threads to run many `.abort()`s in
+parallel, but the main thread can wait for those just fine.)
+
+It is not perfect, but it's probably the best Python can do.
+
+Note that races can still occur between a resource being reserved and written
+to `self.*` for `.abort()` to free, so resource de-allocation is not 100%
+guaranteed, but single-threaded interrupting has the same issue.  
+Do have fallbacks (ie. max reserve times on the external service).
+
+Also note that `.reserve()` and `.abort()` could be also called by a context
+manager as `__enter__` and `__exit__`, ie. by a non-threaded caller (running
+everything in the main thread).
+
+
+## Unsorted notes
+
+TODO: codestyle from contest
+
+```
+- this is not tmt, the goal is to make a python toolbox *for* making runcontest
+  style tools easily, not to replace those tools with tmt-style CLI syntax
+
+  - the whole point is to make usecase-targeted easy-to-use tools that don't
+    intimidate users with 1 KB long command line, and runcontest is a nice example
+
+  - TL;DR - use a modular pythonic approach, not a gluetool-style long CLI
+```

atex-0.9.dist-info/RECORD

@@ -0,0 +1,43 @@
+atex/__init__.py,sha256=LdX67gprtHYeAkjLhFPKzpc7ECv2rHxUbHKDGbGXO1c,517
+atex/fmf.py,sha256=gkJXIaRO7_KvwJR-V6Tc1NVn4a9Hq7hoBLQLhxYIdbg,8834
+atex/cli/__init__.py,sha256=erHv68SsybRbdgJ60013y9jVqY1ec-cb9T9ThPCJ_HY,2408
+atex/cli/fmf.py,sha256=HfbTgFbCwK4Nuyq6vtGutcq_4-4kj-tmoqzXUn3AYtY,3573
+atex/cli/libvirt.py,sha256=vGcan9u0Nor_DaEjgOj3Yzjtlq4WNVLr5oYMb8zFb6M,3743
+atex/cli/testingfarm.py,sha256=DHMupwU3AjbkwkzvzdhlxtsuBnxVaba5i8T8g9sKRug,7569
+atex/connection/__init__.py,sha256=dj8ZBcEspom7Z_UjecfLGBRNvLZ3dyGR9q19i_B4xpY,3880
+atex/connection/podman.py,sha256=JUsi4jOkNFGsEHCqUQF003R2yRfJPsMnUSBEZM3H3Kk,1953
+atex/connection/ssh.py,sha256=c8v01mj0pf6VU8z2cLLCvPGPmGUZp8wQsTG441k3Qy4,13674
+atex/executor/__init__.py,sha256=XCfhi7QDELjey7N1uzhMjc46Kp1Jsd5bOCf52I27SCE,85
+atex/executor/duration.py,sha256=x06sItKOZi6XA8KszQwZGpIb1Z_L-HWqIwZKo2SDo0s,1759
+atex/executor/executor.py,sha256=OcKBXQ77OktmzPWAbdCb81Yj44Jr4SEkwW_CxdEhFP4,15674
+atex/executor/reporter.py,sha256=MceFmHFt0bTEClBZbRI1WnFbfMhR0e1noOzcu7gjKuQ,3403
+atex/executor/scripts.py,sha256=6xqP-m_9UCVb3z0tCWIMczimz_cTeCIJucSp0AGYsRw,5567
+atex/executor/testcontrol.py,sha256=O4V4jRx7lZv-El1i9jkPha-aWzjq_8m6loK7sNGVD14,12695
+atex/orchestrator/__init__.py,sha256=GSHtWoZjJrLiatjUHsMAkYB8VebSQD9ApQw0ymCrGTQ,212
+atex/orchestrator/aggregator.py,sha256=naypN8T42iPau_4eLeXak-INR1_vsYhP_QSp3xLM_FA,3837
+atex/orchestrator/orchestrator.py,sha256=qAwka7MqpAWA6mLgsMl1m1HpQP2mERcaAASNvJwgfTM,14570
+atex/provision/__init__.py,sha256=Pj_JnDaJA8DzKRGmLcc2JiwPsuQc2TYlE-hDR56SwTY,4083
+atex/provision/libvirt/VM_PROVISION,sha256=7pkZ-ozgTyK4qNGC-E-HUznr4IhbosWSASbB72Gknl8,2664
+atex/provision/libvirt/__init__.py,sha256=pKG5IpZSC2IHs5wL2ecQx_fd9AzAXEbZmDzA7RyZsfM,119
+atex/provision/libvirt/libvirt.py,sha256=7wux-zhQoNODJoPB4iMqPlGEJe7rEEZeLSOtMU3TB0M,18160
+atex/provision/libvirt/locking.py,sha256=e9zbtR0uzJZBAzucfhs9VQaDCx-jp0pWJi1pb7JPK4w,5630
+atex/provision/libvirt/setup-libvirt.sh,sha256=oCMy9SCnbC_QuAzO2sFwvB5ui1kMQ6uviHsgdXyoFXc,2428
+atex/provision/podman/__init__.py,sha256=V9miIDQV-CyDVcbf2-1qtDbXDhhZzJza5oXas-JxI8o,66
+atex/provision/podman/podman.py,sha256=FJK4uLDuHBj8_3Wlg4a-4JgGIRGyLX4v7UVcxEorHbA,9578
+atex/provision/testingfarm/__init__.py,sha256=kZncgLGdRCR4FMaRQr2GTwJ8vjlA-24ri8JO2ueZJuw,113
+atex/provision/testingfarm/api.py,sha256=9xeoIN28KumJyMUEUoyEUZcIxVH9G34Wecyy3bp2Cmg,21612
+atex/provision/testingfarm/testingfarm.py,sha256=-Q33FAretSGPWdmNJUywJZVfXYGtsAfLP5uqRZTGfQQ,8631
+atex/util/__init__.py,sha256=cWHFbtQ4mDlKe6lXyPDWRmWJOTcHDGfVuW_-GYa8hB0,1473
+atex/util/dedent.py,sha256=SEuJMtLzqz3dQ7g7qyZzEJ9VYynVlk52tQCJY-FveXo,603
+atex/util/libvirt.py,sha256=kDZmT6xLYEZkQNLZY98gJ2M48DDWXxHF8rQY9PnjB3U,660
+atex/util/log.py,sha256=70f6YiF2RwNUfE5BXZMSQonzfI_uxQoD7_S4bRD_Btw,2466
+atex/util/named_mapping.py,sha256=UBMe9TetjV-DGPhjYjJ42YtC40FVPKAAEROXl9MA5fo,4700
+atex/util/path.py,sha256=x-kXqiWCVodfZWbEwtC5A8LFvutpDIPYv2m0boZSlXU,504
+atex/util/ssh_keygen.py,sha256=9yuSl2yBV7pG3Qfsf9tossVC00nbIUrAeLdbwTykpjk,384
+atex/util/subprocess.py,sha256=IQT9QHe2kMaaO_XPSry-DwObYstGsq6_QdwdbhYDjko,1826
+atex/util/threads.py,sha256=46-5nV-qJqi1YZ4qEshmZXGUxr8j9_9xT9eEpkjRr5I,3355
+atex-0.9.dist-info/METADATA,sha256=iLGFHVxeK9KWrPaRdL4hkbNkb-OgJkeohHhxYz90tAI,6857
+atex-0.9.dist-info/WHEEL,sha256=qtCwoSJWgHk21S1Kb4ihdzI2rlJ1ZKaIurTj_ngOhyQ,87
+atex-0.9.dist-info/entry_points.txt,sha256=pLqJdcfeyQTgup2h6dWb6SvkHhtOl-W5Eg9zV8moK0o,39
+atex-0.9.dist-info/licenses/COPYING.txt,sha256=oEuj51jdmbXcCUy7pZ-KE0BNcJTR1okudRp5zQ0yWnU,670
+atex-0.9.dist-info/RECORD,,

atex/cli/minitmt.py

@@ -1,175 +0,0 @@
-import sys
-#import re
-import pprint
-#import subprocess
-from pathlib import Path
-
-from .. import connection, provision, minitmt
-from ..orchestrator import aggregator
-
-
-def _fatal(msg):
-    print(msg, file=sys.stderr)
-    sys.exit(1)
-
-
-def _get_context(args):
-    context = {}
-    if args.context:
-        for c in args.context:
-            key, value = c.split("=", 1)
-            context[key] = value
-    return context or None
-
-
-def discover(args):
-    result = minitmt.fmf.FMFTests(args.root, args.plan, context=_get_context(args))
-    for name in result.tests:
-        print(name)
-
-
-def show(args):
-    result = minitmt.fmf.FMFTests(args.root, args.plan, context=_get_context(args))
-    if tests := list(result.match(args.test)):
-        for test in tests:
-            print(f"\n--- {test.name} ---")
-            pprint.pprint(test.data)
-    else:
-        _fatal(f"Not reachable via {args.plan} discovery: {args.test}")
-
-
-def execute(args):
-    # remote system connection
-    ssh_keypath = Path(args.ssh_identity)
-    if not ssh_keypath.exists():
-        _fatal(f"SSH Identity {args.ssh_identity} does not exist")
-    ssh_options = {
-        "User": args.user,
-        "Hostname": args.host,
-        "IdentityFile": ssh_keypath,
-    }
-    env = dict(x.split("=",1) for x in args.env)
-
-    # dummy Remote that just wraps the connection
-    class DummyRemote(provision.Remote, connection.ssh.ManagedSSHConn):
-        @staticmethod
-        def release():
-            return
-
-        @staticmethod
-        def alive():
-            return True
-
-    # result aggregation
-    with aggregator.CSVAggregator(args.results_csv, args.results_dir) as csv_aggregator:
-        platform_aggregator = csv_aggregator.for_platform(args.platform)
-
-        # tests discovery and selection
-        result = minitmt.fmf.FMFTests(args.root, args.plan, context=_get_context(args))
-        if args.test:
-            tests = list(result.match(args.test))
-            if not tests:
-                _fatal(f"Not reachable via plan {args.plan} discovery: {args.test}")
-        else:
-            tests = list(result.as_fmftests())
-            if not tests:
-                _fatal(f"No tests found for plan {args.plan}")
-
-        # test run
-        with DummyRemote(ssh_options) as remote:
-            executor = minitmt.executor.Executor(remote, platform_aggregator, env=env)
-            executor.upload_tests(args.root)
-            executor.setup_plan(result)
-            for test in tests:
-                executor.run_test(test)
-
-
-def setup_script(args):
-    result = minitmt.fmf.FMFTests(args.root, args.plan, context=_get_context(args))
-    try:
-        test = result.as_fmftest(args.test)
-    except KeyError:
-        print(f"Not reachable via {args.plan} discovery: {args.test}")
-        raise SystemExit(1) from None
-    output = minitmt.scripts.test_setup(
-        test=test,
-        tests_dir=args.remote_root,
-        debug=args.script_debug,
-    )
-    print(output, end="")
-
-
-def parse_args(parser):
-    parser.add_argument("--root", help="path to directory with fmf tests", default=".")
-    parser.add_argument("--context", "-c", help="tmt style key=value context", action="append")
-    cmds = parser.add_subparsers(
-        dest="_cmd", help="minitmt feature", metavar="<cmd>", required=True,
-    )
-
-    cmd = cmds.add_parser(
-        "discover", aliases=("di",),
-        help="list tests, post-processed by tmt plans",
-    )
-    cmd.add_argument("plan", help="tmt plan to use for discovery")
-
-    cmd = cmds.add_parser(
-        "show",
-        help="show fmf data of a test",
-    )
-    cmd.add_argument("plan", help="tmt plan to use for discovery")
-    cmd.add_argument("test", help="fmf style test regex")
-
-    cmd = cmds.add_parser(
-        "execute", aliases=("ex",),
-        help="run a plan (or test) on a remote system",
-    )
-    #grp = cmd.add_mutually_exclusive_group()
-    #grp.add_argument("--test", "-t", help="fmf style test regex")
-    #grp.add_argument("--plan", "-p", help="tmt plan name (path) inside metadata root")
-    cmd.add_argument("--env", "-e", help="environment to pass to prepare/test", action="append")
-    cmd.add_argument("--test", "-t", help="fmf style test regex")
-    cmd.add_argument(
-        "--plan", "-p", help="tmt plan name (path) inside metadata root", required=True,
-    )
-    cmd.add_argument("--platform", help="platform name, ie. rhel9@x86_64", required=True)
-    cmd.add_argument("--user", help="ssh user to connect via", required=True)
-    cmd.add_argument("--host", help="ssh host to connect to", required=True)
-    cmd.add_argument(
-        "--ssh-identity", help="path to a ssh keyfile for login", required=True,
-    )
-    cmd.add_argument(
-        "--results-csv", help="path to would-be-created .csv.gz results", required=True,
-    )
-    cmd.add_argument(
-        "--results-dir", help="path to would-be-created dir for uploaded files", required=True,
-    )
-
-    cmd = cmds.add_parser(
-        "setup-script",
-        help="generate a script prepping tests for run",
-    )
-    cmd.add_argument("--remote-root", help="path to tests repo on the remote", required=True)
-    cmd.add_argument("--script-debug", help="do 'set -x' in the script", action="store_true")
-    cmd.add_argument("plan", help="tmt plan to use for discovery")
-    cmd.add_argument("test", help="full fmf test name (not regex)")
-
-
-def main(args):
-    if args._cmd in ("discover", "di"):
-        discover(args)
-    elif args._cmd == "show":
-        show(args)
-    elif args._cmd in ("execute", "ex"):
-        execute(args)
-    elif args._cmd == "setup-script":
-        setup_script(args)
-    else:
-        raise RuntimeError(f"unknown args: {args}")
-
-
-CLI_SPEC = {
-    "aliases": ("tmt",),
-    "help": "simple test executor using atex.minitmt",
-    "args": parse_args,
-    "main": main,
-}

atex/minitmt/__init__.py

@@ -1,23 +0,0 @@
-"""
-TODO Minitmt documentation - reference README, etc.
-"""
-
-import importlib as _importlib
-import pkgutil as _pkgutil
-
-__all__ = [
-    info.name for info in _pkgutil.iter_modules(__spec__.submodule_search_locations)
-]
-
-
-def __dir__():
-    return __all__
-
-
-# lazily import submodules
-def __getattr__(attr):
-    # importing a module known to exist
-    if attr in __all__:
-        return _importlib.import_module(f".{attr}", __name__)
-    else:
-        raise AttributeError(f"module '{__name__}' has no attribute '{attr}'")