cleepcli 1.45.0__tar.gz → 1.46.0__tar.gz

{cleepcli-1.45.0/cleepcli.egg-info → cleepcli-1.46.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: cleepcli
-Version: 1.45.0
+Version: 1.46.0
 Summary: Cleep-cli helps developers to build great Cleep applications from command line.
 Home-page: https://github.com/CleepDevice/cleep-cli
 Author: Tanguy Bonneau
@@ -42,7 +42,7 @@ This utility helps developers to build Cleep applications providing some useful
     - `coreget` to clone or pull official Cleep repository.
     - `coredocs` to generate core documentation in HTML format.
     - `coresync` to synchronize sources from repository to execution folders.
-    - `coretests` to perform core tests suite.
+    - `coretests` to perform core unit tests suite (`--functional` for real OS binary tests on device / OS migration).
     - `coretestscov` to display last core tests code coverage report.
     - `cpuprof` to run CPU profiler on application execution instance.
     - `memprof` to run memory profiler on application execution instance.
@@ -52,7 +52,7 @@ This utility helps developers to build Cleep applications providing some useful
     - `moddelete` to delete installed files of a module.
     - `moddocs` to generate module documentation.
     - `modsync` to synchronize sources from module folder to execution folders.
-    - `modtests` to execute module tests.
+    - `modtests` to execute module unit tests (`--functional` for real OS/device tests).
     - `modtestscov` to display last module tests code coverage report.
     - `modbuild` to create application package (.zip)
     - `modcheckback` to check app backend (python files structure and content)
@@ -142,7 +142,7 @@ The [watchdog](https://pypi.org/project/watchdog/) python library is used to det
 
 ## CI (GitHub Actions)
 
-A tag `vX.Y.Z` (or a manual **workflow_dispatch**) builds a **py3** wheel + sdist on `ubuntu-latest` (Python 3.9) via `python -m build` and uploads them to PyPI.
+A tag `vX.Y.Z` (or a manual **workflow_dispatch**) builds a **py3** wheel + sdist on `ubuntu-latest` (Python 3.12) via `python -m build` and uploads them to PyPI.
 
 GitHub **environment** `actions` (Settings → Environments), secret:
 

{cleepcli-1.45.0 → cleepcli-1.46.0}/README.md

@@ -6,7 +6,7 @@ This utility helps developers to build Cleep applications providing some useful
     - `coreget` to clone or pull official Cleep repository.
     - `coredocs` to generate core documentation in HTML format.
     - `coresync` to synchronize sources from repository to execution folders.
-    - `coretests` to perform core tests suite.
+    - `coretests` to perform core unit tests suite (`--functional` for real OS binary tests on device / OS migration).
     - `coretestscov` to display last core tests code coverage report.
     - `cpuprof` to run CPU profiler on application execution instance.
     - `memprof` to run memory profiler on application execution instance.
@@ -16,7 +16,7 @@ This utility helps developers to build Cleep applications providing some useful
     - `moddelete` to delete installed files of a module.
     - `moddocs` to generate module documentation.
     - `modsync` to synchronize sources from module folder to execution folders.
-    - `modtests` to execute module tests.
+    - `modtests` to execute module unit tests (`--functional` for real OS/device tests).
     - `modtestscov` to display last module tests code coverage report.
     - `modbuild` to create application package (.zip)
     - `modcheckback` to check app backend (python files structure and content)
@@ -106,7 +106,7 @@ The [watchdog](https://pypi.org/project/watchdog/) python library is used to det
 
 ## CI (GitHub Actions)
 
-A tag `vX.Y.Z` (or a manual **workflow_dispatch**) builds a **py3** wheel + sdist on `ubuntu-latest` (Python 3.9) via `python -m build` and uploads them to PyPI.
+A tag `vX.Y.Z` (or a manual **workflow_dispatch**) builds a **py3** wheel + sdist on `ubuntu-latest` (Python 3.12) via `python -m build` and uploads them to PyPI.
 
 GitHub **environment** `actions` (Settings → Environments), secret:
 

{cleepcli-1.45.0 → cleepcli-1.46.0}/bin/ccli

@@ -114,12 +114,13 @@ def coresync():
 @click.option('--xml', is_flag=True, default=False, help='Use xml reporting instead of default one (report).')
 @click.option('--quiet', is_flag=True, default=True, help='Display or not coverage warnings.')
 @click.option('-p', '--pattern', default=None, help='Run tests on specified file pattern only')
-def coretests(coverage, output, xml, quiet, pattern):
+@click.option('--functional', is_flag=True, help='Run functional tests (real OS binaries) instead of unit tests.')
+def coretests(coverage, output, xml, quiet, pattern, functional):
     """
     Execute core tests
     """
     m = Test()
-    res = m.core_tests(coverage, output, xml, quiet, pattern)
+    res = m.core_tests(coverage, output, xml, quiet, pattern, functional=functional)
 
     if not res:
         sys.exit(1)
@@ -385,7 +386,8 @@ def test():
 @click.option('--coverage', is_flag=True, help='Display coverage report.')
 @click.option('--copyto', help='Copy .coverage file to specified dir.')
 @click.option('-p', '--pattern', help='Execute tests matching pattern.')
-def modtests(ctx, module, coverage, copyto, pattern):
+@click.option('--functional', is_flag=True, help='Run functional tests (real OS/device) instead of unit tests.')
+def modtests(ctx, module, coverage, copyto, pattern, functional):
     """
     Execute module tests
     """
@@ -393,7 +395,7 @@ def modtests(ctx, module, coverage, copyto, pattern):
     ctx.invoke(modsync, module=module)
 
     m = Test()
-    res = m.module_tests(module, coverage, copyto, pattern)
+    res = m.module_tests(module, coverage, copyto, pattern, functional=functional)
 
     if not res:
         sys.exit(1)

{cleepcli-1.45.0 → cleepcli-1.46.0}/bin/cleep-cli

@@ -114,12 +114,13 @@ def coresync():
 @click.option('--xml', is_flag=True, default=False, help='Use xml reporting instead of default one (report).')
 @click.option('--quiet', is_flag=True, default=True, help='Display or not coverage warnings.')
 @click.option('-p', '--pattern', default=None, help='Run tests on specified file pattern only')
-def coretests(coverage, output, xml, quiet, pattern):
+@click.option('--functional', is_flag=True, help='Run functional tests (real OS binaries) instead of unit tests.')
+def coretests(coverage, output, xml, quiet, pattern, functional):
     """
     Execute core tests
     """
     m = Test()
-    res = m.core_tests(coverage, output, xml, quiet, pattern)
+    res = m.core_tests(coverage, output, xml, quiet, pattern, functional=functional)
 
     if not res:
         sys.exit(1)
@@ -385,7 +386,8 @@ def test():
 @click.option('--coverage', is_flag=True, help='Display coverage report.')
 @click.option('--copyto', help='Copy .coverage file to specified dir.')
 @click.option('-p', '--pattern', help='Execute tests matching pattern.')
-def modtests(ctx, module, coverage, copyto, pattern):
+@click.option('--functional', is_flag=True, help='Run functional tests (real OS/device) instead of unit tests.')
+def modtests(ctx, module, coverage, copyto, pattern, functional):
     """
     Execute module tests
     """
@@ -393,7 +395,7 @@ def modtests(ctx, module, coverage, copyto, pattern):
     ctx.invoke(modsync, module=module)
 
     m = Test()
-    res = m.module_tests(module, coverage, copyto, pattern)
+    res = m.module_tests(module, coverage, copyto, pattern, functional=functional)
 
     if not res:
         sys.exit(1)

{cleepcli-1.45.0 → cleepcli-1.46.0}/cleepcli/console.py

@@ -4,6 +4,7 @@
 import sys
 import subprocess
 import time
+import atexit
 from threading import Timer, Thread
 try: # pragma: no cover
     from Queue import Queue, Empty
@@ -58,12 +59,40 @@ class EndlessConsole(Thread):
         self.__stderr_queue = Queue()
         self.__stdout_thread = None
         self.__stderr_thread = None
+        self.__proc = None
+        self.__pid = None
+        # Child runs in its own session (setsid). Reap it if the CLI exits via
+        # Ctrl-C before kill() — otherwise coverage/pytest can orphan and OOM a Pi.
+        atexit.register(self.__reap_child)
 
     def __del__(self):
         """
         Destructor
         """
         self.__stop()
+        self.__reap_child()
+
+    def __reap_child(self):
+        """Kill leftover process group if the subprocess is still running."""
+        pid = self.__pid
+        proc = self.__proc
+        if not pid or pid == 1:
+            return
+        try:
+            if proc is not None and proc.poll() is None:
+                self.logger.debug('Reaping leftover process group pid=%s', pid)
+                if ON_POSIX:
+                    os.killpg(os.getpgid(pid), signal.SIGKILL)
+                else:  # pragma: no cover
+                    proc.kill()
+                self.killed = True
+        except Exception:  # pragma: no cover
+            pass
+        finally:
+            try:
+                atexit.unregister(self.__reap_child)
+            except Exception:  # pragma: no cover
+                pass
 
     def __enqueue_output(self, output, queue):
         for line in iter(output.readline, b''):
@@ -103,6 +132,7 @@ class EndlessConsole(Thread):
         self.logger.debug('Process killed manually')
         self.killed = True
         self.__stop()
+        self.__reap_child()
 
     def __send_stds(self):
         """
@@ -156,7 +186,9 @@ class EndlessConsole(Thread):
             close_fds=True,
             preexec_fn=os.setsid
         )
-        pid = proc.pid
+        self.__proc = proc
+        self.__pid = proc.pid
+        pid = self.__pid
         self.logger.trace('PID=%d' % pid)
 
         if self.callback:
@@ -214,14 +246,19 @@ class EndlessConsole(Thread):
         except Exception: # pragma: no cover
             pass
 
-        # make sure process (and child processes) is really killed
-        if self.killed and pid != 1:
+        # Always reap the process group if still alive. The child is started in its
+        # own session (setsid): if the CLI exits on Ctrl-C without kill(), pytest/
+        # coverage would otherwise keep running and can OOM a Raspberry Pi.
+        if pid != 1:
             try:
-                if ON_POSIX:
-                    os.killpg(os.getpgid(pid), signal.SIGKILL)
-                else: # pragma: no cover
-                    proc.kill()
-            except Exception: # pragma: no cover
+                if proc.poll() is None:
+                    self.logger.debug('Terminating leftover process group pid=%s', pid)
+                    if ON_POSIX:
+                        os.killpg(os.getpgid(pid), signal.SIGKILL)
+                    else:  # pragma: no cover
+                        proc.kill()
+                    self.killed = True
+            except Exception:  # pragma: no cover
                 pass
 
         # process is over

{cleepcli-1.45.0 → cleepcli-1.46.0}/cleepcli/module.py

@@ -488,6 +488,8 @@ pytest_plugins = ('cleep.libs.tests.fixtures',)
 pythonpath = .
 testpaths = tests
 python_files = test_*.py
+markers =
+    functional: real OS/device tests (CLEEP_FUNCTIONAL_TESTS / modtests --functional)
 """
     DOCS_CONF_PY = """# -*- coding: utf-8 -*-
 import os, sys

{cleepcli-1.45.0 → cleepcli-1.46.0}/cleepcli/test.py

@@ -116,11 +116,11 @@ class Test():
             self.__stderr.append(stderr)
 
     def __console_callback(self, stdout, stderr):
-        if stdout and stdout.find('----------------------------------------------------------------------') < 0:
+        # Forward pytest/unittest output as-is so failure tracebacks stay readable in CI
+        if stdout:
             self.logger.info("  |  " + stdout)
-        if stderr and stderr.find('----------------------------------------------------------------------') < 0:
-            self.logger.info("  | " + stderr)
         if stderr:
+            self.logger.info("  | " + stderr)
             self.__stderr.append(stderr)
 
     def __console_end_callback(self, return_code, killed):
@@ -243,7 +243,7 @@ cd "%(path)s"
 
         return res
 
-    def module_tests(self, module_name, display_coverage=False, copy_to=None, pattern=None):
+    def module_tests(self, module_name, display_coverage=False, copy_to=None, pattern=None, functional=False):
         """
         Execute module unit tests and display process output on stdout
 
@@ -252,6 +252,7 @@ cd "%(path)s"
             display_coverage (bool): display coverage report (default False)
             copy_to (str): copy coverage file to specified path
             pattern (str): run testcase that match pattern (no coverage)
+            functional (bool): run functional tests (real OS / device) instead of unit tests
 
         Returns:
             bool: True if process succeed, False otherwise
@@ -267,11 +268,16 @@ cd "%(path)s"
         if module_version is None:
             return False
 
-        self.logger.info('Running unit tests...')
+        if functional:
+            self.logger.info('Running module functional tests...')
+        else:
+            self.logger.info('Running unit tests...')
         coverage_file_path = self.__get_coverage_file(module_name, module_version)
         module_root_path = self.__get_module_root_path(module_name)
         if pattern:
-            cmd = self.__get_tests_cmd_with_pattern(module_root_path, pattern)
+            cmd = self.__get_tests_cmd_with_pattern(module_root_path, pattern, functional=functional)
+        elif functional:
+            cmd = self.__get_functional_tests_cmd(module_root_path)
         else:
             cmd = self.__get_tests_cmd_with_coverage(module_root_path, coverage_file_path)
         self.logger.debug('Test cmd: %s' % cmd)
@@ -284,18 +290,22 @@ cd "%(path)s"
             time.sleep(0.25)
 
         # display coverage report
-        if display_coverage:
+        if display_coverage and not functional:
             self.logger.debug('Display coverage')
             self.logger.info(self.module_tests_coverage(module_name))
 
         # copy coverage file to standart location (in module tests directory) for coverage report tools
-        if copy_to:
+        if copy_to and not functional:
             if not os.path.isfile(coverage_file_path):
                 self.logger.error('Coverage file missing: %s' % coverage_file_path)
                 return False
             shutil.copyfile(coverage_file_path, os.path.join(copy_to, '.coverage'))
 
         self.logger.debug('Return code: %s' % self.__endless_command_return_code)
+        # pytest exit code 5 = no tests collected (app has no @pytest.mark.functional)
+        if functional and self.__endless_command_return_code == 5:
+            self.logger.info('No functional tests for module "%s" — OK' % module_name)
+            return True
         if self.__endless_command_return_code!=0:
             return False
 
@@ -309,16 +319,30 @@ cd "%(path)s"
         """
         return """
 cd "%s"
-PYTHONPATH=. COVERAGE_FILE=%s coverage run --omit="*/lib/python*/*","*/tests/*","test_*" --source=backend --concurrency=thread -m pytest tests
+PYTHONPATH=. COVERAGE_FILE=%s coverage run --omit="*/lib/python*/*","*/tests/*","test_*" --source=backend --concurrency=thread -m pytest --tb=short -ra -m "not functional" tests
         """ % (module_root_path, coverage_file_path)
 
-    def __get_tests_cmd_with_pattern(self, module_root_path, pattern):
+    def __get_functional_tests_cmd(self, module_root_path):
+        """
+        Return cmdline to run module functional tests only (no coverage).
+        """
+        return """
+cd "%s"
+CLEEP_FUNCTIONAL_TESTS=1 PYTHONPATH=. python3 -m pytest -v --tb=short -ra -m functional tests
+        """ % module_root_path
+
+    def __get_tests_cmd_with_pattern(self, module_root_path, pattern, functional=False):
         """
         Return cmdline to run tests matching specified pattern (no coverage).
         """
+        if functional:
+            return """
+cd "%s"
+CLEEP_FUNCTIONAL_TESTS=1 PYTHONPATH=. python3 -m pytest --tb=short -ra -m functional tests -k "%s"
+            """ % (module_root_path, pattern)
         return """
 cd "%s"
-PYTHONPATH=. python3 -m pytest tests -k "%s"
+PYTHONPATH=. python3 -m pytest --tb=short -ra -m "not functional" tests -k "%s"
         """ % (module_root_path, pattern)
 
     def module_tests_coverage(self, module_name, missing=False, as_json=False, quiet=True):
@@ -379,57 +403,7 @@ PYTHONPATH=. python3 -m pytest tests -k "%s"
         """
         return config.REPO_DIR
 
-    def __list_core_files(self, core_path, pattern=None):
-        """
-        Return all python files on core with their associated tests
-
-        Args:
-            core_path (string): core path
-            pattern (str): custom pattern to filter files to process
-
-        Returns:
-            dict: list of files with test file path::
-
-                {
-                    python file (string): test file (string)
-                }
-
-        """
-        files = []
-        tests_path = self.__get_core_tests_path()
-
-        self.logger.info('Searching core files...')
-        for root, _, filenames in os.walk(core_path):
-            for filename in filenames:
-                # build paths
-                filepath = os.path.join(root, filename)
-                core_relative_path = root.replace('%s' % core_path, '')
-                core_relative_path = core_relative_path[1:] if core_relative_path.startswith('/') else core_relative_path
-                test_filepath = os.path.join(tests_path, core_relative_path, 'test_%s' % filename)
-
-                # drop useless files
-                if not filename.endswith('.py'):
-                    self.logger.debug('  Drop file "%s": not python file', filepath)
-                    continue
-                if filename.startswith('__init__'):
-                    self.logger.debug('  Drop file "%s": python init file', filepath)
-                    continue
-                if filename.startswith('test_'):
-                    self.logger.debug('  Drop file "%s": test file', filepath)
-                    continue
-                if root.endswith('libs/tests'):
-                    self.logger.debug('  Drop file "%s": test lib', filepath)
-                    continue
-                if pattern and not re.search(pattern, test_filepath) and not re.search(pattern, filepath):
-                    self.logger.debug('  Drop file "%s": does not match pattern %s', filepath, pattern)
-                    continue
-
-                self.logger.debug('  Found "%s" [%s]' % (filepath, test_filepath))
-                files.append((filepath, test_filepath))
-
-        return files
-
-    def core_tests(self, display_coverage=False, display_test_output=False, xml=False, quiet=True, pattern=None):
+    def core_tests(self, display_coverage=False, display_test_output=False, xml=False, quiet=True, pattern=None, functional=False):
         """
         Execute core unit tests and display process output on stdout
 
@@ -439,6 +413,7 @@ PYTHONPATH=. python3 -m pytest tests -k "%s"
             xml (bool): use xml coverage command instead of report command
             quiet (bool): display or not coverage.py warnings
             pattern (str): custom pattern to filter tests (pytest -k)
+            functional (bool): run functional tests (real OS binaries) instead of unit tests
 
         Returns:
             bool: True if process succeed.
@@ -451,15 +426,25 @@ PYTHONPATH=. python3 -m pytest tests -k "%s"
             self.logger.error('Unable to clear previous tests results')
             return False
 
-        self.logger.info('Running core unit tests (pytest)...')
+        if functional:
+            self.logger.info('Running core functional tests (pytest, real OS binaries)...')
+            marker_opt = '-m functional'
+            env_prefix = 'CLEEP_FUNCTIONAL_TESTS=1 '
+        else:
+            self.logger.info('Running core unit tests (pytest)...')
+            marker_opt = '-m "not functional"'
+            env_prefix = ''
         pattern_opt = '-k "%s"' % pattern if pattern else ''
-        verbosity = '-v' if display_test_output else '-q'
+        # Always keep short failure tracebacks (+ failed/error summary) so CI logs are actionable
+        verbosity = '-v --tb=short -ra' if display_test_output else '-q --tb=short -ra'
         cmd = """
 cd "%(repo_dir)s"
-PYTHONPATH=. coverage run --omit="*/lib/python*/*","*/tests/*","*test_*.py" --source=cleep --concurrency=gevent -m pytest %(verbosity)s cleep/tests %(pattern_opt)s
+%(env_prefix)sPYTHONPATH=. coverage run --omit="*/lib/python*/*","*/tests/*","*test_*.py" --source=cleep --concurrency=gevent -m pytest %(verbosity)s %(marker_opt)s cleep/tests %(pattern_opt)s
         """ % {
             'repo_dir': repo_dir,
+            'env_prefix': env_prefix,
             'verbosity': verbosity,
+            'marker_opt': marker_opt,
             'pattern_opt': pattern_opt,
         }
         self.logger.debug('Test cmd: %s' % cmd)
@@ -469,8 +454,15 @@ PYTHONPATH=. coverage run --omit="*/lib/python*/*","*/tests/*","*test_*.py" --so
         c = EndlessConsole(cmd, console_callback, self.__console_end_callback)
         c.start()
 
-        while self.__endless_command_running:
-            time.sleep(0.25)
+        try:
+            while self.__endless_command_running:
+                time.sleep(0.25)
+        except KeyboardInterrupt:
+            self.logger.warning('Interrupted — stopping tests...')
+            c.kill()
+            while self.__endless_command_running:
+                time.sleep(0.1)
+            raise
 
         success = self.__endless_command_return_code == 0
 

{cleepcli-1.45.0 → cleepcli-1.46.0}/cleepcli/version.py

@@ -1,4 +1,4 @@
 #!/usr/bin/env python
 # -*- coding: utf-8 -*-
 
-VERSION = "1.45.0"
+VERSION = "1.46.0"

{cleepcli-1.45.0 → cleepcli-1.46.0/cleepcli.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: cleepcli
-Version: 1.45.0
+Version: 1.46.0
 Summary: Cleep-cli helps developers to build great Cleep applications from command line.
 Home-page: https://github.com/CleepDevice/cleep-cli
 Author: Tanguy Bonneau
@@ -42,7 +42,7 @@ This utility helps developers to build Cleep applications providing some useful
     - `coreget` to clone or pull official Cleep repository.
     - `coredocs` to generate core documentation in HTML format.
     - `coresync` to synchronize sources from repository to execution folders.
-    - `coretests` to perform core tests suite.
+    - `coretests` to perform core unit tests suite (`--functional` for real OS binary tests on device / OS migration).
     - `coretestscov` to display last core tests code coverage report.
     - `cpuprof` to run CPU profiler on application execution instance.
     - `memprof` to run memory profiler on application execution instance.
@@ -52,7 +52,7 @@ This utility helps developers to build Cleep applications providing some useful
     - `moddelete` to delete installed files of a module.
     - `moddocs` to generate module documentation.
     - `modsync` to synchronize sources from module folder to execution folders.
-    - `modtests` to execute module tests.
+    - `modtests` to execute module unit tests (`--functional` for real OS/device tests).
     - `modtestscov` to display last module tests code coverage report.
     - `modbuild` to create application package (.zip)
     - `modcheckback` to check app backend (python files structure and content)
@@ -142,7 +142,7 @@ The [watchdog](https://pypi.org/project/watchdog/) python library is used to det
 
 ## CI (GitHub Actions)
 
-A tag `vX.Y.Z` (or a manual **workflow_dispatch**) builds a **py3** wheel + sdist on `ubuntu-latest` (Python 3.9) via `python -m build` and uploads them to PyPI.
+A tag `vX.Y.Z` (or a manual **workflow_dispatch**) builds a **py3** wheel + sdist on `ubuntu-latest` (Python 3.12) via `python -m build` and uploads them to PyPI.
 
 GitHub **environment** `actions` (Settings → Environments), secret: