robotframework-pabot 4.2.0__tar.gz → 4.3.1__tar.gz

{robotframework_pabot-4.2.0/src/robotframework_pabot.egg-info → robotframework_pabot-4.3.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: robotframework-pabot
-Version: 4.2.0
+Version: 4.3.1
 Summary: Parallel test runner for Robot Framework
 Home-page: https://pabot.org
 Download-URL: https://pypi.python.org/pypi/robotframework-pabot
@@ -156,7 +156,7 @@ Supports all [Robot Framework command line options](https://robotframework.org/r
   Indicator for a file that can contain shared variables for distributing resources. This needs to be used together with 
   pabotlib option. Resource file syntax is same as Windows ini files. Where a section is a shared set of variables.
 
---argumentfile [INTEGER] [FILEPATH]          
+--argumentfile[INTEGER] [FILEPATH]          
   Run same suites with multiple [argumentfile](http://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#argument-files) options.
 
   For example:
@@ -266,14 +266,18 @@ After this come the suite names.
 
 With ```--ordering FILENAME``` you can have a list that controls order also. The syntax is same as .pabotsuitenames file syntax but does not contain 4 hash rows that are present in .pabotsuitenames. 
 
+Note: The `--ordering` file is intended only for defining the execution order of suites and tests. The actual selection of what to run must still be done using options like `--test`, `--suite`, `--include`, or `--exclude`.
+
 There different possibilities to influence the execution:
 
   * The order of suites can be changed.
   * If a directory (or a directory structure) should be executed sequentially, add the directory suite name to a row as a ```--suite``` option.
   * If the base suite name is changing with robot option [```--name / -N```](https://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#setting-the-name) you can also give partial suite name without the base suite.
   * You can add a line with text `#WAIT` to force executor to wait until all previous suites have been executed.
-  * You can group suites and tests together to same executor process by adding line `{` before the group and `}`after.
-  * You can introduce dependencies using the word `#DEPENDS` after a test declaration. Can be used several times if it is necessary to refer to several different tests. Please take care that in case of circular dependencies an exception will be thrown. An example could be.
+  * You can group suites and tests together to same executor process by adding line `{` before the group and `}` after. Note that `#WAIT` cannot be used inside a group.
+  * You can introduce dependencies using the word `#DEPENDS` after a test declaration. This keyword can be used several times if it is necessary to refer to several different tests. Please take care that in case of circular dependencies an exception will be thrown. Note that each `#WAIT` splits suites into separate execution blocks, and it's not possible to define dependencies for suites or tests that are inside another `#WAIT` block or inside another `{}` brackets.
+  * Note: Within a group `{}`, neither execution order nor the `#DEPENDS` keyword currently works. This is due to limitations in Robot Framework, which is invoked within Pabot subprocesses. These limitations may be addressed in a future release of Robot Framework. For now, tests or suites within a group will be executed in the order Robot Framework discovers them — typically in alphabetical order.
+  * An example could be:
 
 ```
 --test robotTest.1 Scalar.Test With Environment Variables #DEPENDS robotTest.1 Scalar.Test with BuiltIn Variables of Robot Framework
@@ -281,8 +285,10 @@ There different possibilities to influence the execution:
 --test robotTest.2 Lists.Test with Keywords and a list
 #WAIT
 --test robotTest.2 Lists.Test with a Keyword that accepts multiple arguments
+{
 --test robotTest.2 Lists.Test with some Collections keywords
 --test robotTest.2 Lists.Test to access list entries
+}
 --test robotTest.3 Dictionary.Test that accesses Dictionaries
 --test robotTest.3 Dictionary.Dictionaries for named arguments #DEPENDS robotTest.3 Dictionary.Test that accesses Dictionaries
 --test robotTest.1 Scalar.Test Case With Variables #DEPENDS robotTest.3 Dictionary.Test that accesses Dictionaries
@@ -296,12 +302,13 @@ There different possibilities to influence the execution:
   * By using the command `#SLEEP X`, where `X` is an integer in the range [0-3600] (in seconds), you can 
   define a startup delay for each subprocess. `#SLEEP` affects the next line unless the next line starts a 
   group with `{`, in which case the delay applies to the entire group. If the next line begins with `--test` 
-  or `--suite`, the delay is applied to that specific item. Any other occurrences of `#SLEEP` are ignored.
+  or `--suite`, the delay is applied to that specific item. Any other occurrences of `#SLEEP` are ignored. 
+  Note that `#SLEEP` has no effect within a group, i.e., inside a subprocess.
 
 The following example clarifies the behavior:
 
 ```sh
-pabot --process 2 --ordering order.txt data_1
+pabot --processes 2 --ordering order.txt data_1
 ```
 
 where order.txt is:

{robotframework_pabot-4.2.0 → robotframework_pabot-4.3.1}/README.md

@@ -130,7 +130,7 @@ Supports all [Robot Framework command line options](https://robotframework.org/r
   Indicator for a file that can contain shared variables for distributing resources. This needs to be used together with 
   pabotlib option. Resource file syntax is same as Windows ini files. Where a section is a shared set of variables.
 
---argumentfile [INTEGER] [FILEPATH]          
+--argumentfile[INTEGER] [FILEPATH]          
   Run same suites with multiple [argumentfile](http://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#argument-files) options.
 
   For example:
@@ -240,14 +240,18 @@ After this come the suite names.
 
 With ```--ordering FILENAME``` you can have a list that controls order also. The syntax is same as .pabotsuitenames file syntax but does not contain 4 hash rows that are present in .pabotsuitenames. 
 
+Note: The `--ordering` file is intended only for defining the execution order of suites and tests. The actual selection of what to run must still be done using options like `--test`, `--suite`, `--include`, or `--exclude`.
+
 There different possibilities to influence the execution:
 
   * The order of suites can be changed.
   * If a directory (or a directory structure) should be executed sequentially, add the directory suite name to a row as a ```--suite``` option.
   * If the base suite name is changing with robot option [```--name / -N```](https://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#setting-the-name) you can also give partial suite name without the base suite.
   * You can add a line with text `#WAIT` to force executor to wait until all previous suites have been executed.
-  * You can group suites and tests together to same executor process by adding line `{` before the group and `}`after.
-  * You can introduce dependencies using the word `#DEPENDS` after a test declaration. Can be used several times if it is necessary to refer to several different tests. Please take care that in case of circular dependencies an exception will be thrown. An example could be.
+  * You can group suites and tests together to same executor process by adding line `{` before the group and `}` after. Note that `#WAIT` cannot be used inside a group.
+  * You can introduce dependencies using the word `#DEPENDS` after a test declaration. This keyword can be used several times if it is necessary to refer to several different tests. Please take care that in case of circular dependencies an exception will be thrown. Note that each `#WAIT` splits suites into separate execution blocks, and it's not possible to define dependencies for suites or tests that are inside another `#WAIT` block or inside another `{}` brackets.
+  * Note: Within a group `{}`, neither execution order nor the `#DEPENDS` keyword currently works. This is due to limitations in Robot Framework, which is invoked within Pabot subprocesses. These limitations may be addressed in a future release of Robot Framework. For now, tests or suites within a group will be executed in the order Robot Framework discovers them — typically in alphabetical order.
+  * An example could be:
 
 ```
 --test robotTest.1 Scalar.Test With Environment Variables #DEPENDS robotTest.1 Scalar.Test with BuiltIn Variables of Robot Framework
@@ -255,8 +259,10 @@ There different possibilities to influence the execution:
 --test robotTest.2 Lists.Test with Keywords and a list
 #WAIT
 --test robotTest.2 Lists.Test with a Keyword that accepts multiple arguments
+{
 --test robotTest.2 Lists.Test with some Collections keywords
 --test robotTest.2 Lists.Test to access list entries
+}
 --test robotTest.3 Dictionary.Test that accesses Dictionaries
 --test robotTest.3 Dictionary.Dictionaries for named arguments #DEPENDS robotTest.3 Dictionary.Test that accesses Dictionaries
 --test robotTest.1 Scalar.Test Case With Variables #DEPENDS robotTest.3 Dictionary.Test that accesses Dictionaries
@@ -270,12 +276,13 @@ There different possibilities to influence the execution:
   * By using the command `#SLEEP X`, where `X` is an integer in the range [0-3600] (in seconds), you can 
   define a startup delay for each subprocess. `#SLEEP` affects the next line unless the next line starts a 
   group with `{`, in which case the delay applies to the entire group. If the next line begins with `--test` 
-  or `--suite`, the delay is applied to that specific item. Any other occurrences of `#SLEEP` are ignored.
+  or `--suite`, the delay is applied to that specific item. Any other occurrences of `#SLEEP` are ignored. 
+  Note that `#SLEEP` has no effect within a group, i.e., inside a subprocess.
 
 The following example clarifies the behavior:
 
 ```sh
-pabot --process 2 --ordering order.txt data_1
+pabot --processes 2 --ordering order.txt data_1
 ```
 
 where order.txt is:

{robotframework_pabot-4.2.0 → robotframework_pabot-4.3.1}/src/pabot/__init__.py

@@ -7,4 +7,4 @@ try:
 except ImportError:
     pass
 
-__version__ = "4.2.0"
+__version__ = "4.3.1"

{robotframework_pabot-4.2.0 → robotframework_pabot-4.3.1}/src/pabot/arguments.py

@@ -1,5 +1,9 @@
+import atexit
 import multiprocessing
+import os
+import glob
 import re
+import tempfile
 from typing import Dict, List, Optional, Tuple
 
 from robot import __version__ as ROBOT_VERSION
@@ -66,9 +70,72 @@ def parse_args(
         options_for_subprocesses["name"] = "Suites"
     opts = _delete_none_keys(options)
     opts_sub = _delete_none_keys(options_for_subprocesses)
+    _replace_arg_files(pabot_args, opts_sub)
     return opts, datasources, pabot_args, opts_sub
 
 
+# remove options from argumentfile according to different scenarios.
+# -t/--test/--task shall be removed if --testlevelsplit options exists
+# -s/--suite shall be removed if --testlevelsplit options does not exist
+def _replace_arg_files(pabot_args, opts_sub):
+    _cleanup_old_pabot_temp_files()
+    if not opts_sub.get('argumentfile') or not opts_sub['argumentfile']:
+        return
+    arg_file_list = opts_sub['argumentfile']
+    temp_file_list = []
+    test_level = pabot_args.get('testlevelsplit')
+
+    for arg_file_path in arg_file_list:
+        with open(arg_file_path, 'r') as arg_file:
+            arg_file_lines = arg_file.readlines()
+        if not arg_file_lines:
+            continue
+
+        fd, temp_path = tempfile.mkstemp(prefix="pabot_temp_", suffix=".txt")
+        with os.fdopen(fd, 'wb') as temp_file:
+            for line in arg_file_lines:
+                if test_level and _is_test_option(line):
+                    continue
+                elif not test_level and _is_suite_option(line):
+                    continue
+                temp_file.write(line.encode('utf-8'))
+
+        temp_file_list.append(temp_path)
+
+    opts_sub['argumentfile'] = temp_file_list
+    atexit.register(cleanup_temp_file, temp_file_list)
+
+
+def _is_suite_option(line):
+    return line.startswith('-s ') or line.startswith('--suite ')
+
+
+def _is_test_option(line):
+    return line.startswith('-t ') or line.startswith('--test ') or line.startswith('--task ')
+
+
+# clean the temp argument files before exiting the pabot process
+def cleanup_temp_file(temp_file_list):
+    for temp_file in temp_file_list:
+        if os.path.exists(temp_file):
+            try:
+                os.remove(temp_file)
+            except Exception:
+                pass
+
+
+# Deletes all possible pabot_temp_ files from os temp directory
+def _cleanup_old_pabot_temp_files():
+    temp_dir = tempfile.gettempdir()
+    pattern = os.path.join(temp_dir, "pabot_temp_*.txt")
+    old_files = glob.glob(pattern)
+    for file_path in old_files:
+        try:
+            os.remove(file_path)
+        except Exception:
+            pass
+
+
 def _parse_shard(arg):
     # type: (str) -> Tuple[int, int]
     parts = arg.split("/")

{robotframework_pabot-4.2.0 → robotframework_pabot-4.3.1}/src/pabot/execution_items.py

@@ -7,6 +7,40 @@ from robot.utils import PY2, is_unicode
 
 import re
 
+
+def create_dependency_tree(items):  
+    # type: (List[ExecutionItem]) -> List[List[ExecutionItem]]
+    independent_tests = list(filter(lambda item: not item.depends, items))
+    dependency_tree = [independent_tests]
+    dependent_tests = list(filter(lambda item: item.depends, items))
+    unknown_dependent_tests = dependent_tests
+    while len(unknown_dependent_tests) > 0:
+        run_in_this_stage, run_later = [], []
+        for d in unknown_dependent_tests:
+            stage_indexes = []
+            for i, stage in enumerate(dependency_tree):
+                for test in stage:
+                    if test.name in d.depends:
+                        stage_indexes.append(i)
+            # All #DEPENDS test are already run:
+            if len(stage_indexes) == len(d.depends):
+                run_in_this_stage.append(d)
+            else:
+                run_later.append(d)
+        unknown_dependent_tests = run_later
+        if len(run_in_this_stage) == 0:
+            text = "There are circular or unmet dependencies using #DEPENDS. Check this/these test(s): " + str(run_later)
+            raise DataError(text)
+        else:
+            dependency_tree.append(run_in_this_stage)
+    flattened_dependency_tree = sum(dependency_tree, [])
+    if len(flattened_dependency_tree) != len(items):
+        raise DataError(
+            "Invalid test configuration: Circular or unmet dependencies detected between test suites. Please check your #DEPENDS definitions."
+        )
+    return dependency_tree
+
+
 @total_ordering
 class ExecutionItem(object):
     isWait = False
@@ -78,7 +112,7 @@ class GroupItem(ExecutionItem):
     type = "group"
 
     def __init__(self):
-        self.name = "Group_"
+        self.name = "Group"
         self._items = []
         self._element_type = None
 
@@ -91,13 +125,26 @@ class GroupItem(ExecutionItem):
             )
         if len(self._items) > 0:
             self.name += "_"
-        if self.get_sleep() < item.get_sleep():     # TODO: check!
+        if self.get_sleep() < item.get_sleep():
             self.set_sleep(item.get_sleep())
         self.name += item.name
         self._element_type = item.type
         self._items.append(item)
+    
+    def change_items_order_by_depends(self):
+        ordered_name = "Group"
+        dependency_tree = create_dependency_tree(self._items)
+        ordered = [item for sublist in dependency_tree for item in sublist]
+        for item in ordered:
+            ordered_name += f"_{item.name}"
+        self.name = ordered_name
+        self._items = ordered
 
     def modify_options_for_executor(self, options):
+        # Since a GroupItem contains either tests or suites, options are cleared
+        # and only the Group's content is used to avoid duplicate execution.
+        options['suite'] = []
+        options['test'] = []
         for item in self._items:
             if item.type not in options:
                 options[item.type] = []
@@ -131,6 +178,7 @@ class RunnableItem(ExecutionItem):
             if len(depends_indexes) == 0
             else line_name[0:depends_indexes[0]].strip()
         )
+        assert len(self.name) != 0, f"Suite or test name cannot be empty and then contain #DEPENDS like: {name}"
         self.depends = (
             self._split_dependencies(line_name, depends_indexes)
             if len(depends_indexes) != 0
@@ -208,6 +256,8 @@ class TestItem(RunnableItem):
         def modify_options_for_executor(self, options):
             if "rerunfailed" in options:
                 del options["rerunfailed"]
+            if "rerunfailedsuites" in options:
+                del options["rerunfailedsuites"]
             name = self.name
             for char in ["[", "?", "*"]:
                 name = name.replace(char, "[" + char + "]")
@@ -218,6 +268,8 @@ class TestItem(RunnableItem):
         def modify_options_for_executor(self, options):
             if "rerunfailed" in options:
                 del options["rerunfailed"]
+            if "rerunfailedsuites" in options:
+                del options["rerunfailedsuites"]
 
     def difference(self, from_items):
         # type: (List[ExecutionItem]) -> List[ExecutionItem]

{robotframework_pabot-4.2.0 → robotframework_pabot-4.3.1}/src/pabot/pabot.py

@@ -80,6 +80,7 @@ from .execution_items import (
     TestItem,
     RunnableItem,
     SleepItem,
+    create_dependency_tree,
 )
 from .result_merger import merge
 
@@ -290,7 +291,9 @@ def _create_command_for_execution(caller_id, datasources, is_last, item, outs_di
             item.last_level,
             item.processes,
         )
-        + datasources
+        # If the datasource ends with a backslash '\', it is deleted to ensure 
+        # correct handling of the escape character later on.
+        + [os.path.normpath(s) for s in datasources]
     )
     return _mapOptionalQuote(cmd)
 
@@ -761,6 +764,7 @@ def _options_to_cli_arguments(opts):  # type: (dict) -> List[str]
 
 
 def _group_by_groups(tokens):
+    # type: (List[ExecutionItem]) -> List[ExecutionItem]
     result = []
     group = None
     for token in tokens:
@@ -778,6 +782,7 @@ def _group_by_groups(tokens):
                 raise DataError(
                     "Ordering: Group end tag '}' encountered before start '{'"
                 )
+            group.change_items_order_by_depends()
             group = None
             continue
         if group is not None:
@@ -949,7 +954,7 @@ def solve_suite_names(outs_dir, datasources, options, pabot_args):
             )
             execution_item_lines = [parse_execution_item_line(l) for l in lines[4:]]
             if corrupted or h != file_h or file_hash != hash_of_file or pabot_args.get("pabotprerunmodifier"):
-                if file_h[0] != h[0] and file_h[2] == h[2]:
+                if file_h is not None and file_h[0] != h[0] and file_h[2] == h[2]:
                     suite_names = _levelsplit(
                         generate_suite_names_with_builder(outs_dir, datasources, options),
                         pabot_args,
@@ -985,7 +990,8 @@ def _levelsplit(
 
 
 def _group_by_wait(lines):
-    suites = [[]]  # type: List[List[ExecutionItem]]
+    # type: (List[ExecutionItem]) -> List[List[ExecutionItem]]
+    suites = [[]]
     for suite in lines:
         if not suite.isWait:
             if suite:
@@ -1088,8 +1094,6 @@ def _fix_items(items):  # type: (List[ExecutionItem]) -> List[ExecutionItem]
     _remove_empty_groups(result)
     if result and result[0].isWait:
         result = result[1:]
-    if result and result[-1].isWait:
-        result = result[:-1]
     return result
 
 
@@ -1334,6 +1338,8 @@ def _options_for_rebot(options, start_time_string, end_time_string):
     rebot_options["test"] = []
     rebot_options["exclude"] = []
     rebot_options["include"] = []
+    if rebot_options.get("runemptysuite"):
+        rebot_options["processemptysuite"] = True
     if ROBOT_VERSION >= "2.8":
         options["monitormarkers"] = "off"
     for key in [
@@ -1358,6 +1364,7 @@ def _options_for_rebot(options, start_time_string, end_time_string):
         "randomize",
         "runemptysuite",
         "rerunfailed",
+        "rerunfailedsuites",
         "skip",
         "skiponfailure",
         "skipteardownonexit",
@@ -1476,6 +1483,19 @@ def _copy_output_artifacts(options, file_extensions=None, include_subfolders=Fal
     return copied_artifacts
 
 
+def _check_pabot_results_for_missing_xml(base_dir):
+    missing = []
+    for root, dirs, _ in os.walk(base_dir):
+        if root == base_dir:
+            for subdir in dirs:
+                subdir_path = os.path.join(base_dir, subdir)
+                has_xml = any(fname.endswith('.xml') for fname in os.listdir(subdir_path))
+                if not has_xml:
+                    missing.append(os.path.join(subdir_path, 'robot_stderr.out'))
+            break
+    return missing
+
+
 def _report_results(outs_dir, pabot_args, options, start_time_string, tests_root_name):
     if "pythonpath" in options:
         del options["pythonpath"]
@@ -1491,6 +1511,7 @@ def _report_results(outs_dir, pabot_args, options, start_time_string, tests_root
             "failed": 0,
             "skipped": 0,
         }
+    missing_outputs = []
     if pabot_args["argumentfiles"]:
         outputs = []  # type: List[str]
         for index, _ in pabot_args["argumentfiles"]:
@@ -1507,14 +1528,27 @@ def _report_results(outs_dir, pabot_args, options, start_time_string, tests_root
                     outputfile=os.path.join("pabot_results", "output%s.xml" % index),
                 )
             ]
+            missing_outputs.extend(_check_pabot_results_for_missing_xml(os.path.join(outs_dir, index)))
         if "output" not in options:
             options["output"] = "output.xml"
         _write_stats(stats)
-        return rebot(*outputs, **_options_for_rebot(options, start_time_string, _now()))
+        exit_code = rebot(*outputs, **_options_for_rebot(options, start_time_string, _now()))
     else:
-        return _report_results_for_one_run(
+        exit_code = _report_results_for_one_run(
             outs_dir, pabot_args, options, start_time_string, tests_root_name, stats
         )
+        missing_outputs.extend(_check_pabot_results_for_missing_xml(outs_dir))
+    if missing_outputs:
+        _write(("[ " + _wrap_with(Color.YELLOW, 'WARNING') + " ] "
+                "One or more subprocesses encountered an error and the "
+                "internal .xml files could not be generated. Please check the "
+                "following stderr files to identify the cause:"))
+        for missing in missing_outputs:
+            _write(repr(missing))
+        _write((f"[ " + _wrap_with(Color.RED, 'ERROR') + " ] "
+                "The output, log and report files produced by Pabot are "
+                "incomplete and do not contain all test cases."))
+    return exit_code if not missing_outputs else 252
 
 
 def _write_stats(stats):
@@ -1549,9 +1583,9 @@ def _report_results_for_one_run(
     _write_stats(stats)
     if (
         "report" in options
-        and options["report"] == "NONE"
+        and options["report"].upper() == "NONE"
         and "log" in options
-        and options["log"] == "NONE"
+        and options["log"].upper() == "NONE"
     ):
         options[
             "output"
@@ -1867,19 +1901,28 @@ def _chunk_items(items, chunk_size):
         base_item = chunked_items[0]
         if not base_item:
             continue
-        execution_items = SuiteItems([item.execution_item for item in chunked_items])
-        chunked_item = QueueItem(
-            base_item.datasources,
-            base_item.outs_dir,
-            base_item.options,
-            execution_items,
-            base_item.command,
-            base_item.verbose,
-            (base_item.argfile_index, base_item.argfile),
-            processes=base_item.processes,
-            timeout=base_item.timeout,
-        )
-        yield chunked_item
+        if isinstance(base_item.execution_item, TestItem):
+            for item in chunked_items:
+                chunked_item = _queue_item(base_item, item.execution_item)
+                yield chunked_item
+        else:
+            execution_items = SuiteItems([item.execution_item for item in chunked_items])
+            chunked_item = _queue_item(base_item, execution_items)
+            yield chunked_item
+
+
+def _queue_item(base_item, execution_items):
+    return QueueItem(
+        base_item.datasources,
+        base_item.outs_dir,
+        base_item.options,
+        execution_items,
+        base_item.command,
+        base_item.verbose,
+        (base_item.argfile_index, base_item.argfile),
+        processes=base_item.processes,
+        timeout=base_item.timeout,
+    )
 
 
 def _find_ending_level(name, group):
@@ -1991,8 +2034,8 @@ def main_program(args):
         options, datasources, pabot_args, opts_for_run = parse_args(args)
         if pabot_args["help"]:
             help_print = __doc__.replace(
-                "PLACEHOLDER_README.MD",
-                read_args_from_readme()
+                    "PLACEHOLDER_README.MD",
+                    read_args_from_readme()
                 )
             print(help_print.replace("[PABOT_VERSION]", PABOT_VERSION))
             return 0
@@ -2028,7 +2071,7 @@ def main_program(args):
             _write((
                 "All tests were executed, but the --no-rebot argument was given, "
                 "so the results were not compiled, and no summary was generated. "
-                f"All results have been saved in the {os.path.join(os.path.curdir, 'pabot_results')} folder."
+                f"All results have been saved in the {outs_dir} folder."
             ))
             _write("===================================================")
             return 0 if not _ABNORMAL_EXIT_HAPPENED else 252
@@ -2069,20 +2112,40 @@ def _parse_ordering(filename):  # type: (str) -> List[ExecutionItem]
         with open(filename, "r") as orderingfile:
             return [
                 parse_execution_item_line(line.strip())
-                for line in orderingfile.readlines()
+                for line in orderingfile.readlines() if line.strip() != ""
             ]
     except FileNotFoundError:
         raise DataError("Error: File '%s' not found." % filename)
-    except ValueError as e:
+    except (ValueError, AssertionError) as e:
         raise DataError("Error in ordering file: %s: %s" % (filename, e))
-    except:
+    except Exception:
         raise DataError("Error parsing ordering file '%s'" % filename)
 
 
+def _check_ordering(ordering_file, suite_names):  # type: (List[ExecutionItem], List[ExecutionItem]) -> None
+    list_of_suite_names = [s.name for s in suite_names]
+    number_of_tests_or_suites = 0
+    if ordering_file:
+        for item in ordering_file:
+            if item.type in ['suite', 'test']:
+                if not any((s == item.name or s.endswith("." + item.name)) for s in list_of_suite_names):
+                    # If test name is too long, it gets name ' Invalid', so skip that
+                    # Additionally, the test is skipped also if the user wants a higher-level suite to be executed sequentially by using 
+                    # the --suite option, and the given name is part of the full name of any test or suite.
+                    if item.name != ' Invalid' and not (item.type == 'suite' and any((s == item.name or s.startswith(item.name + ".")) for s in list_of_suite_names)):
+                        raise DataError("%s item '%s' in --ordering file does not match suite or test names in .pabotsuitenames file.\nPlease verify content of --ordering file." % (item.type.title(), item.name))
+                number_of_tests_or_suites += 1
+        if number_of_tests_or_suites > len(list_of_suite_names):
+            raise DataError('Ordering file contains more tests and/or suites than exists. Check that there is no duplicates etc. in ordering file and that to .pabotsuitenames.')
+
+
 def _group_suites(outs_dir, datasources, options, pabot_args):
     suite_names = solve_suite_names(outs_dir, datasources, options, pabot_args)
     _verify_depends(suite_names)
     ordering_arg = _parse_ordering(pabot_args.get("ordering")) if (pabot_args.get("ordering")) is not None else None
+    if ordering_arg:
+        _verify_depends(ordering_arg)
+        _check_ordering(ordering_arg, suite_names)
     ordering_arg_with_sleep = _set_sleep_times(ordering_arg)
     ordered_suites = _preserve_order(suite_names, ordering_arg_with_sleep)
     shard_suites = solve_shard_suites(ordered_suites, pabot_args)
@@ -2142,7 +2205,7 @@ def _verify_depends(suite_names):
     suites_with_found_dependencies = list(
         filter(
             lambda suite: any(
-                runnable_suite.name == suite.depends
+                runnable_suite.name in suite.depends
                 for runnable_suite in runnable_suites
             ),
             suites_with_depends,
@@ -2153,63 +2216,31 @@ def _verify_depends(suite_names):
             "Invalid test configuration: Some test suites have dependencies (#DEPENDS) that cannot be found."
         )
     suites_with_circular_dependencies = list(
-        filter(lambda suite: suite.depends == suite.name, suites_with_depends)
+        filter(lambda suite: suite.name in suite.depends, suites_with_depends)
     )
     if suites_with_circular_dependencies:
         raise DataError(
             "Invalid test configuration: Test suites cannot depend on themselves."
         )
-    grouped_suites = list(
-        filter(lambda suite: isinstance(suite, GroupItem), suite_names)
-    )
-    if grouped_suites and suites_with_depends:
-        raise DataError(
-            "Invalid test configuration: Cannot use both #DEPENDS and grouped suites."
-        )
 
 
 def _group_by_depend(suite_names):
+    # type: (List[ExecutionItem]) -> List[List[ExecutionItem]]
     group_items = list(filter(lambda suite: isinstance(suite, GroupItem), suite_names))
     runnable_suites = list(
         filter(lambda suite: isinstance(suite, RunnableItem), suite_names)
     )
-    if group_items or not runnable_suites:
-        return [suite_names]
-    independent_tests = list(filter(lambda suite: not suite.depends, runnable_suites))
-    dependency_tree = [independent_tests]
-    dependent_tests = list(filter(lambda suite: suite.depends, runnable_suites))
-    unknown_dependent_tests = dependent_tests
-    while len(unknown_dependent_tests) > 0:
-        run_in_this_stage, run_later = [], []
-        for d in unknown_dependent_tests:
-            stage_indexes = []
-            for i, stage in enumerate(dependency_tree):
-                for test in stage:
-                    if test.name in d.depends:
-                        stage_indexes.append(i)
-            # All #DEPENDS test are already run:
-            if len(stage_indexes) == len(d.depends):
-                run_in_this_stage.append(d)
-            else:
-                run_later.append(d)
-        unknown_dependent_tests = run_later
-        if len(run_in_this_stage) == 0:
-            text = "There are circular or unmet dependencies using #DEPENDS. Check this/these test(s): " + str(run_later)
-            raise DataError(text)
-        else:
-            dependency_tree.append(run_in_this_stage)
-    flattened_dependency_tree = sum(dependency_tree, [])
-    if len(flattened_dependency_tree) != len(runnable_suites):
-        raise DataError(
-            "Invalid test configuration: Circular or unmet dependencies detected between test suites. Please check your #DEPENDS definitions."
-        )
+    dependency_tree = create_dependency_tree(runnable_suites)
+    # Since groups cannot depend on others, they are placed at the beginning.
+    dependency_tree[0][0:0] = group_items
     return dependency_tree
 
 
 def _all_grouped_suites_by_depend(grouped_suites):
+    # type: (List[List[ExecutionItem]]) -> List[List[ExecutionItem]]
     grouped_by_depend = []
-    for group_suite in grouped_suites:
-        grouped_by_depend += _group_by_depend(group_suite)
+    for group_suite in grouped_suites:  # These groups are divided by #WAIT
+        grouped_by_depend.extend(_group_by_depend(group_suite))
     return grouped_by_depend