sirmordred 0.10.10rc1__tar.gz → 1.1.5rc1__tar.gz

{sirmordred-0.10.10rc1 → sirmordred-1.1.5rc1}/PKG-INFO

@@ -1,27 +1,26 @@
 Metadata-Version: 2.1
 Name: sirmordred
-Version: 0.10.10rc1
+Version: 1.1.5rc1
 Summary: Drive GrimoireLab tools to produce a dashboard
 Home-page: https://chaoss.github.io/grimoirelab/
 License: GPL-3.0+
 Keywords: development,grimoirelab
 Author: GrimoireLab Developers
-Requires-Python: >=3.8,<4.0
-Classifier: Development Status :: 4 - Beta
+Requires-Python: >=3.9,<4.0
+Classifier: Development Status :: 5 - Production/Stable
 Classifier: Intended Audience :: Developers
 Classifier: Intended Audience :: Science/Research
 Classifier: License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)
 Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.8
 Classifier: Programming Language :: Python :: 3.9
 Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
 Classifier: Topic :: Software Development
 Requires-Dist: cereslib (>=0.3)
-Requires-Dist: colorlog (==4.1.0)
+Requires-Dist: colorlog (>=6.4.1,<7.0.0)
 Requires-Dist: elasticsearch (==6.3.1)
 Requires-Dist: elasticsearch-dsl (==6.3.1)
-Requires-Dist: file-read-backwards (==2.0.0)
+Requires-Dist: file-read-backwards (>=3.1.0,<4.0.0)
 Requires-Dist: graal (>=0.3)
 Requires-Dist: grimoire-elk (>=0.102)
 Requires-Dist: grimoirelab-panels (>=0.1)
@@ -37,7 +36,7 @@ Project-URL: Bug Tracker, https://github.com/chaoss/grimoirelab-sirmordred/issue
 Project-URL: Repository, https://github.com/chaoss/grimoirelab-sirmordred
 Description-Content-Type: text/markdown
 
-# SirMordred [![Build Status](https://github.com/chaoss/grimoirelab-sirmordred/workflows/tests/badge.svg)](https://github.com/chaoss/grimoirelab-sirmordred/actions?query=workflow:tests+branch:master+event:push) [![Coverage Status](https://coveralls.io/repos/github/chaoss/grimoirelab-sirmordred/badge.svg?branch=master)](https://coveralls.io/github/chaoss/grimoirelab-sirmordred?branch=master) [![PyPI version](https://badge.fury.io/py/sirmordred.svg)](https://badge.fury.io/py/sirmordred)
+# SirMordred [![Build Status](https://github.com/chaoss/grimoirelab-sirmordred/workflows/tests/badge.svg)](https://github.com/chaoss/grimoirelab-sirmordred/actions?query=workflow:tests+branch:main+event:push) [![Coverage Status](https://coveralls.io/repos/github/chaoss/grimoirelab-sirmordred/badge.svg?branch=main)](https://coveralls.io/github/chaoss/grimoirelab-sirmordred?branch=main) [![PyPI version](https://badge.fury.io/py/sirmordred.svg)](https://badge.fury.io/py/sirmordred)
 
 SirMordred is the tool used to coordinate the execution of the GrimoireLab platform, via two main configuration files, the `setup.cfg` and `projects.json`, which are summarized in their corresponding sections.
 
@@ -143,7 +142,7 @@ The template of a backend section is shown above.
 Further information about Perceval backends parameters are available at:
 
 * Params details: https://perceval.readthedocs.io/en/latest/perceval/perceval-backends.html
-* Examples: https://github.com/chaoss/grimoirelab-sirmordred/blob/master/tests/test_studies.cfg
+* Examples: https://github.com/chaoss/grimoirelab-sirmordred/blob/main/tests/test_studies.cfg
 
 Note that some backend sections allow to specify specific enrichment options, which are listed below.
 
@@ -160,7 +159,7 @@ Note that some backend sections allow to specify specific enrichment options, wh
 
 A template of a study section is shown above. A complete list of studies parameters is available at:
 
-* https://github.com/chaoss/grimoirelab-sirmordred/blob/master/tests/test_studies.cfg
+* https://github.com/chaoss/grimoirelab-sirmordred/blob/main/tests/test_studies.cfg
 
 ## Projects.json [&uarr;](#contents)
 
@@ -1473,7 +1472,7 @@ studies = [enrich_demography:weblate] (optional)
 
 Micro Mordred is a simplified version of Mordred which omits the use of its scheduler. Thus, Micro Mordred allows running single Mordred tasks (e.g., raw collection, enrichment) per execution.
 
-Micro Mordred is located in the [sirmordred/utils](https://github.com/chaoss/grimoirelab-sirmordred/tree/master/sirmordred/utils/micro.py) folder of this same repository. It can be executed via command line, its parameters are summarized below:
+Micro Mordred is located in the [sirmordred/utils](https://github.com/chaoss/grimoirelab-sirmordred/tree/main/sirmordred/utils/micro.py) folder of this same repository. It can be executed via command line, its parameters are summarized below:
 ```
 --debug: execute Micro Mordred in debug mode
 

{sirmordred-0.10.10rc1 → sirmordred-1.1.5rc1}/README.md

@@ -1,4 +1,4 @@
-# SirMordred [![Build Status](https://github.com/chaoss/grimoirelab-sirmordred/workflows/tests/badge.svg)](https://github.com/chaoss/grimoirelab-sirmordred/actions?query=workflow:tests+branch:master+event:push) [![Coverage Status](https://coveralls.io/repos/github/chaoss/grimoirelab-sirmordred/badge.svg?branch=master)](https://coveralls.io/github/chaoss/grimoirelab-sirmordred?branch=master) [![PyPI version](https://badge.fury.io/py/sirmordred.svg)](https://badge.fury.io/py/sirmordred)
+# SirMordred [![Build Status](https://github.com/chaoss/grimoirelab-sirmordred/workflows/tests/badge.svg)](https://github.com/chaoss/grimoirelab-sirmordred/actions?query=workflow:tests+branch:main+event:push) [![Coverage Status](https://coveralls.io/repos/github/chaoss/grimoirelab-sirmordred/badge.svg?branch=main)](https://coveralls.io/github/chaoss/grimoirelab-sirmordred?branch=main) [![PyPI version](https://badge.fury.io/py/sirmordred.svg)](https://badge.fury.io/py/sirmordred)
 
 SirMordred is the tool used to coordinate the execution of the GrimoireLab platform, via two main configuration files, the `setup.cfg` and `projects.json`, which are summarized in their corresponding sections.
 
@@ -104,7 +104,7 @@ The template of a backend section is shown above.
 Further information about Perceval backends parameters are available at:
 
 * Params details: https://perceval.readthedocs.io/en/latest/perceval/perceval-backends.html
-* Examples: https://github.com/chaoss/grimoirelab-sirmordred/blob/master/tests/test_studies.cfg
+* Examples: https://github.com/chaoss/grimoirelab-sirmordred/blob/main/tests/test_studies.cfg
 
 Note that some backend sections allow to specify specific enrichment options, which are listed below.
 
@@ -121,7 +121,7 @@ Note that some backend sections allow to specify specific enrichment options, wh
 
 A template of a study section is shown above. A complete list of studies parameters is available at:
 
-* https://github.com/chaoss/grimoirelab-sirmordred/blob/master/tests/test_studies.cfg
+* https://github.com/chaoss/grimoirelab-sirmordred/blob/main/tests/test_studies.cfg
 
 ## Projects.json [↑](#contents)
 
@@ -1434,7 +1434,7 @@ studies = [enrich_demography:weblate] (optional)
 
 Micro Mordred is a simplified version of Mordred which omits the use of its scheduler. Thus, Micro Mordred allows running single Mordred tasks (e.g., raw collection, enrichment) per execution.
 
-Micro Mordred is located in the [sirmordred/utils](https://github.com/chaoss/grimoirelab-sirmordred/tree/master/sirmordred/utils/micro.py) folder of this same repository. It can be executed via command line, its parameters are summarized below:
+Micro Mordred is located in the [sirmordred/utils](https://github.com/chaoss/grimoirelab-sirmordred/tree/main/sirmordred/utils/micro.py) folder of this same repository. It can be executed via command line, its parameters are summarized below:
 ```
 --debug: execute Micro Mordred in debug mode
 

{sirmordred-0.10.10rc1 → sirmordred-1.1.5rc1}/pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "sirmordred"
-version = "0.10.10-rc.1"
+version = "1.1.5-rc.1"
 description = "Drive GrimoireLab tools to produce a dashboard"
 authors = [
     "GrimoireLab Developers"
@@ -23,7 +23,7 @@ packages = [
 ]
 
 classifiers = [
-   "Development Status :: 4 - Beta",
+   "Development Status :: 5 - Production/Stable",
    "Intended Audience :: Developers",
    "Intended Audience :: Science/Research",
    "Topic :: Software Development",
@@ -41,12 +41,12 @@ classifiers = [
 'panels_config.py' = 'sirmordred.utils.panels_config:main'
 
 [tool.poetry.dependencies]
-python = "^3.8"
+python = "^3.9"
 
-colorlog = "4.1.0"
+colorlog = "^6.4.1"
 elasticsearch = "6.3.1"
 elasticsearch-dsl = "6.3.1"
-file-read-backwards = "2.0.0"
+file-read-backwards = "^3.1.0"
 grimoirelab-toolkit = { version = ">=0.3", allow-prereleases = true}
 sortinghat = { version = ">=0.7.20", allow-prereleases = true}
 kidash = { version = ">=0.5", allow-prereleases = true}
@@ -62,8 +62,8 @@ graal = { version = ">=0.3", allow-prereleases = true}
 
 [tool.poetry.dev-dependencies]
 httpretty = "^1.1.4"
-flake8 = "^4.0.1"
-coverage = "^6.3.2"
+flake8 = "^7.1.1"
+coverage = "^7.2.3"
 
 [build-system]
 requires = ["poetry-core>=1.0.0"]

sirmordred-1.1.5rc1/sirmordred/_version.py

@@ -0,0 +1,2 @@
+# File auto-generated by semverup on 2025-01-15 11:22:44.910834
+__version__ = "1.1.5-rc.1"

{sirmordred-0.10.10rc1 → sirmordred-1.1.5rc1}/sirmordred/sirmordred.py

@@ -50,6 +50,7 @@ from sirmordred.task_identities import TaskIdentitiesMerge
 from sirmordred.task_manager import TasksManager
 from sirmordred.task_panels import TaskPanels, TaskPanelsMenu
 from sirmordred.task_projects import TaskProjects
+from sortinghat.cli.client import SortingHatClient
 
 logger = logging.getLogger(__name__)
 
@@ -198,7 +199,7 @@ class SirMordred:
             repos_backend = self._get_repos_by_backend()
             for backend in repos_backend:
                 # Start new Threads and add them to the threads list to complete
-                t = TasksManager(backend_tasks, backend, stopper, self.config, small_delay)
+                t = TasksManager(backend_tasks, backend, stopper, self.config, self.client, small_delay)
                 threads.append(t)
                 t.start()
 
@@ -206,7 +207,7 @@ class SirMordred:
         if len(global_tasks) > 0:
             # FIXME timer is applied to all global_tasks, does it make sense?
             # All tasks are executed in the same thread sequentially
-            gt = TasksManager(global_tasks, "Global tasks", stopper, self.config, big_delay)
+            gt = TasksManager(global_tasks, "Global tasks", stopper, self.config, self.client, big_delay)
             threads.append(gt)
             gt.start()
             if big_delay > 0:
@@ -248,14 +249,14 @@ class SirMordred:
         if self.conf['phases']['panels']:
             tasks = [TaskPanels, TaskPanelsMenu]
             stopper.set()
-            tm = TasksManager(tasks, "Global tasks", stopper, self.config)
+            tm = TasksManager(tasks, "Global tasks", stopper, self.config, self.client)
             tm.start()
             tm.join()
 
         logger.info("Loading projects")
         tasks = [TaskProjects]
         stopper.set()
-        tm = TasksManager(tasks, "Global tasks", stopper, self.config)
+        tm = TasksManager(tasks, "Global tasks", stopper, self.config, self.client)
         tm.start()
         tm.join()
         logger.info("Projects loaded")
@@ -280,7 +281,7 @@ class SirMordred:
 
         # check we have access to the needed ES
         if not self.check_es_access():
-            print('Can not access Elasticsearch service. Exiting sirmordred ...')
+            print('Can not access ElasticSearch/OpenSearch service. Exiting sirmordred ...')
             sys.exit(1)
 
         # If bestiary is configured check that it is working
@@ -289,6 +290,9 @@ class SirMordred:
                 print('Can not access bestiary service. Exiting sirmordred ...')
                 sys.exit(1)
 
+        # Create SortingHat Client
+        self.__create_sh_client(self.config)
+
         # Initial round: panels and projects loading
         self.__execute_initial_load()
 
@@ -336,3 +340,28 @@ class SirMordred:
                 logger.error(var)
 
         logger.info("Finished SirMordred engine ...")
+
+    def __create_sh_client(self, config):
+        self.config = config
+        self.conf = config.get_conf()
+
+        sortinghat = self.conf.get('sortinghat', None)
+        self.db_sh = sortinghat['database'] if sortinghat else None
+        self.db_user = sortinghat['user'] if sortinghat else None
+        self.db_password = sortinghat['password'] if sortinghat else None
+        self.db_host = sortinghat['host'] if sortinghat else '127.0.0.1'
+        self.db_path = sortinghat.get('path', None) if sortinghat else None
+        self.db_port = sortinghat.get('port', None) if sortinghat else None
+        self.db_ssl = sortinghat.get('ssl', False) if sortinghat else False
+        self.db_verify_ssl = sortinghat.get('verify_ssl', True) if sortinghat else True
+        self.db_tenant = sortinghat.get('tenant', True) if sortinghat else None
+        self.db_unaffiliate_group = sortinghat['unaffiliated_group'] if sortinghat else None
+        if sortinghat and not hasattr(self, 'client'):
+            self.client = SortingHatClient(host=self.db_host, port=self.db_port,
+                                           path=self.db_path, ssl=self.db_ssl,
+                                           user=self.db_user, password=self.db_password,
+                                           verify_ssl=self.db_verify_ssl,
+                                           tenant=self.db_tenant)
+            self.client.connect()
+        elif not sortinghat:
+            self.client = None

{sirmordred-0.10.10rc1 → sirmordred-1.1.5rc1}/sirmordred/task.py

@@ -29,7 +29,6 @@ import re
 from grimoire_elk.elk import get_ocean_backend
 from grimoire_elk.utils import get_connector_from_name, get_elastic
 from grimoire_elk.enriched.utils import grimoire_con
-from sortinghat.cli.client import SortingHatClient
 
 logger = logging.getLogger(__name__)
 
@@ -42,10 +41,11 @@ class Task():
                          'studies', 'node_regex', 'anonymize']
     PARAMS_WITH_SPACES = ['blacklist-jobs']
 
-    def __init__(self, config):
+    def __init__(self, config, sortinghat_client=None):
         self.backend_section = None
         self.config = config
         self.conf = config.get_conf()
+        self.client = sortinghat_client
 
         sortinghat = self.conf.get('sortinghat', None)
         self.db_sh = sortinghat['database'] if sortinghat else None
@@ -58,15 +58,6 @@ class Task():
         self.db_verify_ssl = sortinghat.get('verify_ssl', True) if sortinghat else True
         self.db_tenant = sortinghat.get('tenant', True) if sortinghat else None
         self.db_unaffiliate_group = sortinghat['unaffiliated_group'] if sortinghat else None
-        if sortinghat:
-            self.client = SortingHatClient(host=self.db_host, port=self.db_port,
-                                           path=self.db_path, ssl=self.db_ssl,
-                                           user=self.db_user, password=self.db_password,
-                                           verify_ssl=self.db_verify_ssl,
-                                           tenant=self.db_tenant)
-            self.client.connect()
-        else:
-            self.client = None
 
         self.grimoire_con = grimoire_con(conn_retries=12)  # 30m retry
 
@@ -156,9 +147,9 @@ class Task():
                 continue
 
             # If param is boolean, no values must be added
-            if type(section_param) == bool:
+            if isinstance(section_param, bool):
                 params.append("--" + p) if section_param else None
-            elif type(section_param) == list:
+            elif isinstance(section_param, list):
                 # '--blacklist-jobs', 'a', 'b', 'c'
                 # 'a', 'b', 'c' must be added as items in the list
                 params.append("--" + p)

{sirmordred-0.10.10rc1 → sirmordred-1.1.5rc1}/sirmordred/task_autorefresh.py

@@ -43,8 +43,8 @@ logger = logging.getLogger(__name__)
 class TaskAutorefresh(Task):
     """Refresh the last modified identities for all the backends."""
 
-    def __init__(self, config):
-        super().__init__(config)
+    def __init__(self, config, sortinghat_client):
+        super().__init__(config, sortinghat_client)
 
         self.last_autorefresh_backend = {}
 

{sirmordred-0.10.10rc1 → sirmordred-1.1.5rc1}/sirmordred/task_collection.py

@@ -22,13 +22,14 @@
 #
 
 import logging
-import time
 import traceback
 
 from grimoire_elk.elk import feed_backend
 from grimoire_elk.elastic_items import ElasticItems
 from grimoire_elk.elastic import ElasticSearch
 
+from grimoirelab_toolkit.datetime import datetime_utcnow
+
 from sirmordred.error import DataCollectionError
 from sirmordred.task import Task
 from sirmordred.task_projects import TaskProjects
@@ -40,8 +41,8 @@ logger = logging.getLogger(__name__)
 class TaskRawDataCollection(Task):
     """ Basic class shared by all collection tasks """
 
-    def __init__(self, config, backend_section=None, allowed_repos=None):
-        super().__init__(config)
+    def __init__(self, config, sortinghat_client=None, backend_section=None, allowed_repos=None):
+        super().__init__(config, sortinghat_client)
 
         self.backend_section = backend_section
         self.allowed_repos = set(allowed_repos) if allowed_repos else None
@@ -74,7 +75,7 @@ class TaskRawDataCollection(Task):
             logging.info('%s collect disabled', self.backend_section)
             return errors
 
-        t2 = time.time()
+        time_start = datetime_utcnow()
         logger.info('[%s] collection phase starts', self.backend_section)
         print("Collection for {}: starting...".format(self.backend_section))
         clean = False
@@ -142,8 +143,7 @@ class TaskRawDataCollection(Task):
                 raise DataCollectionError('Failed to collect data from %s' % url)
             logger.info('[%s] collection finished for %s', self.backend_section, self.anonymize_url(repo))
 
-        t3 = time.time()
-        spent_time = time.strftime("%H:%M:%S", time.gmtime(t3 - t2))
+        spent_time = str(datetime_utcnow() - time_start).split('.')[0]
         logger.info('[%s] collection phase finished in %s',
                     self.backend_section, spent_time)
         print("Collection for {}: finished after {} hours".format(self.backend_section,

{sirmordred-0.10.10rc1 → sirmordred-1.1.5rc1}/sirmordred/task_enrich.py

@@ -53,8 +53,8 @@ logger = logging.getLogger(__name__)
 class TaskEnrich(Task):
     """ Basic class shared by all enriching tasks """
 
-    def __init__(self, config, backend_section=None, allowed_repos=None):
-        super().__init__(config)
+    def __init__(self, config, sortinghat_client, backend_section=None, allowed_repos=None):
+        super().__init__(config, sortinghat_client)
         self.backend_section = backend_section
         self.allowed_repos = set(allowed_repos) if allowed_repos else None
         # This will be options in next iteration
@@ -249,12 +249,16 @@ class TaskEnrich(Task):
         next_autorefresh = self.__update_last_autorefresh()
 
         logger.debug('Getting last modified identities from SH since %s for %s', after, self.backend_section)
+
         author_fields = ["author_uuid"]
+        for role in enrich_backend.roles:
+            author_fields.append(role + '_uuid')
         try:
             meta_fields = enrich_backend.meta_fields
             author_fields += meta_fields
         except AttributeError:
             pass
+
         logger.debug("Refreshing identity ids for %s", self.backend_section)
         total = 0
         time_start = datetime.now()

{sirmordred-0.10.10rc1 → sirmordred-1.1.5rc1}/sirmordred/task_identities.py

@@ -38,8 +38,8 @@ logger = logging.getLogger(__name__)
 class TaskIdentitiesMerge(Task):
     """ Task for processing identities in SortingHat """
 
-    def __init__(self, conf):
-        super().__init__(conf)
+    def __init__(self, conf, sortinghat_client):
+        super().__init__(conf, sortinghat_client)
         self.last_autorefresh = datetime.utcnow()  # Last autorefresh date
 
     def is_backend_task(self):

{sirmordred-0.10.10rc1 → sirmordred-1.1.5rc1}/sirmordred/task_manager.py

@@ -50,7 +50,7 @@ class TasksManager(threading.Thread):
     IDENTITIES_TASKS_ON_LOCK = threading.Lock()
     IDENTITIES_TASKS_ON = False
 
-    def __init__(self, tasks_cls, backend_section, stopper, config, timer=0):
+    def __init__(self, tasks_cls, backend_section, stopper, config, sortinghat_client, timer=0):
         """
         :tasks_cls : tasks classes to be executed using the backend
         :backend_section: perceval backend section name
@@ -64,6 +64,7 @@ class TasksManager(threading.Thread):
         self.stopper = stopper  # To stop the thread from parent
         self.timer = timer
         self.thread_id = None
+        self.client = sortinghat_client
 
     def add_task(self, task):
         self.tasks.append(task)
@@ -80,7 +81,7 @@ class TasksManager(threading.Thread):
         logger.debug(self.tasks_cls)
         for tc in self.tasks_cls:
             # create the real Task from the class
-            task = tc(self.config)
+            task = tc(self.config, self.client)
             task.set_backend_section(self.backend_section)
             self.tasks.append(task)
 

{sirmordred-0.10.10rc1 → sirmordred-1.1.5rc1}/sirmordred/task_panels.py

@@ -264,8 +264,8 @@ class TaskPanels(Task):
     # Panels to be uploaded always, no matter the data sources configured
     panels_common = panels_multi_ds + ["panels/json/about.json"]
 
-    def __init__(self, conf):
-        super().__init__(conf)
+    def __init__(self, conf, sortinghat_client=None):
+        super().__init__(conf, sortinghat_client)
         # Read panels and menu description from yaml file
         with open(self.conf['general']['menu_file'], 'r') as f:
             try:
@@ -473,8 +473,8 @@ class TaskPanelsMenu(Task):
         }
     }
 
-    def __init__(self, conf):
-        super().__init__(conf)
+    def __init__(self, conf, sortinghat_client=None):
+        super().__init__(conf, sortinghat_client)
         # Read panels and menu description from yaml file """
         with open(self.conf['general']['menu_file'], 'r') as f:
             try:

{sirmordred-0.10.10rc1 → sirmordred-1.1.5rc1}/sirmordred/utils/micro.py

@@ -33,6 +33,7 @@ from sirmordred.task_identities import TaskIdentitiesMerge
 from sirmordred.task_enrich import TaskEnrich
 from sirmordred.task_panels import TaskPanels, TaskPanelsMenu
 from sirmordred.task_projects import TaskProjects
+from sortinghat.cli.client import SortingHatClient
 
 COLOR_LOG_FORMAT_SUFFIX = "\033[1m %(log_color)s "
 LOG_COLORS = {'DEBUG': 'white', 'INFO': 'cyan', 'WARNING': 'yellow', 'ERROR': 'red', 'CRITICAL': 'red,bg_white'}
@@ -50,6 +51,34 @@ def main():
                   args.panels)
 
 
+def create_sortinghat_client(config):
+    """Create a SortingHat client"""
+
+    conf = config.get_conf()
+
+    sortinghat = conf.get('sortinghat', None)
+    if not sortinghat:
+        return None
+
+    db_user = sortinghat['user'] if sortinghat else None
+    db_password = sortinghat['password'] if sortinghat else None
+    db_host = sortinghat['host'] if sortinghat else '127.0.0.1'
+    db_path = sortinghat.get('path', None) if sortinghat else None
+    db_port = sortinghat.get('port', None) if sortinghat else None
+    db_ssl = sortinghat.get('ssl', False) if sortinghat else False
+    db_verify_ssl = sortinghat.get('verify_ssl', True) if sortinghat else True
+    db_tenant = sortinghat.get('tenant', True) if sortinghat else None
+
+    client = SortingHatClient(host=db_host, port=db_port,
+                              path=db_path, ssl=db_ssl,
+                              user=db_user, password=db_password,
+                              verify_ssl=db_verify_ssl,
+                              tenant=db_tenant)
+    client.connect()
+
+    return client
+
+
 def micro_mordred(config, backend_sections, repos_to_check, raw, identities_merge, enrich, panels):
     """Execute the Mordred tasks using the configuration file.
 
@@ -66,12 +95,17 @@ def micro_mordred(config, backend_sections, repos_to_check, raw, identities_merg
         for backend in backend_sections:
             get_raw(config, backend, repos_to_check)
 
+    if identities_merge or enrich:
+        sortinghat_client = create_sortinghat_client(config)
+    else:
+        sortinghat_client = None
+
     if identities_merge:
-        get_identities_merge(config)
+        get_identities_merge(config, sortinghat_client)
 
     if enrich:
         for backend in backend_sections:
-            get_enrich(config, backend, repos_to_check)
+            get_enrich(config, sortinghat_client, backend, repos_to_check)
 
     if panels:
         get_panels(config)
@@ -97,29 +131,31 @@ def get_raw(config, backend_section, repos_to_check=None):
         sys.exit(-1)
 
 
-def get_identities_merge(config):
+def get_identities_merge(config, sortinghat_client):
     """Execute the merge identities phase
 
+    :param sortinghat_client: a SortingHat client
     :param config: a Mordred config object
     """
     TaskProjects(config).execute()
-    task = TaskIdentitiesMerge(config)
+    task = TaskIdentitiesMerge(config, sortinghat_client)
     task.execute()
     logging.info("Merging identities finished!")
 
 
-def get_enrich(config, backend_section, repos_to_check=None):
+def get_enrich(config, sortinghat_client, backend_section, repos_to_check=None):
     """Execute the enrich phase for a given backend section
 
     Repos are only checked if they are in BOTH `repos_to_check` and the `projects.json`
 
     :param config: a Mordred config object
+    :param sortinghat_client: a SortingHat client
     :param backend_section: the backend section where the enrich phase is executed
     :param repos_to_check: A list of repo URLs to check, or None to check all repos
     """
 
     TaskProjects(config).execute()
-    task = TaskEnrich(config, backend_section=backend_section, allowed_repos=repos_to_check)
+    task = TaskEnrich(config, sortinghat_client, backend_section=backend_section, allowed_repos=repos_to_check)
     try:
         task.execute()
         logging.info("Loading enriched data finished!")

{sirmordred-0.10.10rc1 → sirmordred-1.1.5rc1}/tests/test_task.py

@@ -33,6 +33,8 @@ sys.path.insert(0, '..')
 from sirmordred.config import Config
 from sirmordred.task import Task
 
+from sortinghat.cli.client import SortingHatClient
+
 CONF_FILE = 'test.cfg'
 BACKEND_NAME = 'stackexchange'
 COLLECTION_URL_STACKEXCHANGE = 'http://127.0.0.1:9200'
@@ -47,14 +49,23 @@ def read_file(filename, mode='r'):
 
 class TestTask(unittest.TestCase):
     """Task tests"""
+    def setUp(self):
+        self.config = Config(CONF_FILE)
+        self.conf = self.config.get_conf()
+        sh = self.conf.get('sortinghat')
+        self.sortinghat_client = SortingHatClient(host=sh['host'], port=sh.get('port', None),
+                                                  path=sh.get('path', None), ssl=sh.get('ssl', False),
+                                                  user=sh['user'], password=sh['password'],
+                                                  verify_ssl=sh.get('verify_ssl', True),
+                                                  tenant=sh.get('tenant', True))
+        self.sortinghat_client.connect()
 
     def test_initialization(self):
         """Test whether attributes are initializated"""
 
-        config = Config(CONF_FILE)
-        task = Task(config)
+        task = Task(self.config, self.sortinghat_client)
 
-        self.assertEqual(task.config, config)
+        self.assertEqual(task.config, self.config)
         self.assertEqual(task.db_sh, task.conf['sortinghat']['database'])
         self.assertEqual(task.db_user, task.conf['sortinghat']['user'])
         self.assertEqual(task.db_password, task.conf['sortinghat']['password'])
@@ -63,15 +74,13 @@ class TestTask(unittest.TestCase):
     def test_run(self):
         """Test whether the Task could be run"""
 
-        config = Config(CONF_FILE)
-        task = Task(config)
+        task = Task(self.config, self.sortinghat_client)
         self.assertEqual(task.execute(), None)
 
     def test_compose_p2o_params(self):
         """Test whether p2o params are built correctly for a backend and a repository"""
 
-        config = Config(CONF_FILE)
-        task = Task(config)
+        task = Task(self.config, self.sortinghat_client)
         params = task._compose_p2o_params("stackexchange", "https://stackoverflow.com/questions/tagged/example")
         self.assertDictEqual(params, {'url': "https://stackoverflow.com/questions/tagged/example"})
 
@@ -92,8 +101,7 @@ class TestTask(unittest.TestCase):
     def test_extract_repo_tags(self):
         """Test the extraction of tags in repositories"""
 
-        config = Config(CONF_FILE)
-        task = Task(config)
+        task = Task(self.config, self.sortinghat_client)
         url, tags = task._extract_repo_tags("git", "https://github.com/zhquan_example/repo --labels=[ENG, SUPP]")
         self.assertEqual(url, "https://github.com/zhquan_example/repo")
         self.assertListEqual(tags, ["ENG", "SUPP"])
@@ -113,8 +121,7 @@ class TestTask(unittest.TestCase):
 
         expected_repo_params = json.loads(read_file('data/task-params-expected'))
 
-        config = Config(CONF_FILE)
-        task = Task(config)
+        task = Task(self.config, self.sortinghat_client)
 
         for backend in expected_repo_params.keys():
             repo = expected_repo_params.get(backend)['repo']
@@ -126,8 +133,7 @@ class TestTask(unittest.TestCase):
     def test_get_collection_url(self):
         """Test whether the collection url could be overwritten in a backend"""
 
-        config = Config(CONF_FILE)
-        task = Task(config)
+        task = Task(self.config, self.sortinghat_client)
         task.backend_section = "stackexchange"
 
         self.assertEqual(task._get_collection_url(), COLLECTION_URL_STACKEXCHANGE)