ciocore 7.0.2b5__py2.py3-none-any.whl → 8.0.0__py2.py3-none-any.whl

{ciocore-7.0.2b5.dist-info → ciocore-8.0.0.dist-info}/METADATA

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: ciocore
-Version: 7.0.2b5
+Version: 8.0.0
 Summary: Core functionality for Conductor's client tools
 Home-page: https://github.com/ConductorTechnologies/ciocore
 Author: conductor
@@ -9,21 +9,22 @@ Classifier: Operating System :: OS Independent
 Classifier: Programming Language :: Python
 Classifier: Topic :: Multimedia :: Graphics :: 3D Rendering
 Description-Content-Type: text/markdown
-Requires-Dist: certifi ==2019.11.28
+Requires-Dist: requests[use_chardet_on_py3] ==2.28.1
 Requires-Dist: pyjwt ==1.7.1
-Requires-Dist: requests <=2.25.1,>=2.10.0
 Requires-Dist: future >=0.18.2
-Requires-Dist: urllib3 ==1.25.11
+Requires-Dist: cioseq <1.0.0,>=0.4.1
+Requires-Dist: Click <9.0.0,>=8.1.3
+Requires-Dist: markdown <4.0.0,>=3.5.2
+Requires-Dist: colorlog <7.0.0,>=6.8.2
 
-# Conductor Core
+## Introduction
 
-Use the command-line uploader and downloader or develop your own tools using the Python API.
+Conductor Core is a Python package that provides an API and a command-line interface for interacting with Conductor.
 
 ## Install
 
-**To install the latest version.**
 ```bash
-pip install --upgrade ciocore
+pip install ciocore
 ```
 
 Run the conductor command to confirm the package was installed.
@@ -31,6 +32,13 @@ Run the conductor command to confirm the package was installed.
 conductor --help
 ```
 
+Get help for a specific sub command
+```bash
+conductor downloader --help
+conductor uploader --help
+conductor software --help
+```
+
 Example API usage
 ```python
 from ciocore import api_client
@@ -43,14 +51,24 @@ See [CONTRIBUTING](CONTRIBUTING.md)
 
 ## Changelog
 
-## Unreleased: 
-* 7.0.2-beta.5
-  * fix bug in package tree where the platform was not being set correctly
-  * make tests runnable in python 2.7
-  * make all tests pass in python 2.7 and 3.9
-* 7.0.2-beta.3
-  * rewrite inst type descriptions
-  * filter out bad 3dsmax entries from the packages
+## Version:8.0.0 -- 07 Apr 2024
+
+* The `conductor` commandline interface (CLI) has undergone a complete overhaul to improve its functionality and performance. It is now installed as a Python console script, which simplifies the installation process on all platforms and ensures that the tool is always accessible. In addition, this standardized installation allows for better integration into our Companion application and potentially customers' own proprietary tools.
+* The commandline downloader has been rewritten to improve its performance and reliability. In previous versions, the downloader would fetch all download URLs from the server before starting any downloads. This approach was inefficient and would lead to performance issues when handling jobs with large numbers of tasks. The new downloader fetches tasks in small batches, which allows the download to start almost immediately, regardless of the number of tasks. It provides several new commandline flags, such as filtering based on filenames, and better logging, giving users more control and visibility over the download process. In addition, the new downloader has been written as an event based system that can be seamlessly integrated into other pipeline tools and GUIs. Pipeline TDs need only to register handlers to any of the events that the downloader emits.
+* The `conductor` commandline now supports a new subcommand, `docs`, which opens API and commandline documentation in a local web browser. The documentation is generated using `mkdocs` and is included in the package. This command is useful as it ensures the **API** documentation is relevant to the version of the tool being used.
+* The `conductor` commandline now supports a new subcommand, `packages`, which displays all available software packages on Conductor's cloud. The command can output the document as text, markdown, or nicely formatted HTML. The data is pulled from the Conductor server and is always up-to-date.
+* The Python API now provides four new endpoints: `kill_tasks`, `kill_jobs`, `get_jobs`, and `get_log`.
+* The Python API now provides access to entries in the Dashboard's Extra Environment Variables section. This feature allows administrators to set environment variables that are passed to all jobs run on the cloud. It's useful for setting up access to license servers or other shared resources. The API is open to clients so that they may insert account-widse variables with the correct level of precedence. 
+* In addition, the environment variable schema has been extended to support a merge policy of `prepend`. This policy allows users to add new values to the beginning of the list of values for an environment variable and is useful for search paths.
+
+
+## Version:7.0.2 -- 10 Nov 2023
+
+* Bug fix: Fixed an issue where a list was copied with a Python3 only construct.
+* Compatibility improvement: Tests can now be run in Python 2.7 as well.
+* Cross-version support: Ensured that all tests pass successfully in both Python 2.7 and 3.9.
+* Improvement: Rewrite instance type descriptions on the client. Ensure uniqueness.
+* Workaround: Removed invalid 3dsmax entries from the packages response.
 
 ## Version:7.0.1 -- 30 Oct 2023
 

ciocore-8.0.0.dist-info/RECORD

@@ -0,0 +1,127 @@
+ciocore/VERSION,sha256=aiaXYaVWcRcFJ54x0J1ICf_ev0aM-5SwdkDiuIfIUqU,5
+ciocore/__init__.py,sha256=aTP7LeeosQA8BZE67gDV4jgfTK5zxmwZRjiTRu_ZWj0,646
+ciocore/api_client.py,sha256=TyHXGmK4uTmKV93O_IpAvttZhbexIIZTGP-IVn_WyWM,24599
+ciocore/cli.py,sha256=9OdoazUovfUHxpiAHahH7HJjJcdRLO0_QsdPXjGjlW8,15509
+ciocore/client_db.py,sha256=tTz3bl2xeDPPcYSDS3g3QgV_xYihJMx0Kj6OeN2klK0,12978
+ciocore/common.py,sha256=mBIS6KiYoQsjWe6aIFUGRRvCMl8BIN2kmLZ4J8icap8,14982
+ciocore/compat.py,sha256=5uEXPSog_jxsDMaHBswAKEtfyXT25VgU6WNGIhz9PHU,256
+ciocore/conductor_submit.py,sha256=ONE0LsA5hGavTJIOXXYx8qzl8_vBPADwhd6Ytq_0E0c,9382
+ciocore/config.py,sha256=rCL7kaFn1tYgSglN8q9Wx6SwMpoXTq0BMQGwPRVwVIg,8973
+ciocore/data.py,sha256=Ji0qUk8nJXBNakoHSqBiVx8O58SbZXyt273SHlEDn3U,7027
+ciocore/exceptions.py,sha256=4Oq-WX-qiN6kPUdBCHvvd6mtSQ0nCkDbJxWt2CNtpv8,1504
+ciocore/file_utils.py,sha256=swA7th9WhDEloW69YViRTKB-oeC9UmNdEGegfH1r-Gw,17176
+ciocore/hardware_set.py,sha256=FlRQiGCLRcSW7Oko_gzgVK8ZqJ_J92eT8e_AleAbS2E,17047
+ciocore/loggeria.py,sha256=2xdQRFb9NyXynU2O_pSOszJWcpoHgPwTUWJvERg7ODY,15251
+ciocore/package_environment.py,sha256=MEHV7jfs3NJIEYCIaW8JfJdBmelvPHZMmBzPlXETiRo,7808
+ciocore/package_query.py,sha256=2m5EBXfu1lmqupZrFF8f8mfkX_PgijpdMxCtFI5e5s0,5574
+ciocore/package_tree.py,sha256=vkORKXxQ7dO8l0_96eFwm-5AUVL0rP9bhgWYhW_v3lo,15649
+ciocore/post_install.py,sha256=zu5Ctz2ANbKD-f5G2ODLIhKkWENBi4F3UKKu50OEWrg,1000
+ciocore/retry.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+ciocore/validator.py,sha256=f_K7gxz122W_i5AxVx6dKhckOygl8TnmQiVj7tyX5zw,2344
+ciocore/worker.py,sha256=h6JophGBWqLdLdvphlzkkVMt0ojLw7_1Vrz4dikPRYE,21743
+ciocore/auth/__init__.py,sha256=cdS-xZzMq41yXM5cz8sUlcYgo8CJYh8HcCCWmhbDgf0,606
+ciocore/auth/server.py,sha256=8btX9-EokUl6q55V8muDmEV2tvvbTBD0BHeWFbwkzUc,3892
+ciocore/docsite/404.html,sha256=DQCrQsaKh8M0zlCFZmjFHUtqy7_gx_i2Rpc3OoDdDI4,17207
+ciocore/docsite/index.html,sha256=p_Dq_6_8cwZZPDI5zjahfCWUkfbSj14NJtoQFAZv0WI,20945
+ciocore/docsite/logo.png,sha256=gArgFFWdw8w985-0TkuGIgU_pW9sziEMZdqytXb5WLo,2825
+ciocore/docsite/objects.inv,sha256=s2FKStLlVIQbfG7U2-nw-7rz4unvd1W0u00YtLBxAKo,758
+ciocore/docsite/sitemap.xml,sha256=M_V85zl0y2adRvzJAnoCxlZH_Hl7TLnIb1A-6l_xGmI,109
+ciocore/docsite/sitemap.xml.gz,sha256=DLMqiMagzHE-fj8auM7l5Hw-22HXwKT5x6XfgNtEg28,127
+ciocore/docsite/apidoc/api_client/index.html,sha256=mHDq-zdWuilDKUluzmQNdOsImRoGEnnK5Ul2ApajgdU,170218
+ciocore/docsite/apidoc/apidoc/index.html,sha256=NQn8wjapxa7O2IQhsbE7Y-VMfMFL6Tby-L7qIF6K3m4,26171
+ciocore/docsite/apidoc/config/index.html,sha256=1GWkyClM7LJlLBpx07BqCi3xDkU-SKVPjNUEeKuwbNs,72559
+ciocore/docsite/apidoc/data/index.html,sha256=9DonjpASPYZGYRHjyaodkK3AxSlTHsvm1xdq2cqMxPI,50850
+ciocore/docsite/apidoc/hardware_set/index.html,sha256=DMlpzQCy2ypJ078tcFNrZAvG2sU7P5-ZwU4ZRCObtQg,123042
+ciocore/docsite/apidoc/package_environment/index.html,sha256=M2L6Pu00jYj5AdhHQXrNZjOKVWCXYtgL8I34zL6B0i4,69248
+ciocore/docsite/apidoc/package_tree/index.html,sha256=wKW5dsHBlcDyvCGakSNzYHX8z99F1Vhu2NyqAW6c1ko,109393
+ciocore/docsite/assets/_mkdocstrings.css,sha256=K3bqYEmxlOHQ3-M11JNbBWHCBDBLarkFRm8HuEYrAG4,341
+ciocore/docsite/assets/images/favicon.png,sha256=AjhUxD_Eslt5XuSVHIAZ494Fk__rb5GLXR8qm0elfP4,1870
+ciocore/docsite/assets/javascripts/bundle.4e31edb1.min.js,sha256=vMxCR_BtNIcbmbPV6j8Z-YDLQ9ckt4RzvGuzCTg034s,97250
+ciocore/docsite/assets/javascripts/bundle.4e31edb1.min.js.map,sha256=8Oyq3j4kjJ2wbHtEGTiDI5Svk1xp2kEnoKKUiYA_geM,890964
+ciocore/docsite/assets/javascripts/lunr/tinyseg.js,sha256=GwRMI5YQ72X87_YDlWdEK7FpLUE_Xuhob-6BsaMGd2E,22878
+ciocore/docsite/assets/javascripts/lunr/wordcut.js,sha256=PTj95orzSJFSiORe6zOK-CVshhH6LP7zB-arlYCLNWc,677463
+ciocore/docsite/assets/javascripts/lunr/min/lunr.ar.min.js,sha256=iaHcsJSoSR2WjBUaslSgMZXIf_KtqiQIx2mSgoSEcSU,17074
+ciocore/docsite/assets/javascripts/lunr/min/lunr.da.min.js,sha256=KhJzQEjq504KBeXBCP3yTCtx382NpxIzbKnj0nq5KVY,4654
+ciocore/docsite/assets/javascripts/lunr/min/lunr.de.min.js,sha256=vKp8XBm8cwwFu5HiKuDjhuGsSgWAxCt9lPm4dXqtras,6119
+ciocore/docsite/assets/javascripts/lunr/min/lunr.du.min.js,sha256=RS2gT-RJucZcYThP7FgtxjgkvU5Vn64sOSaneQto6XY,6208
+ciocore/docsite/assets/javascripts/lunr/min/lunr.es.min.js,sha256=3sTskL1vR4yj5lY2qOP-31ErI9AaUaAdfngi9n67qvg,11499
+ciocore/docsite/assets/javascripts/lunr/min/lunr.fi.min.js,sha256=aBVB_PWxmHrYsAdm6cBjiez5nD166Bf6iR-SP2kbhmg,9342
+ciocore/docsite/assets/javascripts/lunr/min/lunr.fr.min.js,sha256=TdVqolUedwrSbvS4UJLNMUR2ceDoSzlRQnhUf1QXc3c,10669
+ciocore/docsite/assets/javascripts/lunr/min/lunr.hi.min.js,sha256=Yx__UuNiPy37o1rwdsa4taRQ7v_jClz6x0WNq3fL4KY,3383
+ciocore/docsite/assets/javascripts/lunr/min/lunr.hu.min.js,sha256=h2yWm79KK4Rzj04KFJIu4zHLfCGTpiH2Xbf4Nv3TOjg,9437
+ciocore/docsite/assets/javascripts/lunr/min/lunr.hy.min.js,sha256=DvvLkCvbfsDd3_peK6NHt0xJ_8X0P9c__tg1KOuhLpU,1264
+ciocore/docsite/assets/javascripts/lunr/min/lunr.it.min.js,sha256=AKUHPfLpIuX97RxaTUARrsAkcqudkYKnxlCNYLYlxFM,11232
+ciocore/docsite/assets/javascripts/lunr/min/lunr.ja.min.js,sha256=xP3tMI9jNoGm7VO2wQ2MpZFv39UPa4LskU1yW-DLi9Y,2313
+ciocore/docsite/assets/javascripts/lunr/min/lunr.jp.min.js,sha256=BBFO3DbcEU7c5a-7PH95X0XHfYZzwe3NylZ13atdUZk,36
+ciocore/docsite/assets/javascripts/lunr/min/lunr.kn.min.js,sha256=oGTQ10ODNJV6RCtiYUD7cvTt6OvhfrXE4laC2HyIu_g,3494
+ciocore/docsite/assets/javascripts/lunr/min/lunr.ko.min.js,sha256=c9G9sd5K72xZIPUWkkxascRA21Cm9cYSa9D7fPXTdmQ,7972
+ciocore/docsite/assets/javascripts/lunr/min/lunr.multi.min.js,sha256=l2H0da5582Qg_3FN-ZUO-yZO_8-SMO_VW1EaDeiZ6Xk,817
+ciocore/docsite/assets/javascripts/lunr/min/lunr.nl.min.js,sha256=OZ7ybWHtLVgG9Y6nI9-soJ9KfJtR4xW90XSCyOMBOj4,6026
+ciocore/docsite/assets/javascripts/lunr/min/lunr.no.min.js,sha256=V_nbkpD57TYIB1X1PuNCoE9VpZVOSt2RXRwwGjdGRXM,4754
+ciocore/docsite/assets/javascripts/lunr/min/lunr.pt.min.js,sha256=OiXSq6ojBjzxP8VsGa5MxQUMtiFRFenQj0pvS3h4AGc,10171
+ciocore/docsite/assets/javascripts/lunr/min/lunr.ro.min.js,sha256=z6JqjqQpPYHoVHuiMh1wOg_piB8HIfyl8zo_gL8Vbuk,10958
+ciocore/docsite/assets/javascripts/lunr/min/lunr.ru.min.js,sha256=UO88D6tKPy0mDolzxjNjqg3PArVZ6p0R1fArYZdYXBc,10331
+ciocore/docsite/assets/javascripts/lunr/min/lunr.sa.min.js,sha256=QcVYpDVcPq6PCieuhwcSc48UsL2IA4BCrIq4tBHlgX8,4901
+ciocore/docsite/assets/javascripts/lunr/min/lunr.stemmer.support.min.js,sha256=LnITYFhGoYAU7HUJEMsYqKTGC7IyABA7P62e8nsQxwU,3647
+ciocore/docsite/assets/javascripts/lunr/min/lunr.sv.min.js,sha256=lCsWW8yCGOMkb8RzWvmxwvZlFDZtAAC-BRvxu4ltQT4,4523
+ciocore/docsite/assets/javascripts/lunr/min/lunr.ta.min.js,sha256=F6MtxJwnydJ5RWJafo0GbUL2TqHOWls03kJ3I_v5rio,2406
+ciocore/docsite/assets/javascripts/lunr/min/lunr.te.min.js,sha256=B5yPzwKK7vOI6Y7IYHyOpQupZgxGeXyKEZ0F-PggWDg,2330
+ciocore/docsite/assets/javascripts/lunr/min/lunr.th.min.js,sha256=NIRhJmhD4xayaCAASpbT_YehV5gp0kTOUQwXKT7vBDk,1031
+ciocore/docsite/assets/javascripts/lunr/min/lunr.tr.min.js,sha256=6L9XsQM7i9kuoEqUxTMBWqGn8KjWgwGUuXTpTh5Kyjw,15009
+ciocore/docsite/assets/javascripts/lunr/min/lunr.vi.min.js,sha256=LjnD_XMF9uZvwEJHlqdpbLjjbcD1w6g-obxE3-P-y1Y,784
+ciocore/docsite/assets/javascripts/lunr/min/lunr.zh.min.js,sha256=2MW3LDbsnDGXYCVG6bpbV62j9yXVZKVRvxbquvtUjog,2158
+ciocore/docsite/assets/javascripts/workers/search.dfff1995.min.js,sha256=dSn8S0AZNSnDklowowFe_v5geq6WOXg3zQiH_JuRUTY,39431
+ciocore/docsite/assets/javascripts/workers/search.dfff1995.min.js.map,sha256=WRu84gT6TJxoh21TdmByytMRPpYv8PHzHKdSvS1psrM,214982
+ciocore/docsite/assets/stylesheets/main.83068744.min.css,sha256=gwaHRErdgDoSsYabs3TkMGBpuUc9kTXNLWwucSPCqZ0,122334
+ciocore/docsite/assets/stylesheets/main.83068744.min.css.map,sha256=q8dWQkgGSfCcYT2Le1J0Tmxpcqxol5kZKaokp9d8Af0,42278
+ciocore/docsite/assets/stylesheets/palette.ecc896b0.min.css,sha256=7MiWsGpIt19TLduugHr1OQv1TJ9rEsndnkc2V8D9VjM,12245
+ciocore/docsite/assets/stylesheets/palette.ecc896b0.min.css.map,sha256=T8ZT8Mw8ws4vxyx_b1QAwyC9_pnv7r0KVaUdu2ST7ug,3639
+ciocore/docsite/cmdline/docs/index.html,sha256=2SFe5X1oq58UDuYBvUXCcgSBseP5AMgfu-ldKKK1WXA,20051
+ciocore/docsite/cmdline/downloader/index.html,sha256=DAVErhWEPuEtkQr7y9D_wjylzM9j4nWIWvTi4_gMoaA,23171
+ciocore/docsite/cmdline/packages/index.html,sha256=j1YeMgdwZ6FyJGJii05AdtUWKRjyu1fOr3_bKJsLfEU,20923
+ciocore/docsite/cmdline/uploader/index.html,sha256=y0728lIAYC4b9mfpjNVfwQqXoqJrsdG49UY5u7B1p94,25123
+ciocore/docsite/how-to-guides/index.html,sha256=OFrFBTb9tuMn60Jd4oeM10Mw-d-0pyqwoKoo9g1biBM,20100
+ciocore/docsite/search/search_index.json,sha256=ZMAbHLAlwOTujgA32XJTA7SHogRYh9i-ioPFOxjjJ0s,182633
+ciocore/docsite/stylesheets/extra.css,sha256=_Cxe9Dhg1BBi6Kqaz_iZD9z9VyqxA9vtONRjP4PVic0,354
+ciocore/docsite/stylesheets/tables.css,sha256=LE_zwGRxGcdPIy-9QiVPecOzlEBSqZb_WP5vDkFE0ZM,3235
+ciocore/downloader/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+ciocore/downloader/base_downloader.py,sha256=9PVmP1qy-dop4MbVsmLFvf5W6LKQLUKHPDYE6AEt71g,25784
+ciocore/downloader/download_runner_base.py,sha256=cWTWKhFX5FLG84Wh8_s7xpk3n0nmQ4fDiD2rYV6a3u0,1612
+ciocore/downloader/job_downloader.py,sha256=HAhr95RyCsj3KTp5W58Znc4JmqHY-mrM5b8vo6kRIJ0,5697
+ciocore/downloader/legacy_downloader.py,sha256=LIbT3BxuY56lq_UW_io9FSgK_2VIKwFVMENyF-5KllA,51158
+ciocore/downloader/log.py,sha256=WCRNx0LObx8JBO5MQucNHQGBXMgSBLGdH0VALN8dFDo,2105
+ciocore/downloader/logging_download_runner.py,sha256=iOuW8OvfAYW5MpP9zpUXh94qkWLC0GeG4PDGH7JSqJM,2865
+ciocore/downloader/perpetual_downloader.py,sha256=cD7lnBH75-c-ZVVPHZc1vSnDhgJOnGlPT85zn7IjGgA,2105
+ciocore/downloader/registry.py,sha256=_JIOuqpWkJkgJGN33nt-DCvqN9Gw3xeFhzPq4RUxIoE,2903
+ciocore/downloader/reporter.py,sha256=p1NK9k6iQ-jt7lRvZR0xFz0cGb2yo8tQcjlvYKR9SWM,4501
+ciocore/uploader/__init__.py,sha256=hxRFJf5Lo86rtRObFXSjjot8nybQd-SebSfYCbgZwow,24
+ciocore/uploader/_uploader.py,sha256=AaCwRo1rGkdMA-XACx9k4Z73R97ndJUVMemUZNV-Ark,37353
+ciocore/uploader/upload_stats/__init__.py,sha256=Lg1y4zq1i0cwc6Hh2K1TAQDYymLff49W-uIo1xjcvdI,5309
+ciocore/uploader/upload_stats/stats_formats.py,sha256=giNirtObU66VALWghPFSRhg3q_vw5MvESsnXhb_I3y8,2402
+tests/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+tests/extra_env_fixtures.py,sha256=8qvU4d8SXGKzRVNR5whVqKCQOwOVMiFVfbKBAjxa2gE,1119
+tests/instance_type_fixtures.py,sha256=uIzQduqKQVgjllMuyXaYnRC-pwqk5lnTx3NY2M5Nujo,4320
+tests/package_fixtures.py,sha256=CsJnhB7oYzIxJH7b1tCOPyvnnVSCqEbSPhtCnsHL-nA,5070
+tests/project_fixtures.py,sha256=iBm_th_JtAw76vlNu7Jjhh9tLH4oOaNi-MgtPzCV7yQ,138
+tests/test_api_client.py,sha256=tnYkIYvDILGJRatZM0nrtr83x8yBJstMP0Rt151U2gI,6757
+tests/test_base_downloader.py,sha256=SS7tWKv2ZZhpUDk4UCg1TkrNrpntjSewgzLl1mEubSE,3603
+tests/test_cli.py,sha256=yWWGLip8WgnDAB0vg7uhBU5eeEJieFLQ78Pn7IN-uWI,6088
+tests/test_common.py,sha256=tY_-SY-JmJX09UehFs9RIDqZ785AmhfTl6eVKJeIUFY,763
+tests/test_config.py,sha256=-_G682Ss3Zr1FmcMkjla1zAZprX2tQKpKc5_wD28rII,13340
+tests/test_data.py,sha256=NIBXpCjG3Os3vpc1CkiVONrebro8D_jqQyJ0N3kbucU,5433
+tests/test_downloader.py,sha256=hceljsjnuvk5Vk5X4mHgavIEcpbv8ylPwpz7rTwJ-aE,4721
+tests/test_hardware_set.py,sha256=hW7A_suyYdU7WkB7qoHSBPLxaAP2CKqI0i_ULfO5GeY,4408
+tests/test_imports_2and3.py,sha256=ehqpRYPVY7djBcb8OT_cnh86iCJJ9wuMWnfSR9RHxmY,507
+tests/test_job_downloader.py,sha256=_dZqyLZhc2Bq2n7-skERfodHx1JgFyHw8TamHp6ID9I,7025
+tests/test_package_environment.py,sha256=CdiC2PDVSnbcwTb4fsDTWqGYSzs1n5ca2KMoyISckGA,5893
+tests/test_package_tree.py,sha256=xCwNwYUmJrfmgCP2FGoHRFG-L0JPy8s4-66icxAls4o,6780
+tests/test_submit.py,sha256=-ORI75pWzFGKUCkqHuPzaMxkK7qGh1zuAOMcTQ6Q1Hk,4940
+tests/test_uploader.py,sha256=B1llTJt_fqR6e_V_Jxfw9z73QgkFlEPU87xLYGzt-TQ,2914
+tests/test_validator.py,sha256=2fY66ayNc08PGyj2vTI-V_1yeCWJDngkj2zkUM5TTCI,1526
+tests/mocks/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+tests/mocks/glob.py,sha256=J2MH7nqi6NJOHuGdVWxhfeBd700_Ckj6cLh_8jSNkfg,215
+ciocore-8.0.0.dist-info/METADATA,sha256=8iBZp-N5NyE1NnpPJccLrL4LPQHjDuetVnKEsb-Gk-Y,18155
+ciocore-8.0.0.dist-info/WHEEL,sha256=DZajD4pwLWue70CAfc7YaxT1wLUciNBvN_TTcvXpltE,110
+ciocore-8.0.0.dist-info/entry_points.txt,sha256=cCqcALMYbC4d8545V9w0Zysfg9MVuKWhzDQ2er4UfGE,47
+ciocore-8.0.0.dist-info/top_level.txt,sha256=SvlM5JlqULzAz00JZWfiUhfjhqDzYzSWssA87zdJl0o,14
+ciocore-8.0.0.dist-info/RECORD,,

{ciocore-7.0.2b5.dist-info → ciocore-8.0.0.dist-info}/WHEEL

@@ -1,5 +1,5 @@
 Wheel-Version: 1.0
-Generator: bdist_wheel (0.41.2)
+Generator: bdist_wheel (0.43.0)
 Root-Is-Purelib: true
 Tag: py2-none-any
 Tag: py3-none-any

ciocore-8.0.0.dist-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+conductor = ciocore.cli:main

tests/extra_env_fixtures.py

@@ -0,0 +1,57 @@
+
+
+EXTRA_ENV = [
+    {
+      "account_id": "6649535867387904",
+      "env": [],
+    },
+    {
+      "account_id": "5767615549800448",
+      "env": [
+        {
+          "merge_policy": "prepend",
+          "name": "test",
+          "value": "test"
+        }
+      ],
+    },
+    {
+      "account_id": "6649535867387904",
+      "env": [],
+    },
+    {
+      "account_id": "5669544198668288",
+      "env": [
+        {
+          "merge_policy": "append",
+          "name": "PATH",
+          "value": "/path/to/scripts"
+        },
+        {
+          "merge_policy": "exclusive",
+          "name": "RENDER_LOCATION",
+          "value": "cloud"
+        },
+        {
+          "merge_policy": "exclusive",
+          "name": "VARIABLE_USED_IN_SCRIPTS",
+          "value": "true"
+        },
+        {
+          "merge_policy": "exclusive",
+          "name": "testvar",
+          "value": "somevalue"
+        }
+      ],
+    },
+    {
+      "account_id": "6649535867387904",
+      "env": [
+        {
+          "merge_policy": "exclusive",
+          "name": "JMEYER",
+          "value": "JMEYER_ENV_VALUE"
+        }
+      ],
+    }
+  ]

tests/instance_type_fixtures.py

@@ -55,12 +55,46 @@ WIN_INSTANCE_TYPES = [
 
 ALL_INSTANCE_TYPES = WIN_INSTANCE_TYPES + LIN_INSTANCE_TYPES
 
+AWS_INSTANCE_TYPES = [
+ {
+      "cloud": "aws",
+      "cpu": 72,
+      "memory": 144,
+      "name": "c5.18xlarge",
+      "orchestrator": "batch",
+      "operating_system": "linux",
+      "cores": 72,
+       "description": "c5  18xlarge",
+  },
+  {
+      "cloud": "aws",
+      "cpu": 2,
+      "memory": 8,
+      "name": "m5.large",
+      "orchestrator": "batch",
+      "operating_system": "linux",
+      "cores": 2,
+      "description": "m large",
+  },
+  {
+      "cloud": "aws",
+      "cpu": 4,
+      "memory": 16,
+      "name": "m5.xlarge",
+      "orchestrator": "batch",
+      "operating_system": "linux",
+      "cores": 4,
+      "description": "m5 xlarge",
+  }
+]
+
+
 
 CW_INSTANCE_TYPES = [
     {
         "cores": 4,
         "memory": 16,
-        "name": "a-4-16",
+        "name": "cw-a-4-16",
         "categories": [{"label": "low", "order": 1}, {"label": "extra", "order": 9}],
         "operating_system": "linux",
         "description": "Desc 4 C 16 M A",
@@ -68,7 +102,7 @@ CW_INSTANCE_TYPES = [
     {
         "cores": 8,
         "memory": 16,
-        "name": "b-8-16",
+        "name": "cw-b-8-16",
         "categories": [{"label": "low", "order": 1}],
         "operating_system": "linux",
         "description": "Desc 8 C 16 M B",
@@ -76,7 +110,7 @@ CW_INSTANCE_TYPES = [
     {
         "cores": 4,
         "memory": 32,
-        "name": "c-4-32",
+        "name": "cw-c-4-32",
         "categories": [{"label": "mid", "order": 2}, {"label": "extra", "order": 9}],
         "operating_system": "linux",
         "description": "Desc 4 C 32 M C",
@@ -84,7 +118,7 @@ CW_INSTANCE_TYPES = [
     {
         "cores": 8,
         "memory": 32,
-        "name": "d-8-32",
+        "name": "cw-d-8-32",
         "categories": [{"label": "mid", "order": 2}],
         "operating_system": "linux",
         "description": "Desc 8 C 32 M D",
@@ -92,7 +126,7 @@ CW_INSTANCE_TYPES = [
     {
         "cores": 4,
         "memory": 64,
-        "name": "e-4-32",
+        "name": "cw-e-4-32",
         "categories": [{"label": "high", "order": 3}, {"label": "extra", "order": 9}],
         "operating_system": "linux",
         "description": "Desc 4 C 32 M E",
@@ -100,7 +134,7 @@ CW_INSTANCE_TYPES = [
     {
         "cores": 8,
         "memory": 64,
-        "name": "f-8-32",
+        "name": "cw-f-8-32",
         "categories": [{"label": "high", "order": 3}],
         "operating_system": "linux",
         "description": "Desc 8 C 32 M F",
@@ -108,7 +142,7 @@ CW_INSTANCE_TYPES = [
     {
         "cores": 8,
         "memory": 64,
-        "name": "g-8-32",
+        "name": "cw-g-8-32",
         "categories": None,
         "operating_system": "linux",
         "description": "Desc 8 C 32 M G",
@@ -119,7 +153,7 @@ CW_INSTANCE_TYPES_WITH_GPUS = [
     {
         "cores": 8,
         "memory": 64,
-        "name": "f-8-32-gpu",
+        "name": "cw-f-8-32-gpu",
         "categories": [{"label": "high", "order": 3}],
         "operating_system": "linux",
         "description": "Desc 8 C 32 M F gpu",

tests/project_fixtures.py

@@ -0,0 +1,8 @@
+
+
+PROJECTS = [
+    "Deadpool",
+    "Harry Potter & the chamber of secrets",
+    "Captain Corelli's Mandolin",
+    "Gone with the Wind",
+]

tests/test_api_client.py

@@ -2,9 +2,9 @@
 
    isort:skip_file
 """
- 
+import sys
 import unittest
-
+import json
 try:
     from unittest import mock
 except ImportError:
@@ -12,6 +12,7 @@ except ImportError:
 
 from ciocore import api_client
 
+from ciocore.api_client import request_extra_environment
 
 class ApiClientTest(unittest.TestCase):
     @staticmethod
@@ -45,3 +46,121 @@ class ApiClientTest(unittest.TestCase):
             fn = api_client.get_creds_path(api_key=True).replace("\\", "/")
             self.assertEqual(fn, "/users/joebloggs/.config/conductor/api_key_credentials")
 
+
+class TestTruncateMiddle(unittest.TestCase):
+
+    def test_truncation_not_needed(self):
+        self.assertEqual(api_client.truncate_middle("short", 10), "short")
+
+    def test_truncation_with_even_max_length(self):
+        self.assertEqual(api_client.truncate_middle("1234567890ABCDEF", 8), "1234~DEF")
+
+    def test_truncation_with_odd_max_length(self):
+        self.assertEqual(api_client.truncate_middle("1234567890ABCDEF", 9), "1234~CDEF")
+
+    def test_empty_string(self):
+        self.assertEqual(api_client.truncate_middle("", 5), "")
+
+    def test_non_string_input(self):
+        with self.assertRaises(TypeError):
+            api_client.truncate_middle(12345, 5)
+
+class TestRegisterClient(unittest.TestCase):
+    USER_AGENT_MAX_PATH_LENGTH = 10  # Example max length for testing
+
+    @classmethod
+    def setUpClass(cls):
+        cls.original_executable = sys.executable
+        sys.executable = '/usr/bin/python3'  # Example path for testing
+
+    @classmethod
+    def tearDownClass(cls):
+        sys.executable = cls.original_executable
+
+    def test_register_client_with_version(self):
+        client_name = 'ApiClient'
+        client_version = '1.0'
+
+        with mock.patch('platform.python_version', return_value='3.8.2'), \
+             mock.patch('platform.system', return_value='Linux'), \
+             mock.patch('platform.release', return_value='5.4.0-42-generic'):
+            user_agent = api_client.ApiClient.register_client(client_name, client_version)
+
+        expected_user_agent = (
+            f"ApiClient/1.0 (python 3.8.2; Linux 5.4.0-42-generic; "
+        )
+        self.assertTrue(user_agent.startswith(expected_user_agent))
+
+    def test_register_client_without_version(self):
+        client_name = 'ApiClient'
+
+        with mock.patch('platform.python_version', return_value='3.8.2'), \
+             mock.patch('platform.system', return_value='Linux'), \
+             mock.patch('platform.release', return_value='5.4.0-42-generic'):
+            user_agent =  api_client.ApiClient.register_client(client_name)
+
+
+        expected_user_agent = (
+            f"ApiClient/unknown (python 3.8.2; Linux 5.4.0-42-generic; "
+        )
+
+        self.assertTrue(user_agent.startswith(expected_user_agent))
+
+
+class TestRequestExtraEnvironment(unittest.TestCase):
+
+    def setUp(self):
+        self.api_response_data = {
+            "data": [
+                {"account_id": "123", "env": ["VAR1=value1", "VAR2=value2"]},
+                {"account_id": "456", "env": ["VAR3=value3", "VAR4=value4"]}
+            ]
+        }
+        self.response_ok = mock.MagicMock(status_code=200, text=json.dumps(self.api_response_data))
+        self.response_error = mock.MagicMock(status_code=500, text=json.dumps({"error": "Internal Server Error"}))
+
+    @mock.patch("ciocore.api_client.ApiClient")
+    @mock.patch("ciocore.api_client.read_conductor_credentials")
+    @mock.patch("ciocore.api_client.account_id_from_jwt")
+    def test_request_extra_environment_success(self, mock_account_id_from_jwt, mock_read_conductor_credentials, mock_ApiClient):
+        # Set up mocks for successful execution
+        mock_read_conductor_credentials.return_value = "valid_token"
+        mock_account_id_from_jwt.return_value = "123"
+        mock_api_instance = mock_ApiClient.return_value
+        mock_api_instance.make_request.return_value = (self.response_ok.text, self.response_ok.status_code)
+
+        result = request_extra_environment()
+
+        self.assertEqual(result, ["VAR1=value1", "VAR2=value2"])
+        mock_ApiClient.assert_called_once()
+        mock_read_conductor_credentials.assert_called_once_with(True)
+        mock_account_id_from_jwt.assert_called_once_with("valid_token")
+
+    @mock.patch("ciocore.api_client.ApiClient")
+    def test_request_extra_environment_api_failure(self, mock_ApiClient):
+        # Set up mock for API failure
+        mock_api_instance = mock_ApiClient.return_value
+        mock_api_instance.make_request.return_value = (self.response_error.text, self.response_error.status_code)
+
+        # Assert exception raised when the API call fails
+        with self.assertRaises(Exception) as context:
+            request_extra_environment()
+
+        self.assertIn('Failed to get extra environment', str(context.exception))
+        mock_ApiClient.assert_called_once()
+
+    @mock.patch("ciocore.api_client.ApiClient")
+    @mock.patch("ciocore.api_client.read_conductor_credentials")
+    @mock.patch("ciocore.api_client.account_id_from_jwt")
+    def test_request_extra_environment_no_account_env(self, mock_account_id_from_jwt, mock_read_conductor_credentials, mock_ApiClient):
+        # Set up mocks to simulate valid token and account ID but no matching environment
+        mock_read_conductor_credentials.return_value = "valid_token"
+        mock_account_id_from_jwt.return_value = "invalid_id"  # This won't match any 'account_id' in response
+        mock_api_instance = mock_ApiClient.return_value
+        mock_api_instance.make_request.return_value = (self.response_ok.text, self.response_ok.status_code)
+
+        with self.assertRaises(Exception) as context:
+            request_extra_environment()
+
+        self.assertEqual("Error: Could not get account environment!", str(context.exception))
+        mock_ApiClient.assert_called_once()

tests/test_base_downloader.py

@@ -0,0 +1,104 @@
+from ciocore.downloader.base_downloader import (
+    BaseDownloader,
+    DEFAULT_NUM_THREADS,
+    DEFAULT_PROGRESS_INTERVAL,
+    DEFAULT_MAX_ATTEMPTS,
+    DEFAULT_DELAY,
+    DEFAULT_JITTER,
+    DEFAULT_PAGE_SIZE,
+)
+import unittest
+from unittest import mock
+
+from ciocore import api_client
+
+from unittest.mock import  patch
+
+from concurrent.futures import ThreadPoolExecutor
+
+
+class TestBaseDownloaderInit(unittest.TestCase):
+    def test_default_values(self):
+        # Create an instance of the class
+        downloader = BaseDownloader()
+
+        # Assertions
+        self.assertIsNone(downloader.output_path)
+        self.assertFalse(downloader.force)
+        self.assertEqual(downloader.num_threads, DEFAULT_NUM_THREADS)
+        self.assertEqual(downloader.max_queue_size, DEFAULT_NUM_THREADS * 2)
+        self.assertEqual(
+            downloader.progress_interval, DEFAULT_PROGRESS_INTERVAL / 1000.0
+        )
+        self.assertEqual(downloader.page_size, DEFAULT_PAGE_SIZE)
+        self.assertIsInstance(downloader.client, api_client.ApiClient)
+        self.assertEqual(downloader.max_attempts, DEFAULT_MAX_ATTEMPTS)
+        self.assertEqual(downloader.delay, DEFAULT_DELAY)
+        self.assertEqual(downloader.jitter, DEFAULT_JITTER)
+        self.assertIsNone(downloader.regex)
+
+
+    def test_custom_values(self):
+        output_path = "/path/to/destination"
+        num_threads = 4
+        progress_interval = 500
+        page_size = 10
+        force = True
+        regex = r"\d+"
+        max_attempts = 3
+        delay = 2
+        jitter = 0.5
+
+        downloader = BaseDownloader(
+            output_path=output_path,
+            num_threads=num_threads,
+            progress_interval=progress_interval,
+            page_size=page_size,
+            force=force,
+            regex=regex,
+            max_attempts=max_attempts,
+            delay=delay,
+            jitter=jitter,
+        )
+
+        # Assertions
+        self.assertEqual(downloader.output_path, output_path)
+        self.assertTrue(downloader.force)
+        self.assertEqual(downloader.num_threads, num_threads)
+        self.assertEqual(downloader.max_queue_size, num_threads * 2)
+        self.assertAlmostEqual(downloader.progress_interval, progress_interval / 1000.0)
+        self.assertEqual(downloader.page_size, page_size)
+        self.assertIsInstance(downloader.client, api_client.ApiClient)
+        self.assertEqual(downloader.max_attempts, max_attempts)
+        self.assertEqual(downloader.delay, delay)
+        self.assertEqual(downloader.jitter, jitter)
+        self.assertIsNotNone(downloader.regex)
+
+
+class TestBaseDownloaderRun(unittest.TestCase):
+    def setUp(self):
+        self.downloader = BaseDownloader()
+
+    def tearDown(self):
+        pass
+
+    def test_run_method(self):
+        with patch(
+            "ciocore.downloader.base_downloader.ThreadPoolExecutor"
+        ) as mock_executor:
+            my_mock_executor = mock.MagicMock(spec=ThreadPoolExecutor)
+
+            mock_executor.return_value.__enter__.return_value = my_mock_executor
+
+            tasks = [{"id": 1, "name": "task1"}, {"id": 2, "name": "task2"}]
+            next_locator = False
+            mock_get_some_tasks = mock.MagicMock(return_value=(tasks, next_locator))
+
+            self.downloader.get_some_tasks = mock_get_some_tasks
+            self.downloader.download_tasks = mock.MagicMock()
+            self.downloader.event_queue = mock.MagicMock()
+
+            self.downloader.run()
+
+            mock_get_some_tasks.assert_called_with(None)
+            self.downloader.download_tasks.assert_called_with(tasks, my_mock_executor)