development-framework-cli 1.7.3__py3-none-any.whl

development_framework_cli-1.7.3.dist-info/METADATA

@@ -0,0 +1,141 @@
+Metadata-Version: 2.4
+Name: development-framework-cli
+Version: 1.7.3
+Summary: CLI tool for working with Data Modelling Storage Service
+Project-URL: Homepage, https://github.com/equinor/dm-cli
+Project-URL: Repository, https://github.com/equinor/dm-cli
+License: MIT
+License-File: LICENSE
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.13
+Requires-Python: >=3.13
+Requires-Dist: emoji>=2.15.0
+Requires-Dist: frozendict>=2.4.7
+Requires-Dist: python-dateutil>=2.9.0.post0
+Requires-Dist: requests>=2.32.5
+Requires-Dist: tenacity>=9.1.4
+Requires-Dist: tqdm>=4.67.3
+Requires-Dist: typer[all]>=0.24.1
+Requires-Dist: typing-extensions>=4.15.0
+Provides-Extra: dev
+Requires-Dist: pre-commit>=4.5.1; extra == 'dev'
+Requires-Dist: pytest>=9.0.2; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# Data Modelling CLI Tool [![PyPi version](https://img.shields.io/pypi/v/development-framework-cli)](https://pypi.org/project/development-framework-cli) [![PyPi downloads](https://img.shields.io/pypi/dm/development-framework-cli)](https://pypi.org/project/development-framework-cli) ![Visitors](https://api.visitorbadge.io/api/visitors?path=equinor%2Fdm-cli&countColor=%23263759&style=flat)
+
+CLI tool for working with [Data Modelling Storage Service](https://github.com/equinor/data-modelling-storage-service)
+
+## Installing
+> [!NOTE]
+> Requires Python version >= 3.13
+
+Install the tool via pip
+```sh
+$ pip3 install development-framework-cli
+```
+
+After a successful install, the program `'dm'` will be available in your python environment
+
+## Usage
+
+```txt
+Usage: dm [OPTIONS] COMMAND [ARGS]...
+
+Command Line Interface (CLI) tool for working with the Data Modelling Framework. This tool is mainly used to upload data source definitions, models and
+entities, and creating RecipeLink-tables.
+
+╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
+│ --force               -f            Force the operation. Overwriting and potentially deleting data.                                                         │
+│ --url                 -u      TEXT  URL to the Data Modelling Storage Service (DMSS). [default: http://localhost:5000]                                      │
+│ --token               -t      TEXT  Token for authentication against DMSS. [default: no-token]                                                              │
+│ --debug               -d            Print stack trace of suppressed exceptions                                                                              │
+│ --version             -v            Print version and exit                                                                                                  │
+│ --install-completion                Install completion for the current shell.                                                                               │
+│ --show-completion                   Show completion for the current shell, to copy it or customize the installation.                                        │
+│ --help                              Show this message and exit.                                                                                             │
+╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
+╭─ Commands ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
+│ create-lookup              Create a named Ui-/StorageRecipe-lookup-table from all RecipeLinks in a package existing in DMSS (requires admin privileges).    │
+│ ds                         Import and reset data sources                                                                                                    │
+│ entities                   Import, delete, or validate entities and/or blueprints                                                                           │
+│ export                     Export one or more entities.                                                                                                     │
+│ import-plugin-blueprints   Import blueprints from a plugin into the standard location 'system/Plugins/<plugin-name>'.                                       │
+│ reset                      Reset all data sources (deletes and re-uploads all packages to DMSS).                                                            │
+╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
+```
+
+> [!TIP]
+> For each of the `commands` listed above, you can run `dm <COMMAND> --help` to see subcommand-specific help messages, e.g. `dm ds import --help` or `dm pkg --help`
+
+### Required directory structure
+Certain commands expect a specific directory structure, such as the commands `dm reset`, `dm ds init`, and `dm ds reset`.
+For these commands, the `path` argument must be the path to a directory with two subdirectories, `data_sources` and `data`.
+
+```sh
+$ tree app
+app
+├── data
+  └── DemoApplicationDataSource
+    ├── instances
+│     └── demoApplication.json
+|     └── models
+|       └── DemoApplication.json
+└── data_sources
+    └── DemoApplicationDataSource.json
+```
+
+To add meta information to a package (for example to the models root package), a file with name "package.json" can be placed inside the folder.
+
+
+### Supported reference syntax
+The CLI tool will understand and resolve the following address formats during import.
+
+
+```bash
+# URI - Full absolute path prefixed with protocol
+dmss://datasource/package/entity
+dmss://datasource/package/subfolder/entity
+dmss://datasource/$123-456.attributeName
+
+# Alias - Require dependencies to be defined somewhere in the source tree
+ALIAS:package/entity
+ALIAS:entity
+
+# Data Source - Relative from the destination data source
+/package/entity
+/package/subfolder/entity
+/$123-456.attributeName
+
+# Package - Relative from the source package
+entity
+subfolder/entity
+
+# Dotted - Relative from the file (UNIX directory traversal)
+./../entity
+../subfolder/entity
+../../subfolder/entity
+```
+
+## Development
+
+```sh
+$ python3 -m venv .venv
+$ source .venv/bin/activate
+# Install the package from local files in edit mode (-e)
+$ pip3 install -e .
+$ dm --help
+```
+
+### Testing
+
+1. Install the package with dev dependencies: `pip3 install -e ".[dev]"`
+2. Run the tests: `pytest`
+
+## Feedback
+Please feel free to leave feedback in issues/PRs.
+
+## Contributing
+If you would like to contribute, please read our [Contribution guide](https://equinor.github.io/dm-docs/contributing/).

development_framework_cli-1.7.3.dist-info/RECORD

@@ -0,0 +1,71 @@
+dm_cli/__init__.py,sha256=2c0WxZjtTHwotq4Jkk-B28UcCDeX2TNO-BSbdbD95x8,46
+dm_cli/cli.py,sha256=mpxL7qeJQlpTeFlM7Fqho-6ASYrfcgIYfqxnVmvMZHI,7004
+dm_cli/dmss.py,sha256=3yzKc3Hb3iXE_Pq7-Dop2eQ61KLc2WcEsMOzfp-JM-g,3066
+dm_cli/domain.py,sha256=BMBK-WOQwBvDBEr_AoywdXAUPXMHIKIZLnMHpkuzqPs,5650
+dm_cli/enums.py,sha256=7PFPV1u5mexc-p6s8gG-wb7B5SmAORh5esmPQ-zxUtg,1236
+dm_cli/import_entity.py,sha256=SJn9IxUBWS9iTkafYAfZfkk7bZsip4TfZmQZRbHe3LI,4275
+dm_cli/import_package.py,sha256=wBXqF0rJrRusCFsLQkFBCyYbrLJdHQ8756oW04v9ggM,7526
+dm_cli/package_tree_from_zip.py,sha256=bpr9BoRaYKaowaP-zDif6km7mYJLbbby0f0kZHH6DTc,4624
+dm_cli/state.py,sha256=F2L_0bj0P840WDvCmqR0TCsko0AsVyjiOP8SVKocG7s,198
+dm_cli/command_group/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+dm_cli/command_group/data_source.py,sha256=pYVPicGlabf1HRUNqoKQcSlQfVRtzJjp2xqmWb_9kKA,7651
+dm_cli/command_group/entities.py,sha256=7pPL_Ejkm2He6CDjVfSz39QDGWBVNCtkU4dVox5du24,4198
+dm_cli/dmss_api/__init__.py,sha256=rO1FT4B25_nBww-fLBlehjkOfqgkDV3EWbG5ZJcO7cA,731
+dm_cli/dmss_api/api_client.py,sha256=AZz3BN28Jk1AUeS5ZTizJ0ZW8bGOCPlVF2MapfX1uMQ,39037
+dm_cli/dmss_api/configuration.py,sha256=mToiL-ruKriKSfuhJr2CJNnQEA2ASGviNS1vatHxe34,17323
+dm_cli/dmss_api/exceptions.py,sha256=dbM7LyJ7_cBPWLhUyTEwU1VcLNF9WRG0ZiwaztZ5G7c,5055
+dm_cli/dmss_api/model_utils.py,sha256=fyT0M2tzu1Uc9kNp8rqVVImBChh1DTyunEkECFtYwKA,82475
+dm_cli/dmss_api/rest.py,sha256=GpRcHbwGnXiu9HbiObaNss9-rrdGtmELxGhAHdL3J1k,14265
+dm_cli/dmss_api/api/__init__.py,sha256=12fUKkdPETk9RckbCl32EL1X6OhwMwwWMmCOmPnls94,225
+dm_cli/dmss_api/api/access_control_api.py,sha256=I4o0cqQMUDmyDKOAV53jGBSKZwYGLwY2DbOMSPwAGXc,13451
+dm_cli/dmss_api/api/attribute_api.py,sha256=tgD5OjbCbVNmPAIFtF_4X-XtKnJhgbzU4KJuoDg-bEw,6459
+dm_cli/dmss_api/api/blob_api.py,sha256=j1u11vmysi9FYmmSHtqDH7gGriUT83uahELoLFFzhHc,12976
+dm_cli/dmss_api/api/blueprint_api.py,sha256=Y2ksyFgNo6uzdFZUQW7cFlw1VGrXHjotA3QMvY1KkfU,12515
+dm_cli/dmss_api/api/datasource_api.py,sha256=KXvY--XsJ65rtXDd7IiX8AvIS_--sjKDMz8FGmmRP8E,17726
+dm_cli/dmss_api/api/default_api.py,sha256=WkHiGQZPyTp19OLC_0ZA3VIWVkkAb_aIaTMZkG97S2M,182276
+dm_cli/dmss_api/api/document_api.py,sha256=q1hJ6usJh6b55OiJrHY_6f_DY43tVhu1TsoZQ7Z9JLU,37828
+dm_cli/dmss_api/api/entity_api.py,sha256=bKdPS-_9PgNxPiC4nvy4vbSzy5vB-xiUNiz75QD5B64,18900
+dm_cli/dmss_api/api/export_api.py,sha256=kAGwTLJY6qSUri7waCIvysg1IGxVzr4efq3s-EZADag,12271
+dm_cli/dmss_api/api/file_api.py,sha256=wZpQD83P5JpHG2IQzkXVUGruD3Ax56eH7WRjVc75-R8,7285
+dm_cli/dmss_api/api/health_check_api.py,sha256=4siOuShRIz43zNF4G0oH_-Er3niHjuYpOb1pNwr7G1c,5768
+dm_cli/dmss_api/api/lookup_table_api.py,sha256=uEy92V59eJpsxtngHoQoH9AyUryqCEIiTac6VYQc0R0,12639
+dm_cli/dmss_api/api/meta_api.py,sha256=P6DqHw2Sa37H6cmZSc9Kzu6yMry5wG7Dd-ntt1jdvdc,6975
+dm_cli/dmss_api/api/personal_access_token_api.py,sha256=a8nIs-Lbf9VBQAT8OzJr7VDWhx1wXAomY_rq3-nP_Zs,16745
+dm_cli/dmss_api/api/reference_api.py,sha256=8uQnY69rKR4HfoweFElmgnhYGMRbL6MyKMU5nLe9hNk,6203
+dm_cli/dmss_api/api/search_api.py,sha256=NDOl0f3Ck5DxLrDqO58M-daGLtzUozMM56Hrf0ItOpY,7870
+dm_cli/dmss_api/api/whoami_api.py,sha256=bI7imyDYqedM-up20y7UfJ4RS2-MM0yC6AcXkji3Ihg,6077
+dm_cli/dmss_api/apis/__init__.py,sha256=6ZAbQnCEUI8MECJMbXVvaP3O-iZy3iQLBoSXlP-tRg0,1368
+dm_cli/dmss_api/model/__init__.py,sha256=U9808NsoSPJPzCkNdXn1lsoaUXKJu6LeAYzjE6xq5Kc,349
+dm_cli/dmss_api/model/access_control_list.py,sha256=ytBc-1fHcFgVFyxQ5N-WNHlDzj6WqQA_gPS1A29MjE4,12675
+dm_cli/dmss_api/model/access_level.py,sha256=AMhP_AS7G_k6ar903IECo-G3FkuJqHU8SwP_3bNwpOg,12009
+dm_cli/dmss_api/model/acl.py,sha256=ogczX9AszMg9dSpk1IUAWUDnkdjLhkzQ8fskCwIvEGo,12632
+dm_cli/dmss_api/model/data_source_information.py,sha256=BeTOHx_NGGkL9JCAfDIDy1Tl6MtHy6pbEdOfHya5fZg,11976
+dm_cli/dmss_api/model/data_source_request.py,sha256=6TmSFJ6V2beqL6qiX5Q3SzUldOyZA6SDhNiDAxF5D1A,11935
+dm_cli/dmss_api/model/dependency.py,sha256=es0ly3G-qXxO3G8hcJqCXbvUOMP9rRSwhucskGHMnCo,12472
+dm_cli/dmss_api/model/entity.py,sha256=CjQDalkvAWk7Lmm30j7T5OfYuYZwSLlNYhBKogkmY_M,11602
+dm_cli/dmss_api/model/error_response.py,sha256=SZCPbiE_vfWPesCEehIlP_nhs3UnywQkXwdv484Nusg,13086
+dm_cli/dmss_api/model/export_meta_response.py,sha256=Bc95wErM0AKbym27G62vftJF0QCXcb6cYDtt6O-hg04,12363
+dm_cli/dmss_api/model/get_blueprint_response.py,sha256=rTlCzAohS8UuktzhDQWkxAGa9v4AZqNuGKu1OZKAm8c,12675
+dm_cli/dmss_api/model/lookup.py,sha256=breMaW0pXGXPNx6luEQO5B9Co4fMpTMFm4P9G5zboyc,12625
+dm_cli/dmss_api/model/pat_data.py,sha256=gkQdIQSmVnR4pAKw7Y3UmXr2kST0RH-c-imEyayYg5U,12923
+dm_cli/dmss_api/model/recipe.py,sha256=hNbyp0wa4xzeD1naiE2KjLg9vomXqZp8RbkxP9pDYZs,14427
+dm_cli/dmss_api/model/recipe_attribute.py,sha256=f2-P-bF5LGNia08T86Sc9V73jnAH-zhKo9aE_7J0_48,13522
+dm_cli/dmss_api/model/reference.py,sha256=fxp8Z66wSVaKgxFJ4R2fiB_CKbq72aD9bQ7yhem5Cg4,11764
+dm_cli/dmss_api/model/reference_entity.py,sha256=ITan_axXrDu9W7qTybA6p3OQB5mjIQFtkJQ8XhlFaBc,12151
+dm_cli/dmss_api/model/repository.py,sha256=W1T7QfqXpk1DDPSkLvpB_TgvbzV8LoE9ZmMeVsV81m8,13879
+dm_cli/dmss_api/model/repository_type.py,sha256=zeqC9WRY9PETTnXcUCx8JIWgCgxkuyYgkkdIKSYN3M8,12168
+dm_cli/dmss_api/model/storage_attribute.py,sha256=xRML8nAgZ4GkfhHEVCG496IAU6O2AnRbY9froqbtMlU,13333
+dm_cli/dmss_api/model/storage_data_types.py,sha256=TElPaXKgiZ_YUqXAbjqw3tkmGn271uc1NkClSzSoRqg,12198
+dm_cli/dmss_api/model/storage_recipe.py,sha256=uZh_zFithPxO3_lXuiHchhf2ZRX3_pJd-xP3HVFaDm0,13218
+dm_cli/dmss_api/models/__init__.py,sha256=7pzdYw1ktxZRe29ZN_ZIdid8hPu_0JZx0KPaGjV0fR0,1516
+dm_cli/utils/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+dm_cli/utils/file_structure.py,sha256=efIcl7kr7LGm9-2AKt6M6uXe0L8NHNBg0Icy1xO8rSM,1020
+dm_cli/utils/reference.py,sha256=V11edpj8mJSCbXlWlv8HW2zO4AwHRqDnmwdxKv5_qpA,5593
+dm_cli/utils/resolve_local_ids.py,sha256=I9kXVZTQgqr07eu4CghGQVU99MAEXCdn6hacVX0AYOY,5726
+dm_cli/utils/utils.py,sha256=T-O4mwMOJH-qwGFEVDbogb8RVXfloEt0wA8A5fRWCSc,6882
+dm_cli/utils/zip.py,sha256=Zdq6Z978i_1iDAvbrqZwJNsw4QersVtK0HeqFiRagdA,2781
+development_framework_cli-1.7.3.dist-info/METADATA,sha256=JDFFza5Yjl5Mm1mkN8tMiuKJcKUnQ5bNDx7h9UCOyDU,8174
+development_framework_cli-1.7.3.dist-info/WHEEL,sha256=QccIxa26bgl1E6uMy58deGWi-0aeIkkangHcxk2kWfw,87
+development_framework_cli-1.7.3.dist-info/entry_points.txt,sha256=PAOXUTc_jhfY9Hfs7BNFzh4TlbrOGYU7HgTM-P7h074,38
+development_framework_cli-1.7.3.dist-info/licenses/LICENSE,sha256=zefthAHZDOYIcHUXefkN_Fvuy4s-nYfoLX_dsycVDxc,1064
+development_framework_cli-1.7.3.dist-info/RECORD,,

development_framework_cli-1.7.3.dist-info/WHEEL

@@ -0,0 +1,4 @@
+Wheel-Version: 1.0
+Generator: hatchling 1.29.0
+Root-Is-Purelib: true
+Tag: py3-none-any

development_framework_cli-1.7.3.dist-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+dm = dm_cli.cli:app

development_framework_cli-1.7.3.dist-info/licenses/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2022 Equinor
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

dm_cli/__init__.py

@@ -0,0 +1 @@
+VERSION = "1.7.3"  # x-release-please-version

dm_cli/cli.py

@@ -0,0 +1,180 @@
+#! /usr/bin/env python
+import io
+import os
+from pathlib import Path
+from typing import List, Optional
+from zipfile import ZipFile
+
+import emoji
+import typer
+from typing_extensions import Annotated
+
+from dm_cli import VERSION
+from dm_cli.command_group.data_source import data_source_app, reset_data_source
+from dm_cli.command_group.entities import entities_app, import_entity
+from dm_cli.dmss import dmss_api, dmss_exception_wrapper, export
+from dm_cli.state import state
+from dm_cli.utils.file_structure import get_app_dir_structure, get_json_files_in_dir
+from dm_cli.utils.utils import (
+    get_root_packages_in_data_sources,
+    validate_entities_in_data_sources,
+)
+from dm_cli.utils.zip import save_as_zip_file, unpack_and_save_zipfile
+
+app = typer.Typer(pretty_exceptions_short=True)
+app.add_typer(data_source_app, name="ds")
+app.add_typer(entities_app, name="entities")
+
+
+def version_callback(print_version: bool):
+    if print_version:
+        print(VERSION)
+        raise typer.Exit()
+
+
+@app.callback()
+def main(
+    force: bool = typer.Option(
+        False, "--force", "-f", help="Force the operation. Overwriting and potentially deleting data."
+    ),
+    dmss_url: str = typer.Option(
+        "http://localhost:5000", "--url", "-u", help="URL to the Data Modelling Storage Service (DMSS)."
+    ),
+    token: str = typer.Option("no-token", "--token", "-t", help="Token for authentication against DMSS."),
+    debug: bool = typer.Option(False, "--debug", "-d", help="Print stack trace of suppressed exceptions"),
+    version: Optional[bool] = typer.Option(
+        None, "--version", "-v", callback=version_callback, is_eager=True, help="Print version and exit"
+    ),
+):
+    """
+    Command Line Interface (CLI) tool for working with the Data Modelling Framework.
+    This tool is mainly used to upload data source definitions, models and entities, and creating RecipeLink-tables.
+    """
+    state.force = force
+    state.dmss_url = dmss_url
+    state.token = token
+    state.debug = debug
+
+    dmss_api.api_client.default_headers["Authorization"] = f"Bearer {token}"
+    dmss_api.api_client.configuration.host = dmss_url
+
+
+@app.command("import-plugin-blueprints")
+def import_plugin_blueprints(
+    path: Annotated[
+        str,
+        typer.Argument(
+            help="Path to file or folder on local filesystem to import. Trailing '/' will result in the content being imported instead of the folder itself."
+        ),
+    ],
+    validate: Annotated[bool, typer.Option(help="if True, all entities uploaded will be validated.")] = True,
+):
+    """
+    Import blueprints from a plugin into the standard location 'system/Plugins/<plugin-name>'.
+    """
+    state.force = True
+    dmss_exception_wrapper(
+        import_entity, source=f"{path}/blueprints/", destination=f"system/Plugins/{Path(path).name}", validate=validate
+    )
+
+
+@app.command("create-lookup")
+def create_lookup(
+    name: Annotated[str, typer.Argument(help="Name of the lookup (application context) to create.")],
+    paths: Annotated[
+        List[Path],
+        typer.Argument(
+            help="one or more remote location for recursively looking for RecipeLinks. "
+            "A remote location is a path in DMSS on the format <dataSource>/<rootPackage>/<subPacakge>/..."
+        ),
+    ],
+):
+    """
+    Create a named Ui-/StorageRecipe-lookup-table from all RecipeLinks in a package existing in DMSS (requires admin privileges).
+    """
+    # TODO change type of paths argument to be list of str to avoid this path as strings conversion below
+    paths_as_strings = [str(path) for path in paths]
+    print(f"Creating lookup table from paths: {paths_as_strings}")
+    dmss_exception_wrapper(dmss_api.create_lookup, recipe_package=paths_as_strings, application=name)
+
+
+@app.command("export")
+def export_entity(
+    target: Annotated[str, typer.Argument(help="Address to the entity to export. Format: <dataSource>/<path>.")],
+    export_location: Annotated[
+        str,
+        typer.Argument(
+            help="Path on local filesystem to store the exported document(s) to. If not provided, will export to current folder."
+        ),
+    ] = "",
+    unpack: Annotated[
+        bool,
+        typer.Option(help="Whether or not to unpack the zip file containing the exported entities(s)."),
+    ] = False,
+):
+    """
+    Export one or more entities.
+
+    Example: if you have a data_source "DemoApplicationDataSource" with the root package "models" and a sub package "Windmill",
+    the entire "Windmill" package can be exported using: target = "DemoApplicationDataSource/models/Windmill".
+    The 'target' parameter can either point to a single entity or to a package.
+
+    If the file or folder to export already exists on local disk, an exception is raised.
+    """
+    response = dmss_exception_wrapper(export, target)
+
+    if not export_location:
+        export_location = os.getcwd()
+    with ZipFile(io.BytesIO(response.content), "r") as zip_file:
+        zip_file.filename = f"{target.split('/')[-1]}.zip"
+        if unpack:
+            unpack_and_save_zipfile(export_location=export_location, zip_file=zip_file)
+        else:
+            save_as_zip_file(export_location=export_location, filename=zip_file.filename, data=response.content)
+
+
+@app.command("reset")
+def reset(
+    path: Annotated[
+        str,
+        typer.Argument(help="The path on local file system to the data source you want to reset."),
+    ],
+    validate_entities: Annotated[
+        bool, typer.Option(help="if True, all entities uploaded to DMSS will be validated.")
+    ] = True,
+    resolve_local_ids: Annotated[
+        bool, typer.Option(help="if True, will resolve all local ids found in all entities")
+    ] = False,
+):
+    """
+    Reset all data sources (deletes and re-uploads all packages to DMSS).
+    """
+    # Check for presence of expected directories, 'data_sources' and 'data'
+    data_sources_dir, data_dir = get_app_dir_structure(Path(path))
+
+    data_source_definitions = get_json_files_in_dir(data_sources_dir)
+    if not data_source_definitions:
+        print(emoji.emojize(f"\t:warning: No data source definitions were found in '{data_sources_dir}'."))
+
+    for data_source_definition_filename in data_source_definitions:
+        data_source_name = data_source_definition_filename.replace(".json", "")
+
+        data_source_data_dir = data_dir.joinpath(data_source_name)
+        if not data_source_data_dir.is_dir():
+            print(
+                emoji.emojize(
+                    f"\t:warning: No data source data directory was found by the name '{data_source_name}' in '{data_dir}'."
+                )
+            )
+            continue
+
+        dmss_exception_wrapper(
+            reset_data_source, data_source=data_source_name, path=path, resolve_local_ids=resolve_local_ids
+        )
+    data_source_contents = get_root_packages_in_data_sources(path)
+    if validate_entities:
+        dmss_exception_wrapper(validate_entities_in_data_sources, data_source_contents)
+
+
+if __name__ == "__main__":
+    app()

dm_cli/command_group/data_source.py

@@ -0,0 +1,176 @@
+import json
+from pathlib import Path
+
+import emoji
+import typer
+from rich import print
+from tenacity import (
+    retry,
+    retry_if_exception_type,
+    stop_after_attempt,
+    wait_random_exponential,
+)
+from typing_extensions import Annotated
+
+from dm_cli.dmss import ApplicationException, dmss_api, dmss_exception_wrapper
+from dm_cli.dmss_api.exceptions import ServiceException
+from dm_cli.import_entity import import_folder_entity, remove_by_path_ignore_404
+from dm_cli.utils.file_structure import get_app_dir_structure, get_json_files_in_dir
+from dm_cli.utils.utils import (
+    get_root_packages_in_data_sources,
+    validate_entities_in_data_sources,
+)
+
+data_source_app = typer.Typer(help="Import and reset data sources")
+
+
+@data_source_app.command("import", help="Import a single data source definition")
+def import_data_source(
+    path: Annotated[Path, typer.Argument(help="Path on local filesystem to a data source JSON file.")],
+):
+    """
+    Import a single data source definition to DMSS.
+    """
+
+    @retry(
+        wait=wait_random_exponential(multiplier=1, max=60),
+        stop=stop_after_attempt(5),
+        reraise=True,
+        retry=retry_if_exception_type(ServiceException),
+    )
+    def retry_wrapper():
+        data_source_path = Path(path)
+        if not data_source_path.is_file():
+            raise FileNotFoundError(f"The path '{path}' is not a file.")
+
+        print(f"IMPORTING DATA SOURCE '{data_source_path.name}'")
+
+        # Read the data source definition
+        with open(data_source_path) as file:
+            document = json.load(file)
+            existing_data_sources = dmss_exception_wrapper(dmss_api.data_source_get_all)
+            if any(existing_document["name"] == document["name"] for existing_document in existing_data_sources):
+                print(f"WARNING: data source {document['name']} already exists. Updating existing data source.")
+
+            dmss_exception_wrapper(dmss_api.data_source_save, document["name"], document)
+            print(f"\tImported data source '{document['name']}' ✓")
+
+    retry_wrapper()
+
+
+@data_source_app.command("import-all", help="Import all datasources found in the directory given by 'path'")
+def import_all_data_sources(
+    path: Annotated[
+        Path, typer.Argument(help="Path on local filesystem to the folder containing the data sources to import.")
+    ],
+):
+    """
+    Import all data source definitions to DMSS.
+    """
+    data_sources_dir = Path(path)
+    if not data_sources_dir.is_dir():
+        raise FileNotFoundError(f"The path '{path}' is not a directory.")
+
+    print("IMPORTING DATA SOURCES")
+
+    # List all data source definitions under <data_sources_dir>
+    data_sources = get_json_files_in_dir(data_sources_dir)
+    if not data_sources:
+        print(emoji.emojize(f"\t:warning: No data source definitions were found in '{data_sources_dir}'"))
+
+    for filename in data_sources:
+        # Import the data source definition
+        filepath = data_sources_dir.joinpath(filename)
+        import_data_source(filepath)
+
+
+@data_source_app.command("init")
+def initialize_data_source(
+    path: Annotated[Path, typer.Argument(help="Path on local filesystem to data source folder.")],
+    validate_entities: Annotated[
+        bool, typer.Option(help="If True, all entities uploaded to DMSS will be validated.")
+    ] = True,
+):
+    """
+    Initialize the data sources and import all packages.
+    The remote packages in a data source will be deleted before new data sources are imported.
+    """
+    # Check for presence of expected directories, 'data_sources' and 'data'
+    data_sources_dir, data_dir = get_app_dir_structure(Path(path))
+
+    data_source_definitions = get_json_files_in_dir(data_sources_dir)
+    if not data_source_definitions:
+        print(emoji.emojize(f"\t:warning: No data source definitions were found in '{data_sources_dir}'."))
+    for data_source_definition_filename in data_source_definitions:
+        import_data_source_file(data_sources_dir, data_dir, data_source_definition_filename, False)
+    data_source_contents = get_root_packages_in_data_sources(path)
+    if validate_entities:
+        dmss_exception_wrapper(validate_entities_in_data_sources, data_source_contents)
+
+
+def import_data_source_file(
+    data_sources_dir: str, data_dir: str, data_source_definition_filename: str, resolve_local_ids: bool
+):
+    data_source_definition_filepath = Path(data_sources_dir).joinpath(data_source_definition_filename)
+    data_source_name = data_source_definition_filename.replace(".json", "")
+
+    data_source_data_dir = data_dir / data_source_name
+    if not data_source_data_dir.is_dir():
+        print(
+            emoji.emojize(
+                f"\t:warning: No data source data directory was found by the name '{data_source_name}' in '{data_dir}'."
+            )
+        )
+    else:
+        import_data_source(data_source_definition_filepath)
+        with open(data_source_definition_filepath) as file:
+            data_source_document = json.load(file)
+            global_folders = data_source_document.get("global_folders", [])
+            root_packages = [f for f in data_source_data_dir.iterdir() if f.is_dir() and f.name not in global_folders]
+            # Remove existing root packages from the data source.
+            # This will also remove any files in the global folders that are references from files in the root packages.
+            for root_package in root_packages:
+                dmss_exception_wrapper(remove_by_path_ignore_404, f"/{data_source_name}/{root_package.name}")
+
+            # Import all root packages
+            for root_package in root_packages:
+                print(f"Importing PACKAGE '{data_source_data_dir / root_package.name}' --> '{data_source_name}'")
+                dmss_exception_wrapper(
+                    import_folder_entity,
+                    source_path=data_source_data_dir / root_package.name,
+                    destination=data_source_name,
+                    # Use the document raw endpoint,
+                    # so that uploaded packages will not be resolved,
+                    # this is to support uploading core blueprints.
+                    raw_package_import=True,
+                    resolve_local_ids=resolve_local_ids,
+                )
+
+
+@data_source_app.command("reset")
+def reset_data_source(
+    data_source: Annotated[str, typer.Argument(help="Name of data source to reset")],
+    path: Annotated[Path, typer.Argument(help="Path on local filesystem to data source folder.")],
+    resolve_local_ids: Annotated[bool, typer.Argument(help="Resolve local ids")] = False,
+):
+    """
+    Reset a single data source (deletes and re-uploads root-packages)
+    """
+    app_dir = Path(path)
+    if not app_dir.is_dir():
+        raise FileNotFoundError(f"The path '{path}' is not a directory.")
+
+    # Check for presence of expected directories, 'data_sources' and 'data'
+    data_sources_dir, data_dir = get_app_dir_structure(app_dir)
+
+    # Check for the data source definition file (json)
+    data_source_path = data_sources_dir.joinpath(f"{data_source}.json")
+    if not data_source_path.is_file():
+        raise FileNotFoundError(f"There is no data source definition for '{data_source}' in '{data_sources_dir}'.")
+    # Check for the data source data directory (contains packages)
+    data_source_data_dir = data_dir.joinpath(data_source)
+    if not data_source_data_dir.is_dir():
+        raise FileNotFoundError(f"There is no data source directory for '{data_source}' in '{data_dir}'.")
+
+    # Import all packages in the data source
+    import_data_source_file(data_sources_dir, data_dir, f"{data_source}.json", resolve_local_ids)