gOdoo-cli 0.1.6__tar.gz → 0.3.3__tar.gz

{godoo_cli-0.1.6 → godoo_cli-0.3.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: godoo-cli
-Version: 0.1.6
+Version: 0.3.3
 Summary: Wrapper around Odoo-Bin with some convinience RPC functions.
 Home-page: https://github.com/OpenJKSoftware/gOdoo
 License: LGPL-3

{godoo_cli-0.1.6 → godoo_cli-0.3.3}/pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "gOdoo-cli"
-version = "0.1.6"
+version = "0.3.3"
 description = "Wrapper around Odoo-Bin with some convinience RPC functions."
 authors = ["Joshua Kreuder <Joshua_Kreuder@outlook.com>"]
 license = "LGPL-3"

{godoo_cli-0.1.6 → godoo_cli-0.3.3}/setup.py

@@ -40,7 +40,7 @@ entry_points = \
 
 setup_kwargs = {
     'name': 'godoo-cli',
-    'version': '0.1.6',
+    'version': '0.3.3',
     'description': 'Wrapper around Odoo-Bin with some convinience RPC functions.',
     'long_description': '# gOdoo Dev Environment\n\n![OdooLogo](https://raw.githubusercontent.com/OpenJKSoftware/gOdoo/main/assets/odoo_logo.png)\n![ComposeLogo](https://raw.githubusercontent.com/docker/compose/v2/logo.png)\n\n[<img src="https://raw.githubusercontent.com/OpenJKSoftware/gOdoo/main/assets/godoo-main-cli.png" width="1000"/>](image.png)\n\n**gOdoo** is short for **go Odoo**. \\\nIt is a [Vscode Devcontainer](https://code.visualstudio.com/docs/remote/containers) Environment for [Odoo](https://odoo.com/)\nwith Python CLI `godoo` convenience wrapper around `odoo-bin`.\n\nThis repository is the base source for the Python package [godoo-cli](https://pypi.org/project/godoo-cli/) and serves as\nan all batteries included development environment.\n\nMade Possible by: [WEMPE Elektronic GmbH](https://wetech.de)\n\n# gOdoo-cli\n\nPython package that provides `godoo` command line interface around `odoo-bin`.\n\nIt\'s build with [Typer](https://github.com/tiangolo/typer) to provide some convenience Wrappers for Odoo development and\nDeployment.\n\nMost flags can be configured by Env variables. \\\nUse `godoo --help` to find out more. HINT: Install tab-completion with `godoo --install-completion`\n\n# Docker\n\nThis workspace also contains Docker and Docker-Compose files. \\\n\nThey are used to provide either easy Odoo instances where the source is pulled according to\n[ODOO_MANIFEST.yml](./ODOO_MANIFEST.yml), or as a all batteries included devcontainer for VScode.\n\n## Requirements\n\n- [Docker Compose](https://github.com/docker/compose)\n- [Traefik](https://doc.traefik.io/traefik/) container running with docker provider and "traefik" named docker network.\n  Example: [Traefik Devproxy](https://github.com/joshkreud/traefik_devproxy)\n- SSH Agent running. (check `echo $SSH_AUTH_SOCK`)\\\n  This gets passed trough in the Buildprocess to clone Thirdparty repos.\n\n## Just wanna have a quick and easy Odoo Instance?\n\n```bash\ngit clone https://github.com/OpenJKSoftware/gOdoo\ncd godoo\n. scripts/container_requirements.sh # Check Requirements\ndocker-compose build\ndocker-compose up\n# wait......\n# wait a bit mode ...\n# just a little bit longer ..\n# There we go.\n# Odoo should be reachable on \'https://godoo.docker.localhost\' assuming you didn\'t change .env TRAEFIK_HOST_RULE or COMPOSE_PROJECT_NAME\n```\n\n# Devcontainer\n\n## Features\n\n- All batteries included [Devcontainer](https://code.visualstudio.com/docs/remote/containers) with postgres service\n  Container and local DNS resolvig managed by [Traefik](https://doc.traefik.io/traefik/).\n- Easy fully working Odoo instance by `docker-compose up` with https access.\n- `godoo` CLI wrapper around Odoo. (Most flags can be configured by Environment Variables and are already preconfigured\n  in the Containers. See [.env.sample](./.env.sample))\n- `odoo-bin` is added to PATH and can thus be invoked from every folder.\n- Odoo will run in Proxy_Mode behind a Traefik reverse proxy for easy access on\n  `https://$COMPOSE_PROJECT_NAME.docker.localhost`\n- [Odoo Pylint plugin](https://github.com/OCA/pylint-odoo) preconfigured in vscode\n- Preinstalled vscode Extensions Highlights:\n  - [SQL Tools](https://marketplace.visualstudio.com/items?itemName=mtxr.sqltools) with preconfigured connection for\n    easy Database access in the Sidebar.\n  - [Docker Extension](https://marketplace.visualstudio.com/items?itemName=ms-azuretools.vscode-docker) controls\n    container host.\n  - [Odoo Snippets](https://marketplace.visualstudio.com/items?itemName=mstuttgart.odoo-snippets)\n  - [Odoo Developments](https://marketplace.visualstudio.com/items?itemName=scapigliato.vsc-odoo-development) can Grab\n    Odoo Model information from a running Server\n  - [Todo Tree](https://marketplace.visualstudio.com/items?itemName=Gruntfuggly.todo-tree)\n\n## Usage\n\n1. For Docker on windows: Clone the repo into the WSL2 Filesystem for better IO performance\n2. Have [Traefik](https://github.com/traefik/traefik) Running on `docker.localhost`\n   [Example](https://github.com/joshkreud/traefik_devproxy) \\\n   There must be a Docker network called `traefik` that can reach traefik.\n3. Open Devcontianer:\n   - If you have the Devcontainer CLI: `devcontainer open .`\n   - If not open the workspace in Local Vscode. In the Command pallete search for `Reopen in container`\n4. From **within the container** start Odoo using one of the following commands:\n   - You can enable godoo tab-completion by `godoo --install-completion`\n   - `make` -> Loads Odoo + Workspace Addons\n   - `make bare` -> Loads Odoo with ony `web` installed.\n   - `make kill` -> Search for `odoo-bin` processes and kill them\n   - `make reset` -> Drops DB, deletes config file and datafolder\n   - The full init script is available via "`godoo`". (See --help for Options)\n5. Open Odoo `https://$COMPOSE_PROJECT_NAME.docker.localhost`\\\n   For example `COMPOSE_PROJECT_NAME=godoo` --> [https://godoo.docker.localhost](https://godoo.docker.localhost)\n6. Login with `admin:admin`\n7. Profit!\n\n### Access to Odoo and Thirdparty addon Source\n\nYou can access the Odoo source by opening the VsCode workspace [full.code-workspace](full.code-workspace) from within\nthe Container. This will open a [Multi-Root Workspace](https://code.visualstudio.com/docs/editor/multi-root-workspaces).\nReally waiting for https://github.com/microsoft/vscode-remote-release/issues/3665 here.\n\n## Reset Devcontainer Data\n\nWhen you screwed up so bad its time to just start Over godoo has you covered:\n\n### Automatic Reset\n\nThere are 3 Options to reset the Dev Env.\n\n1. From **Outside** the Container run `make reset` in the project root to delete docker volumes and restart the\n   container. (Vscode will prompt to reconnect if still open)\n2. From **Outside** the Container run `make reset-hard` in the project root to force rebuild the main Odoo container and\n   then do the same as `make reset`\n3. From **Inside** the Container run `make reset` to drop the DB and delete varlib and the bootstrap flag, which is way\n   quicker than the other options.\n\n### Manual Reset\n\n1. Close vscode\n2. Remove `app` and `db` container from docker.\n3. Remove volumes: `db, odoo_thirdparty, odoo_web, vscode_extensions`\n4. Restart Devcontainer\n\n## Python Debugging\n\n### VsCode Debugging\n\nDebugging doesn\'t reliably work with\n[Odoo Multiprocess](https://www.odoo.com/documentation/14.0/developer/misc/other/cmdline.html#multiprocessing) mode\nenabled. \\\nThe container ships with a Vscode Debug profile, that sets `--workers 0` to allow for Debugging Breakpoints. See [.vscode/launch.json](./.vscode/launch.json)\n\n### Interactive Shell\n\nUse `godoo shell` to enter an interactive shell on the Database.\n\n# Odoo Modules\n\n## Third Party Modules (manifest.yml)\n\nThe `godoo` bootstrap function, will download some modules using git. \\\nWhich Repos to download is specified in `ODOO_MANIFEST.yml` ([Default](./ODOO_MANIFEST.yml)) \\\nNot all of the cloned addons are automatically installed. \\\nInstall them via the Apps Page in Odoo using `godoo rpc modules install` or using `odoo-bin`.\\\nModules downloaded on the Odoo Marketplace can be dropped as a `.zip` archive in [./thirdparty](./thirdparty)\n',
     'author': 'Joshua Kreuder',

{godoo_cli-0.1.6 → godoo_cli-0.3.3}/src/godoo_cli/__init__.py

@@ -1,2 +1,2 @@
-__version__ = "0.1.6"
+__version__ = "0.3.3"
 from .cli import launch_cli

{godoo_cli-0.1.6 → godoo_cli-0.3.3}/src/godoo_cli/cli.py

@@ -9,14 +9,15 @@ from .commands import (
     bootstrap_odoo,
     get_source,
     get_source_file,
+    install_module_dependencies,
     launch_odoo,
-    makedec_cli_app,
     odoo_shell,
     odoo_test,
     rpc_cli_app,
+    set_odoo_config,
     uninstall_modules,
 )
-from .helpers import set_logging
+from .helpers.system import set_logging
 
 
 def main_callback(
@@ -54,21 +55,20 @@ def main_cli():
     app.add_typer(
         typer_instance=rpc_cli_app(),
         name="rpc",
-        help="Various RPC Wrappers",
-    )
-    app.add_typer(
-        typer_instance=makedec_cli_app(),
-        name="makedev",
-        help="Set config and RPC settings to generate a staging environment",
+        help="Commands that act on a running Odoo instance through RPC",
     )
 
     # Normal Subcommands
+    app.command("config", help="Set odoo.conf options")(set_odoo_config)
     app.command("launch", help="Launch Odoo, Bootstrap if bootstrapflag is not present")(launch_odoo)
     app.command("bootstrap", help="Bootstrap Odoo")(bootstrap_odoo)
-    app.command("get-source-file", help="Get Raw file from odoo git remote or specific git remote.")(get_source_file)
-    app.command("get-source", help="Download/Unzip Odoo Source and thirdparty addons.")(get_source)
+    app.command("source-get-file", help="Get Raw file from odoo git remote or specific git remote.")(get_source_file)
+    app.command("source-get-depends", help="Install dependencies from __manifest__.py in specified modules")(
+        install_module_dependencies
+    )
+    app.command("source-get", help="Download/Unzip Odoo Source and thirdparty addons.")(get_source)
     app.command("test", help="Bootstrap or Launch odoo in Testing Mode")(odoo_test)
-    app.command("shell", help="Shell into Odoo")(odoo_shell)
+    app.command("shell", help="Start interactive Odoo shell")(odoo_shell)
     app.command("uninstall", help="Uninstall Modules")(uninstall_modules)
     return app
 

{godoo_cli-0.1.6 → godoo_cli-0.3.3}/src/godoo_cli/commands/__init__.py

@@ -1,7 +1,7 @@
 from .bootstrap import bootstrap_odoo
+from .config import set_odoo_config
 from .launch import launch_odoo
-from .makedev import makedec_cli_app
 from .rpc import rpc_cli_app
 from .shell import odoo_shell, uninstall_modules
-from .source_get import get_source, get_source_file
+from .source_get import get_source, get_source_file, install_module_dependencies
 from .test import odoo_test

{godoo_cli-0.1.6 → godoo_cli-0.3.3}/src/godoo_cli/commands/bootstrap.py

@@ -1,67 +1,19 @@
 import logging
 import re
-import subprocess
-from ast import literal_eval
 from pathlib import Path
 from typing import List
 
 import typer
 
-from ..helpers import run_cmd
+from ..helpers.bootstrap import _install_py_reqs_by_odoo_cmd
 from ..helpers.cli import typer_retuner, typer_unpacker
-from ..helpers.odoo_files import _get_python_requirements_of_modules, get_addon_paths, get_odoo_module_paths
+from ..helpers.odoo_files import get_addon_paths, get_odoo_module_paths
+from ..helpers.system import run_cmd
 from .source_get import get_source
 
 LOGGER = logging.getLogger(__name__)
 
 
-def _install_py_reqs_for_modules(addon_paths: List[Path], module_names: List[str] = None):
-    """Install Python Requirements mentioned in odoo module manifests of given modules
-
-    Parameters
-    ----------
-    addon_paths : List[Path]
-        odoo-bin addons path
-    module_names : List[str], Optional
-        List of modules to filter for, by default "all available modules"
-
-    Returns
-    -------
-    CompletedProcess
-    """
-    py_reqs = _get_python_requirements_of_modules(addon_paths=addon_paths, filter_module_names=module_names)
-    if py_reqs:
-        installed_packages = run_cmd(
-            "pip list --format json --disable-pip-version-check", check=True, shell=True, stdout=subprocess.PIPE
-        ).stdout.decode("utf-8")
-        installed_packages = [m.get("name") for m in literal_eval(installed_packages)]
-        if missing_packages := [p for p in py_reqs if p not in installed_packages]:
-            LOGGER.info("Installing Python requirements: '%s'", ", ".join(missing_packages))
-            return run_cmd(f"pip install {' '.join(missing_packages)} --disable-pip-version-check", shell=True)
-    LOGGER.debug("No py Requirements to be installed")
-
-
-def _install_py_reqs_by_odoo_cmd(addon_paths: List[Path], odoo_bin_cmd: str):
-    """Install Python reqs for modules mentioned in odoo-bin commandline --init or -i directives.
-
-    Parameters
-    ----------
-    addon_paths : List[Path]
-        odoo-bin addons-path
-    odoo_bin_cmd : str
-        odoo-bin commandline
-
-    Returns
-    -------
-    CompletedProcess
-    """
-    install_modules = []
-    for m in re.finditer(r'(--init|-i) "?([^ \n]+)"?', odoo_bin_cmd):
-        install_modules += m.group(2).split(",")
-    if install_modules:
-        return _install_py_reqs_for_modules(addon_paths=addon_paths, module_names=install_modules)
-
-
 def _boostrap_command(
     odoo_main_path: Path,
     odoo_conf_path: Path,
@@ -202,7 +154,6 @@ def bootstrap_odoo(
     addon_paths = get_addon_paths(
         odoo_main_repo=ctx.obj.odoo_main_path,
         workspace_addon_path=ctx.obj.workspace_addon_path,
-        zip_addon_path=thirdparty_addon_path / "custom",  # !Todo Pull out into variable
         thirdparty_addon_path=thirdparty_addon_path,
     )
 

godoo_cli-0.3.3/src/godoo_cli/commands/config.py

@@ -0,0 +1,37 @@
+"""
+Set odoo.conf variables
+"""
+import logging
+from configparser import ConfigParser
+from pathlib import Path
+from typing import List
+
+import typer
+
+from ..helpers.cli import typer_unpacker
+
+LOGGER = logging.getLogger(__name__)
+
+
+@typer_unpacker
+def set_odoo_config(
+    ctx: typer.Context,
+    options: List[str] = typer.Argument(..., help="odoo.conf options by key=value"),
+):
+    """
+    Set odoo.conf values.
+    """
+    conf_path = Path(ctx.obj.odoo_conf_path)
+    odoo_conf = ConfigParser()
+    odoo_conf.read(conf_path)
+    LOGGER.info("Setting Odoo Conf Options for Stage")
+    custom_opts = {}
+    for op in options:
+        if "=" not in op:
+            LOGGER.error("Cannot parse Option: '%s' --> missing = sign", op)
+            raise ValueError("")
+        op_split = op.split("=")
+        custom_opts[op_split[0]] = op_split[1]
+    LOGGER.debug("Writing Conf Options: %s", custom_opts)
+    odoo_conf["options"].update(custom_opts)
+    odoo_conf.write(conf_path.open("w"))

{godoo_cli-0.1.6 → godoo_cli-0.3.3}/src/godoo_cli/commands/launch.py

@@ -6,11 +6,10 @@ from typing import List
 import typer
 
 from ..commands.rpc.cli import rpc_callback
-from ..helpers import run_cmd
 from ..helpers.cli import typer_retuner, typer_unpacker
 from ..helpers.odoo_files import get_odoo_module_paths
+from ..helpers.system import run_cmd
 from .bootstrap import bootstrap_odoo
-from .makedev import makedev_config, makedev_rpc
 from .rpc import import_to_odoo
 
 LOGGER = logging.getLogger(__name__)
@@ -62,8 +61,7 @@ def launch_odoo(
     install_workspace_addons: bool = typer.Option(True, help="Install Workspace addons"),
     update_source: bool = typer.Option(True, help="Git Pull Addons"),
     addons_remove_unspecified: bool = typer.Option(True, help="Cleanup/Remove unspecified Addons"),
-    load_data_path: Path = typer.Option(None, help="Starts Async Importer Job with provided path"),
-    prep_stage: bool = typer.Option(False, help="Starts Async Stage Prepper"),
+    load_data_path: List[Path] = typer.Option(None, help="Starts Async Importer Job with provided path(s)"),
     extra_args: List[str] = typer.Option([], help="Extra args to Pass to odoo Launch"),
     extra_bootstrap_args: List[str] = typer.Option([], help="Extra args to Pass to odoo Bootstrap"),
     log_file_path: Path = typer.Option(None, dir_okay=False, writable=True, help="Logfile Path"),
@@ -79,8 +77,6 @@ def launch_odoo(
         log_file_path.unlink(missing_ok=True)
         extra_odoo_args.append("--logfile " + str(log_file_path.absolute()))
 
-    if ctx.obj.odoo_conf_path.exists() and prep_stage:
-        makedev_config(ctx)
     LOGGER.info("Bootstrap Flag Status: %s", ctx.obj.bootstrap_flag_location.exists())
     bootstraped = False
     ret = ""
@@ -110,31 +106,25 @@ def launch_odoo(
         extra_odoo_args += ea
 
     if dev_mode:
-        extra_odoo_args.append("--dev xml,qweb" if load_data_path else "--dev xml,qweb,reload")
+        extra_odoo_args.append(
+            "--dev xml,qweb" if load_data_path else "--dev xml,qweb,reload"
+        )  # Prevent server restart if Importer threads will be spawned.
 
     if bootstraped and not launch:
         return typer_retuner(ret)
 
     rpc_callback(ctx)  # Add RPC Options using Defaults and Envvars
 
-    if prep_stage:
-        LOGGER.info("Starting Stage Prepper Thread")
-        loader_thread = threading.Thread(
-            target=makedev_rpc,
-            args=(ctx,),
-            name="Stage Cutter",
-        )
-        loader_thread.start()
-
     if load_data_path:
-        LOGGER.info("Starting Data Importer Thread")
-        loader_thread = threading.Thread(
-            target=import_to_odoo,
-            name="DataLoader",
-            args=(ctx,),
-            kwargs={"read_path": load_data_path.absolute()},
-        )
-        loader_thread.start()
+        for p in load_data_path:
+            LOGGER.info("Starting Data Importer Thread for: '%s'", str(p))
+            loader_thread = threading.Thread(
+                target=import_to_odoo,
+                name="DataLoader",
+                args=(ctx,),
+                kwargs={"read_path": p.absolute()},
+            )
+            loader_thread.start()
 
     cmd_string = _launch_command(
         odoo_path=ctx.obj.odoo_main_path,

{godoo_cli-0.1.6 → godoo_cli-0.3.3}/src/godoo_cli/commands/shell.py

@@ -3,8 +3,8 @@ from typing import List
 
 import typer
 
-from ..helpers import run_cmd
 from ..helpers.cli import typer_retuner, typer_unpacker
+from ..helpers.system import run_cmd
 
 LOGGER = logging.getLogger(__name__)
 

{godoo_cli-0.1.6 → godoo_cli-0.3.3}/src/godoo_cli/commands/source_get.py

@@ -1,7 +1,8 @@
+"""Commands to clone Odoo and addon source code"""
 import configparser
 import logging
 import shutil
-import zipfile
+import tempfile
 from enum import Enum
 from pathlib import Path
 from typing import List
@@ -10,10 +11,11 @@ import typer
 from ruamel.yaml import YAML
 
 from ..git import GitUrl, git_ensure_addon_repos, git_ensure_odoo_repo
-from ..helpers import download_file
+from ..helpers.bootstrap import _install_py_reqs_for_modules
 from ..helpers.cli import typer_unpacker
-from ..helpers.odoo_files import get_addon_paths
+from ..helpers.odoo_files import get_addon_paths, get_odoo_module_paths, get_zip_addon_path
 from ..helpers.odoo_manifest import remove_unused_folders
+from ..helpers.system import download_file
 
 LOGGER = logging.getLogger(__name__)
 
@@ -25,27 +27,46 @@ class UpdateMode(str, Enum):
     thirdparty = "thirdparty"
 
 
-def unzip_addons(zip_folder: Path, target_addon_folder: Path, remove_excess: bool = False):
-    """Take .zip files from zip_folder and extracts them into the odoo addons folder.
+def unpack_addon_archives(archive_folder: Path, target_addon_folder: Path, remove_excess: bool = False):
+    """Take archive files from archive_folder and extract them into target_addon_folder.
 
     Parameters
     ----------
-    zip_folder : Path
+    archive_folder : Path
         Where to look for zip files
     target_addon_folder : Path
         where to place them
     remove_excess : bool , optional
         remove all and then unzip, by default False
     """
+    target_addon_folder.mkdir(exist_ok=True, parents=True)
     if remove_excess:
+        LOGGER.debug("Clearing out unarchive folder: %s", target_addon_folder)
         for folder in target_addon_folder.iterdir():
             shutil.rmtree(folder)
-    LOGGER.info("Extracting Zip Addons to: %s", target_addon_folder)
-    for zip_file in zip_folder.glob("*.zip"):
-        target_addon_folder.mkdir(exist_ok=True, parents=True)
-        LOGGER.info("Extracting Zip Addon: %s", zip_file)
-        with zipfile.ZipFile(zip_file, "r") as zip_ref:
-            zip_ref.extractall(target_addon_folder)
+    LOGGER.info("Extracting archive addons to: %s", target_addon_folder)
+    for zip_file in archive_folder.glob("*.zip"):
+        LOGGER.info("Extracting addon archive: %s", zip_file)
+        with tempfile.TemporaryDirectory() as td:
+            td = Path(td)
+            shutil.unpack_archive(zip_file, td)
+            # We can have zip files with one or more modules.
+            # Either the first folder contains multiple or its a module by itself
+            # So first get the real modules form the zip root or one level down and then move them to subpaths
+            zip_module_paths = get_odoo_module_paths([td] + list(td.glob("*/")))
+            if not zip_module_paths:
+                LOGGER.warning("Could not find valid modules in thirdparty zip: %s", zip_file)
+                continue
+            LOGGER.debug(
+                "Found modules in Zipfile:\n%s",
+                "\n".join([str(f.relative_to(td)) for f in zip_module_paths]),
+            )
+            target_folder = target_addon_folder / ("single_mods" if len(zip_module_paths) == 1 else zip_file.stem)
+            target_folder.mkdir(exist_ok=True)
+            for m in zip_module_paths:
+                module_target = target_folder / m.stem
+                shutil.rmtree(module_target, ignore_errors=True)
+                shutil.move(m, module_target)
 
 
 def update_odoo_conf_addon_paths(odoo_conf: Path, addon_paths: List[Path]):
@@ -67,6 +88,30 @@ def update_odoo_conf_addon_paths(odoo_conf: Path, addon_paths: List[Path]):
     config.write(odoo_conf.open("w"))
 
 
+@typer_unpacker
+def install_module_dependencies(
+    ctx: typer.Context,
+    thirdparty_addon_path: Path = typer.Option(
+        ...,
+        envvar="ODOO_THIRDPARTY_LOCATION",
+        help="Root folder of the Thirdparty addon repos",
+    ),
+    module_list: List[str] = typer.Argument(
+        ...,
+        help="Modules to check for dependencies (can use all for all available addons)",
+    ),
+):
+    odoo_addon_paths = get_addon_paths(
+        odoo_main_repo=ctx.obj.odoo_main_path,
+        workspace_addon_path=ctx.obj.workspace_addon_path,
+        thirdparty_addon_path=thirdparty_addon_path,
+    )
+    if len(module_list) == 1 and module_list[0] == "all":
+        search_addon_paths = [p for p in odoo_addon_paths if ctx.obj.odoo_main_path not in p.parents]
+        module_list = [p.stem for p in get_odoo_module_paths(search_addon_paths)]
+    _install_py_reqs_for_modules(odoo_addon_paths, module_list)
+
+
 @typer_unpacker
 def get_source_file(
     manifest_yml: Path = typer.Option(
@@ -101,6 +146,7 @@ def get_source_file(
 @typer_unpacker
 def get_source(
     ctx: typer.Context,
+    update_mode: UpdateMode = typer.Argument(UpdateMode.all, help="What to Update"),
     manifest_yml: Path = typer.Option(
         ..., envvar="ODOO_MANIFEST", help="Git.yml file, that specified what to download with wich prefix"
     ),
@@ -110,7 +156,6 @@ def get_source(
     thirdparty_zip_source: Path = typer.Option(
         ..., envvar="ODOO_THIRDPARTY_ZIP_LOCATION", help="Source folder, where to look for Addon zips"
     ),
-    update_mode: UpdateMode = typer.Option(UpdateMode.all, help="What to Update"),
     add_compare_comments: bool = typer.Option(
         False, help="Wether to add github.com three dot compare links as comments."
     ),
@@ -124,10 +169,10 @@ def get_source(
 ):
     """Update Odoo Source and Thirdparty source."""
     LOGGER.info("Updating Source Repos")
-    zip_addon_path = thirdparty_addon_path / "custom"  # !Todo Pull out into variable (same in bootstrap)
+    zip_addon_path = get_zip_addon_path(thirdparty_addon_path)
 
     if update_mode in ["all", "zip"]:
-        unzip_addons(thirdparty_zip_source, zip_addon_path)
+        unpack_addon_archives(thirdparty_zip_source, zip_addon_path, remove_excess=remove_unspecified_addons)
 
     if update_mode in ["all", "odoo"]:
         git_ensure_odoo_repo(
@@ -145,18 +190,17 @@ def get_source(
             generate_yml_compare_comments=add_compare_comments,
             download_archive=ctx.obj.source_download_archive,
         )
-    if remove_unspecified_addons:
-        remove_unused_folders(
-            thirdparty_addon_path=thirdparty_addon_path,
-            thirdparty_repos=git_repos.get("thirdparty", []),
-            keep_folders=[zip_addon_path],
-        )
+        if remove_unspecified_addons:
+            remove_unused_folders(
+                thirdparty_addon_path=thirdparty_addon_path,
+                thirdparty_repos=git_repos.get("thirdparty", []),
+                keep_folders=[zip_addon_path],
+            )
 
     if (conf_path := ctx.obj.odoo_conf_path).exists():
         odoo_addon_paths = get_addon_paths(
             odoo_main_repo=ctx.obj.odoo_main_path,
             workspace_addon_path=ctx.obj.workspace_addon_path,
-            zip_addon_path=zip_addon_path,
             thirdparty_addon_path=thirdparty_addon_path,
         )
         update_odoo_conf_addon_paths(odoo_conf=conf_path, addon_paths=odoo_addon_paths)

{godoo_cli-0.1.6 → godoo_cli-0.3.3}/src/godoo_cli/git/git_repo.py

@@ -1,7 +1,9 @@
+""" Module to provide GIT interaction."""
 import logging
+import shutil
 from pathlib import Path
 
-from git import InvalidGitRepositoryError, Repo
+from git import GitCommandError, InvalidGitRepositoryError, Repo
 
 from .git_url import GitUrl
 from .zip_download import git_download_zip
@@ -9,6 +11,32 @@ from .zip_download import git_download_zip
 LOGGER = logging.getLogger(__name__)
 
 
+def _git_clean_clone(repo_src: str, target_folder: Path, **kwargs):
+    """Clears targetfolder and does a clean clone_from
+
+    Parameters
+    ----------
+    repo_src : str
+        Clone url
+    target_folder : Path
+        Download folder
+    branch : str
+        branch to clone
+
+    Returns
+    -------
+    _type_re
+        _description_
+    """
+    LOGGER.debug("Cloning Repo: %s, to '%s', Kwargs: '%s'", repo_src, target_folder, kwargs)
+    if not isinstance(target_folder, Path):
+        target_folder = Path(target_folder)
+    if target_folder.exists():
+        LOGGER.debug("Clearing Repo folder: %s", target_folder)
+        shutil.rmtree(target_folder)
+    return Repo.clone_from(repo_src, target_folder, **kwargs)
+
+
 def git_pull_checkout_reset(
     repo: Repo, branch: str = "master", commit: str = "", pull: str = "", reset_hard: bool = True
 ):
@@ -30,24 +58,28 @@ def git_pull_checkout_reset(
         will call reset --hard after checkout to ensure clean git repo, by default True
     """
 
-    try:
-        if pull:
-            LOGGER.debug("Pulling Repo: %s, %s", repo.git_dir, pull)
+    if pull:
+        LOGGER.debug("Pulling Repo: %s, %s", repo.working_dir, pull)
+        try:
             repo.remotes[0].pull(pull)
-        if commit:
-            if str(repo.head.commit) != str(commit):
-                LOGGER.debug("Checking out %s to Commit: %s", repo.git_dir, commit)
-                repo.git.checkout(commit)
-            return
-        if branch:
-            LOGGER.debug("Checking Out repo %s to Branch: %s", repo.git_dir, branch)
-            repo.git.checkout(branch)
-        if reset_hard:
-            repo.git.reset("--hard")
-    except Exception as e:
-        raise Exception(
-            str(f"Error while ensuring Git repo: {str(repo)}, branch='{branch}', commit='{commit}', fetch='{pull}'")
-        ) from e
+        except GitCommandError as e:
+            if "fatal: refusing to merge unrelated histories" in e.stderr:
+                clone_kwargs = {}
+                if branch:
+                    clone_kwargs["branch"] = branch
+                repo = _git_clean_clone(repo.remotes[0].url, repo.working_dir, **clone_kwargs)
+            else:
+                raise e
+    if commit:
+        if str(repo.head.commit) != str(commit):
+            LOGGER.debug("Checking out %s to Commit: %s", repo.git_dir, commit)
+            repo.git.checkout(commit)
+        return
+    if branch:
+        LOGGER.debug("Checking Out repo %s to Branch: %s", repo.git_dir, branch)
+        repo.git.checkout(branch)
+    if reset_hard:
+        repo.git.reset("--hard")
 
 
 def git_ensure_ref(
@@ -81,23 +113,29 @@ def git_ensure_ref(
     target_folder.mkdir(exist_ok=True, parents=True)
     try:
         repo = Repo(target_folder)
-        current = repo.head.commit
+        current = str(repo.head.commit)
 
         if not pull and commit and str(commit) != str(current):
             pull = str(commit)
 
         if not pull and branch and not commit:
-            pull = str(branch)
-        git_pull_checkout_reset(repo=repo, branch=branch, commit=commit, pull=pull)
+            remote_name = str(repo.remotes[0].name)
+            remote_commit = repo.git.rev_parse(remote_name + "/" + str(branch))
+            LOGGER.debug(
+                "Repo: %s comparing local head '%s' with remote head '%s'", repo.working_dir, current, remote_commit
+            )
+            if str(remote_commit) != current:
+                pull = str(branch)
+
+            git_pull_checkout_reset(repo=repo, branch=branch, commit=commit, pull=pull)
 
-        if current == repo.head.commit:
+        if current == str(repo.head.commit):
             LOGGER.info("Repo Commit matches. Skipping: '%s' --> '%s'", repo_src, target_folder)
         else:
             LOGGER.info("Pulled Repo from: '%s'. Head is now at: %s", repo_src, repo.head.commit)
     except InvalidGitRepositoryError:
-        LOGGER.debug("Cloning Repo: %s, to '%s', branch '%s'", repo_src, target_folder, branch)
-        repo = Repo.clone_from(repo_src, target_folder, branch=branch, **kwargs)
-        git_pull_checkout_reset(repo=repo, branch=branch, commit=commit, pull=pull)
+        repo = _git_clean_clone(repo_src, target_folder, branch=branch, **kwargs)
+        git_pull_checkout_reset(repo=repo, branch=branch, commit=commit, pull=False)
 
         LOGGER.info(
             "Cloned Repo: '%s'. Branch='%s' Head='%s'",

{godoo_cli-0.1.6 → godoo_cli-0.3.3}/src/godoo_cli/git/zip_download.py

@@ -3,7 +3,7 @@ import tempfile
 import zipfile
 from pathlib import Path
 
-from ..helpers import download_file
+from ..helpers.system import download_file
 from .git_url import GitUrl
 
 LOGGER = logging.getLogger(__name__)

godoo_cli-0.3.3/src/godoo_cli/helpers/__init__.py

@@ -0,0 +1 @@
+"""Provides Various Helper Functions for Odoo"""

godoo_cli-0.3.3/src/godoo_cli/helpers/bootstrap.py

@@ -0,0 +1,58 @@
+import logging
+import re
+import subprocess
+from ast import literal_eval
+from pathlib import Path
+from typing import List
+
+from .odoo_files import _get_python_requirements_of_modules
+from .system import run_cmd
+
+LOGGER = logging.getLogger(__name__)
+
+
+def _install_py_reqs_for_modules(addon_paths: List[Path], module_names: List[str] = None):
+    """Install Python Requirements mentioned in odoo module manifests of given modules
+
+    Parameters
+    ----------
+    addon_paths : List[Path]
+        odoo-bin addons path
+    module_names : List[str], Optional
+        List of modules to filter for, by default "all available modules"
+
+    Returns
+    -------
+    CompletedProcess
+    """
+    py_reqs = _get_python_requirements_of_modules(addon_paths=addon_paths, filter_module_names=module_names)
+    if py_reqs:
+        installed_packages = run_cmd(
+            "pip list --format json --disable-pip-version-check", check=True, shell=True, stdout=subprocess.PIPE
+        ).stdout.decode("utf-8")
+        installed_packages = [m.get("name") for m in literal_eval(installed_packages)]
+        if missing_packages := [p for p in py_reqs if p not in installed_packages]:
+            LOGGER.info("Installing Python requirements: '%s'", ", ".join(missing_packages))
+            return run_cmd(f"pip install {' '.join(missing_packages)} --disable-pip-version-check", shell=True)
+    LOGGER.debug("No py Requirements to be installed")
+
+
+def _install_py_reqs_by_odoo_cmd(addon_paths: List[Path], odoo_bin_cmd: str):
+    """Install Python reqs for modules mentioned in odoo-bin commandline --init or -i directives.
+
+    Parameters
+    ----------
+    addon_paths : List[Path]
+        odoo-bin addons-path
+    odoo_bin_cmd : str
+        odoo-bin commandline
+
+    Returns
+    -------
+    CompletedProcess
+    """
+    install_modules = []
+    for m in re.finditer(r'(--init|-i) "?([^ \n]+)"?', odoo_bin_cmd):
+        install_modules += m.group(2).split(",")
+    if install_modules:
+        return _install_py_reqs_for_modules(addon_paths=addon_paths, module_names=install_modules)

{godoo_cli-0.1.6 → godoo_cli-0.3.3}/src/godoo_cli/helpers/odoo_files.py

@@ -9,6 +9,10 @@ from git import Repo
 LOGGER = logging.getLogger(__name__)
 
 
+def get_zip_addon_path(thirdparty_path: Path) -> Path:
+    return thirdparty_path / "custom"
+
+
 def get_odoo_module_paths(search_folders: Union[List[Path], Path]) -> List[Path]:
     """List all Valid odoo module names in one or many Odoo addons folder.
 
@@ -101,7 +105,6 @@ def get_depends_of_module(
         return []
     already_done_modules.append(module_to_check.absolute())
 
-    LOGGER.debug("Loading Manifest: %s", manifest_path.absolute())
     manifest = literal_eval(manifest_path.read_text())
     module_depends = manifest.get("depends", [])
     sub_depends = []
@@ -122,7 +125,6 @@ def get_depends_of_module(
 def get_addon_paths(
     odoo_main_repo: Path,
     workspace_addon_path: Path,
-    zip_addon_path: Path,
     thirdparty_addon_path: Path,
 ) -> List[Path]:
     """Get Odoo Addon Paths for odoo.conf.
@@ -133,8 +135,6 @@ def get_addon_paths(
         Path to main odoo repo
     workspace_addon_path : Path
         Path to workspace addons
-    zip_addon_path : Path
-        path to zip addons
     thirdparty_addon_path : Path
         path to git cloned addon repos
 
@@ -146,6 +146,7 @@ def get_addon_paths(
     odoo_addon_paths = [odoo_main_repo / "addons"]
     if get_odoo_module_paths(workspace_addon_path):
         odoo_addon_paths.append(workspace_addon_path)
+    zip_addon_path = get_zip_addon_path(thirdparty_addon_path)
     zip_addon_repos = [f for f in zip_addon_path.iterdir() if f.is_dir() and get_odoo_module_paths(f)]
     odoo_addon_paths += zip_addon_repos
     git_thirdparty_addon_repos = [
@@ -155,6 +156,27 @@ def get_addon_paths(
     return odoo_addon_paths
 
 
+def _get_odoo_main_path(addon_paths: List[Path]) -> Path:
+    """Get Odoo main path by location of odoo-bin from addon folders.
+
+    Parameters
+    ----------
+    addon_paths : List[Path]
+        List of possible paths
+
+    Returns
+    -------
+    Path
+        Path with odoo-bin and main addons folder
+    """
+    for path in addon_paths:
+        # The main odoo addon path is odoo/addons.
+        # so one up should be the launch script
+        bin_path = path.parent / "odoo-bin"
+        if bin_path.exists() and bin_path.is_file():
+            return path
+
+
 def _get_python_requirements_of_modules(addon_paths: List[Path], filter_module_names: List[str] = None):
     """Install python requirements mentioned in module manifests
 
@@ -170,31 +192,42 @@ def _get_python_requirements_of_modules(addon_paths: List[Path], filter_module_n
 
     if not filter_module_names:
         filter_module_names = available_module_names
-    filter_module_names = [f for f in filter_module_names if f not in ["base", "web"]]
-    LOGGER.info("Checking python requirements of Modules: %s", ", ".join(sorted(filter_module_names)))
+    filter_module_names = [f for f in filter_module_names if f != "base"]
 
-    if unavailable_modules := [m for m in filter_module_names if m not in available_module_names]:
-        LOGGER.warning("Couldn't search Python reqs for unavailable Modules: %s", ", ".join(unavailable_modules))
+    LOGGER.info("Checking python requirements of Modules:\n%s", "\n".join(sorted(filter_module_names)))
+    if unavailable_modules := set(filter_module_names).difference(available_module_names):
+        LOGGER.warning("Could not find __manifest__ for:\n%s", "\n".join(unavailable_modules))
+        LOGGER.debug("Search Paths:\n%s", "\n".join(map(str, addon_paths)))
 
     check_modules = [mp for mp in available_modules if mp.stem in filter_module_names]
+
+    if odoo_main_path := _get_odoo_main_path(addon_paths):
+        # When the Odoo main path is in the supplied path, exclude those modules.
+        # Odoos base requirements are set in their requirements.txt
+        LOGGER.debug("Py install: Excluding Odoo main Modules in: '%s'", odoo_main_path)
+        check_modules = [mp for mp in check_modules if odoo_main_path not in mp.parents]
+
     check_modules_dependencies = []
     for module in check_modules:
         check_modules_dependencies += get_depends_of_module(
             available_modules, module, already_done_modules=check_modules_dependencies
         )
     LOGGER.debug(
-        "adding child modules to check list: %s", ", ".join(sorted([p.stem for p in check_modules_dependencies]))
+        "adding children of requested modules to check list:\n%s",
+        "\n".join(sorted([p.stem for p in check_modules_dependencies])),
     )
 
     check_modules += check_modules_dependencies
 
+    check_modules = set(check_modules)
+
     if not check_modules:
         LOGGER.debug("No Modules provided to check for python Requirements")
         return
     python_depends = []
     for module_path in check_modules:
+        LOGGER.debug("Checking for External Depends: %s", module_path)
         manifest_path = module_path / "__manifest__.py"
-        LOGGER.debug("Loading Manifest: %s", manifest_path.absolute())
         manifest = literal_eval(manifest_path.read_text())
         if module_depends := manifest.get("external_dependencies", {}).get("python"):
             python_depends += module_depends

godoo_cli-0.1.6/src/godoo_cli/helpers/__init__.py → godoo_cli-0.3.3/src/godoo_cli/helpers/system.py

@@ -1,13 +1,15 @@
-"""Provides Various Helper Functions for Odoo"""
-
+"""Helper functions around the host system"""
 import logging
 import os
 import subprocess
-import sys
 from pathlib import Path
 
+import click
 import requests
 from rich.logging import RichHandler
+from rich.traceback import install as install_rich_traceback
+
+from . import cli as godoo_cli_helpers
 
 LOGGER = logging.getLogger(__name__)
 
@@ -68,6 +70,7 @@ def set_logging(verbose: bool = False) -> None:
     verbose : bool
         Wether to Log debug Messages
     """
+    install_rich_traceback(suppress=[click, godoo_cli_helpers])
     if verbose:
         logging.basicConfig(
             level=logging.DEBUG,

godoo_cli-0.1.6/src/godoo_cli/commands/makedev.py

@@ -1,138 +0,0 @@
-"""
-Disconnects SMTP From odoo via Config.
-Disconnects Label Printers via RPC.
-Installs and configures Web Ribbon Banner.
-Disables Audit rules via RPC.
-"""
-
-import logging
-from configparser import ConfigParser
-
-import typer
-from godoo_rpc.login import wait_for_odoo
-
-from ..commands.rpc.cli import rpc_callback
-from ..helpers.cli import typer_unpacker
-
-LOGGER = logging.getLogger(__name__)
-
-
-def remove_label_printers(odoo_api, printer_url="host.docker.internal"):
-    """Set Odoo Label Printer URL.
-
-    Parameters
-    ----------
-    odoo_api : _type_
-        Logged in Odoo API
-    printer_url : str, optional
-        printer url, by default "host.docker.internal"
-    """
-    printer_env = odoo_api.session.env["wetech.label.printer"]
-    l_printer_ids = printer_env.search([])
-    l_printers = printer_env.browse(l_printer_ids)
-    LOGGER.info("Setting Label Printer URL for %s Printers", len(l_printers))
-    l_printers.write({"hostname": printer_url})
-
-
-def staging_banner(odoo_api, banner_text: str = "", banner_background_color: str = ""):
-    """Set Odoo top left Banner.
-
-    Parameters
-    ----------
-    odoo_api : _type_
-        Logged in Odoo API
-    banner_text : str, optional
-        Text to display on banner, by default ""
-    banner_background_color : str, optional
-        rbg("","","",""), by default ""
-    """
-    mod = odoo_api.session.env["ir.module.module"]
-    mod_ids = mod.search(["&", ("name", "in", ["web_environment_ribbon"]), ("state", "=", "uninstalled")])
-    modules = mod.browse(mod_ids)
-    if modules:
-        LOGGER.info("Installing Modules: %s", ",".join(modules.mapped("name")))
-        modules.button_immediate_install()
-    param_env = odoo_api.session.env["ir.config_parameter"]
-    ribbon_text_id = param_env.search([("key", "=", "ribbon.name")])
-    param_env.browse(ribbon_text_id).value = banner_text
-    LOGGER.info("Setting Banner Text to: %s", banner_text)
-    ribbon_text_id = param_env.search([("key", "=", "ribbon.background.color")])
-    param_env.browse(ribbon_text_id).value = banner_background_color
-    LOGGER.info("Setting Banner background color: %s", banner_background_color)
-
-
-def disable_audit_logs(odoo_api):
-    """Disable audit log rules.
-
-    Parameters
-    ----------
-    odoo_api : _type_
-        Logged in Odoo API
-    """
-    mod = odoo_api.session.env["auditlog.rule"]
-    rule_ids = mod.search([("state", "=", "subscribed")])
-    rules = mod.browse(rule_ids)
-    if rules:
-        LOGGER.info("Disabling '%s' audit log rules", len(rules))
-        rules.unsubscribe()
-
-
-@typer_unpacker
-def makedev_config(ctx: typer.Context):
-    """
-    Set odoo.conf values for Staging Env.
-    """
-    odoo_conf = ConfigParser()
-    odoo_conf.read(ctx.obj.odoo_conf_path)
-    LOGGER.info("Setting Odoo Conf Options for Stage")
-    odoo_conf["options"].update(
-        {
-            "smtp_server": "localhost",
-            "smtp_password": "False",
-            "smtp_user": "False",
-            "logfile": "",
-            "limit_memory_hard": "2684354560",
-            "limit_memory_soft": "2147483648",
-            "workers": "3",
-            "max_cron_threads": "1",
-            "limit_time_cpu": "36000",
-            "limit_time_real": "42000",
-        }
-    )
-
-    with ctx.obj.odoo_conf_path.open("w") as conf:
-        odoo_conf.write(conf)
-
-
-@typer_unpacker
-def makedev_rpc(
-    ctx: typer.Context,
-    banner_text: str = typer.Option(
-        "Development", envvar="ODOO_BANNER_TEXT", help="Text to display in Top left ribbon banner"
-    ),
-    banner_background_color: str = typer.Option(
-        "rgba(255,0,0,.6)", envvar="ODOO_BANNER_BG_COLOR", help="Top left ribbon banner color"
-    ),
-):
-    """
-    Set some Odoo Options via RPC to disable prod features.
-    """
-
-    LOGGER.info("Starting Stagesplitter")
-    odoo_api = wait_for_odoo(
-        odoo_host=ctx.obj.odoo_rpc_host,
-        odoo_db=ctx.obj.odoo_main_db,
-        odoo_user=ctx.obj.odoo_rpc_user,
-        odoo_password=ctx.obj.odoo_rpc_password,
-    )
-
-    remove_label_printers(odoo_api=odoo_api)
-    staging_banner(odoo_api=odoo_api, banner_text=banner_text, banner_background_color=banner_background_color)
-    disable_audit_logs(odoo_api=odoo_api)
-
-
-def makedec_cli_app():
-    app = typer.Typer(callback=rpc_callback, no_args_is_help=True)
-    app.command("rpc", help="Set odoo.conf values for Staging Env.")(makedev_rpc)
-    app.command("config", help="Set some Odoo Options via RPC to disable prod features.")(makedev_config)
-    return app