zrb 0.0.109__tar.gz → 0.0.110__tar.gz

{zrb-0.0.109 → zrb-0.0.110}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: zrb
-Version: 0.0.109
+Version: 0.0.110
 Summary: Super framework for your super app
 Author-email: Go Frendi Gunawan <gofrendiasgard@gmail.com>
 Requires-Python: >=3.10.0

{zrb-0.0.109 → zrb-0.0.110}/docs/concepts/tasks/README.md

@@ -12,7 +12,7 @@ There are many task types in Zrb. Every task has its own specific use cases:
 - [ResourceMaker](resource-maker.md): Generate artifacts/resources based on templates
 - [FlowTask](flow-task.md): Put `CmdTask` and `python task` into single flow.
 - [RemoteCmdTask](remote-cmd-task.md)
-- [RsyncTask](remote-cmd-task.md)
+- [RsyncTask](rsync-task.md)
 - [Checkers (HttpChecker, PortChecker, and PathChecker)](checkers.md): Check parent task's readiness.
 
 As every task are extended from `BaseTask`, you will see that most of them share some common parameters.
@@ -400,13 +400,45 @@ Every task share some common methods like `run`, `check`, and `to_function`.
 
 Deep copy current task
 
-## `add_envs`
+## `inject_env`
 
-## `add_env_files`
+To be overridden
 
-## `add_inputs`
+## `insert_env`
 
-## `add_upstreams`
+## `add_env`
+
+## `inject_env_file`
+
+To be overridden
+
+## `insert_env_file`
+
+## `add_env_file`
+
+## `inject_input`
+
+To be overridden
+
+## `insert_input`
+
+## `add_input`
+
+## `inject_upstream`
+
+To be overridden
+
+## `insert_upstream`
+
+## `add_upstream`
+
+## `inject_checker`
+
+To be overridden
+
+## `insert_checker`
+
+## `add_checker`
 
 ## `set_name`
 

{zrb-0.0.109 → zrb-0.0.110}/docs/concepts/tasks/remote-cmd-task.md

@@ -27,4 +27,13 @@ install_curl = RemoteCmdTask(
 runner.register(install_curl)
 ```
 
+RemoteCmdTask exposes several environments that you can use on your `cmd` and `cmd_path`
+
+- `_CONFIG_HOST`
+- `_CONFIG_PORT`
+- `_CONFIG_SSH_KEY`
+- `_CONFIG_USER`
+- `_CONFIG_PASSWORD`
+- `_CONFIG_MAP_<UPPER_SNAKE_CASE_NAME>`
+
 🔖 [Table of Contents](../../README.md) / [Concepts](../README.md) / [Tasks](README.md)

{zrb-0.0.109 → zrb-0.0.110}/docs/concepts/tasks/rsync-task.md

@@ -1,13 +1,13 @@
 🔖 [Table of Contents](../../README.md) / [Concepts](../README.md) / [Tasks](README.md)
 
-# RSyncTask
+# RsyncTask
 
 ```python
 from zrb import (
-    runner, CmdTask, RSyncTask, RemoteConfig, PasswordInput, StrInput
+    runner, CmdTask, RsyncTask, RemoteConfig, PasswordInput, StrInput
 )
 
-upload = RSyncTask(
+upload = RsyncTask(
     name='upload',
     inputs=[
         PasswordInput(name='passsword'),
@@ -18,17 +18,20 @@ upload = RSyncTask(
         RemoteConfig(
             host='192.168.1.10,
             user='ubuntu,
-            password='{{input.password}}'
+            password='{{input.password}}',
+            config_map={
+                'dir': '192-168-1-10'
+            }
         )
     ],
     is_remote_src=False,
-    is_remote_dst=True
-    src='{{input.src}}',
+    src='$_CONFIG_MAP_DIR/{{input.src}}',
+    is_remote_dst=True,
     dst='{{input.dst}}',
 )
 runner.register(upload)
 
-download = RSyncTask(
+download = RsyncTask(
     name='download',
     inputs=[
         PasswordInput(name='passsword'),
@@ -43,11 +46,20 @@ download = RSyncTask(
         )
     ],
     is_remote_src=True,
-    is_remote_dst=False
     src='{{input.src}}',
-    dst='{{input.dst}}',
+    is_remote_dst=False,
+    dst='$_CONFIG_MAP_DIR/{{input.dst}}',
 )
 runner.register(download)
 ```
 
+RsyncTask exposes several environments that you can use on your `src` and `dst`
+
+- `_CONFIG_HOST`
+- `_CONFIG_PORT`
+- `_CONFIG_SSH_KEY`
+- `_CONFIG_USER`
+- `_CONFIG_PASSWORD`
+- `_CONFIG_MAP_<UPPER_SNAKE_CASE_NAME>`
+
 🔖 [Table of Contents](../../README.md) / [Concepts](../README.md) / [Tasks](README.md)

zrb-0.0.110/docs/oops-i-did-it-again/using-the-same-variable-to-define-different-task.md

@@ -0,0 +1,39 @@
+🔖 [Table of Contents](../README.md) / [Oops, I Did It Again](README.md)
+
+# Using The Same Variable to Define Different Task
+
+```python
+from zrb import runner, CmdTask
+
+prepare = CmdTask(
+    name='prepare-python-project',
+    cmd='pip install -R requirements.txt'
+)
+runner.register(prepare)
+
+prepare = CmdTask(
+    name='prepare-node-project',
+    cmd='npm install'
+)
+runner.register(prepare)
+
+run_fastapi = CmdTask(
+    name='run-fastapi',
+    cmd='uvicon main:app'
+    upstreams=[
+        prepare, # <-- Here is the problem, `npm install`` or `pip install`? 
+    ]
+)
+runner.register(run_fastapi)
+```
+
+You can see that `prepare-python-project` and `prepare-node-project` are assigned to the same variable.
+
+Using that variable as upstream or checker will lead to a tricky situation. In our case, we want to perform `pip install` before starting Fast API. But since we re-assign the variable to `prepare-node-project`, we will got `npm install` instead.
+
+# Avoiding the Problem
+
+Beware of your variable name. Give your variable the same name as your task name.
+
+
+🔖 [Table of Contents](../README.md) / [Oops, I Did It Again](README.md)

{zrb-0.0.109 → zrb-0.0.110}/docs/quirks.md

@@ -18,6 +18,5 @@
     - `env` will override each other, the last one takes greater priority
     - If you define a `DockerComposeTask`, it will automatically fill your environment with the ones you use in your docker-compose file. The environment defined that way will have a very low priority. They will be overridden by both `env_files` and `env`.
 - You cannot have an input named: `_task`, `_args` or `_execution_id`
-- You cannot have an environment named `_execution_id`
 
 🔖 [Table of Contents](README.md)

{zrb-0.0.109 → zrb-0.0.110}/docs/tutorials/copy-and-reuse-task.md

@@ -20,10 +20,10 @@ local_hello = hello.copy()
 
 # Update name, input, and env
 local_hello.set_name('hello-local')
-local_hello.add_inputs(
+local_hello.add_input(
     StrInput(name='name', description='Name', default='dunia')
 )
-local_hello.add_envs(
+local_hello.add_env(
     Env(name='GREETINGS', os_name='', default='Halo')
 )
 

{zrb-0.0.109 → zrb-0.0.110}/docs/tutorials/extending-cmd-task.md

@@ -56,18 +56,23 @@ class SlackPrintTask(CmdTask):
         self._slack_app_token = slack_app_token
         self._message = message
 
-    def run(self, *args: Any, **kwargs: Any):
-        # Inject environment variables
-        self.inject_env_map(
-            env_map={
-                'CHANNEL_ID': self.render_str(self._slack_channel_id),
-                'TOKEN': self.render_str(self._slack_app_token),
-                'MESSAGE': self.render_str(self._message)
-            }
+    def inject_envs(self):
+        self.add_envs(
+            Env(
+                name='CHANNEL_ID', os_name='',
+                default=self.render_str(self._slack_channel_id)
+            ),
+            Env(
+                name='TOKEN', os_name='',
+                default=self.render_str(self._slack_app_token)
+            ),
+            Env(
+                name='MESSAGE', os_name='',
+                default=self.render_str(self._message)
+            )
         )
-        return super().run(*args, **kwargs)
 
-    def _get_cmd_str(self, *args: Any, **kwargs: Any):
+    def get_cmd_script(self, *args: Any, **kwargs: Any):
         # contruct json payload and replace all `"` with `\\"`
         json_payload = jsons.dumps({
             'channel': '$CHANNEL_ID',

{zrb-0.0.109 → zrb-0.0.110}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "flit_core.buildapi"
 
 [project]
 name = "zrb"
-version = "0.0.109"
+version = "0.0.110"
 authors = [
   { name="Go Frendi Gunawan", email="gofrendiasgard@gmail.com" },
 ]

{zrb-0.0.109 → zrb-0.0.110}/src/zrb/action/runner.py

@@ -28,7 +28,7 @@ class Runner():
 
     def register(self, task: AnyTask):
         task.set_has_cli_interface()
-        cmd_name = task.get_complete_cmd_name()
+        cmd_name = task.get_full_cmd_name()
         logger.debug(colored(f'Register task: {cmd_name}', attrs=['dark']))
         self._tasks.append(task)
         logger.debug(colored(f'Task registered: {cmd_name}', attrs=['dark']))
@@ -77,7 +77,7 @@ class Runner():
         return group
 
     def _create_cli_command(self, task: AnyTask) -> click.Command:
-        task_inputs = task.get_all_inputs()
+        task_inputs = task._get_combined_inputs()
         task_cmd_name = task.get_cmd_name()
         task_description = task.get_description()
         task_function = task.to_function(

{zrb-0.0.109 → zrb-0.0.110}/src/zrb/builtin/git.py

@@ -1,7 +1,6 @@
 from zrb.helper.typing import Any
 from zrb.builtin.group import git_group
 from zrb.task.decorator import python_task
-from zrb.task.task import Task
 from zrb.task_input.str_input import StrInput
 from zrb.task_input.bool_input import BoolInput
 from zrb.runner import runner
@@ -26,7 +25,7 @@ from zrb.helper.python_task import show_lines
             prompt='Commit hash/Tag',
             default='HEAD'
         ),
-       BoolInput(
+        BoolInput(
             name='include-new',
             description='include new files',
             prompt='Include new files',
@@ -52,7 +51,6 @@ async def get_file_changes(*args: Any, **kwargs: Any):
     include_new = kwargs.get('include_new', True)
     include_removed = kwargs.get('include_removed', True)
     include_updated = kwargs.get('include_updated', True)
-    task: Task = kwargs['_task']
     modified_file_states = get_modified_file_states(commit)
     modified_file_keys = []
     output = []

zrb-0.0.110/src/zrb/helper/accessories/name.py

@@ -0,0 +1,45 @@
+from zrb.helper.typecheck import typechecked
+import random
+import string
+
+
+@typechecked
+def get_random_name(
+    separator: str = '-',
+    add_random_digit: bool = True,
+    digit_count: int = 4
+) -> str:
+    prefixes = [
+        'albedo', 'argent', 'argentum', 'aurora', 'aurum', 'azure',
+        'basilisk', 'cerulean', 'chimeric', 'citrin', 'coral', 'crimson',
+        'diamond', 'draco', 'dragon', 'emerald', 'ethereal', 'ferrum',
+        'flammeus', 'garnet', 'glacial', 'glimmering', 'glistening', 'golden',
+        'helios', 'igneous', 'imperial', 'jade', 'luminous', 'luna', 'lunar',
+        'mystic', 'nephrite', 'nocturnal', 'obsidian', 'opal', 'pearl',
+        'platinum', 'prismatic', 'ruby', 'sapphire', 'serpentine', 'silver',
+        'sol', 'solar', 'spiritual', 'stellar',  'tempest', 'topaz',
+        'turquoise', 'verde', 'vermillion', 'vitreous', 'zephyr', 'zircon'
+    ]
+    suffixes = [
+        'aether', 'albedo', 'alchemy', 'arcana', 'aureum', 'aetheris',
+        'anima', 'astralis', 'caelestis', 'chrysopoeia', 'cosmicum',
+        'crystallum', 'deum', 'divinitas', 'draconis', 'elementorum', 'elixir',
+        'essentia', 'eternis', 'ethereus', 'fatum', 'flamma', 'fulgur',
+        'hermetica', 'ignis', 'illuminationis', 'imperium', 'incantatum',
+        'infinitum', 'lapis', 'lux', 'magicae', 'magnum', 'materia',
+        'metallum', 'mysticum', 'natura', 'occultum', 'omnipotentis',
+        'opulentia', 'philosophia', 'philosophorum', 'praeparatum',
+        'praestantissimum', 'prima', 'primordium', 'quintessentia',
+        'regeneratio', 'ritualis', 'sanctum', 'spiritus', 'tenebris',
+        'terra', 'tinctura', 'transmutationis', 'universalis', 'vapores',
+        'venenum', 'veritas', 'vitae', 'volatus'
+    ]
+    prefix = random.choice(prefixes)
+    suffix = random.choice(suffixes)
+    parts = [prefix, suffix]
+    if add_random_digit:
+        random_digit = ''.join(
+            random.choices(string.digits, k=digit_count)
+        )
+        parts.append(random_digit)
+    return separator.join(parts)

{zrb-0.0.109 → zrb-0.0.110}/src/zrb/helper/env_map/fetch.py

@@ -16,7 +16,7 @@ def fetch_env_map_from_group(
         sub_env_map: Mapping[str, str] = fetch_env_map_from_group(
             env_map, sub_group
         )
-        env_map = cascade_env_map(env_map, sub_env_map)
+        env_map = _cascade_env_map(env_map, sub_env_map)
     return env_map
 
 
@@ -25,33 +25,33 @@ def fetch_env_map_from_task(
     env_map: Mapping[str, str], task: AnyTask
 ):
     task_env_map: Mapping[str, str] = {}
-    for env_file in task.get_env_files():
+    for env_file in task._get_env_files():
         envs = env_file.get_envs()
-        task_env_map = add_envs_to_env_map(task_env_map, envs)
-    task_env_map = add_envs_to_env_map(task_env_map, task._envs)
-    env_map = cascade_env_map(env_map, task_env_map)
-    for upstream in task.get_upstreams():
+        task_env_map = _add_envs_to_env_map(task_env_map, envs)
+    task_env_map = _add_envs_to_env_map(task_env_map, task._envs)
+    env_map = _cascade_env_map(env_map, task_env_map)
+    for upstream in task._get_upstreams():
         task_env_map = fetch_env_map_from_task(env_map, upstream)
-    for checker in task.get_checkers():
+    for checker in task._get_checkers():
         task_env_map = fetch_env_map_from_task(env_map, checker)
     return env_map
 
 
 @typechecked
-def add_envs_to_env_map(
+def _add_envs_to_env_map(
     env_map: Mapping[str, str], envs: List[Env]
 ) -> Mapping[str, str]:
     for env in envs:
         if env.os_name == '':
             continue
-        env_name = get_env_name(env)
-        env_default = get_env_default(env)
+        env_name = _get_env_name(env)
+        env_default = _get_env_default(env)
         env_map[env_name] = env_default
     return env_map
 
 
 @typechecked
-def cascade_env_map(
+def _cascade_env_map(
     env_map: Mapping[str, str],
     other_env_map: Mapping[str, str]
 ) -> Mapping[str, str]:
@@ -63,14 +63,14 @@ def cascade_env_map(
 
 
 @typechecked
-def get_env_name(env: Env) -> str:
+def _get_env_name(env: Env) -> str:
     if env.os_name is None:
         return env.name
     return env.os_name
 
 
 @typechecked
-def get_env_default(env: Env) -> str:
+def _get_env_default(env: Env) -> str:
     if is_probably_jinja(env.default):
         return ''
     return env.default

zrb-0.0.110/src/zrb/shell-scripts/rsync-util.sh

@@ -0,0 +1,12 @@
+set -e
+auth_rsync(){
+    if [ "$_CONFIG_SSH_KEY" != "" ]
+    then
+        rsync --mkpath -avz -e "ssh -i $_CONFIG_SSH_KEY -p $_CONFIG_PORT" $@
+    elif [ "$_CONFIG_PASSWORD" != "" ]
+    then
+        sshpass -p "$_CONFIG_PASSWORD" rsync --mkpath -avz -e "ssh -p $_CONFIG_PORT" $@
+    else
+        rsync --mkpath -avz -e "ssh -p $_CONFIG_PORT" $@
+    fi
+}

{zrb-0.0.109 → zrb-0.0.110}/src/zrb/task/any_task.py

@@ -65,23 +65,35 @@ class AnyTask(ABC):
         pass
 
     @abstractmethod
-    def add_upstreams(self, *upstreams: TAnyTask):
+    def add_upstream(self, *upstreams: TAnyTask):
         pass
 
     @abstractmethod
-    def add_inputs(self, *inputs: AnyInput):
+    def insert_input(self, *inputs: AnyInput):
         pass
 
     @abstractmethod
-    def add_envs(self, *envs: Env):
+    def add_input(self, *inputs: AnyInput):
         pass
 
     @abstractmethod
-    def add_env_files(self, *env_files: EnvFile):
+    def insert_env(self, *envs: Env):
         pass
 
     @abstractmethod
-    def set_execution_id(self, execution_id: str):
+    def add_env(self, *envs: Env):
+        pass
+
+    @abstractmethod
+    def insert_env_file(self, *env_files: EnvFile):
+        pass
+
+    @abstractmethod
+    def add_env_file(self, *env_files: EnvFile):
+        pass
+
+    @abstractmethod
+    def _set_execution_id(self, execution_id: str):
         pass
 
     @abstractmethod
@@ -139,31 +151,51 @@ class AnyTask(ABC):
         pass
 
     @abstractmethod
-    def get_complete_cmd_name(self) -> str:
+    def get_full_cmd_name(self) -> str:
+        pass
+
+    @abstractmethod
+    def inject_env_files(self):
+        pass
+
+    @abstractmethod
+    def _get_env_files(self) -> List[EnvFile]:
+        pass
+
+    @abstractmethod
+    def inject_envs(self):
+        pass
+
+    @abstractmethod
+    def _get_envs(self) -> List[Env]:
         pass
 
     @abstractmethod
-    def get_env_files(self) -> List[EnvFile]:
+    def inject_inputs(self):
         pass
 
     @abstractmethod
-    def get_envs(self) -> List[Env]:
+    def _get_inputs(self) -> List[AnyInput]:
         pass
 
     @abstractmethod
-    def get_inputs(self) -> List[AnyInput]:
+    def inject_checkers(self):
         pass
 
     @abstractmethod
-    def get_checkers(self) -> Iterable[TAnyTask]:
+    def _get_checkers(self) -> Iterable[TAnyTask]:
         pass
 
     @abstractmethod
-    def get_upstreams(self) -> Iterable[TAnyTask]:
+    def inject_upstreams(self):
         pass
 
     @abstractmethod
-    def get_all_inputs(self) -> Iterable[AnyInput]:
+    def _get_upstreams(self) -> Iterable[TAnyTask]:
+        pass
+
+    @abstractmethod
+    def _get_combined_inputs(self) -> Iterable[AnyInput]:
         pass
 
     @abstractmethod
@@ -206,12 +238,6 @@ class AnyTask(ABC):
     def get_env_map(self) -> Mapping[str, Any]:
         pass
 
-    @abstractmethod
-    def inject_env_map(
-        self, env_map: Mapping[str, str], override: bool = False
-    ):
-        pass
-
     @abstractmethod
     def render_any(
         self, val: Any, data: Optional[Mapping[str, Any]] = None

{zrb-0.0.109 → zrb-0.0.110}/src/zrb/task/base_remote_cmd_task.py

@@ -2,6 +2,7 @@ from zrb.helper.typing import (
     Any, Callable, Iterable, Mapping, Optional, Union, TypeVar
 )
 from zrb.helper.typecheck import typechecked
+from zrb.helper.util import to_snake_case
 from zrb.task.any_task import AnyTask
 from zrb.task.any_task_event_handler import (
     OnTriggered, OnWaiting, OnSkipped, OnStarted, OnReady, OnRetry, OnFailed
@@ -34,13 +35,15 @@ class RemoteConfig:
         user: str = '',
         password: str = '',
         ssh_key: str = '',
-        port: int = 22
+        port: int = 22,
+        config_map: Optional[Mapping[str, str]] = None
     ):
         self.host = host
         self.user = user
         self.password = password
         self.ssh_key = ssh_key
         self.port = port
+        self.config_map = {} if config_map is None else config_map
 
 
 @typechecked
@@ -123,28 +126,50 @@ class SingleBaseRemoteCmdTask(CmdTask):
     def copy(self) -> TSingleBaseRemoteCmdTask:
         return copy.deepcopy(self)
 
-    def _get_shell_env_map(self) -> Mapping[str, Any]:
-        env_map = super()._get_shell_env_map()
-        env_map['_CONFIG_HOST'] = self.render_str(self._remote_config.host)
-        env_map['_CONFIG_PORT'] = str(self.render_int(
-            self._remote_config.port)
-        )
-        env_map['_CONFIG_SSH_KEY'] = self.render_str(
-            self._remote_config.ssh_key
-        )
-        env_map['_CONFIG_USER'] = self.render_str(self._remote_config.user)
-        env_map['_CONFIG_PASSWORD'] = self.render_str(
-            self._remote_config.password
+    def inject_envs(self):
+        super().inject_envs()
+        # add remote config properties as env
+        self.add_env(
+            Env(
+                name='_CONFIG_HOST', os_name='',
+                default=self.render_str(self._remote_config.host)
+            ),
+            Env(
+                name='_CONFIG_PORT', os_name='',
+                default=str(self.render_int(self._remote_config.port))
+            ),
+            Env(
+                name='_CONFIG_SSH_KEY', os_name='',
+                default=self.render_str(self._remote_config.ssh_key)
+            ),
+            Env(
+                name='_CONFIG_USER', os_name='',
+                default=self.render_str(self._remote_config.user)
+            ),
+            Env(
+                name='_CONFIG_PASSWORD', os_name='',
+                default=self.render_str(self._remote_config.password)
+            ),
         )
-        return env_map
+        for key, val in self._remote_config.config_map.items():
+            upper_snake_key = to_snake_case(key).upper()
+            rendered_val = self.render_str(val)
+            # add remote config map as env
+            self.add_env(
+                Env(
+                    name='_CONFIG_MAP_' + upper_snake_key,
+                    os_name='',
+                    default=rendered_val
+                )
+            )
 
-    def _get_cmd_str(self, *args: Any, **kwargs: Any) -> str:
+    def get_cmd_script(self, *args: Any, **kwargs: Any) -> str:
         cmd_str = '\n'.join([
-            self._create_cmd_str(
+            self._create_cmd_script(
                 self._pre_cmd_path, self._pre_cmd, *args, **kwargs
             ),
-            super()._get_cmd_str(*args, **kwargs),
-            self._create_cmd_str(
+            super().get_cmd_script(*args, **kwargs),
+            self._create_cmd_script(
                 self._post_cmd_path, self._post_cmd, *args, **kwargs
             ),
         ])