blue-assistant 4.243.1__tar.gz → 4.273.1__tar.gz

{blue_assistant-4.243.1/blue_assistant.egg-info → blue_assistant-4.273.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.2
 Name: blue_assistant
-Version: 4.243.1
+Version: 4.273.1
 Summary: 🧠 An AI Assistant.
 Home-page: https://github.com/kamangir/blue-assistant
 Author: Arash Abadpour (Kamangir)
@@ -112,8 +112,8 @@ graph LR
 
 |   |   |
 | --- | --- |
-| [`hue`](https://raw.githubusercontent.com/kamangir/blue-assistant/main/blue_assistant/script/repository/hue) [![image](https://github.com/kamangir/assets/raw/main/blue-assistant/20250314_143702.jpg?raw=true)](https://raw.githubusercontent.com/kamangir/blue-assistant/main/blue_assistant/script/repository/hue) "send a color command to the Hue LED lights in my apartment." | [`blue-amo`](https://raw.githubusercontent.com/kamangir/blue-assistant/main/blue_assistant/script/repository/blue_amo/README.md) [![image](https://github.com/kamangir/assets/raw/main/blue-amo-2025-02-03-nswnx6/stitching_the_frames-2.png?raw=true)](https://raw.githubusercontent.com/kamangir/blue-assistant/main/blue_assistant/script/repository/blue_amo/README.md) A story developed and visualized, by AI. |
-| [`orbital-data-explorer`](https://raw.githubusercontent.com/kamangir/blue-assistant/main/blue_assistant/script/repository/orbital_data_explorer) [![image](https://github.com/kamangir/assets/blob/main/blue-assistant/orbital-data-explorer.png?raw=true)](https://raw.githubusercontent.com/kamangir/blue-assistant/main/blue_assistant/script/repository/orbital_data_explorer) Access to the [Orbital Data Explorer](https://ode.rsl.wustl.edu/), through AI. ⏸️ |  |
+| [`orbital-data-explorer`](https://raw.githubusercontent.com/kamangir/blue-assistant/main/blue_assistant/script/repository/orbital_data_explorer) [![image](https://github.com/kamangir/assets/blob/main/blue-assistant/orbital-data-explorer.png?raw=true)](https://raw.githubusercontent.com/kamangir/blue-assistant/main/blue_assistant/script/repository/orbital_data_explorer) Access to the [Orbital Data Explorer](https://ode.rsl.wustl.edu/). 🔥 | [`🌀 blue script`](https://raw.githubusercontent.com/kamangir/blue-assistant/main/blue_assistant/script/) [![image](https://github.com/kamangir/assets/raw/main/blue-plugin/marquee.png?raw=true)](https://raw.githubusercontent.com/kamangir/blue-assistant/main/blue_assistant/script/) A minimal AI DAG interface. |
+| [`@hue`](https://raw.githubusercontent.com/kamangir/blue-assistant/main/blue_assistant/script/repository/hue) [![image](https://github.com/kamangir/assets/raw/main/blue-assistant/20250314_143702.jpg?raw=true)](https://raw.githubusercontent.com/kamangir/blue-assistant/main/blue_assistant/script/repository/hue) "send a color command to the Hue LED lights in my apartment." | [`blue-amo`](https://raw.githubusercontent.com/kamangir/blue-assistant/main/blue_assistant/script/repository/blue_amo/README.md) [![image](https://github.com/kamangir/assets/raw/main/blue-amo-2025-02-03-nswnx6/stitching_the_frames-2.png?raw=true)](https://raw.githubusercontent.com/kamangir/blue-assistant/main/blue_assistant/script/repository/blue_amo/README.md) Story development and visualization. |
 
 ---
 
@@ -124,4 +124,4 @@ Also home to [`@web`](https://raw.githubusercontent.com/kamangir/blue-assistant/
 
 [![pylint](https://github.com/kamangir/blue-assistant/actions/workflows/pylint.yml/badge.svg)](https://github.com/kamangir/blue-assistant/actions/workflows/pylint.yml) [![pytest](https://github.com/kamangir/blue-assistant/actions/workflows/pytest.yml/badge.svg)](https://github.com/kamangir/blue-assistant/actions/workflows/pytest.yml) [![bashtest](https://github.com/kamangir/blue-assistant/actions/workflows/bashtest.yml/badge.svg)](https://github.com/kamangir/blue-assistant/actions/workflows/bashtest.yml) [![PyPI version](https://img.shields.io/pypi/v/blue-assistant.svg)](https://pypi.org/project/blue-assistant/) [![PyPI - Downloads](https://img.shields.io/pypi/dd/blue-assistant)](https://pypistats.org/packages/blue-assistant)
 
-built by 🌀 [`blue_options-4.236.1`](https://github.com/kamangir/awesome-bash-cli), based on 🧠 [`blue_assistant-4.243.1`](https://github.com/kamangir/blue-assistant).
+built by 🌀 [`blue_options-4.236.1`](https://github.com/kamangir/awesome-bash-cli), based on 🧠 [`blue_assistant-4.273.1`](https://github.com/kamangir/blue-assistant).

{blue_assistant-4.243.1 → blue_assistant-4.273.1}/README.md

@@ -67,8 +67,8 @@ graph LR
 
 |   |   |
 | --- | --- |
-| [`hue`](./blue_assistant/script/repository/hue) [![image](https://github.com/kamangir/assets/raw/main/blue-assistant/20250314_143702.jpg?raw=true)](./blue_assistant/script/repository/hue) "send a color command to the Hue LED lights in my apartment." | [`blue-amo`](./blue_assistant/script/repository/blue_amo/README.md) [![image](https://github.com/kamangir/assets/raw/main/blue-amo-2025-02-03-nswnx6/stitching_the_frames-2.png?raw=true)](./blue_assistant/script/repository/blue_amo/README.md) A story developed and visualized, by AI. |
-| [`orbital-data-explorer`](./blue_assistant/script/repository/orbital_data_explorer) [![image](https://github.com/kamangir/assets/blob/main/blue-assistant/orbital-data-explorer.png?raw=true)](./blue_assistant/script/repository/orbital_data_explorer) Access to the [Orbital Data Explorer](https://ode.rsl.wustl.edu/), through AI. ⏸️ |  |
+| [`orbital-data-explorer`](./blue_assistant/script/repository/orbital_data_explorer) [![image](https://github.com/kamangir/assets/blob/main/blue-assistant/orbital-data-explorer.png?raw=true)](./blue_assistant/script/repository/orbital_data_explorer) Access to the [Orbital Data Explorer](https://ode.rsl.wustl.edu/). 🔥 | [`🌀 blue script`](./blue_assistant/script/) [![image](https://github.com/kamangir/assets/raw/main/blue-plugin/marquee.png?raw=true)](./blue_assistant/script/) A minimal AI DAG interface. |
+| [`@hue`](./blue_assistant/script/repository/hue) [![image](https://github.com/kamangir/assets/raw/main/blue-assistant/20250314_143702.jpg?raw=true)](./blue_assistant/script/repository/hue) "send a color command to the Hue LED lights in my apartment." | [`blue-amo`](./blue_assistant/script/repository/blue_amo/README.md) [![image](https://github.com/kamangir/assets/raw/main/blue-amo-2025-02-03-nswnx6/stitching_the_frames-2.png?raw=true)](./blue_assistant/script/repository/blue_amo/README.md) Story development and visualization. |
 
 ---
 
@@ -79,4 +79,4 @@ Also home to [`@web`](./blue_assistant/web/)
 
 [![pylint](https://github.com/kamangir/blue-assistant/actions/workflows/pylint.yml/badge.svg)](https://github.com/kamangir/blue-assistant/actions/workflows/pylint.yml) [![pytest](https://github.com/kamangir/blue-assistant/actions/workflows/pytest.yml/badge.svg)](https://github.com/kamangir/blue-assistant/actions/workflows/pytest.yml) [![bashtest](https://github.com/kamangir/blue-assistant/actions/workflows/bashtest.yml/badge.svg)](https://github.com/kamangir/blue-assistant/actions/workflows/bashtest.yml) [![PyPI version](https://img.shields.io/pypi/v/blue-assistant.svg)](https://pypi.org/project/blue-assistant/) [![PyPI - Downloads](https://img.shields.io/pypi/dd/blue-assistant)](https://pypistats.org/packages/blue-assistant)
 
-built by 🌀 [`blue_options-4.236.1`](https://github.com/kamangir/awesome-bash-cli), based on 🧠 [`blue_assistant-4.243.1`](https://github.com/kamangir/blue-assistant).
+built by 🌀 [`blue_options-4.236.1`](https://github.com/kamangir/awesome-bash-cli), based on 🧠 [`blue_assistant-4.273.1`](https://github.com/kamangir/blue-assistant).

{blue_assistant-4.243.1 → blue_assistant-4.273.1}/blue_assistant/.abcli/script/run.sh

@@ -3,7 +3,8 @@
 function blue_assistant_script_run() {
     local options=$1
     local do_dryrun=$(abcli_option_int "$options" dryrun 0)
-    local do_download=$(abcli_option_int "$options" download 0)
+    local use_cache=$(abcli_option_int "$options" cache 1)
+    local do_download=$(abcli_option_int "$options" download $use_cache)
     local do_upload=$(abcli_option_int "$options" upload $(abcli_not $do_dryrun))
 
     local script_options=$2
@@ -20,6 +21,7 @@ function blue_assistant_script_run() {
         run \
         --script_name $script_name \
         --object_name $object_name \
+        --use_cache $use_cache \
         "${@:4}"
     [[ $? -ne 0 ]] && return 1
 

{blue_assistant-4.243.1 → blue_assistant-4.273.1}/blue_assistant/.abcli/tests/script_run.sh

@@ -11,11 +11,13 @@ function test_blue_assistant_script_run() {
     for script_name in $(echo "$list_of_script_name" | tr + " "); do
         abcli_log "testing $script_name ..."
 
+        local object_name=test_blue_assistant_script_run-$(abcli_string_timestamp_short)
+
         abcli_eval ,$options \
             blue_assistant_script_run \
             ~upload,$options \
             script=$script_name \
-            test_blue_assistant_script_run-$(abcli_string_timestamp_short) \
+            $object_name \
             "${@:2}" \
             --test_mode 1 \
             --verbose 1

{blue_assistant-4.243.1 → blue_assistant-4.273.1}/blue_assistant/.abcli/web/crawl.sh

@@ -3,11 +3,15 @@
 function blue_assistant_web_crawl() {
     local options=$1
     local do_dryrun=$(abcli_option_int "$options" dryrun 0)
+    local use_cache=$(abcli_option_int "$options" cache 0)
+    local do_download=$(abcli_option_int "$options" download $use_cache)
     local do_upload=$(abcli_option_int "$options" upload $(abcli_not $do_dryrun))
 
     local seed_urls=${2:-void}
 
     local object_name=$(abcli_clarify_object $3 web-crawl-$(abcli_string_timestamp_short))
+    [[ "$do_download" == 1 ]] &&
+        abcli_download - $object_name
 
     abcli_log "crawling $seed_urls -> $object_name ..."
 
@@ -16,6 +20,7 @@ function blue_assistant_web_crawl() {
         crawl \
         --seed_urls $seed_urls \
         --object_name $object_name \
+        --use_cache $use_cache \
         "${@:4}"
     [[ $? -ne 0 ]] && return 1
 

{blue_assistant-4.243.1 → blue_assistant-4.273.1}/blue_assistant/README.py

@@ -1,14 +1,28 @@
 import os
 
+from blue_options.help.functions import get_help
 from blue_objects import file, README
 
 from blue_assistant import NAME, VERSION, ICON, REPO_NAME
+from blue_assistant.help.functions import help_functions
 
 
 items = README.Items(
     [
         {
-            "name": "hue",
+            "name": "orbital-data-explorer",
+            "url": "./blue_assistant/script/repository/orbital_data_explorer",
+            "marquee": "https://github.com/kamangir/assets/blob/main/blue-assistant/orbital-data-explorer.png?raw=true",
+            "description": "Access to the [Orbital Data Explorer](https://ode.rsl.wustl.edu/). 🔥",
+        },
+        {
+            "name": "🌀 blue script",
+            "marquee": "https://github.com/kamangir/assets/raw/main/blue-plugin/marquee.png?raw=true",
+            "description": "A minimal AI DAG interface.",
+            "url": "./blue_assistant/script/",
+        },
+        {
+            "name": "@hue",
             "url": "./blue_assistant/script/repository/hue",
             "marquee": "https://github.com/kamangir/assets/raw/main/blue-assistant/20250314_143702.jpg?raw=true",
             "description": '"send a color command to the Hue LED lights in my apartment."',
@@ -17,13 +31,7 @@ items = README.Items(
             "name": "blue-amo",
             "url": "./blue_assistant/script/repository/blue_amo/README.md",
             "marquee": "https://github.com/kamangir/assets/raw/main/blue-amo-2025-02-03-nswnx6/stitching_the_frames-2.png?raw=true",
-            "description": "A story developed and visualized, by AI.",
-        },
-        {
-            "name": "orbital-data-explorer",
-            "url": "./blue_assistant/script/repository/orbital_data_explorer",
-            "marquee": "https://github.com/kamangir/assets/blob/main/blue-assistant/orbital-data-explorer.png?raw=true",
-            "description": "Access to the [Orbital Data Explorer](https://ode.rsl.wustl.edu/), through AI. ⏸️",
+            "description": "Story development and visualization.",
         },
     ]
 )
@@ -39,6 +47,11 @@ def build():
             NAME=NAME,
             VERSION=VERSION,
             REPO_NAME=REPO_NAME,
+            help_function=lambda tokens: get_help(
+                tokens,
+                help_functions,
+                mono=True,
+            ),
         )
         for readme in [
             {
@@ -54,6 +67,7 @@ def build():
             {"path": "script/repository/hue/docs/round-1.md"},
             {"path": "script/repository/hue/docs"},
             #
+            {"path": "script/"},
             {"path": "web/"},
         ]
     )

{blue_assistant-4.243.1 → blue_assistant-4.273.1}/blue_assistant/__init__.py

@@ -4,7 +4,7 @@ ICON = "🧠"
 
 DESCRIPTION = f"{ICON} An AI Assistant."
 
-VERSION = "4.243.1"
+VERSION = "4.273.1"
 
 REPO_NAME = "blue-assistant"
 

{blue_assistant-4.243.1 → blue_assistant-4.273.1}/blue_assistant/help/script.py

@@ -30,7 +30,7 @@ def help_run(
     tokens: List[str],
     mono: bool,
 ) -> str:
-    options = xtra("download,dryrun,~upload", mono=mono)
+    options = xtra("~cache,download,dryrun,~upload", mono=mono)
 
     script_options = "script=<script>"
 

{blue_assistant-4.243.1 → blue_assistant-4.273.1}/blue_assistant/help/web.py

@@ -7,7 +7,7 @@ def help_crawl(
     tokens: List[str],
     mono: bool,
 ) -> str:
-    options = xtra("dryrun,~upload", mono=mono)
+    options = xtra("cache,~download,dryrun,~upload", mono=mono)
 
     args = [
         "[--max_iterations <100000>]",

{blue_assistant-4.243.1 → blue_assistant-4.273.1}/blue_assistant/script/__main__.py

@@ -48,6 +48,12 @@ parser.add_argument(
     default=1,
     help="0 | 1",
 )
+parser.add_argument(
+    "--use_cache",
+    type=int,
+    default=1,
+    help="0 | 1",
+)
 args = parser.parse_args()
 
 delim = " " if args.delim == "space" else args.delim
@@ -70,7 +76,9 @@ elif args.task == "run":
     )
 
     if success:
-        success = script.run()
+        success = script.run(
+            use_cache=args.use_cache == 1,
+        )
 else:
     success = None
 

{blue_assistant-4.243.1 → blue_assistant-4.273.1}/blue_assistant/script/actions/__init__.py

@@ -1,14 +1,15 @@
-from typing import Dict, Callable, Tuple
+from typing import Dict, Callable
 
 from blue_assistant.script.repository.base.classes import BaseScript
 from blue_assistant.script.actions.generic import generic_action
 from blue_assistant.script.actions.generate_image import generate_image
 from blue_assistant.script.actions.generate_text import generate_text
-from blue_assistant.logger import logger
+from blue_assistant.script.actions.web_crawl import web_crawl
 
 
 dict_of_actions: Dict[str, Callable[[BaseScript, str], bool]] = {
     "generic": generic_action,
     "generate_image": generate_image,
     "generate_text": generate_text,
+    "web_crawl": web_crawl,
 }

{blue_assistant-4.243.1 → blue_assistant-4.273.1}/blue_assistant/script/actions/generate_image.py

@@ -17,6 +17,7 @@ NAME = module.name(__file__, NAME)
 def generate_image(
     script: BaseScript,
     node_name: str,
+    use_cache: bool,
 ) -> bool:
     logger.info(f"{NAME}: {script} @ {node_name} ...")
 

{blue_assistant-4.243.1 → blue_assistant-4.273.1}/blue_assistant/script/actions/generate_text.py

@@ -18,6 +18,7 @@ NAME = module.name(__file__, NAME)
 def generate_text(
     script: BaseScript,
     node_name: str,
+    use_cache: bool,
 ) -> bool:
     logger.info(f"{NAME}: {script} @ {node_name} ...")
 

{blue_assistant-4.243.1 → blue_assistant-4.273.1}/blue_assistant/script/actions/generic.py

@@ -10,6 +10,7 @@ NAME = module.name(__file__, NAME)
 def generic_action(
     script: BaseScript,
     node_name: str,
+    use_cache: bool,
 ) -> bool:
     logger.info(f"{NAME}: {script} @ {node_name} ...")
     return True

blue_assistant-4.273.1/blue_assistant/script/actions/web_crawl.py

@@ -0,0 +1,49 @@
+from blueness import module
+
+from blue_options.logger import log_list
+
+from blue_assistant import NAME
+from blue_assistant.web.crawl import crawl_list_of_urls
+from blue_assistant.script.repository.base.classes import BaseScript
+from blue_assistant.logger import logger
+
+
+NAME = module.name(__file__, NAME)
+
+
+def web_crawl(
+    script: BaseScript,
+    node_name: str,
+    use_cache: bool,
+) -> bool:
+    logger.info(f"{NAME}: {script} @ {node_name} ...")
+
+    seed_url_var_name = script.nodes[node_name].get("seed_urls", "")
+    if not isinstance(seed_url_var_name, str):
+        logger.error(f"{node_name}: seed_urls must be a string.")
+        return False
+    if not seed_url_var_name:
+        logger.error(f"{node_name}: seed_urls not found.")
+        return False
+
+    # to allow both :::<var-name> and <var-name> - for convenience :)
+    if seed_url_var_name.startswith(":::"):
+        seed_url_var_name = seed_url_var_name[3:].strip()
+
+    if seed_url_var_name not in script.vars:
+        logger.error(f"{node_name}: {seed_url_var_name}: seed_urls not found in vars.")
+        return False
+    seed_urls = script.vars[seed_url_var_name]
+    log_list(logger, seed_urls, "seed url(s)")
+
+    visited_urls = crawl_list_of_urls(
+        seed_urls=seed_urls,
+        object_name=script.object_name,
+        max_iterations=script.nodes[node_name]["max_iterations"],
+        use_cache=use_cache,
+    )
+
+    script.nodes[node_name]["visited_urls"] = visited_urls
+    script.nodes[node_name]["output"] = "TBA"
+
+    return True

{blue_assistant-4.243.1 → blue_assistant-4.273.1}/blue_assistant/script/repository/base/classes.py

@@ -27,8 +27,6 @@ class BaseScript:
         self.object_name = object_name
 
         self.test_mode = test_mode
-        if self.test_mode:
-            logger.info("💰 test mode is on.")
 
         self.verbose = verbose
 
@@ -41,14 +39,70 @@ class BaseScript:
         success, self.metadata = file.load_yaml(metadata_filename)
         assert success, f"cannot load {self.name}/metadata.yaml"
 
-        logger.info("loaded {} node(s)".format(len(self.nodes)))
+        self.metadata.setdefault("script", {})
+        assert isinstance(
+            self.script,
+            dict,
+        ), "script: expected dict, received {}.".format(
+            self.script.__class__.__name__,
+        )
+
+        self.script.setdefault("nodes", {})
+        assert isinstance(
+            self.nodes,
+            dict,
+        ), "nodes: expected dict, received {}.".format(
+            self.nodes.__class__.__name__,
+        )
+
+        self.script.setdefault("vars", {})
+        assert isinstance(
+            self.vars,
+            dict,
+        ), "vars: expected dict, received {}.".format(
+            self.vars.__class__.__name__,
+        )
+
+        if self.test_mode:
+            logger.info("🧪  test mode is on.")
+
+            for node_name, node in self.nodes.items():
+                if "test_mode" in self.script:
+                    updates = self.script["test_mode"]
+                    logger.info(f"🧪  vars.update({updates})")
+                    self.vars.update(updates)
+
+                if "test_mode" in node:
+                    updates = node["test_mode"]
+                    logger.info(f"🧪  {node_name}.update({updates})")
+                    node.update(updates)
 
-        logger.info("loaded {} variable(s)".format(len(self.vars)))
+        logger.info(
+            "loaded {} node(s): {}".format(
+                len(self.nodes),
+                ", ".join(self.nodes.keys()),
+            )
+        )
+
+        logger.info(
+            "loaded {} var(s){}".format(
+                len(self.vars),
+                "" if verbose else ": {}".format(", ".join(self.vars.keys())),
+            )
+        )
         if verbose:
             for var_name, var_value in self.vars.items():
                 logger.info("{}: {}".format(var_name, var_value))
 
-        assert self.generate_graph(), "cannot generate graph"
+        assert self.generate_graph(), "cannot generate graph."
+
+    def __str__(self) -> str:
+        return "{}[{} var(s), {} node(s) -> {}]".format(
+            self.__class__.__name__,
+            len(self.vars),
+            len(self.nodes),
+            self.object_name,
+        )
 
     def apply_vars(self, text: str) -> str:
         for var_name, var_value in self.vars.items():
@@ -91,12 +145,16 @@ class BaseScript:
             [node_name],
         )
 
-    def run(self) -> bool:
+    def run(
+        self,
+        use_cache: bool = True,
+    ) -> bool:
         logger.info(
-            "{}.run: {}:{} -> {}".format(
+            "{}.run: {}:{} -{}> {}".format(
                 NAME,
                 self.__class__.__name__,
                 self.name,
+                "use-cache-" if use_cache else "",
                 self.object_name,
             )
         )
@@ -126,12 +184,12 @@ class BaseScript:
     # Aliases
     @property
     def script(self) -> Dict:
-        return self.metadata.get("script", {})
+        return self.metadata["script"]
 
     @property
     def nodes(self) -> Dict[str, Dict]:
-        return self.metadata.get("script", {}).get("nodes", {})
+        return self.metadata["script"]["nodes"]
 
     @property
     def vars(self) -> Dict:
-        return self.metadata.get("script", {}).get("vars", {})
+        return self.metadata["script"]["vars"]

{blue_assistant-4.243.1 → blue_assistant-4.273.1}/blue_assistant/script/repository/blue_amo/actions/slicing_into_frames.py

@@ -10,6 +10,7 @@ NAME = module.name(__file__, NAME)
 def slicing_into_frames(
     script: BaseScript,
     node_name: str,
+    use_cache: bool,
 ) -> bool:
     logger.info(f"{NAME}: processing the output...")
 

{blue_assistant-4.243.1 → blue_assistant-4.273.1}/blue_assistant/script/repository/blue_amo/actions/stitching_the_frames.py

@@ -18,6 +18,7 @@ NAME = module.name(__file__, NAME)
 def stitching_the_frames(
     script: BaseScript,
     node_name: str,
+    use_cache: bool,
 ) -> bool:
     list_of_frames_filenames: List[str] = [
         filename

{blue_assistant-4.243.1 → blue_assistant-4.273.1}/blue_assistant/script/repository/blue_amo/classes.py

@@ -66,14 +66,19 @@ class BlueAmoScript(GenericScript):
     def perform_action(
         self,
         node_name: str,
+        use_cache: bool,
     ) -> bool:
-        if not super().perform_action(node_name=node_name):
+        if not super().perform_action(
+            node_name=node_name,
+            use_cache=use_cache,
+        ):
             return False
 
         if node_name in dict_of_actions:
             return dict_of_actions[node_name](
                 script=self,
                 node_name=node_name,
+                use_cache=use_cache,
             )
 
         return True

{blue_assistant-4.243.1 → blue_assistant-4.273.1}/blue_assistant/script/repository/generic/classes.py

@@ -22,6 +22,7 @@ class GenericScript(BaseScript):
     def perform_action(
         self,
         node_name: str,
+        use_cache: bool,
     ) -> bool:
         action_name = self.nodes[node_name].get("action", "unknown")
         logger.info(f"---- node: {node_name} ---- ")
@@ -33,10 +34,14 @@ class GenericScript(BaseScript):
         return dict_of_actions[action_name](
             script=self,
             node_name=node_name,
+            use_cache=use_cache,
         )
 
-    def run(self) -> bool:
-        if not super().run():
+    def run(
+        self,
+        use_cache: bool = True,
+    ) -> bool:
+        if not super().run(use_cache=use_cache):
             return False
 
         success: bool = True
@@ -68,7 +73,10 @@ class GenericScript(BaseScript):
                     )
                     continue
 
-                if not self.perform_action(node_name=node_name):
+                if not self.perform_action(
+                    node_name=node_name,
+                    use_cache=use_cache,
+                ):
                     success = False
                     break
 

blue_assistant-4.273.1/blue_assistant/script/repository/orbital_data_explorer/classes.py

@@ -0,0 +1,7 @@
+from blue_objects import file, path
+
+from blue_assistant.script.repository.generic.classes import GenericScript
+
+
+class OrbitalDataExplorerScript(GenericScript):
+    name = path.name(file.path(__file__))

blue_assistant-4.273.1/blue_assistant/web/__init__.py

@@ -0,0 +1,2 @@
+from blue_assistant.web.crawl import crawl_list_of_urls
+from blue_assistant.web.fetch import fetch_links_and_text

{blue_assistant-4.243.1 → blue_assistant-4.273.1}/blue_assistant/web/__main__.py

@@ -6,7 +6,7 @@ from blue_options.logger import log_dict
 from blue_objects.metadata import post_to_object
 
 from blue_assistant import NAME
-from blue_assistant.web.functions import crawl_list_of_urls, fetch_links_and_text
+from blue_assistant.web import crawl_list_of_urls, fetch_links_and_text
 from blue_assistant.logger import logger
 
 NAME = module.name(__file__, NAME)
@@ -40,6 +40,12 @@ parser.add_argument(
     "--object_name",
     type=str,
 )
+parser.add_argument(
+    "--use_cache",
+    type=int,
+    default=0,
+    help="0 | 1",
+)
 args = parser.parse_args()
 
 success = False
@@ -48,6 +54,7 @@ if args.task == "crawl":
         seed_urls=args.seed_urls.split("+"),
         object_name=args.object_name,
         max_iterations=args.max_iterations,
+        use_cache=args.use_cache == 1,
     )
 
     if args.verbose == 1:
@@ -59,7 +66,7 @@ if args.task == "crawl":
         dict_of_urls,
     )
 elif args.task == "fetch":
-    links, text = fetch_links_and_text(
+    summary = fetch_links_and_text(
         url=args.url,
         verbose=True,
     )
@@ -67,10 +74,7 @@ elif args.task == "fetch":
     success = post_to_object(
         args.object_name,
         NAME.replace(".", "-"),
-        {
-            "links": list(links),
-            "text": text,
-        },
+        summary,
     )
 else:
     success = None