blue-sandbox 5.47.1__tar.gz → 5.87.1__tar.gz

{blue_sandbox-5.47.1/blue_sandbox.egg-info → blue_sandbox-5.87.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.2
 Name: blue_sandbox
-Version: 5.47.1
+Version: 5.87.1
 Summary: 🌀 A sandbox for ideas and experiments.
 Home-page: https://github.com/kamangir/blue-sandbox
 Author: Arash Abadpour (Kamangir)
@@ -13,8 +13,12 @@ Classifier: Operating System :: OS Independent
 Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: blueness
-Requires-Dist: blue-options
 Requires-Dist: abcli
+Requires-Dist: blue-options
+Requires-Dist: blue_objects
+Requires-Dist: blue_geo
+Requires-Dist: blueflow
+Requires-Dist: blue_plugin
 Requires-Dist: boto3
 Requires-Dist: geojson
 Requires-Dist: geopandas
@@ -27,6 +31,9 @@ Requires-Dist: pylint
 Requires-Dist: pytest
 Requires-Dist: python-dotenv[cli]
 Requires-Dist: tqdm
+Requires-Dist: azure-storage-blob
+Requires-Dist: fiona
+Requires-Dist: tensorboard
 Dynamic: author
 Dynamic: author-email
 Dynamic: classifier
@@ -47,11 +54,11 @@ pip install blue-sandbox
 
 |   |   |   |
 | --- | --- | --- |
-| 🌐[``@damage``](https://github.com/kamangir/blue-sandbox/blob/main/blue_sandbox/microsoft_building_damage_assessment/README.md) [![image](https://github.com/microsoft/building-damage-assessment/raw/main/figures/damage.png)](https://github.com/kamangir/blue-sandbox/blob/main/blue_sandbox/microsoft_building_damage_assessment/README.md) Satellite imagery damage assessment workflow | 🌀[`experiment 2`](#) [![image](https://github.com/kamangir/assets/raw/main/blue-plugin/marquee.png?raw=true)](#) description of experiment 2 ... | 🌀[`experiment 3`](#) [![image](https://github.com/kamangir/assets/raw/main/blue-plugin/marquee.png?raw=true)](#) description of experiment 3 ... |
+| 🌐[``@damages``](https://github.com/kamangir/blue-sandbox/blob/main/blue_sandbox/microsoft_building_damage_assessment/README.md) [![image](https://github.com/microsoft/building-damage-assessment/raw/main/figures/damage.png)](https://github.com/kamangir/blue-sandbox/blob/main/blue_sandbox/microsoft_building_damage_assessment/README.md) Satellite imagery damage assessment workflow | 🌀[`experiment 2`](#) [![image](https://github.com/kamangir/assets/raw/main/blue-plugin/marquee.png?raw=true)](#) description of experiment 2 ... | 🌀[`experiment 3`](#) [![image](https://github.com/kamangir/assets/raw/main/blue-plugin/marquee.png?raw=true)](#) description of experiment 3 ... |
 
 ---
 
 
 [![pylint](https://github.com/kamangir/blue-sandbox/actions/workflows/pylint.yml/badge.svg)](https://github.com/kamangir/blue-sandbox/actions/workflows/pylint.yml) [![pytest](https://github.com/kamangir/blue-sandbox/actions/workflows/pytest.yml/badge.svg)](https://github.com/kamangir/blue-sandbox/actions/workflows/pytest.yml) [![bashtest](https://github.com/kamangir/blue-sandbox/actions/workflows/bashtest.yml/badge.svg)](https://github.com/kamangir/blue-sandbox/actions/workflows/bashtest.yml) [![PyPI version](https://img.shields.io/pypi/v/blue-sandbox.svg)](https://pypi.org/project/blue-sandbox/) [![PyPI - Downloads](https://img.shields.io/pypi/dd/blue-sandbox)](https://pypistats.org/packages/blue-sandbox)
 
-built by 🌀 [`blue_options-4.177.1`](https://github.com/kamangir/awesome-bash-cli), based on 🌀 [`blue_sandbox-5.47.1`](https://github.com/kamangir/blue-sandbox).
+built by 🌀 [`blue_options-4.180.1`](https://github.com/kamangir/awesome-bash-cli), based on 🌀 [`blue_sandbox-5.87.1`](https://github.com/kamangir/blue-sandbox).

{blue_sandbox-5.47.1 → blue_sandbox-5.87.1}/README.md

@@ -8,11 +8,11 @@ pip install blue-sandbox
 
 |   |   |   |
 | --- | --- | --- |
-| 🌐[``@damage``](https://github.com/kamangir/blue-sandbox/blob/main/blue_sandbox/microsoft_building_damage_assessment/README.md) [![image](https://github.com/microsoft/building-damage-assessment/raw/main/figures/damage.png)](https://github.com/kamangir/blue-sandbox/blob/main/blue_sandbox/microsoft_building_damage_assessment/README.md) Satellite imagery damage assessment workflow | 🌀[`experiment 2`](#) [![image](https://github.com/kamangir/assets/raw/main/blue-plugin/marquee.png?raw=true)](#) description of experiment 2 ... | 🌀[`experiment 3`](#) [![image](https://github.com/kamangir/assets/raw/main/blue-plugin/marquee.png?raw=true)](#) description of experiment 3 ... |
+| 🌐[``@damages``](https://github.com/kamangir/blue-sandbox/blob/main/blue_sandbox/microsoft_building_damage_assessment/README.md) [![image](https://github.com/microsoft/building-damage-assessment/raw/main/figures/damage.png)](https://github.com/kamangir/blue-sandbox/blob/main/blue_sandbox/microsoft_building_damage_assessment/README.md) Satellite imagery damage assessment workflow | 🌀[`experiment 2`](#) [![image](https://github.com/kamangir/assets/raw/main/blue-plugin/marquee.png?raw=true)](#) description of experiment 2 ... | 🌀[`experiment 3`](#) [![image](https://github.com/kamangir/assets/raw/main/blue-plugin/marquee.png?raw=true)](#) description of experiment 3 ... |
 
 ---
 
 
 [![pylint](https://github.com/kamangir/blue-sandbox/actions/workflows/pylint.yml/badge.svg)](https://github.com/kamangir/blue-sandbox/actions/workflows/pylint.yml) [![pytest](https://github.com/kamangir/blue-sandbox/actions/workflows/pytest.yml/badge.svg)](https://github.com/kamangir/blue-sandbox/actions/workflows/pytest.yml) [![bashtest](https://github.com/kamangir/blue-sandbox/actions/workflows/bashtest.yml/badge.svg)](https://github.com/kamangir/blue-sandbox/actions/workflows/bashtest.yml) [![PyPI version](https://img.shields.io/pypi/v/blue-sandbox.svg)](https://pypi.org/project/blue-sandbox/) [![PyPI - Downloads](https://img.shields.io/pypi/dd/blue-sandbox)](https://pypistats.org/packages/blue-sandbox)
 
-built by 🌀 [`blue_options-4.177.1`](https://github.com/kamangir/awesome-bash-cli), based on 🌀 [`blue_sandbox-5.47.1`](https://github.com/kamangir/blue-sandbox).
+built by 🌀 [`blue_options-4.180.1`](https://github.com/kamangir/awesome-bash-cli), based on 🌀 [`blue_sandbox-5.87.1`](https://github.com/kamangir/blue-sandbox).

blue_sandbox-5.87.1/blue_sandbox/.abcli/alias.sh

@@ -0,0 +1,5 @@
+#! /usr/bin/env bash
+
+alias @damages="blue_sandbox_microsoft_building_damage_assessment"
+
+alias @sandbox=blue_sandbox

blue_sandbox-5.87.1/blue_sandbox/.abcli/install.sh

@@ -0,0 +1,7 @@
+#! /usr/bin/env bash
+
+function abcli_install_blue_sandbox() {
+    abcli_log "✅"
+}
+
+abcli_install_module blue_sandbox 1.1.1

{blue_sandbox-5.47.1 → blue_sandbox-5.87.1}/blue_sandbox/.abcli/microsoft_building_damage_assessment/ingest.sh

@@ -26,8 +26,8 @@ function blue_sandbox_microsoft_building_damage_assessment_ingest() {
     pushd $object_path >/dev/null
 
     # https://github.com/microsoft/building-damage-assessment/blob/main/SATELLITE_WORKFLOW.md#1-image-acquisition-and-label-generation
-    mkdir -pv data/demo/raw/
-    cd data/demo/raw/
+    mkdir -pv raw
+    cd raw
 
     if [[ "$do_download" == 1 ]]; then
         # TODO: use $event_name and $count (options +=)
@@ -57,8 +57,6 @@ function blue_sandbox_microsoft_building_damage_assessment_ingest() {
         rm -v maxar_lahaina_8_12_2023-visual.vrt
     fi
 
-    cd ../../..
-
     popd >/dev/null
 
     abcli_eval dryrun=$do_dryrun \

blue_sandbox-5.87.1/blue_sandbox/.abcli/microsoft_building_damage_assessment/install.sh

@@ -0,0 +1,49 @@
+#! /usr/bin/env bash
+
+function blue_sandbox_microsoft_building_damage_assessment_install() {
+    local options=$1
+    local do_dryrun=$(abcli_option_int "$options" dryrun 0)
+    local recreate_env=$(abcli_option_int "$options" recreate_env 0)
+
+    if [[ -d "$abcli_path_git/building-damage-assessment" ]]; then
+        abcli_log "✅  building-damage-assessment"
+    else
+        abcli_git_clone https://github.com/microsoft/building-damage-assessment.git
+        [[ $? -ne 0 ]] && return 1
+    fi
+
+    [[ "$abcli_is_github_workflow" == true ]] &&
+        return 0
+
+    if [[ "$recreate_env" == 1 ]]; then
+        abcli_conda_rm name=bda
+        [[ $? -ne 0 ]] && return 1
+    fi
+
+    local exists=$(abcli_conda_exists name=bda)
+    if [[ "$exists" == 0 ]]; then
+        # https://github.com/microsoft/building-damage-assessment?tab=readme-ov-file#setup
+        conda env create \
+            -f $abcli_path_git/building-damage-assessment/environment.yml
+        [[ $? -ne 0 ]] && return 1
+    fi
+
+    conda activate bda
+    [[ $? -ne 0 ]] && return 1
+
+    pushd $abcli_path_git/blue-sandbox >/dev/null
+    pip3 install -e .
+    [[ $? -ne 0 ]] && return 1
+    popd >/dev/null
+
+    [[ "$abcli_is_mac" == true ]] &&
+        brew install wget
+
+    return 0
+}
+
+function abcli_install_microsoft_building_damage_assessment() {
+    blue_sandbox_microsoft_building_damage_assessment_install "$@"
+}
+
+abcli_install_module microsoft_building_damage_assessment 1.1.1

blue_sandbox-5.87.1/blue_sandbox/.abcli/microsoft_building_damage_assessment/label.sh

@@ -0,0 +1,26 @@
+#! /usr/bin/env bash
+
+function blue_sandbox_microsoft_building_damage_assessment_label() {
+    local options=$1
+    local do_dryrun=$(abcli_option_int "$options" dryrun 0)
+    local do_download=$(abcli_option_int "$options" download $(abcli_not $do_dryrun))
+    local do_upload=$(abcli_option_int "$options" upload $(abcli_not $do_dryrun))
+
+    local object_name=$(abcli_clarify_object $2 .)
+    [[ "$do_download" == 1 ]] &&
+        abcli_download - $object_name
+
+    abcli_log "labeling $object_name ..."
+
+    abcli_eval dryrun=$do_dryrun \
+        python3 -m blue_sandbox.microsoft_building_damage_assessment \
+        label \
+        --object_name $object_name \
+        "${@:3}"
+    local status="$?"
+
+    [[ "$do_upload" == 1 ]] &&
+        abcli_upload - $object_name
+
+    return $status
+}

blue_sandbox-5.87.1/blue_sandbox/.abcli/microsoft_building_damage_assessment/tensorboard.sh

@@ -0,0 +1,21 @@
+#! /usr/bin/env bash
+
+function blue_sandbox_microsoft_building_damage_assessment_tensorboard() {
+    local options=$1
+    local do_dryrun=$(abcli_option_int "$options" dryrun 0)
+    local do_browse=$(abcli_option_int "$options" browse $(abcli_not $do_dryrun))
+    local do_download=$(abcli_option_int "$options" download $(abcli_not $do_dryrun))
+    local port=$(abcli_option "$options" port 8889)
+
+    local model_object_name=$(abcli_clarify_object $2 .)
+    [[ "$do_download" == 1 ]] &&
+        abcli_download - $model_object_name
+
+    [[ "$do_browse" == 1 ]] &&
+        abcli_browse http://localhost:$port/
+
+    abcli_eval ,$options \
+        tensorboard \
+        --logdir $ABCLI_OBJECT_ROOT/$model_object_name/logs/ \
+        --port $port
+}

blue_sandbox-5.87.1/blue_sandbox/.abcli/microsoft_building_damage_assessment/train.sh

@@ -0,0 +1,53 @@
+#! /usr/bin/env bash
+
+function blue_sandbox_microsoft_building_damage_assessment_train() {
+    local options=$1
+    local do_dryrun=$(abcli_option_int "$options" dryrun 0)
+    local do_download=$(abcli_option_int "$options" download $(abcli_not $do_dryrun))
+    local do_upload=$(abcli_option_int "$options" upload $(abcli_not $do_dryrun))
+
+    local dataset_object_name=$(abcli_clarify_object $2 .)
+    [[ "$do_download" == 1 ]] &&
+        abcli_download - $dataset_object_name
+
+    local model_object_name=$(abcli_clarify_object $3 $dataset_object_name-model-$(abcli_string_timestamp_short))
+
+    abcli_log "training $dataset_object_name -> $model_object_name ..."
+
+    conda activate bda
+    [[ $? -ne 0 ]] && return 1
+
+    abcli_eval dryrun=$do_dryrun \
+        python3 -m blue_sandbox.microsoft_building_damage_assessment \
+        train \
+        --dataset_object_name $dataset_object_name \
+        --model_object_name $model_object_name \
+        "${@:4}"
+    [[ $? -ne 0 ]] && return 1
+
+    abcli_eval dryrun=$do_dryrun,path=$abcli_path_git/building-damage-assessment \
+        python3 project_setup.py \
+        --skip_health_check \
+        --config $ABCLI_OBJECT_ROOT/$model_object_name/config.yml
+    [[ $? -ne 0 ]] && return 1
+
+    abcli_eval dryrun=$do_dryrun,path=$abcli_path_git/building-damage-assessment \
+        python3 create_masks.py \
+        --config $ABCLI_OBJECT_ROOT/$model_object_name/config.yml
+    [[ $? -ne 0 ]] && return 1
+
+    abcli_eval dryrun=$do_dryrun,path=$abcli_path_git/building-damage-assessment \
+        python3 fine_tune.py \
+        --config $ABCLI_OBJECT_ROOT/$model_object_name/config.yml
+    [[ $? -ne 0 ]] && return 1
+
+    abcli_eval dryrun=$do_dryrun,path=$abcli_path_git/building-damage-assessment \
+        python3 inference.py \
+        --config $ABCLI_OBJECT_ROOT/$model_object_name/config.yml
+    [[ $? -ne 0 ]] && return 1
+
+    [[ "$do_upload" == 1 ]] &&
+        abcli_upload - $model_object_name
+
+    return 0
+}

{blue_sandbox-5.47.1 → blue_sandbox-5.87.1}/blue_sandbox/.abcli/tests/help.sh

@@ -7,9 +7,13 @@ function test_blue_sandbox_help() {
     for module in \
         "@sandbox" \
         \
-        "@damage" \
-        "@damage ingest" \
-        "@damage ingest list" \
+        "@damages" \
+        "@damages ingest" \
+        "@damages ingest list" \
+        "@damages install" \
+        "@damages label" \
+        "@damages tensorboard" \
+        "@damages train" \
         \
         "blue_sandbox"; do
         abcli_eval ,$options \

blue_sandbox-5.87.1/blue_sandbox/.abcli/tests/microsoft_building_damage_assessment_label.sh

@@ -0,0 +1,12 @@
+#! /usr/bin/env bash
+
+function test_blue_sandbox_microsoft_building_damage_assessment_label() {
+    local options=$1
+
+    abcli_eval ,$options \
+        blue_sandbox_microsoft_building_damage_assessment \
+        label \
+        ~upload,$options \
+        $DAMAGES_TEST_DATASET_OBJECT_NAME \
+        "${@:2}"
+}

blue_sandbox-5.87.1/blue_sandbox/.abcli/tests/microsoft_building_damage_assessment_train.sh

@@ -0,0 +1,20 @@
+#! /usr/bin/env bash
+
+function test_blue_sandbox_microsoft_building_damage_assessment_train() {
+    local options=$1
+
+    abcli_log_warning "disabled."
+    return 0
+
+    abcli_eval ,$options \
+        blue_sandbox_microsoft_building_damage_assessment \
+        train \
+        ~upload,$options \
+        $DAMAGES_TEST_DATASET_OBJECT_NAME \
+        test_blue_sandbox_microsoft_building_damage_assessment_train-$(abcli_string_timestamp_short) \
+        --verbose 1 \
+        "${@:2}"
+
+    # ignore error - TODO: disable
+    return 0
+}

{blue_sandbox-5.47.1 → blue_sandbox-5.87.1}/blue_sandbox/README.py

@@ -11,7 +11,7 @@ from blue_sandbox.microsoft_building_damage_assessment import (
 items = [
     "{}[`{}`]({}) [![image]({})]({}) {}".format(
         "🌐",
-        "`@damage`",
+        "`@damages`",
         "https://github.com/kamangir/blue-sandbox/blob/main/blue_sandbox/microsoft_building_damage_assessment/README.md",
         "https://github.com/microsoft/building-damage-assessment/raw/main/figures/damage.png",
         "https://github.com/kamangir/blue-sandbox/blob/main/blue_sandbox/microsoft_building_damage_assessment/README.md",

{blue_sandbox-5.47.1 → blue_sandbox-5.87.1}/blue_sandbox/__init__.py

@@ -4,7 +4,7 @@ ICON = "🌀"
 
 DESCRIPTION = f"{ICON} A sandbox for ideas and experiments."
 
-VERSION = "5.47.1"
+VERSION = "5.87.1"
 
 REPO_NAME = "blue-sandbox"
 

blue_sandbox-5.87.1/blue_sandbox/config.env

@@ -0,0 +1 @@
+DAMAGES_TEST_DATASET_OBJECT_NAME=Maui-Hawaii-fires-Aug-23-ingest-2025-01-10-qqJqhm

blue_sandbox-5.87.1/blue_sandbox/env.py

@@ -0,0 +1,13 @@
+import os
+from blue_options.env import load_config, load_env
+
+load_env(__name__)
+load_config(__name__)
+
+
+DAMAGES_TEST_DATASET_OBJECT_NAME = os.getenv(
+    "DAMAGES_TEST_DATASET_OBJECT_NAME",
+    "",
+)
+
+ENCODED_BLOB_SAS_TOKEN = os.getenv("ENCODED_BLOB_SAS_TOKEN", "")

blue_sandbox-5.87.1/blue_sandbox/microsoft_building_damage_assessment/README.py

@@ -0,0 +1,45 @@
+from typing import Dict
+
+from blue_sandbox.env import DAMAGES_TEST_DATASET_OBJECT_NAME
+
+list_of_steps: Dict[str, Dict] = {
+    "ingest": {
+        "object_name": DAMAGES_TEST_DATASET_OBJECT_NAME,
+        "image_name": "Maui-Hawaii-fires-Aug-23-damage-2025-01-09-GgnjQC",
+    },
+    "label": {
+        "object_name": "",
+        "image_name": DAMAGES_TEST_DATASET_OBJECT_NAME,
+    },
+    "train": {
+        "object_name": "",
+        "image_name": "Maui-Hawaii-fires-Aug-23-model-2025-01-10-NQb8IS",
+    },
+    "predict": {"object_name": "", "image_name": ""},
+    "summarize": {"object_name": "", "image_name": ""},
+}
+
+items = (
+    [f"`{step}`" for step in list_of_steps]
+    + [
+        (
+            "[`{}`](https://kamangir-public.s3.ca-central-1.amazonaws.com/{}.tar.gz)".format(
+                step["object_name"],
+                step["image_name"],
+            )
+            if step["object_name"]
+            else ""
+        )
+        for step in list_of_steps.values()
+    ]
+    + [
+        (
+            "![image](https://github.com/kamangir/assets/blob/main/blue-sandbox/{}.png?raw=true)".format(
+                step["image_name"],
+            )
+            if step["image_name"]
+            else ""
+        )
+        for step in list_of_steps.values()
+    ]
+)

blue_sandbox-5.87.1/blue_sandbox/microsoft_building_damage_assessment/__main__.py

@@ -0,0 +1,82 @@
+import argparse
+
+from blueness import module
+from blueness.argparse.generic import sys_exit
+
+from blue_sandbox import NAME
+from blue_sandbox.microsoft_building_damage_assessment.ingest import ingest
+from blue_sandbox.microsoft_building_damage_assessment.label import label
+from blue_sandbox.microsoft_building_damage_assessment.train import train
+from blue_sandbox.microsoft_building_damage_assessment.sas_token import (
+    decode_token,
+    encode_token,
+)
+from blue_sandbox.help.microsoft_building_damage_assessment import list_of_events
+from blue_sandbox.logger import logger
+
+NAME = module.name(__file__, NAME)
+
+parser = argparse.ArgumentParser(NAME)
+parser.add_argument(
+    "task",
+    type=str,
+    help="decode_token | encode_token | ingest | label | train",
+)
+parser.add_argument(
+    "--object_name",
+    type=str,
+)
+parser.add_argument(
+    "--dataset_object_name",
+    type=str,
+)
+parser.add_argument(
+    "--model_object_name",
+    type=str,
+)
+parser.add_argument(
+    "--event_name",
+    type=str,
+    default=list_of_events[0],
+    help=" | ".join(list_of_events),
+)
+parser.add_argument(
+    "--verbose",
+    type=bool,
+    default=0,
+    help="0|1",
+)
+parser.add_argument(
+    "--token",
+    type=str,
+)
+args = parser.parse_args()
+
+success = False
+if args.task == "decode_token":
+    success = True
+    print(decode_token(args.token))
+elif args.task == "encode_token":
+    success = True
+    print(encode_token(args.token))
+elif args.task == "ingest":
+    success = ingest(
+        object_name=args.object_name,
+        event_name=args.event_name,
+        verbose=args.verbose == 1,
+    )
+elif args.task == "label":
+    success = label(
+        object_name=args.object_name,
+        verbose=args.verbose == 1,
+    )
+elif args.task == "train":
+    success = train(
+        dataset_object_name=args.dataset_object_name,
+        model_object_name=args.model_object_name,
+        verbose=args.verbose == 1,
+    )
+else:
+    success = None
+
+sys_exit(logger, NAME, args.task, success)

blue_sandbox-5.87.1/blue_sandbox/microsoft_building_damage_assessment/ingest.py

@@ -0,0 +1,31 @@
+from blueness import module
+
+from blue_objects import mlflow, metadata
+
+from blue_sandbox import NAME
+from blue_sandbox.logger import logger
+
+
+NAME = module.name(__file__, NAME)
+
+
+def ingest(
+    event_name: str,
+    object_name: str,
+    verbose: bool = False,
+) -> bool:
+    logger.info(f"{NAME}.ingest({event_name}) -> {object_name}")
+
+    return all(
+        [
+            mlflow.set_tags(
+                object_name,
+                {"event": event_name},
+            ),
+            metadata.post_to_object(
+                object_name,
+                "ingest",
+                {"event": event_name},
+            ),
+        ]
+    )

blue_sandbox-5.87.1/blue_sandbox/microsoft_building_damage_assessment/label.py

@@ -0,0 +1,38 @@
+import os
+
+from blueness import module
+from blue_objects import metadata, file, objects
+from blue_objects.env import abcli_path_git
+
+from blue_sandbox import NAME
+from blue_sandbox.logger import logger
+
+
+NAME = module.name(__file__, NAME)
+
+
+def label(
+    object_name: str,
+    verbose: bool = False,
+) -> bool:
+    logger.info(f"{NAME}.label: {object_name}")
+
+    geojson_filename = os.path.join(
+        abcli_path_git,
+        "building-damage-assessment/data/demo/labels/Maui_Wildfires_August_0.geojson",
+    )
+
+    if not file.copy(
+        geojson_filename,
+        objects.path_of(
+            filename="label.geojson",
+            object_name=object_name,
+        ),
+    ):
+        return False
+
+    return metadata.post_to_object(
+        object_name,
+        "label",
+        {"geojson": geojson_filename},
+    )

blue_sandbox-5.87.1/blue_sandbox/microsoft_building_damage_assessment/sas_token.py

@@ -0,0 +1,9 @@
+import base64
+
+
+def encode_token(token: str) -> str:
+    return base64.b64encode(token.encode()).decode()
+
+
+def decode_token(encoded_token: str) -> str:
+    return base64.b64decode(encoded_token.encode()).decode()

blue_sandbox-5.87.1/blue_sandbox/microsoft_building_damage_assessment/train.py

@@ -0,0 +1,95 @@
+import os
+
+from blueness import module
+from blue_objects import mlflow, metadata, file, objects, path
+from blue_objects.env import abcli_path_git
+
+from blue_sandbox import NAME
+from blue_sandbox.env import ENCODED_BLOB_SAS_TOKEN
+from blue_sandbox.microsoft_building_damage_assessment.sas_token import decode_token
+from blue_sandbox.logger import logger
+
+
+NAME = module.name(__file__, NAME)
+
+
+# ... copy `configs/example_config.yml` and fill the first three sections.
+def train(
+    dataset_object_name: str,
+    model_object_name: str,
+    verbose: bool = False,
+) -> bool:
+    logger.info(f"{NAME}.train: {dataset_object_name} -> {model_object_name}")
+
+    config_filename = os.path.join(
+        abcli_path_git,
+        "building-damage-assessment/configs/example_config.yml",
+    )
+    success, config = file.load_yaml(config_filename)
+    if not success:
+        return False
+
+    config["experiment_dir"] = objects.object_path(model_object_name)
+    config["experiment_name"] = model_object_name
+
+    config["imagery"]["rgb_fn"] = config["imagery"]["raw_fn"] = objects.path_of(
+        filename="raw/maxar_lahaina_8_12_2023-visual.tif",
+        object_name=dataset_object_name,
+    )
+
+    config["inference"]["output_subdir"] = objects.path_of(
+        filename="outputs/",
+        object_name=model_object_name,
+    )
+    config["inference"]["checkpoint_fn"] = objects.path_of(
+        filename="checkpoints/last.ckpt",
+        object_name=model_object_name,
+    )
+
+    config["infrastructure"]["container_name"] = "sandbox"
+    config["infrastructure"][
+        "storage_account"
+    ] = "https://kamangir.blob.core.windows.net/"
+    config["infrastructure"]["sas_token"] = decode_token(ENCODED_BLOB_SAS_TOKEN)
+    config["infrastructure"]["relative_path"] = dataset_object_name
+
+    config["labels"]["fn"] = objects.path_of(
+        filename="label.geojson",
+        object_name=dataset_object_name,
+    )
+
+    config["training"]["log_dir"] = objects.path_of(
+        filename="logs/",
+        object_name=model_object_name,
+    )
+    config["training"]["checkpoint_subdir"] = objects.path_of(
+        filename="checkpoints/",
+        object_name=model_object_name,
+    )
+
+    if not file.save_yaml(
+        objects.path_of(
+            filename="config.yml",
+            object_name=model_object_name,
+        ),
+        config,
+    ):
+        return False
+
+    return all(
+        [
+            mlflow.set_tags(
+                model_object_name,
+                {
+                    "dataset": dataset_object_name,
+                },
+            ),
+            metadata.post_to_object(
+                model_object_name,
+                "train",
+                {
+                    "dataset": dataset_object_name,
+                },
+            ),
+        ]
+    )

blue_sandbox-5.87.1/blue_sandbox/sample.env

@@ -0,0 +1 @@
+ENCODED_BLOB_SAS_TOKEN=value

{blue_sandbox-5.47.1 → blue_sandbox-5.87.1/blue_sandbox.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.2
 Name: blue_sandbox
-Version: 5.47.1
+Version: 5.87.1
 Summary: 🌀 A sandbox for ideas and experiments.
 Home-page: https://github.com/kamangir/blue-sandbox
 Author: Arash Abadpour (Kamangir)
@@ -13,8 +13,12 @@ Classifier: Operating System :: OS Independent
 Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: blueness
-Requires-Dist: blue-options
 Requires-Dist: abcli
+Requires-Dist: blue-options
+Requires-Dist: blue_objects
+Requires-Dist: blue_geo
+Requires-Dist: blueflow
+Requires-Dist: blue_plugin
 Requires-Dist: boto3
 Requires-Dist: geojson
 Requires-Dist: geopandas
@@ -27,6 +31,9 @@ Requires-Dist: pylint
 Requires-Dist: pytest
 Requires-Dist: python-dotenv[cli]
 Requires-Dist: tqdm
+Requires-Dist: azure-storage-blob
+Requires-Dist: fiona
+Requires-Dist: tensorboard
 Dynamic: author
 Dynamic: author-email
 Dynamic: classifier
@@ -47,11 +54,11 @@ pip install blue-sandbox
 
 |   |   |   |
 | --- | --- | --- |
-| 🌐[``@damage``](https://github.com/kamangir/blue-sandbox/blob/main/blue_sandbox/microsoft_building_damage_assessment/README.md) [![image](https://github.com/microsoft/building-damage-assessment/raw/main/figures/damage.png)](https://github.com/kamangir/blue-sandbox/blob/main/blue_sandbox/microsoft_building_damage_assessment/README.md) Satellite imagery damage assessment workflow | 🌀[`experiment 2`](#) [![image](https://github.com/kamangir/assets/raw/main/blue-plugin/marquee.png?raw=true)](#) description of experiment 2 ... | 🌀[`experiment 3`](#) [![image](https://github.com/kamangir/assets/raw/main/blue-plugin/marquee.png?raw=true)](#) description of experiment 3 ... |
+| 🌐[``@damages``](https://github.com/kamangir/blue-sandbox/blob/main/blue_sandbox/microsoft_building_damage_assessment/README.md) [![image](https://github.com/microsoft/building-damage-assessment/raw/main/figures/damage.png)](https://github.com/kamangir/blue-sandbox/blob/main/blue_sandbox/microsoft_building_damage_assessment/README.md) Satellite imagery damage assessment workflow | 🌀[`experiment 2`](#) [![image](https://github.com/kamangir/assets/raw/main/blue-plugin/marquee.png?raw=true)](#) description of experiment 2 ... | 🌀[`experiment 3`](#) [![image](https://github.com/kamangir/assets/raw/main/blue-plugin/marquee.png?raw=true)](#) description of experiment 3 ... |
 
 ---
 
 
 [![pylint](https://github.com/kamangir/blue-sandbox/actions/workflows/pylint.yml/badge.svg)](https://github.com/kamangir/blue-sandbox/actions/workflows/pylint.yml) [![pytest](https://github.com/kamangir/blue-sandbox/actions/workflows/pytest.yml/badge.svg)](https://github.com/kamangir/blue-sandbox/actions/workflows/pytest.yml) [![bashtest](https://github.com/kamangir/blue-sandbox/actions/workflows/bashtest.yml/badge.svg)](https://github.com/kamangir/blue-sandbox/actions/workflows/bashtest.yml) [![PyPI version](https://img.shields.io/pypi/v/blue-sandbox.svg)](https://pypi.org/project/blue-sandbox/) [![PyPI - Downloads](https://img.shields.io/pypi/dd/blue-sandbox)](https://pypistats.org/packages/blue-sandbox)
 
-built by 🌀 [`blue_options-4.177.1`](https://github.com/kamangir/awesome-bash-cli), based on 🌀 [`blue_sandbox-5.47.1`](https://github.com/kamangir/blue-sandbox).
+built by 🌀 [`blue_options-4.180.1`](https://github.com/kamangir/awesome-bash-cli), based on 🌀 [`blue_sandbox-5.87.1`](https://github.com/kamangir/blue-sandbox).

{blue_sandbox-5.47.1 → blue_sandbox-5.87.1}/blue_sandbox.egg-info/SOURCES.txt

@@ -28,11 +28,17 @@ blue_sandbox/.abcli/browse.sh
 blue_sandbox/.abcli/install.sh
 blue_sandbox/.abcli/microsoft_building_damage_assessment.sh
 blue_sandbox/.abcli/microsoft_building_damage_assessment/ingest.sh
+blue_sandbox/.abcli/microsoft_building_damage_assessment/install.sh
+blue_sandbox/.abcli/microsoft_building_damage_assessment/label.sh
+blue_sandbox/.abcli/microsoft_building_damage_assessment/tensorboard.sh
+blue_sandbox/.abcli/microsoft_building_damage_assessment/train.sh
 blue_sandbox/.abcli/microsoft_building_damage_assessment/ingest/list.sh
 blue_sandbox/.abcli/tests/README.sh
 blue_sandbox/.abcli/tests/help.sh
 blue_sandbox/.abcli/tests/microsoft_building_damage_assessment_ingest.sh
 blue_sandbox/.abcli/tests/microsoft_building_damage_assessment_ingest_list.sh
+blue_sandbox/.abcli/tests/microsoft_building_damage_assessment_label.sh
+blue_sandbox/.abcli/tests/microsoft_building_damage_assessment_train.sh
 blue_sandbox/.abcli/tests/version.sh
 blue_sandbox/help/__init__.py
 blue_sandbox/help/__main__.py
@@ -40,4 +46,7 @@ blue_sandbox/help/functions.py
 blue_sandbox/microsoft_building_damage_assessment/README.py
 blue_sandbox/microsoft_building_damage_assessment/__init__.py
 blue_sandbox/microsoft_building_damage_assessment/__main__.py
-blue_sandbox/microsoft_building_damage_assessment/ingest.py
+blue_sandbox/microsoft_building_damage_assessment/ingest.py
+blue_sandbox/microsoft_building_damage_assessment/label.py
+blue_sandbox/microsoft_building_damage_assessment/sas_token.py
+blue_sandbox/microsoft_building_damage_assessment/train.py

{blue_sandbox-5.47.1 → blue_sandbox-5.87.1}/blue_sandbox.egg-info/requires.txt

@@ -1,6 +1,10 @@
 blueness
-blue-options
 abcli
+blue-options
+blue_objects
+blue_geo
+blueflow
+blue_plugin
 boto3
 geojson
 geopandas
@@ -13,3 +17,6 @@ pylint
 pytest
 python-dotenv[cli]
 tqdm
+azure-storage-blob
+fiona
+tensorboard

{blue_sandbox-5.47.1 → blue_sandbox-5.87.1}/requirements.txt

@@ -1,6 +1,11 @@
 blueness
-blue-options
 abcli
+blue-options
+blue_objects
+blue_geo
+blueflow
+blue_plugin
+
 boto3
 geojson
 geopandas
@@ -13,3 +18,7 @@ pylint
 pytest
 python-dotenv[cli]
 tqdm
+
+azure-storage-blob
+fiona
+tensorboard

blue_sandbox-5.47.1/blue_sandbox/.abcli/alias.sh

@@ -1,5 +0,0 @@
-#! /usr/bin/env bash
-
-alias @damage="blue_sandbox_microsoft_building_damage_assessment"
-
-alias @sandbox=blue_sandbox

blue_sandbox-5.47.1/blue_sandbox/.abcli/install.sh

@@ -1,12 +0,0 @@
-#! /usr/bin/env bash
-
-function abcli_install_blue_sandbox() {
-    abcli_git_clone https://github.com/microsoft/building-damage-assessment.git
-
-    [[ "$abcli_is_mac" == true ]] &&
-        brew install wget
-
-    return 0
-}
-
-abcli_install_module blue_sandbox 1.1.1

blue_sandbox-5.47.1/blue_sandbox/config.env

@@ -1 +0,0 @@
-BLUE_PLUGIN_CONFIG=value

blue_sandbox-5.47.1/blue_sandbox/env.py

@@ -1,11 +0,0 @@
-import os
-from blue_options.env import load_config, load_env
-
-load_env(__name__)
-load_config(__name__)
-
-
-BLUE_PLUGIN_CONFIG = os.getenv(
-    "BLUE_PLUGIN_CONFIG",
-    "",
-)

blue_sandbox-5.47.1/blue_sandbox/microsoft_building_damage_assessment/README.py

@@ -1,34 +0,0 @@
-from typing import Dict
-
-list_of_steps: Dict[str, Dict] = {
-    "ingest": "Maui-Hawaii-fires-Aug-23-damage-2025-01-09-GgnjQC",
-    "label": "",
-    "train": "",
-    "predict": "",
-    "summarize": "",
-}
-
-items = (
-    [f"`{step}`" for step in list_of_steps]
-    + [
-        (
-            "[`{}`](https://kamangir-public.s3.ca-central-1.amazonaws.com/{}.tar.gz)".format(
-                object_name,
-                object_name,
-            )
-            if object_name
-            else ""
-        )
-        for object_name in list_of_steps.values()
-    ]
-    + [
-        (
-            "[![image](https://github.com/kamangir/assets/blob/main/blue-sandbox/{}.png?raw=true)](#)".format(
-                object_name,
-            )
-            if object_name
-            else ""
-        )
-        for object_name in list_of_steps.values()
-    ]
-)

blue_sandbox-5.47.1/blue_sandbox/microsoft_building_damage_assessment/__main__.py

@@ -1,47 +0,0 @@
-import argparse
-
-from blueness import module
-from blueness.argparse.generic import sys_exit
-
-from blue_sandbox import NAME
-from blue_sandbox.microsoft_building_damage_assessment.ingest import ingest
-from blue_sandbox.help.microsoft_building_damage_assessment import list_of_events
-from blue_sandbox.logger import logger
-
-NAME = module.name(__file__, NAME)
-
-parser = argparse.ArgumentParser(NAME)
-parser.add_argument(
-    "task",
-    type=str,
-    help="ingest",
-)
-parser.add_argument(
-    "--object_name",
-    type=str,
-)
-parser.add_argument(
-    "--event_name",
-    type=str,
-    default=list_of_events[0],
-    help=" | ".join(list_of_events),
-)
-parser.add_argument(
-    "--verbose",
-    type=bool,
-    default=0,
-    help="0|1",
-)
-args = parser.parse_args()
-
-success = False
-if args.task == "ingest":
-    success = ingest(
-        object_name=args.object_name,
-        event_name=args.event_name,
-        verbose=args.verbose == 1,
-    )
-else:
-    success = None
-
-sys_exit(logger, NAME, args.task, success)

blue_sandbox-5.47.1/blue_sandbox/microsoft_building_damage_assessment/ingest.py

@@ -1,19 +0,0 @@
-from blueness import module
-
-from blue_sandbox import NAME
-from blue_sandbox.logger import logger
-
-
-NAME = module.name(__file__, NAME)
-
-
-def ingest(
-    event_name: str,
-    object_name: str,
-    verbose: bool = False,
-) -> bool:
-    logger.info(f"{NAME}.ingest({event_name}) -> {object_name}")
-
-    logger.info("🪄")
-
-    return True