mlcflow 1.2.3__tar.gz → 1.2.5__tar.gz

{mlcflow-1.2.3 → mlcflow-1.2.5}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mlcflow
-Version: 1.2.3
+Version: 1.2.5
 Summary: An automation interface tailored for CPU/GPU benchmarking
 Author-email: MLCommons <systems@mlcommons.org>
 License: 
@@ -204,10 +204,10 @@ Dynamic: license-file
 [![License](https://img.shields.io/badge/License-Apache%202.0-green)](https://github.com/mlcommons/mlcflow/blob/main/LICENSE.md)
 [![Downloads](https://static.pepy.tech/badge/mlcflow)](https://pepy.tech/project/mlcflow)
 
-[![MLC core actions test](https://github.com/mlcommons/mlcflow/actions/workflows/test-mlc-core-actions.yaml/badge.svg)](https://github.com/mlcommons/mlcflow/actions/workflows/test-mlc-core-actions.yaml)
-[![MLC script automation features test](https://github.com/mlcommons/mlperf-automations/actions/workflows/test-mlc-script-features.yml/badge.svg?cache-bust=1)](https://github.com/mlcommons/mlperf-automations/actions/workflows/test-mlc-script-features.yml)
-[![MLPerf inference resnet50](https://github.com/mlcommons/mlcflow/actions/workflows/mlperf-inference-resnet50.yml/badge.svg)](https://github.com/mlcommons/mlcflow/actions/workflows/mlperf-inference-resnet50.yml)
-[![MLPerf inference bert (deepsparse, tf, onnxruntime, pytorch)](https://github.com/mlcommons/mlcflow/actions/workflows/mlperf-inference-bert.yml/badge.svg)](https://github.com/mlcommons/mlcflow/actions/workflows/mlperf-inference-bert.yml)
+[![MLC core actions test](https://github.com/mlcommons/mlcflow/actions/workflows/test-mlc-core-actions.yaml/badge.svg?branch=dev)](https://github.com/mlcommons/mlcflow/actions/workflows/test-mlc-core-actions.yaml)
+[![MLC script automation features test](https://github.com/mlcommons/mlperf-automations/actions/workflows/test-mlc-script-features.yml/badge.svg?branch=dev)](https://github.com/mlcommons/mlperf-automations/actions/workflows/test-mlc-script-features.yml)
+[![MLPerf inference resnet50](https://github.com/mlcommons/mlcflow/actions/workflows/mlperf-inference-resnet50.yml/badge.svg?branch=dev)](https://github.com/mlcommons/mlcflow/actions/workflows/mlperf-inference-resnet50.yml)
+[![MLPerf inference bert (deepsparse, tf, onnxruntime, pytorch)](https://github.com/mlcommons/mlcflow/actions/workflows/mlperf-inference-bert.yml/badge.svg?branch=dev)](https://github.com/mlcommons/mlcflow/actions/workflows/mlperf-inference-bert.yml)
 
 
 MLCFlow is a versatile CLI and Python interface developed by MLCommons in collaboration with a dedicated team of volunteers (see [Contributors](https://github.com/mlcommons/mlcflow/blob/main/CONTRIBUTORS.md)). It serves as a streamlined replacement for the [CMind](https://github.com/mlcommons/ck/tree/master/cm) tool, designed to drive the automation workflows of MLPerf benchmarks more efficiently. You can use this tool for any of your workflow automation tasks. 
@@ -259,7 +259,7 @@ Each target has its own set of specific actions to tailor automation workflows a
 | Target | Action          |
 |--------|-----------------|
 | script    | run, find/search, rm, mv, cp, add, test, docker-run, show, experiment, doc   |
-| cache    | find/search, rm, show  |
+| cache    | find/search, rm, show, list, prune, mark-tmp  |
 | repo    | pull, search, rm, list, find/search , add        |
 
 
@@ -356,4 +356,3 @@ classDiagram
     Item o-- Repo
     Automation o-- Action
 ```
-

{mlcflow-1.2.3 → mlcflow-1.2.5}/README.md

@@ -3,10 +3,10 @@
 [![License](https://img.shields.io/badge/License-Apache%202.0-green)](https://github.com/mlcommons/mlcflow/blob/main/LICENSE.md)
 [![Downloads](https://static.pepy.tech/badge/mlcflow)](https://pepy.tech/project/mlcflow)
 
-[![MLC core actions test](https://github.com/mlcommons/mlcflow/actions/workflows/test-mlc-core-actions.yaml/badge.svg)](https://github.com/mlcommons/mlcflow/actions/workflows/test-mlc-core-actions.yaml)
-[![MLC script automation features test](https://github.com/mlcommons/mlperf-automations/actions/workflows/test-mlc-script-features.yml/badge.svg?cache-bust=1)](https://github.com/mlcommons/mlperf-automations/actions/workflows/test-mlc-script-features.yml)
-[![MLPerf inference resnet50](https://github.com/mlcommons/mlcflow/actions/workflows/mlperf-inference-resnet50.yml/badge.svg)](https://github.com/mlcommons/mlcflow/actions/workflows/mlperf-inference-resnet50.yml)
-[![MLPerf inference bert (deepsparse, tf, onnxruntime, pytorch)](https://github.com/mlcommons/mlcflow/actions/workflows/mlperf-inference-bert.yml/badge.svg)](https://github.com/mlcommons/mlcflow/actions/workflows/mlperf-inference-bert.yml)
+[![MLC core actions test](https://github.com/mlcommons/mlcflow/actions/workflows/test-mlc-core-actions.yaml/badge.svg?branch=dev)](https://github.com/mlcommons/mlcflow/actions/workflows/test-mlc-core-actions.yaml)
+[![MLC script automation features test](https://github.com/mlcommons/mlperf-automations/actions/workflows/test-mlc-script-features.yml/badge.svg?branch=dev)](https://github.com/mlcommons/mlperf-automations/actions/workflows/test-mlc-script-features.yml)
+[![MLPerf inference resnet50](https://github.com/mlcommons/mlcflow/actions/workflows/mlperf-inference-resnet50.yml/badge.svg?branch=dev)](https://github.com/mlcommons/mlcflow/actions/workflows/mlperf-inference-resnet50.yml)
+[![MLPerf inference bert (deepsparse, tf, onnxruntime, pytorch)](https://github.com/mlcommons/mlcflow/actions/workflows/mlperf-inference-bert.yml/badge.svg?branch=dev)](https://github.com/mlcommons/mlcflow/actions/workflows/mlperf-inference-bert.yml)
 
 
 MLCFlow is a versatile CLI and Python interface developed by MLCommons in collaboration with a dedicated team of volunteers (see [Contributors](https://github.com/mlcommons/mlcflow/blob/main/CONTRIBUTORS.md)). It serves as a streamlined replacement for the [CMind](https://github.com/mlcommons/ck/tree/master/cm) tool, designed to drive the automation workflows of MLPerf benchmarks more efficiently. You can use this tool for any of your workflow automation tasks. 
@@ -58,7 +58,7 @@ Each target has its own set of specific actions to tailor automation workflows a
 | Target | Action          |
 |--------|-----------------|
 | script    | run, find/search, rm, mv, cp, add, test, docker-run, show, experiment, doc   |
-| cache    | find/search, rm, show  |
+| cache    | find/search, rm, show, list, prune, mark-tmp  |
 | repo    | pull, search, rm, list, find/search , add        |
 
 
@@ -155,4 +155,3 @@ classDiagram
     Item o-- Repo
     Automation o-- Action
 ```
-

{mlcflow-1.2.3 → mlcflow-1.2.5}/mlc/action.py

@@ -185,6 +185,15 @@ class Action:
             self._index = Index(self.repos_path, self.repos)
         return self._index
 
+    def _item_from_index_entry(self, res, target_name):
+        """Create an Item from an index entry and skip entries with invalid meta."""
+        it = Item(res['path'], res['repo'])
+        if not isinstance(it.meta, dict):
+            logger.warning(
+                f"Skipping {target_name} item at {it.path}: missing or invalid meta")
+            return None
+        return it
+
     def __init__(self):
         setup_logging(log_path=os.getcwd(), log_file='.mlc-log.txt')
         self.logger = logger
@@ -728,8 +737,13 @@ class Action:
         # For targets like cache, sometimes user would need to clear the entire cache folder present in the system
         # this helps to fetch entire data pertaining to particular target
         if fetch_all:
+            if not target_index:
+                return {'return': 0, 'list': result}
+
             for res in target_index:
-                result.append(Item(res['path'], res['repo']))
+                it = self._item_from_index_entry(res, target)
+                if it:
+                    result.append(it)
             return {'return': 0, 'list': result}
 
         if not uid and not alias and i.get('details'):
@@ -775,14 +789,16 @@ class Action:
                 for res in target_index:
                     if (res["uid"] == uid or (alias and res["alias"] == alias)) and (
                             not item_repo or item_repo == res['repo']):
-                        it = Item(res['path'], res['repo'])
-                        result.append(it)
-                        found = True
+                        it = self._item_from_index_entry(res, target)
+                        if it:
+                            result.append(it)
+                            found = True
                 if not found and folder_name:
                     for res in target_index:
                         if os.path.basename(res["path"]) == folder_name:
-                            it = Item(res['path'], res['repo'])
-                            result.append(it)
+                            it = self._item_from_index_entry(res, target)
+                            if it:
+                                result.append(it)
             else:
                 tags = i.get("tags")
                 if tags:
@@ -803,11 +819,12 @@ class Action:
                 n_tags = [p[1:] for p in n_tags_]
                 p_tags = list(set(tags_to_match) - set(n_tags_))
                 for res in target_index:
-                    c_tags = res["tags"]
+                    c_tags = res.get("tags") or []
                     if (exact_tags_match and set(p_tags) == set(c_tags)) or (not exact_tags_match and set(
                             p_tags).issubset(set(c_tags)) and set(n_tags).isdisjoint(set(c_tags))):
-                        it = Item(res['path'], res['repo'])
-                        result.append(it)
+                        it = self._item_from_index_entry(res, target)
+                        if it:
+                            result.append(it)
         return {'return': 0, 'list': result}
 
     find = search

{mlcflow-1.2.3 → mlcflow-1.2.5}/mlc/cache_action.py

@@ -5,6 +5,7 @@ import time
 from . import utils
 from .logger import logger
 
+
 class CacheAction(Action):
     """
     ####################################################################################################################
@@ -15,14 +16,16 @@ class CacheAction(Action):
     2. show
     3. list
     4. remove(rm)
+    5. prune
+    6. mark-tmp
 
     """
 
     def __init__(self, parent=None):
-        #super().__init__(parent)
+        # super().__init__(parent)
         self.parent = parent
         self.__dict__.update(vars(parent))
-    
+
     def search(self, i):
         """
     ####################################################################################################################
@@ -30,11 +33,11 @@ class CacheAction(Action):
     Action: Find (Alias: Search)
     ####################################################################################################################
 
-    The `find` (or `search`) action retrieves the path of cache generated by a particular script. 
+    The `find` (or `search`) action retrieves the path of cache generated by a particular script.
 
     Syntax:
 
-    mlc find cache --tags=<list_of_tag_used_to_run_the_particular_script> 
+    mlc find cache --tags=<list_of_tag_used_to_run_the_particular_script>
 
     Example Command:
 
@@ -42,7 +45,7 @@ class CacheAction(Action):
 
         """
         i['target_name'] = "cache"
-        #logger.debug(f"Searching for cache with input: {i}")
+        # logger.debug(f"Searching for cache with input: {i}")
         r = self.parent.search(i)
         if r['return'] > 0:
             return r
@@ -83,13 +86,13 @@ class CacheAction(Action):
 
     Syntax:
 
-    mlc rm cache --tags=<list_of_tag_used_to_run_the_particular_script> 
+    mlc rm cache --tags=<list_of_tag_used_to_run_the_particular_script>
 
     Options:
         1. `-f`: Force removes caches without confirmation. Without `-f`, the user will be prompted for confirmation before deletion.
 
     To remove all generated caches, use:
-        
+
     mlc rm cache
 
     Example Command:
@@ -98,9 +101,68 @@ class CacheAction(Action):
 
         """
         i['target_name'] = "cache"
-        #logger.debug(f"Removing cache with input: {i}")
+        # logger.debug(f"Removing cache with input: {i}")
         return self.parent.rm(i)
 
+    def mark_tmp(self, i):
+        """
+    ####################################################################################################################
+    Target: Cache
+    Action: Mark Tmp (mark-tmp)
+    ####################################################################################################################
+
+    The `mark-tmp` action marks one or more cache entries as `tmp` without removing the cache contents.
+
+    Syntax:
+
+    mlc mark-tmp cache --tags=<list_of_tags_used_to_run_the_particular_script>
+
+    Example Command:
+
+    mlc mark-tmp cache --tags=get,dataset,igbh
+
+        """
+        i['target_name'] = "cache"
+        res = self.search(i)
+        if res['return'] > 0:
+            return res
+
+        if len(res['list']) == 0:
+            logger.warning(f"No cache found for {i}")
+            return {'return': 0, "warnings": [
+                {"code": "EMPTY_TARGET", "description": f"No cache found for {i}"}]}
+
+        updated_count = 0
+
+        for item in res['list']:
+            tags = item.meta.get("tags", [])
+            if 'tmp' in tags:
+                continue
+
+            tags.append('tmp')
+            item.meta["tags"] = tags
+            meta_yaml_path = os.path.join(item.path, "meta.yaml")
+            meta_json_path = os.path.join(item.path, "meta.json")
+
+            if os.path.exists(meta_yaml_path):
+                save_result = utils.save_yaml(meta_yaml_path, meta=item.meta)
+            else:
+                save_result = utils.save_json(meta_json_path, meta=item.meta)
+
+            if save_result['return'] > 0:
+                return save_result
+
+            self.get_index().update(item.meta, "cache", item.path, item.repo)
+            updated_count += 1
+
+        logger.info(f"Marked {updated_count} cache item(s) as tmp")
+
+        return {
+            'return': 0,
+            'message': f"Marked {updated_count} cache item(s) as tmp",
+            'list': res['list']
+        }
+
     def show(self, run_args):
         """
     ####################################################################################################################
@@ -109,9 +171,9 @@ class CacheAction(Action):
     ####################################################################################################################
 
     Retrieves the path and metadata of caches generated while running scripts through MLCFlow.
-        
+
     Syntax:
-        
+
     mlc show cache --tags=<list_of_tags_used_while_running_script>
 
     Example Command:
@@ -128,7 +190,11 @@ class CacheAction(Action):
             return res
 
         logger.info(f"Showing cache with tags: {run_args.get('tags')}")
-        cached_meta_keys_to_show = ["uid", "tags", "dependent_cached_path", "associated_script_item"]
+        cached_meta_keys_to_show = [
+            "uid",
+            "tags",
+            "dependent_cached_path",
+            "associated_script_item"]
         cached_state_keys_to_show = ["new_env", "new_state", "version"]
         for item in res['list']:
             print(f"""Location: {item.path}:
@@ -137,7 +203,8 @@ Cache Meta:""")
                 if key in item.meta:
                     print(f"""    {key}: {item.meta[key]}""")
             print("""Cached State:""")
-            cached_state_meta_file = os.path.join(item.path, "mlc-cached-state.json")
+            cached_state_meta_file = os.path.join(
+                item.path, "mlc-cached-state.json")
             if not os.path.exists(cached_state_meta_file):
                 continue
             try:
@@ -149,17 +216,16 @@ Cache Meta:""")
                             print(f"""    {key}:""", end="")
                             if meta[key] and isinstance(meta[key], dict):
                                 print("")
-                                utils.printd(meta[key], yaml=False, sort_keys=True, begin_spaces=8)
+                                utils.printd(
+                                    meta[key], yaml=False, sort_keys=True, begin_spaces=8)
                             else:
                                 print(f""" {meta[key]}""")
             except json.JSONDecodeError as e:
                 logger.error(f"Error decoding JSON: {e}")
             print("......................................................")
             print("")
-            
-        return {'return': 0}
 
-    
+        return {'return': 0}
 
     def prune(self, args):
         """
@@ -176,12 +242,13 @@ Cache Meta:""")
 
         """
         self.action_type = "cache"
-        run_args = {"fetch_all": True}  # to fetch the details of all the caches generated
-        
+        # to fetch the details of all the caches generated
+        run_args = {"fetch_all": True}
+
         res = self.search(run_args)
         if res['return'] > 0:
             return res
-        
+
         for item in res['list']:
             expiration_time = item.meta.get('cache_expiration')
             if expiration_time is not None and expiration_time < time.time():
@@ -192,8 +259,6 @@ Cache Meta:""")
                     self.rm(ii)
 
         return {'return': 0}
-        
-
 
     def list(self, args):
         """
@@ -210,16 +275,18 @@ Cache Meta:""")
 
         """
         self.action_type = "cache"
-        run_args = {"fetch_all": True}  # to fetch the details of all the caches generated
-        
+        # to fetch the details of all the caches generated
+        run_args = {"fetch_all": True}
+
         res = self.search(run_args)
         if res['return'] > 0:
             return res
-        
+
         logger.info(f"Listing all the caches and their paths")
         print("......................................................")
         for item in res['list']:
-            print(f"tags: {item.meta['tags'] if item.meta.get('tags') else 'None'}")
+            print(
+                f"tags: {item.meta['tags'] if item.meta.get('tags') else 'None'}")
             print(f"Location: {item.path}")
             print("......................................................")
 

mlcflow-1.2.5/mlc/error_codes.py

@@ -0,0 +1,209 @@
+import re
+from enum import Enum, auto
+
+
+class ErrorCode(Enum):
+    """Enum class for error codes in MLCFlow"""
+    # General errors (2000-2007)
+    AUTOMATION_SCRIPT_NOT_FOUND = (
+        2000, "The specified automation script was not found")
+    PATH_DOES_NOT_EXIST = (2001, "Provided path does not exists")
+    FILE_NOT_FOUND = (2002, "Required file was not found")
+    PERMISSION_DENIED = (2003, "Insufficient permission to execute the script")
+    IO_Error = (2004, "File I/O operation failed")
+    AUTOMATION_CUSTOM_ERROR = (2005, "Custom error triggered by the script")
+    UNSUPPORTED_OS = (
+        2006, "The Operating System is not supported by the script")
+    MISSING_ENV_VARIABLE = (2007, "Required environment variables are missing")
+
+    def __init__(self, code, description):
+        self.code = code
+        self.description = description
+
+
+ERROR_CODES = {error.code for error in ErrorCode}
+ERROR_CODE_PATTERNS = [
+    re.compile(r'(?:error|exit|return)\s+code\s*[:=]?\s*(\d+)', re.IGNORECASE),
+    re.compile(r'\[errno\s+(\d+)\]', re.IGNORECASE),
+    re.compile(r'signal\s+(\d+)', re.IGNORECASE),
+]
+
+
+class WarningCode(Enum):
+    """Enum class for warning codes in MLCFlow"""
+    # General warnings (1000-1007)
+    IO_WARNING = (1000, "File I/O operation warning")
+    AUTOMATION_SCRIPT_NOT_TESTED = (
+        1001,
+        "the script is not tested on the current operatinig system or is in a development state")
+    AUTOMATION_SCRIPT_SKIPPED = (
+        1002, "The script has been skipped during execution")
+    AUTOMATION_CUSTOM_ERROR = (1003, "Custom warning triggered by the script")
+    NON_INTERACTIVE_ENV = (1004, "Non interactive environment detected")
+    ELEVATED_PERMISSION_NEEDED = (1005, "Elevated permission needed")
+    EMPTY_TARGET = (1006, "The specified target is empty")
+
+    def __init__(self, code, description):
+        self.code = code
+        self.description = description
+
+
+def get_error_info(error_code):
+    """Get the error message for a given error code"""
+    try:
+        return {"error_code": ErrorCode(
+            error_code).code, "error_message": ErrorCode(error_code).description}
+    except ValueError:
+        return f"Unknown error code: {error_code}"
+
+
+def get_warning_info(warning_code):
+    """Get the warning message for a given warning code"""
+    try:
+        return {"warning_code": WarningCode(
+            warning_code).code, "warning_message": WarningCode(warning_code).description}
+    except ValueError:
+        return f"Unknown warning code: {warning_code}"
+
+
+def is_warning_code(code):
+    """Check if a given code is a warning code"""
+    try:
+        # Check if code is in warning range (2000-2399)
+        if 2000 <= code <= 2399:
+            WarningCode(code)
+            return True
+        return False
+    except ValueError:
+        return False
+
+
+def is_error_code(code):
+    """Check if a given code is an error code"""
+    try:
+        # Check if code is in error range (1000-1399)
+        if 1000 <= code <= 1399:
+            ErrorCode(code)
+            return True
+        return False
+    except ValueError:
+        return False
+
+
+def get_code_type(code):
+    """Get the type of a code (error or warning)"""
+    if is_error_code(code):
+        return "error"
+    elif is_warning_code(code):
+        return "warning"
+    else:
+        return "unknown"
+
+
+def _normalize_code(code):
+    """Convert a code value to int when possible."""
+    if code is None:
+        return None
+    if isinstance(code, int):
+        return code
+    try:
+        return int(str(code).strip())
+    except (TypeError, ValueError):
+        return None
+
+
+def detect_error_code(return_code=None, error_message=""):
+    """Detect the most useful error code from a return value or error text."""
+    normalized_return_code = _normalize_code(return_code)
+    message = error_message or ""
+
+    for pattern in ERROR_CODE_PATTERNS:
+        match = pattern.search(message)
+        if match:
+            detected = _normalize_code(match.group(1))
+            # Prefer the code extracted from the error text when the outer
+            # result only carries a generic status such as 1.
+            if normalized_return_code in (None, 0, 1):
+                return detected
+
+    return normalized_return_code
+
+
+def get_error_guidance(return_code=None, error_message=""):
+    """Return actionable guidance for known error codes and failure patterns."""
+    error_code = detect_error_code(return_code, error_message)
+    guidance = {
+        "error_code": error_code,
+        "error_message": None,
+        "suggestions": [],
+    }
+
+    if error_code in ERROR_CODES:
+        error_info = get_error_info(error_code)
+        if isinstance(error_info, dict):
+            guidance["error_message"] = error_info["error_message"]
+
+    message = (error_message or "").lower()
+
+    if ("no space left on device" in message or
+            "disk full" in message or
+            "not enough space" in message):
+        guidance["error_message"] = guidance["error_message"] or \
+            "Likely disk space exhaustion while running the script"
+        guidance["suggestions"] = [
+            "Free disk space in the work/cache directories and retry the command.",
+            "Remove old artifacts or caches if they are no longer needed.",
+        ]
+    # 139 = 128 + SIGSEGV(11), while some tools report the raw signal number
+    # 11 or its negative form -11.
+    elif ("segmentation fault" in message or error_code in [139, -11, 11]):
+        guidance["error_message"] = guidance["error_message"] or \
+            "A native program crashed with a segmentation fault"
+        guidance["suggestions"] = [
+            "Rerun with verbose logs to identify which native command crashed.",
+            "Check native dependencies, compiler/runtime compatibility, and input files.",
+        ]
+    # Common downloader/network failure codes used by curl and similar tools:
+    # 6/7 resolve/connect failures, 28 timeout, 35 TLS/connect issue,
+    # 56 connection reset/read failure, 60 certificate validation failure.
+    elif ("network" in message or
+          "connection" in message or
+          "timed out" in message or
+          "temporary failure in name resolution" in message or
+          "could not resolve host" in message or
+          error_code in [6, 7, 28, 35, 56, 60]):
+        guidance["error_message"] = guidance["error_message"] or \
+            "Likely network or download failure while running the script"
+        guidance["suggestions"] = [
+            "Check internet connectivity, proxy/firewall settings, and remote endpoint availability.",
+            "Retry the command after verifying the network connection.",
+        ]
+    elif error_code == 126:
+        guidance["error_message"] = guidance["error_message"] or \
+            "Command found but it could not be executed"
+        guidance["suggestions"] = [
+            "Check file permissions and whether the target command is executable.",
+        ]
+    elif error_code == 127:
+        guidance["error_message"] = guidance["error_message"] or \
+            "Command not found during script execution"
+        guidance["suggestions"] = [
+            "Verify that the required tool is installed and available on PATH.",
+        ]
+    elif error_code == 137:
+        guidance["error_message"] = guidance["error_message"] or \
+            "Process was terminated, often due to out-of-memory or a kill signal"
+        guidance["suggestions"] = [
+            "Check system memory limits and retry with fewer parallel jobs if possible.",
+        ]
+    elif error_code == 130:
+        guidance["error_message"] = guidance["error_message"] or \
+            "The command was interrupted by the user or the environment"
+        guidance["suggestions"] = [
+            "Retry the command if the interruption was unexpected.",
+        ]
+
+    if not guidance["error_message"] and not guidance["suggestions"]:
+        return None
+
+    return guidance

{mlcflow-1.2.3 → mlcflow-1.2.5}/mlc/index.py

@@ -232,8 +232,12 @@ class Index:
 
                 if os.path.isfile(yaml_path):
                     config_path = yaml_path
+                    with open(config_path, "r") as f:
+                        data = yaml.safe_load(f) or {}
                 elif os.path.isfile(json_path):
                     config_path = json_path
+                    with open(config_path, "r") as f:
+                        data = json.load(f) or {}
                 else:
                     # No config file found, remove from index if exists
                     delete_flag = False
@@ -257,6 +261,7 @@ class Index:
                     if delete_flag:
                         changed = True
                     continue
+
                 if current_item_keys is not None:
                     current_item_keys.add(config_path)
                 mtime = self.get_item_mtime(config_path)
@@ -266,6 +271,19 @@ class Index:
                 if old_mtime == mtime and not repos_changed:
                     continue
 
+                # Validate script meta against schema during indexing if meta
+                # changed or repos changed
+                if folder_type == "script":
+                    # logger.debug(f"-----Meta changed for {automation_path}, validating against schema... -----")
+                    errors, warnings = validate_meta(data, config_path)
+                    for e in errors:
+                        logger.error(f"Meta validation error: {e}")
+                    for w in warnings:
+                        logger.debug(f"Meta validation warning: {w}")
+                    if errors:
+                        raise ValueError(
+                            f"Meta validation failed for {config_path}. Fix the above error(s) and try again.")
+
                 self.modified_times[config_path] = {
                     "mtime": mtime,
                     "date_time": datetime.fromtimestamp(mtime).strftime("%Y-%m-%d %H:%M:%S")
@@ -400,17 +418,6 @@ class Index:
             tags = data.get("tags", [])
             alias = data.get("alias", None)
 
-            # Validate script meta against schema during indexing
-            if folder_type == "script":
-                errors, warnings = validate_meta(data, config_file)
-                for e in errors:
-                    logger.error(f"Meta validation error: {e}")
-                for w in warnings:
-                    logger.debug(f"Meta validation warning: {w}")
-                if errors:
-                    raise ValueError(
-                        f"Meta validation failed for {config_file}. Fix the above error(s) and try again.")
-
             # Remove stale entry for the same meta file path if exists
             self._delete_index_entries(folder_type, "path", folder_path)