idf-build-apps 2.11.1__tar.gz → 2.12.0__tar.gz

{idf_build_apps-2.11.1 → idf_build_apps-2.12.0}/.pre-commit-config.yaml

@@ -17,13 +17,13 @@ repos:
           - --use-current-year
         exclude: 'idf_build_apps/vendors/'
   - repo: https://github.com/astral-sh/ruff-pre-commit
-    rev: 'v0.11.10'
+    rev: 'v0.12.5'
     hooks:
       - id: ruff
         args: ['--fix']
       - id: ruff-format
   - repo: https://github.com/pre-commit/mirrors-mypy
-    rev: 'v1.15.0'
+    rev: 'v1.17.0'
     hooks:
       - id: mypy
         args: ['--warn-unused-ignores']

{idf_build_apps-2.11.1 → idf_build_apps-2.12.0}/CHANGELOG.md

@@ -2,6 +2,20 @@
 
 All notable changes to this project will be documented in this file.
 
+## v2.12.0 (2025-08-11)
+
+### Feat
+
+- recording build_disable_reason and test_disable_reason
+- support --archives, --archive-details ARCHIVE or --files args of idf.py size
+- support config `manifest_exclude_regexes`
+
+## v2.11.2 (2025-06-10)
+
+### Fix
+
+- missing `FolderRule` in .manifest
+
 ## v2.11.1 (2025-06-04)
 
 ### Fix

{idf_build_apps-2.11.1 → idf_build_apps-2.12.0}/CONTRIBUTING.md

@@ -6,18 +6,14 @@ Hi! We're glad that you're interested in contributing to `idf-build-apps`. This
 
 Here's a table shows the supported ESP-IDF versions and the corresponding Python versions.
 
-| ESP-IDF Version | ESP-IDF Supported Python Versions | idf-build-apps Releases |
-|-----------------|-----------------------------------|-------------------------|
-| 4.1             | 2.7, 3.4+                         | 1.x                     |
-| 4.2             | 2.7, 3.4+                         | 1.x                     |
-| 4.3             | 2.7, 3.4+                         | 1.x                     |
-| 4.4             | 2.7, 3.4+                         | 1.x                     |
-| 5.0             | 3.7+                              | main (2.x)              |
-| 5.1             | 3.7+                              | main (2.x)              |
-| 5.2             | 3.7+                              | main (2.x)              |
-| 5.3             | 3.8+                              | main (2.x)              |
-| 5.4             | 3.8+                              | main (2.x)              |
-| master (5.5)    | 3.9+                              | main (2.x)              |
+| ESP-IDF Version | ESP-IDF Version EOL | ESP-IDF Minimum Python Version | idf-build-apps Releases |
+|-----------------|---------------------|--------------------------------|-------------------------|
+| 5.1             | 2025.12.30          | 3.7                            | 2.x                     |
+| 5.2             | 2026.08.16          | 3.8                            | 2.x                     |
+| 5.3             | 2027.01.25          | 3.8                            | 2.x                     |
+| 5.4             | 2027.07.05          | 3.8                            | 2.x                     |
+| 5.5             | 2028.01.21          | 3.9                            | 2.x                     |
+| 6.0 (master)    | N/A                 | 3.10                           | 3.x WIP                 |
 
 `idf-build-apps` is following the semantic versioning. The major version of `idf-build-apps` is the same as the ESP-IDF version it supports. For example, `idf-build-apps` 1.x supports ESP-IDF 4.x, and `idf-build-apps` 2.x supports ESP-IDF 5.x.
 

{idf_build_apps-2.11.1 → idf_build_apps-2.12.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: idf-build-apps
-Version: 2.11.1
+Version: 2.12.0
 Summary: Tools for building ESP-IDF related apps.
 Author-email: Fu Hanxi <fuhanxi@espressif.com>
 Requires-Python: >=3.7

idf_build_apps-2.12.0/docs/en/explanations/build_status.rst

@@ -0,0 +1,122 @@
+##############
+ Build Status
+##############
+
+This page explains the different build statuses that apps can have during the build process and what each status means.
+
+Each app in `idf-build-apps` has a build status that indicates its current state in the build pipeline. Understanding these statuses is crucial for interpreting build results and troubleshooting issues.
+
+********************
+ Available Statuses
+********************
+
+The following build statuses are available:
+
+``unknown``
+===========
+
+The default status when an app is first discovered. The app's build status has not yet been determined.
+
+This status indicates that:
+
+-  The app has been found but not yet processed
+-  Further ``idf.py reconfigure`` is needed to determine if the build can proceed
+
+``disabled``
+============
+
+The app supports the target architecture, but has been disabled by manifest rules or configuration.
+
+This status indicates that:
+
+-  The app is compatible with the target but explicitly disabled
+-  Manifest rules contain ``disable`` clauses that match this app and target combination
+
+Apps with this status will not be built unless specifically included with ``--include-disabled-apps``.
+
+``skipped``
+===========
+
+The app supports the target and could potentially be built, but has been skipped due to dependency-driven build logic or dry run mode.
+
+This status indicates that:
+
+-  **Dependency-driven build**: The app doesn't depend on any of the modified components/files in the current build
+-  **Dry run mode**: The ``--dry-run`` flag was used, so no actual building occurs
+
+Apps with this status will not be built unless specifically included with ``--include-skipped-apps``.
+
+``should be built``
+===================
+
+The app should be built but hasn't been built yet.
+
+This status indicates that:
+
+-  The app has passed all checks and is ready for building
+-  Dependency analysis shows the app should be included in this build
+-  The app has not been built yet in the current run
+
+``build failed``
+================
+
+The app build process was attempted but failed.
+
+This status indicates that:
+
+-  The build process started but encountered an error
+-  Compilation, linking, or other build steps failed
+-  The failure reason is typically available in the build comment and build logs
+
+``build success``
+=================
+
+The app was successfully built.
+
+This status indicates that:
+
+-  All build steps completed without errors
+-  The app binary was generated successfully
+-  The build process finished normally
+
+************************
+ Status Transition Flow
+************************
+
+The typical flow of build statuses is:
+
+#. **unknown** → Initial state when app is discovered
+#. **unknown** → **disabled** (if app doesn't support target or is disabled by manifest)
+#. **unknown** → **skipped** (if dependency checks show app shouldn't be built)
+#. **unknown** → **should be built** (if app passes all checks)
+#. **should be built** → **build failed** (if build encounters errors)
+#. **should be built** → **build success** (if build completes successfully)
+
+.. note::
+
+   Apps in ``disabled`` or ``skipped`` status will not transition to ``should be built`` unless the conditions change or specific inclusion flags are used.
+
+**********************
+ Viewing Build Status
+**********************
+
+You can view the build status of apps in several ways:
+
+-  **JSON output**: Use ``--json-output`` to save detailed status information
+-  **Build logs**: Status and comments are included in build output
+-  **Summary reports**: Final status counts are shown at the end of builds
+
+**************************
+ Including Apps by Status
+**************************
+
+By default, only apps with ``unknown``, ``should be built``, ``build failed``, and ``build success`` statuses are processed. You can include additional apps using:
+
+-  ``--include-disabled-apps``: Include apps with ``disabled`` status
+-  ``--include-skipped-apps``: Include apps with ``skipped`` status
+
+This is useful for:
+
+-  Testing disabled apps during development
+-  Force-building apps that would normally be skipped
+-  Comprehensive testing regardless of dependency analysis

{idf_build_apps-2.11.1 → idf_build_apps-2.12.0}/docs/en/index.rst

@@ -11,6 +11,7 @@ This documentation is for idf-build-apps. idf-build-apps is a tool that allows d
    explanations/config_rules
    explanations/find
    explanations/build
+   explanations/build_status
    explanations/dependency_driven_build
 
 .. toctree::

{idf_build_apps-2.11.1 → idf_build_apps-2.12.0}/idf_build_apps/__init__.py

@@ -8,7 +8,7 @@ Tools for building ESP-IDF related apps.
 # ruff: noqa: E402
 # avoid circular imports
 
-__version__ = '2.11.1'
+__version__ = '2.12.0'
 
 from .session_args import (
     SessionArgs,

{idf_build_apps-2.11.1 → idf_build_apps-2.12.0}/idf_build_apps/app.py

@@ -95,6 +95,7 @@ class App(BaseModel):
 
     _build_log_filename: t.Optional[str] = None
     _size_json_filename: t.Optional[str] = None
+    size_json_extra_args: t.Optional[t.List[str]] = None
 
     dry_run: bool = False
     verbose: bool = False
@@ -105,15 +106,16 @@ class App(BaseModel):
     # build_apps() related
     index: t.Optional[int] = None
 
-    # build status related
     build_status: BuildStatus = BuildStatus.UNKNOWN
     build_comment: t.Optional[str] = None
+    test_comment: t.Optional[str] = None
 
     _build_duration: float = 0
     _build_timestamp: t.Optional[datetime] = None
 
     __EQ_IGNORE_FIELDS__ = [
         'build_comment',
+        'test_comment',
     ]
     __EQ_TUNE_FIELDS__ = {
         'app_dir': lambda x: (os.path.realpath(os.path.expanduser(x))),
@@ -324,8 +326,7 @@ class App(BaseModel):
         # put the expanded variable files in a temporary directory
         # will remove if the content is the same as the original one
         expanded_dir = os.path.join(self.work_dir, 'expanded_sdkconfig_files', os.path.basename(self.build_dir))
-        if not os.path.isdir(expanded_dir):
-            os.makedirs(expanded_dir)
+        os.makedirs(expanded_dir, exist_ok=True)
 
         for f in self.sdkconfig_defaults_candidates + ([self.sdkconfig_path] if self.sdkconfig_path else []):
             # use filepath if abs/rel already point to itself
@@ -338,57 +339,47 @@ class App(BaseModel):
                     continue
 
             expanded_fp = os.path.join(expanded_dir, os.path.basename(f))
-            with open(f) as fr:
-                with open(expanded_fp, 'w') as fw:
-                    for line in fr:
-                        line = os.path.expandvars(line)
-
-                        m = self.SDKCONFIG_LINE_REGEX.match(line)
-                        if m:
-                            key = m.group(1)
-                            if key == 'CONFIG_IDF_TARGET':
-                                sdkconfig_files_defined_target = m.group(2)
-
-                            if isinstance(self, CMakeApp):
-                                if key in self.SDKCONFIG_TEST_OPTS:
-                                    self.cmake_vars[key] = m.group(2)
-                                    continue
-
-                                if key in self.SDKCONFIG_IGNORE_OPTS:
-                                    continue
-
-                        fw.write(line)
-
-            with open(f) as fr:
-                with open(expanded_fp) as new_fr:
-                    if fr.read() == new_fr.read():
-                        LOGGER.debug('Use sdkconfig file %s', f)
-                        try:
-                            os.unlink(expanded_fp)
-                        except OSError:
-                            LOGGER.debug('Failed to remove file %s', expanded_fp)
-                        real_sdkconfig_files.append(f)
-                    else:
-                        LOGGER.debug('Expand sdkconfig file %s to %s', f, expanded_fp)
-                        real_sdkconfig_files.append(expanded_fp)
-                        # copy the related target-specific sdkconfig files
-                        par_dir = os.path.abspath(os.path.join(f, '..'))
-                        for target_specific_file in (
-                            os.path.join(par_dir, str(p))
-                            for p in Path(par_dir).glob(os.path.basename(f) + f'.{self.target}')
-                        ):
-                            LOGGER.debug(
-                                'Copy target-specific sdkconfig file %s to %s', target_specific_file, expanded_dir
-                            )
-                            shutil.copy(target_specific_file, expanded_dir)
-
-        # remove if expanded folder is empty
-        try:
-            os.rmdir(expanded_dir)
-        except OSError:
-            pass
+            with open(f) as fr, open(expanded_fp, 'w') as fw:
+                for line in fr:
+                    line = os.path.expandvars(line)
+
+                    m = self.SDKCONFIG_LINE_REGEX.match(line)
+                    if m:
+                        key, value = m.group(1), m.group(2)
+                        if key == 'CONFIG_IDF_TARGET':
+                            sdkconfig_files_defined_target = value
+
+                        if isinstance(self, CMakeApp):
+                            if key in self.SDKCONFIG_TEST_OPTS:
+                                self.cmake_vars[key] = value
+                                continue
+                            if key in self.SDKCONFIG_IGNORE_OPTS:
+                                continue
+
+                    fw.write(line)
+
+            with open(f) as fr, open(expanded_fp) as new_fr:
+                if fr.read() == new_fr.read():
+                    LOGGER.debug('Use sdkconfig file %s', f)
+                    try:
+                        os.unlink(expanded_fp)
+                    except OSError:
+                        LOGGER.debug('Failed to remove file %s', expanded_fp)
+                    real_sdkconfig_files.append(f)
+                else:
+                    LOGGER.debug('Expand sdkconfig file %s to %s', f, expanded_fp)
+                    real_sdkconfig_files.append(expanded_fp)
+                    # copy the related target-specific sdkconfig files
+                    par_dir = os.path.abspath(os.path.join(f, '..'))
+                    for target_specific_file in (
+                        os.path.join(par_dir, str(p))
+                        for p in Path(par_dir).glob(os.path.basename(f) + f'.{self.target}')
+                    ):
+                        LOGGER.debug('Copy target-specific sdkconfig file %s to %s', target_specific_file, expanded_dir)
+                        shutil.copy(target_specific_file, expanded_dir)
 
         try:
+            os.rmdir(expanded_dir)
             os.rmdir(os.path.join(self.work_dir, 'expanded_sdkconfig_files'))
         except OSError:
             pass
@@ -428,14 +419,14 @@ class App(BaseModel):
 
     @property
     def supported_targets(self) -> t.List[str]:
+        if self.sdkconfig_files_defined_idf_target:
+            return [self.sdkconfig_files_defined_idf_target]
+
         if self.MANIFEST:
             return self.MANIFEST.enable_build_targets(
                 self.app_dir, self.sdkconfig_files_defined_idf_target, self.config_name
             )
 
-        if self.sdkconfig_files_defined_idf_target:
-            return [self.sdkconfig_files_defined_idf_target]
-
         return DEFAULT_BUILD_TARGETS.get()
 
     @property
@@ -656,11 +647,10 @@ class App(BaseModel):
                 [
                     sys.executable,
                     str(IDF_SIZE_PY),
-                ]
-                + (['--json'] if IDF_VERSION < Version('5.1') else ['--format', 'json'])
-                + [
+                    *(['--json'] if IDF_VERSION < Version('5.1') else ['--format', 'json']),
                     '-o',
                     self.size_json_path,
+                    *(self.size_json_extra_args or []),
                     map_file,
                 ],
                 check=True,
@@ -668,14 +658,13 @@ class App(BaseModel):
         else:
             with open(self.size_json_path, 'w') as fw:
                 subprocess_run(
-                    (
-                        [
-                            sys.executable,
-                            str(IDF_SIZE_PY),
-                            '--json',
-                            map_file,
-                        ]
-                    ),
+                    [
+                        sys.executable,
+                        str(IDF_SIZE_PY),
+                        '--json',
+                        *(self.size_json_extra_args or []),
+                        map_file,
+                    ],
                     log_terminal=False,
                     log_fs=fw,
                     check=True,
@@ -735,9 +724,35 @@ class App(BaseModel):
         modified_components: t.Optional[t.List[str]] = None,
         modified_files: t.Optional[t.List[str]] = None,
     ) -> None:
+        """Check if this app should be built based on the modified files and components."""
         if self.build_status != BuildStatus.UNKNOWN:
             return
 
+        if self.target not in self.supported_targets:
+            # default error message
+            self.build_comment = (
+                f'Target {self.target} not in default build targets {",".join(DEFAULT_BUILD_TARGETS.get())}'
+            )
+
+            if self.MANIFEST:
+                rule = self.MANIFEST.most_suitable_rule(self.app_dir)
+
+                # disable > enable
+                for clause in rule.disable:
+                    if clause.get_value(self.target, self.config_name or ''):
+                        self.build_comment = f'Disabled by manifest rule: {clause}'
+                        break
+                else:
+                    # Check if it's not enabled by manifest rules
+                    if rule.enable:
+                        # Has enable rules but target not in enabled targets
+                        self.build_comment = 'Not enabled by manifest rules:\n'
+                        self.build_comment += '\n'.join(f'- {clause}' for clause in rule.enable)
+
+            self.build_status = BuildStatus.DISABLED
+            self._checked_should_build = True
+            return
+
         if not check_app_dependencies:
             self.build_status = BuildStatus.SHOULD_BE_BUILT
             self._checked_should_build = True
@@ -804,6 +819,35 @@ class App(BaseModel):
         self.build_comment = 'current build does not modify any components or files required by this app'
         self._checked_should_build = True
 
+    def check_should_test(self) -> None:
+        """Check if testing is disabled for this app and set test_disable_reason."""
+        if not self.MANIFEST:
+            return
+
+        rule = self.MANIFEST.most_suitable_rule(self.app_dir)
+
+        # Check if testing is enabled for this target
+        if self.target not in self.verified_targets:
+            # default error message
+            self.test_comment = f'Target {self.target} not in verified targets {",".join(self.verified_targets)}'
+
+            # disable_test > disable > enable
+            for clause in rule.disable_test:
+                if clause.get_value(self.target, self.config_name or ''):
+                    self.test_comment = f'Disabled by manifest rule: {clause}'
+                    return
+
+            # Check if disabled by general disable rules
+            for clause in rule.disable:
+                if clause.get_value(self.target, self.config_name or ''):
+                    self.test_comment = f'Disabled by manifest rule: {clause}'
+                    return
+
+            # If not explicitly disabled but not in enabled targets
+            if rule.enable:
+                self.test_comment = 'Not enabled by manifest rules:\n'
+                self.test_comment += '\n'.join(f'- {clause}' for clause in rule.enable)
+
 
 class MakeApp(App):
     MAKE_PROJECT_LINE: t.ClassVar[str] = r'include $(IDF_PATH)/make/project.mk'

{idf_build_apps-2.11.1 → idf_build_apps-2.12.0}/idf_build_apps/args.py

@@ -240,14 +240,24 @@ class DependencyDrivenBuildArguments(GlobalArguments):
             validate_method=[ValidateMethod.TO_LIST],
             nargs='+',
         ),
-        description='space-separated list of file patterns to search for the manifest files. '
+        description='space-separated list of file glob patterns to search for the manifest files. '
         'The matched files will be loaded as the manifest files.',
         default=None,  # type: ignore
     )
+    manifest_exclude_regexes: t.Optional[t.List[str]] = field(
+        FieldMetadata(
+            validate_method=[ValidateMethod.TO_LIST],
+            nargs='+',
+        ),
+        description='space-separated list of regex to exclude when searching for manifest files. '
+        'Files matching these patterns will be ignored. '
+        'By default excludes files under "managed_components" directories.',
+        default=['/managed_components/'],  # type: ignore
+    )
     manifest_rootpath: str = field(
         None,
         description='Root path to resolve the relative paths defined in the manifest files. '
-        'By default set to the current directory. Support environment variables.',
+        'By default set to the current directory.',
         default=os.curdir,  # type: ignore
     )
     modified_components: t.Optional[t.List[str]] = field(
@@ -337,6 +347,22 @@ class DependencyDrivenBuildArguments(GlobalArguments):
             for pat in [to_absolute_path(p, self.manifest_rootpath) for p in self.manifest_filepatterns]:
                 matched_paths.update(glob.glob(str(pat), recursive=True))
 
+            exclude_regexes = {re.compile(regex) for regex in self.manifest_exclude_regexes or []}
+
+            # Filter out files matching excluded patterns
+            if matched_paths:
+                filtered_paths = set()
+                for path in matched_paths:
+                    posix_path = Path(path).as_posix()
+                    for regex in exclude_regexes:
+                        if regex.search(posix_path):
+                            LOGGER.debug(f'Excluding manifest file {path} due to excluded regex match')
+                            break
+                    else:
+                        filtered_paths.add(path)
+
+                matched_paths = filtered_paths
+
             if matched_paths:
                 if self.manifest_files:
                     self.manifest_files.extend(matched_paths)
@@ -482,6 +508,13 @@ class FindBuildArguments(DependencyDrivenBuildArguments):
         validation_alias=AliasChoices('size_json_filename', 'size_file'),
         default=None,  # type: ignore
     )
+    size_json_extra_args: t.Optional[t.List[str]] = field(
+        FieldMetadata(
+            validate_method=[ValidateMethod.TO_LIST],
+        ),
+        description='Additional arguments to pass to esp_idf_size tool',
+        default=None,  # type: ignore
+    )
     config_rules: t.Optional[t.List[str]] = field(
         FieldMetadata(
             validate_method=[ValidateMethod.TO_LIST],
@@ -607,7 +640,7 @@ class FindBuildArguments(DependencyDrivenBuildArguments):
         elif self.enable_preview_targets:
             self.default_build_targets = deepcopy(ALL_TARGETS)
             LOGGER.info('Overriding default build targets to %s', self.default_build_targets)
-            DEFAULT_BUILD_TARGETS.set(self.default_build_targets)  # type: ignore
+            DEFAULT_BUILD_TARGETS.set(self.default_build_targets)
 
         if self.disable_targets and DEFAULT_BUILD_TARGETS.get():
             LOGGER.info('Disable targets: %s', self.disable_targets)
@@ -791,7 +824,7 @@ class BuildArguments(FindBuildArguments):
             hidden=True,
         ),
         description='Record size json filepath of the built apps to the specified file. '
-        'Each line is a json string. Can expand placeholders @p. Support environment variables.',
+        'Each line is a json string. Can expand placeholders @p.',
         validation_alias=AliasChoices('collect_size_info_filename', 'collect_size_info'),
         default=None,  # type: ignore
         exclude=True,  # computed field is used
@@ -802,7 +835,7 @@ class BuildArguments(FindBuildArguments):
             hidden=True,
         ),
         description='Record serialized app model of the built apps to the specified file. '
-        'Each line is a json string. Can expand placeholders @p. Support environment variables.',
+        'Each line is a json string. Can expand placeholders @p.',
         validation_alias=AliasChoices('collect_app_info_filename', 'collect_app_info'),
         default=None,  # type: ignore
         exclude=True,  # computed field is used
@@ -812,8 +845,7 @@ class BuildArguments(FindBuildArguments):
             deprecates={'junitxml': {}},
             hidden=True,
         ),
-        description='Path to the junitxml file to record the build results. Can expand placeholder @p. '
-        'Support environment variables.',
+        description='Path to the junitxml file to record the build results. Can expand placeholder @p.',
         validation_alias=AliasChoices('junitxml_filename', 'junitxml'),
         default=None,  # type: ignore
         exclude=True,  # computed field is used