updates2mqtt 1.7.3__tar.gz → 1.8.1__tar.gz

{updates2mqtt-1.7.3 → updates2mqtt-1.8.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: updates2mqtt
-Version: 1.7.3
+Version: 1.8.1
 Summary: System update and docker image notification and execution over MQTT
 Keywords: mqtt,docker,oci,container,updates,automation,home-assistant,homeassistant,selfhosting
 Author: jey burrows
@@ -72,12 +72,12 @@ Read the release notes, and optionally click *Update* to trigger a Docker *pull*
 
 Updates2MQTT perioidically checks for new versions of components being available, and publishes new version info to MQTT. HomeAssistant auto discovery is supported, so all updates can be seen in the same place as Home Assistant's own components and add-ins.
 
-Currently only Docker containers are supported, either via an image registry check (using either v1 Docker APIs or the OCI v2 API), or a git repo for source (see [Local Builds](local_builds.md)), with specific handling for Docker, Github Container Registry, Gitlab, Codeberg, Microsoft Container Registry and LinuxServer Registry, with adaptive behaviour to cope with most
+Currently only Docker containers are supported, either via an image registry check (using either v1 Docker APIs or the OCI v2 API), or a git repo for source (see [Local Builds](local_builds.md)), with specific handling for Docker, Github Container Registry, Gitlab, Codeberg, Microsoft Container Registry, Quay and LinuxServer Registry, with adaptive behaviour to cope with most
 others.  The design is modular, so other update sources can be added, at least for notification. The next anticipated is **apt** for Debian based systems.
 
 Components can also be updated, either automatically or triggered via MQTT, for example by hitting the *Install* button in the HomeAssistant update dialog. Icons and release notes can be specified for a better HA experience. See [Home Assistant Integration](home_assistant.md) for details.
 
-To get started, read the [Installation](installation.md) and [Configuration](configuration.md) pages.
+To get started, read the [Installation](installation.md) and [Configuration](configuration/index.md) pages.
 
 For a quick spin, try this:
 
@@ -91,6 +91,9 @@ or without Docker, using [uv](https://docs.astral.sh/uv/)
 export MQTT_HOST=192.168.1.1;export MQTT_USER=user1;export MQTT_PASS=user1;uv run --with updates2mqtt python -m updates2mqtt
 ```
 
+It also comes with a basic command line tool that will perform the analysis for a single running container, or fetch
+manifests, JSON blobs and lists of tags from remote registries (known to work with GitHub, GitLab, Codeberg, Quay, LSCR and Microsoft MCR).
+
 ## Release Support
 
 Presently only Docker containers are supported, although others are planned, probably with priority for `apt`.
@@ -128,7 +131,7 @@ restarter:
 While `updates2mqtt` will discover and monitor all containers running under the Docker daemon,
 there are some options to make to those containers to tune how it works.
 
-These happen by adding environment variables to the containers, typically inside an `.env`
+These happen by adding environment variables or docker labels to the containers, typically inside an `.env`
 file, or as `environment` options inside `docker-compose.yaml`.
 
 ### Automated updates
@@ -148,45 +151,6 @@ restarter:
 Automated updates can also apply to local builds, where a `git_repo_path` has been defined - if there are remote
 commits available to pull, then a `git pull`, `docker compose build` and `docker compose up` will be executed.
 
-### Environment Variables
-
-The following environment variables can be used to configure containers for `updates2mqtt`:
-
-| Env Var                    | Description                                                                                  | Default         |
-|----------------------------|----------------------------------------------------------------------------------------------|-----------------|
-| `UPD2MQTT_UPDATE`          | Update mode, either `Passive` or `Auto`. If `Auto`, updates will be installed automatically. | `Passive`       |
-| `UPD2MQTT_PICTURE`         | URL to an icon to use in Home Assistant.                                                     | Docker logo URL |
-| `UPD2MQTT_RELNOTES`        | URL to release notes for the package.                                                        |                 |
-| `UPD2MQTT_GIT_REPO_PATH`   | Relative path to a local git repo if the image is built locally.                             |                 |
-| `UPD2MQTT_IGNORE`          | If set to `True`, the container will be ignored by Updates2MQTT.                             | False           |
-                        |                 |
-| `UPD2MQTT_VERSION_POLICY` | Change how version derived from container label or image hash, `Version`,`Digest`,`Version_Digest` with default of `Auto`|
-| `UPD2MQTT_REGISTRY_TOKEN` | Access token for authentication to container distribution API, as alternative to making a call to `token` service |
-
-### Docker Labels
-
-Alternatively, use Docker labels
-
-| Label                          | Env Var                    |
-|--------------------------------|----------------------------|
-| `updates2mqtt.update`          | `UPD2MQTT_UPDATE`          |
-| `updates2mqtt.picture`         | `UPD2MQTT_PCITURE`         |
-| `updates2mqtt.relnotes`        | `UPD2MQTT_RELNOTES`        |
-| `updates2mqtt.git_repo_path`   | `UPD2MQTT_GIT_REPO_PATH`   |
-| `updates2mqtt.ignore`          | `UPD2MQTT_IGNORE`          |
-| `updates2mqtt.version_policy`  | `UPD2MQTT_VERSION_POLICY`  |
-| `updates2mqtt.registry_token`  | `UPD2MQTT_REGISTRY_TOKEN`  |
-
-
-
-```yaml title="Example Compose Snippet"
-restarter:
-    image: docker:cli
-    command: ["/bin/sh", "-c", "while true; do sleep 86400; docker restart mailserver; done"]
-    labels:
-        updates2mqtt.relnotes: https://component.my.com/release_notes
-```
-
 
 ## Related Projects
 
@@ -196,7 +160,7 @@ Other apps useful for self-hosting with the help of MQTT:
 
 Find more at [awesome-mqtt](https://github.com/rhizomatics/awesome-mqtt)
 
-For a more powerful Docker update manager, try [What's Up Docker](https://getwud.github.io/wud/)
+For a more powerful Docker focussed update manager, try [What's Up Docker](https://getwud.github.io/wud/)
 
 ## Development
 

{updates2mqtt-1.7.3 → updates2mqtt-1.8.1}/README.md

@@ -34,12 +34,12 @@ Read the release notes, and optionally click *Update* to trigger a Docker *pull*
 
 Updates2MQTT perioidically checks for new versions of components being available, and publishes new version info to MQTT. HomeAssistant auto discovery is supported, so all updates can be seen in the same place as Home Assistant's own components and add-ins.
 
-Currently only Docker containers are supported, either via an image registry check (using either v1 Docker APIs or the OCI v2 API), or a git repo for source (see [Local Builds](local_builds.md)), with specific handling for Docker, Github Container Registry, Gitlab, Codeberg, Microsoft Container Registry and LinuxServer Registry, with adaptive behaviour to cope with most
+Currently only Docker containers are supported, either via an image registry check (using either v1 Docker APIs or the OCI v2 API), or a git repo for source (see [Local Builds](local_builds.md)), with specific handling for Docker, Github Container Registry, Gitlab, Codeberg, Microsoft Container Registry, Quay and LinuxServer Registry, with adaptive behaviour to cope with most
 others.  The design is modular, so other update sources can be added, at least for notification. The next anticipated is **apt** for Debian based systems.
 
 Components can also be updated, either automatically or triggered via MQTT, for example by hitting the *Install* button in the HomeAssistant update dialog. Icons and release notes can be specified for a better HA experience. See [Home Assistant Integration](home_assistant.md) for details.
 
-To get started, read the [Installation](installation.md) and [Configuration](configuration.md) pages.
+To get started, read the [Installation](installation.md) and [Configuration](configuration/index.md) pages.
 
 For a quick spin, try this:
 
@@ -53,6 +53,9 @@ or without Docker, using [uv](https://docs.astral.sh/uv/)
 export MQTT_HOST=192.168.1.1;export MQTT_USER=user1;export MQTT_PASS=user1;uv run --with updates2mqtt python -m updates2mqtt
 ```
 
+It also comes with a basic command line tool that will perform the analysis for a single running container, or fetch
+manifests, JSON blobs and lists of tags from remote registries (known to work with GitHub, GitLab, Codeberg, Quay, LSCR and Microsoft MCR).
+
 ## Release Support
 
 Presently only Docker containers are supported, although others are planned, probably with priority for `apt`.
@@ -90,7 +93,7 @@ restarter:
 While `updates2mqtt` will discover and monitor all containers running under the Docker daemon,
 there are some options to make to those containers to tune how it works.
 
-These happen by adding environment variables to the containers, typically inside an `.env`
+These happen by adding environment variables or docker labels to the containers, typically inside an `.env`
 file, or as `environment` options inside `docker-compose.yaml`.
 
 ### Automated updates
@@ -110,45 +113,6 @@ restarter:
 Automated updates can also apply to local builds, where a `git_repo_path` has been defined - if there are remote
 commits available to pull, then a `git pull`, `docker compose build` and `docker compose up` will be executed.
 
-### Environment Variables
-
-The following environment variables can be used to configure containers for `updates2mqtt`:
-
-| Env Var                    | Description                                                                                  | Default         |
-|----------------------------|----------------------------------------------------------------------------------------------|-----------------|
-| `UPD2MQTT_UPDATE`          | Update mode, either `Passive` or `Auto`. If `Auto`, updates will be installed automatically. | `Passive`       |
-| `UPD2MQTT_PICTURE`         | URL to an icon to use in Home Assistant.                                                     | Docker logo URL |
-| `UPD2MQTT_RELNOTES`        | URL to release notes for the package.                                                        |                 |
-| `UPD2MQTT_GIT_REPO_PATH`   | Relative path to a local git repo if the image is built locally.                             |                 |
-| `UPD2MQTT_IGNORE`          | If set to `True`, the container will be ignored by Updates2MQTT.                             | False           |
-                        |                 |
-| `UPD2MQTT_VERSION_POLICY` | Change how version derived from container label or image hash, `Version`,`Digest`,`Version_Digest` with default of `Auto`|
-| `UPD2MQTT_REGISTRY_TOKEN` | Access token for authentication to container distribution API, as alternative to making a call to `token` service |
-
-### Docker Labels
-
-Alternatively, use Docker labels
-
-| Label                          | Env Var                    |
-|--------------------------------|----------------------------|
-| `updates2mqtt.update`          | `UPD2MQTT_UPDATE`          |
-| `updates2mqtt.picture`         | `UPD2MQTT_PCITURE`         |
-| `updates2mqtt.relnotes`        | `UPD2MQTT_RELNOTES`        |
-| `updates2mqtt.git_repo_path`   | `UPD2MQTT_GIT_REPO_PATH`   |
-| `updates2mqtt.ignore`          | `UPD2MQTT_IGNORE`          |
-| `updates2mqtt.version_policy`  | `UPD2MQTT_VERSION_POLICY`  |
-| `updates2mqtt.registry_token`  | `UPD2MQTT_REGISTRY_TOKEN`  |
-
-
-
-```yaml title="Example Compose Snippet"
-restarter:
-    image: docker:cli
-    command: ["/bin/sh", "-c", "while true; do sleep 86400; docker restart mailserver; done"]
-    labels:
-        updates2mqtt.relnotes: https://component.my.com/release_notes
-```
-
 
 ## Related Projects
 
@@ -158,7 +122,7 @@ Other apps useful for self-hosting with the help of MQTT:
 
 Find more at [awesome-mqtt](https://github.com/rhizomatics/awesome-mqtt)
 
-For a more powerful Docker update manager, try [What's Up Docker](https://getwud.github.io/wud/)
+For a more powerful Docker focussed update manager, try [What's Up Docker](https://getwud.github.io/wud/)
 
 ## Development
 

{updates2mqtt-1.7.3 → updates2mqtt-1.8.1}/pyproject.toml

@@ -7,7 +7,7 @@ authors = [
 ]
 
 requires-python = ">=3.13"
-version = "1.7.3"
+version = "1.8.1"
 license="Apache-2.0"
 keywords=["mqtt", "docker", "oci","container","updates", "automation","home-assistant","homeassistant","selfhosting"]
 
@@ -60,7 +60,8 @@ dev = [
     "pytest-subprocess>=1.5.3",
     "coverage",
     "icdiff",
-    "genbadge[all]"
+    "genbadge[all]",
+    "actionlint-py"
 ]
 docs=[
     "mkdocs",
@@ -69,12 +70,11 @@ docs=[
     "mkdocs-autorefs",
     "mkdocs-pagetree-plugin",
     "mkdocs-coverage",
-    "mkdocstrings[python]",
     "pymdown-extensions",
-    "mkdocs-git-revision-date-localized-plugin",
     "mkdocs-meta-descriptions-plugin",
     "pngquant",
-    "mkdocs-mermaid2-plugin"
+    "mkdocs-mermaid2-plugin",
+    "mkdocs-htmlproofer-plugin"
 ]
 
 [build-system]

{updates2mqtt-1.7.3 → updates2mqtt-1.8.1}/src/updates2mqtt/app.py

@@ -21,7 +21,6 @@ from .mqtt import MqttPublisher
 log = structlog.get_logger()
 
 CONF_FILE = Path("conf/config.yaml")
-PKG_INFO_FILE = Path("./common_packages.yaml")
 UPDATE_INTERVAL = 60 * 60 * 4
 
 # #TODO:
@@ -56,7 +55,15 @@ class App:
         self.scan_count: int = 0
         self.last_scan: str | None = None
         if self.cfg.docker.enabled:
-            self.scanners.append(DockerProvider(self.cfg.docker, self.cfg.node, self.self_bounce))
+            self.scanners.append(
+                DockerProvider(
+                    self.cfg.docker,
+                    self.cfg.node,
+                    packages=self.cfg.packages,
+                    github_cfg=self.cfg.github,
+                    self_bounce=self.self_bounce,
+                )
+            )
         self.stopped = Event()
         self.healthcheck_topic = self.cfg.node.healthcheck.topic_template.format(node_name=self.cfg.node.name)
 
@@ -71,9 +78,6 @@ class App:
         session = uuid.uuid4().hex
         for scanner in self.scanners:
             slog = log.bind(source_type=scanner.source_type, session=session)
-            slog.info("Cleaning topics before scan")
-            if self.scan_count == 0:
-                await self.publisher.clean_topics(scanner, None, force=True)
             if self.stopped.is_set():
                 break
             slog.info("Scanning ...")
@@ -84,7 +88,7 @@ class App:
             if self.stopped.is_set():
                 slog.debug("Breaking scan loop on stopped event")
                 break
-            await self.publisher.clean_topics(scanner, session, force=False)
+            await self.publisher.clean_topics(scanner)
             self.scan_count += 1
             slog.info(f"Scan #{self.scan_count} complete")
         self.last_scan_timestamp = datetime.now(UTC).isoformat()

{updates2mqtt-1.7.3 → updates2mqtt-1.8.1}/src/updates2mqtt/cli.py

@@ -4,7 +4,7 @@ import structlog
 from omegaconf import DictConfig, OmegaConf
 from rich import print_json
 
-from updates2mqtt.config import DockerConfig, NodeConfig, RegistryConfig
+from updates2mqtt.config import DockerConfig, GitHubConfig, NodeConfig, RegistryConfig
 from updates2mqtt.helpers import Throttler
 from updates2mqtt.integrations.docker import DockerProvider
 from updates2mqtt.integrations.docker_enrich import (
@@ -24,18 +24,20 @@ log = structlog.get_logger()
 """
 Super simple CLI
 
-python updates2mqtt.cli container=frigate
+Command can be `container`,`tags`,`manifest` or `blob`
 
-python updates2mqtt.cli container=frigate api=docker_client log_level=DEBUG
+* `container=container-name`
+* `container=hash`
+* `tags=ghcr.io/
+* `blob=mcr.microsoft.com/dotnet/sdk:latest`
+* `tags=quay.io/linuxserver.io/babybuddy`
+* `blob=ghcr.io/blakeblackshear/frigate@sha256:759c36ee869e3e60258350a2e221eae1a4ba1018613e0334f1bc84eb09c4bbbc`
 
-ython3 updates2mqtt/cli.py blob=ghcr.io/homarr-labs/homarr@sha256:af79a3339de5ed8ef7f5a0186ff3deb86f40b213ba75249291f2f68aef082a25 | jq '.config.Labels'
+In addition, a `log_level=DEBUG` or other level can be added, `github_token` to try a personal access
+token for GitHub release info retrieval, or `api=docker_client` to use the older API (defaults to `api=OCI_V2`)
 
-python3 updates2mqtt/cli.py manifest=ghcr.io/blakeblackshear/frigate:stable
 
-python3 updates2mqtt/cli.py blob=ghcr.io/blakeblackshear/frigate@sha256:ef8d56a7d50b545af176e950ce328aec7f0b7bc5baebdca189fe661d97924980
-
-python3 updates2mqtt/cli.py manifest=ghcr.io/blakeblackshear/frigate@sha256:c68fd78fd3237c9ba81b5aa927f17b54f46705990f43b4b5d5596cfbbb626af4
-"""  # noqa: E501
+"""
 
 OCI_MANIFEST_TYPES: list[str] = [
     "application/vnd.oci.image.manifest.v1+json",
@@ -91,7 +93,9 @@ ALL_OCI_MEDIA_TYPES: list[str] = (
 )
 
 
-def dump_url(doc_type: str, img_ref: str) -> None:
+def dump_url(doc_type: str, img_ref: str, cli_conf: DictConfig) -> None:
+    structlog.configure(wrapper_class=structlog.make_filtering_bound_logger(cli_conf.get("log_level", "WARNING")))
+
     lookup = ContainerDistributionAPIVersionLookup(Throttler(), RegistryConfig())
     img_info = DockerImageInfo(img_ref)
     if not img_info.index_name or not img_info.name:
@@ -110,35 +114,49 @@ def dump_url(doc_type: str, img_ref: str) -> None:
             log.warning("No tag or digest found in %s", img_ref)
             return
         url = f"https://{api_host}/v2/{img_info.name}/manifests/{img_info.tag_or_digest}"
+    elif doc_type == "tags":
+        url = f"https://{api_host}/v2/{img_info.name}/tags/list"
     else:
         return
 
     token: str | None = lookup.fetch_token(img_info.index_name, img_info.name)
 
     response: Response | None = fetch_url(url, bearer_token=token, follow_redirects=True, response_type=ALL_OCI_MEDIA_TYPES)
-    if response:
+    if response and response.is_error:
+        log.warning(f"{response.status_code}: {url}")
+        log.warning(response.text)
+    elif response and response.is_success:
         log.debug(f"{response.status_code}: {url}")
         log.debug("HEADERS")
         for k, v in response.headers.items():
             log.debug(f"{k}: {v}")
         log.debug("CONTENTS")
+
         print_json(response.text)
 
 
 def main() -> None:
     # will be a proper cli someday
     cli_conf: DictConfig = OmegaConf.from_cli()
-    structlog.configure(wrapper_class=structlog.make_filtering_bound_logger(cli_conf.get("log_level", "WARNING")))
 
     if cli_conf.get("blob"):
-        dump_url("blob", cli_conf.get("blob"))
+        dump_url("blob", cli_conf.get("blob"), cli_conf)
     elif cli_conf.get("manifest"):
-        dump_url("manifest", cli_conf.get("manifest"))
+        dump_url("manifest", cli_conf.get("manifest"), cli_conf)
+    elif cli_conf.get("tags"):
+        dump_url("tags", cli_conf.get("tags"), cli_conf)
 
     else:
+        structlog.configure(wrapper_class=structlog.make_filtering_bound_logger(cli_conf.get("log_level", "INFO")))
+
         docker_scanner = DockerProvider(
-            DockerConfig(registry=RegistryConfig(api=cli_conf.get("api", "OCI_V2"))), NodeConfig(), None
+            DockerConfig(registry=RegistryConfig(api=cli_conf.get("api", "OCI_V2"))),
+            NodeConfig(),
+            packages={},
+            github_cfg=GitHubConfig(access_token=cli_conf.get("github_token")),
+            self_bounce=None,
         )
+        docker_scanner.initialize()
         discovery: Discovery | None = docker_scanner.rescan(
             Discovery(docker_scanner, cli_conf.get("container", "frigate"), "cli", "manual")
         )

{updates2mqtt-1.7.3 → updates2mqtt-1.8.1}/src/updates2mqtt/config.py

@@ -67,6 +67,11 @@ class MqttConfig:
     protocol: str = "${oc.env:MQTT_VERSION,3.11}"
 
 
+@dataclass
+class GitHubConfig:
+    access_token: str | None = None
+
+
 @dataclass
 class MetadataSourceConfig:
     enabled: bool = True
@@ -84,6 +89,21 @@ class VersionPolicy(StrEnum):
     VERSION = "VERSION"
     DIGEST = "DIGEST"
     VERSION_DIGEST = "VERSION_DIGEST"
+    TIMESTAMP = "TIMESTAMP"
+
+
+@dataclass
+class DockerPackageUpdateInfo:
+    image_name: str = MISSING  # untagged image ref
+    version_policy: VersionPolicy = VersionPolicy.AUTO
+
+
+@dataclass
+class PackageUpdateInfo:
+    docker: DockerPackageUpdateInfo | None = field(default_factory=DockerPackageUpdateInfo)
+    logo_url: str | None = None
+    release_notes_url: str | None = None
+    source_repo_url: str | None = None
 
 
 @dataclass
@@ -149,25 +169,14 @@ class Config:
     mqtt: MqttConfig = field(default_factory=MqttConfig)  # pyright: ignore[reportArgumentType, reportCallIssue]
     homeassistant: HomeAssistantConfig = field(default_factory=HomeAssistantConfig)
     docker: DockerConfig = field(default_factory=DockerConfig)
+    github: GitHubConfig = field(default_factory=GitHubConfig)
     scan_interval: int = 60 * 60 * 3
+    packages: dict[str, PackageUpdateInfo] = field(default_factory=dict)
 
 
 @dataclass
-class DockerPackageUpdateInfo:
-    image_name: str = MISSING  # untagged image ref
-
-
-@dataclass
-class PackageUpdateInfo:
-    docker: DockerPackageUpdateInfo | None = field(default_factory=DockerPackageUpdateInfo)
-    logo_url: str | None = None
-    release_notes_url: str | None = None
-    source_repo_url: str | None = None
-
-
-@dataclass
-class UpdateInfoConfig:
-    common_packages: dict[str, PackageUpdateInfo] = field(default_factory=lambda: {})
+class CommonPackages:
+    common_packages: dict[str, PackageUpdateInfo] = field(default_factory=dict)
 
 
 class IncompleteConfigException(BaseException):
@@ -188,13 +197,13 @@ def load_app_config(conf_file_path: Path, return_invalid: bool = False) -> Confi
             try:
                 log.debug(f"Creating config directory {conf_file_path.parent} if not already present")
                 conf_file_path.parent.mkdir(parents=True, exist_ok=True)
-            except Exception:
-                log.warning("Unable to create config directory", path=conf_file_path.parent)
+            except Exception as e:
+                log.warning("Unable to create config directory: %s", e, path=conf_file_path.parent)
         try:
             conf_file_path.write_text(OmegaConf.to_yaml(base_cfg))
             log.info(f"Auto-generated a new config file at {conf_file_path}")
-        except Exception:
-            log.warning("Unable to write config file", path=conf_file_path)
+        except Exception as e:
+            log.warning("Unable to write config file: %s", e, path=conf_file_path)
         cfg = base_cfg
     else:
         cfg = base_cfg

{updates2mqtt-1.7.3 → updates2mqtt-1.8.1}/src/updates2mqtt/helpers.py

@@ -145,8 +145,8 @@ class APIStats:
         """Log line friendly string summary"""
         return (
             f"fetches: {self.fetches}, cache ratio: {self.hit_ratio():.2%}, revalidated: {self.revalidated}, "
-            + f"errors: {', '.join(f'{status_code}:{fails}' for status_code, fails in self.failed.items())}, "
-            + f"oldest cache hit: {self.max_cache_age:.2f}, avg elapsed: {self.average_elapsed()}"
+            + f"errors: {', '.join(f'{status_code}:{fails}' for status_code, fails in self.failed.items()) or '0'}, "
+            + f"oldest cache hit: {self.max_cache_age:.2f}s, avg elapsed: {self.average_elapsed()}s"
         )
 
 
@@ -196,8 +196,9 @@ def fetch_url(
                 allow_stale=allow_stale,
             )
         )
+        log_headers: list[tuple[str, str]] = [h for h in headers if len(h) > 1 and h[0] != "Authorization"]
         with SyncCacheClient(headers=headers, follow_redirects=follow_redirects, policy=cache_policy) as client:
-            log.debug(f"Fetching URL {url}, redirects={follow_redirects}, headers={headers}, cache_ttl={cache_ttl}")
+            log.debug(f"Fetching URL {url}, redirects={follow_redirects}, headers={log_headers}, cache_ttl={cache_ttl}")
             response: Response = client.request(method=method, url=url, extensions={"hishel_ttl": cache_ttl})
             cache_metadata: CacheMetadata = CacheMetadata(response)
             if not response.is_success:
@@ -221,6 +222,45 @@ def fetch_url(
     return None
 
 
-def validate_url(url: str, cache_ttl: int = 300) -> bool:
+def validate_url(url: str, cache_ttl: int = 1500) -> bool:
     response: Response | None = fetch_url(url, method="HEAD", cache_ttl=cache_ttl, follow_redirects=True)
     return response is not None and response.status_code != 404
+
+
+def sanitize_name(name: str, replacement: str = "_", max_len: int = 64) -> str:
+    """Strict sanitization that removes/replaces common problematic characters for MQTT or HA
+
+    - Replaces spaces with underscores
+    - Removes control characters
+    - Ensures alphanumeric safety for broader compatibility
+
+    Args:
+        name: The topic component string to sanitize
+        replacement: Character to replace invalid characters with (default: "_")
+        max_len: Largest acceptable name size
+
+    Returns:
+        Sanitized topic string safe for most MQTT brokers
+
+    """
+    if not name:
+        raise ValueError("Name cannot be empty")
+    orig_name: str = name
+    name = re.sub(r"[^A-Za-z0-9_\-\.]+", replacement, name)
+
+    # Replace multiple consecutive replacement chars with single one
+    if replacement:
+        pattern = re.escape(replacement) + "+"
+        name = re.sub(pattern, replacement, name)
+
+    # Trim to max length
+    topic_bytes = name.encode("utf-8")
+    if len(topic_bytes) > max_len:
+        name = topic_bytes[:max_len].decode("utf-8", errors="ignore")
+
+    if not name:
+        raise ValueError("Topic became empty after sanitization")
+    if name != orig_name:
+        log.info("Component name %s changed to %s for MQTT/HA compatibility", orig_name, name)
+
+    return name