flashdata-google-news 0.1.0__py3-none-any.whl

flashdata_google_news-0.1.0.dist-info/METADATA

@@ -0,0 +1,97 @@
+Metadata-Version: 2.5
+Name: flashdata-google-news
+Version: 0.1.0
+Summary: Save a news topic snapshot for a daily briefing with the FlashData JSON CLI
+Project-URL: Homepage, https://flashdata.dev/docs/sources/google_news
+Project-URL: Documentation, https://flashdata.dev/docs/sources/google_news
+Project-URL: Pricing, https://flashdata.dev/pricing
+Project-URL: Support, https://flashdata.dev
+Project-URL: Source, https://github.com/rocosen/flashdata-mcp/tree/main/packages/python-cli
+Author-email: FlashData <support@flashdata.dev>
+License-Expression: MIT
+License-File: LICENSE
+Keywords: cli,flashdata,google-news,json,uvx
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3
+Requires-Python: >=3.10
+Requires-Dist: httpx<1,>=0.28.1
+Requires-Dist: jsonschema<5,>=4.23
+Requires-Dist: pydantic<3,>=2.7.4
+Description-Content-Type: text/markdown
+
+# Google News CLI — FlashData
+
+Save a news topic snapshot for a daily briefing. This command calls the `google_news` API and returns JSON for your
+shell script, data pipeline or agent tool runner. It does not invoke a language model.
+
+Returns news search results, not article full text.
+
+## Install and run
+
+Requires Python 3.10 or newer. [uv installation](https://docs.astral.sh/uv/getting-started/installation/)
+provides `uvx`, which runs this package in an isolated environment.
+
+1. [Create a FlashData account](https://flashdata.dev/signup), obtain an API Key and fund Credits.
+2. Set `FLASHDATA_API_KEY` using your shell or secret manager. The CLI does not accept a key flag
+   or persist credentials. [Authentication and source documentation](https://flashdata.dev/docs/sources/google_news).
+3. Run this example after reviewing the input and cost:
+
+```sh
+uvx flashdata-google-news@0.1.0 --query 'reusable rockets' --options-json '{"num": 5}' --output result.json
+```
+
+`--output` creates a new file with owner-only permissions where supported and refuses existing
+files before making a paid request. Without it, the full JSON response goes to stdout. Errors
+go to stderr. Exit 0 means the response was exported, 2 means local input/configuration failure,
+and 1 means request or output failure. For downloads, exit 0 can describe an unfinished job.
+
+For repeated use, install `uv tool install flashdata-google-news==0.1.0`; the command name is `flashdata-google-news`.
+Alternatively use `python -m pip install flashdata-google-news==0.1.0` in your virtual environment.
+
+## Inputs and output
+
+Use `--query` for the primary input. Additional options use
+`--options-json` as a JSON object, with strict field/type/range validation. Source is fixed by
+this package. Google count and locale options remain omitted unless supplied.
+
+| Option | Meaning |
+| --- | --- |
+| `num` | Requested result count. Default: `null`. |
+| `page` | Search result page. Default: `null`. |
+| `autocorrect` | Enable spelling autocorrection. Default: `null`. |
+| `gl` | Two-letter country code, for example us. Default: `null`. |
+| `hl` | Interface language code, for example en. Default: `null`. |
+| `location` | Optional geographic search location. Default: `null`. |
+| `tbs` | Optional Google time filter, for example qdr:w. Default: `null`. |
+
+Run `uvx flashdata-google-news@0.1.0 --schema` to inspect all fields, nested filters, allowed values and
+defaults without a Key or an API call. `--help` and `--version` are also local-only.
+
+The JSON output preserves the API result envelope and optional fields: results[].news: titles, links and available dates/snippets; usage.credits.
+Missing fields remain missing; empty results remain empty. No article text, translated captions,
+model conclusions, rights grants or keyword volumes are invented. Exact source contracts are
+documented [here](https://flashdata.dev/docs/sources/google_news).
+
+## Fees and failure handling
+
+This MIT-licensed CLI is free to install. FlashData API requests consume paid Credits;
+[current pricing](https://flashdata.dev/pricing) applies. Extended YouTube Search has a separate
+rate. Media delivery is billed by delivered volume, and your cloud-storage costs are separate.
+There is no automatic pagination, retry, polling, scheduled execution or model call.
+
+Each HTTP request has a 120-second network timeout (15-second connect timeout) and a 20-MiB response
+limit. A timeout or failure may occur after a paid request has been accepted. Check Jobs in
+the [FlashData console](https://flashdata.dev) before resubmitting. HTTP 401 means check the Key;
+402 means insufficient Credits; 429 means rate/quota limit. A partial output file is retained
+on failure. Do not retry a paid query just because output handling failed.
+
+Input values and your API Key are sent to FlashData to perform the request. A completed
+download-result read may fetch a signed JSON object from the result origin returned by
+FlashData; the API Key is never sent to that object URL. Only process data appropriate for
+your output destination. [Privacy policy](https://flashdata.dev/legal/privacy).
+
+Source and MIT license are included in the source distribution.
+
+[Source documentation](https://flashdata.dev/docs/sources/google_news) ·
+[Pricing](https://flashdata.dev/pricing) · [Sign up](https://flashdata.dev/signup)

flashdata_google_news-0.1.0.dist-info/RECORD

@@ -0,0 +1,12 @@
+flashdata_google_news_cli/__init__.py,sha256=kUR5RAFc7HCeiqdlX36dZOHkUI5wI6V_43RpEcD8b-0,22
+flashdata_google_news_cli/_capability.py,sha256=pitfzHUqC81-9ctbzL_hbXtEnyOGn-5vcOoY1BatV20,57
+flashdata_google_news_cli/_client.py,sha256=bkIpwrFeokVc-zRSUg1Jnv8r2DnDLAsvPeOwjuF0rMQ,6484
+flashdata_google_news_cli/_inputs.py,sha256=tV6mQp_9qnZa3-242YXdJ3GEfZh9l-lnvFMrdHEFGNY,9015
+flashdata_google_news_cli/_storage.py,sha256=iOJOGcOETKPE4HZF4RsBG3DiROEG1EFuexK1v_LClyQ,4538
+flashdata_google_news_cli/cli.py,sha256=1mMVfwCu-ohQyuy8MqDw1GeMA-qLqz3gWGhXFlk0a5E,7683
+flashdata_google_news_cli/storage.schema.json,sha256=ppAXH8fRKewYqiI57_r6rfeLSQ4pKqIZbeWB1jzbiPY,2370
+flashdata_google_news-0.1.0.dist-info/METADATA,sha256=-xLdwJstoT899OwuVV6dITc2aJX56gkVTUyaSZPHmgA,5262
+flashdata_google_news-0.1.0.dist-info/WHEEL,sha256=W3fkpkm7-wf9vBI5Z-7s0eWkeM-spu78I8Neb98DeEg,87
+flashdata_google_news-0.1.0.dist-info/entry_points.txt,sha256=ogX-YKZTTP3Cdp5aiJBak8R_kY3CV4LRe_ptF0M3gmo,77
+flashdata_google_news-0.1.0.dist-info/licenses/LICENSE,sha256=_WaID44HsDW1TORBvMsdgzkCyWysJarIM_3IVHobB7k,1079
+flashdata_google_news-0.1.0.dist-info/RECORD,,

flashdata_google_news-0.1.0.dist-info/WHEEL

@@ -0,0 +1,4 @@
+Wheel-Version: 1.0
+Generator: hatchling 1.32.4
+Root-Is-Purelib: true
+Tag: py3-none-any

flashdata_google_news-0.1.0.dist-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+flashdata-google-news = flashdata_google_news_cli.cli:main

flashdata_google_news-0.1.0.dist-info/licenses/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 FlashData contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

flashdata_google_news_cli/__init__.py

@@ -0,0 +1 @@
+__version__ = "0.1.0"

flashdata_google_news_cli/_capability.py

@@ -0,0 +1,2 @@
+SOURCE = 'google_news'
+COMMAND = 'flashdata-google-news'

flashdata_google_news_cli/_client.py

@@ -0,0 +1,152 @@
+"""Bounded, single-attempt requests for the standalone command-line tools."""
+
+from urllib.parse import urlsplit
+from uuid import uuid4
+
+import httpx
+
+QUERY_URL = "https://data.flashdata.dev/v1/queries/realtime"
+ASYNC_URL = "https://data.flashdata.dev/v1/queries"
+JOBS_URL = "https://api.flashdata.dev/v2/jobs"
+AUTH_URL = "https://api.flashdata.dev/v2/api-access"
+MAX_RESPONSE_BYTES = 20 * 1024 * 1024
+UNKNOWN_OUTCOME = (
+    "The request may already have been accepted and charged. "
+    "Check Jobs in the FlashData console before resubmitting. No automatic retry was made."
+)
+READ_ERROR = "Could not complete this read. Retry the same job read; do not submit a new job."
+
+
+class FlashDataError(Exception):
+    """An error safe to display without raw responses or credentials."""
+
+
+class Client:
+    def __init__(self, api_key, *, transport=None):
+        self.api_key = api_key
+        self.transport = transport
+
+    def _request(self, method, url, payload=None, *, authenticated=True):
+        headers = {"Accept": "application/json", "User-Agent": "flashdata-cli/0.1.0"}
+        if authenticated:
+            headers.update({"X-API-Key": self.api_key, "X-Request-Id": str(uuid4())})
+        try:
+            with (
+                httpx.Client(
+                    timeout=httpx.Timeout(120.0, connect=15.0),
+                    follow_redirects=False,
+                    trust_env=False,
+                    transport=self.transport,
+                ) as client,
+                client.stream(method, url, json=payload, headers=headers) as response,
+            ):
+                body = bytearray()
+                for chunk in response.iter_bytes():
+                    if len(body) + len(chunk) > MAX_RESPONSE_BYTES:
+                        raise FlashDataError(
+                            "Response exceeds 20 MiB. Retrieve it from Jobs; do not resubmit."
+                        )
+                    body.extend(chunk)
+                # iter_bytes already decodes content encodings; do not decode twice.
+                return httpx.Response(
+                    response.status_code,
+                    content=bytes(body),
+                    headers={"location": response.headers.get("location", "")},
+                )
+        except httpx.HTTPError:
+            raise FlashDataError(UNKNOWN_OUTCOME if method == "POST" else READ_ERROR) from None
+
+    @staticmethod
+    def _json(response, *, submission=False):
+        message = UNKNOWN_OUTCOME if submission else READ_ERROR
+        errors = {
+            401: "API key rejected. Check FLASHDATA_API_KEY.",
+            402: "Insufficient FlashData Credits. Fund your account before a new request.",
+            403: "This account or key cannot access the requested source.",
+            429: "FlashData rate or quota limit reached. Wait before a new request.",
+        }
+        if response.status_code != 200:
+            raise FlashDataError(
+                f"{errors.get(response.status_code, message)} HTTP {response.status_code}."
+            )
+        try:
+            data = response.json()
+        except ValueError:
+            raise FlashDataError(message) from None
+        if not isinstance(data, dict):
+            raise FlashDataError(message)
+        return data
+
+    def query(self, payload):
+        data = self._json(self._request("POST", QUERY_URL, payload), submission=True)
+        if data.get("status") not in (None, "done", "completed") or not isinstance(
+            data.get("results"), list
+        ):
+            raise FlashDataError(UNKNOWN_OUTCOME)
+        job = data.get("job")
+        if isinstance(job, dict) and job.get("status") == "failed":
+            raise FlashDataError("The job failed. Inspect it in Jobs before a new request.")
+        return data
+
+    def submit(self, payload):
+        from ._inputs import DownloadJobInput
+
+        data = self._json(self._request("POST", ASYNC_URL, payload), submission=True)
+        try:
+            DownloadJobInput(job_id=data.get("id"))
+        except ValueError:
+            raise FlashDataError(UNKNOWN_OUTCOME) from None
+        if data.get("status") not in ("pending", "processing"):
+            raise FlashDataError(UNKNOWN_OUTCOME)
+        return {k: data[k] for k in ("id", "status", "source", "created_at") if k in data}
+
+    def status(self, job_id):
+        from ._inputs import DownloadJobInput
+
+        DownloadJobInput(job_id=job_id)
+        data = self._json(self._request("GET", f"{JOBS_URL}/{job_id}"))
+        job = data.get("job")
+        if not isinstance(job, dict) or job.get("source") != "youtube_download":
+            raise FlashDataError("This reader only accepts YouTube download jobs.")
+        if job.get("id") != job_id:
+            raise FlashDataError("Job receipt does not match the requested ID.")
+        # Job params can contain storage credentials. Export lifecycle fields only.
+        return {
+            "job": {
+                k: job[k]
+                for k in ("id", "source", "status", "created_at", "updated_at", "completed_at")
+                if k in job
+            }
+        }
+
+    def result(self, job_id):
+        status = self.status(job_id)
+        if status["job"].get("status") != "completed":
+            return status
+        identity = self._json(self._request("GET", AUTH_URL))
+        link = self._request("GET", f"{JOBS_URL}/{job_id}/results")
+        try:
+            origin = identity.get("result_origin")
+            trusted = urlsplit(origin) if isinstance(origin, str) else None
+            target = urlsplit(link.headers.get("location", ""))
+            if (
+                link.status_code != 302
+                or not trusted
+                or trusted.scheme != "https"
+                or not trusted.hostname
+                or trusted.username
+                or trusted.password
+                or trusted.path not in ("", "/")
+                or trusted.query
+                or trusted.fragment
+                or target.scheme != "https"
+                or target.username
+                or target.password
+                or target.fragment
+                or (target.hostname, target.port or 443) != (trusted.hostname, trusted.port or 443)
+            ):
+                raise ValueError()
+        except ValueError:
+            raise FlashDataError("Result URL does not match trusted FlashData storage.") from None
+        output = self._json(self._request("GET", target.geturl(), authenticated=False))
+        return {**status, "output": output}

flashdata_google_news_cli/_inputs.py

@@ -0,0 +1,213 @@
+"""Validated tool arguments. Credentials are never part of these schemas."""
+
+import re
+from typing import Annotated, Literal
+from urllib.parse import parse_qs, urlsplit
+
+from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator
+
+Query = Annotated[str, Field(min_length=1, max_length=2000)]
+Language = Annotated[str, Field(pattern=r"^[A-Za-z0-9-]{1,35}$")]
+
+
+class _Input(BaseModel):
+    model_config = ConfigDict(extra="forbid", strict=True, str_strip_whitespace=True)
+
+
+class GoogleInput(_Input):
+    """Google source parameters; omitted settings use provider behavior."""
+
+    query: Query = Field(description="Search query, between 1 and 2,000 characters.")
+    num: int | None = Field(default=None, ge=1, le=100, description="Requested result count.")
+    page: int | None = Field(default=None, ge=1, le=100, description="Search result page.")
+    autocorrect: bool | None = Field(default=None, description="Enable spelling autocorrection.")
+    gl: Annotated[str, Field(pattern=r"^[A-Za-z]{2}$")] | None = Field(
+        default=None, description="Two-letter country code, for example us."
+    )
+    hl: Language | None = Field(
+        default=None, description="Interface language code, for example en."
+    )
+    location: Annotated[str, Field(min_length=1, max_length=200)] | None = Field(
+        default=None, description="Optional geographic search location."
+    )
+    tbs: Annotated[str, Field(min_length=1, max_length=200)] | None = Field(
+        default=None, description="Optional Google time filter, for example qdr:w."
+    )
+
+
+class GoogleSearchInput(GoogleInput):
+    """Preserve the original Google Search tool defaults."""
+
+    num: int = Field(default=10, ge=1, le=100)
+    page: int = Field(default=1, ge=1, le=100)
+    autocorrect: bool = True
+
+
+class YouTubeSearchFilters(_Input):
+    """Optional YouTube search filters, including feature flags."""
+
+    duration: Literal["<4", "4-20", ">20"] | None = None
+    sort_by: Literal["rating", "relevance", "view_count", "upload_date"] | None = None
+    type: Literal["video", "channel", "playlist", "movie"] | None = None
+    upload_date: Literal["today", "last_hour", "this_week", "this_month", "this_year"] | None = None
+    video_360: bool | None = Field(default=None, alias="360")
+    video_3d: bool | None = Field(default=None, alias="3d")
+    video_4k: bool | None = Field(default=None, alias="4k")
+    creative_commons: bool | None = None
+    hd: bool | None = None
+    hdr: bool | None = None
+    live: bool | None = None
+    location: bool | None = None
+    purchased: bool | None = None
+    subtitles: bool | None = None
+    vr180: bool | None = None
+
+
+class YouTubeSearchInput(_Input):
+    """YouTube video search parameters."""
+
+    query: Query = Field(description="YouTube search query, between 1 and 2,000 characters.")
+    max_results: int = Field(default=10, ge=1, le=20, description="Requested number of videos.")
+    filters: YouTubeSearchFilters | None = None
+
+
+class YouTubeExtendedSearchInput(YouTubeSearchInput):
+    """Larger searches have a separate source and rate; count is not guaranteed."""
+
+    max_results: int = Field(
+        default=20, ge=1, le=700, description="Requested result limit, up to 700."
+    )
+
+
+class _VideoInput(_Input):
+    video_id: str = Field(description="11-character YouTube video ID or a YouTube video URL.")
+
+    @field_validator("video_id")
+    @classmethod
+    def normalize_video_id(cls, value: str) -> str:
+        if re.fullmatch(r"[A-Za-z0-9_-]{11}", value):
+            return value
+        try:
+            url = urlsplit(value)
+            parts = url.path.split("/")
+            candidate = ""
+            if url.scheme in ("http", "https") and not url.username and not url.password:
+                if url.hostname == "youtu.be" and (
+                    len(parts) == 2 or (len(parts) == 3 and not parts[2])
+                ):
+                    candidate = parts[1]
+                elif url.hostname in (
+                    "youtube.com",
+                    "www.youtube.com",
+                    "m.youtube.com",
+                    "music.youtube.com",
+                ):
+                    if url.path == "/watch":
+                        candidate = parse_qs(url.query).get("v", [""])[0]
+                    elif (len(parts) == 3 or (len(parts) == 4 and not parts[3])) and parts[1] in (
+                        "shorts",
+                        "embed",
+                        "live",
+                    ):
+                        candidate = parts[2]
+            if re.fullmatch(r"[A-Za-z0-9_-]{11}", candidate):
+                return candidate
+        except ValueError:
+            pass
+        raise ValueError("Enter an 11-character YouTube ID or a youtube.com / youtu.be video URL.")
+
+
+class YouTubeMetadataInput(_VideoInput):
+    """YouTube metadata parameters."""
+
+    include_chapters: bool = Field(default=True, description="Include available video chapters.")
+    include_formats: bool = Field(default=True, description="Include available format metadata.")
+
+
+class YouTubeTranscriptInput(_VideoInput):
+    """YouTube transcript parameters; no implicit language or caption fallback."""
+
+    language: Language = Field(default="en", description="Requested caption language code.")
+    auto: bool = Field(
+        default=True,
+        description="True requests automatic captions; false requests manually authored captions.",
+    )
+
+
+class YouTubeCaptionsInput(YouTubeTranscriptInput):
+    """Select uploader captions by default, without fallback."""
+
+    auto: bool = Field(default=False, description="False selects manual; true automatic captions.")
+
+
+class YouTubeTrainabilityInput(_VideoInput):
+    """Inspect license labels; this does not establish training permission."""
+
+
+class YouTubeChannelInput(_Input):
+    handle: Annotated[str, Field(min_length=1, max_length=100)] = Field(
+        description="YouTube channel handle, for example @jawed."
+    )
+    video_limit: int = Field(default=20, ge=1, le=100)
+
+
+class YouTubeSuggestionsInput(_Input):
+    query: Query = Field(description="YouTube search prefix.")
+    language: Language = "en"
+    region: Annotated[str, Field(pattern=r"^[A-Za-z]{2}$")] = "US"
+
+
+class YouTubeDownloadInput(_VideoInput):
+    """Media options only. Storage credentials are injected by the application."""
+
+    download_type: Literal["audio_video", "video", "audio"] = "audio_video"
+    video_quality: (
+        Annotated[
+            str,
+            Field(pattern=r"^(best|worst|(?:<=|==|>=)(?:144|360|480|720|1080|1440|2160|4320))$"),
+        ]
+        | None
+    ) = None
+    fps: Annotated[str, Field(pattern=r"^(?:>=|<=|==)(?:[1-9]\d?|1\d\d|2[0-3]\d|240)$")] | None = (
+        None
+    )
+    audio_language: Literal["default", "original"] | None = None
+    audio_quality: Literal["best", "320", "256", "192", "128"] | None = None
+    yt_dlp_format: Annotated[str, Field(min_length=1, max_length=1024)] | None = None
+    upload_metadata: bool = False
+    output_format: Literal["mp4", "webm", "mkv", "m4a", "mp3", "opus"] | None = None
+    start_at: Annotated[str, Field(pattern=r"^\d{1,2}:[0-5]\d:[0-5]\d$")] | None = None
+    end_at: Annotated[str, Field(pattern=r"^\d{1,2}:[0-5]\d:[0-5]\d$")] | None = None
+
+    @model_validator(mode="after")
+    def validate_options(self):
+        audio = self.download_type == "audio"
+        formats = ("m4a", "mp3", "opus") if audio else ("mp4", "webm", "mkv")
+        if self.output_format is not None and self.output_format not in formats:
+            raise ValueError("output_format is incompatible with download_type.")
+        if self.yt_dlp_format and any(ord(c) < 32 or ord(c) == 127 for c in self.yt_dlp_format):
+            raise ValueError("yt_dlp_format must not contain control characters.")
+        if not self.yt_dlp_format:
+            if audio and (self.video_quality is not None or self.fps is not None):
+                raise ValueError("video_quality and fps do not apply to audio downloads.")
+            if self.download_type == "video" and self.audio_language is not None:
+                raise ValueError("audio_language does not apply to video-only downloads.")
+        if not audio and self.audio_quality is not None:
+            raise ValueError("audio_quality applies only to audio downloads.")
+        times = {}
+        for field in ("start_at", "end_at"):
+            value = getattr(self, field)
+            if value is not None:
+                h, m, s = map(int, value.split(":"))
+                times[field] = h * 3600 + m * 60 + s
+                if times[field] > 43200:
+                    raise ValueError("Clip timestamps must not exceed 12:00:00.")
+        if "end_at" in times and times["end_at"] <= times.get("start_at", 0):
+            raise ValueError("end_at must be later than start_at.")
+        return self
+
+
+class DownloadJobInput(_Input):
+    job_id: Annotated[str, Field(pattern=r"^job_[A-Za-z0-9-]{1,100}$")] = Field(
+        description="Job ID from a YouTube download submission."
+    )

flashdata_google_news_cli/_storage.py

@@ -0,0 +1,108 @@
+"""Customer storage validation. Never include credentials in errors."""
+
+import json
+import re
+from pathlib import Path
+from urllib.parse import parse_qs
+
+from jsonschema import Draft7Validator
+
+from ._client import FlashDataError
+
+STORAGE_SCHEMA = json.loads(Path(__file__).with_name("storage.schema.json").read_text())
+
+
+def storage_config(raw):
+    try:
+        storage = json.loads(raw)
+    except (ValueError, TypeError):
+        raise FlashDataError("Download storage must contain a JSON object.") from None
+    if not Draft7Validator(STORAGE_SCHEMA).is_valid(storage):
+        raise FlashDataError("Invalid download storage credential. Check its fields and types.")
+    kind = storage["storage_type"]
+    bucket = storage["storage_url"].strip()
+    if not re.fullmatch(r"[a-z0-9][a-z0-9.-]{1,61}[a-z0-9]", bucket) or ".." in bucket:
+        raise FlashDataError(
+            "storage_url must be a bucket/container name; put folders in object_prefix."
+        )
+    if kind == "s3" and (
+        re.fullmatch(r"\d{1,3}(?:\.\d{1,3}){3}", bucket) or ".-" in bucket or "-." in bucket
+    ):
+        raise FlashDataError("storage_url must be a valid AWS S3 bucket name.")
+    storage["storage_url"] = bucket
+    provider_fields = (
+        ["service_account"]
+        if kind == "gcs"
+        else ["account_name", "sas_token"]
+        if kind == "azure"
+        else ["access_key", "secret_key", "region"]
+        if kind == "cos"
+        else ["access_key", "secret_key", "region", "endpoint_url"]
+    )
+    if set(storage) - {"storage_type", "storage_url", "object_prefix", *provider_fields}:
+        raise FlashDataError("The storage credential contains fields for a different provider.")
+    if "object_prefix" in storage:
+        prefix = storage["object_prefix"].strip().removesuffix("/")
+        if (
+            not prefix
+            or len(prefix) > 512
+            or any(
+                part in (".", "..") or not re.fullmatch(r"[A-Za-z0-9._-]+", part)
+                for part in prefix.split("/")
+            )
+        ):
+            raise FlashDataError("object_prefix must use safe folder names, up to 512 characters.")
+        storage["object_prefix"] = prefix
+    if kind == "gcs":
+        account = storage.get("service_account", {})
+        if (
+            not all(account.get(k) for k in ("project_id", "client_email", "private_key"))
+            or not re.fullmatch(
+                r"[^@\s]+@[^@\s]+\.gserviceaccount\.com", account.get("client_email", "")
+            )
+            or "BEGIN PRIVATE KEY" not in account.get("private_key", "")
+        ):
+            raise FlashDataError(
+                "GCS requires project_id, service-account client_email and a PEM private_key."
+            )
+        storage["service_account"] = {
+            key: account[key] for key in ("project_id", "client_email", "private_key")
+        }
+    elif kind == "azure":
+        sas = storage.get("sas_token", "").removeprefix("?")
+        fields = parse_qs(sas)
+        if (
+            not storage.get("account_name")
+            or not re.fullmatch(r"[a-z0-9]+(?:-[a-z0-9]+)*", bucket)
+            or not all(fields.get(k) for k in ("sig", "se", "sv"))
+            or "w" not in fields.get("sp", [""])[0]
+            or fields.get("sr") != ["c"]
+            or "\n" in sas
+            or "\r" in sas
+            or len(sas) > 4096
+        ):
+            raise FlashDataError(
+                "Azure requires an account name and container SAS with write permission and expiry."
+            )
+        storage["sas_token"] = sas
+    else:
+        if not all(storage.get(k, "").strip() for k in ("access_key", "secret_key")):
+            raise FlashDataError(
+                "Download storage requires access_key and secret_key in the provider credential."
+            )
+        if kind == "s3_compatible" and not storage.get("endpoint_url"):
+            raise FlashDataError("S3-compatible storage requires endpoint_url.")
+        region = storage.get("region", "")
+        if (
+            (kind in ("cos", "bos", "tos") and "." in bucket)
+            or (
+                kind == "cos"
+                and (
+                    not re.fullmatch(r"ap-[a-z0-9-]+", region) or not re.search(r"-\d{5,}$", bucket)
+                )
+            )
+            or (kind == "bos" and region not in ("bj", "su", "hkg"))
+            or (kind == "tos" and not re.fullmatch(r"(?:cn|ap|eu|us)-[a-z0-9-]+", region))
+        ):
+            raise FlashDataError("Check the storage provider region and bucket name.")
+    return storage

flashdata_google_news_cli/cli.py

@@ -0,0 +1,201 @@
+"""CLI entry point shared at build time by fifteen independently named packages."""
+
+import argparse
+import json
+import os
+import sys
+from pathlib import Path
+from urllib.parse import urlsplit
+
+from pydantic import ValidationError
+
+from . import __version__
+from ._client import UNKNOWN_OUTCOME, Client, FlashDataError
+from ._inputs import (
+    DownloadJobInput,
+    GoogleInput,
+    YouTubeCaptionsInput,
+    YouTubeChannelInput,
+    YouTubeDownloadInput,
+    YouTubeExtendedSearchInput,
+    YouTubeMetadataInput,
+    YouTubeSearchInput,
+    YouTubeSuggestionsInput,
+    YouTubeTrainabilityInput,
+    YouTubeTranscriptInput,
+)
+from ._storage import storage_config
+
+CONTRACTS = {
+    **{
+        f"google_{kind}": GoogleInput
+        for kind in ("search", "news", "images", "shopping", "videos", "places")
+    },
+    "youtube_search": YouTubeSearchInput,
+    "youtube_search_extended": YouTubeExtendedSearchInput,
+    "youtube_metadata": YouTubeMetadataInput,
+    "youtube_transcript": YouTubeTranscriptInput,
+    "youtube_captions": YouTubeCaptionsInput,
+    "youtube_trainability": YouTubeTrainabilityInput,
+    "youtube_channel": YouTubeChannelInput,
+    "youtube_suggestions": YouTubeSuggestionsInput,
+    "youtube_download": YouTubeDownloadInput,
+}
+
+
+class Parser(argparse.ArgumentParser):
+    def error(self, message):
+        raise FlashDataError("Invalid command arguments. Use --help for supported options.")
+
+
+def parser_for(source, command):
+    parser = Parser(
+        prog=command,
+        description=(
+            f"Run the {source} FlashData API and export JSON. API calls use paid Credits. "
+            "Read FLASHDATA_API_KEY from the environment; never pass credentials as arguments."
+        ),
+    )
+    parser.add_argument("--version", action="version", version=f"{command} {__version__}")
+    parser.add_argument(
+        "--schema", action="store_true", help="Print the input schema; no API call."
+    )
+    parser.add_argument(
+        "--output", type=Path, help="New JSON file (0600); existing files are refused."
+    )
+    primary = (
+        "handle"
+        if source == "youtube_channel"
+        else "query"
+        if (
+            source.startswith("google_")
+            or source in ("youtube_search", "youtube_search_extended", "youtube_suggestions")
+        )
+        else "video-id"
+    )
+    parser.add_argument("--" + primary, help="Primary query, channel handle, or YouTube ID/URL.")
+    parser.add_argument(
+        "--options-json", help="Source options as a JSON object, without credentials."
+    )
+    if source == "youtube_download":
+        parser.add_argument("--operation", choices=["status", "result", "submit"], default="status")
+        parser.add_argument("--job-id", help="Previously saved job ID for status/result.")
+        parser.add_argument(
+            "--allow-submit", action="store_true", help="Permit one paid storage write."
+        )
+    return parser
+
+
+def payload_for(source, args, environ):
+    operation = getattr(args, "operation", "query")
+    if operation in ("status", "result"):
+        if args.video_id or args.options_json or args.allow_submit:
+            raise FlashDataError("Status/result accepts --job-id, without submission inputs.")
+        return operation, DownloadJobInput(job_id=args.job_id).model_dump()
+    try:
+        options = json.loads(args.options_json) if args.options_json is not None else {}
+    except ValueError:
+        raise FlashDataError("--options-json must be a JSON object.") from None
+    if not isinstance(options, dict) or any(
+        field in options for field in ("source", "query", "video_id", "handle", "storage")
+    ):
+        raise FlashDataError(
+            "Use dedicated primary inputs; source is fixed and secrets use env vars."
+        )
+    values = dict(options)
+    for primary in ("query", "video_id", "handle"):
+        value = getattr(args, primary, None)
+        if value is not None:
+            values[primary] = value
+    data = CONTRACTS[source].model_validate(values).model_dump(exclude_none=True, by_alias=True)
+    primary = next(key for key in ("query", "video_id", "handle") if key in data)
+    payload = {"source": source, primary: data.pop(primary), "params": data}
+    if operation == "submit":
+        if args.job_id or not args.allow_submit:
+            raise FlashDataError(
+                "New delivery requires --allow-submit; saved jobs use status/result."
+            )
+        storage = storage_config(environ.get("FLASHDATA_DOWNLOAD_STORAGE", ""))
+        endpoint = storage.get("endpoint_url")
+        if endpoint:
+            url = urlsplit(endpoint)
+            if (
+                url.scheme != "https"
+                or not url.hostname
+                or url.username
+                or url.password
+                or url.query
+                or url.fragment
+            ):
+                raise FlashDataError("Storage endpoint must be an HTTPS URL without credentials.")
+        payload["params"]["storage"] = storage
+    return operation, payload
+
+
+def run(
+    source, command, argv=None, *, environ=None, stdout=None, stderr=None, client_factory=Client
+):
+    environ = os.environ if environ is None else environ
+    stdout = sys.stdout if stdout is None else stdout
+    stderr = sys.stderr if stderr is None else stderr
+    output = None
+    request_started = False
+    output_complete = False
+    try:
+        args = parser_for(source, command).parse_args(argv)
+        if args.schema:
+            stdout.write(json.dumps(CONTRACTS[source].model_json_schema(), indent=2) + "\n")
+            return 0
+        operation, payload = payload_for(source, args, environ)
+        key = environ.get("FLASHDATA_API_KEY", "").strip()
+        if not key or any(ord(c) < 32 or ord(c) == 127 for c in key):
+            raise FlashDataError("Set a valid FLASHDATA_API_KEY environment variable.")
+        if args.output:
+            # Reserve the new path before spending Credits, without replacing user files.
+            fd = os.open(args.output, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
+            output = os.fdopen(fd, "w", encoding="utf-8")
+        client = client_factory(key)
+        request_started = True
+        result = (
+            client.query(payload)
+            if operation == "query"
+            else client.submit(payload)
+            if operation == "submit"
+            else client.status(payload["job_id"])
+            if operation == "status"
+            else client.result(payload["job_id"])
+        )
+        destination = output if output else stdout
+        destination.write(json.dumps(result, ensure_ascii=False, allow_nan=False, indent=2) + "\n")
+        destination.flush()
+        output_complete = True
+        return 0
+    except ValidationError:
+        stderr.write("Invalid source input. See --schema for fields, types, ranges and defaults.\n")
+        return 2
+    except FlashDataError as error:
+        stderr.write(str(error) + "\n")
+        return 1 if request_started else 2
+    except (OSError, ValueError, TypeError):
+        stderr.write(
+            (
+                "Could not save the response. " + UNKNOWN_OUTCOME
+                if request_started
+                else "Could not prepare output or inputs. Check the new output path."
+            )
+            + "\n"
+        )
+        return 1 if request_started else 2
+    finally:
+        if output:
+            output.close()
+            if not output_complete:
+                stderr.write(
+                    "The newly reserved output file is incomplete; check Jobs before retrying.\n"
+                )
+
+
+def main():
+    from ._capability import COMMAND, SOURCE
+
+    raise SystemExit(run(SOURCE, COMMAND))

flashdata_google_news_cli/storage.schema.json

@@ -0,0 +1,87 @@
+{
+  "type": "object",
+  "additionalProperties": false,
+  "required": [
+    "storage_type",
+    "storage_url"
+  ],
+  "description": "Customer-owned destination, using the same fields as Download API params.storage. Supply credentials from your configured storage; do not invent values or display secrets. GCS uses service_account; Azure uses account_name and a container SAS with write permission; other providers use access_key/secret_key and provider endpoint/region as needed.",
+  "properties": {
+    "storage_type": {
+      "type": "string",
+      "enum": [
+        "azure",
+        "bos",
+        "cos",
+        "gcs",
+        "oss",
+        "s3",
+        "s3_compatible",
+        "tos"
+      ]
+    },
+    "storage_url": {
+      "type": "string",
+      "minLength": 1,
+      "maxLength": 1024,
+      "description": "Bucket name (Azure: container name). Not a local path or a full object URL."
+    },
+    "object_prefix": {
+      "type": "string",
+      "minLength": 1,
+      "maxLength": 513,
+      "description": "Optional object folder prefix. Use a distinct prefix when keeping multiple versions of a video."
+    },
+    "access_key": {
+      "type": "string",
+      "maxLength": 4096
+    },
+    "secret_key": {
+      "type": "string",
+      "maxLength": 4096
+    },
+    "endpoint_url": {
+      "type": "string",
+      "maxLength": 4096,
+      "description": "Provider endpoint, required for S3-compatible storage. Must be reachable from the download Worker."
+    },
+    "region": {
+      "type": "string",
+      "maxLength": 4096
+    },
+    "account_name": {
+      "type": "string",
+      "pattern": "^[a-z0-9]{3,24}$"
+    },
+    "sas_token": {
+      "type": "string",
+      "maxLength": 4097,
+      "description": "Azure container SAS with write permission; never return this secret to the user in output."
+    },
+    "service_account": {
+      "type": "object",
+      "required": [
+        "project_id",
+        "client_email",
+        "private_key"
+      ],
+      "properties": {
+        "project_id": {
+          "type": "string",
+          "minLength": 1,
+          "maxLength": 256
+        },
+        "client_email": {
+          "type": "string",
+          "minLength": 1
+        },
+        "private_key": {
+          "type": "string",
+          "minLength": 1,
+          "maxLength": 16384
+        }
+      },
+      "additionalProperties": true
+    }
+  }
+}