salesforce-data-customcode 7.0.0rc2__tar.gz → 7.0.0rc4__tar.gz

{salesforce_data_customcode-7.0.0rc2 → salesforce_data_customcode-7.0.0rc4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: salesforce-data-customcode
-Version: 7.0.0rc2
+Version: 7.0.0rc4
 Summary: Data Cloud Custom Code SDK
 License-Expression: Apache-2.0
 License-File: LICENSE.txt
@@ -176,8 +176,8 @@ For a batch transform, use `Client`. You should only need the following methods:
 * `find_file_path(file_name)` – Resolve a bundled file (placed under `payload/files/`) to a `pathlib.Path` that exists. Works the same locally and inside Data Cloud — see [Bundled file resolution](#bundled-file-resolution) below for the full lookup order. Raises `FileNotFoundError` if the file isn't found.
 * `read_dlo(name)` – Read from a Data Lake Object by name
 * `read_dmo(name)` – Read from a Data Model Object by name
-* `write_to_dlo(name, spark_dataframe, write_mode)` – Write to a Data Model Object by name with a Spark dataframe
-* `write_to_dmo(name, spark_dataframe, write_mode)` – Write to a Data Lake Object by name with a Spark dataframe
+* `write_to_dlo(name, spark_dataframe, write_mode)` – Write to a Data Lake Object by name with a Spark dataframe
+* `write_to_dmo(name, spark_dataframe, write_mode)` – Write to a Data Model Object by name with a Spark dataframe
 
 For a streaming (delta) transform, use `StreamingClient`, which exposes the streaming counterparts:
 * `read_dlo_deltas()` – Read the streaming change feed (deltas) of a Data Lake Object as a streaming DataFrame.
@@ -201,6 +201,27 @@ client.write_to_dlo('output_DLO')
 > [!WARNING]
 > Currently we only support reading from DMOs and writing to DMOs or reading from DLOs and writing to DLOs, but they cannot mix.
 
+### Writing Currency fields
+
+A DLO or DMO with a Currency field has a system field, `cdp_sys_record_currency__c`, that holds each record's ISO currency code. Data Explorer UI uses it to display currency values, and shows a blank value if it's null (Query Editor still shows it).
+
+Code Extensions don't set `cdp_sys_record_currency__c` for you, since only your script knows what currency its amounts are in. When writing to an object with Currency fields, set it yourself:
+
+```python
+from pyspark.sql.functions import lit
+
+# Either source has Currency fields: keep its cdp_sys_record_currency__c in your output
+df = source.select("id__c", "amount__c", "cdp_sys_record_currency__c")
+
+# OR amounts are in a known currency
+df = df.withColumn("cdp_sys_record_currency__c", lit("EUR"))
+```
+
+- Use an ISO code that exists in your org's configured currency rates.
+- When aggregating, group by the currency code so amounts in different currencies aren't summed together.
+
+See [`docs/examples/script/currency_totals`](docs/examples/script/currency_totals) for a complete example.
+
 ### Streaming (delta) transforms
 
 Streaming BYOC transforms process a Data Lake Object's Change Data Feed continuously instead of reading a bounded snapshot. Use a `StreamingClient` and its `*_deltas` methods in place of the batch `Client` read/write methods:
@@ -249,6 +270,29 @@ If none of these exist, `find_file_path` raises `FileNotFoundError` with the lis
 
 `$LIBRARY_PATH` is set automatically to the root of the package at runtime inside Data Cloud.
 
+### Generating lineage
+
+`generate_lineage` runs a script entry point locally and derives the field-level `dataLineage` section of `config.json` from the Spark plans of the DataFrames it writes:
+
+```python
+from datacustomcode.lineage import generate_lineage
+
+result = generate_lineage("payload/entrypoint.py", sf_cli_org="my-org")
+print(result.to_dict()["dataLineage"])
+# [{"targetField": "Output__dll.name__c",
+#   "sourceFields": ["Input__dll.first__c", "Input__dll.last__c"]}, ...]
+```
+
+The entry point runs exactly as it would locally (same config, reader, and credentials), except that writes are traced and validated against the target DLO instead of being performed. It runs in a separate Python process, which inherits your working directory, environment, and `sys.path`; errors raised by the script are re-raised from `generate_lineage`. Reads use the reader's `default_row_limit` (1000 by default; set it in your `config.yaml`).
+
+> **Review the output before you use it.** The generated lineage is a starting point for review, not a guarantee. It reflects only the code paths and data your local run exercised, and it can't see inside UDFs. Accurate and complete `dataLineage` in your package is your responsibility as the package author.
+
+- A target field's sources are the fields its *value* is computed from. Fields that only decide which rows, groups or branch are used aren't sources, however the logic is written: filter conditions, join keys, `groupBy` keys, sort keys, window `partitionBy`/`orderBy` keys, `when()`/`if()` conditions, and aggregate `FILTER (WHERE …)` clauses. For example, `groupBy("region").agg(sum("amount"))` and `sum("amount").over(Window.partitionBy("region"))` both trace to `amount` only.
+- Fields computed only from literals are listed in `noSources`. So are fields whose value doesn't come from any input field's value, such as `row_number()`, `rank()`, `count("*")`, or `sum(when(cond, 1))`.
+- Fields that can't be traced are listed in `unresolved` with a reason and get no lineage. This covers data that didn't come from `read_dlo`/`read_dmo` (for example a `toPandas()` → `createDataFrame` round trip), `mapInPandas`, and subqueries.
+- `pairCount` is the total number of (target, source) field pairs.
+- Only batch script packages are supported.
+
 
 ## CLI
 

{salesforce_data_customcode-7.0.0rc2 → salesforce_data_customcode-7.0.0rc4}/README.md

@@ -151,8 +151,8 @@ For a batch transform, use `Client`. You should only need the following methods:
 * `find_file_path(file_name)` – Resolve a bundled file (placed under `payload/files/`) to a `pathlib.Path` that exists. Works the same locally and inside Data Cloud — see [Bundled file resolution](#bundled-file-resolution) below for the full lookup order. Raises `FileNotFoundError` if the file isn't found.
 * `read_dlo(name)` – Read from a Data Lake Object by name
 * `read_dmo(name)` – Read from a Data Model Object by name
-* `write_to_dlo(name, spark_dataframe, write_mode)` – Write to a Data Model Object by name with a Spark dataframe
-* `write_to_dmo(name, spark_dataframe, write_mode)` – Write to a Data Lake Object by name with a Spark dataframe
+* `write_to_dlo(name, spark_dataframe, write_mode)` – Write to a Data Lake Object by name with a Spark dataframe
+* `write_to_dmo(name, spark_dataframe, write_mode)` – Write to a Data Model Object by name with a Spark dataframe
 
 For a streaming (delta) transform, use `StreamingClient`, which exposes the streaming counterparts:
 * `read_dlo_deltas()` – Read the streaming change feed (deltas) of a Data Lake Object as a streaming DataFrame.
@@ -176,6 +176,27 @@ client.write_to_dlo('output_DLO')
 > [!WARNING]
 > Currently we only support reading from DMOs and writing to DMOs or reading from DLOs and writing to DLOs, but they cannot mix.
 
+### Writing Currency fields
+
+A DLO or DMO with a Currency field has a system field, `cdp_sys_record_currency__c`, that holds each record's ISO currency code. Data Explorer UI uses it to display currency values, and shows a blank value if it's null (Query Editor still shows it).
+
+Code Extensions don't set `cdp_sys_record_currency__c` for you, since only your script knows what currency its amounts are in. When writing to an object with Currency fields, set it yourself:
+
+```python
+from pyspark.sql.functions import lit
+
+# Either source has Currency fields: keep its cdp_sys_record_currency__c in your output
+df = source.select("id__c", "amount__c", "cdp_sys_record_currency__c")
+
+# OR amounts are in a known currency
+df = df.withColumn("cdp_sys_record_currency__c", lit("EUR"))
+```
+
+- Use an ISO code that exists in your org's configured currency rates.
+- When aggregating, group by the currency code so amounts in different currencies aren't summed together.
+
+See [`docs/examples/script/currency_totals`](docs/examples/script/currency_totals) for a complete example.
+
 ### Streaming (delta) transforms
 
 Streaming BYOC transforms process a Data Lake Object's Change Data Feed continuously instead of reading a bounded snapshot. Use a `StreamingClient` and its `*_deltas` methods in place of the batch `Client` read/write methods:
@@ -224,6 +245,29 @@ If none of these exist, `find_file_path` raises `FileNotFoundError` with the lis
 
 `$LIBRARY_PATH` is set automatically to the root of the package at runtime inside Data Cloud.
 
+### Generating lineage
+
+`generate_lineage` runs a script entry point locally and derives the field-level `dataLineage` section of `config.json` from the Spark plans of the DataFrames it writes:
+
+```python
+from datacustomcode.lineage import generate_lineage
+
+result = generate_lineage("payload/entrypoint.py", sf_cli_org="my-org")
+print(result.to_dict()["dataLineage"])
+# [{"targetField": "Output__dll.name__c",
+#   "sourceFields": ["Input__dll.first__c", "Input__dll.last__c"]}, ...]
+```
+
+The entry point runs exactly as it would locally (same config, reader, and credentials), except that writes are traced and validated against the target DLO instead of being performed. It runs in a separate Python process, which inherits your working directory, environment, and `sys.path`; errors raised by the script are re-raised from `generate_lineage`. Reads use the reader's `default_row_limit` (1000 by default; set it in your `config.yaml`).
+
+> **Review the output before you use it.** The generated lineage is a starting point for review, not a guarantee. It reflects only the code paths and data your local run exercised, and it can't see inside UDFs. Accurate and complete `dataLineage` in your package is your responsibility as the package author.
+
+- A target field's sources are the fields its *value* is computed from. Fields that only decide which rows, groups or branch are used aren't sources, however the logic is written: filter conditions, join keys, `groupBy` keys, sort keys, window `partitionBy`/`orderBy` keys, `when()`/`if()` conditions, and aggregate `FILTER (WHERE …)` clauses. For example, `groupBy("region").agg(sum("amount"))` and `sum("amount").over(Window.partitionBy("region"))` both trace to `amount` only.
+- Fields computed only from literals are listed in `noSources`. So are fields whose value doesn't come from any input field's value, such as `row_number()`, `rank()`, `count("*")`, or `sum(when(cond, 1))`.
+- Fields that can't be traced are listed in `unresolved` with a reason and get no lineage. This covers data that didn't come from `read_dlo`/`read_dmo` (for example a `toPandas()` → `createDataFrame` round trip), `mapInPandas`, and subqueries.
+- `pairCount` is the total number of (target, source) field pairs.
+- Only batch script packages are supported.
+
 
 ## CLI
 

{salesforce_data_customcode-7.0.0rc2 → salesforce_data_customcode-7.0.0rc4}/pyproject.toml

@@ -18,7 +18,7 @@ license = "Apache-2.0"
 name = "salesforce-data-customcode"
 readme = "README.md"
 requires-python = ">=3.10,<3.12"
-version = "7.0.0rc2"
+version = "7.0.0rc4"
 
 [tool.black]
 exclude = '''
@@ -88,10 +88,10 @@ warn_unused_configs = true
 
 [tool.poetry]
 include = [
-  {path = "src/datacustomcode/templates/**/*"},
-  {path = "src/datacustomcode/config.yaml"}
+  { path = "src/datacustomcode/templates/**/*" },
+  { path = "src/datacustomcode/config.yaml" }
 ]
-packages = [{include = "datacustomcode", from = "src"}]
+packages = [{ include = "datacustomcode", from = "src" }]
 
 [tool.poetry.build]
 generate-setup-file = false
@@ -115,7 +115,7 @@ coverage = ">=7.0.0,<8.0.0"
 ipykernel = "^6.29.5"
 mypy = "*"
 pipreqs = "*"
-poetry-dynamic-versioning = {extras = ["plugin"], version = "^1.8.2"}
+poetry-dynamic-versioning = { extras = ["plugin"], version = "^1.8.2" }
 pre-commit = "*"
 pytest = "*"
 pytest-cov = "*"
@@ -127,7 +127,7 @@ types-requests = "*"
 [tool.poetry.plugins."poetry.plugin"]
 
 [tool.poetry.requires-plugins]
-poetry-dynamic-versioning = {version = ">=1.0.0,<2.0.0", extras = ["plugin"]}
+poetry-dynamic-versioning = { version = ">=1.0.0,<2.0.0", extras = ["plugin"] }
 
 [tool.poetry.scripts]
 datacustomcode = "datacustomcode.cli:cli"
@@ -152,7 +152,6 @@ testpaths = ["tests"]
 [tool.ruff]
 fix = true
 line-length = 88
-target-version = 'py310'
 lint.ignore = [
   # do not assign a lambda expression, use a def
   'E731',
@@ -211,6 +210,7 @@ lint.select = [
   'RUF',
   'S102'
 ]
+target-version = 'py310'
 
 [tool.setuptools.package_data]
 datacustomcode = ["templates/**/*", "config.yaml"]

{salesforce_data_customcode-7.0.0rc2 → salesforce_data_customcode-7.0.0rc4}/src/datacustomcode/__init__.py

@@ -24,6 +24,8 @@ __all__ = [
     "Credentials",
     "DefaultSparkEinsteinPredictions",
     "DefaultSparkLLMGateway",
+    "LocalDeltasReader",
+    "LocalDeltasWriter",
     "PrintDataCloudWriter",
     "QueryAPIDataCloudReader",
     "SparkEinsteinPredictions",
@@ -60,6 +62,14 @@ def __getattr__(name: str):
         from datacustomcode.io.reader.query_api import QueryAPIDataCloudReader
 
         return QueryAPIDataCloudReader
+    elif name == "LocalDeltasReader":
+        from datacustomcode.io.reader.local_deltas import LocalDeltasReader
+
+        return LocalDeltasReader
+    elif name == "LocalDeltasWriter":
+        from datacustomcode.io.writer.local_deltas import LocalDeltasWriter
+
+        return LocalDeltasWriter
     elif name == "SparkLLMGateway":
         from datacustomcode.llm_gateway import SparkLLMGateway
 

{salesforce_data_customcode-7.0.0rc2 → salesforce_data_customcode-7.0.0rc4}/src/datacustomcode/client.py

@@ -231,11 +231,12 @@ def einstein_predict_col(
 def named_credential_request_col(
     request: "HTTPRequest",
     body: Optional["Column"] = None,
+    path: Optional["Column"] = None,
 ) -> "Column":
     """Build a Spark Column that makes one Named Credential callout per row.
 
-    The endpoint, method, and headers are fixed for the call (taken from
-    ``request``); only ``body`` varies per row. Use this instead of
+    The method and headers are fixed for the call (taken from ``request``);
+    ``body`` and, optionally, ``path`` vary per row. Use this instead of
     :meth:`Client.named_credential_request` when the callout runs across a
     DataFrame so each row is dispatched independently rather than one-shot on
     the driver.
@@ -253,13 +254,20 @@ def named_credential_request_col(
             headers are applied to every row.
         body: Optional per-row ``Column`` holding the request body as a
             string (or null for no body).
+        path: Optional per-row ``Column`` appended to ``request.url``, e.g.
+            ``concat(lit("?address="), col("address"))`` or ``lit("/v1/models")``.
+            The callout always stays on ``request.url``'s Named Credential: a
+            path not starting with ``/`` or ``?`` is joined with ``/``. A null
+            or empty row value uses ``request.url`` as is.
 
     Returns:
         A Spark ``Column`` of ``StructType`` with fields ``status``,
         ``response``, ``error_code``, and ``error_message``.
     """
     named_credential = Client()._get_spark_named_credential()
-    return named_credential.request_col(request, body=body)
+    if path is None:
+        return named_credential.request_col(request, body=body)
+    return named_credential.request_col(request, body=body, path=path)
 
 
 class DataCloudObjectType(Enum):
@@ -368,7 +376,13 @@ class _BaseClient:
             instance._spark_named_credential = spark_named_credential
             # Initialize Readers and Writers from config
             # and/or provided reader and writer
-            if reader is None or writer is None:
+            reader_forced = (
+                config.reader_config is not None and config.reader_config.force
+            )
+            writer_forced = (
+                config.writer_config is not None and config.writer_config.force
+            )
+            if reader is None or writer is None or reader_forced or writer_forced:
                 # We need a spark because we will initialize readers and writers.
                 # Reuse the process-wide session if one client already built it,
                 # so a Client and a StreamingClient share a single connection.
@@ -396,9 +410,7 @@ class _BaseClient:
                 raise ValueError(
                     "Reader config is required when reader is not provided"
                 )
-            elif reader is None or (
-                config.reader_config is not None and config.reader_config.force
-            ):
+            elif reader is None or reader_forced:
                 reader_init = config.reader_config.to_object(spark)  # type: ignore
             else:
                 reader_init = reader
@@ -406,9 +418,7 @@ class _BaseClient:
                 raise ValueError(
                     "Writer config is required when writer is not provided"
                 )
-            elif writer is None or (
-                config.writer_config is not None and config.writer_config.force
-            ):
+            elif writer is None or writer_forced:
                 writer_init = config.writer_config.to_object(spark)  # type: ignore
             else:
                 writer_init = writer

salesforce_data_customcode-7.0.0rc4/src/datacustomcode/io/cdf.py

@@ -0,0 +1,34 @@
+# Copyright (c) 2025, Salesforce, Inc.
+# SPDX-License-Identifier: Apache-2
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from __future__ import annotations
+
+from typing import Final
+
+from datacustomcode.io.writer.base import MERGE_RECORD_TYPE_COLUMN, MergeRecordType
+
+COMMIT_VERSION: Final = "_commit_version"
+COMMIT_TIMESTAMP: Final = "_commit_timestamp"
+MERGE_RECORD_TYPE: Final = MERGE_RECORD_TYPE_COLUMN
+
+CDF_METADATA_COLUMNS: Final = (COMMIT_VERSION, COMMIT_TIMESTAMP, MERGE_RECORD_TYPE)
+
+__all__ = [
+    "COMMIT_VERSION",
+    "COMMIT_TIMESTAMP",
+    "MERGE_RECORD_TYPE",
+    "CDF_METADATA_COLUMNS",
+    "MergeRecordType",
+]

salesforce_data_customcode-7.0.0rc4/src/datacustomcode/io/reader/local_deltas.py

@@ -0,0 +1,164 @@
+# Copyright (c) 2025, Salesforce, Inc.
+# SPDX-License-Identifier: Apache-2
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from __future__ import annotations
+
+import json
+import logging
+from pathlib import Path
+import sys
+from typing import (
+    TYPE_CHECKING,
+    Optional,
+    Union,
+)
+
+from datacustomcode.config import config
+from datacustomcode.io import cdf
+from datacustomcode.io.reader.base import BaseDataCloudReader
+from datacustomcode.io.reader.query_api import QueryAPIDataCloudReader
+
+if TYPE_CHECKING:
+    from pyspark.sql import DataFrame as PySparkDataFrame, SparkSession
+    from pyspark.sql.types import AtomicType, StructType
+
+logger = logging.getLogger(__name__)
+
+
+class LocalDeltasReader(BaseDataCloudReader):
+    """Local reader for streaming transforms"""
+
+    CONFIG_NAME = "LocalDeltasReader"
+
+    def __init__(
+        self,
+        spark: SparkSession,
+        credentials_profile: str = "default",
+        dataspace: Optional[str] = None,
+        sf_cli_org: Optional[str] = None,
+        default_row_limit: Optional[int] = None,
+        fixtures_root: str = "payload/streaming_fixtures",
+    ) -> None:
+        super().__init__(spark)
+        self._fixtures_root = Path(fixtures_root)
+        self._credentials_profile = credentials_profile
+        self._dataspace = dataspace
+        self._sf_cli_org = sf_cli_org
+        # Reuse the batch reader for INITIAL_SYNC / REBUILD reads.
+        self._batch = QueryAPIDataCloudReader(
+            spark=spark,
+            credentials_profile=credentials_profile,
+            dataspace=dataspace,
+            sf_cli_org=sf_cli_org,
+            default_row_limit=default_row_limit,
+        )
+        self._current_layer = "dlo"
+
+    def read_dlo(
+        self,
+        name: str,
+        schema: Union[AtomicType, StructType, str, None] = None,
+    ) -> PySparkDataFrame:
+        return self._batch.read_dlo(name, schema)
+
+    def read_dmo(
+        self,
+        name: str,
+        schema: Union[AtomicType, StructType, str, None] = None,
+    ) -> PySparkDataFrame:
+        return self._batch.read_dmo(name, schema)
+
+    def read_dlo_deltas(self) -> PySparkDataFrame:
+        self._current_layer = "dlo"
+        return self._open_stream(self._streaming_source())
+
+    def read_dmo_deltas(self) -> PySparkDataFrame:
+        self._current_layer = "dmo"
+        return self._open_stream(self._streaming_source())
+
+    def _streaming_source(self) -> str:
+        source = config.streaming_source
+        if not source:
+            raise RuntimeError(
+                "No streaming source configured. Set streamingSource.name in "
+                "config.json."
+            )
+        return source
+
+    def _open_stream(self, name: str) -> PySparkDataFrame:
+        drop_dir = self._fixtures_root / name
+
+        if not (drop_dir / "_schema.json").exists():
+            seeded = self._try_seed(name)
+            if not seeded:
+                print(
+                    f"\nStreaming source {self._current_layer}='{name}' is "
+                    f"empty.\n"
+                    f"  Populate the stream source with at least one row in "
+                    f"Data Cloud, then re-run `datacustomcode run`.\n"
+                )
+                sys.exit(0)
+
+            print(
+                f"\nWrote sample streaming fixture in: {drop_dir} "
+                f"based on the contents of the streaming source {name}.\n"
+                f"  You may add more JSON files alongside it to simulate "
+                f"additional changes.\n"
+            )
+
+        schema = self._build_stream_schema(name)
+        return (
+            self.spark.readStream.format("json")
+            .schema(schema)
+            .option("maxFilesPerTrigger", 1)  # one file = one batch
+            .option("latestFirst", "false")  # oldest mtime first
+            .load(str(drop_dir))
+        )
+
+    def _build_stream_schema(self, name: str) -> "StructType":
+        """Compose (source schema + CDF metadata columns) for readStream."""
+        from pyspark.sql.types import (
+            LongType,
+            StringType,
+            StructField,
+            StructType,
+            TimestampType,
+        )
+
+        source_schema = self._resolve_source_schema(name)
+        cdf_fields = [
+            StructField(cdf.COMMIT_VERSION, LongType(), True),
+            StructField(cdf.COMMIT_TIMESTAMP, TimestampType(), True),
+            StructField(cdf.MERGE_RECORD_TYPE, StringType(), True),
+        ]
+        return StructType(list(source_schema.fields) + cdf_fields)
+
+    def _resolve_source_schema(self, name: str) -> "StructType":
+        from pyspark.sql.types import StructType
+
+        schema_file = self._fixtures_root / name / "_schema.json"
+        return StructType.fromJson(json.loads(schema_file.read_text()))
+
+    def _try_seed(self, name: str) -> bool:
+        """Creates source schema and sample change file."""
+        from datacustomcode.io.reader.streaming_seeder import StreamingSourceSeeder
+
+        seeder = StreamingSourceSeeder(
+            spark=self.spark,
+            credentials_profile=self._credentials_profile,
+            dataspace=self._dataspace,
+            sf_cli_org=self._sf_cli_org,
+        )
+        return seeder.seed_source(name, self._current_layer, str(self._fixtures_root))

salesforce_data_customcode-7.0.0rc4/src/datacustomcode/io/reader/streaming_seeder.py

@@ -0,0 +1,114 @@
+# Copyright (c) 2025, Salesforce, Inc.
+# SPDX-License-Identifier: Apache-2
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from __future__ import annotations
+
+from datetime import datetime, timezone
+import json
+import math
+from pathlib import Path
+from typing import (
+    TYPE_CHECKING,
+    Any,
+    Optional,
+)
+
+from datacustomcode.io import cdf
+from datacustomcode.io.reader.query_api import QueryAPIDataCloudReader
+
+if TYPE_CHECKING:
+    import pandas as pd
+    from pyspark.sql import SparkSession
+
+SEED_LIMIT = 10
+
+
+def _clean_for_json(value: Any) -> Any:
+    if isinstance(value, float):
+        if math.isnan(value):
+            return None
+        if value.is_integer():
+            return int(value)
+    return value
+
+
+class StreamingSourceSeeder:
+    def __init__(
+        self,
+        spark: SparkSession,
+        credentials_profile: str = "default",
+        dataspace: Optional[str] = None,
+        sf_cli_org: Optional[str] = None,
+    ) -> None:
+        self.spark = spark
+        self.reader = QueryAPIDataCloudReader(
+            spark=spark,
+            credentials_profile=credentials_profile,
+            dataspace=dataspace,
+            sf_cli_org=sf_cli_org,
+        )
+
+    def seed_source(self, name: str, layer: str, fixtures_root: str) -> bool:
+        """Seed streaming fixtures and source schema.
+
+        Returns:
+            ``True`` when schema + fixtures are written
+            ``False`` when the source is empty.
+            Query API errors (missing creds, missing
+            source) propagate as ``RuntimeError``.
+        """
+        read_fn = self.reader.read_dlo if layer == "dlo" else self.reader.read_dmo
+
+        try:
+            head_df = read_fn(name).limit(1)
+        except Exception as exc:
+            raise RuntimeError(f"Failed to read {layer}='{name}': {exc}.") from exc
+
+        head_pandas = head_df.toPandas()
+        if len(head_pandas) == 0:
+            return False
+
+        schema = head_df.schema
+        out_dir = Path(fixtures_root) / name
+        out_dir.mkdir(parents=True, exist_ok=True)
+        schema_file_path = out_dir / "_schema.json"
+        schema_file_path.write_text(json.dumps(schema.jsonValue()))
+        print(
+            f"\nStreaming source schema written to {schema_file_path} "
+            f"({len(schema.fields)} fields)"
+        )
+
+        # Mix UPSERT and DELETE in the seed so the customer sees both
+        # operation types in the starter fixture. Annotated because
+        # pyspark's ``.toPandas()`` return type is ``PandasDataFrameLike``,
+        # which mypy doesn't recognize as having ``to_dict``.
+        snapshot: pd.DataFrame = read_fn(name).limit(SEED_LIMIT).toPandas()
+        seed_rows = []
+        for i, record in enumerate(snapshot.to_dict("records")):
+            op = cdf.MergeRecordType.DELETE if i % 2 else cdf.MergeRecordType.UPSERT
+
+            cleaned = {k: _clean_for_json(v) for k, v in record.items()}
+            seed_rows.append(
+                {
+                    **cleaned,
+                    cdf.COMMIT_VERSION: i + 1,
+                    cdf.COMMIT_TIMESTAMP: datetime.now(timezone.utc).isoformat(),
+                    cdf.MERGE_RECORD_TYPE: op.value,
+                }
+            )
+        (out_dir / "000_seed.json").write_text(
+            "\n".join(json.dumps(r, default=str) for r in seed_rows)
+        )
+        return True