@grepr/cli 1.12.7 → 1.14.3-5fb972a

package/build/dist/docs-index/7b1cb971-fd70-45e1-a1f1-26c71999fde5.txt

@@ -1,117 +0,0 @@
-Learn how to optimize your Google Cloud observability with Grepr and the Google Pub/Sub to Datadog feature.
-
-# Google Cloud Platform observability with Grepr
-
-To enhance your Google Cloud Platform (GCP) observability, you can stream GCP logs to Grepr, process and transform the logs using a Grepr pipeline, then forward the processed logs to any Grepr-supported observability platform. The Grepr integration with GCP uses Google's `Pub/Sub to Datadog` Dataflow template with a configured Grepr Datadog integration, and requires only minimal configuration changes to route logs through a Grepr pipeline. To learn more about the `Pub/Sub to Datadog` feature, see [Pub/Sub to Datadog template](https://cloud.google.com/dataflow/docs/guides/templates/provided/pubsub-to-datadog).
-
-## Requirements
-
-> The integration with GCP requires a Datadog integration configured in Grepr, including a valid Datadog API key. However, when you configure a Grepr pipeline using this integration, you can use any Grepr-supported sink. For example, a pipeline that sources GCP logs with a Datadog integration can use a Splunk integration as a sink.
-
-To integrate Grepr with GCP and Datadog, you must have:
-
-* A Datadog API key for your account. You use the same API key in your Grepr integration and the GCP component configurations.
-
-* A Datadog integration configured in Grepr. See [Configure Datadog connections with a Grepr integration](/integrations/datadog).
-
-* A Grepr ingestion URL. This URL is available when you create a Grepr pipeline using the Datadog integration. To learn how to create a Grepr pipeline, see the [Build your first Grepr pipeline](/tutorials/first-pipeline) tutorial.
-
-* Completed the configuration of the GCP to Datadog infrastructure. See [Stream logs from Google Cloud to Datadog](https://cloud.google.com/architecture/partners/stream-cloud-logs-to-datadog). Make sure you have:
-
-  * A Pub/Sub topic and subscription for collecting logs and sending them to Grepr.
-  * A GCP Cloud Logging sink configured to route logs to your Pub/Sub topic.
-  * Correct GCP IAM permissions.
-
-## Configure the GCP and Datadog integration with Grepr
-
-To configure the integration, you modify the Google `Pub/Sub to Datadog` Dataflow template to send logs to Grepr instead of directly to Datadog:
-
-1. In the GCP Console, go to **Dataflow > Jobs > Create Job From Template**.
-2. Select **Pub/Sub to Datadog** from the available templates.
-3. In **Required Parameters**, enter the following value:
-   * `Datadog Logs API URL: <your-grepr-ingestion-url>`
-4. In **Optional Parameters**, enter the following value:
-   * `Logs API key: <your-datadog-api-key>`
-
-> The `Logs API key` parameter in your Dataflow job must exactly match the Datadog API key configured in your Grepr Datadog integration. Mismatched API keys will cause authentication failures and prevent log delivery.
-
-### Example configuration
-
-```yaml
-# Standard Datadog configuration:
-url: https://http-intake.logs.datadoghq.com/api/v2/logs
-apiKey: <datadog-api-key>
-
-# Modified for Grepr:
-url: https://<integration-id>-<org-id>.ingest.grepr.ai
-apiKey: <same-datadog-api-key>
-```
-
-To get the Grepr ingestion URL, go to your pipeline's source configuration page in the Grepr UI and click **Copy ingest URL**.
-
-## Verifying the integration
-
-After deploying your Dataflow job:
-
-1. **Check Dataflow Status**: Verify the job is running without errors in the GCP Console.
-2. **Monitor Grepr Pipeline**: In the Grepr UI, confirm your pipeline shows active log ingestion and processing metrics.
-3. **Verify Datadog Delivery**: Check that processed logs are arriving in Datadog with tags applied by Grepr. For example, `processor:grepr`.
-
-## Troubleshooting
-
-### Common issues
-
-* **Authentication Errors**: Verify the Datadog API key matches between the Grepr integration and the Dataflow job.
-* **Connection Failures**: Ensure Dataflow workers can reach the Grepr ingestion endpoint.
-* **Missing Logs**: Verify the Cloud Logging sink is properly routing logs to your Pub/Sub topic.
-
-### Monitoring
-
-Monitor the health of the integration with:
-
-* GCP Dataflow job metrics and logs.
-* Grepr pipeline throughput and processing metrics.
-* Datadog log intake monitoring.
-* Pub/Sub subscription metrics for backlog monitoring.
-
-## Frequently Asked Questions
-
-### How do I stream logs from Google Cloud Platform to Grepr?
-
-Use the Google Pub/Sub to Datadog Dataflow template configured with Grepr. Set up a Cloud Logging sink to route logs to a Pub/Sub topic. Then create a Dataflow job using the Pub/Sub to Datadog template, but configure it to send logs to your Grepr ingestion URL instead of directly to Datadog.
-
-### What authentication is required for GCP integration with Grepr?
-
-You need a Datadog API key that is configured in both your Grepr Datadog integration and your GCP Dataflow job. The API key must be identical in both places. Additionally, you need appropriate GCP IAM permissions to create Dataflow jobs and access Pub/Sub topics.
-
-### What is the Pub/Sub to Datadog Dataflow template?
-
-It is a Google-provided template that processes logs from a Pub/Sub topic and forwards them to Datadog. Grepr uses this template by configuring it to send logs to Grepr's ingestion endpoint instead, allowing you to process GCP logs through Grepr before final delivery.
-
-### Do I need to use Datadog as my sink when using GCP integration with Grepr?
-
-You must configure a Datadog integration in Grepr to use the GCP integration, but you can use any Grepr-supported sink for processed logs. For example, you can ingest GCP logs via Datadog integration and send processed logs to Splunk, New Relic, or other platforms.
-
-### How do I configure the Pub/Sub to Datadog Dataflow job for Grepr?
-
-In GCP Console, create a Dataflow job from the Pub/Sub to Datadog template. Set the Datadog Logs API URL to your Grepr ingestion URL (format: https\://\<integration-id>-\<org-id>.ingest.grepr.ai) and the Logs API key to your Datadog API key. Ensure the key matches your Grepr Datadog integration.
-
-### What happens if the Datadog API key doesn't match between Grepr and Dataflow?
-
-If the API keys don't match, authentication will fail and logs will not be delivered. The mismatch prevents the Dataflow job from authenticating with Grepr, causing the integration to break. Always verify that the API key is identical in both your Grepr integration and your Dataflow job configuration.
-
-### How do I set up the GCP infrastructure required for this integration?
-
-You need to set up a Pub/Sub topic and subscription for log collection, configure a Cloud Logging sink to route logs to the Pub/Sub topic, and ensure your GCP IAM permissions allow creating and managing Dataflow jobs. See Google's documentation on streaming logs to Datadog for detailed infrastructure setup.
-
-### How can I reduce costs using GCP logs with Grepr?
-
-Process logs through Grepr pipelines to filter, aggregate, and reduce log volume before final storage. Use Grepr's log reducer to identify and exclude redundant messages. This reduces the volume of data stored in your final observability platform, lowering your overall observability costs.
-
-### Can I transform GCP logs in Grepr before sending to Datadog?
-
-Yes, that is the main benefit of the integration. Logs flow from GCP Cloud Logging through Pub/Sub to Grepr via Dataflow. In Grepr, you can apply transformations, filtering, log reduction, and other processing before sending to Datadog or other sinks.
-
-### How do I monitor the GCP to Grepr integration?
-
-Monitor the Dataflow job status in GCP Console for errors and processing metrics. In Grepr, check your pipeline's ingestion metrics to verify logs are arriving. Check your Dataflow job logs for any authentication or processing errors. Verify processed logs are appearing in your final sink platform.

package/build/dist/docs-index/7b1d93cf-18e4-4466-92f6-70440fc412a6.json

@@ -1 +0,0 @@
-{"docType":"schema"}

package/build/dist/docs-index/7b1d93cf-18e4-4466-92f6-70440fc412a6.txt

@@ -1,46 +0,0 @@
-# SpansBackfillIcebergTableSource
-
-**Type**: `object`
-
-## Properties
-
-- name: datasetId (required)
-  type: string
-  description: The ID of the dataset to read data from.
-
-- name: end (required)
-  type: string
-  description: End of time interval to query data for.
-
-- name: limit (optional)
-  type: integer
-  description: The maximum number of rows to process
-
-- name: name (required)
-  type: string
-
-- name: offset (optional)
-  type: integer
-  description: Offset on the number of rows to process
-
-- name: query (required)
-  type: `query`
-
-- name: sortOrder (optional)
-  type: enum: `ASCENDING`, `DESCENDING`, `UNSORTED`
-  description: The order in which the rows should be sorted by
-
-- name: start (required)
-  type: string
-  description: Start of time interval to query data for.
-
-- name: type (required)
-  type: enum: `spans-backfill-iceberg-table-source`
-  description: Reads spans from an Iceberg table for backfill, skipping spans already sent to the given vendor sinks (idempotent backfill source).
-
-- name: variables (optional)
-  type: object
-  description: Variables that can be used to modify the query while parsed
-
-- name: vendorSinkIntegrationIds (optional)
-  type: array of string

package/build/dist/docs-index/7b51330d-0afd-4de7-9dc4-25daa9f91857.json

@@ -1 +0,0 @@
-{"docType":"doc"}

package/build/dist/docs-index/7b51330d-0afd-4de7-9dc4-25daa9f91857.txt

@@ -1,125 +0,0 @@
-Connect a model provider with a Grepr LLM integration, and cap how many tokens your agents consume per time window.
-
-# Configure LLM provider connections with a Grepr integration
-
-An LLM integration holds the credentials Grepr uses to call a large language model provider. Grepr AI agents can use LLM integrations for two purposes: a generative model used for reasoning, or an embedding model used for similarity analysis.
-
-An LLM integration can be assigned a token budget, which is a mechanism that caps the amount of tokens your agents can spend. When several agents share one integration, the budget applies to all of them together.
-
-> LLM integrations, along with the rest of the Grepr AI agent features, are in a closed beta. To request access, contact <support@grepr.ai>.
-
-## Requirements
-
-To create an LLM integration, you need:
-
-* The credential your provider requires, which is an API key for most providers, or a gateway URL and a key for the gateway.
-
-## Choose a provider \[#providers]
-
-An LLM integration serves one of two purposes, so create the one that matches what you need.
-
-For the model an agent reasons with, create an integration for your model provider. You can also create a gateway integration to route your agents through a gateway you run, which reaches whatever models you configure it with, so use one to choose the model yourself, apply your own routing rules, or keep model traffic inside your own infrastructure.
-
-For the embedding model that agent memory uses, create an integration for a provider that supplies embeddings. One integration can serve both purposes, so a single integration is often enough to run an agent with memory turned on.
-
-The **Type** menu in the **Add LLM** dialog lists the providers available to you.
-
-Don't see your provider in the list? Send a request to <support@grepr.ai>.
-
-## Create an LLM integration \[#create]
-
-1. On the **Integrations** page, next to the **LLMs** section heading, click **Add new**.
-2. In the **Add LLM** dialog, select the provider from the **Type** menu.
-3. Complete the remaining fields, which depend on the provider you selected.
-4. Click **Create**.
-
-You cannot change an integration's type after you create it. To move to a different provider, create a new integration and point your agents at it.
-
-Select the tab for your provider to see the fields it requires:
-
-| Field | Description |
-| --- | --- |
-| **Name** | A name for the integration, used to identify it when you configure an agent. |
-| **API Key** | Your Anthropic API key. |
-| **Max tokens per window** | Under **Advanced**. The total tokens, counting input, cached input, and output tokens, that agent investigations using this integration can consume in each window. Leave empty for the default of 200,000,000. This is a safety net, not a spending cap: for firmer cost control, set a spend limit with your provider, on the project or workspace the API key belongs to or on the LiteLLM virtual key. |
-| **Max tokens window (hours)** | Under **Advanced**. How often the token count starts over. Windows align to UTC and must divide 24 (1, 2, 3, 4, 6, 8, 12, or 24). Leave empty for the default of 1 hour. |
-
-An Anthropic integration serves as an agent's model. Anthropic does not supply embeddings, so an agent with memory turned on also needs a Gemini or OpenAI integration for its embedding model.
-
-| Field | Description |
-| --- | --- |
-| **Name** | A name for the integration, used to identify it when you configure an agent. |
-| **API Key** | Your Gemini API key. |
-| **Max tokens per window** | Under **Advanced**. The total tokens, counting input, cached input, and output tokens, that agent investigations using this integration can consume in each window. Leave empty for the default of 200,000,000. This is a safety net, not a spending cap: for firmer cost control, set a spend limit with your provider, on the project or workspace the API key belongs to or on the LiteLLM virtual key. |
-| **Max tokens window (hours)** | Under **Advanced**. How often the token count starts over. Windows align to UTC and must divide 24 (1, 2, 3, 4, 6, 8, 12, or 24). Leave empty for the default of 1 hour. |
-
-A Gemini integration can serve as both an agent's model and the embedding model for agent memory.
-
-| Field | Description |
-| --- | --- |
-| **Name** | A name for the integration, used to identify it when you configure an agent. |
-| **Gateway URL** | The base URL of your LiteLLM proxy, such as `https://litellm.example.com/team-a`. Required. |
-| **Virtual key** | The LiteLLM virtual key Grepr authenticates with. |
-| **Validate virtual key when saving** | Turn this on to have Grepr verify the virtual key against the gateway before it saves the integration. |
-| **Max tokens per window** | Under **Advanced**. The total tokens, counting input, cached input, and output tokens, that agent investigations using this integration can consume in each window. Leave empty for the default of 200,000,000. This is a safety net, not a spending cap: for firmer cost control, set a spend limit with your provider, on the project or workspace the API key belongs to or on the LiteLLM virtual key. |
-| **Max tokens window (hours)** | Under **Advanced**. How often the token count starts over. Windows align to UTC and must divide 24 (1, 2, 3, 4, 6, 8, 12, or 24). Leave empty for the default of 1 hour. |
-
-Grepr reaches whatever models your gateway exposes, so the model names available to an agent are the ones your LiteLLM configuration routes.
-
-| Field | Description |
-| --- | --- |
-| **Name** | A name for the integration, used to identify it when you configure an agent. |
-| **API Key** | Your OpenAI API key. |
-
-An OpenAI integration provides the embedding model that agent memory uses.
-
-## Cap token consumption with a budget window \[#max-tokens]
-
-An integration that serves an agent's model carries a token budget, which is what caps the spend of every agent sharing it. The budget has two settings:
-
-* **Max tokens per window** is the total tokens every agent sharing the integration can consume in one window.
-* **Max tokens window (hours)** is the length of that window. The value must divide 24 evenly, so the valid lengths are 1, 2, 3, 4, 6, 8, 12, and 24 hours. Windows are aligned to UTC.
-
-Before it starts an investigation, Grepr reserves the tokens that investigation is allowed to consume and checks the reservation against two budgets: the integration's budget, and the agent's per-window budget. Both must have room. An agent whose own budget is untouched still waits when the shared integration has none left, so when investigations stop starting, check which of the two is exhausted on the agent's page before you raise a limit. To learn more about the agent-level settings, see [_](/ai-agents/create-an-agent#limits).
-
-## Update an API key \[#update-key]
-
-To replace a key, click the three-dot menu on the integration card, select **Edit**, enter the new key, and click **Save**. Every agent using the integration picks up the new key automatically.
-
-## Grant team access \[#team-access]
-
-When you have teams in your Grepr organization, you can grant access to the integration to one or more of those teams. To configure team access when you create or edit an integration:
-
-1. In the **Grant Team Access** section, click the plus sign icon (+).
-
-2. Select the team or teams that should be granted access to the integration.
-
-3. Click **Create**.
-
-To learn more, see [Manage teams in the Grepr platform](/admin/manage-teams) and [Permissions in the Grepr platform](/grepr-platform/permissions).
-
-## View token usage \[#usage]
-
-Each LLM integration card on the **Integrations** page charts the tokens its agents consumed, broken down by token type, and shows how much of the budget window remains. The card totals every agent sharing the integration and lists which agents those are, so to see how much one agent is spending, open that agent's page.
-
-## Frequently Asked Questions
-
-### Which LLM providers can a Grepr AI agent use as its model?
-
-Create an LLM integration for your model provider, or a gateway integration to route your agents through a gateway you run, which reaches whatever models you configure it with. Use a gateway to choose the model yourself, apply your own routing rules, or keep model traffic inside your own infrastructure. The Type menu in the Add LLM dialog lists the providers available to you, and Grepr adds integrations continuously, so contact support\@grepr.ai to get connected to the provider you use.
-
-### Which LLM providers can Grepr use for agent memory embeddings?
-
-Create an LLM integration for a provider that supplies embeddings. One integration can serve as both an agent's reasoning model and its embedding model, so a single integration is often enough to run an agent with memory turned on.
-
-### How does Grepr cap the tokens my AI agents consume?
-
-An LLM integration that serves an agent's model carries a max-tokens budget and a window length in hours that must divide 24 evenly. Grepr reserves an investigation's tokens up front and requires room in both the shared integration budget and the agent's own per-window budget before the investigation starts.
-
-### Can I change an LLM integration's provider after creating it?
-
-No. The type is fixed when you create the integration. To move to a different provider, create a new integration and point your agents at it.
-
-### Why have my agent's investigations stopped starting?
-
-The most common cause is an exhausted token budget. Because several agents can share one LLM integration, the shared budget can run out while the agent's own budget is untouched. The agent's page reports which of the two blocked the investigation and when the window resets.

package/build/dist/docs-index/7bc8ba36-e470-40f9-8a05-6a24db67edc3.json

@@ -1 +0,0 @@
-{"docType":"doc"}

package/build/dist/docs-index/7bc8ba36-e470-40f9-8a05-6a24db67edc3.txt

@@ -1,264 +0,0 @@
-Learn how the Grepr SQL transform supports the processing and transformation of log events in a Grepr pipeline.
-
-# Transform events with SQL
-
-You can use the SQL transform to process data with SQL statements. With support for multiple statement types and complex data flows, the SQL transform gives you a powerful and flexible way to process, filter, and enrich data using a familiar SQL syntax.
-
-The SQL transform complies with the ANSI SQL standard. It includes the full capabilities of Apache Flink SQL 2.2.0, including access to a comprehensive set of built-in functions and operators for data processing. This document describes a subset of the available capabilities. To learn more about Flink SQL, see [SQL](https://nightlies.apache.org/flink/flink-docs-release-2.2/docs/dev/table/sql/overview/) in the Flink 2.2.0 documentation.
-
-To learn how to use the SQL transform in the Grepr UI or the REST API, see [How do I use the SQL transform?](#sql-transform-usage).
-
-## When should I use the SQL transform?
-
-The SQL transform is ideal when you need to perform complex data transformations, filtering, or enrichment using a declarative SQL syntax, including:
-
-* Data normalization and standardization
-
-  * Message format standardization: Converting various log formats to a standard structure.
-  * Timestamp normalization: Converting different timestamp formats to standard milliseconds.
-  * Field standardization: Ensuring consistent field names and types across services.
-
-* Log enhancement and enrichment
-
-  * Computed metrics: Adding calculated fields like processing delays and error rates.
-  * Categorization: Adding severity levels, priority flags, or classification labels.
-  * Contextual data: Extracting and promoting key information from log messages.
-
-* Security and compliance
-
-  * Data redaction: Removing or masking sensitive information (passwords, tokens, PII).
-  * Audit trail enhancement: Adding tracking fields for compliance requirements.
-  * Anomaly flagging: Marking unusual patterns or suspicious activities.
-
-* Performance and monitoring
-
-  * SLA monitoring: Extracting response times and performance metrics.
-  * Error analysis: Categorizing and enriching error logs for better debugging.
-  * Trend analysis: Preparing log data for time-series analysis and alerting.
-
-* Data pipeline optimization
-
-  * Selective processing: Filtering logs to reduce downstream processing load.
-  * Format conversion: Converting logs for compatibility with downstream systems.
-  * Batch preparation: Aggregating and preparing data for batch analytics.
-
-## How does the SQL transform process data? \[#sql-transform-processing]
-
-The SQL transform processes data through one or more SQL statements. You specify SQL statements that define the data structure after processing, and the SQL transform processes the input data to transform it into those structures. You can configure the transform to send the results of running the SQL statement to either:
-
-* A `view`, which is a temporary table for intermediate results. Subsequent statements can then reference the `view` during their processing.
-* An `output` object, which converts the results of a view statement into a Flink DataStream object that downstream statements can consume. Each `output` must be assigned a unique name.
-
-`view` and `output` statements are defined as part of a configuration that also includes details such as:
-
-* Any `views` created by upstream statements that contain input data for the statement.
-* Tables to reference during processing that are part of a data lake dataset.
-
-When the SQL transform runs, it validates that:
-
-* All `views` created by the transform use unique names.
-* At least one `output` object exists to produce results, and all `output` names in the transform are unique.
-
-## How do I use the SQL transform? \[#sql-transform-usage]
-
-You use SQL transforms by adding them to your Grepr pipelines. To add a SQL transform to a pipeline in the Grepr UI, add a SQL action to one of the pipeline action locations. See [Process events with SQL](/transforms/filter-events#sql).
-
-To add a transform in the REST API, you define the SQL transform as part of the pipeline's job graph using a [SqlOperation](/apis/api-spec#schema/SqlOperation). The SqlOperation has the following schema:
-
-```json
-{
-  "name": "<operation_name>",
-  "type": "sql-operation",
-  "inputs": {
-    "<input-tablename-1>": "<LOG_EVENT|VARIANT|COMPLETE_SPAN>",
-    "<input-tablename-2>": "<LOG_EVENT|VARIANT|COMPLETE_SPAN>",
-    "...": "..."
-  },
-  "statements": [
-    {
-      "type": "sql_view",
-      "tableName": "<output-object-name>",
-      "sqlQuery": "<sql-statement>",
-      "shouldMaterialize": boolean
-    },
-    {
-      "type": "sql_output",
-      "outputName": "<output-stream-name>",
-      "outputType": "<LOG_EVENT|VARIANT|COMPLETE_SPAN>",
-      "sqlQuery": "<sql-statement>"
-    }
-  ],
-  "availableDatasets": [
-    "<dataset-identifier-1>",
-    "<dataset-identifier-2>",
-    "..."
-  ],
-  "globalStateTtl": "<time-to-live-duration>",
-  "watermarkDelay": "<watermark-delay-duration>"
-}
-```
-
-| Field | Type | Description |
-| --- | --- | --- |
-| `name` | string | A name identifying the SQL transform. This field is required. |
-| `type` | string | The type of operation. For the SQL transform, this value is always `sql-operation`. This field is required. |
-| `inputs` | object | Key-value pairs where the key is an input table name and the value is the data type. Each input table corresponds to an input stream that is connected to this operation in the job graph, and the type specifies the input's schema. Each input appears as a table that can be queried in SQL statements. |
-| `statements` | array | One or more SQL statements to run. Multiple statements run in the same order that you define them. Statements are either `view` statements or `output` statements. This field is required. See [How does the SQL transform process data?](#sql-transform-processing). |
-| `availableDatasets`| array | Identifiers for datasets containing tables that the statements in this transform configuration can use. |
-| `globalStateTtl` | string | The global state time-to-live. When running stateful streaming jobs, any unused state that is not used after this time is dropped to reduce memory pressure. |
-| `shouldMaterialize`| boolean | Whether to materialize a query's result set. Use this option to force downstream transforms to reuse query results instead of re-running the query. |
-
-To learn about the SQL transform input and output types, see [SQL transform supported data types](/transforms/sql-transform/data-types).
-
-The following example of a SQL transform configuration creates a view named `error_logs` that filters for input log events with a severity of 13 or higher. It then creates an output stream named `critical_errors` that selects log events from the `error_logs` view where the message contains the string "CRITICAL":
-
-```json
-{
-  "name": "example_sql_operation",
-  "type": "sql-operation",
-  "inputs": {
-    "logs": "LOG_EVENT"
-  },
-  "statements": [
-    {
-      "type": "sql_view",
-      "tableName": "error_logs",
-      "sqlQuery": "SELECT * FROM logs WHERE severity >= 13",
-      "shouldMaterialize": false
-    },
-    {
-      "type": "sql_output",
-      "outputName": "critical_errors",
-      "outputType": "LOG_EVENT",
-      "sqlQuery": "SELECT * FROM error_logs WHERE message LIKE '%CRITICAL%'"
-    }
-  ],
-  "availableDatasets": [],
-  "globalStateTtl": "PT2M",
-  "watermarkDelay": "PT30S"
-}
-```
-
-## Performance considerations
-
-The following are best practices for optimizing query performance when using the SQL transform.
-
-### Query optimization
-
-* Use `WHERE` clauses early to filter data and reduce the resources required by complex queries.
-* Avoid `SELECT *` when possible. Instead, explicitly list the columns required for your processing.
-* Because they are CPU-intensive, limit the use of regular expressions on high-volume streams.
-
-The following is an inefficient query. This query applies a regular expression to all log messages, even when the message doesn't contain relevant content:
-
-```sql
-SELECT *,
-  REGEXP_EXTRACT(message, '(\\\\d{4}-\\\\d{2}-\\\\d{2})', 1) as date_extracted
-FROM logs
-```
-
-A better approach is to filter the logs first using a `WHERE` clause, and then apply the regular expression to the filtered logs:
-
-```sql
-SELECT *,  -- Only processes logs that likely contain dates
-  -- Regex only runs on pre-filtered logs (much more efficient)
-  REGEXP_EXTRACT(message, '(\\\\d{4}-\\\\d{2}-\\\\d{2})', 1) as date_extracted
-FROM logs
-WHERE message LIKE '%-%-%'  -- Quick pre-filter using simple pattern matching
-```
-
-Removing the `SELECT *` and explicitly listing only the required fields can further improve performance for this query.
-
-### Memory and state management
-
-* Computed fields increase the size of events and might impact downstream processing.
-* Complex aggregations require more memory. If necessary, reduce time-to-live values to reduce memory usage.
-* Because window operations buffer data, memory requirements increase with larger windows.
-* Because JSON operations allocate temporary objects, you should avoid using them in tight loops.
-
-### Resource usage patterns
-
-When you write SQL queries, consider the typical resource usage for common operations:
-
-* String operations, such as UPPER, LOWER, and CONCAT, have low CPU overhead.
-* Regular expression operations, such as REGEXP_REPLACE and REGEXP_EXTRACT, have high CPU overhead.
-* JSON operations, such as JSON_VALUE and JSON_QUERY, have moderate CPU overhead.
-* Mathematical operations have low CPU overhead.
-* Window aggregations have high memory usage during window buffering.
-
-## Error handling
-
-The SQL transform fails if:
-
-* The SQL syntax is invalid. Malformed SQL queries are rejected during parsing.
-* Field types are incompatible. For example, the standard fields in a log event must match their expected types when you include them in a query.
-* Runtime errors occur when running a query. For example, division by zero or an invalid regular expression.
-* JSON parsing errors occur, such as running invalid JSON operations on the attributes field.
-
-## Limitations
-
-### Streaming vs batch processing
-
-The SQL transform processes both streaming and batch data. Compared to batch processing, streaming has the following limitations:
-
-* `JOIN` operations are limited.
-* Complex aggregations or joins might require state management. Your pipeline can fail if the amount of state used for aggregations or joins becomes too large. You can manage the amount of state using the time-to-live configuration setting (`globalStateTtl`) in the SQL transform configuration. By default, `globalStateTtl` is set to 2 minutes.
-
-Windowing limitations:
-
-* Window functions require using windowing table-valued functions (TVFs), such as `TUMBLE`, `HOP`, `SESSION`, or `CUMULATE`.
-* You cannot use unbounded aggregations. Instead, use windowing for aggregations in streaming mode.
-
-Data ordering:
-
-* Events might arrive out of order. Consider using watermarks for time-based operations.
-* `ROW_NUMBER()` and ranking functions might not guarantee consistent ordering across restarts.
-
-## Next steps
-
-* To learn about using the SQL transform in the Grepr pipeline UI, see [Filter and transform events with actions](/transforms/filter-events).
-* To learn about supported data types for the SQL transform, see [SQL transform supported data types](/transforms/sql-transform/data-types).
-* To see examples of using the SQL transform, see [SQL transform examples](/transforms/sql-transform/sql-transform-examples).
-* To learn about using supported SQL functions and operators, see [SQL transform functions and operators](/transforms/sql-transform/sql-functions).
-
-## Frequently Asked Questions
-
-### What is the SQL transform and what can it do?
-
-The SQL transform is a Grepr component that processes log events using ANSI SQL statements, including Apache Flink SQL. It supports data normalization, log enrichment, security and compliance operations like data redaction, performance monitoring, and pipeline optimization using familiar SQL syntax.
-
-### When should I use the SQL transform?
-
-Use SQL transform for complex data transformations like format standardization, timestamp normalization, field enrichment with computed metrics, data redaction for security, anomaly flagging, SLA monitoring, error categorization, and selective filtering to reduce downstream processing.
-
-### How do I create a SQL view versus a SQL output?
-
-A view is a temporary table for intermediate results that subsequent statements can reference. An output converts the results of a view into a Flink DataStream object that downstream statements or pipeline sinks can consume. Each output must have a unique name and at least one output must exist in your SQL transform.
-
-### Can I create multiple SQL statements in a single transform?
-
-Yes. The SQL transform supports multiple SQL statements that run in the order you define them. Each statement can reference views created by previous statements or input tables, allowing you to build complex multi-stage data processing pipelines.
-
-### What SQL standard does Grepr support?
-
-The SQL transform complies with ANSI SQL standard and includes full capabilities of Apache Flink SQL 2.2.0. It supports SELECT, CREATE VIEW, GROUP BY, JOIN, window functions, and a comprehensive set of built-in functions and operators for data processing.
-
-### How do I preserve original fields when transforming with SQL?
-
-Include the wildcard character (\*) in your SELECT statement to retain all fields from the original event. For example: SELECT \*, CASE WHEN severity > 16 THEN 21 ELSE severity END as severity FROM logs keeps all original fields while transforming severity.
-
-### How do I optimize SQL transform performance?
-
-Filter data early with WHERE clauses before complex operations, avoid SELECT \* when possible by listing only required columns, limit regex operations on high-volume streams, pre-filter before applying CPU-intensive operations like regex extraction, and materialize views when they are reused in multiple subsequent queries.
-
-### What are views and when should I use shouldMaterialize?
-
-Views are temporary tables containing intermediate results. Use shouldMaterialize: true when a view is referenced in multiple subsequent SQL statements to force results to be reused instead of re-running the query, improving performance for complex transformations.
-
-### How do I handle late-arriving data in SQL transforms?
-
-Use the watermarkDelay parameter in your SQL transform configuration to handle out-of-order data. Events that arrive later than the specified watermark delay are handled according to your Flink SQL configuration, typically being dropped or assigned to the nearest window.
-
-### What happens if my SQL query fails?
-
-The SQL transform fails if SQL syntax is invalid, field types are incompatible with schema expectations, runtime errors occur like division by zero, or JSON parsing errors happen. The pipeline will error and you should review the SQL statements and input data types to correct the issue.

package/build/dist/docs-index/7bf0a3db-c218-47b3-9d81-58074f76ee44.json

@@ -1 +0,0 @@
-{"docType":"doc"}