tdsql-mcp 1.4.2__tar.gz → 1.4.4__tar.gz

{tdsql_mcp-1.4.2 → tdsql_mcp-1.4.4}/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.4
+Metadata-Version: 2.5
 Name: tdsql-mcp
-Version: 1.4.2
+Version: 1.4.4
 Summary: MCP server for Teradata Vantage — SQL execution and native analytics function reference for AI agents
 Project-URL: Homepage, https://github.com/ksturgeon-td/tdsql-mcp
 Project-URL: Repository, https://github.com/ksturgeon-td/tdsql-mcp
@@ -19,7 +19,7 @@ Classifier: Programming Language :: Python :: 3.12
 Classifier: Topic :: Database
 Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
 Requires-Python: >=3.10
-Requires-Dist: mcp>=1.0.0
+Requires-Dist: mcp<2.0.0,>=1.0.0
 Requires-Dist: python-dotenv>=1.0.0
 Requires-Dist: teradatasql>=20.0.0
 Description-Content-Type: text/markdown

{tdsql_mcp-1.4.2 → tdsql_mcp-1.4.4}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "tdsql-mcp"
-version = "1.4.2"
+version = "1.4.4"
 description = "MCP server for Teradata Vantage — SQL execution and native analytics function reference for AI agents"
 readme = "README.md"
 requires-python = ">=3.10"
@@ -26,7 +26,7 @@ classifiers = [
     "Topic :: Scientific/Engineering :: Artificial Intelligence",
 ]
 dependencies = [
-    "mcp>=1.0.0",
+    "mcp>=1.0.0,<2.0.0",
     "teradatasql>=20.0.0",
     "python-dotenv>=1.0.0",
 ]

{tdsql_mcp-1.4.2 → tdsql_mcp-1.4.4}/skills/teradata-sql-analytics/SKILL.md

@@ -31,6 +31,7 @@ If you know a native function exists but haven't loaded its syntax topic, say so
 
 **Topic triggers — load these immediately when the task involves:**
 - Apache Iceberg, Delta Lake, Open Table Format, OTF tables, DATALAKE objects, three-tier notation (`datalake.db.table`) → read [syntax/open-table-format.md](syntax/open-table-format.md)
+- Iceberg/Delta Lake catalog metadata, list datalakes, list OTF tables, DBC.DatalakeInfoV, DBC.ManagedOTFTablesV, DBC.OtfStatsV → read [syntax/catalog-views.md](syntax/catalog-views.md)
 - S3, Azure Blob, GCS, object store, READ_NOS, WRITE_NOS, CREATE FOREIGN TABLE, NOS foreign tables → read [syntax/object-store.md](syntax/object-store.md)
 - `describe_table` does not work for OTF or foreign tables — use `HELP TABLE` passed through `execute_query` instead
 

{tdsql_mcp-1.4.2 → tdsql_mcp-1.4.4}/skills/teradata-sql-analytics/syntax/authorization-objects.md

@@ -8,10 +8,11 @@ Rather than embedding API keys or access tokens directly in a function call, you
 
 ## CREATE / REPLACE AUTHORIZATION
 
-Two forms are supported: standard user/password credentials, and IAM role assumption (AWS only).
+Three forms are supported: standard user/password credentials, credentials with a DEFINER or INVOKER execution context, and IAM role assumption (AWS only).
 
 ```sql
 { CREATE | REPLACE } AUTHORIZATION [DatabaseName.]authorization_name
+    [ AS { DEFINER | INVOKER } TRUSTED ]
     { user_password_auth | extended_auth }
 
 -- Form 1: user_password_auth (all providers)
@@ -26,6 +27,32 @@ EXTERNALID 'external_id_value'
 [ DURATION_SECONDS 'duration_in_seconds' ]
 ```
 
+**`AS DEFINER` vs `AS INVOKER` vs `TRUSTED`:**
+
+| Clause | Meaning |
+|--------|---------|
+| `AS DEFINER` | Shared access — usable by multiple users of the database. Can be created in any database. |
+| `AS INVOKER` | Exclusive access by the creating user. Typically must be created in the current user's own database; exact privilege requirements may vary. |
+| `TRUSTED` | Required when the auth object is referenced in an `EXTERNAL SECURITY` clause (CREATE FOREIGN TABLE, CREATE FUNCTION MAPPING). |
+
+> **EXTERNAL SECURITY — the reference must match how the auth object was created:**
+>
+> | Auth created as | `EXTERNAL SECURITY` reference | Qualified `db.auth` allowed? |
+> |---|---|---|
+> | Plain (no `AS` clause) | `EXTERNAL SECURITY <db>.<auth>` (no keywords) | **Yes** — verified for DATALAKE and FOREIGN TABLE |
+> | `AS DEFINER TRUSTED` | `EXTERNAL SECURITY DEFINER TRUSTED <auth>` | **No** — Error 3706; auth must be in the same DB as the object |
+> | `AS INVOKER TRUSTED` | `EXTERNAL SECURITY INVOKER TRUSTED <auth>` | No (per syntax docs; not verified on EC) |
+>
+> A mismatch between the auth creation form and the EXTERNAL SECURITY reference raises
+> Error 6953 (`authorization definition does not match`).
+>
+> **Elastic Compute:** For DATALAKE objects, the auth object must be in a **GLOBAL database**
+> so it replicates across all CE instances alongside the datalake. For NOS foreign tables,
+> the auth object can be in the same local or global database as the table.
+>
+> **GRANT EXECUTE:** After creating an auth object, grant `EXECUTE` at the database level
+> so roles can use it. See the [Permissions](#permissions) section below.
+
 **Form 1 — user_password_auth:**
 - **`CREATE`** — creates a new authorization object; fails if it already exists
 - **`REPLACE`** — creates or replaces; use this to rotate credentials without dropping first
@@ -47,7 +74,7 @@ The three fields (`USER`, `PASSWORD`, `SESSION_TOKEN`) map to different provider
 
 | Provider | USER | PASSWORD | SESSION_TOKEN |
 |----------|------|----------|---------------|
-| **AWS Bedrock** | AccessKey | SecretKey | SessionKey *(optional)* |
+| **AWS Bedrock / S3** | AccessKey | SecretKey | SessionToken *(optional — for STS temporary credentials only)* |
 | **Azure** | ApiBase (endpoint URL) | ApiKey | ApiVersion *(required)* |
 | **Google Cloud (GCP)** | Project | Region | AccessToken *(required)* |
 | **NVIDIA NIM** | ApiBase (endpoint URL) | ApiKey | *(not used)* |
@@ -66,11 +93,50 @@ The three fields (`USER`, `PASSWORD`, `SESSION_TOKEN`) map to different provider
 ## Examples
 
 ```sql
--- AWS Bedrock (SessionKey optional — omit for long-term credentials)
+-- NOS / OTF: DEFINER TRUSTED (shared service account — all users get same access)
+CREATE AUTHORIZATION mydb.s3_definer_auth
+    AS DEFINER TRUSTED
+    USER     '{AWS_ACCESS_KEY}'
+    PASSWORD '{AWS_SECRET_KEY}';
+-- Optional: add SESSION_TOKEN for temporary AWS credentials (STS-issued)
+--   SESSION_TOKEN '{AWS_SESSION_TOKEN}'
+
+-- NOS / OTF: INVOKER TRUSTED (per-invoker runtime check)
+CREATE AUTHORIZATION mydb.s3_invoker_auth
+    AS INVOKER TRUSTED
+    USER     '{AWS_ACCESS_KEY}'
+    PASSWORD '{AWS_SECRET_KEY}';
+
+-- How they appear in EXTERNAL SECURITY clauses:
+-- (NOS foreign table — auth must be in the SAME database as the table;
+--  reference by UNQUALIFIED name only — qualified db.auth raises Error 3706)
+CREATE AUTHORIZATION mydb.s3_definer_auth        -- same db as the table
+    AS DEFINER TRUSTED
+    USER '{AWS_ACCESS_KEY}' PASSWORD '{AWS_SECRET_KEY}';
+
+CREATE MULTISET FOREIGN TABLE mydb.orders_ft,
+    EXTERNAL SECURITY DEFINER TRUSTED s3_definer_auth   -- unqualified name
+    USING ( LOCATION ('/S3/s3.amazonaws.com/my-bucket/') STOREDAS ('PARQUET') )
+NO PRIMARY INDEX;
+
+-- (OTF DATALAKE — plain auth in a GLOBAL database; no keywords in EXTERNAL SECURITY)
+CREATE AUTHORIZATION global_db.s3_auth
+    USER '{AWS_ACCESS_KEY}' PASSWORD '{AWS_SECRET_KEY}';
+
+CREATE DATALAKE my_lake
+    EXTERNAL SECURITY CATALOG global_db.s3_auth,   -- no DEFINER/TRUSTED keywords
+    EXTERNAL SECURITY STORAGE global_db.s3_auth
+USING catalog_type ('glue') ... TABLE FORMAT iceberg;
+```
+
+### AI Function Authorization (no AS DEFINER/INVOKER)
+
+```sql
+-- AWS Bedrock (SESSION_TOKEN optional — include only for STS-issued temporary credentials)
 CREATE AUTHORIZATION db.td_gen_aws_auth
     USER     '{AWS_ACCESS_KEY}'
     PASSWORD '{AWS_SECRET_KEY}'
-    SESSION_TOKEN '{AWS_SESSION_TOKEN}';
+    SESSION_TOKEN '{AWS_SESSION_TOKEN}'; -- omit for long-term IAM user credentials
 
 -- Azure (SESSION_TOKEN = ApiVersion — required)
 CREATE AUTHORIZATION db.td_gen_azure_auth

tdsql_mcp-1.4.4/skills/teradata-sql-analytics/syntax/catalog-views.md

@@ -0,0 +1,257 @@
+# Teradata Catalog Views (DBC.*)
+
+DBC views expose the Teradata data dictionary. Always use the `V` (view) variants
+(e.g. `DBC.TablesV`) over the base tables — they enforce access control properly.
+
+## Databases & Users
+```sql
+-- List all accessible databases
+SELECT DatabaseName, DBKind, CommentString
+FROM DBC.DatabasesV
+ORDER BY DatabaseName;
+-- DBKind: 'D' = database, 'U' = user
+
+-- List users
+SELECT UserName, DefaultDatabase, CreateDate
+FROM DBC.UsersV
+ORDER BY UserName;
+```
+
+## Tables & Views
+```sql
+-- All tables in a database
+SELECT TableName, TableKind, CreateTimeStamp, LastAlterTimeStamp
+FROM DBC.TablesV
+WHERE DatabaseName = 'mydb'
+ORDER BY TableKind, TableName;
+-- TableKind: 'T'=table, 'V'=view, 'O'=NoPI table, 'Q'=queue, 'E'=error, 'I'=join index
+
+-- Search for tables by name pattern
+SELECT DatabaseName, TableName, TableKind
+FROM DBC.TablesV
+WHERE TableName LIKE '%customer%'
+ORDER BY DatabaseName, TableName;
+
+-- View definition (DDL text)
+SELECT RequestText FROM DBC.TablesV
+WHERE DatabaseName = 'mydb' AND TableName = 'my_view';
+```
+
+## Columns
+```sql
+-- All columns for a table
+SELECT ColumnId, ColumnName, ColumnType, ColumnLength,
+       Nullable, DefaultValue, ColumnFormat, CommentString
+FROM DBC.ColumnsV
+WHERE DatabaseName = 'mydb' AND TableName = 'mytable'
+ORDER BY ColumnId;
+
+-- Find tables that contain a specific column
+SELECT DatabaseName, TableName, ColumnName, ColumnType
+FROM DBC.ColumnsV
+WHERE ColumnName = 'customer_id'
+ORDER BY DatabaseName, TableName;
+
+-- Find columns matching a pattern across all databases
+SELECT DatabaseName, TableName, ColumnName
+FROM DBC.ColumnsV
+WHERE ColumnName LIKE '%email%'
+ORDER BY DatabaseName, TableName;
+```
+
+## Indexes & Primary Index
+```sql
+-- Indexes on a table
+SELECT IndexName, IndexType, UniqueFlag, ColumnName, ColumnPosition
+FROM DBC.IndicesV
+WHERE DatabaseName = 'mydb' AND TableName = 'mytable'
+ORDER BY IndexType, ColumnPosition;
+-- IndexType: 'P'=primary, 'S'=secondary, 'U'=unique, 'J'=join index
+```
+
+## Table Size & Storage
+```sql
+-- Table sizes in a database (from collected stats — no full scan)
+SELECT TableName,
+       SUM(CurrentPerm) / 1024 / 1024 AS size_mb,
+       SUM(PeakPerm) / 1024 / 1024 AS peak_mb
+FROM DBC.TableSizeV
+WHERE DatabaseName = 'mydb'
+GROUP BY TableName
+ORDER BY size_mb DESC;
+
+-- Database space summary
+SELECT DatabaseName,
+       SUM(MaxPerm) / 1024 / 1024 / 1024     AS max_gb,
+       SUM(CurrentPerm) / 1024 / 1024 / 1024  AS used_gb
+FROM DBC.DiskSpaceV
+GROUP BY DatabaseName
+ORDER BY used_gb DESC;
+```
+
+## Statistics
+```sql
+-- Collected statistics on a table
+SELECT ColumnName, StatsType, LastCollectTimeStamp, SampleSize
+FROM DBC.StatsV
+WHERE DatabaseName = 'mydb' AND TableName = 'mytable'
+ORDER BY ColumnName;
+```
+
+## Access Rights
+```sql
+-- What access does the current user have on a database?
+SELECT AccessRight, DatabaseName, TableName
+FROM DBC.UserRightsV
+WHERE UserName = USER
+ORDER BY DatabaseName, TableName;
+```
+
+## Database Hierarchy
+```sql
+-- List direct children of a parent database (Elastic Compute: global databases)
+-- DBC.ChildrenV columns are Child and Parent (NOT DatabaseName / ParentName)
+SELECT Child AS DatabaseName FROM DBC.ChildrenV WHERE Parent = 'TD_GLOBAL';
+
+-- Find all databases that are children of TD_PARENT (local/user databases)
+SELECT Child AS DatabaseName FROM DBC.ChildrenV WHERE Parent = 'TD_PARENT';
+```
+
+## Common Lookup Patterns
+```sql
+-- Fully qualified table info in one query
+SELECT
+    c.DatabaseName,
+    c.TableName,
+    t.TableKind,
+    c.ColumnName,
+    c.ColumnType,
+    c.Nullable
+FROM DBC.ColumnsV c
+JOIN DBC.TablesV t
+    ON t.DatabaseName = c.DatabaseName AND t.TableName = c.TableName
+WHERE c.DatabaseName = 'mydb'
+ORDER BY c.TableName, c.ColumnId;
+```
+
+---
+
+## Open Table Format (OTF) Catalog Views
+
+Use these views to discover and inspect Iceberg tables, Delta Lake tables, datalake objects, and OTF statistics registered in Teradata Vantage. These cover metadata that cannot be queried via `DBC.TablesV` or `DBC.ColumnsV`.
+
+**Keyword index:** Iceberg, Delta Lake, Open Table Format, OTF, datalake, external catalog, Hive metastore, AWS Glue, Databricks Unity Catalog, Polaris, Gravitino, Lake Formation, OTF statistics, managed tables, external servers, `HELP DATALAKE`, `HELP DATABASE`, `HELP TABLE`.
+
+These views supplement the HELP commands — see `open-table-format` topic for full DDL/DML syntax and HELP command usage.
+
+### DBC.DatalakeInfoV — Registered Datalakes (Iceberg / Delta Lake catalogs)
+
+> **Access note:** On some systems, `DBC.DatalakeInfoV` raises **Error 3523** (not accessible)
+> even for `TD_CREATOR` / `TD_ADMIN` users. If you get Error 3523 or an empty result set,
+> fall back to `DBC.ServerV` and `SHOW DATALAKE <name>` for discovery.
+
+```sql
+SELECT DatalakeName, OTFTableFormat, CatalogType, CatalogLocation,
+       StorageLocation, StorageEndPoint, StorageRegion,
+       UnityCatalogName, StorageAccountName
+FROM DBC.DatalakeInfoV
+ORDER BY DatalakeName;
+-- String values include embedded single quotes, e.g. CatalogType returns 'glue' not glue
+```
+
+Columns include: `DatalakeName`, `OTFTableFormat` (ICEBERG/DELTA), `CatalogType`
+(hive/glue/unity/rest/fabric/biglake), `CatalogLocation` (catalog URL / AWS region),
+`StorageLocation`, `StorageEndPoint`, `StorageRegion`, `UnityCatalogName`, `StorageAccountName`.
+
+Once a datalake name is known, get its full DDL (auth names, catalog config):
+```sql
+SHOW DATALAKE <datalake_name>;
+```
+
+### DBC.ServerV — External Servers (primary discovery path)
+
+Use `DBC.ServerV` as the primary fallback when `DBC.DatalakeInfoV` is inaccessible or
+returns no rows. It lists all registered foreign servers, including OTF datalakes.
+
+```sql
+SELECT ServerName, DataBaseName, AuthorizationName, AuthorizationType,
+       CatalogAuthName, TableFormat
+FROM DBC.ServerV
+ORDER BY ServerName;
+-- DataBaseName: the database this server is registered in (usually TD_SERVER_DB)
+-- TableFormat: ICEBERG, DELTA, or blank for non-OTF servers
+-- AuthorizationType: may appear blank on some deployments
+```
+
+> **Note:** `DBC.ServerV` does not have `ServerType` or `CommentString` columns.
+
+### DBC.ManagedOTFTablesV — Managed OTF Tables
+
+```sql
+SELECT DatalakeName, DatabaseName, TableName, TableFormat,
+       CreateTimeStamp, LastAlterTimeStamp
+FROM DBC.ManagedOTFTablesV
+ORDER BY DatalakeName, DatabaseName, TableName;
+```
+
+### DBC.OtfStatsV — OTF Statistics
+
+```sql
+SELECT DatabaseName, TableName, ColumnName, StatsType,
+       LastCollectTimeStamp, SampleSize
+FROM DBC.OtfStatsV
+WHERE DatabaseName = 'my_lake'
+ORDER BY TableName, ColumnName;
+```
+
+### DBC.AllStatsV — All Statistics (Teradata + OTF)
+
+```sql
+SELECT DatabaseName, TableName, ColumnName, StatsType,
+       LastCollectTimeStamp
+FROM DBC.AllStatsV
+WHERE DatabaseName = 'mydb'
+ORDER BY TableName;
+```
+
+`DBC.StatsV` covers only relational tables. Use `DBC.AllStatsV` when you need a single view across both table types.
+
+---
+
+## Alternate OTF Discovery Path — TD_SERVER_DB and HELP FOREIGN SERVER
+
+`TD_SERVER_DB` is a system database that holds all foreign server registrations, including DATALAKE objects and QueryGrid foreign servers. Use this path when `DBC.DatalakeInfoV` returns no rows or you need to enumerate all foreign servers to find which ones are OTF datalakes.
+
+### Step 1 — List all foreign servers
+
+```sql
+HELP DATABASE TD_SERVER_DB;
+```
+
+Returns a row per object. Rows with `Kind = 'K'` are foreign servers. These can be either QueryGrid foreign servers or DATALAKE objects — the next step distinguishes them.
+
+### Step 2 — Inspect a foreign server
+
+```sql
+HELP FOREIGN SERVER TD_SERVER_DB.<foreign_server_name>;
+```
+
+- If the object is an **OTF DATALAKE**, this returns the databases inside it — equivalent to `HELP DATALAKE <datalake_name>`.
+- If it is a QueryGrid foreign server, the output will reflect that server's structure instead.
+
+### Full discovery workflow
+
+```sql
+-- Step 1: find all foreign server entries
+HELP DATABASE TD_SERVER_DB;
+-- Look for rows where Kind = 'K'
+
+-- Step 2: for each Kind='K' entry, inspect it
+HELP FOREIGN SERVER TD_SERVER_DB.my_lake_name;
+-- If OTF: returns list of databases inside the datalake
+-- Then use HELP DATABASE and HELP TABLE to drill down:
+HELP DATABASE my_lake_name.my_otf_database;
+HELP TABLE my_lake_name.my_otf_database.my_table;
+```
+
+> **When to use this path:** Use `TD_SERVER_DB` + `HELP FOREIGN SERVER` when the user asks about available external data, Iceberg catalogs, or datalake objects and you do not already know the datalake name. Start here to enumerate what exists, then use `HELP DATALAKE` / `HELP DATABASE` / `HELP TABLE` to drill into specifics.

{tdsql_mcp-1.4.2 → tdsql_mcp-1.4.4}/skills/teradata-sql-analytics/syntax/guidelines.md

@@ -6,6 +6,11 @@ Teradata Vantage has built-in distributed table operators for most analytics, ML
 
 **Before writing any SQL for analytics, transformation, or ML: check this guide and the relevant syntax topic.**
 
+> **This server is reference only — it does not execute SQL.** It serves syntax and native-function
+> documentation; running a query, an EXPLAIN, or any DDL/DML is the job of whatever SQL client or other
+> MCP server you have connected. So the guidance below says *what to run and why*, and never names a tool
+> on this server to run it with.
+
 ---
 
 ## Minimize Data Movement — Critical at Teradata Scale
@@ -27,13 +32,19 @@ SELECT * FROM TD_ColumnSummary(
 ) AS t;
 ```
 
-**2. Use SAMPLE or TOP N when you need to see rows**
+**2. Use SHOW TABLE / SHOW SELECT to inspect structure; TOP N only for sample rows**
 
 ```sql
--- Explore structure and values without touching the full table
+-- Confirm column names and types — no data scan
+SHOW TABLE db.large_table;
+SHOW SELECT * FROM db.my_view;
+
+-- Sample rows when you need actual data values (structure already known)
 SELECT TOP 100 * FROM db.large_table SAMPLE 0.001;
 ```
 
+`TOP N` on a large table can perform badly even when N is small. Use `SHOW TABLE` or `SHOW SELECT *` for structure checks.
+
 **3. Filter before you function — push predicates as deep as possible**
 
 ```sql
@@ -87,9 +98,9 @@ SELECT * FROM TD_XGBoost(
 ) AS t;
 ```
 
-**6. Use execute_query with small max_rows for validation only**
+**6. Return rows only to validate, never to process**
 
-`execute_query` is for checking results, previewing schemas, and validating output — not for analytics. Default `max_rows=100` exists for this reason. If you find yourself wanting to increase `max_rows` significantly to "process" data, that is a signal to use a native function instead.
+Fetching rows is for checking results, previewing a schema, and validating output — not for analytics. Cap it small; a hundred rows is plenty. If you find yourself wanting to raise that cap substantially in order to "process" the data, that is the signal to use a native function instead and let the database do the work.
 
 **7. Aggregate before returning — let the database do the grouping**
 
@@ -117,6 +128,49 @@ Native functions distribute across all AMPs. The result set returned to the agen
 
 ## Common Operations → Native Function Mapping
 
+### Schema & Data Discovery
+
+When users ask about available data — especially external, Iceberg, or datalake sources — use in-database discovery commands and DBC views rather than querying data directly. The table below maps common user intents to the right discovery path and topic.
+
+| User intent | Discovery approach | Topic |
+|-------------|-------------------|-------|
+| "What databases / schemas exist?" | `SELECT DatabaseName FROM DBC.DatabasesV` | `catalog-views` |
+| "What tables are in database X?" | `SELECT TableName, TableKind FROM DBC.TablesV WHERE DatabaseName = 'X'` | `catalog-views` |
+| "What columns does table X have?" | `SELECT ColumnName, ColumnType FROM DBC.ColumnsV WHERE ...` | `catalog-views` |
+| "What Iceberg / Delta Lake / OTF sources are registered?" | `SELECT * FROM DBC.DatalakeInfoV` — if empty, fall back to `HELP DATABASE TD_SERVER_DB` + `HELP FOREIGN SERVER TD_SERVER_DB.<name>` | `catalog-views`, `open-table-format` |
+| "What datalakes are available?" | Same as above — `DBC.DatalakeInfoV` first, then TD_SERVER_DB path | `catalog-views`, `open-table-format` |
+| "What external / foreign servers exist?" | `HELP DATABASE TD_SERVER_DB` — rows with `Kind = 'K'` are foreign servers | `catalog-views` |
+| "What Iceberg tables are in datalake X?" | `HELP DATALAKE X` (known name) — or `HELP FOREIGN SERVER TD_SERVER_DB.X` (unknown name) | `open-table-format`, `catalog-views` |
+| "What tables are in OTF database X.Y?" | `HELP DATABASE X.Y` | `open-table-format` |
+| "What columns does OTF table X.Y.Z have?" | `HELP TABLE X.Y.Z` | `open-table-format` |
+| "What NOS / object-store foreign tables exist?" | `SELECT TableName FROM DBC.TablesV WHERE DatabaseName = 'X' AND TableKind = 'O'` | `catalog-views`, `object-store` |
+| "What QueryGrid foreign servers are there?" | `HELP DATABASE TD_SERVER_DB` — rows with `Kind = 'K'` that are not OTF datalakes | `catalog-views` |
+
+**Schema inspection — use SHOW, not SELECT TOP:**
+
+```sql
+-- WRONG: can perform very badly on large tables; avoid for structure inspection
+SELECT TOP 1 * FROM db.large_table;
+
+-- RIGHT: SHOW TABLE returns DDL (column names, types, constraints) — no data scan
+SHOW TABLE db.large_table;
+
+-- RIGHT: SHOW SELECT returns the resolved column list for a view or derived query
+SHOW SELECT * FROM db.my_view;
+SHOW SELECT * FROM db.large_table;
+```
+
+Use `SHOW TABLE` or `SHOW SELECT *` whenever the goal is to confirm structure, not data. Reserve `TOP N` for when you actually need sample rows.
+
+**OTF discovery decision tree:**
+1. Try `DBC.DatalakeInfoV` — lists all registered datalakes when the user has view access.
+2. If empty or access denied, try `HELP DATABASE TD_SERVER_DB` → find `Kind = 'K'` rows → `HELP FOREIGN SERVER TD_SERVER_DB.<name>` for each.
+3. Once a datalake name is known, use `HELP DATALAKE <name>` / `HELP DATABASE <datalake>.<db>` / `HELP TABLE <datalake>.<db>.<table>` to drill in.
+
+Load `get_syntax_help(topic="catalog-views")` for DBC view syntax and the full TD_SERVER_DB workflow. Load `get_syntax_help(topic="open-table-format")` for HELP DATALAKE syntax and OTF DDL/DML.
+
+---
+
 ### Data Exploration & Statistics
 
 | Instead of this (manual SQL) | Use this (native function) | Topic |
@@ -203,6 +257,12 @@ Native functions distribute across all AMPs. The result set returned to the agen
 | External approximate nearest neighbor index | `TD_HNSW` / `TD_HNSWPredict` | `vector-search` |
 | External embedding storage type | `VECTOR` / `Vector32` data type | `data-types-casting` |
 
+**Inline NL Query → Embedding → Vector Search:** For RAG retrieval, use the full CTE pipeline pattern — embed the query inline with `AI_TEXTEMBEDDINGS`, normalize with `TD_VectorNormalize(Approach('UNITVECTOR'))`, and search with `TD_VectorDistance` against a pre-built corpus embedding table, all in a single SQL statement. See `vector-search` topic, "Inline NL Query → Embedding → Vector Search Pipeline" section.
+
+**Building a corpus embedding table:** Use the full workflow: source text table → `AI_TEXTEMBEDDINGS` → `TD_VectorNormalize(Approach('UNITVECTOR'))` → CTAS. See `vector-search` topic, "Full Corpus Build Workflow" section.
+
+**Vector dimension introspection:** Use `embedding.LENGTH()` to get the number of dimensions from a VECTOR column — never infer from UDT byte size. `SELECT embedding.LENGTH() AS dims FROM db.table SAMPLE 1;`
+
 ### Statistical Testing
 
 | Instead of this | Use this (native function) | Topic |
@@ -278,16 +338,16 @@ See the `ml-patterns` topic for complete end-to-end ML pipeline examples. See `u
 
 ## Query Validation and Optimization with EXPLAIN
 
-For any non-trivial query, run `explain_query` before executing. Read the plan and optimize if needed — do not just use EXPLAIN for syntax checking.
+For any non-trivial query, run `EXPLAIN` on it before executing. Read the plan and optimize if needed — do not use EXPLAIN merely as a syntax check.
 
 **When to always EXPLAIN first:**
 - Queries joining two or more large tables
 - Queries without a PI-aligned filter (likely full table scan)
 - Queries with subqueries, correlated subqueries, or complex predicates
-- Any query you're about to run with `execute_statement` that modifies data
+- Any statement you are about to run that modifies data (INSERT, UPDATE, DELETE, MERGE, DDL)
 
 **Decision loop:**
-1. Run `explain_query(sql=...)`
+1. Run `EXPLAIN <your statement>`
 2. Scan for red flags (see below) — if found, fix and re-EXPLAIN before executing
 3. Only execute once the plan looks reasonable
 
@@ -312,27 +372,6 @@ For full EXPLAIN interpretation guidance, optimization playbook, and stats colle
 
 ---
 
-## External Data Access
-
-### Open Table Format (OTF) — Iceberg and Delta Lake
-
-For working with Apache Iceberg or Delta Lake tables in external catalogs (Hive, Glue, Unity, REST):
-- Load `get_syntax_help(topic='open-table-format')` before writing any OTF DDL or DML
-- OTF tables use three-tier dot notation: `datalake.database.table`
-- **HELP TABLE, not describe_table:** `DBC.ColumnsV` does not cover OTF tables. Use `HELP TABLE my_lake.db.table;` passed through `execute_query` to inspect OTF table columns
-- **HELP commands are first-class statements:** `HELP DATALAKE`, `HELP DATABASE`, `HELP TABLE` query the external catalog directly — do not rewrite them as SELECT queries against DBC views
-- For CREATE/ALTER/DROP TABLE and DML (INSERT/UPDATE/DELETE) on OTF tables, follow the syntax exactly — OTF has significant restrictions vs. relational SQL
-
-### Native Object Store (NOS) — S3, Azure, GCS
-
-For ad-hoc object store access (READ_NOS), persistent access (CREATE FOREIGN TABLE), or export (WRITE_NOS):
-- Load `get_syntax_help(topic='object-store')` before writing any NOS SQL
-- **HELP TABLE, not describe_table:** `DBC.ColumnsV` does not cover foreign tables. Use `HELP TABLE mydb.foreign_table;` or `READ_NOS(USING ... RETURNTYPE('NOSREAD_SCHEMA'))` to discover schema
-- For import workflows, use the foreign table → CAST view → permanent table pattern documented in the `object-store` topic
-- Collect statistics on payload attributes used in joins or filters to improve query plans against foreign tables
-
----
-
 ## When Manual SQL Is Appropriate
 
 Native functions do not cover everything. Use hand-written SQL for:
@@ -341,7 +380,7 @@ Native functions do not cover everything. Use hand-written SQL for:
 - Date/time arithmetic (`date-time`)
 - CASE expressions and NULL handling (`conditional`)
 - Window functions for lag/lead features, running totals (`window-functions`)
-- Schema discovery via MCP tools first — `list_databases`, `list_tables`, `describe_table` cover the common cases; fall back to manual DBC.* queries only for capabilities not covered by those tools (`catalog-views`)
+- Schema discovery queries against DBC.* views (`catalog-views`)
 - Bit/byte manipulation — `BITAND`, `BITOR`, `BITXOR`, `BITNOT`, `SHIFTLEFT`/`SHIFTRIGHT`, `ROTATELEFT`/`ROTATERIGHT`, `GETBIT`, `SETBIT`, `COUNTSET`, `SUBBITSTR`, `TO_BYTE` — no ANSI equivalents; do not use `&`, `|`, `^`, `~` operators (`bit-byte-functions`)
 - JSON data — native `JSON` type with BSON/UBJSON binary formats; JSONPath extraction; shredding and publishing — all in-database (`json-functions`)
 - One-off computations not covered by any native function

{tdsql_mcp-1.4.2 → tdsql_mcp-1.4.4}/skills/teradata-sql-analytics/syntax/index.md

@@ -38,10 +38,10 @@ Use `get_syntax_help(topic="<name>")` to load any topic below.
 | `data-cleaning` | NULL handling, deduplication, string cleaning, outlier detection, type validation |
 | `data-prep` | Feature engineering: binning, encoding, scaling, pivoting, polynomial features, dimensionality reduction, Fit/Transform pairs, TD_SMOTE oversampling |
 | `utility-functions` | TD_FillRowID, TD_NumApply, TD_RoundColumns, TD_StrApply |
-| `text-analytics` | Text tokenization, classification, and entity extraction: TD_NgramSplitter, TD_NaiveBayesTextClassifier, TD_NERExtractor |
+| `text-analytics` | Text tokenization, classification, tagging, and entity extraction: TD_NgramSplitter, TD_TextParser, TD_TextTagger, TD_TFIDF, TD_SentimentExtractor, TD_WordEmbeddings, TD_NaiveBayesTextClassifier, TD_TextMorph, TD_POSTagger, TD_NERExtractor |
 | `hypothesis-testing` | Statistical hypothesis tests: TD_ANOVA, TD_ChiSq, TD_FTest, TD_ZTest |
 | `association-analysis` | Frequent itemset mining and collaborative filtering: TD_Apriori, TD_CFilter |
-| `path-analysis` | Event sequence analysis: Attribution, Sessionize, nPath |
+| `path-analysis` | Event sequence analysis: Attribution, Sessionize, nPath, TD_PathSummarizer |
 | `model-evaluation` | Model evaluation and explainability: TD_TrainTestSplit, TD_ClassificationEvaluator, TD_RegressionEvaluator, TD_ROC, TD_Silhouette, TD_SHAP |
 | `ml-patterns` | End-to-end ML pipeline patterns: CTE prediction pipeline, elbow method, train/evaluate/retrain loop, class imbalance workflow, micromodeling |
 | `vector-search` | Vector similarity search: TD_VectorDistance (exact), TD_HNSW/TD_HNSWPredict (approximate), KMeans IVF pattern |
@@ -100,6 +100,7 @@ Recommended topic reading order for common end-to-end tasks. Load these topics i
 | **Micromodeling (per-segment models)** | `ml-functions` (TD_GLM) → `ml-patterns` (micromodeling) |
 | **Semantic search / RAG embeddings** | `authorization-objects` → `llm-providers` → `embeddings` → `vector-search` |
 | **In-database ONNX inference** | `byom-model-loading` → `embeddings` (ONNXEmbeddings) → `vector-search` |
+| **Discover available Iceberg / Delta Lake / OTF sources** | `catalog-views` (DBC.DatalakeInfoV + TD_SERVER_DB section) → `open-table-format` |
 | **Query Iceberg / Delta Lake tables** | `open-table-format` |
 | **Import NOS data into Vantage** | `object-store` (foreign table → CAST view → permanent table) |
 | **Export Vantage data to S3/Azure/GCS** | `object-store` (WRITE_NOS) |
@@ -109,5 +110,8 @@ Recommended topic reading order for common end-to-end tasks. Load these topics i
 | **Digital signal filtering** | `uaf-concepts` → `uaf-utility` (TD_FILTERFACTORY1D) → `uaf-dsp` (TD_CONVOLVE) |
 
 ---
-> **Adding topics:** Drop a new `.md` file into `src/tdsql_mcp/syntax/` and it appears here
-> automatically — no code changes needed.
+> **Adding topics:** Drop a new `.md` file into the matching category subfolder
+> (`core/`, `functions/`, `analytics/`, `uaf/`, `geospatial/`, `reference/`). The
+> filename (without `.md`) is the topic slug — it **must be globally unique
+> across subfolders**. Run `bash scripts/check-unique-slugs.sh` before pushing.
+> Add a row to the matching table above so the LLM can discover it.