@mastra/oracledb 0.0.0 → 0.2.0-alpha.0

package/dist/docs/references/reference-rag-metadata-filters.md

@@ -0,0 +1,227 @@
+> Discover all available pages from the documentation index: https://mastra.ai/llms.txt
+
+# Metadata filters
+
+Mastra provides a unified metadata filtering syntax across all vector stores, based on MongoDB/Sift query syntax. Each vector store translates these filters into its native query format. For example, PgVector uses PostgreSQL JSONB predicates, while OracleDB stores metadata as Oracle JSON and compiles filters to `JSON_VALUE`, `JSON_EXISTS`, `REGEXP_LIKE`, and `LIKE` predicates with bound values.
+
+## Basic example
+
+```typescript
+import { PgVector } from '@mastra/pg'
+
+const store = new PgVector({
+  id: 'pg-vector',
+  connectionString,
+})
+
+const results = await store.query({
+  indexName: 'my_index',
+  queryVector: queryVector,
+  topK: 10,
+  filter: {
+    category: 'electronics', // Simple equality
+    price: { $gt: 100 }, // Numeric comparison
+    tags: { $in: ['sale', 'new'] }, // Array membership
+  },
+})
+```
+
+## Supported operators
+
+### Basic Comparison
+
+`$eq`Matches values equal to specified value{ age: { $eq: 25 } }Supported by: All except Couchbase`$ne`Matches values not equal{ status: { $ne: 'inactive' } }Supported by: All except Couchbase`$gt`Greater than{ price: { $gt: 100 } }Supported by: All except Couchbase`$gte`Greater than or equal{ rating: { $gte: 4.5 } }Supported by: All except Couchbase`$lt`Less than{ stock: { $lt: 20 } }Supported by: All except Couchbase`$lte`Less than or equal{ priority: { $lte: 3 } }Supported by: All except Couchbase
+
+### Array Operators
+
+`$in`Matches any value in array{ category: { $in: \["A", "B"] } }Supported by: All except Couchbase`$nin`Matches none of the values{ status: { $nin: \["deleted", "archived"] } }Supported by: All except Couchbase`$all`Matches arrays containing all elements{ tags: { $all: \["urgent", "high"] } }Supported by: Astra, Pinecone, Upstash, MongoDB, OracleDB`$elemMatch`Matches array elements meeting criteria{ scores: { $elemMatch: { $gt: 80 } } }Supported by: libSQL, PgVector, MongoDB, OracleDB
+
+### Logical Operators
+
+`$and`Logical AND{ $and: \[{ price: { $gt: 100 } }, { stock: { $gt: 0 } }] }Supported by: All except Vectorize, Couchbase`$or`Logical OR{ $or: \[{ status: "active" }, { priority: "high" }] }Supported by: All except Vectorize, Couchbase`$not`Logical NOT{ price: { $not: { $lt: 100 } } }Supported by: Astra, Qdrant, Upstash, PgVector, libSQL, MongoDB, OracleDB`$nor`Logical NOR{ $nor: \[{ status: "deleted" }, { archived: true }] }Supported by: Qdrant, Upstash, PgVector, libSQL, MongoDB, OracleDB
+
+### Element Operators
+
+`$exists`Matches documents with field{ rating: { $exists: true } }Supported by: All except Vectorize, Chroma, Couchbase
+
+### Custom Operators
+
+`$contains`Text contains substring{ description: { $contains: "sale" } }Supported by: Upstash, libSQL, PgVector, OracleDB`$regex`Regular expression match{ name: { $regex: "^test" } }Supported by: Qdrant, PgVector, Upstash, MongoDB, OracleDB`$size`Array length check{ tags: { $size: 3 } }Supported by: Astra, libSQL, PgVector, MongoDB, OracleDB`$geo`Geospatial query{ location: { $geo: { type: "radius", ... } } }Supported by: Qdrant`$datetime`Datetime range query{ created: { $datetime: { range: { gt: "2024-01-01" } } } }Supported by: Qdrant`$hasId`Vector ID existence check{ $hasId: \["id1", "id2"] }Supported by: Qdrant`$hasVector`Vector existence check{ $hasVector: true }Supported by: Qdrant
+
+## Common rules and restrictions
+
+1. Field names can't:
+
+   - Contain dots (.) unless referring to nested fields
+   - Start with $ or contain null characters
+   - Be empty strings
+
+2. Values must be:
+
+   - Valid JSON types (string, number, boolean, object, array)
+   - Not undefined
+   - Properly typed for the operator (e.g., numbers for numeric comparisons)
+
+3. Logical operators:
+
+   - Must contain valid conditions
+   - Can't be empty
+   - Must be properly nested
+   - Can only be used at top level or nested within other logical operators
+   - Can't be used at field level or nested inside a field
+   - Can't be used inside an operator
+   - Valid: `{ "$and": [{ "field": { "$gt": 100 } }] }`
+   - Valid: `{ "$or": [{ "$and": [{ "field": { "$gt": 100 } }] }] }`
+   - Invalid: `{ "field": { "$and": [{ "$gt": 100 }] } }`
+   - Invalid: `{ "field": { "$gt": { "$and": [{...}] } } }`
+
+4. $not operator:
+
+   - Must be an object
+   - Can't be empty
+   - Can be used at field level or top level
+   - Valid: `{ "$not": { "field": "value" } }`
+   - Valid: `{ "field": { "$not": { "$eq": "value" } } }`
+
+5. Operator nesting:
+
+   - Logical operators must contain field conditions, not direct operators
+   - Valid: `{ "$and": [{ "field": { "$gt": 100 } }] }`
+   - Invalid: `{ "$and": [{ "$gt": 100 }] }`
+
+## Store-specific notes
+
+### Astra
+
+- Nested field queries are supported using dot notation
+- Array fields must be explicitly defined as arrays in the metadata
+- Metadata values are case-sensitive
+
+### ChromaDB
+
+- Where filters only return results where the filtered field exists in metadata
+- Empty metadata fields aren't included in filter results
+- Metadata fields must be present for negative matches (e.g., $ne won't match documents missing the field)
+
+### Cloudflare Vectorize
+
+- Requires explicit metadata indexing before filtering can be used
+- Use `createMetadataIndex()` to index fields you want to filter on
+- Up to 10 metadata indexes per Vectorize index
+- String values are indexed up to first 64 bytes (truncated on UTF-8 boundaries)
+- Number values use float64 precision
+- Filter JSON must be under 2048 bytes
+- Field names can't contain dots (.) or start with $
+- Field names limited to 512 characters
+- Vectors must be re-upserted after creating new metadata indexes to be included in filtered results
+- Range queries may have reduced accuracy with very large datasets (\~10M+ vectors)
+
+### libSQL
+
+- Supports nested object queries with dot notation
+- Array fields are validated to ensure they contain valid JSON arrays
+- Numeric comparisons maintain proper type handling
+- Empty arrays in conditions are handled gracefully
+- Metadata is stored in a JSONB column for efficient querying
+
+### OracleDB
+
+- Metadata is stored as Oracle JSON alongside each `VECTOR` row
+- Scalar comparisons use `JSON_VALUE`, while array, existence, and element-match checks use `JSON_EXISTS`
+- `$regex` uses Oracle `REGEXP_LIKE`; string `$contains` uses case-insensitive `LIKE`
+- Nested fields are supported with dot notation and are converted to quoted Oracle JSON paths
+- User-provided metadata values are bound as parameters instead of interpolated into SQL
+
+### PgVector
+
+- Full support for PostgreSQL's native JSON querying capabilities
+- Efficient handling of array operations using native array functions
+- Proper type handling for numbers, strings, and booleans
+- Nested field queries use PostgreSQL's JSON path syntax internally
+- Metadata is stored in a JSONB column for efficient indexing
+
+### Pinecone
+
+- Metadata field names are limited to 512 characters
+- Numeric values must be within the range of ±1e38
+- Arrays in metadata are limited to 64KB total size
+- Nested objects are flattened with dot notation
+- Metadata updates replace the entire metadata object
+
+### Qdrant
+
+- Supports advanced filtering with nested conditions
+- Payload (metadata) fields must be explicitly indexed for filtering
+- Use `createPayloadIndex()` to index fields you want to filter on:
+
+```typescript
+// Index a field before filtering on it
+await store.createPayloadIndex({
+  indexName: 'my_index',
+  fieldName: 'source',
+  fieldSchema: 'keyword', // 'keyword' | 'integer' | 'float' | 'geo' | 'text' | 'bool' | 'datetime' | 'uuid'
+})
+
+// Now filtering works
+const results = await store.query({
+  indexName: 'my_index',
+  queryVector: queryVector,
+  filter: { source: 'document-a' },
+})
+```
+
+- Efficient handling of geo-spatial queries
+- Special handling for null and empty values
+- Vector-specific filtering capabilities
+- Datetime values must be in RFC 3339 format
+
+### Upstash
+
+- 512-character limit for metadata field keys
+- Query size is limited (avoid large IN clauses)
+- No support for null/undefined values in filters
+- Translates to SQL-like syntax internally
+- Case-sensitive string comparisons
+- Metadata updates are atomic
+
+### MongoDB
+
+- Full support for MongoDB/Sift query syntax for metadata filters
+- Supports all standard comparison, array, logical, and element operators
+- Supports nested fields and arrays in metadata
+- Filtering can be applied to both `metadata` and the original document content using the `filter` and `documentFilter` options, respectively
+- `filter` applies to the metadata object; `documentFilter` applies to the original document fields
+- No artificial limits on filter size or complexity (subject to MongoDB query limits)
+- Indexing metadata fields is recommended for optimal performance
+
+### Couchbase
+
+- Currently doesn't have support for metadata filters. Filtering must be done client-side after retrieving results or by using the Couchbase SDK's Search capabilities directly for more complex queries.
+
+### Amazon S3 Vectors
+
+- Equality values must be primitives (string/number/boolean). `null`/`undefined`, arrays, objects, and Date aren't allowed for equality. Range operators accept numbers or Date (Dates are normalized to epoch ms).
+- `$in`/`$nin` require **non-empty arrays of primitives**; Date elements are allowed and normalized to epoch ms. **Array equality** isn't supported.
+- Implicit AND is canonicalized (`{a:1,b:2}` → `{$and:[{a:1},{b:2}]`). Logical operators must contain field conditions and use non-empty arrays. They may appear only at the root or within other logical operators (not inside field values).
+- Keys listed in `nonFilterableMetadataKeys` at index creation are stored but not filterable. This setting is immutable.
+- $exists requires a boolean value.
+- undefined/null/empty filters are treated as no filter.
+- Each metadata key name limited to 63 characters.
+- Total metadata per vector: Up to 40 KB (filterable + non-filterable)
+- Total metadata keys per vector: Up to 10
+- Filterable metadata per vector: Up to 2 KB
+- Non-filterable metadata keys per vector index: Up to 10
+
+## Related
+
+- [Astra](https://mastra.ai/reference/vectors/astra)
+- [Chroma](https://mastra.ai/reference/vectors/chroma)
+- [Cloudflare Vectorize](https://mastra.ai/reference/vectors/vectorize)
+- [libSQL](https://mastra.ai/reference/vectors/libsql)
+- [MongoDB](https://mastra.ai/reference/vectors/mongodb)
+- [OracleDB](https://mastra.ai/reference/vectors/oracledb)
+- [PgStore](https://mastra.ai/reference/vectors/pg)
+- [Pinecone](https://mastra.ai/reference/vectors/pinecone)
+- [Qdrant](https://mastra.ai/reference/vectors/qdrant)
+- [Upstash](https://mastra.ai/reference/vectors/upstash)
+- [Amazon S3 Vectors](https://mastra.ai/reference/vectors/s3vectors)

package/dist/docs/references/reference-storage-oracledb.md

@@ -0,0 +1,239 @@
+> Discover all available pages from the documentation index: https://mastra.ai/llms.txt
+
+# OracleDB storage
+
+The OracleDB storage provider stores Mastra application state in Oracle Database. It implements Mastra's composite storage interface, so one `OracleStore` instance can back memory, workflow snapshots, observability, scores, scorer definitions, MCP client metadata, and agent registry data.
+
+## Installation
+
+**npm**:
+
+```bash
+npm install @mastra/oracledb@latest
+```
+
+**pnpm**:
+
+```bash
+pnpm add @mastra/oracledb@latest
+```
+
+**Yarn**:
+
+```bash
+yarn add @mastra/oracledb@latest
+```
+
+**Bun**:
+
+```bash
+bun add @mastra/oracledb@latest
+```
+
+## Usage
+
+```ts
+import { OracleStore } from '@mastra/oracledb'
+
+const storage = new OracleStore({
+  id: 'oracle-storage',
+  user: process.env.ORACLE_DATABASE_USER,
+  password: process.env.ORACLE_DATABASE_PASSWORD,
+  connectString: process.env.ORACLE_DATABASE_CONNECT_STRING,
+})
+```
+
+Use it with Mastra:
+
+```ts
+import { Mastra } from '@mastra/core/mastra'
+
+export const mastra = new Mastra({
+  storage,
+})
+```
+
+## Parameters
+
+**id** (`string`): Unique identifier for this storage instance.
+
+**user** (`string`): Oracle Database user. Required unless using pool or externalAuth.
+
+**password** (`string`): Password for the Oracle Database user. Required unless using pool or externalAuth.
+
+**connectString** (`string`): Oracle connect string, service name, TNS alias, or Autonomous Database connect descriptor. Required unless using pool.
+
+**pool** (`oracledb.Pool`): Existing Oracle connection pool. When provided, Mastra uses the pool but doesn't close it when store.close() is called.
+
+**poolManager** (`OraclePoolManager`): Shared Oracle pool manager. Use this to share one Oracle pool between OracleStore and OracleVector.
+
+**schemaName** (`string`): Oracle schema name used to qualify storage tables.
+
+**poolMin** (`number`): Minimum number of Oracle pool connections. (Default: `0`)
+
+**poolMax** (`number`): Maximum number of Oracle pool connections. (Default: `4`)
+
+**poolIncrement** (`number`): Number of connections to add when the pool grows. (Default: `1`)
+
+**configDir** (`string`): Directory containing Oracle Network configuration files such as tnsnames.ora.
+
+**walletLocation** (`string`): Oracle wallet directory for mTLS connections such as Autonomous Database.
+
+**walletPassword** (`string`): Password for the Oracle wallet, when required by the wallet configuration.
+
+**externalAuth** (`boolean`): Use Oracle external authentication instead of username/password authentication.
+
+**disableInit** (`boolean`): When true, automatic schema initialization is disabled. Use this when schema changes are applied separately before the app starts. (Default: `false`)
+
+**messageBatchSize** (`number`): Number of messages sent per Oracle executeMany call when saving messages. The operation still commits once at the transaction boundary. (Default: `200`)
+
+**skipDefaultIndexes** (`boolean`): When true, default storage indexes aren't created during initialization.
+
+**indexes** (`OracleCreateIndexOptions[]`): Custom Oracle index definitions to create during initialization. Indexes are routed to the storage domain that owns the target table.
+
+**migrationTableName** (`string`): Oracle table used to track storage schema migrations. (Default: `'MASTRA_ORACLE_MIGRATIONS'`)
+
+**vectorRegistryTableName** (`string`): OracleVector registry table used to discover semantic-recall vector tables when threads or messages are deleted. Set this to match OracleVector's registryTableName when that option is customized.
+
+## Connection examples
+
+The basic username/password constructor is shown above. For Autonomous Database, add wallet options to the same constructor:
+
+```ts
+const storage = new OracleStore({
+  id: 'oracle-storage',
+  user: process.env.ORACLE_DATABASE_USER,
+  password: process.env.ORACLE_DATABASE_PASSWORD,
+  connectString: process.env.ORACLE_DATABASE_CONNECT_STRING,
+  walletLocation: process.env.ORACLE_DATABASE_WALLET_DIR,
+  walletPassword: process.env.ORACLE_DATABASE_WALLET_PASSWORD,
+  configDir: process.env.ORACLE_DATABASE_CONFIG_DIR,
+})
+```
+
+For external authentication, set `externalAuth: true` and omit `password`. To reuse an existing `oracledb.Pool`, pass it as `pool`. Mastra uses it but doesn't close it.
+
+`OracleStore` backs memory, workflow snapshots, observability, scores, scorer definitions, MCP client metadata, and agent registry data. When using the store outside a `Mastra` instance, call `await storage.init()` and access a domain with `await storage.getStore('memory')`.
+
+## Initialization
+
+When you pass `OracleStore` to `Mastra`, `init()` is called automatically before storage operations run. If you use `OracleStore` directly, call `init()` before reading or writing:
+
+```ts
+await storage.init()
+```
+
+> **Warning:** If initialization is disabled or skipped, storage operations require the Oracle tables and indexes to already exist.
+
+`OracleStore.init()` runs repeatable migrations and records the result in the migration ledger table. The default ledger table is `MASTRA_ORACLE_MIGRATIONS`.
+
+```ts
+await storage.migrate()
+const history = await storage.listMigrations()
+```
+
+Repeatable migrations are idempotent. They reconcile the tables and indexes owned by each storage domain on startup, which lets new domain indexes or compatible schema additions apply without changing application code.
+
+Initialization also creates the provider's default indexes for common Mastra query paths. Use `skipDefaultIndexes` when indexes are managed separately, or pass `indexes` for custom Oracle indexes. Custom definitions support Oracle options such as `bitmap`, `online`, `invisible`, `parallel`, `compress`, `noLogging`, and `reverse`, as well as function-based expressions like `JSON_VALUE(...)`.
+
+Custom indexes are useful when your app repeatedly filters on JSON metadata or when database administrators (DBAs) want to test an index before the optimizer uses it:
+
+```ts
+const storage = new OracleStore({
+  id: 'oracle-storage',
+  user,
+  password,
+  connectString,
+  indexes: [
+    {
+      name: 'idx_messages_status',
+      table: 'mastra_messages',
+      columns: [
+        "JSON_VALUE(metadata, '$.status' RETURNING VARCHAR2(32) NULL ON ERROR)",
+        'thread_id',
+      ],
+      online: true,
+      invisible: true,
+    },
+  ],
+})
+```
+
+Use `invisible` for staged rollout, then remove it after validating query plans. Use `skipDefaultIndexes: true` only when a DBA-managed indexing strategy replaces the defaults.
+
+Use `disableInit: true` when schema changes are applied by a separate deployment step or by a database administrator.
+
+## Schema export
+
+Use `exportSchemas()` to generate Oracle DDL without connecting to a database. This is useful when schema changes are reviewed or applied outside application startup.
+
+```ts
+import { exportSchemas } from '@mastra/oracledb'
+
+const ddl = exportSchemas({
+  schemaName: 'MASTRA_APP',
+  domains: [
+    'memory',
+    'workflows',
+    'observability',
+    'scores',
+    'scorerDefinitions',
+    'mcpClients',
+    'agents',
+  ],
+})
+
+console.log(ddl)
+```
+
+`domains` defaults to every supported domain, including `vector`, when omitted.
+
+## Operational notes
+
+Use the same `OraclePoolManager` when `OracleStore` and `OracleVector` should share one Oracle connection lifecycle:
+
+```ts
+import { OracleStore, OracleVector } from '@mastra/oracledb'
+
+const storage = new OracleStore({ id: 'oracle-storage', user, password, connectString })
+const vector = new OracleVector({
+  id: 'oracle-vector',
+  poolManager: storage.getPoolManager(),
+})
+```
+
+`OracleStore` exposes `storage.db` and `await storage.getPool()` for advanced use cases. When using these APIs directly, you're responsible for transaction boundaries and connection lifecycle.
+
+JSON metadata, payloads, and snapshots are stored in native Oracle JSON columns and encoded server-side, so the rows are readable directly with standard Oracle JDBC tools such as DBeaver and SQL Developer.
+
+## Usage example
+
+### Adding OracleDB memory to an agent
+
+```ts
+import { Agent } from '@mastra/core/agent'
+import { Memory } from '@mastra/memory'
+import { OracleStore } from '@mastra/oracledb'
+
+const storage = new OracleStore({
+  id: 'oracle-storage',
+  user: process.env.ORACLE_DATABASE_USER,
+  password: process.env.ORACLE_DATABASE_PASSWORD,
+  connectString: process.env.ORACLE_DATABASE_CONNECT_STRING,
+})
+
+export const oracleAgent = new Agent({
+  id: 'oracle-agent',
+  name: 'Oracle Agent',
+  instructions: 'You are an assistant with persistent OracleDB-backed memory.',
+  model: 'openai/gpt-5.6-sol',
+  memory: new Memory({ storage }),
+})
+```
+
+## Related
+
+- [OracleDB vector store](https://mastra.ai/reference/vectors/oracledb)
+- [Storage overview](https://mastra.ai/reference/storage/overview)
+- [Working memory](https://mastra.ai/docs/memory/working-memory)
+- [Workflow snapshots](https://mastra.ai/docs/workflows/snapshots)