@alvera-ai/platform-sdk 0.10.0-rc.2 → 0.10.0-rc.21

package/.agent/data_sources.md

@@ -0,0 +1,234 @@
+# Data Sources
+
+A **data source** declares an external system that the platform
+ingests data from — a managed file drop, an upload endpoint,
+a scheduled pull from an external SaaS, and so on. Each data
+source scopes to exactly one datalake. A tenant can own many
+data sources across many datalakes.
+
+Data sources are **Platform-DB-resident**: their metadata lives
+in the platform's tenant-wide DB, not in the per-datalake
+schemas. As a consequence, data sources can be created against
+a `new`-status datalake (no migration wait — see `datalakes.md`
+§5 Lifecycle "Platform-DB / Datalake-DB split").
+
+SDK namespace: `api.dataSources`.
+
+## 1. Wire shape
+
+```typescript
+import type {
+  DataSourceRequestWritable,
+  DataSourceResponse,
+} from '@alvera-ai/platform-sdk'
+
+const body: DataSourceRequestWritable = {
+  name: 'Vendor Inbound',
+  uri: 'api.vendor.example.com',
+  description: 'External vendor billing integration',
+  status: 'active',
+  is_default: false,
+}
+
+const { data: created } = await api.dataSources.create(
+  tenantSlug,
+  datalakeSlug,
+  body,
+)
+// created.id        — server-derived UUID
+// created.name      === body.name
+// created.uri       === body.uri
+// created.datalake  — references the parent datalake
+```
+
+The body is flat — no discriminator. Every data source carries
+the same five caller-supplied fields plus the universal server-
+derived set (`id`, `slug`, `created_at`, `updated_at`).
+
+## 2. Rules the type cannot encode
+
+### `uri` is a free-form endpoint identifier
+
+The `uri` field is typed as `string` but the platform doesn't
+enforce a strict URL grammar. Values observed include hostname
+patterns (`api.stripe.com`, `appLeadCaptureForms.airtable.com`)
+and host-with-port patterns (`12345.athenahealth.com`). The
+field is the agent-facing identifier the platform routes against
+when downstream resources (data activation clients, tools) bind
+to the source; pick a stable, identifiable value the operator
+can recognize.
+
+### `is_default` is per-datalake
+
+At most one data source per datalake carries `is_default: true`.
+Promoting a new source to default demotes the prior default
+(operator-mediated; the SDK doesn't expose explicit demotion).
+Most SDK consumers ship sources with `is_default: false`.
+
+### A default Manual Upload data source is auto-created per datalake
+
+When a datalake completes migration, the platform auto-creates
+a data source named `"Alvera Manual Upload - <datalake_name>"`
+with `uri: "<datalake_id>.alvera.ai"` and one bound Manual
+Upload tool. This row carries `is_default: true` and serves as
+the default sink for ad-hoc CSV / JSON uploads.
+
+The default row CAN be PUT-updated, but the auto-creation runs
+once at datalake migration. Filter on `is_default` when
+listing for management UIs that should show only consumer-
+created sources.
+
+### `status` is an enum
+
+The `status` field carries the source's lifecycle state. The
+wire enum has three values: `'draft' | 'active' | 'inactive'`.
+`'active'` is the typical create-time value; `'inactive'`
+short-circuits the source from new ingestion runs without
+removing it; `'draft'` is the unconfigured state, useful for
+multi-step UI flows.
+
+## 3. Field ownership
+
+**Server-derived (Response-only).** The universal set from
+`type_naming.md`. Data sources add no extras.
+
+**Caller-supplied (round-trip).** Present on both
+`DataSourceRequestWritable` and `DataSourceResponse`:
+
+```
+name           required string — agent-facing label
+uri            required string — endpoint identifier (free-form)
+description    optional string — narrative
+status         required enum   — lifecycle state
+is_default     required bool   — at most one true per datalake
+```
+
+**Write-only (Request-only).** None — data sources don't hold
+credentials directly. Credential handling is the consuming
+resource's responsibility (e.g. tools that connect to the
+source carry the authentication; the source itself is just
+a name + uri).
+
+The `slug` returned in the Response is the canonical reference
+for downstream scoping. Never pre-compute it client-side — see
+`type_naming.md` "Never pre-compute the slug client-side".
+
+## 4. Error envelopes
+
+Standard JSON:API envelopes per `errors.md`. Common rejections:
+
+| `source.pointer`                   | Cause                                           |
+|------------------------------------|-------------------------------------------------|
+| `/name`                            | uniqueness within datalake (name collision)      |
+| `/uri`                             | empty/structurally invalid OR uniqueness within  |
+|                                    | datalake (uri collision)                          |
+| `/status`                          | value not in the typed enum                      |
+| `/is_default`                      | another source already marked default            |
+
+All of these come back as a 422 `AlveraApiError` thrown from the
+SDK on submit — both structural failures (Layer 1) and semantic
+rejections like uniqueness / is_default conflict (Layer 2). The
+SDK does not validate client-side. See `errors.md`.
+
+## 5. Lifecycle
+
+Data sources are **Platform-DB-resident** — no async migration,
+no readiness poll. `.create` returns a ready-to-use row.
+
+### Create
+
+```typescript
+const { data: source } = await api.dataSources.create(
+  tenantSlug,
+  datalakeSlug,
+  body,
+)
+// source is immediately usable as a binding target for tools,
+// data activation clients, etc.
+```
+
+### Read shapes
+
+```
+.list(tenantSlug, datalakeSlug)              Paged list:
+                                              { data: DataSourceResponse[], meta }
+.get(tenantSlug, datalakeSlug, idOrSlug)     One row
+.metadata(tenantSlug, datalakeSlug)           Markdown string — catalog
+.metadataDetails(tenantSlug, datalakeSlug,    Markdown string — one source
+                 idOrSlug)
+```
+
+The two `metadata` methods return `string` — agent-facing
+markdown. `metadataDetails` takes the same id/slug as `.get`
+(no composite key like `templates.metadataDetails`).
+
+### Update
+
+`PUT` replays the full body per `mutations.md`. Data sources
+have no write-only fields, so the simple GET-then-spread
+pattern works (no re-supply needed):
+
+```typescript
+const { data: current } = await api.dataSources.get(
+  tenantSlug, datalakeSlug, sourceId,
+)
+const next = { ...current, description: 'Now updated' }
+const { data: updated } = await api.dataSources.update(
+  tenantSlug, datalakeSlug, sourceId, next,
+)
+```
+
+### Delete (operator-only — no SDK surface)
+
+There is **no `DELETE` endpoint** for data sources, and
+correspondingly no `api.dataSources.delete(...)` method on the
+typed client. Removal is operator-driven via the platform's
+admin console.
+
+For completeness, when an operator does remove a data source
+the platform applies asymmetric cascade behavior by dependent
+resource:
+
+- All data activation clients bound via `data_source_id`
+  cascade-delete with the source.
+- Workflow actions referencing the data source REJECT the
+  delete with a 422 — the operator must detach those actions
+  first.
+
+The auto-created default Manual Upload source (see §2 "A
+default Manual Upload data source is auto-created per
+datalake") is **not removable** — even operator-driven deletes
+reject it. The default exists for the lifetime of its parent
+datalake.
+
+Consumer-side, treat data sources as monotonically-growing
+within a datalake. If your code path conceptually "removes" a
+source (e.g. an integration test cleaning up), the only
+SDK-visible option is to mark it `status: 'inactive'` via PUT —
+the source row stays in the datalake.
+
+## 6. Gotchas
+
+1. **`.list()` is scoped to one datalake, not the tenant.** A
+   data source created on datalake A never appears in
+   `api.dataSources.list(tenantSlug, slugOfDatalakeB)`. The
+   platform filters strictly by the datalake in the path —
+   sibling-datalake sources are never surfaced in the active
+   datalake's response. If you're tracking sources across
+   multiple datalakes, iterate by datalake or use cross-datalake
+   discovery via a higher-level surface.
+
+2. **`is_default` collisions surface at create time.** Posting
+   a source with `is_default: true` against a datalake that
+   already has a default returns a 422. The SDK does NOT
+   silently demote the existing default. Either pass
+   `is_default: false` and promote separately via operator
+   action, or coordinate with the existing default's owner.
+
+3. **Data sources don't carry credentials.** A source is a
+   *named endpoint*, not an authenticated connection.
+   Authentication for the endpoint lives on the consuming
+   resource — typically a Tool's body (with its own
+   `tool_body_type` discriminator). The source-tool pair is
+   how connection credentials enter the system, not the
+   source alone.