@collective-intelligence/extensions-sdk-backend 0.1.29 → 0.1.31

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -114,7 +114,9 @@ Details keep only camelCase identifier, code, count, and flag metadata: `id`,
114
114
  `resourceType`, `scopeKind`, `authorityMode`, `configured`,
115
115
  `permissionGranted`, and `reachable`; other keys are dropped.
116
116
 
117
- The API exposes ordinary values and `PromiseLike` results. Effect services and private host implementations are intentionally outside this package.
117
+ The root `.` exposes ordinary values and `PromiseLike` results. The `./effect` export holds the Effect adapter; see [Effect adapter (`./effect`)](#effect-adapter-effect). Private host implementations stay outside this package.
118
+
119
+ `effect` (`^4.1.0`) is a required peer. Install it next to the SDK and keep one `effect` version in the application. The package root does not load it; only `./effect` does.
118
120
 
119
121
  For local validation, `test/consumer` is a private workspace with explicit SDK dependencies. Run `bun run test:pack` to verify the published boundary: it copies the consumer without local `node_modules` and installs the packed SDK artifacts in a temporary directory, so workspace links cannot satisfy its imports.
120
122
 
@@ -431,6 +433,173 @@ Declare `entitySchemaFqn: "*"` for a connector-wide fallback adapter. Hosts sele
431
433
  one exact schema adapter first, then one wildcard adapter; ambiguous matches are
432
434
  errors. The wildcard uses the existing declaration shape.
433
435
 
436
+ ## Effect adapter (`./effect`)
437
+
438
+ `@collective-intelligence/extensions-sdk-backend/effect` holds the Effect Schema
439
+ adapter for `@collective-intelligence/extensions-sdk` and the Effect Dataset
440
+ Provider ports. Use it only when an extension is authored with Effect. It was
441
+ the package `@collective-intelligence/extensions-sdk-backend-effect`, which now
442
+ re-exports these names and is deprecated.
443
+
444
+ `ExternalDatasetProvider` and `ManagedDatasetProvider` have two meanings. At
445
+ `./effect` they are the Effect tags of the provider services; at `.` they are
446
+ the abstract base classes. The class that
447
+ `makeExternalDatasetProviderImplementation` or
448
+ `makeManagedDatasetProviderImplementation` returns extends the `.` class.
449
+ The tag keys keep the prefix
450
+ `@collective-intelligence/extensions-sdk-backend-effect/`, so a tag of an older
451
+ copy of the deprecated package still matches.
452
+
453
+ `ci package build` bundles `./effect` into the release and keeps `.`
454
+ Host-provided. An earlier CLI keeps `./effect` external, and
455
+ `ci package validate` rejects the release; see the CLI `CHANGELOG.md`.
456
+
457
+ ```ts
458
+ import { effectSchema } from "@collective-intelligence/extensions-sdk-backend/effect"
459
+ import { Schema } from "effect"
460
+
461
+ const Count = effectSchema(Schema.NumberFromString, { codec: true })
462
+
463
+ await Count.validate("42") // { value: 42 }
464
+ await Count.encode(42) // { value: "42" }
465
+ ```
466
+
467
+ `effectSchema(...)` infers Effect's encoded type as the package wire type and
468
+ its decoded type as the implementation value. It delegates validation,
469
+ defaults, refinements, transforms, and encoding to Effect, including
470
+ asynchronous schemas.
471
+
472
+ Effect's static `Codec` type does not reveal whether a schema uses a forbidden
473
+ encoding getter. For that reason, `effectSchema(schema)` safely creates a
474
+ decoder-only source. A package author must explicitly select codec capability
475
+ for an output schema:
476
+
477
+ ```ts
478
+ const InputOnly = effectSchema(nativeSchema)
479
+ const InputAndOutput = effectSchema(nativeCodec, { codec: true })
480
+ ```
481
+
482
+ The default source has no runtime `encode` function or output projection and
483
+ therefore does not satisfy the package codec capability required by output
484
+ declarations. Both decoding and encoding must be service-free when codec
485
+ capability is selected.
486
+
487
+ The adapter projects Effect's encoded side to deterministic Draft 2020-12 for
488
+ both codec directions as a portable interchange artifact. Runtime payloads
489
+ are decoded and encoded only by Effect; projections are never runtime
490
+ validation authority. Effect's decoded/type-side JSON Schema is not the
491
+ emitted wire contract. Projection fidelity is conservatively marked
492
+ `structural`: Effect remains authoritative for semantics that JSON Schema may
493
+ only approximate. Native Effect schema objects stay in runtime sidecars and
494
+ are never part of portable manifests or declaration hashes.
495
+
496
+ The projection closes ordinary `Schema.Struct` objects, including nested
497
+ structs, while retaining the value schema of explicit `Schema.Record` index
498
+ signatures. It uses Effect's `onExcessProperty: "error"` JSON Schema generation
499
+ option for this portable artifact. Effect's default decoder can still ignore
500
+ unknown properties, so an author asserting `exact` fidelity must separately
501
+ prove wire equivalence for the schema and decoding options in use.
502
+
503
+ Effect's `withDecodingDefault*` defaults are opaque Effects, so the projection
504
+ cannot show them. `withWorkflowDefault(value)` makes a struct key optional,
505
+ applies `value` when decoding, and annotates the same value as the projected
506
+ JSON Schema `default`, which editors show and consumer profiles validate:
507
+
508
+ ```ts
509
+ import { withWorkflowDefault } from "@collective-intelligence/extensions-sdk-backend/effect"
510
+
511
+ const Config = Schema.Struct({
512
+ mode: Schema.Literals(["text", "structured"]).pipe(
513
+ withWorkflowDefault("text")
514
+ )
515
+ })
516
+ ```
517
+
518
+ When the author has separately proved that the encoded Effect contract and
519
+ its projected JSON Schema accept exactly the same wire values, an exact-only
520
+ consumer can be selected explicitly:
521
+
522
+ ```ts
523
+ const ExactInput = effectSchema(Schema.Boolean, { fidelity: "exact" })
524
+ const ExactCodec = effectSchema(Schema.String, {
525
+ codec: true,
526
+ fidelity: "exact"
527
+ })
528
+ ```
529
+
530
+ This is an author assertion, not provider inference; transformed or refined
531
+ schemas should remain `structural` unless equivalence is covered by tests.
532
+
533
+ Interactive forms require closed object schemas. Opt into rejection of unknown
534
+ properties for both native decoding/encoding and the wire projection:
535
+
536
+ ```ts
537
+ const Form = effectSchema(Schema.Struct({ name: Schema.String }), {
538
+ codec: true,
539
+ fidelity: "exact",
540
+ onExcessProperty: "error"
541
+ })
542
+ ```
543
+
544
+ When the option is omitted, projections close unmodeled object properties while
545
+ native validation retains Effect's permissive behavior. Explicitly selecting
546
+ `onExcessProperty: "ignore"` also leaves those properties open in the projection.
547
+ The `"error"` policy also applies to nested objects; keys modeled by an Effect
548
+ Record remain valid. Projections preserve referenced definitions. Each adapter
549
+ owns its validation policy even when the same native schema was previously
550
+ adapted with another policy. `source.native` remains the original Effect schema;
551
+ the option configures the adapter, not direct calls to that native schema.
552
+
553
+ ### Dataset Providers
554
+
555
+ `ManagedDatasetProvider` and `ExternalDatasetProvider` are invocation-scoped
556
+ Effect tags over the portable dataset vocabulary. Managed operations are
557
+ readiness, datasetProvision, datasetIngest, datasetQuery, datasetInspect, and
558
+ datasetDelete. External operations are readiness, listSources,
559
+ discoverDefinition, datasetQuery, and datasetInspect.
560
+
561
+ Use `makeExternalDatasetProviderImplementation` or
562
+ `makeManagedDatasetProviderImplementation` to expose one of those services as
563
+ the no-argument constructor required by `declareExternalDatasetProvider` or
564
+ `declareManagedDatasetProvider`:
565
+
566
+ ```ts
567
+ import {
568
+ ExternalDatasetProvider,
569
+ makeExternalDatasetProviderImplementation
570
+ } from "@collective-intelligence/extensions-sdk-backend/effect"
571
+ import type { PackageRuntimeContext } from "@collective-intelligence/extensions-sdk-backend"
572
+ import * as Context from "effect/Context"
573
+ import * as Effect from "effect/Effect"
574
+ import * as Layer from "effect/Layer"
575
+
576
+ import {
577
+ decodeProviderConfig,
578
+ providerLayer,
579
+ resolveProviderCredentials
580
+ } from "./provider"
581
+
582
+ const acquireService = (context: PackageRuntimeContext) =>
583
+ Effect.gen(function* acquireProvider() {
584
+ const config = yield* decodeProviderConfig(context.config)
585
+ const credentials = yield* resolveProviderCredentials(context, config)
586
+ const services = yield* Layer.build(providerLayer({ config, credentials }))
587
+ return Context.get(services, ExternalDatasetProvider)
588
+ })
589
+
590
+ export const ExampleExternalDatasetProvider =
591
+ makeExternalDatasetProviderImplementation(acquireService)
592
+ ```
593
+
594
+ The acquisition callback is the provider-owned boundary for decoding package
595
+ configuration, resolving host-authorized credentials, and constructing its
596
+ Layer. The adapter owns the common runtime bridge: it opens and closes an
597
+ Effect scope for every operation, forwards the host `AbortSignal` to
598
+ `Effect.runPromise`, preserves `DatasetProviderError` in the portable
599
+ `DatasetProviderResult` envelope, and delegates every provider method to the
600
+ acquired service. It supplies no default Layer and holds no provider state
601
+ between invocations.
602
+
434
603
  ## Enum value spelling
435
604
 
436
605
  Repository-owned enum values use lower `snake_case`. This includes component kinds, dataset statuses, schema consumer profiles, and hosted contribution identity. Notification enum values retain their existing spelling. External protocol values, including HTTP methods and JSON Schema keywords and formats, retain the protocol spelling.
@@ -0,0 +1,15 @@
1
+ import type { DatasetInspectInput, DatasetInspectResult, DatasetProviderError, DatasetResultPage, DefinitionProposal, DiscoverDefinitionInput, ListSourcesInput, QueryInput, Readiness, ReadinessInput, SourcePage } from "@collective-intelligence/extensions-sdk";
2
+ import type * as Effect from "effect/Effect";
3
+ import * as Context from "effect/Context";
4
+ /** Invocation-scoped External Dataset Provider authority; consumers own Layers. */
5
+ export interface ExternalDatasetProviderService {
6
+ readonly readiness: (input: ReadinessInput) => Effect.Effect<Readiness, DatasetProviderError>;
7
+ readonly listSources: (input: ListSourcesInput) => Effect.Effect<SourcePage, DatasetProviderError>;
8
+ readonly discoverDefinition: (input: DiscoverDefinitionInput) => Effect.Effect<DefinitionProposal, DatasetProviderError>;
9
+ readonly datasetQuery: (input: QueryInput) => Effect.Effect<DatasetResultPage, DatasetProviderError>;
10
+ readonly datasetInspect: (input: DatasetInspectInput) => Effect.Effect<DatasetInspectResult, DatasetProviderError>;
11
+ }
12
+ declare const ExternalDatasetProvider_base: Context.ServiceClass<ExternalDatasetProvider, "@collective-intelligence/extensions-sdk-backend-effect/ExternalDatasetProvider", ExternalDatasetProviderService>;
13
+ export declare class ExternalDatasetProvider extends ExternalDatasetProvider_base {
14
+ }
15
+ export {};
@@ -0,0 +1,16 @@
1
+ import type { DatasetDeleteInput, DatasetDeleteResult, DatasetIngestInput, DatasetIngestResult, DatasetInspectInput, DatasetInspectResult, DatasetProviderError, DatasetProvisionInput, DatasetProvisionResult, DatasetResultPage, QueryInput, Readiness, ReadinessInput } from "@collective-intelligence/extensions-sdk";
2
+ import type * as Effect from "effect/Effect";
3
+ import * as Context from "effect/Context";
4
+ /** Invocation-scoped Managed Dataset Provider authority; consumers own Layers. */
5
+ export interface ManagedDatasetProviderService {
6
+ readonly readiness: (input: ReadinessInput) => Effect.Effect<Readiness, DatasetProviderError>;
7
+ readonly datasetProvision: (input: DatasetProvisionInput) => Effect.Effect<DatasetProvisionResult, DatasetProviderError>;
8
+ readonly datasetIngest: (input: DatasetIngestInput) => Effect.Effect<DatasetIngestResult, DatasetProviderError>;
9
+ readonly datasetQuery: (input: QueryInput) => Effect.Effect<DatasetResultPage, DatasetProviderError>;
10
+ readonly datasetInspect: (input: DatasetInspectInput) => Effect.Effect<DatasetInspectResult, DatasetProviderError>;
11
+ readonly datasetDelete: (input: DatasetDeleteInput) => Effect.Effect<DatasetDeleteResult, DatasetProviderError>;
12
+ }
13
+ declare const ManagedDatasetProvider_base: Context.ServiceClass<ManagedDatasetProvider, "@collective-intelligence/extensions-sdk-backend-effect/ManagedDatasetProvider", ManagedDatasetProviderService>;
14
+ export declare class ManagedDatasetProvider extends ManagedDatasetProvider_base {
15
+ }
16
+ export {};
@@ -0,0 +1,31 @@
1
+ import type { DatasetDeleteInput, DatasetDeleteResult, DatasetIngestInput, DatasetIngestResult, DatasetInspectInput, DatasetInspectResult, DatasetProviderError, DatasetProviderResult, DatasetProvisionInput, DatasetProvisionResult, DatasetResultPage, DefinitionProposal, DiscoverDefinitionInput, ListSourcesInput, QueryInput, Readiness, ReadinessInput, SourcePage } from "@collective-intelligence/extensions-sdk";
2
+ import type { PackageRuntimeContext } from "@collective-intelligence/extensions-sdk-backend";
3
+ import type * as Scope from "effect/Scope";
4
+ import * as Effect from "effect/Effect";
5
+ import type { ExternalDatasetProviderService } from "./external-dataset-provider.js";
6
+ import type { ManagedDatasetProviderService } from "./managed-dataset-provider.js";
7
+ type DatasetProviderServiceAcquirer<Service> = (context: PackageRuntimeContext) => Effect.Effect<Service, DatasetProviderError, Scope.Scope>;
8
+ export type ExternalDatasetProviderServiceAcquirer = DatasetProviderServiceAcquirer<ExternalDatasetProviderService>;
9
+ export type ManagedDatasetProviderServiceAcquirer = DatasetProviderServiceAcquirer<ManagedDatasetProviderService>;
10
+ /** Adapts an invocation-scoped Effect External Dataset Provider to the backend SDK. */
11
+ export declare const makeExternalDatasetProviderImplementation: (acquireService: ExternalDatasetProviderServiceAcquirer) => {
12
+ new (): {
13
+ readiness(input: ReadinessInput, context: PackageRuntimeContext): Promise<DatasetProviderResult<Readiness>>;
14
+ listSources(input: ListSourcesInput, context: PackageRuntimeContext): Promise<DatasetProviderResult<SourcePage>>;
15
+ discoverDefinition(input: DiscoverDefinitionInput, context: PackageRuntimeContext): Promise<DatasetProviderResult<DefinitionProposal>>;
16
+ datasetQuery(input: QueryInput, context: PackageRuntimeContext): Promise<DatasetProviderResult<DatasetResultPage>>;
17
+ datasetInspect(input: DatasetInspectInput, context: PackageRuntimeContext): Promise<DatasetProviderResult<DatasetInspectResult>>;
18
+ };
19
+ };
20
+ /** Adapts an invocation-scoped Effect Managed Dataset Provider to the backend SDK. */
21
+ export declare const makeManagedDatasetProviderImplementation: (acquireService: ManagedDatasetProviderServiceAcquirer) => {
22
+ new (): {
23
+ readiness(input: ReadinessInput, context: PackageRuntimeContext): Promise<DatasetProviderResult<Readiness>>;
24
+ datasetProvision(input: DatasetProvisionInput, context: PackageRuntimeContext): Promise<DatasetProviderResult<DatasetProvisionResult>>;
25
+ datasetIngest(input: DatasetIngestInput, context: PackageRuntimeContext): Promise<DatasetProviderResult<DatasetIngestResult>>;
26
+ datasetQuery(input: QueryInput, context: PackageRuntimeContext): Promise<DatasetProviderResult<DatasetResultPage>>;
27
+ datasetInspect(input: DatasetInspectInput, context: PackageRuntimeContext): Promise<DatasetProviderResult<DatasetInspectResult>>;
28
+ datasetDelete(input: DatasetDeleteInput, context: PackageRuntimeContext): Promise<DatasetProviderResult<DatasetDeleteResult>>;
29
+ };
30
+ };
31
+ export {};