@collective-intelligence/extensions-sdk-backend 0.1.29 → 0.1.31
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +170 -1
- package/dist/effect/datasets/external-dataset-provider.d.ts +15 -0
- package/dist/effect/datasets/managed-dataset-provider.d.ts +16 -0
- package/dist/effect/datasets/provider-implementations.d.ts +31 -0
- package/dist/effect/schemas/dataset-schema.d.ts +862 -0
- package/dist/effect/schemas/defaults.d.ts +13 -0
- package/dist/effect/schemas/schema.d.ts +23 -0
- package/dist/effect.d.ts +10 -0
- package/dist/effect.js +1 -0
- package/dist/index.js +1 -2
- package/package.json +17 -7
package/README.md
CHANGED
|
@@ -114,7 +114,9 @@ Details keep only camelCase identifier, code, count, and flag metadata: `id`,
|
|
|
114
114
|
`resourceType`, `scopeKind`, `authorityMode`, `configured`,
|
|
115
115
|
`permissionGranted`, and `reachable`; other keys are dropped.
|
|
116
116
|
|
|
117
|
-
The
|
|
117
|
+
The root `.` exposes ordinary values and `PromiseLike` results. The `./effect` export holds the Effect adapter; see [Effect adapter (`./effect`)](#effect-adapter-effect). Private host implementations stay outside this package.
|
|
118
|
+
|
|
119
|
+
`effect` (`^4.1.0`) is a required peer. Install it next to the SDK and keep one `effect` version in the application. The package root does not load it; only `./effect` does.
|
|
118
120
|
|
|
119
121
|
For local validation, `test/consumer` is a private workspace with explicit SDK dependencies. Run `bun run test:pack` to verify the published boundary: it copies the consumer without local `node_modules` and installs the packed SDK artifacts in a temporary directory, so workspace links cannot satisfy its imports.
|
|
120
122
|
|
|
@@ -431,6 +433,173 @@ Declare `entitySchemaFqn: "*"` for a connector-wide fallback adapter. Hosts sele
|
|
|
431
433
|
one exact schema adapter first, then one wildcard adapter; ambiguous matches are
|
|
432
434
|
errors. The wildcard uses the existing declaration shape.
|
|
433
435
|
|
|
436
|
+
## Effect adapter (`./effect`)
|
|
437
|
+
|
|
438
|
+
`@collective-intelligence/extensions-sdk-backend/effect` holds the Effect Schema
|
|
439
|
+
adapter for `@collective-intelligence/extensions-sdk` and the Effect Dataset
|
|
440
|
+
Provider ports. Use it only when an extension is authored with Effect. It was
|
|
441
|
+
the package `@collective-intelligence/extensions-sdk-backend-effect`, which now
|
|
442
|
+
re-exports these names and is deprecated.
|
|
443
|
+
|
|
444
|
+
`ExternalDatasetProvider` and `ManagedDatasetProvider` have two meanings. At
|
|
445
|
+
`./effect` they are the Effect tags of the provider services; at `.` they are
|
|
446
|
+
the abstract base classes. The class that
|
|
447
|
+
`makeExternalDatasetProviderImplementation` or
|
|
448
|
+
`makeManagedDatasetProviderImplementation` returns extends the `.` class.
|
|
449
|
+
The tag keys keep the prefix
|
|
450
|
+
`@collective-intelligence/extensions-sdk-backend-effect/`, so a tag of an older
|
|
451
|
+
copy of the deprecated package still matches.
|
|
452
|
+
|
|
453
|
+
`ci package build` bundles `./effect` into the release and keeps `.`
|
|
454
|
+
Host-provided. An earlier CLI keeps `./effect` external, and
|
|
455
|
+
`ci package validate` rejects the release; see the CLI `CHANGELOG.md`.
|
|
456
|
+
|
|
457
|
+
```ts
|
|
458
|
+
import { effectSchema } from "@collective-intelligence/extensions-sdk-backend/effect"
|
|
459
|
+
import { Schema } from "effect"
|
|
460
|
+
|
|
461
|
+
const Count = effectSchema(Schema.NumberFromString, { codec: true })
|
|
462
|
+
|
|
463
|
+
await Count.validate("42") // { value: 42 }
|
|
464
|
+
await Count.encode(42) // { value: "42" }
|
|
465
|
+
```
|
|
466
|
+
|
|
467
|
+
`effectSchema(...)` infers Effect's encoded type as the package wire type and
|
|
468
|
+
its decoded type as the implementation value. It delegates validation,
|
|
469
|
+
defaults, refinements, transforms, and encoding to Effect, including
|
|
470
|
+
asynchronous schemas.
|
|
471
|
+
|
|
472
|
+
Effect's static `Codec` type does not reveal whether a schema uses a forbidden
|
|
473
|
+
encoding getter. For that reason, `effectSchema(schema)` safely creates a
|
|
474
|
+
decoder-only source. A package author must explicitly select codec capability
|
|
475
|
+
for an output schema:
|
|
476
|
+
|
|
477
|
+
```ts
|
|
478
|
+
const InputOnly = effectSchema(nativeSchema)
|
|
479
|
+
const InputAndOutput = effectSchema(nativeCodec, { codec: true })
|
|
480
|
+
```
|
|
481
|
+
|
|
482
|
+
The default source has no runtime `encode` function or output projection and
|
|
483
|
+
therefore does not satisfy the package codec capability required by output
|
|
484
|
+
declarations. Both decoding and encoding must be service-free when codec
|
|
485
|
+
capability is selected.
|
|
486
|
+
|
|
487
|
+
The adapter projects Effect's encoded side to deterministic Draft 2020-12 for
|
|
488
|
+
both codec directions as a portable interchange artifact. Runtime payloads
|
|
489
|
+
are decoded and encoded only by Effect; projections are never runtime
|
|
490
|
+
validation authority. Effect's decoded/type-side JSON Schema is not the
|
|
491
|
+
emitted wire contract. Projection fidelity is conservatively marked
|
|
492
|
+
`structural`: Effect remains authoritative for semantics that JSON Schema may
|
|
493
|
+
only approximate. Native Effect schema objects stay in runtime sidecars and
|
|
494
|
+
are never part of portable manifests or declaration hashes.
|
|
495
|
+
|
|
496
|
+
The projection closes ordinary `Schema.Struct` objects, including nested
|
|
497
|
+
structs, while retaining the value schema of explicit `Schema.Record` index
|
|
498
|
+
signatures. It uses Effect's `onExcessProperty: "error"` JSON Schema generation
|
|
499
|
+
option for this portable artifact. Effect's default decoder can still ignore
|
|
500
|
+
unknown properties, so an author asserting `exact` fidelity must separately
|
|
501
|
+
prove wire equivalence for the schema and decoding options in use.
|
|
502
|
+
|
|
503
|
+
Effect's `withDecodingDefault*` defaults are opaque Effects, so the projection
|
|
504
|
+
cannot show them. `withWorkflowDefault(value)` makes a struct key optional,
|
|
505
|
+
applies `value` when decoding, and annotates the same value as the projected
|
|
506
|
+
JSON Schema `default`, which editors show and consumer profiles validate:
|
|
507
|
+
|
|
508
|
+
```ts
|
|
509
|
+
import { withWorkflowDefault } from "@collective-intelligence/extensions-sdk-backend/effect"
|
|
510
|
+
|
|
511
|
+
const Config = Schema.Struct({
|
|
512
|
+
mode: Schema.Literals(["text", "structured"]).pipe(
|
|
513
|
+
withWorkflowDefault("text")
|
|
514
|
+
)
|
|
515
|
+
})
|
|
516
|
+
```
|
|
517
|
+
|
|
518
|
+
When the author has separately proved that the encoded Effect contract and
|
|
519
|
+
its projected JSON Schema accept exactly the same wire values, an exact-only
|
|
520
|
+
consumer can be selected explicitly:
|
|
521
|
+
|
|
522
|
+
```ts
|
|
523
|
+
const ExactInput = effectSchema(Schema.Boolean, { fidelity: "exact" })
|
|
524
|
+
const ExactCodec = effectSchema(Schema.String, {
|
|
525
|
+
codec: true,
|
|
526
|
+
fidelity: "exact"
|
|
527
|
+
})
|
|
528
|
+
```
|
|
529
|
+
|
|
530
|
+
This is an author assertion, not provider inference; transformed or refined
|
|
531
|
+
schemas should remain `structural` unless equivalence is covered by tests.
|
|
532
|
+
|
|
533
|
+
Interactive forms require closed object schemas. Opt into rejection of unknown
|
|
534
|
+
properties for both native decoding/encoding and the wire projection:
|
|
535
|
+
|
|
536
|
+
```ts
|
|
537
|
+
const Form = effectSchema(Schema.Struct({ name: Schema.String }), {
|
|
538
|
+
codec: true,
|
|
539
|
+
fidelity: "exact",
|
|
540
|
+
onExcessProperty: "error"
|
|
541
|
+
})
|
|
542
|
+
```
|
|
543
|
+
|
|
544
|
+
When the option is omitted, projections close unmodeled object properties while
|
|
545
|
+
native validation retains Effect's permissive behavior. Explicitly selecting
|
|
546
|
+
`onExcessProperty: "ignore"` also leaves those properties open in the projection.
|
|
547
|
+
The `"error"` policy also applies to nested objects; keys modeled by an Effect
|
|
548
|
+
Record remain valid. Projections preserve referenced definitions. Each adapter
|
|
549
|
+
owns its validation policy even when the same native schema was previously
|
|
550
|
+
adapted with another policy. `source.native` remains the original Effect schema;
|
|
551
|
+
the option configures the adapter, not direct calls to that native schema.
|
|
552
|
+
|
|
553
|
+
### Dataset Providers
|
|
554
|
+
|
|
555
|
+
`ManagedDatasetProvider` and `ExternalDatasetProvider` are invocation-scoped
|
|
556
|
+
Effect tags over the portable dataset vocabulary. Managed operations are
|
|
557
|
+
readiness, datasetProvision, datasetIngest, datasetQuery, datasetInspect, and
|
|
558
|
+
datasetDelete. External operations are readiness, listSources,
|
|
559
|
+
discoverDefinition, datasetQuery, and datasetInspect.
|
|
560
|
+
|
|
561
|
+
Use `makeExternalDatasetProviderImplementation` or
|
|
562
|
+
`makeManagedDatasetProviderImplementation` to expose one of those services as
|
|
563
|
+
the no-argument constructor required by `declareExternalDatasetProvider` or
|
|
564
|
+
`declareManagedDatasetProvider`:
|
|
565
|
+
|
|
566
|
+
```ts
|
|
567
|
+
import {
|
|
568
|
+
ExternalDatasetProvider,
|
|
569
|
+
makeExternalDatasetProviderImplementation
|
|
570
|
+
} from "@collective-intelligence/extensions-sdk-backend/effect"
|
|
571
|
+
import type { PackageRuntimeContext } from "@collective-intelligence/extensions-sdk-backend"
|
|
572
|
+
import * as Context from "effect/Context"
|
|
573
|
+
import * as Effect from "effect/Effect"
|
|
574
|
+
import * as Layer from "effect/Layer"
|
|
575
|
+
|
|
576
|
+
import {
|
|
577
|
+
decodeProviderConfig,
|
|
578
|
+
providerLayer,
|
|
579
|
+
resolveProviderCredentials
|
|
580
|
+
} from "./provider"
|
|
581
|
+
|
|
582
|
+
const acquireService = (context: PackageRuntimeContext) =>
|
|
583
|
+
Effect.gen(function* acquireProvider() {
|
|
584
|
+
const config = yield* decodeProviderConfig(context.config)
|
|
585
|
+
const credentials = yield* resolveProviderCredentials(context, config)
|
|
586
|
+
const services = yield* Layer.build(providerLayer({ config, credentials }))
|
|
587
|
+
return Context.get(services, ExternalDatasetProvider)
|
|
588
|
+
})
|
|
589
|
+
|
|
590
|
+
export const ExampleExternalDatasetProvider =
|
|
591
|
+
makeExternalDatasetProviderImplementation(acquireService)
|
|
592
|
+
```
|
|
593
|
+
|
|
594
|
+
The acquisition callback is the provider-owned boundary for decoding package
|
|
595
|
+
configuration, resolving host-authorized credentials, and constructing its
|
|
596
|
+
Layer. The adapter owns the common runtime bridge: it opens and closes an
|
|
597
|
+
Effect scope for every operation, forwards the host `AbortSignal` to
|
|
598
|
+
`Effect.runPromise`, preserves `DatasetProviderError` in the portable
|
|
599
|
+
`DatasetProviderResult` envelope, and delegates every provider method to the
|
|
600
|
+
acquired service. It supplies no default Layer and holds no provider state
|
|
601
|
+
between invocations.
|
|
602
|
+
|
|
434
603
|
## Enum value spelling
|
|
435
604
|
|
|
436
605
|
Repository-owned enum values use lower `snake_case`. This includes component kinds, dataset statuses, schema consumer profiles, and hosted contribution identity. Notification enum values retain their existing spelling. External protocol values, including HTTP methods and JSON Schema keywords and formats, retain the protocol spelling.
|
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
import type { DatasetInspectInput, DatasetInspectResult, DatasetProviderError, DatasetResultPage, DefinitionProposal, DiscoverDefinitionInput, ListSourcesInput, QueryInput, Readiness, ReadinessInput, SourcePage } from "@collective-intelligence/extensions-sdk";
|
|
2
|
+
import type * as Effect from "effect/Effect";
|
|
3
|
+
import * as Context from "effect/Context";
|
|
4
|
+
/** Invocation-scoped External Dataset Provider authority; consumers own Layers. */
|
|
5
|
+
export interface ExternalDatasetProviderService {
|
|
6
|
+
readonly readiness: (input: ReadinessInput) => Effect.Effect<Readiness, DatasetProviderError>;
|
|
7
|
+
readonly listSources: (input: ListSourcesInput) => Effect.Effect<SourcePage, DatasetProviderError>;
|
|
8
|
+
readonly discoverDefinition: (input: DiscoverDefinitionInput) => Effect.Effect<DefinitionProposal, DatasetProviderError>;
|
|
9
|
+
readonly datasetQuery: (input: QueryInput) => Effect.Effect<DatasetResultPage, DatasetProviderError>;
|
|
10
|
+
readonly datasetInspect: (input: DatasetInspectInput) => Effect.Effect<DatasetInspectResult, DatasetProviderError>;
|
|
11
|
+
}
|
|
12
|
+
declare const ExternalDatasetProvider_base: Context.ServiceClass<ExternalDatasetProvider, "@collective-intelligence/extensions-sdk-backend-effect/ExternalDatasetProvider", ExternalDatasetProviderService>;
|
|
13
|
+
export declare class ExternalDatasetProvider extends ExternalDatasetProvider_base {
|
|
14
|
+
}
|
|
15
|
+
export {};
|
|
@@ -0,0 +1,16 @@
|
|
|
1
|
+
import type { DatasetDeleteInput, DatasetDeleteResult, DatasetIngestInput, DatasetIngestResult, DatasetInspectInput, DatasetInspectResult, DatasetProviderError, DatasetProvisionInput, DatasetProvisionResult, DatasetResultPage, QueryInput, Readiness, ReadinessInput } from "@collective-intelligence/extensions-sdk";
|
|
2
|
+
import type * as Effect from "effect/Effect";
|
|
3
|
+
import * as Context from "effect/Context";
|
|
4
|
+
/** Invocation-scoped Managed Dataset Provider authority; consumers own Layers. */
|
|
5
|
+
export interface ManagedDatasetProviderService {
|
|
6
|
+
readonly readiness: (input: ReadinessInput) => Effect.Effect<Readiness, DatasetProviderError>;
|
|
7
|
+
readonly datasetProvision: (input: DatasetProvisionInput) => Effect.Effect<DatasetProvisionResult, DatasetProviderError>;
|
|
8
|
+
readonly datasetIngest: (input: DatasetIngestInput) => Effect.Effect<DatasetIngestResult, DatasetProviderError>;
|
|
9
|
+
readonly datasetQuery: (input: QueryInput) => Effect.Effect<DatasetResultPage, DatasetProviderError>;
|
|
10
|
+
readonly datasetInspect: (input: DatasetInspectInput) => Effect.Effect<DatasetInspectResult, DatasetProviderError>;
|
|
11
|
+
readonly datasetDelete: (input: DatasetDeleteInput) => Effect.Effect<DatasetDeleteResult, DatasetProviderError>;
|
|
12
|
+
}
|
|
13
|
+
declare const ManagedDatasetProvider_base: Context.ServiceClass<ManagedDatasetProvider, "@collective-intelligence/extensions-sdk-backend-effect/ManagedDatasetProvider", ManagedDatasetProviderService>;
|
|
14
|
+
export declare class ManagedDatasetProvider extends ManagedDatasetProvider_base {
|
|
15
|
+
}
|
|
16
|
+
export {};
|
|
@@ -0,0 +1,31 @@
|
|
|
1
|
+
import type { DatasetDeleteInput, DatasetDeleteResult, DatasetIngestInput, DatasetIngestResult, DatasetInspectInput, DatasetInspectResult, DatasetProviderError, DatasetProviderResult, DatasetProvisionInput, DatasetProvisionResult, DatasetResultPage, DefinitionProposal, DiscoverDefinitionInput, ListSourcesInput, QueryInput, Readiness, ReadinessInput, SourcePage } from "@collective-intelligence/extensions-sdk";
|
|
2
|
+
import type { PackageRuntimeContext } from "@collective-intelligence/extensions-sdk-backend";
|
|
3
|
+
import type * as Scope from "effect/Scope";
|
|
4
|
+
import * as Effect from "effect/Effect";
|
|
5
|
+
import type { ExternalDatasetProviderService } from "./external-dataset-provider.js";
|
|
6
|
+
import type { ManagedDatasetProviderService } from "./managed-dataset-provider.js";
|
|
7
|
+
type DatasetProviderServiceAcquirer<Service> = (context: PackageRuntimeContext) => Effect.Effect<Service, DatasetProviderError, Scope.Scope>;
|
|
8
|
+
export type ExternalDatasetProviderServiceAcquirer = DatasetProviderServiceAcquirer<ExternalDatasetProviderService>;
|
|
9
|
+
export type ManagedDatasetProviderServiceAcquirer = DatasetProviderServiceAcquirer<ManagedDatasetProviderService>;
|
|
10
|
+
/** Adapts an invocation-scoped Effect External Dataset Provider to the backend SDK. */
|
|
11
|
+
export declare const makeExternalDatasetProviderImplementation: (acquireService: ExternalDatasetProviderServiceAcquirer) => {
|
|
12
|
+
new (): {
|
|
13
|
+
readiness(input: ReadinessInput, context: PackageRuntimeContext): Promise<DatasetProviderResult<Readiness>>;
|
|
14
|
+
listSources(input: ListSourcesInput, context: PackageRuntimeContext): Promise<DatasetProviderResult<SourcePage>>;
|
|
15
|
+
discoverDefinition(input: DiscoverDefinitionInput, context: PackageRuntimeContext): Promise<DatasetProviderResult<DefinitionProposal>>;
|
|
16
|
+
datasetQuery(input: QueryInput, context: PackageRuntimeContext): Promise<DatasetProviderResult<DatasetResultPage>>;
|
|
17
|
+
datasetInspect(input: DatasetInspectInput, context: PackageRuntimeContext): Promise<DatasetProviderResult<DatasetInspectResult>>;
|
|
18
|
+
};
|
|
19
|
+
};
|
|
20
|
+
/** Adapts an invocation-scoped Effect Managed Dataset Provider to the backend SDK. */
|
|
21
|
+
export declare const makeManagedDatasetProviderImplementation: (acquireService: ManagedDatasetProviderServiceAcquirer) => {
|
|
22
|
+
new (): {
|
|
23
|
+
readiness(input: ReadinessInput, context: PackageRuntimeContext): Promise<DatasetProviderResult<Readiness>>;
|
|
24
|
+
datasetProvision(input: DatasetProvisionInput, context: PackageRuntimeContext): Promise<DatasetProviderResult<DatasetProvisionResult>>;
|
|
25
|
+
datasetIngest(input: DatasetIngestInput, context: PackageRuntimeContext): Promise<DatasetProviderResult<DatasetIngestResult>>;
|
|
26
|
+
datasetQuery(input: QueryInput, context: PackageRuntimeContext): Promise<DatasetProviderResult<DatasetResultPage>>;
|
|
27
|
+
datasetInspect(input: DatasetInspectInput, context: PackageRuntimeContext): Promise<DatasetProviderResult<DatasetInspectResult>>;
|
|
28
|
+
datasetDelete(input: DatasetDeleteInput, context: PackageRuntimeContext): Promise<DatasetProviderResult<DatasetDeleteResult>>;
|
|
29
|
+
};
|
|
30
|
+
};
|
|
31
|
+
export {};
|