@sanity/embeddings-index-ui 1.0.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2023 Sanity
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,148 @@
+# @sanity/embeddings-index-ui
+> This package contains plugins for **Sanity Studio v3**.
+
+`@sanity/embeddings-index-ui` Sanity Studio plugins that interacts with the `/embeddings-index` HTTP API.
+
+The embeddings index API allows the creation of named embeddings vector indexes.
+An embeddings index contains embeddings for all Sanity documents matching a configured GROQ filter in a dataset.
+A GROQ projection is applied to matching documents before vectorization.
+
+Indexes can be queried using semantic text search, and returns a list of matching document ids,
+sorted by relevance.
+
+When an index is first created, all documents matching the configured filter will be synced into the index.
+This can take some time depending on the number of documents that need to be synced.
+
+For a CLI alternative, check out the [Embeddings index CLI](https://github.com/sanity-io/embeddings-index-cli) package.
+
+## Installation
+
+```sh
+npm install @sanity/embeddings-index-ui
+```
+
+`@sanity/embeddings-index-ui` contains the following Sanity Studio plugins: 
+
+* [embeddingsIndexReferenceInput](#embeddings-index-reference-input): semantic search mode for reference inputs
+* [embeddingsIndexDashboard](#embeddings-index-dashboard): manage indexes in a Sanity studio tool
+
+Consult each section for usage details.
+
+## Embeddings index reference input
+
+<img width="619" alt="image" src="https://github.com/sanity-io/sanity/assets/835514/55d372fe-c5fe-40dd-882b-10c6e8794442">
+
+The `embeddingsIndexReferenceInput` plugin allows references fields to opt-in to embeddings index search.
+This users to search for references using natural language to find documents based on semantic meaning,
+bypassing the need for exact word matches.
+
+### Usage
+
+Add `embeddingsIndexReferenceInput` as a plugin in `sanity.config.ts` (or .js):
+
+```ts
+import {defineConfig} from 'sanity'
+import {embeddingsIndexReferenceInput} from '@sanity/embeddings-index-ui'
+
+export default defineConfig({
+  //...
+  plugins: [embeddingsIndexReferenceInput()],
+})
+```
+
+Next, enabled semantic search using `options.embeddingsIndex` on reference fields:
+
+```ts
+defineField({
+  name: 'myField',
+  type: 'reference',
+  to: [{type: 'myType'}],
+  options: {
+    embeddingsIndex: {
+      indexName: 'my-index',
+      maxResults: 10, // default,
+      searchMode: 'embeddings' // defaults value is 'default'
+    }
+  }
+})
+```
+
+Setting `options.embeddings.indexName` on a reference field will allow enabled searching into the named index.
+
+This will add a "search mode" toggle button to the field.
+
+*Note*: The search will use `to` types as a filter into the index, so it important
+that the types the reference field expects actually exist in the index 
+(ie, the embeddings index `filter` contains one or more documents relevant to the reference field).
+
+*Caveats*:
+`options.filter` is not respected by the semantic search.
+
+## Embeddings index dashboard
+An UI alternative to the [Embeddings index CLI](https://github.com/sanity-io/embeddings-index-cli)
+Manage embeddings indexes in a Studio dashboard.
+
+<img width="1227" alt="image" src="https://github.com/sanity-io/sanity/assets/835514/279b03b8-d2c0-4cc1-bbe6-9d335937f25a">
+
+
+## Usage
+
+Add `embeddingsIndexDashboard` as a plugin in `sanity.config.ts` (or .js):
+
+```ts
+import {defineConfig} from 'sanity'
+import {embeddingsIndexDashboard} from '@sanity/embeddings-index-ui'
+
+export default defineConfig({
+  //...
+  plugins: [    
+    process.env.NODE_ENV === 'development'
+    ? embeddingsIndexDashboard()
+    : {name: 'embeddings-index-dashboard-disabled'}
+  ],
+})
+```
+
+This will add the Embeddings tool to the studio nav-bar, only when studio is running in developer mode (localhost).
+
+You might instead want to enable this tool based on roles instead:
+
+```ts
+import {defineConfig} from 'sanity'
+import {embeddingsIndexDashboard} from '@sanity/embeddings-index-ui'
+
+export default defineConfig({
+  //...
+  plugins: [embeddingsIndexDashboard()],
+
+  tools: (prev, context) => {
+    const enabledForRoles = ['developer']
+    const canManageEmbeddingsIndex = context.currentUser?.roles
+      .map((role) => role.name)
+      .some((roleName) => enabledForRoles.includes(roleName))
+    return canManageEmbeddingsIndex ? prev : prev.filter((tool) => tool.name !== 'embeddings-index')
+  },
+})
+```
+
+
+
+## License
+
+[MIT](LICENSE) © Sanity
+
+## Develop & test
+
+This plugin uses [@sanity/plugin-kit](https://github.com/sanity-io/plugin-kit)
+with default configuration for build & watch scripts.
+
+See [Testing a plugin in Sanity Studio](https://github.com/sanity-io/plugin-kit#testing-a-plugin-in-sanity-studio)
+on how to run this plugin with hotreload in the studio.
+
+
+### Release new version
+
+Run ["CI & Release" workflow](https://github.com/sanity-io/embeddings-plugin/actions/workflows/main.yml).
+Make sure to select the main branch and check "Release new version".
+
+Semantic release will only release on configured branches, so it is safe to run release on any branch.

package/dist/index.d.ts

@@ -0,0 +1,67 @@
+import {Plugin as Plugin_2} from 'sanity'
+import {SanityClient} from 'sanity'
+
+export declare function deleteIndex(indexName: string, client: SanityClient): Promise<IndexState>
+
+export declare const embeddingsIndexDashboard: Plugin_2<void>
+
+export declare const embeddingsIndexReferenceInput: Plugin_2<void>
+
+export declare function getIndexes(client: SanityClient): Promise<IndexState[]>
+
+export declare interface IndexState extends NamedIndex {
+  status: string
+  startDocumentCount: number
+  remainingDocumentCount: number
+  failedDocumentCount: number
+}
+
+export declare interface NamedIndex {
+  indexName: string
+  dataset: string
+  project: string
+  projection: string
+  filter: string
+}
+
+export declare interface QueryConfig {
+  query: string
+  indexName: string
+  maxResults?: number
+  filter?: {
+    type?: string | string[]
+  }
+}
+
+export declare function queryIndex(
+  queryConfig: QueryConfig,
+  client: SanityClient,
+): Promise<QueryResult[]>
+
+export declare interface QueryResult {
+  score: number
+  value: {
+    documentId: string
+    type?: string
+  }
+}
+
+export {}
+
+declare module 'sanity' {
+  interface ReferenceBaseOptions {
+    embeddingsIndex?: {
+      /**
+       * Name of the index
+       */
+      indexName: string
+      maxResults?: number
+      /**
+       * Determines if which search mode is enabled by default for the reference field.
+       * Default is the studio default search, while 'embeddings' enables
+       * Defaults to 'default' behaviour
+       */
+      searchMode?: 'embeddings' | 'default'
+    }
+  }
+}