@sanity/embeddings-index-ui 1.1.1 → 1.1.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +48 -49
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -1,19 +1,22 @@
|
|
|
1
|
-
#
|
|
2
|
-
> This package contains plugins for **Sanity Studio v3**.
|
|
1
|
+
# Sanity Embeddings Index UI
|
|
3
2
|
|
|
4
|
-
Sanity
|
|
3
|
+
> Using this feature requires Sanity to send data to OpenAI[.]com, and Pinecone[.]io for storing vector interpretations of documents.
|
|
5
4
|
|
|
6
|
-
|
|
7
|
-
An embeddings index contains embeddings for all Sanity documents matching a configured GROQ filter in a dataset.
|
|
8
|
-
A GROQ projection is applied to matching documents before vectorization.
|
|
5
|
+
Sanity Studio v3 plugins that interact with the `/embeddings-index` HTTP API.
|
|
9
6
|
|
|
10
|
-
|
|
11
|
-
sorted by relevance.
|
|
7
|
+
The Embeddings Index API enables creating named embeddings vector indexes.
|
|
12
8
|
|
|
13
|
-
|
|
14
|
-
|
|
9
|
+
An embeddings index contains embeddings for all Sanity documents matching a configured [GROQ filter](https://www.sanity.io/docs/how-queries-work) in a dataset.
|
|
10
|
+
A [GROQ projection](https://www.sanity.io/docs/query-cheat-sheet) is applied to matching documents before vectorization.
|
|
15
11
|
|
|
16
|
-
|
|
12
|
+
You can query indexes using semantic text search to obtain a list of matching document IDs sorted by relevance.
|
|
13
|
+
|
|
14
|
+
When an index is first created, all documents matching the configured filter are synced into the index.
|
|
15
|
+
Creating an index can take time, depending on the number of existing documents and the indexer load.
|
|
16
|
+
|
|
17
|
+
For a CLI alternative, check out the [Embeddings Index CLI](https://github.com/sanity-io/embeddings-index-cli) package.
|
|
18
|
+
|
|
19
|
+
> Using this feature requires Sanity to send data to OpenAI.com, and Pinecone.io for storing vector interpretations of documents.
|
|
17
20
|
|
|
18
21
|
## Installation
|
|
19
22
|
|
|
@@ -24,21 +27,20 @@ npm install @sanity/embeddings-index-ui
|
|
|
24
27
|
`@sanity/embeddings-index-ui` contains the following Sanity Studio plugins:
|
|
25
28
|
|
|
26
29
|
* [embeddingsIndexReferenceInput](#embeddings-index-reference-input): semantic search mode for reference inputs
|
|
27
|
-
* [embeddingsIndexDashboard](#embeddings-index-dashboard): manage indexes in a Sanity
|
|
30
|
+
* [embeddingsIndexDashboard](#embeddings-index-dashboard): manage indexes in a Sanity Studio UI tool
|
|
28
31
|
|
|
29
|
-
|
|
32
|
+
For more information about using the plugins, see the relevant sections below.
|
|
30
33
|
|
|
31
34
|
## Embeddings index reference input
|
|
32
35
|
|
|
33
36
|
<img width="619" alt="image" src="https://github.com/sanity-io/sanity/assets/835514/55d372fe-c5fe-40dd-882b-10c6e8794442">
|
|
34
37
|
|
|
35
|
-
The `embeddingsIndexReferenceInput` plugin allows
|
|
36
|
-
This users to search for references using natural language to
|
|
37
|
-
bypassing the need for exact word matches.
|
|
38
|
+
The `embeddingsIndexReferenceInput` plugin allows reference fields to opt in to embeddings index search.
|
|
39
|
+
This enables users to search for references using natural language, and to retrieve documents based on semantic meaning, rather than exact word matches.
|
|
38
40
|
|
|
39
41
|
### Usage
|
|
40
42
|
|
|
41
|
-
Add `embeddingsIndexReferenceInput` as a plugin
|
|
43
|
+
Add `embeddingsIndexReferenceInput` as a plugin to `sanity.config.ts` (or `.js`):
|
|
42
44
|
|
|
43
45
|
```ts
|
|
44
46
|
import {defineConfig} from 'sanity'
|
|
@@ -50,37 +52,37 @@ export default defineConfig({
|
|
|
50
52
|
})
|
|
51
53
|
```
|
|
52
54
|
|
|
53
|
-
|
|
55
|
+
Then, enable semantic search using `options.embeddingsIndex` on reference fields.
|
|
56
|
+
Example of a default configuration for a reference field:
|
|
54
57
|
|
|
55
58
|
```ts
|
|
56
59
|
defineField({
|
|
57
60
|
name: 'myField',
|
|
58
61
|
type: 'reference',
|
|
59
|
-
to: [{type: 'myType'}],
|
|
62
|
+
to: [{type: 'myType'}], // The type(s) of document(s) to include
|
|
60
63
|
options: {
|
|
61
64
|
embeddingsIndex: {
|
|
62
|
-
indexName: 'my-index',
|
|
63
|
-
maxResults: 10, //
|
|
64
|
-
searchMode: 'embeddings' //
|
|
65
|
+
indexName: 'my-index', // Name of the embeddings index
|
|
66
|
+
maxResults: 10, // Max. number of returned results per request. Default: 10
|
|
67
|
+
searchMode: 'embeddings' // Sets default search mode for the field. Enables toggling between 'embeddings' (semantic search) and 'default' (default search based on GROQ filter)
|
|
65
68
|
}
|
|
66
69
|
}
|
|
67
70
|
})
|
|
68
71
|
```
|
|
69
72
|
|
|
70
|
-
Setting `options.
|
|
73
|
+
Setting `options.embeddings.indexName` on a reference field enables searching into the named index.
|
|
71
74
|
|
|
72
|
-
|
|
75
|
+
*Note*: the search uses `to` types as a filter for the index. Therefore, the types that the
|
|
76
|
+
the reference field expects must exist in the index: the GROQ query specified in the embeddings index
|
|
77
|
+
`filter` must include one or more documents that are relevant to the reference field.
|
|
73
78
|
|
|
74
|
-
*
|
|
75
|
-
that the types the reference field expects actually exist in the index
|
|
76
|
-
(ie, the embeddings index `filter` contains one or more documents relevant to the reference field).
|
|
79
|
+
*Caveats*: the semantic search functionality does not honor `options.filter`.
|
|
77
80
|
|
|
78
|
-
|
|
79
|
-
`options.filter` is not respected by the semantic search.
|
|
81
|
+
### Default embeddings index configuration
|
|
80
82
|
|
|
81
|
-
|
|
83
|
+
You can enable a default configuration for the reference inputs through the plugin configuration.
|
|
82
84
|
|
|
83
|
-
|
|
85
|
+
Example:
|
|
84
86
|
|
|
85
87
|
```ts
|
|
86
88
|
import {defineConfig} from 'sanity'
|
|
@@ -89,15 +91,15 @@ import {embeddingsIndexReferenceInput} from '@sanity/embeddings-index-ui'
|
|
|
89
91
|
export default defineConfig({
|
|
90
92
|
//...
|
|
91
93
|
plugins: [embeddingsIndexReferenceInput({
|
|
92
|
-
indexName: 'my-index', //
|
|
93
|
-
maxResults: 15, //
|
|
94
|
-
searchMode: 'embeddings' //
|
|
94
|
+
indexName: 'my-index', // Inputs use 'my-index' as the default index
|
|
95
|
+
maxResults: 15, // Inputs return max. 15 results per request
|
|
96
|
+
searchMode: 'embeddings' // Semantic search is the default search mode
|
|
95
97
|
})],
|
|
96
98
|
})
|
|
97
99
|
```
|
|
98
100
|
|
|
99
|
-
|
|
100
|
-
|
|
101
|
+
If you assign a default `indexName` to the plugin, you can also enable embeddings search
|
|
102
|
+
by setting `options.embeddingsIndex: true` for a reference field:
|
|
101
103
|
|
|
102
104
|
```ts
|
|
103
105
|
defineField({
|
|
@@ -111,15 +113,15 @@ defineField({
|
|
|
111
113
|
```
|
|
112
114
|
|
|
113
115
|
## Embeddings index dashboard
|
|
114
|
-
An UI alternative to the [Embeddings index CLI](https://github.com/sanity-io/embeddings-index-cli)
|
|
115
|
-
Manage embeddings indexes in a Studio dashboard.
|
|
116
116
|
|
|
117
|
-
|
|
117
|
+
A UI alternative to the [Embeddings ˆndex CLI](https://github.com/sanity-io/embeddings-index-cli) to
|
|
118
|
+
manage embeddings indexes in a Studio dashboard.
|
|
118
119
|
|
|
120
|
+
<img width="1227" alt="image" src="https://github.com/sanity-io/sanity/assets/835514/279b03b8-d2c0-4cc1-bbe6-9d335937f25a">
|
|
119
121
|
|
|
120
|
-
|
|
122
|
+
### Usage
|
|
121
123
|
|
|
122
|
-
Add `embeddingsIndexDashboard` as a plugin
|
|
124
|
+
Add `embeddingsIndexDashboard` as a plugin to `sanity.config.ts` (or `.js`):
|
|
123
125
|
|
|
124
126
|
```ts
|
|
125
127
|
import {defineConfig} from 'sanity'
|
|
@@ -135,9 +137,9 @@ export default defineConfig({
|
|
|
135
137
|
})
|
|
136
138
|
```
|
|
137
139
|
|
|
138
|
-
This
|
|
140
|
+
This adds the Embeddings Index tool to the studio navigation bar, but only when the studio is running in developer mode (`localhost`).
|
|
139
141
|
|
|
140
|
-
|
|
142
|
+
If you want to enable the tool based on user access roles:
|
|
141
143
|
|
|
142
144
|
```ts
|
|
143
145
|
import {defineConfig} from 'sanity'
|
|
@@ -157,20 +159,17 @@ export default defineConfig({
|
|
|
157
159
|
})
|
|
158
160
|
```
|
|
159
161
|
|
|
160
|
-
|
|
161
|
-
|
|
162
162
|
## License
|
|
163
163
|
|
|
164
164
|
[MIT](LICENSE) © Sanity
|
|
165
165
|
|
|
166
|
-
## Develop
|
|
166
|
+
## Develop and test
|
|
167
167
|
|
|
168
168
|
This plugin uses [@sanity/plugin-kit](https://github.com/sanity-io/plugin-kit)
|
|
169
|
-
with default configuration for build
|
|
169
|
+
with default configuration for build and watch scripts.
|
|
170
170
|
|
|
171
171
|
See [Testing a plugin in Sanity Studio](https://github.com/sanity-io/plugin-kit#testing-a-plugin-in-sanity-studio)
|
|
172
|
-
on how to run this plugin with
|
|
173
|
-
|
|
172
|
+
on how to run this plugin with hot reload in the studio.
|
|
174
173
|
|
|
175
174
|
### Release new version
|
|
176
175
|
|