@sanity/assist 1.2.15-lang.1 → 1.2.15-lang.10

package/README.md

@@ -16,14 +16,20 @@
   - [Disable for a field](#disable-for-a-field)
   - [Disable for an array type](#disable-for-an-array-type)
   - [Unsupported types](#unsupported-types)
+  - [Hidden and readOnly fields](#hidden-and-readonly-fields)
   - [Reference support](#reference-support)
   - [Troubleshooting](#troubleshooting)
 - [Included document types](#included-document-types)
 - [Field and type filters](#field-and-type-filters)
-- [Caption generation](#caption-generation)
+- [Image description generation](#image-description-generation)
+- [Image generation](#image-generation)
 - [Full document translation](#full-document-translation)
   - [What it solves](#what-ai-assist-full-document-translations-solves)
   - [Configure](#configure-document-translations)
+- [Field level translations](#field-level-translations)
+  - [What it solves](#what-ai-assist-field-level-translations-solves)
+  - [Configure](#configure-field-translations)
+- [Adding translation actions to fields](#adding-translation-actions-to-fields)
 - [License](#license)
 - [Develop \& test](#develop--test)
   - [Release new version](#release-new-version)
@@ -49,7 +55,7 @@ In your Studio project folder, install the following plugin dependency:
 npm install @sanity/assist sanity@latest
 ```
 
-This plugin requires `sanity` version `3.16.0` or greater.
+This plugin requires `sanity` version `3.26` or greater.
 
 ## Setup
 
@@ -123,7 +129,7 @@ defineType({
  name: 'policy',
   type: 'document',
   options: {
-    aiWritingAssistance: {exclude: true}
+    aiAssist: {exclude: true}
  },
   fields: [
     // ...
@@ -143,7 +149,7 @@ defineType({
       name: 'sku',
       type: 'string',
       options: {
-        aiWritingAssistance: {exclude: true}
+        aiAssist: {exclude: true}
      }
   })
   ]
@@ -162,7 +168,7 @@ defineType({
     defineArrayMember({
       type: 'customProduct',
       options: {
-        aiWritingAssistance: {exclude: true}
+        aiAssist: {exclude: true}
      }
   })
   ]
@@ -181,14 +187,25 @@ The following types are not supported, and behave as excluded types:
 * [File](https://www.sanity.io/docs/file-type) (never supported, even when file has custom fields)
 * [Reference](https://www.sanity.io/docs/reference-type) (supported when configured with embeddingsIndex)
 
-Types and fields with `hidden` or `readonly` with a truthy value (`true` or `function`) are not supported.
-(Hidden and readOnly fields can be referenced in instructions still)
-
 Fields with these types will not be changed by the assistant, do not have AI Assist actions, and cannot be referenced in instructions.
 
 Objects where all fields are excluded or unsupported and arrays where all member types are excluded or unsupported
 will also be excluded.
 
+### Hidden and readOnly fields
+
+In AI Assist 2.0 and later, conditionally `hidden` and `readOnly` fields can have instructions.
+These fields can be written to by an instruction, as long as the field is non-hidden and writable when the instruction is started.
+
+Fields with `hidden` or `readOnly` set to literal `true` will be skipped by AI Assist.
+
+*Note*: An instruction will not re-evaluate these states during a run.
+Ie, if an instructions writes to a field that will make another field in the studio readonly or hidden,
+the running instruction will still consider these as if in their original state.
+
+If it is essential that AI Assist *never* writes to a conditional field,
+it should be marked with `options.aiAssist.exclude: true`.
+
 ### Reference support
 
 #### Create an Embeddings-index
@@ -198,7 +215,7 @@ with the documents it should be able to reference.
 You can manage your indexes directly in the studio using the [Embeddings Index Dashboard plugin](https://github.com/sanity-io/embeddings-index-ui#embeddings-index-api-dashboard-for-sanity-studio).
 
 #### Set schema options
-Set `options.aiWritingAssistance.embeddingsIndex` for reference fields/types you want to enable reference instructions for. 
+Set `options.aiAssist.embeddingsIndex` for reference fields/types you want to enable reference instructions for. 
 Reference fields with this options set can have instructions attached to them, and will be visited when running instructions for object fields and arrays.
 
 AI assist will use the embeddings-index, filtered by the types allowed by the reference to look up contextually relevant references.
@@ -215,7 +232,7 @@ defineField({
   title: 'Article referene',
   to: [ { type: 'article'} ],
   options: {
-    aiWritingAssistance: {
+    aiAssist: {
       embeddingsIndex: 'article-index'
     },
   },
@@ -264,14 +281,121 @@ Note that if the schema targeted by the instruction changes, the following behav
 * instructions that included all fields or types will automatically also include the new fields or types
 * instructions that have excluded one or more fields or types, will NOT include the new fields or types
 
-## Caption generation
-AI Assist can optionally generate captions for images. This has to be enabled on an image-type/field,
-by setting the `options.captionField` on the image type, where `captionField` is the field name of a
+## Image description generation
+AI Assist can optionally generate descriptions for images. This has to be enabled on an image-type/field,
+by setting the `options.aiAssist.imageDescriptionField` on the image type, where `imageDescriptionField` is the field name of a
 custom string-field on the image object:
 
+```tsx
+defineField({
+    type: 'image',
+    name: 'inlineImage',
+    title: 'Image',
+    fields: [
+      defineField({
+        type: 'string',
+        name: 'altText',
+        title: 'Alternative text',
+      }),
+    ],
+    options: {
+      aiAssist: {
+        imageDescriptionField: 'altText',
+      }
+    },
+})
+```
+This will add a "Generate image description" action to the configured field.
+"Generate image description" action will automatically run whenever the image changes.
+
+`imageDescriptionField` can be a nested field, if the image has object field, ie `imageDescriptionField: 'wrapper.altText'`.
+Fields within array items are not supported.
+
+## Image generation
+<img width="600" alt="image" src="https://github.com/sanity-io/assist/assets/835514/c4de6791-f530-4cd1-b0c2-96ef988bc256">
+
+AI Assist can generate assets for images configured with a prompt field.
+
+An image is generated directly by using the "Generate image from prompt" instruction on the prompt field, 
+or indirectly whenever the image prompt field is written to by an AI Assist instruction.
+
+### Configure
+To enable image generation for an image field, the image must:
+- set `options.aiAssist.imageInstructionField` to a child-path relative to the image
+- have a `string` or `text` field that corresponds to the `imageInstructionField` path
+
+This will add a "Generate image from prompt" instruction to the image prompt field. Running it will generate and image.
+
+Additionally, whenever an AI Assist instruction writes to the image prompt field, the image will be generated.
+
+This could be a document instruction, an instruction for the image field or parent object, or directly on the image prompt field.
+
+A common styleguide can achieved by adding an instruction to the image prompt field that rewrites whatever value is there, to include a common style.
+Use AI context documents to apply a reusable styleguide to the prompt rewriting as needed.
+
+#### Example
+
+Given the following document schema
+```ts
+defineType({
+  type: 'document',
+	name: 'article',
+	fields: [
+		defineField({
+		  type: 'image',
+		  name: 'articleImage',
+		  fields: [
+		    defineField({
+		      type: 'text',
+		      name: 'imagePrompt',
+		      title: 'Image prompt',
+		      rows: 2,
+		    }),
+		  ],
+		  options: {
+		    imagePromptField: 'imagePromptField',
+		  },
+		})
+	] 
+})
+```
+
+To directly generate an image based on the value in the prompt field, 
+run the "Generate image from prompt" instruction that is automatically added.
+
+For better image results or to ensure a consistent style, rewrite the prompt before generating the image:
+
+### Example prompt expansion instruction
+<img width="267" alt="image" src="https://github.com/sanity-io/assist/assets/835514/dabc6910-80d3-4a69-940f-49ac5cae9ade">
+
+For better image results, use an instruction that expands the prompt to be more detailed.
+
+Example instruction text:
+
+```
+Rewrite image prompts for image generation according to the following rules:
+- Be Specific: Include detailed descriptions of the scene, objects, colors, and any characters. Instead of saying "a cat in a garden", say "a fluffy gray cat sitting beside pink tulips in a sunny garden".
+- Set the Scene: Describe the environment or background. Mention if it's indoors or outdoors, the time of day, weather conditions, and any specific setting details like a beach, forest, cityscape, etc.
+- Detail Characters: If your image includes people or animals, specify their appearance, clothing, poses, and expressions. For example, "a smiling young woman with short black hair, wearing a red dress, standing under a tree".
+- Color and Style: Mention specific colors and artistic styles you prefer. For instance, "bright, vivid colors with a watercolor effect".
+- Mood and Atmosphere: Describe the mood or atmosphere of the image. Words like 'peaceful', 'dynamic', 'mysterious', or 'joyful' can guide the AI in capturing the right tone.
+- Avoid Ambiguity: Be clear and direct. Avoid using vague or abstract concepts that the AI might struggle to interpret.
+- Follow the Guidelines: Ensure your prompt doesn't include any content against usage policies, such as depictions of real people, copyrighted characters, or sensitive subjects.
+
+Keep it 100 words or less.
+
+The prompt to rewrite is:
+{Reference to image-prompt-field}
+```
+
+The rules can be extracted into an AI Context document and reused in other instructions as needed. This approach can also be used to inform a reusable styleguide for image generation.
 ## Full document translation
+<img width="250" alt="Translate document action" src="https://github.com/sanity-io/assist/assets/835514/932968ee-1a8c-4389-8822-338188f88b40">
+
 AI assist offers full document translations, which is ideal for pairing with [@sanity/document-internationalization](https://github.com/sanity-io/document-internationalization).
 
+Translations are done deeply; visiting nested objects, arrays and even Portable text annotations.
+
 ### What AI Assist full document translations solves
 
 Given a document written in one language, AI assist can translate the document in place to a language specified by a language field in the document.
@@ -349,6 +473,9 @@ assist({
 ```
 
 ## Field level translations
+<img width="250" alt="Translate fields action" src="https://github.com/sanity-io/assist/assets/835514/99819cd4-578e-43b2-8c70-8e39afff5f09">
+
+<img width="250" alt="Translate fields dialog" src="https://github.com/sanity-io/assist/assets/835514/fe3d289c-49b6-46dd-ae2f-cd509a01534a">
 
 AI assist offers field-level translations, which is ideal for using alongside with[sanity-plugin-internationalized-array](https://github.com/sanity-io/sanity-plugin-internationalized-array?tab=readme-ov-file#sanity-plugin-internationalized-array) and (@sanity/language-filter)[https://github.com/sanity-io/language-filter]
 
@@ -597,39 +724,48 @@ assist({
  })
 ```
 
-## Caveats
+## Adding translation actions to fields
 
-Large Language Models (LLMs) are a new technology. Constraints and limitations are still being explored, 
-but some common caveats to the field that you may run into using AI Assist are:
+<img width="250" alt="Translate action on field" src="https://github.com/sanity-io/assist/assets/835514/e6dc0860-90a7-4f7a-b3d2-71893b09862f">
 
-* Limits to instruction length: Long instructions on deep content structures may exhaust model context
-* Timeouts: To be able to write structured content, we're using the largest language models - long-running results may time out or intermittently fail
-* Limited capacity: The underlying LLM APIs used by AI Assist are resource constrained
+<img width="250" alt="Translate fields action on field" src="https://github.com/sanity-io/assist/assets/835514/acc5fa23-2022-4eae-922d-5c83dda7379c">
 
+By default, “Translate document” and “Translate fields…” instructions are only added to the document instruction dropdown.
 
+These instructions can also be added to fields by setting
+`options.aiAssist.translateAction: true` for a field or type.
 
-```tsx
+This allows editors to translate only parts of the document, and can be useful to enable on `internatinalizedArrays` or `locale` wrapper object types.
+
+For document types configured for full document translations, a "Translate" action will be added. Running it will translate the field to the language set in the language field
+
+For document types configured for field translations, a "Translate fields..." action will be added. Running it will open a dialog with language selectors.
+
+
+#### Example
+
+```ts
 defineField({
-    type: 'image',
-    name: 'inlineImage',
-    title: 'Image',
-    fields: [
-      defineField({
-        type: 'string',
-        name: 'caption',
-        title: 'Caption',
-      }),
-    ],
+    name: 'subtitle',
+    type: 'internationalizedArrayString',
+    title: 'Subtitle',
     options: {
-      captionField: 'caption', 
+        aiAssist: {
+            translateAction: true
+        }
     },
 })
 ```
-This will add a "Generate caption" action to the configured field. 
-"Generate caption" action will automatically run whenever the image changes.
 
-`captionField` can be a nested field, if the image has object field, ie `captionField: 'wrapper.caption'`.
-Fields within array items are not supported.
+## Caveats
+
+Large Language Models (LLMs) are a new technology. Constraints and limitations are still being explored, 
+but some common caveats to the field that you may run into using AI Assist are:
+
+* Limits to instruction length: Long instructions on deep content structures may exhaust model context
+* Timeouts: To be able to write structured content, we're using the largest language models - long-running results may time out or intermittently fail
+* Limited capacity: The underlying LLM APIs used by AI Assist are resource constrained
+
 
 ## Third party sub-processors
 

package/dist/index.cjs.mjs

@@ -3,4 +3,5 @@ import cjs from './index.js';
 export const SchemaTypeTool = cjs.SchemaTypeTool;
 export const assist = cjs.assist;
 export const contextDocumentTypeName = cjs.contextDocumentTypeName;
+export const defaultLanguageOutputs = cjs.defaultLanguageOutputs;
 

package/dist/index.d.ts

@@ -8,18 +8,19 @@ import {SchemaType} from 'sanity'
 export declare const assist: Plugin_2<void | AssistPluginConfig>
 
 export declare interface AssistOptions {
-  aiWritingAssistance?: {
+  aiAssist?: {
     /** Set to true to disable assistance for this field or type */
     exclude?: boolean
+    /**
+     * Set to true to add translation field-action to the field.
+     * Only has an effect in document types configured for document or field level translations.
+     */
+    translateAction?: boolean
   }
 }
 
 declare interface AssistPluginConfig {
   translate?: TranslationConfig
-  /**
-   * Set this to false to disable model migration from the alpha version of this plugin
-   */
-  alphaMigration?: boolean
   /**
    * @internal
    */
@@ -28,13 +29,21 @@ declare interface AssistPluginConfig {
 
 export declare const contextDocumentTypeName: 'assist.instruction.context'
 
-declare interface DocumentMember {
+/**
+ * Default implementation for plugin config `translate.field.translationOutputs`
+ *
+ * @see FieldTranslationConfig#translationOutputs
+ */
+export declare const defaultLanguageOutputs: TranslationOutputsFunction
+
+export declare interface DocumentMember {
   schemaType: SchemaType
   path: Path
   name: string
+  value: unknown
 }
 
-declare interface DocumentTranslationConfig {
+export declare interface DocumentTranslationConfig {
   /**
    * Path to language field in documents. Can be a hidden field.
    * For instance: 'config.language'
@@ -56,7 +65,7 @@ declare interface DocumentTranslationConfig {
   documentTypes?: string[]
 }
 
-declare interface FieldTranslationConfig {
+export declare interface FieldTranslationConfig {
   /**
    * `documentTypes` should be an array of strings where each entry must match a name from your document schemas.
    *
@@ -147,19 +156,19 @@ declare interface FieldTranslationConfig {
   translationOutputs?: TranslationOutputsFunction
 }
 
-declare interface Language {
+export declare interface Language {
   id: string
   title?: string
 }
 
-declare type LanguageCallback = (
+export declare type LanguageCallback = (
   client: SanityClient,
   selectedLanguageParams: Record<string, unknown>,
 ) => Promise<Language[]>
 
 export declare function SchemaTypeTool(): JSX_2.Element
 
-declare interface TranslationConfig {
+export declare interface TranslationConfig {
   /**
    * Config for document types with fields in multiple languages in the same document.
    */
@@ -170,13 +179,13 @@ declare interface TranslationConfig {
   document?: DocumentTranslationConfig
 }
 
-declare interface TranslationOutput {
+export declare interface TranslationOutput {
   /** Language id */
   id: string
   outputPath: Path
 }
 
-declare type TranslationOutputsFunction = (
+export declare type TranslationOutputsFunction = (
   documentMember: DocumentMember,
   enclosingType: SchemaType,
   translateFromLanguageId: string,
@@ -195,14 +204,80 @@ declare module 'sanity' {
   interface DocumentOptions extends AssistOptions {}
   interface FileOptions extends AssistOptions {}
   interface GeopointOptions extends AssistOptions {}
-  interface ImageOptions extends AssistOptions {
-    imagePromptField?: string
-    captionField?: string
+  interface ImageOptions {
+    aiAssist?: AssistOptions['aiAssist'] & {
+      /**
+       * When set, an image will be created whenever the `imageInstructionField` is written to by
+       * an AI Assist instruction.
+       *
+       * The value output by AI Assist will be use as an image prompt for an generative image AI.
+       *
+       * This means that instructions directly for the field or instructions that visit the field when running,
+       * will result in the image being changed.
+       *
+       * `imageInstructionField` must be a child-path relative to the image field, ie:
+       * * field
+       * * path.to.field
+       *
+       * ### Example
+       * ```ts
+       * defineType({
+       *   type: 'image',
+       *   name: 'articleImage',
+       *   fields: [
+       *     defineField({
+       *       type: 'text',
+       *       name: 'imagePrompt',
+       *       title: 'Image prompt',
+       *       rows: 2,
+       *     }),
+       *   ],
+       *   options: {
+       *      aiAssist: {
+       *        imageInstructionField: 'imagePrompt',
+       *     }
+       *   },
+       * })
+       * ```
+       */
+      imageInstructionField?: string
+      /**
+       * When set, an image description will be automatically created for the image.
+       *
+       * `imageDescriptionField` must be a child-path relative to the image field, ie:
+       * * field
+       * * path.to.field
+       *
+       * Whenever the image asset for the field is changed in the Studio,
+       * an image description is generated and set into the `imageDescriptionField`.
+       *
+       * ### Example
+       * ```ts
+       * defineType({
+       *   type: 'image',
+       *   name: 'articleImage',
+       *   fields: [
+       *     defineField({
+       *       type: 'string',
+       *       name: 'altText',
+       *       title: 'Alt text',
+       *     }),
+       *   ],
+       *   options: {
+       *      aiAssist: {
+       *        imageDescriptionField: 'altText',
+       *     }
+       *   },
+       * })
+       * ```
+       */
+      imageDescriptionField?: string
+    }
   }
   interface NumberOptions extends AssistOptions {}
   interface ObjectOptions extends AssistOptions {}
   interface ReferenceBaseOptions {
-    aiWritingAssistance?: {
+    aiAssist?: {
       /** Set to true to disable assistance for this field or type */
       exclude?: boolean
       /**