@sanity/assist 1.2.15-lang.2 → 1.2.15-lang.4

package/README.md

@@ -21,9 +21,14 @@
 - [Included document types](#included-document-types)
 - [Field and type filters](#field-and-type-filters)
 - [Caption generation](#caption-generation)
+- [Image generation](#image-generation)
 - [Full document translation](#full-document-translation)
   - [What it solves](#what-ai-assist-full-document-translations-solves)
   - [Configure](#configure-document-translations)
+- [Field level translations](#field-level-translations)
+  - [What it solves](#what-ai-assist-field-level-translations-solves)
+  - [Configure](#configure-field-translations)
+- [Adding translation actions to fields](#adding-translation-actions-to-fields)
 - [License](#license)
 - [Develop \& test](#develop--test)
   - [Release new version](#release-new-version)
@@ -269,7 +274,110 @@ AI Assist can optionally generate captions for images. This has to be enabled on
 by setting the `options.captionField` on the image type, where `captionField` is the field name of a
 custom string-field on the image object:
 
+```tsx
+defineField({
+    type: 'image',
+    name: 'inlineImage',
+    title: 'Image',
+    fields: [
+      defineField({
+        type: 'string',
+        name: 'caption',
+        title: 'Caption',
+      }),
+    ],
+    options: {
+      captionField: 'caption', 
+    },
+})
+```
+This will add a "Generate caption" action to the configured field.
+"Generate caption" action will automatically run whenever the image changes.
+
+`captionField` can be a nested field, if the image has object field, ie `captionField: 'wrapper.caption'`.
+Fields within array items are not supported.
+
+## Image generation
+<img width="600" alt="image" src="https://github.com/sanity-io/assist/assets/835514/c4de6791-f530-4cd1-b0c2-96ef988bc256">
+
+AI Assist can generate assets for images configured with a prompt field.
+
+An image is generated directly by using the "Generate image from prompt" instruction on the prompt field, 
+or indirectly whenever the image prompt field is written to by an AI Assist instruction.
+
+### Configure
+To enable image generation for an image field, the image must:
+- set `options.imagePromptField` to a child-path relative to the image
+- have a `string` or `text` field that corresponds to the `imagePromptField` path
+
+This will add a "Generate image from prompt" instruction to the image prompt field. Running it will generate and image.
+
+Additionally, whenever an AI Assist instruction writes to the image prompt field, the image will be generated.
+
+This could be a document instruction, an instruction for the image field or parent object, or directly on the image prompt field.
+
+A common styleguide can achieved by adding an instruction to the image prompt field that rewrites whatever value is there, to include a common style.
+Use AI context documents to apply a reusable styleguide to the prompt rewriting as needed.
+
+#### Example
+
+Given the following document schema
+```ts
+defineType({
+  type: 'document',
+	name: 'article',
+	fields: [
+		defineField({
+		  type: 'image',
+		  name: 'articleImage',
+		  fields: [
+		    defineField({
+		      type: 'text',
+		      name: 'imagePrompt',
+		      title: 'Image prompt',
+		      rows: 2,
+		    }),
+		  ],
+		  options: {
+		    imagePromptField: 'imagePromptField',
+		  },
+		})
+	] 
+})
+```
+
+To directly generate an image based on the value in the prompt field, 
+run the "Generate image from prompt" instruction that is automatically added.
+
+For better image results or to ensure a consistent style, rewrite the prompt before generating the image:
+
+### Example prompt expansion instruction
+<img width="267" alt="image" src="https://github.com/sanity-io/assist/assets/835514/dabc6910-80d3-4a69-940f-49ac5cae9ade">
+
+For better image results, use an instruction that expands the prompt to be more detailed.
+
+Example instruction text:
+
+```
+Rewrite image prompts for image generation according to the following rules:
+- Be Specific: Include detailed descriptions of the scene, objects, colors, and any characters. Instead of saying "a cat in a garden", say "a fluffy gray cat sitting beside pink tulips in a sunny garden".
+- Set the Scene: Describe the environment or background. Mention if it's indoors or outdoors, the time of day, weather conditions, and any specific setting details like a beach, forest, cityscape, etc.
+- Detail Characters: If your image includes people or animals, specify their appearance, clothing, poses, and expressions. For example, "a smiling young woman with short black hair, wearing a red dress, standing under a tree".
+- Color and Style: Mention specific colors and artistic styles you prefer. For instance, "bright, vivid colors with a watercolor effect".
+- Mood and Atmosphere: Describe the mood or atmosphere of the image. Words like 'peaceful', 'dynamic', 'mysterious', or 'joyful' can guide the AI in capturing the right tone.
+- Avoid Ambiguity: Be clear and direct. Avoid using vague or abstract concepts that the AI might struggle to interpret.
+- Follow the Guidelines: Ensure your prompt doesn't include any content against usage policies, such as depictions of real people, copyrighted characters, or sensitive subjects.
+
+Keep it 100 words or less.
+
+The prompt to rewrite is:
+{Reference to image-prompt-field}
+```
+
+The rules can be extracted into an AI Context document and reused in other instructions as needed. This approach can also be used to inform a reusable styleguide for image generation.
 ## Full document translation
+<img width="250" alt="Translate document action" src="https://github.com/sanity-io/assist/assets/835514/932968ee-1a8c-4389-8822-338188f88b40">
+
 AI assist offers full document translations, which is ideal for pairing with [@sanity/document-internationalization](https://github.com/sanity-io/document-internationalization).
 
 ### What AI Assist full document translations solves
@@ -349,6 +457,9 @@ assist({
 ```
 
 ## Field level translations
+<img width="250" alt="Translate fields action" src="https://github.com/sanity-io/assist/assets/835514/99819cd4-578e-43b2-8c70-8e39afff5f09">
+
+<img width="250" alt="Translate fields dialog" src="https://github.com/sanity-io/assist/assets/835514/fe3d289c-49b6-46dd-ae2f-cd509a01534a">
 
 AI assist offers field-level translations, which is ideal for using alongside with[sanity-plugin-internationalized-array](https://github.com/sanity-io/sanity-plugin-internationalized-array?tab=readme-ov-file#sanity-plugin-internationalized-array) and (@sanity/language-filter)[https://github.com/sanity-io/language-filter]
 
@@ -597,39 +708,48 @@ assist({
  })
 ```
 
-## Caveats
+## Adding translation actions to fields
 
-Large Language Models (LLMs) are a new technology. Constraints and limitations are still being explored, 
-but some common caveats to the field that you may run into using AI Assist are:
+<img width="250" alt="Translate action on field" src="https://github.com/sanity-io/assist/assets/835514/e6dc0860-90a7-4f7a-b3d2-71893b09862f">
 
-* Limits to instruction length: Long instructions on deep content structures may exhaust model context
-* Timeouts: To be able to write structured content, we're using the largest language models - long-running results may time out or intermittently fail
-* Limited capacity: The underlying LLM APIs used by AI Assist are resource constrained
+<img width="250" alt="Translate fields action on field" src="https://github.com/sanity-io/assist/assets/835514/acc5fa23-2022-4eae-922d-5c83dda7379c">
 
+By default, “Translate document” and “Translate fields…” instructions are only added to the document instruction dropdown.
 
+These instructions can also be added to fields by setting
+`options.aiWritingAssistance.translateAction: true` for a field or type.
 
-```tsx
+This allows editors to translate only parts of the document, and can be useful to enable on `internatinalizedArrays` or `locale` wrapper object types.
+
+For document types configured for full document translations, a "Translate" action will be added. Running it will translate the field to the language set in the language field
+
+For document types configured for field translations, a "Translate fields..." action will be added. Running it will open a dialog with language selectors.
+
+
+#### Example
+
+```ts
 defineField({
-    type: 'image',
-    name: 'inlineImage',
-    title: 'Image',
-    fields: [
-      defineField({
-        type: 'string',
-        name: 'caption',
-        title: 'Caption',
-      }),
-    ],
+    name: 'subtitle',
+    type: 'internationalizedArrayString',
+    title: 'Subtitle',
     options: {
-      captionField: 'caption', 
+        aiWritingAssistance: {
+            translateAction: true
+        }
     },
 })
 ```
-This will add a "Generate caption" action to the configured field. 
-"Generate caption" action will automatically run whenever the image changes.
 
-`captionField` can be a nested field, if the image has object field, ie `captionField: 'wrapper.caption'`.
-Fields within array items are not supported.
+## Caveats
+
+Large Language Models (LLMs) are a new technology. Constraints and limitations are still being explored, 
+but some common caveats to the field that you may run into using AI Assist are:
+
+* Limits to instruction length: Long instructions on deep content structures may exhaust model context
+* Timeouts: To be able to write structured content, we're using the largest language models - long-running results may time out or intermittently fail
+* Limited capacity: The underlying LLM APIs used by AI Assist are resource constrained
+
 
 ## Third party sub-processors
 

package/dist/index.d.ts

@@ -11,6 +11,11 @@ export declare interface AssistOptions {
   aiWritingAssistance?: {
     /** Set to true to disable assistance for this field or type */
     exclude?: boolean
+    /**
+     * Set to true to add translation field-action to the field.
+     * Only has an effect in document types configured for document or field level translations.
+     */
+    translateAction?: boolean
   }
 }
 
@@ -196,7 +201,62 @@ declare module 'sanity' {
   interface FileOptions extends AssistOptions {}
   interface GeopointOptions extends AssistOptions {}
   interface ImageOptions extends AssistOptions {
+    /**
+     * When set, an image will be created whenever the `imagePromptField` is written to by
+     * an AI Assist instruction.
+     *
+     * The value output by AI Assist will be use as an image prompt for an generative image AI.
+     *
+     * This means that instructions directly for the field or instructions that visit the field when running,
+     * will result in the image being changed.
+     *
+     * `imagePromptField` must be a child-path relative to the image field.
+     * ### Example
+     * ```ts
+     * defineType({
+     *   type: 'image',
+     *   name: 'articleImage',
+     *   fields: [
+     *     defineField({
+     *       type: 'text',
+     *       name: 'imagePrompt',
+     *       title: 'Image prompt',
+     *       rows: 2,
+     *     }),
+     *   ],
+     *   options: {
+     *     imagePromptField: 'imagePromptField',
+     *   },
+     * })
+     * ```
+     */
     imagePromptField?: string
+    /**
+     * When set, an image caption will be automatically created for the image.
+     *
+     * `captionField` must be a child-path relative to the image field.
+     *
+     * Whenever the image asset for the field is changed in the Studio,
+     * an image caption is generated and set into the `captionField`.
+     *
+     * ### Example
+     * ```ts
+     * defineType({
+     *   type: 'image',
+     *   name: 'articleImage',
+     *   fields: [
+     *     defineField({
+     *       type: 'string',
+     *       name: 'altText',
+     *       title: 'Alt text',
+     *     }),
+     *   ],
+     *   options: {
+     *     captionField: 'altText',
+     *   },
+     * })
+     * ```
+     */
     captionField?: string
   }
   interface NumberOptions extends AssistOptions {}