sanity-plugin-jev-fields 0.0.0-stage → 0.1.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Frederik von Sperling
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,4 +1,185 @@
-# Temporary Holding Version
+# sanity-plugin-jev-fields
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
-If no other versions are published within 30 days, this package and version will be deleted.
+> Beta: expect `0.x` releases to change until the stored value shapes are frozen at `1.0.0`.
+
+Jev signals for Sanity Studio: questions about a field's content, answered by
+[TypeSafe's Jev](https://typesafe.ai) decision model through
+[Vercel AI Gateway](https://vercel.com/ai-gateway) as editors write. Each signal is a chip under
+the field it judges; click it for the details.
+
+- `noul`: a yes/no question, answered with the probability that the answer is yes
+- `score`: a position on an ordered scale you define
+- `choice`: one option from a named set, with a probability for each option
+
+Signals re-evaluate shortly after the field is edited. Opening a document never writes to it.
+
+![A Body field in Sanity Studio with three signal chips under it: Readable 90%, Evidence 1.5 of 3 and Tone Casual. The Evidence details are open: a four-step bar from None to Cited, filled to Anecdotal, and the hint "To move up: add a source or figure."](https://raw.githubusercontent.com/frederikvonsperling/sanity-plugin-jev-fields/main/docs/images/signals.png)
+
+## Install
+
+```sh
+pnpm add sanity-plugin-jev-fields
+```
+
+Requires Sanity Studio 6.
+
+## Usage
+
+Add the plugin, and wrap your schema types with `withJevAnswers` so every signal gets a field to
+store its answer in:
+
+```ts
+// sanity.config.ts
+import {defineConfig} from 'sanity'
+import {jev, withJevAnswers} from 'sanity-plugin-jev-fields'
+
+export default defineConfig({
+  // ...
+  plugins: [jev()],
+  schema: {types: withJevAnswers(schemaTypes)},
+})
+```
+
+Then attach signals to any field with `options.jev`:
+
+```ts
+import {defineArrayMember, defineField} from 'sanity'
+import {choice, noul, score} from 'sanity-plugin-jev-fields'
+
+defineField({
+  name: 'body',
+  type: 'array',
+  of: [defineArrayMember({type: 'block'})],
+  options: {
+    jev: {
+      readable: noul({
+        instructions: 'Is this article easy to read for a general audience?',
+        true: 'Short sentences, plain words, clear structure',
+        false: 'Dense, jargon-heavy or hard to follow',
+        label: 'likely to read easily',
+      }),
+      evidence: score({
+        instructions: 'How well does this article support its claims?',
+        // Lowest first. Text before a colon is the label.
+        criteria: [
+          'none: no backing for its claims',
+          'anecdotal: personal experience only',
+          'data: add a source or figure',
+          'cited: key claims cite their sources',
+        ],
+      }),
+      tone: choice({
+        instructions: 'What is the tone of this article?',
+        criteria: {formal: 'Professional and reserved', casual: 'Conversational and relaxed'},
+      }),
+    },
+  },
+})
+```
+
+A signal reads only the field it is attached to. Portable Text, slugs and nested objects are
+flattened to plain text. Each key (`readable`, `evidence`, `tone`) becomes the name of a field
+next to it that stores the answer; `withJevAnswers` adds those fields and stops with an error if a
+name is already taken.
+
+## Rules
+
+Give a signal a `warn` or `require` rule to act on its answer: `warn` shows a warning on the
+attached field, `require` an error that blocks publishing.
+
+```ts
+options: {
+  jev: {
+    readable: noul({...signal, warn: {atLeast: 0.6}}), // probability, 0–1
+    evidence: score({...signal, require: {atLeast: 2}}), // position on the scale
+    tone: choice({...signal, warn: {oneOf: ['formal', 'casual']}}),
+  },
+}
+```
+
+Rules judge the stored answer, so they say nothing until a signal has been evaluated, and an
+answer that is out of date is still judged as it is.
+
+## What Jev is good at
+
+Jev judges meaning in text: tone, clarity, whether claims are backed up. It reads only text, so
+images and other media in a field are ignored. It is not built for counting or arithmetic: ask
+"Is this under 150 words?" in a validation rule instead. Test signals on content in your own
+languages before relying on them. A field's text may be up to about 32k tokens; the Studio warns
+when a field gets close.
+
+## API key
+
+Open the **Jev** tool in the Studio, click **Set key** and paste an AI Gateway API key. The tool
+shows whether a key is set, when it last changed, and can test the connection.
+
+The key is stored in the document `secrets.jev` in your dataset. Unauthenticated queries can't
+read it, but anyone who can read the dataset can: every Studio user, and every API token with read
+access, such as a frontend's read or preview token. It is also included in dataset exports and
+backups. Use a dedicated key with a spend limit.
+
+If editors and tokens must never see the key, send requests through your own server instead:
+
+```ts
+jev({
+  transport: (request, {signal}) =>
+    fetch('/api/jev', {method: 'POST', body: JSON.stringify(request), signal}),
+})
+```
+
+Your endpoint forwards the body unchanged to `https://ai-gateway.vercel.sh/v1/evaluate` with an
+`Authorization: Bearer <key>` header, and returns the Gateway's response.
+
+## Stored values
+
+```groq
+*[_type == "article"]{
+  title,
+  "readable": readable.probability,        // 0–1
+  "evidence": evidence{score, max, label},  // score runs from 0 to max
+  "tone": tone.choice
+}
+```
+
+Every value also stores `evaluatedAt`, `model` (e.g. `typesafe-ai/jev`) and a `sourceHash` the
+Studio uses to show when a value is out of date.
+
+## Translations
+
+Everything editors see is in the `jev` namespace of the Studio's translations, in US English.
+Add another language with a locale bundle that uses the same keys (see `JevTranslationKey`):
+
+```ts
+import {defineLocaleResourceBundle} from 'sanity'
+import {JEV_NAMESPACE} from 'sanity-plugin-jev-fields'
+
+export const jevNorwegian = defineLocaleResourceBundle({
+  locale: 'nb-NO',
+  namespace: JEV_NAMESPACE,
+  resources: {'strip.set-up': 'Sett opp Jev' /* … */},
+})
+```
+
+Rule messages and config problems stay in English: Sanity's validation has no translation hook,
+and config problems are meant for schema authors.
+
+## Development
+
+```sh
+pnpm install
+pnpm test
+pnpm build
+```
+
+`pnpm record-fixtures` re-records the real Gateway responses in `src/__fixtures__` (needs
+`AI_GATEWAY_API_KEY` in `.env`). [CONTEXT.md](./CONTEXT.md) defines the vocabulary and
+[docs/adr](./docs/adr) records the main decisions.
+
+To release a change, add a changeset to its PR with `pnpm changeset`. The release workflow keeps a
+"Version packages" PR up to date; merging it stages the new version on npm (trusted publishing,
+no token) and creates the GitHub release. A maintainer then approves the staged version on
+npmjs.com, with 2FA, to publish it.
+
+## License
+
+MIT

package/dist/index.d.ts

@@ -0,0 +1,283 @@
+import { SchemaTypeDefinition } from "sanity";
+import "react";
+/** @public */
+type JevQuestion = {
+  type: 'boolean';
+  instructions: string;
+  criteria: {
+    true: string;
+    false: string;
+  };
+} | {
+  type: 'score';
+  instructions: string;
+  criteria: string[];
+} | {
+  type: 'choice';
+  instructions: string;
+  criteria: Record<string, string>;
+};
+/** @public */
+type JevAnswer = {
+  type: 'boolean';
+  probability: number;
+} | {
+  type: 'score';
+  score: number;
+  probabilities: Record<string, number>;
+  confidence?: number;
+} | {
+  type: 'choice';
+  choice: string;
+  probabilities: Record<string, number>;
+  confidence?: number;
+};
+/**
+ * Body of a request to AI Gateway's `/v1/evaluate`. A `transport` receives it as-is, so a
+ * proxy only has to forward it with an `Authorization` header.
+ * @public
+ */
+interface JevRequest {
+  model: string;
+  state: string | Record<string, string>;
+  questions: {
+    q: JevQuestion;
+  };
+  providerOptions: {
+    gateway: {
+      tags: string[];
+    };
+  };
+}
+/**
+ * Sends one request and resolves with the HTTP response, which must carry AI Gateway's
+ * status and JSON body. Errors and retries are handled by the plugin either way.
+ * @public
+ */
+type JevTransport = (request: JevRequest, init: {
+  signal?: AbortSignal;
+}) => Promise<Response>;
+interface JevPluginConfig {
+  /** Vercel AI Gateway key. Anything set here is bundled into the Studio's JavaScript. */
+  apiKey?: string;
+  /** Decision model to call. Defaults to `typesafe-ai/jev`. */
+  model?: string;
+  /** Delay after the last edit before re-evaluating. Defaults to 500ms. */
+  debounceMs?: number;
+  /** Defaults to `https://ai-gateway.vercel.sh/v1/evaluate`. Ignored when `transport` is set. */
+  endpoint?: string;
+  /**
+   * Sends evaluation requests yourself, e.g. through your own server so the API key never
+   * reaches the browser. Takes precedence over `apiKey`.
+   */
+  transport?: JevTransport;
+  /**
+   * Adds a "Jev" tool to the Studio for setting, testing and removing the stored API key.
+   * Defaults to `true`.
+   */
+  tool?: boolean;
+  /**
+   * AI Gateway reporting tags, for cost attribution. Defaults to `['feature:jev-fields']`.
+   * Each request also gets a per-signal tag, e.g. `jev.noul:article.readable`.
+   */
+  tags?: string[];
+}
+/** @public */
+interface ChoiceSignal extends SignalBase {
+  type: 'choice';
+  /** Two to 255 options: option name → what it means. */
+  criteria: Record<string, string>;
+  /** Warn unless the answer is one of these options, e.g. `{oneOf: ['formal', 'casual']}`. */
+  warn?: ChoiceRule;
+  /** Block publishing unless the answer is one of these options. */
+  require?: ChoiceRule;
+}
+/** Options an answer must be one of. @public */
+interface ChoiceRule {
+  oneOf: string[];
+}
+/** One option from a named set. @public */
+export declare const choice: (signal: Omit<ChoiceSignal, "type">) => ChoiceSignal;
+interface ChoiceProbability {
+  _key: string;
+  _type?: 'jev.choiceProbability';
+  option?: string;
+  probability?: number;
+}
+/** @public */
+interface ChoiceValue extends EvaluatedValue {
+  _type?: 'jev.choice';
+  choice?: string;
+  /** The model's confidence in this choice (0–1). */
+  confidence?: number;
+  probabilities?: ChoiceProbability[];
+}
+/** @public */
+interface NoulSignal extends SignalBase {
+  type: 'noul';
+  /** What a "yes" means. Shown when the probability is 50% or higher. */
+  true: string;
+  /** What a "no" means. Shown when the probability is below 50%. */
+  false: string;
+  /** Short phrase after the percentage, e.g. "likely to read easily". */
+  label?: string;
+  /** Warn when the probability is outside these bounds (0–1), e.g. `{atLeast: 0.6}`. */
+  warn?: RangeRule;
+  /** Block publishing when the probability is outside these bounds (0–1). */
+  require?: RangeRule;
+}
+/** A yes/no signal: the chip shows the probability that the answer is yes. @public */
+export declare const noul: (signal: Omit<NoulSignal, "type">) => NoulSignal;
+/** @public */
+interface NoulValue extends EvaluatedValue {
+  _type?: 'jev.noul';
+  /** Probability (0–1) that the answer is yes. */
+  probability?: number;
+}
+/** @public */
+interface ScoreSignal extends SignalBase {
+  type: 'score';
+  /**
+   * Two to ten criteria, lowest first. Text before a colon becomes the short label,
+   * e.g. `'anecdotal: personal experience only'`.
+   */
+  criteria: string[];
+  /**
+   * `traffic` (default): red at the bottom, green at the top.
+   * `reverse`: green at the bottom, for scales where lower is better (e.g. risk).
+   * `neutral`: one colour, for scales that aren't good or bad (e.g. reading level).
+   */
+  colors?: 'traffic' | 'reverse' | 'neutral';
+  /** Warn when the score is outside these bounds, as criterion positions: `{atLeast: 2}`. */
+  warn?: RangeRule;
+  /** Block publishing when the score is outside these bounds. */
+  require?: RangeRule;
+}
+/** A position on an ordered scale. @public */
+export declare const score: (signal: Omit<ScoreSignal, "type">) => ScoreSignal;
+/** @public */
+interface ScoreValue extends EvaluatedValue {
+  _type?: 'jev.score';
+  /** Interpolated position on the scale, from 0 to `max`. */
+  score?: number;
+  /** Index of the top criterion (number of criteria minus one). */
+  max?: number;
+  /** Short label of the nearest criterion. */
+  label?: string;
+  /** The model's confidence in this score (0–1). */
+  confidence?: number;
+}
+interface SignalBase {
+  /** Chip and detail heading. Defaults to the signal's key, e.g. `readable` → "Readable". */
+  title?: string;
+  /** The question Jev answers about the attached field. */
+  instructions: string;
+}
+interface EvaluatedValue {
+  evaluatedAt?: string;
+  /** The model that answered, as AI Gateway names it, e.g. `typesafe-ai/jev`. */
+  model?: string;
+  /** Fingerprint of the evaluated content and question, used to detect stale results. */
+  sourceHash?: string;
+}
+/** Bounds on a probability (Noul, 0–1) or a score (0 to the top criterion). @public */
+interface RangeRule {
+  atLeast?: number;
+  atMost?: number;
+}
+/** @public */
+type JevSignal = NoulSignal | ScoreSignal | ChoiceSignal;
+/** Signals attached to a field, keyed by the name of the field that stores each answer. */
+type JevSignals = Record<string, JevSignal>;
+/**
+ * Adds a field that stores each signal's answer next to every field with `options.jev`.
+ * Plugins can't see the Studio's own schema types, so wrap them in `sanity.config`:
+ *
+ * ```ts
+ * schema: {types: withJevAnswers(schemaTypes)}
+ * ```
+ * @public
+ */
+export declare function withJevAnswers<T extends SchemaTypeDefinition>(types: T[]): T[];
+/** The plugin's translation namespace: `useTranslation(JEV_NAMESPACE)`. */
+export declare const JEV_NAMESPACE: "jev";
+/**
+ * Every string an editor sees, in US English. Other locales can add a bundle for the `jev`
+ * namespace with the same keys. Config problems and validation messages stay in English: they
+ * are for schema authors, and Sanity's validation has no translation hook.
+ */
+declare const resources: {
+  readonly 'strip.label': "Jev";
+  readonly 'strip.set-up': "Set up Jev";
+  readonly 'strip.evaluate-all': "Evaluate all signals now";
+  readonly 'chip.empty': "–";
+  readonly 'chip.error': "Error";
+  readonly 'chip.out-of-date': "Out of date";
+  readonly 'detail.empty-field': "Add content to this field to evaluate it.";
+  readonly 'detail.not-evaluated': "Not evaluated yet.";
+  readonly 'detail.out-of-date': "Out of date: the field changed since this was evaluated.";
+  readonly 'detail.update-key': "Update API key";
+  readonly 'detail.try-again': "Try again";
+  readonly 'field.too-long': "This field is about {{tokens}}k tokens long. Jev reads at most {{limit}}k per question, so longer text may be refused.";
+  readonly 'error.auth': "AI Gateway rejected the API key. Check the key in the Jev tool.";
+  readonly 'error.invalid': "AI Gateway rejected the question: {{detail}}";
+  readonly 'error.busy': "AI Gateway is busy. Try again in a moment.";
+  readonly 'error.failed': "AI Gateway responded with {{status}}.";
+  readonly 'error.failed-with-detail': "AI Gateway responded with {{status}}: {{detail}}";
+  readonly 'error.unexpected': "AI Gateway returned no answer.";
+  readonly 'noul.level.low': "Low";
+  readonly 'noul.level.medium': "Medium";
+  readonly 'noul.level.high': "High";
+  readonly 'noul.yes': "Yes";
+  readonly 'score.summary': "{{meaning}} (score {{score}} of {{max}}).";
+  readonly 'score.next': "To move up: {{next}}.";
+  readonly 'key-dialog.title': "AI Gateway API key";
+  readonly 'key-dialog.label': "API key";
+  readonly 'key-dialog.description': "Create one in the Vercel dashboard under AI Gateway → API Keys, ideally with a spend limit. It is stored in this dataset, so Studio users and API tokens that can read the dataset can see it.";
+  readonly 'key-dialog.cancel': "Cancel";
+  readonly 'key-dialog.save': "Save";
+  readonly 'key-dialog.saving': "Saving…";
+  readonly 'key-dialog.save-failed': "Could not save the key: {{error}}";
+  readonly 'tool.title': "Jev";
+  readonly 'tool.intro': "Jev signals are answered by TypeSafe’s Jev model through Vercel AI Gateway.";
+  readonly 'tool.key.heading': "AI Gateway API key";
+  readonly 'tool.key.status.config': "From plugin config";
+  readonly 'tool.key.status.checking': "Checking";
+  readonly 'tool.key.status.set': "Set";
+  readonly 'tool.key.status.not-set': "Not set";
+  readonly 'tool.key.from-transport': "Requests go through the transport in the plugin config, so no key is needed here.";
+  readonly 'tool.key.from-config': "The key comes from apiKey in the plugin config. A key stored here is ignored.";
+  readonly 'tool.key.ends-with': "Key ending in <Code>{{last4}}</Code>";
+  readonly 'tool.key.changed': "Last changed {{date}}";
+  readonly 'tool.key.checking': "Checking for a stored key…";
+  readonly 'tool.key.none': "No key is stored, so Jev signals cannot evaluate yet.";
+  readonly 'tool.key.set': "Set key";
+  readonly 'tool.key.change': "Change key";
+  readonly 'tool.key.remove': "Remove key";
+  readonly 'tool.key.remove-confirm': "Remove the key? Jev signals stop evaluating until a new one is set.";
+  readonly 'tool.key.remove-cancel': "Cancel";
+  readonly 'tool.key.remove-confirm-button': "Remove";
+  readonly 'tool.key.remove-failed': "Could not remove the key: {{error}}";
+  readonly 'tool.test.run': "Test connection";
+  readonly 'tool.test.running': "Testing…";
+  readonly 'tool.test.passed': "Connection works. {{model}} answered.";
+  readonly 'tool.visibility.heading': "Who can see the key";
+  readonly 'tool.visibility.body': "The key is stored in this dataset in the document <Code>secrets.jev</Code>. It is not public, but anyone who can read the dataset can see it: every Studio user, and every API token with read access, such as a frontend’s read or preview token. It is also included in dataset exports and backups. Use a dedicated key with a spend limit. If editors and tokens must never see the key, set a <Code>transport</Code> that sends requests through your own server.";
+};
+/** A key of the plugin's translations. */
+type JevTranslationKey = keyof typeof resources;
+declare module 'sanity' {
+  interface BaseSchemaTypeOptions {
+    /** Jev signals shown on this field, keyed by the name of the field storing each answer. */
+    jev?: JevSignals;
+  }
+}
+/**
+ * Jev signals for Sanity Studio: yes/no, score and choice questions attached to a field with
+ * `options.jev`, answered by TypeSafe's Jev model through Vercel AI Gateway. Pair it with
+ * `withJevAnswers(schemaTypes)` in `schema.types`.
+ * @public
+ */
+export declare const jev: import("sanity").Plugin<void | JevPluginConfig>;
+export type { ChoiceRule, ChoiceSignal, ChoiceValue, JevAnswer, JevPluginConfig, JevQuestion, JevRequest, JevSignal, JevSignals, JevTranslationKey, JevTransport, NoulSignal, NoulValue, RangeRule, ScoreSignal, ScoreValue };
+//# sourceMappingURL=index.d.ts.map