@sanity/ailf-studio 0.1.17 → 0.1.19

package/README.md

@@ -1,5 +1,9 @@
 # @sanity/ailf-studio
 
+> ⚠️ **Internal package.** This package is published publicly for convenience
+> but is intended for internal Sanity use only. APIs and schemas may change
+> without notice. No support is provided for external consumers.
+
 Sanity Studio dashboard plugin for the **AI Literacy Framework**. Visualizes
 evaluation reports, score trends, comparisons, and content impact — directly
 inside Sanity Studio with no external backend.
@@ -17,10 +21,6 @@ AILF reports are stored.
 pnpm add @sanity/ailf-studio
 ```
 
-> **Note:** The package is published with `restricted` access to the `@sanity`
-> npm scope. You need an npm token with read access — see the root
-> [README](../../README.md#obtain-secrets) for how to obtain one.
-
 #### Within the monorepo
 
 ```bash
@@ -57,6 +57,28 @@ This registers:
 - The `ailf.evalRequest` document type (evaluation request triggers)
 - The **AI Literacy Framework** dashboard tool in the Studio sidebar
 
+#### Document Actions
+
+The plugin registers two document actions for triggering evaluations directly
+from Studio:
+
+- **Run Task Eval** (on `ailf.task` documents) — evaluates a single task. Click
+  ▶ in the document actions menu to run all test cases for the task against the
+  current documentation. The button shows the score when complete (~10–15 min).
+  No secrets needed — it creates an `ailf.evalRequest` document that a
+  server-side webhook dispatches to the pipeline.
+
+- **Run AI Eval** (on content releases) — evaluates all tasks affected by a
+  content release. Appears in the release detail page's action bar. Answers "did
+  my doc changes help or hurt AI agent performance?" Shows score and delta vs
+  baseline when complete.
+
+Both actions use the same mechanism: they create an `ailf.evalRequest` document
+in the Content Lake with `status: "pending"`. A server-side Sanity webhook picks
+up the document and dispatches the pipeline via GitHub Actions. The Studio
+component polls for the resulting report and updates the button label with the
+score.
+
 ### 3. Alternative: tool-only installation
 
 If you only want the dashboard tool without the document schemas (e.g., the
@@ -103,6 +125,22 @@ export default defineConfig({
 })
 ```
 
+## Task Execution Workflows
+
+Tasks created in Studio are automatically included in every pipeline run — no
+registration step needed. There are four ways to execute tasks:
+
+| Method                         | Trigger                                                           | Scope                         |
+| ------------------------------ | ----------------------------------------------------------------- | ----------------------------- |
+| **Run Task Eval** action       | Click ▶ on any `ailf.task` document                               | Single task                   |
+| **Run AI Eval** release action | Click button on a content release page                            | Tasks affected by the release |
+| **CLI pipeline**               | `ailf pipeline` (with optional `--area`/`--task`/`--tag` filters) | All enabled tasks             |
+| **Scheduled pipeline**         | GitHub Actions cron (daily + weekly)                              | All enabled tasks             |
+
+See the
+[CONTRIBUTING_TASKS](https://github.com/sanity-labs/ai-literacy-framework/blob/main/docs/CONTRIBUTING_TASKS.md#running-your-task)
+guide for the full execution flow and details on each method.
+
 ## Dashboard Views
 
 The plugin provides three tab views plus a detail drill-down, accessible from
@@ -178,15 +216,13 @@ To point it at a dedicated report dataset, configure the Studio's dataset:
 
 ```ts
 export default defineConfig({
-  projectId: "3do82whm",
-  dataset: "my-report-dataset", // or use AILF_REPORT_DATASET
+  projectId: "<your-project-id>",
+  dataset: "my-report-dataset",
   plugins: [ailfPlugin()],
 })
 ```
 
-Reports are written by the evaluation pipeline (`ailf pipeline --publish`). See
-the [report store design docs](../../docs/design-docs/report-store/index.md) for
-the full architecture.
+Reports are written by the evaluation pipeline (`ailf pipeline --publish`).
 
 ## Exported API
 
@@ -299,9 +335,9 @@ consuming Studio's bundler (Vite) handles the final bundle.
 
 ## Related Documentation
 
-- [Report Store Design](../../docs/design-docs/report-store/index.md) — full
-  architecture and implementation plan
-- [Visibility & Workflows](../../docs/design-docs/report-store/visibility-workflows.md)
+- [Report Store Design](https://github.com/sanity-labs/ai-literacy-framework/blob/main/docs/design-docs/report-store/index.md)
+  — full architecture and implementation plan
+- [Visibility & Workflows](https://github.com/sanity-labs/ai-literacy-framework/blob/main/docs/design-docs/report-store/visibility-workflows.md)
   — design rationale for the dashboard views
-- [Report Store Architecture](../../docs/design-docs/report-store/architecture.md)
+- [Report Store Architecture](https://github.com/sanity-labs/ai-literacy-framework/blob/main/docs/design-docs/report-store/architecture.md)
   — Sanity Content Lake as the system of record

package/dist/index.d.ts

@@ -436,10 +436,24 @@ declare const reportSchema: {
  * - A gold-standard implementation (reference solution)
  * - When/how the task runs (execution controls)
  *
+ * ## Execution paths
+ *
+ * Published tasks are automatically discovered by the pipeline — no
+ * registration step needed. There are four ways to execute a task:
+ *
+ * 1. **Run Task Eval** — click ▶ on any ailf.task document in Studio.
+ *    Creates an ailf.evalRequest scoped to this task. Webhook dispatches
+ *    the pipeline; score appears inline when complete (~10–15 min).
+ * 2. **Run AI Eval** — click on a content release page. Auto-scopes to
+ *    tasks whose canonical docs are in the release.
+ * 3. **CLI** — `ailf pipeline --task <id>` or `ailf pipeline --area <area>`.
+ * 4. **Scheduled** — GitHub Actions cron (daily baseline, weekly full).
+ *
  * Tasks can be authored natively in Studio or mirrored from external
  * repositories. Mirrored tasks have a read-only `origin` block that
  * tracks their source repo provenance.
  *
+ * @see docs/CONTRIBUTING_TASKS.md#running-your-task — full execution guide
  * @see docs/design-docs/tasks-as-content.md
  * @see docs/design-docs/tasks-as-content.md#decision-8-domain-specific-assertion-types-not-a-promptfoo-subset
  */

package/dist/index.js

@@ -2481,6 +2481,26 @@ var taskSchema = defineType5({
     }),
     // -----------------------------------------------------------------------
     // Execution controls
+    //
+    // Pipeline support status:
+    //   - `enabled`  → ✅ used in GROQ filter (execution.enabled != false)
+    //   - `blocking` → 🔮 forward-looking — schema reserved, not yet enforced
+    //                   by the pipeline. PR check workflow reports scores but
+    //                   does not gate merges on this flag.
+    //   - `trigger.branches`, `trigger.paths` → ✅ used by repo-based task
+    //                   threshold evaluator (RepoThresholdEvaluator)
+    //   - `trigger.labels`, `trigger.schedule` → 🔮 forward-looking — schema
+    //                   reserved, not yet read by the pipeline GROQ query or
+    //                   mapped to TaskDefinition.
+    //   - `threshold.score` → ✅ used by repo-based threshold evaluator
+    //   - `threshold.dimensions` → 🔮 forward-looking — schema reserved, not
+    //                   yet read by the pipeline GROQ query or mapped to
+    //                   TaskDefinition.
+    //
+    // When implementing a forward-looking field, update the GROQ query in
+    // content-lake-task-source.ts, the ContentLakeTask interface, the
+    // mapToTaskDefinition() mapping, and (if pipeline-relevant) the
+    // TaskDefinition port type in @sanity/ailf-core.
     // -----------------------------------------------------------------------
     defineField5({
       description: "Controls when and how this task runs in the pipeline. Tasks without execution controls use defaults: enabled, non-blocking, no filters, no threshold.",

package/package.json

@@ -1,12 +1,12 @@
 {
   "name": "@sanity/ailf-studio",
-  "version": "0.1.17",
+  "version": "0.1.19",
   "description": "AI Literacy Framework — Sanity Studio dashboard plugin",
   "type": "module",
   "license": "MIT",
   "private": false,
   "publishConfig": {
-    "access": "restricted"
+    "access": "public"
   },
   "repository": {
     "type": "git",