@salesforce/afv-skills 1.66.0 → 1.68.0

package/README.md

@@ -1,15 +1,52 @@
-# Salesforce Skills Library
+# Salesforce Skills and Plugins Library
 
-This repository provides a curated collection of Salesforce agent skills for building applications. It includes skills for Agentforce agents, Lightning apps, Flow, Apex, SOQL, Lightning Web Components (LWC), UI bundles, objects and fields, permission sets, and related areas.
+This repository provides Salesforce agent skills, plugins, and sample apps for building applications. It includes skills for Agentforce agents, Lightning apps, Flow, Apex, SOQL, Lightning Web Components (LWC), UI bundles, objects and fields, permission sets, and related areas. The [builder plugins](plugins/builder/) package skills with additional capabilities such as agents, hooks, commands, and Model Context Protocol (MCP) servers. Each client loads the components supported by that plugin's manifest.
 
-The skills are contributed by Salesforce and the broader community. It’s optimized for Agentforce Vibes and can be used with any AI tool that supports skills.
+The skills are contributed by Salesforce and the broader community. The skills library is optimized for Agentforce Vibes and can be used with any AI tool that supports skills. For guided Salesforce development, start with the [Salesforce Development plugin README](plugins/builder/salesforce-development/README.md) and the [Salesforce Development plugin documentation](https://developer.salesforce.com/docs/platform/salesforce-skills-plugins/guide/salesforce-development-overview.html).
 
-> ⚠️ **Expect frequent changes.** The Salesforce skills library is evolving rapidly as we refine patterns and incorporate feedback. Skills may be renamed, restructured, or removed between releases — they do not follow the same stability guarantees as GA platform APIs. If you’ve forked or synced the repository, be prepared for upstream changes that may conflict with local modifications. This repository is always the source of truth.
+## See the development workflow
+
+Go from a natural-language request to an app deployed in Salesforce. These short demos
+were recorded with Salesforce Development 2.5.0 and show a terminal workflow followed by the finished app.
+
+![Claude Code building and deploying a Salesforce Lightning app](docs/media/development-demos/claude-salesforce-development-2.5.0.gif)
+
+| Demo | Length | Watch or download |
+|---|---:|---|
+| Claude Code: Lightning app | 35 seconds | [GIF](docs/media/development-demos/claude-salesforce-development-2.5.0.gif) · [MP4](docs/media/development-demos/claude-salesforce-development-2.5.0.mp4) |
+| Codex CLI: Lightning app | 30 seconds | [GIF](docs/media/development-demos/codex-salesforce-development-2.5.0.gif) · [MP4](docs/media/development-demos/codex-salesforce-development-2.5.0.mp4) |
+| Claude Code: internal React app | 32 seconds | [GIF](docs/media/development-demos/claude-react-salesforce-development-2.5.0.gif) · [MP4](docs/media/development-demos/claude-react-salesforce-development-2.5.0.mp4) |
+
+The Codex Lightning demo also uses Experience LWC 0.3.0; the React demo uses
+Experience React 0.2.1. The clips combine real terminal excerpts with captured views
+of the deployed apps. Accelerated terminal sections are labeled. All Account data
+shown is fictional. See the [recording notes](docs/media/development-demos/README.md).
+
+<details>
+<summary>Preview Codex CLI and React</summary>
+
+### Codex CLI: Lightning app
+
+![Codex CLI building and deploying a Salesforce Lightning app](docs/media/development-demos/codex-salesforce-development-2.5.0.gif)
+
+### Claude Code: internal React app
+
+![Claude Code building and deploying an internal Salesforce React app](docs/media/development-demos/claude-react-salesforce-development-2.5.0.gif)
+
+</details>
+
+> ⚠️ **Expect frequent changes.** The Salesforce skills and plugins library is evolving rapidly as we refine patterns and incorporate feedback. Skills and plugins may be renamed, restructured, or removed between releases — they do not follow the same stability guarantees as GA platform APIs. If you’ve forked or synced the repository, be prepared for upstream changes that may conflict with local modifications. This repository is always the source of truth.
 
 ## 🗂️ Structure
 
 ```
 sf-skills/
+├── .claude-plugin/
+│   └── marketplace.json  # Salesforce plugin marketplace catalog
+├── plugins/
+│   └── builder/          # Plugins for Salesforce development
+│       ├── salesforce-development/
+│       └── ...
 ├── skills/               # Directory-based executable workflows
 │   ├── platform-apex-generate/
 │   ├── platform-custom-object-generate/
@@ -28,7 +65,20 @@ sf-skills/
 | **Tool** | **Usage** |
 |----------|-------------|
 | **Agentforce Vibes** | Skills are auto-installed and auto-updated |
-| **OpenCode, Claude Code, Codex, Cursor, [more](https://agentskills.io/)** | `npx skills add forcedotcom/sf-skills` |
+| **Claude Code — Salesforce Development plugin** | Follow the [Claude Code installation guide](https://developer.salesforce.com/docs/platform/salesforce-skills-plugins/guide/install-salesforce-development.html) for skills, guided workflows, and integrated tools |
+| **Codex — Salesforce Development plugin** | Follow [Codex setup and requirements](plugins/builder/salesforce-development/README.md#codex-lifecycle-support); use a release containing the Codex manifests and Codex CLI 0.145.0 or later |
+| **OpenCode, Claude Code, Codex, Cursor, [more](https://agentskills.io/) — standalone skills** | `npx skills add forcedotcom/sf-skills` |
+
+## 🔌 Plugins
+
+Plugins bundle skills with other components that the coding client installs and loads together. The Salesforce Development plugin provides shared workflows for building, testing, and deploying on the Salesforce Platform, with client-specific commands, hooks, and MCP configuration. Claude Code agents and hooks aren't automatically available in other clients. Specialized builder plugins add capabilities for areas such as Agentforce, code quality, LWC, and React development.
+
+- [Salesforce marketplace manifest](.claude-plugin/marketplace.json) — available plugins, descriptions, and source paths
+- [Salesforce Development plugin README](plugins/builder/salesforce-development/README.md) — prerequisites, quick start, and component inventory
+- [Salesforce Development plugin documentation](https://developer.salesforce.com/docs/platform/salesforce-skills-plugins/guide/salesforce-development-overview.html) — installation, project and org setup, workflows, and troubleshooting; use the plugin README for Codex-specific setup
+- [Claude Code plugin documentation](https://code.claude.com/docs/en/plugins) — plugin components and how plugins work
+
+Check the [plugin changelog](plugins/builder/salesforce-development/CHANGELOG.md) and installed version for release availability. Other clients, including Cursor, can use standalone skills where supported; full plugin support depends on the client and plugin manifest.
 
 ## 📦 Samples
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@salesforce/afv-skills",
-  "version": "1.66.0",
+  "version": "1.68.0",
   "description": "Salesforce's curated collection of agent skills for building applications. Optimized for Agentforce Vibes, compatible with all AI tools.",
   "license": "Apache-2.0",
   "files": [
@@ -12,9 +12,7 @@
   },
   "devDependencies": {
     "@salesforce/ui-bundle-template-app-react-sample-b2e": "^12.4.4",
-    "@salesforce/ui-bundle-template-app-react-sample-b2x": "^12.4.4",
-    "@salesforce/webapp-template-app-react-sample-b2e-experimental": "^1.117.1",
-    "@salesforce/webapp-template-app-react-sample-b2x-experimental": "^1.117.1"
+    "@salesforce/ui-bundle-template-app-react-sample-b2x": "^12.4.4"
   },
   "scripts": {
     "sync-react-b2e-sample": "node scripts/sync-react-b2e-sample.js",

package/skills/agentforce-offline-experimentation-run/SKILL.md

@@ -4,41 +4,39 @@ description: "Use this skill when users want to execute this workflow when a use
 metadata:
   relatedSkills:
   - dx-apexguru-scan
-  version: '1.0'
+  version: "1.0"
   cliTools:
   - tool:
-    - curl
-    semver: '>=7.0'
+    - "curl"
+    semver: ">=7.0"
   - tool:
-    - jq
-    semver: '>=1.6'
+    - "jq"
+    semver: ">=1.6"
   - tool:
-    - python3
-    semver: '>=3.8'
+    - "python3"
+    semver: ">=3.8"
   - tool:
-    - sf
-    semver: '>=2.0.0'
+    - "sf"
+    semver: ">=2.0.0"
 ---
 
 # Managing Agentforce Offline Experimentation
 
 ## When to Use This Skill
 
-Execute this workflow when a user wants to test how a different LLM model would handle existing Agentforce agent conversations. Fetches session traces from Data Cloud STDM, replays prompts via the SFAP AI Gateway REST API with structured tool calling, scores across four metrics (topic agreement, function name accuracy, function argument accuracy via LLM-as-judge, text response quality via LLM-as-judge), and produces CSV + HTML reports with SAFE/REVIEW/UNSAFE verdict badges. Requires Data Cloud with Agentforce Activity data streaming enabled, an authenticated org, and a target model ID (llmgateway__ prefix for SFAP replay). Estimated runtime: 10-30 minutes for 10 sessions.
+Execute this workflow when a user wants to test how a different LLM model would handle existing Agentforce agent conversations. Fetches session traces from Data Cloud STDM, replays prompts via the SFAP AI Gateway REST API with structured tool calling, scores across four metrics (topic agreement, function name accuracy, function argument accuracy via LLM-as-judge, text response quality via LLM-as-judge), and produces a run directory with units.jsonl, results.jsonl, summary.json, and an HTML report. Requires Data Cloud with Agentforce Activity data streaming enabled, an authenticated org, and a target model ID (llmgateway__ prefix for SFAP replay). Estimated runtime: 10-30 minutes for 10 sessions.
 
 ## Workflow
 
-discover-data-space → verify-stdm-availability → discover-agents → resolve-agent-label → list-available-models (if user needs help choosing) → find-sessions → get-conversation-details → get-llm-step-details → obtain-sfap-token → replay-turn-prompts → [score-exact-matches, judge-function-args, judge-text-responses] → generate-csv-report → generate-html-report.
+discover-data-space → verify-stdm-availability → discover-agents → list-available-models (if user needs help choosing) → find-sessions → get-conversation-details → get-llm-step-details → obtain-sfap-token → replay-turn-prompts (scoring happens inline) → generate-run-results → generate-html-report.
 WORKFLOW GATE: Do not proceed past Phase 0 until org alias (auto-discover from authenticated orgs), agent API name (auto-discover from STDM), and target model ID (ask user) are all confirmed.
 SFAP JWT expires after ~2 hours. For 20+ session replays, refresh the token between batches. Each SFAP replay call takes ~1-3 seconds; keep under 20 sessions for reasonable wall-clock time (~10-30 minutes).
-The scoring and judging steps (Phase 3) can run in parallel once replay completes. Exact-match scoring is instant; LLM judge calls add ~1-2 seconds per turn.
 
 ## Critical Constraints
 
 **Preconditions:**
 
 - Data Cloud is provisioned and enabled in the org (check: `DataCloudProvisioned`)
-- Agentforce Activity data streaming is active in Data Cloud (check: `AgentforceActivityStream`)
 - Einstein LLM / GenAI Gateway is configured in the org (check: `EinsteinLlmEnabled`)
 - User has an authenticated session to the target org (check: `OrgAuthenticated`)
 
@@ -53,7 +51,6 @@ Operations grouped by purpose. Use these as the building blocks for the workflow
 | `discover-data-space` | read | implemented | `getDataSpaces` | — |
 | `verify-stdm-availability` | validate | implemented | `queryCdpAnsiSqlV2` | `discover-data-space` |
 | `discover-agents` | read | implemented | `queryCdpAnsiSqlV2` | `verify-stdm-availability` |
-| `resolve-agent-label` | read | implemented | `queryGenAiPlannerDefinition` | `discover-agents` |
 | `list-available-models` | read | implemented | `queryAiModel` | — |
 | `find-sessions` | read | implemented | `queryCdpAnsiSqlV2` | `verify-stdm-availability`, `discover-agents` |
 | `get-conversation-details` | read | proposed | `queryCdpAnsiSqlV2` | `find-sessions` |
@@ -63,8 +60,8 @@ Operations grouped by purpose. Use these as the building blocks for the workflow
 | `score-exact-matches` | read | proposed | — | `replay-turn-prompts` |
 | `judge-function-args` | read | proposed | — | `score-exact-matches`, `obtain-sfap-token` |
 | `judge-text-responses` | read | proposed | — | `replay-turn-prompts`, `obtain-sfap-token` |
-| `generate-csv-report` | write | proposed | — | `score-exact-matches`, `judge-function-args`, `judge-text-responses` |
-| `generate-html-report` | write | proposed | — | `generate-csv-report` |
+| `generate-run-results` | write | proposed | — | `score-exact-matches`, `judge-function-args`, `judge-text-responses` |
+| `generate-html-report` | write | proposed | — | `generate-run-results` |
 
 ### Dependency graph
 
@@ -73,7 +70,6 @@ graph TD
   discover_data_space["discover-data-space (read)"]
   verify_stdm_availability["verify-stdm-availability (validate)"]
   discover_agents["discover-agents (read)"]
-  resolve_agent_label["resolve-agent-label (read)"]
   list_available_models["list-available-models (read)"]
   find_sessions["find-sessions (read)"]
   get_conversation_details["get-conversation-details (read)"]
@@ -83,11 +79,10 @@ graph TD
   score_exact_matches["score-exact-matches (read)"]
   judge_function_args["judge-function-args (read)"]
   judge_text_responses["judge-text-responses (read)"]
-  generate_csv_report["generate-csv-report (write)"]
+  generate_run_results["generate-run-results (write)"]
   generate_html_report["generate-html-report (write)"]
   discover_data_space --> verify_stdm_availability
   verify_stdm_availability --> discover_agents
-  discover_agents --> resolve_agent_label
   verify_stdm_availability --> find_sessions
   discover_agents --> find_sessions
   find_sessions --> get_conversation_details
@@ -99,10 +94,10 @@ graph TD
   obtain_sfap_token --> judge_function_args
   replay_turn_prompts --> judge_text_responses
   obtain_sfap_token --> judge_text_responses
-  score_exact_matches --> generate_csv_report
-  judge_function_args --> generate_csv_report
-  judge_text_responses --> generate_csv_report
-  generate_csv_report --> generate_html_report
+  score_exact_matches --> generate_run_results
+  judge_function_args --> generate_run_results
+  judge_text_responses --> generate_run_results
+  generate_run_results --> generate_html_report
 ```
 
 ### Read operations
@@ -115,24 +110,12 @@ Discover active Data Cloud data spaces in the org
 
 #### `discover-agents`
 
-Query STDM for distinct Agentforce agent API names that have session traces
+Query STDM for distinct agent + version combinations that have session traces
 
 **Call:** `queryCdpAnsiSqlV2`   — status: `implemented`
 
 **Depends on:** `verify-stdm-availability`
 
-#### `resolve-agent-label`
-
-Enrich the selected agent API name with a human-readable MasterLabel
-
-**Call:** `queryGenAiPlannerDefinition`   — status: `implemented`
-
-**Inputs:**
-
-- `agent_api_name` *(`String`)* — Agent API name selected in discover-agents
-
-**Depends on:** `discover-agents`
-
 #### `list-available-models`
 
 List Einstein LLM models configured in the org to help the user choose a target model
@@ -141,13 +124,14 @@ List Einstein LLM models configured in the org to help the user choose a target
 
 #### `find-sessions`
 
-Find recent Agentforce sessions in STDM filtered by agent API name and date range
+Find recent Agentforce sessions in STDM filtered by agent API name, version, and date range
 
 **Call:** `queryCdpAnsiSqlV2`   — status: `implemented`
 
 **Inputs:**
 
 - `agent_api_name` *(`String`)* — Agent API name selected in discover-agents
+- `agent_version_api_name` *(`String`)* — Agent version API name selected in discover-agents
 - `start_date` *(`String`)*
 - `end_date` *(`String`)*
 - `max_sessions` *(`Integer`)*
@@ -209,29 +193,22 @@ LLM-as-judge scoring of text responses for turns that end with text output
 
 ### Write operations
 
-#### `generate-csv-report`
+#### `generate-run-results`
 
-Generate CSV report with per-turn replay results and aggregate summary scores
+sfap_replay.py produces the run directory with units.jsonl (one line per replay unit with request and golden), results.jsonl (one line per scored result), and summary.json (run config, model, counts, aggregate scores). Scoring happens inline during replay — no separate scoring step needed.
 
-**Status:** `proposed`
-
-**Inputs:**
 
-- `output_csv` *(`String`)*
+**Status:** `proposed`
 
 **Depends on:** `score-exact-matches`, `judge-function-args`, `judge-text-responses`
 
 #### `generate-html-report`
 
-Build a self-contained HTML analysis report with verdict badges, scorecards, and failure breakdown
+Build a self-contained HTML analysis report with scorecards and failure breakdown
 
 **Status:** `proposed`
 
-**Inputs:**
-
-- `html_output` *(`String`)*
-
-**Depends on:** `generate-csv-report`
+**Depends on:** `generate-run-results`
 
 ### Other operations
 

package/skills/agentforce-offline-experimentation-run/scripts/bt_compare.py

@@ -0,0 +1,81 @@
+#!/usr/bin/env python3
+"""Compare two replay runs of the same units, unit by unit, against the noise between repeats.
+
+    python3 scripts/bt_compare.py runs/base runs/cand [--noise-from runs/v0] [--out FILE]
+
+Each unit x repeat scored in both runs is a pair. Per group (overall, step kind, node, golden tool)
+the delta is in units: the sum of each unit's cand - base score, averaged over its repeats. Per
+failure bucket, the net is the pairs that scored lower and ended in it, minus those that scored
+higher and left it. A metric's tolerance is the largest difference on it between two repeats of
+the noise run (default: BASE), at least 1 unit; a delta beyond it is a regression or a gain. A
+bootstrap over sessions gives a 95% interval of the mean unit delta, for reading only. A failure
+drops the pair from both runs, except a no answer: a timeout where the other run answered another
+request or model scores 0, in the bucket no_answer.
+
+Exit 0: no metric got worse beyond its tolerance. Exit 1: some did. Exit 2: the runs are not
+comparable (other units, judge, k, seed, steps, router nodes, equivalent tools, timeout or btlib
+code; or more than 10% infra errors, or more than 10% no answers, in a run). --force compares them
+anyway, on the units both share. Exit 2 also when a dir is not a finished run, or bt_compare
+fails: never a verdict.
+
+Writes the comparison to --out (default CAND/compare.json, bt.compare/1). It lists the units that
+moved by unit id, so keep it with the run dirs, out of git.
+"""
+
+import argparse
+import os
+import sys
+import traceback
+
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+
+from btlib import compare as comparelib  # noqa: E402
+from btlib import results  # noqa: E402
+from btlib.redact import redact  # noqa: E402
+
+
+def parse_args(argv=None):
+    p = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    p.add_argument("base", help="run dir of the baseline (sfap_replay.py --out-dir)")
+    p.add_argument("cand", help="run dir of the candidate, on the same units")
+    p.add_argument("--noise-from", metavar="DIR",
+                   help="run dir whose repeats set the tolerances (default: BASE); same units, judge, steps, "
+                        "router nodes, equivalent tools and btlib code as BASE")
+    p.add_argument("--out", metavar="FILE", help="where to write the comparison (default CAND/compare.json)")
+    p.add_argument("--force", action="store_true",
+                   help="compare runs that are not comparable, on the units both share (recorded in the output)")
+    p.add_argument("--bootstrap-samples", type=int, default=comparelib.BOOTSTRAP_SAMPLES, metavar="N",
+                   help="session resamples for the 95%% intervals (default %(default)s; 0: none)")
+    args = p.parse_args(argv)
+    if args.bootstrap_samples < 0:
+        p.error("--bootstrap-samples must be >= 0")
+    return args
+
+
+def main(argv=None):
+    args = parse_args(argv)
+    out = args.out or os.path.join(args.cand, "compare.json")
+    try:
+        res = comparelib.compare(args.base, args.cand, args.noise_from, force=args.force,
+                                 samples=args.bootstrap_samples)
+        results.write_json(out, res)  # 0600: unit ids name sessions
+        lines = comparelib.report_lines(res)
+    except comparelib.CompareError as e:
+        print(f"error: {redact(str(e))}", file=sys.stderr)
+        return 2
+    except OSError as e:
+        print(f"error: cannot write {out}: {e.strerror or e}", file=sys.stderr)
+        return 2
+    except Exception as e:  # a bug, or a run dir it did not foresee: exit 1 would read as a regression
+        frame = traceback.extract_tb(e.__traceback__)[-1]
+        print(f"error: bt_compare failed ({type(e).__name__} at {os.path.basename(frame.filename)}:{frame.lineno}); "
+              "no verdict (exit 2)", file=sys.stderr)
+        return 2
+    for line in lines:
+        print(redact(line))
+    print(f"comparison: {out}")
+    return comparelib.exit_code(res)
+
+
+if __name__ == "__main__":
+    sys.exit(main())