@salesforce/afv-skills 1.34.0 → 1.36.0

package/skills/automation-sandbox-post-copy-config-generate/references/sop_parsing_patterns.md

@@ -0,0 +1,157 @@
+# SOP Parsing Patterns
+
+Heuristics for turning SOP prose into post-copy action records.
+
+## Identify the Post-Copy / Post-Refresh section
+
+SOPs are typically structured as:
+
+1. Pre-Refresh Steps (or Pre-Copy)
+2. Refresh / Copy Steps (the act of cloning)
+3. **Post-Refresh Steps** (or Post-Copy) — this is the section that
+   produces actions. Everything else is filtered out.
+
+Section markers to look for (case-insensitive):
+
+- `Post Refresh Steps` / `Post-Refresh Steps`
+- `Post Copy Steps` / `Post-Copy Steps`
+- `After Refresh`
+- `After Copy`
+
+If the SOP has no such section, ask the user which sections are post-copy.
+
+## Action signal patterns
+
+Each post-copy action in the JSON output corresponds to one SOP instruction
+that **also includes a concrete value** (URL, etc.). If the value is
+missing, skip the action.
+
+### Pattern 1 — "Update X to Y" (Outbound Messages)
+
+> "Update the OBM Messages endpoint for `IR_Account_OBM_PROD` to
+> `https://uat.example.com/account`."
+
+- One entry per outbound message named in the SOP.
+- `ConfigurationName: "OutboundMessages"`,
+  `Fields: { "EndpointUrl": "<literal URL from SOP>", "Object": "<entity>" }`.
+- `Label` = the outbound message Name as it appears in the SOP.
+- `Object` is **mandatory** — infer from the OBM name
+  (`IR_Account_OBM_PROD` → `Account`, `IR_Contact_OBM_PROD` → `Contact`,
+  `IR_Asset_OBM_PROD` → `Asset`) or from explicit text in the SOP.
+  The same Label may apply to two OBMs that differ only by entity, so
+  the post-copy tool relies on `Object` to disambiguate.
+
+If the SOP names the outbound message but does not state the new
+endpoint URL ("Update the OBM endpoints" with no table or list of
+URLs), **skip** and list the omission in the response. If the target
+Object cannot be inferred from the OBM name or surrounding text,
+also skip and surface it.
+
+### Pattern 2 — Tabular updates with explicit URL columns
+
+> A table with columns: `Remote Site Name`, `Prod URL`, `UAT URL`,
+> `Difference found`.
+
+- One entry per row whose `Difference found` (or equivalent) flag
+  indicates the URL changed.
+- `ConfigurationName: "RemoteSiteSettings"`,
+  `Fields: { "RemoteSiteUrl": "<UAT URL from row>" }`.
+- `Label` = the Remote Site Name from the row.
+- Skip rows where Prod and UAT URLs are identical (no update needed).
+- Do **not** add `RemoteSiteName` to `Fields` (lives in `Label`).
+- Do **not** add `IsActive` to `Fields` (lives at the top level).
+
+### Pattern 3 — "Delete all X"
+
+> "Delete all integration endpoints in the OBM Messages setup."
+
+- A single entry with `Fields: { "Action": "Delete" }` and a Label
+  describing the deletion target ("All OBM Endpoints" or similar).
+  `"Delete"` here is a literal instruction the tool understands.
+
+### Pattern 4 — Sequential / phased instructions
+
+> "First, update the OBM endpoints. Then, update the Remote Site Settings."
+
+- Use the order in `ExecutionOrder`. See "Ordering heuristic" below.
+
+### Pattern 5 — Conditional / environment-specific
+
+> "If you are refreshing fcUAT, do X. For all other sandboxes, do Y."
+
+- Emit both entries. Mark the conditional one with `IsActive: false` so
+  the customer can toggle it per-environment, and surface the
+  conditionality in your response message.
+
+## Ordering heuristic — `ExecutionOrder` is a phase number
+
+Treat `ExecutionOrder` like a phase / wave number, not a per-row index.
+**Entries that have no dependency on each other share the same number.**
+The post-copy tool runs all entries with the same number in parallel.
+
+Default phasing (use unless the SOP explicitly says otherwise):
+
+| Phase | Type | Why |
+|-------|------|-----|
+| 1 | OutboundMessages | Endpoint URLs the rest of the system depends on; no inter-OBM dependency |
+| 2 | RemoteSiteSettings | Trusted URLs subsequent integrations need; no inter-site dependency |
+
+Within a single `ConfigurationName`, every entry shares the same phase
+number — there is no inter-row dependency among OBMs or among remote
+sites. Across types, run OBMs before remote sites unless the SOP says
+otherwise.
+
+If the SOP states explicit dependencies ("Step 5 must finish before
+Step 6"), break the dependent entry into a later phase.
+
+## Locating values across multi-sheet sources
+
+Customer SOPs often split the **action list** from the **endpoint
+table**:
+
+- The "Post Refresh / Post Copy" section names which OBMs / Remote
+  Sites need updating, but only by name.
+- A separate sheet (or appendix) holds a Prod URL → UAT URL table.
+
+Before skipping an entry for "missing value", search **every tab /
+sheet of every file the user supplied** for a row keyed by the same
+OBM / RemoteSite / Configuration name. Look for column headers like
+`Prod Endpoint URL`, `UAT Endpoint URL`, `Post-refresh URL`, or
+`Endpoint`. The post-refresh value is the one to emit.
+
+If after searching every supplied source the value is still not
+present, **skip the entry** and list it in the response. Never emit
+empty strings, `null`, or placeholder markers (`<from-backup>`, `TBD`,
+`TODO`) — customers consume the JSON directly and unpopulated fields
+break the tool.
+
+## Filter rules
+
+Skip these — they are not post-copy automation actions:
+
+- Manual ServiceNow ticket creation (process / ops, not metadata).
+- "Email the SSO team" / human notifications.
+- "Schedule a meeting" / calendar actions.
+- "Click Save" / UI navigation breadcrumbs.
+- Pre-refresh data collection ("copy the endpoints to a spreadsheet").
+- Verification / test steps with no metadata write.
+- Any action that targets a configuration type **not** in
+  `configuration_catalog.md` (Custom Labels, Connected Apps, Named
+  Credentials, SSO, etc.) — list these in your response message so the
+  user knows what was filtered and can prioritize extending the catalog.
+- Any action where the SOP names the metadata but does not give a
+  concrete value (URL, etc.). List these too in the response.
+
+If a section is mostly process and not configuration, note in your
+response message that the section was skipped and why.
+
+## Disambiguating Label vs. DeveloperName
+
+When the SOP shows an Outbound Message or Remote Site name like
+`R12_Remote_Site`, use that as the `Label` field — this is the
+DeveloperName, which is what the post-copy tool resolves at runtime.
+
+When the SOP shows a UI Label like "TK Marine Parts Store Online", that's
+ambiguous. Ask the user or pick the most likely API name based on the
+surrounding context (the SOP usually shows the API name in setup
+breadcrumbs).

package/skills/automation-sandbox-post-copy-config-generate/references/source_format_handling.md

@@ -0,0 +1,230 @@
+# Source Format Handling
+
+The customer SOP can arrive in any of the formats below — sometimes
+several at once. This file gives the exact extraction recipe per
+format so action records and endpoint values can be located reliably,
+and explains how to decide whether an embedded image carries data or
+is illustrative.
+
+> **Rule of thumb**: the action list (which OBM / Remote Site to
+> update) and the endpoint table (the actual URL values) are
+> frequently in **different files or different tabs / pages**. Always
+> read every source the user supplies before deciding a value is
+> missing. If a value still cannot be located after exhausting every
+> source, skip the entry — never emit empty / placeholder fields.
+
+---
+
+## PDF (`.pdf`)
+
+```python
+import pypdf
+reader = pypdf.PdfReader(path)
+for i, page in enumerate(reader.pages):
+    text = page.extract_text()
+```
+
+- Many enterprise SOPs are slide decks exported to PDF — the text
+  layer is usually present, but tables may flatten oddly. Inspect the
+  raw extracted text rather than trusting visual alignment.
+- For section navigation, search case-insensitively for: "Post
+  Refresh", "Post-Refresh", "Post Copy", "Post-Copy", "After
+  Refresh", "Update <X>".
+- If `extract_text()` returns empty or near-empty for a page, the
+  page is likely an embedded image. OCR it:
+
+```python
+import pdfplumber, pytesseract
+with pdfplumber.open(path) as pdf:
+    for page in pdf.pages:
+        if not (page.extract_text() or "").strip():
+            img = page.to_image(resolution=200).original
+            text = pytesseract.image_to_string(img)
+```
+
+- If `pytesseract` is unavailable, fall back to `pdftotext -layout`
+  (Poppler) and `tesseract` CLI.
+
+## xlsx (`.xlsx`)
+
+```python
+import openpyxl
+wb = openpyxl.load_workbook(path, data_only=True)
+for sname in wb.sheetnames:
+    ws = wb[sname]
+    for row in ws.iter_rows(values_only=False):
+        for cell in row:
+            ...
+```
+
+- **Always read every sheet.** Customer planners routinely split the
+  task list and the URL table across tabs (real example: Michelin UAT
+  Refresh Planner — task list in `Integration (Indus)`, URL table in
+  `Evolution SFA` columns K/L/M).
+- Iterate **every column** up to `ws.max_column`. Some sheets have
+  data far past column M (e.g., column index 10000+) that the visible
+  layout hides.
+- Read with `data_only=True` so cached formula values are returned.
+- Check `cell.comment` for free-text annotations that often contain
+  the post-refresh value.
+- Look for canonical column headers: `Outbound Message`, `Remote
+  Site`, `Object`, `Endpoint`, `Endpoint URL`, `Prod Endpoint URL`,
+  `UAT Endpoint URL`, `Post-refresh URL`. The post-refresh value is
+  the one to emit.
+- For embedded images on a sheet, use `ws._images` (or unzip the
+  xlsx and inspect `xl/media/`) — see the image rules below.
+
+## csv (`.csv`)
+
+```python
+import csv
+with open(path, newline="") as f:
+    reader = csv.DictReader(f)
+    for row in reader:
+        ...
+```
+
+- Treat the same way as a single xlsx sheet — look for the canonical
+  column headers above.
+
+## JSON (`.json`)
+
+- If the customer already supplies a JSON document, scan every key
+  for the canonical names (`outboundMessage`, `endpointUrl`, `object`,
+  `remoteSiteUrl`, `label`, `name`). Be tolerant of casing
+  (`endpointURL`, `EndpointUrl`, `endpoint_url`).
+- Customer JSON might already be partially in the target shape; do
+  not blindly copy it through — re-validate every entry against
+  `assets/json_schema.json` and the catalog.
+
+## docx (`.docx`)
+
+```python
+from docx import Document
+doc = Document(path)
+for p in doc.paragraphs:
+    ...
+for tbl in doc.tables:
+    for row in tbl.rows:
+        for cell in row.cells:
+            ...
+```
+
+- Extract both paragraph text and table cells. Word tables
+  frequently hold the endpoint table.
+- Inline images (`doc.inline_shapes`) follow the image rules below.
+
+## Markdown / plain text (`.md`, `.txt`)
+
+- Read directly with the `Read` tool. Tables in pipe-delimited
+  Markdown are easy to parse — split on `|`.
+
+## Pasted excerpts in the conversation
+
+- The user may paste content directly. Treat it as plain text.
+- If they paste an image (drag-drop), use the `Read` tool on the
+  image path and follow the image rules below.
+
+---
+
+## Images (`.png`, `.jpg`, `.jpeg`, `.tiff`, `.bmp`)
+
+Two kinds of image arrive in customer SOPs:
+
+| Kind | Examples | What to do |
+|------|----------|------------|
+| **Data-bearing** | screenshot of the Outbound Messages list page showing real Names + Endpoint URLs; captured Remote Site Settings table with real URLs; spreadsheet snippet pasted as a screenshot; configuration page where the endpoint values are visible | OCR and treat the values as authoritative for the entries they cover |
+| **Illustrative** | "this is what the setup screen looks like" with placeholder values; architecture diagram showing system topology; UI flow chart; redacted/blurred values; screenshot whose surrounding text says "for example" / "as shown below your values will look like this" | Do NOT extract values — the image is documentation, not data |
+
+### Heuristic — is this image data-bearing?
+
+Yes (use the values), if any of:
+
+- The surrounding paragraph references it as the source of truth:
+  "the values listed below", "as captured in the spreadsheet
+  above", "copy these into the post-refresh setup", "the URLs you
+  copied before the refresh".
+- The image shows a Salesforce setup table (Outbound Messages list,
+  Remote Site Settings list, Custom Metadata Type list) with real
+  values for known catalog types.
+- Real endpoint URLs are visible (full domain, not redacted, not
+  obviously a placeholder like `https://example.com/...`).
+- The image directly precedes or follows a "Post Refresh Steps"
+  heading and lists the entities named in that step.
+
+No (skip the image), if any of:
+
+- Surrounding text says "for example", "as a sample", "this is what
+  it will look like", "your real values will differ".
+- The values shown are placeholders (`PROD_URL`, `<your domain>`,
+  `username/password`, all-`x` or all-`*` redaction).
+- The image is a flowchart / arch diagram / sequence diagram.
+- The image is a UI walkthrough screenshot whose purpose is to show
+  *where* to click, not *what* to enter.
+- The image is a stock screenshot of the Salesforce Help / Setup
+  interface unrelated to the customer's actual values.
+
+### OCR recipe
+
+```python
+from PIL import Image
+import pytesseract
+img = Image.open(image_path)
+text = pytesseract.image_to_string(img)
+```
+
+For higher accuracy on tables:
+
+```python
+text = pytesseract.image_to_string(img, config="--psm 6")
+```
+
+`--psm 6` treats the image as a single uniform block of text (good
+for tables). Try `--psm 4` for column-of-text layouts.
+
+If `pytesseract` is unavailable, **read the image with the `Read`
+tool and transcribe the visible values into a temporary text
+buffer** — the multimodal `Read` reliably picks up table contents
+in screenshots.
+
+### After OCR
+
+- Normalise whitespace, strip trailing zero-width chars.
+- Fix common OCR confusions in URLs: `0`/`O`, `1`/`l`/`I`, `:`/`;`,
+  `/`/`\`, `_`/`—`. Cross-check with surrounding text where possible.
+- If two URL columns are present (Prod / UAT), use the
+  **post-refresh** column.
+- Treat OCR'd values like any other extracted value — if the text
+  is unreadable or ambiguous, skip the entry and surface it. Do
+  NOT guess to fill a field.
+
+---
+
+## Multi-source merge
+
+When the user supplies multiple files:
+
+1. Read every file end-to-end first; build an internal map of
+   "named entity → values found".
+2. Identify the action list source (which OBMs / Remote Sites need
+   updating) — usually the file with "Post Refresh Steps".
+3. For each named action, look up the value in the map. If found in
+   any source, use it. If not found anywhere, skip and surface.
+4. Emit one entry per (configuration, label) pair — do not duplicate
+   when the same OBM appears in multiple sources.
+
+---
+
+## Failure modes to surface in the response
+
+When you skip an entry, list it explicitly so the customer can
+correct the source. Use one of these labels in the response:
+
+- **Missing value** — the entity is named but no concrete value was
+  found in any supplied source.
+- **Out of catalog** — the action targets a configuration type not
+  yet supported (Custom Labels, Connected Apps, etc.).
+- **Unreadable image** — an image was data-bearing but OCR returned
+  unintelligible output.
+- **Ambiguous Object (OutboundMessages only)** — the OBM name does
+  not reveal the target SObject and no supplemental table provides it.