toga-ai 1.0.0

package/knowledge/2.0/apps/_underscore/features/recursive-item-fulfillments.md

@@ -0,0 +1,111 @@
+---
+title: Recursive Item Fulfillments (upstream mirroring)
+framework: "2.0"
+repo: _underscore
+project: _Underscore
+client: shared
+type: feature
+status: active
+updated: 2026-06-08
+owners: [jcardinal]
+files:
+  - _underscore/Model/Client/ItemFulfillment.php
+  - _underscore/Model/Client/ItemFulfillmentItem.php
+  - _underscore/Model/Client/ItemFulfillmentItemUnit.php
+  - _underscore/Model/Client/ItemFulfillmentPackage.php
+  - _underscore/Model/Compass/AdvanceShippingNotice.php
+  - dbchanges2/Core/2026-02-13 - 75601 - RecursiveItemFulfillmentCreation.sql
+  - dbchanges2/Core/2026-06-04 - RecursiveItemFulfillmentPut.sql
+related:
+  - ../architecture.md
+  - ../../api2/architecture.md
+---
+
+## Summary
+In a multi-tier supply chain a sales order (SO) spawns a purchase order (PO) that becomes
+another SO downstream, and so on. When goods ship, an ItemFulfillment (IF) is created against
+the downstream SO. This feature **mirrors that fulfillment up the chain onto every upstream
+(customer-facing) SO, recursively, to the top** — so the customer-facing order reflects
+shipments that physically happened further down the chain. Triggered whenever any IF,
+IF-item, IF-item-unit, or IF-package is **POSTed or PUT**.
+
+## Key files / entry points
+The whole engine is in `_Model_Client_ItemFulfillment`; the three child models are thin
+triggers. Each registers `postPost`/`postPut` interceptors that funnel into one driver:
+
+| File | Hooks | Role |
+|---|---|---|
+| `Model/Client/ItemFulfillment.php` | `postPost`, `postPut` | engine: `ensureUpstreamItemFulfillment`, `reconcileUpstreamLevel`, `reconcileUpstreamUnits`, `reconcileUpstreamPackages`, `buildUpstreamHeaderPayload`, `buildUpstreamHeaderUpdatePayload` |
+| `Model/Client/ItemFulfillmentItem.php` | `postPost`, `postPut` | resolve owning IF uuid → driver |
+| `Model/Client/ItemFulfillmentItemUnit.php` | `postPost`, `postPut` | resolve owning IF uuid → driver |
+| `Model/Client/ItemFulfillmentPackage.php` | `postPost`, `postPut` | resolve owning IF uuid → driver |
+
+Triggers live on every child (not just the IF) so the sync is robust to all paths: the ASN
+flow (POSTs the IF then each child individually), nested IF POSTs, and direct child edits.
+
+## How it works
+**Driver — `ensureUpstreamItemFulfillment($api, $ifUuid)`:** a static `$isReconciling`
+re-entrancy guard makes nested re-entries (from our own `internalApiRequest` writes, which
+fire those records' interceptors) no-op. A `while` loop walks UP one level at a time via
+`reconcileUpstreamLevel`, with a `$visited` set to prevent cycles.
+
+**One level — `reconcileUpstreamLevel($api, $downIfUuid): ?string`:**
+1. Load downstream IF; no `salesOrderId` (transfer order) → return null (out of scope).
+2. Resolve upstream SO via the **header walk** (bridge tables); none → null (top of chain).
+3. Build desired upstream items: map each downstream IFI's SOI up via the **item walk**,
+   GROUP BY upstream SOI, sum fulfilled qty, then **scale into upstream order units** for
+   bundle decomposition (see Gotchas).
+4. Resolve-or-create the upstream IF (**eager** — header-only POST still builds the chain):
+   load existing via `upstreamItemFulfillmentId` and PUT changed header fields, or POST a
+   new IF with mirrored header + upstream SO then PUT the downstream IF's link.
+5. Reconcile upstream IFIs keyed by `salesOrderItemId`: update qty, create missing, link
+   `upstreamItemFulfillmentItem`, delete stale (units first, then item).
+6. Reconcile units per IFI matched on `unitId`; reconcile packages on the IF matched on
+   `trackingNumberId`: add missing, update changed tracking, delete leftovers.
+7. Return upstream IF uuid → loop continues upward.
+
+All internal calls pass `['depth' => -1]` and null-check responses; hard failures throw,
+rolling back the whole outer transaction (consistency over silent desync).
+
+## Data model
+Bridge tables (client DB, in the blank-client template → all clients):
+`PurchaseOrders_SalesOrders`, `SalesOrders_PurchaseOrders`,
+`PurchaseOrderItems_SalesOrderItems`, `SalesOrderItems_PurchaseOrderItems`.
+
+Upstream links added this feature (`RecordFields`, `isIdentifier=1`):
+`ItemFulfillments.upstreamItemFulfillmentId`, `ItemFulfillmentItems.upstreamItemFulfillmentItemId`.
+Units match on shared `unitId`; packages match on shared `trackingNumberId` (no upstream-id
+columns — the same `Unit` / `TrackingNumber` record is shared up the chain).
+
+Mirrored header fields (only when populated, never cleared upstream): scalars
+`dateItemFulfillment`, `dateDeliveryEta`, `dtSubmitted`; FK references `itemFulfillmentStage`,
+`shipToAddress`, `location`, `fulfilledByUser`. `number` is left unset → auto `F#####`.
+
+## Client variations
+Engine is shared `_Model_Client_*` logic. Compass uses **empty pass-through subclasses**
+(`_Model_Compass_ItemFulfillment`, `…ItemFulfillmentItemUnit`) so base logic applies via
+inheritance. Verified against prod `Client_Compass` chains (≥3 levels deep).
+
+## Gotchas / known issues
+- **Interceptors are DB-driven.** `postPost`/`postPut` PHP does nothing without matching
+  `ApiPayloadInterceptors` rows (recordId 28/29/30/41). **Deploying to any env requires
+  running BOTH Core SQL files** (`…Creation.sql` for POST/POST + RecordFields, and
+  `…Put.sql` for POST/PUT) against that env's Core DB — the PHP won't fire on PUT otherwise.
+- **`depth: -1` always** on orchestration calls. `depth => 0` means *unlimited* FK
+  traversal → OOM. Never use 0.
+- **Bundle scaling:** `upstreamQty = totalDownstreamFulfilled × upstreamOrderedQty /
+  totalDownstreamOrderedQty`, **capped at the raw sum** so the factor can only reduce
+  (roll-up), never inflate. A qty-1 bundle of 6 fully-shipped components → upstream qty 1.
+- **DELETE propagation is NOT implemented.** The engine sets `$outData = null` on DELETE so
+  post-delete hooks never fire; a standalone downstream DELETE doesn't immediately remove its
+  upstream mirror. A later reconciling PUT cleans up stale records via set comparison. Full
+  delete handling would need a `preDelete` mechanism + `PRE/DELETE` interceptor rows.
+- **Out of scope:** NetSuite sync of upstream IFs; transfer-order fulfillments
+  (`transferOrderId`) are skipped.
+- **Performance:** every child write re-runs a full upstream walk (read-heavy, but each
+  upstream record is written at most once — idempotent). Fine for normal fulfillment sizes.
+
+## Related docs
+- `_underscore` architecture (interceptors, `internalApiRequest`, `_Model` layer).
+- `api2` architecture (V2 metadata engine that fires these interceptors; the
+  commit-logs / rollback-data-on-failure transaction invariant the throws rely on).

package/knowledge/2.0/apps/api2/INDEX.md

@@ -0,0 +1,5 @@
+# api2 (API) — 2.0 knowledge
+
+| Doc | Summary | Files |
+|-----|---------|-------|
+| [API (api2 / TOGa API v2) Architecture](architecture.md) | `api2` is the backend powering the public **TOGa 2.0 API**. | api2/Controller/Index.php, api2/Component/Api/V2/V2.php, api2/Component/Api/Cxml/Cxml.php, api2/Component/Api/V2/Response/Response.php, api2/Config/ |

package/knowledge/2.0/apps/api2/architecture.md

@@ -0,0 +1,162 @@
+---
+title: API (api2 / TOGa API v2) Architecture
+framework: "2.0"
+repo: api2
+project: API
+client: shared
+type: architecture
+status: active
+updated: 2026-06-08
+owners: [jcardinal]
+files:
+  - api2/Controller/Index.php
+  - api2/Component/Api/V2/V2.php
+  - api2/Component/Api/Cxml/Cxml.php
+  - api2/Component/Api/V2/Response/Response.php
+  - api2/Config/
+related: []
+---
+
+## Summary
+
+`api2` is the backend powering the public **TOGa 2.0 API**. It serves two protocols from
+one codebase, chosen by hostname in `Controller/Index.php`:
+
+1. **RESTful JSON API** (`/v2/...`) — the primary, fully metadata- and ACL-driven CRUD API
+   third parties integrate with. Engine: `Component/Api/V2/V2.php`.
+2. **cXML gateway** — accepts supplier cXML (PunchOut, OrderRequest, ShipNotice) and
+   translates them into internal JSON API calls. Engine: `Component/Api/Cxml/Cxml.php`.
+
+Runtime: PHP 8.2+, AWS Elastic Beanstalk (Apache/httpd → PHP-FPM). The `_underscore`
+framework is **pulled at deploy, not vendored**. Composer deps: `sentry/sentry`,
+`aws/aws-sdk-php`, `robrichards/xmlseclibs`, `phpmailer/phpmailer`.
+
+## Dependencies
+
+- **`_underscore` (framework core)** — this API is essentially a metadata-driven layer over
+  `_Model`. Read its architecture first.
+- **`apiproxy`** (reverse proxy, not yet captured) — all external traffic hits the proxy
+  first; it forwards here and retries across regions on 5xx so integrators never see a
+  regional failure. _When apiproxy knowledge is captured, add it to this repo's `dependsOn`._
+
+## Request lifecycle
+
+```
+third party → apiproxy (region failover) → EB ALB → Apache → index.php
+  → _underscore boot → _Controller_Index::api()
+      ├─ host api/api1-3/api-writer/apiproxy*/api-production* → JSON → _Component_Api_V2::execute()
+      └─ host cxml/cxml1-3/cxml-writer                        → cXML → _Component_Api_Cxml::execute()
+```
+
+`.htaccess` rewrites all to `index.php` (just `require '_underscore.php'`).
+`_underscore::DEFAULT_CONTROLLER_METHOD = ['_Controller_Index','api']`, so every request
+lands in `api()`.
+
+## Front controller — `Controller/Index.php`
+
+`api()` is the protocol router and the **transaction/logging boundary**:
+- **CORS:** permissive `Access-Control-Allow-*`; returns 200 for `OPTIONS` preflight.
+- **`/health`:** 200 + empty object (EB/LB probe — keep it working).
+- **Sentry:** initialized per request; EC2 instance metadata attached.
+- **Core DB:** registers `Core` (`DB_CORE`), preferring a region-local read host
+  (`CORE_READHOST`). **Core Logs DB** (`DatabaseHost` id 11) registered region-aware as `DB_LOGS`.
+- **Host dispatch:** lowercase `HTTP_HOST`, strip trailing domain, switch on the remainder.
+- **Transaction/logging invariant (JSON path):** wraps the call in transactions; on
+  **success** commit logs then data; on **failure** commit logs but **roll back data**; on
+  uncaught `Throwable` report to Sentry, write a log row/CloudWatch line, commit logs, roll
+  back data. **Preserve this invariant.**
+
+## JSON engine — `Component/Api/V2/V2.php`
+
+One ~2,000-line `execute()` then `processRoutePairs()`:
+
+1. **Parse/normalize** — reads "now" from Core (`SELECT NOW()`) for token expiry; parses
+   method/headers/route (strips `/v2`)/body; builds Core-metadata lookups (`Parameters`,
+   `Records`, `InherentRecordChildren`, `RecordFields`).
+2. **Options** (`fields`, `where`, `join`/`ojoin`, `sort`, `group`, `distinct`, `page`,
+   `recordsPerPage` default 25 / max 10000, `depth` default 3, `calcDepth` default 1) —
+   passed as query params or one base64(JSON) `_` blob.
+3. **Authentication** — `transactionId` is required and **globally unique** (checked vs
+   Core + client logs; reuse → `EV-5`). Auth routes under `/v2/auth/*`: `api`, `refresh`,
+   `oauth`, `oauth/refresh`, `login` (email/pw; AD via LDAP bind for AD-configured clients),
+   `public`, `delegator`/`encrypted`. JWTs HS256, `sub` access/refresh (access 3600s,
+   refresh 30d); signing secret **rotated** with current+previous accepted across the
+   boundary. The JWT `id` claim carries the full identity used for ACL.
+4. **Response envelope** — `{transactionId, timestamp, authority, audience, isSuccess,
+   status, error, messages[], meta{}, data{}}`; `isSuccess` = 2xx status.
+5. **Transaction logging** — every request logged (to client/core Logs DB, or as a JSONL
+   line shipped by CloudWatch when `[api] log_filepath` is set).
+
+## CRUD engine — `processRoutePairs()`
+
+**Metadata-driven** — routes/models/fields/permissions come from Core/Client DB tables, not
+hardcoded controllers. The route after `/v2` is chunked into `[routeName, uuid?]` pairs and
+processed recursively (nested resources). Metadata tables (Core): `Records` (route ↔ model,
+`aclDatabase`, `childPolicy`), `RecordFields`, `InherentRecordChildren`,
+`AclRecordPermissions` (per-role CRUD), `AclFieldPermissions` (per-role field read/write).
+
+Per route pair: resolve Record → **authorize record-level** (`EZ-1` if no grant) → **client
+model override** (`_Model_Client_X` → `_Model_<Slug>_X` when present) → **pre interceptors**
+→ **authorize field-level** → field analysis (columns, FK children, SQL fields, aggregates,
+inherent children) → **execute** the `_Model` op → **serialize** via `getFullModelData()`
+(nested, bounded by `depth`) → **post interceptors**. Scripted APIs and
+`internalApiRequest()` (in-process calls reusing auth/ACL) are supported. Message codes:
+`EN-*` auth, `EZ-*` authorization, `EV-*` validation, `EO-*` operation, `W*`/`D*`.
+
+## cXML gateway — `Component/Api/Cxml/Cxml.php`
+
+Translation layer for EDI/punchout suppliers. `execute()`: normalize XML (DOMDocument,
+strip DOCTYPE, disable external entities) → authenticate via `Sender > Credential` against
+Core `ClientApiIdentities` → dispatch: `OrderRequest` builds a Sales Order and POSTs to
+`/sales-orders`; `ShipNoticeRequest` → `/advance-shipping-notices`; `PunchOutSetupRequest`
+creates a Vision (1.0) quote, Store (1.0) cart, or TOGa Commerce session. **cXML rides on
+top of the V2 JSON engine** (it authenticates and POSTs via `_ApiRequest`), inheriting ACL,
+interceptors, and logging. PunchOut paths write directly to legacy **Vision/Store** (1.0)
+DBs (`DB_VISION_1` / `DB_STORE_1`).
+
+## Multi-database architecture
+
+Multi-tenant; each client has isolated DBs resolved at request time per environment/region.
+Aliases: `DB_CORE`, `DB_CLIENT`, `DB_CLIENT_LOGS`, `DB_CLIENT_ARCHIVE`, `DB_LOGS`, `DB_TEAM`,
+`DB_STORE_1`/`DB_VISION_1` (legacy). `_Database::registerClientDatabases(clientId,
+environment)` joins `Clients`/`Databases`/`DatabaseHosts`/`Environments`, preferring the
+instance's own region; reads→readers, writes→writers, per-connection transactions.
+
+## Deployment (EB)
+
+Same shape as other 2.0 tiers. `.ebextensions/php_include_underscore.config` adds
+`_underscore` to `include_path`; `.platform/hooks/prebuild/git.sh` clones
+`TOGATechnology/_underscore` (branch `_<ENVIRONMENT>`) at build (framework not vendored);
+`015_install_composer.sh` runs `composer install`; instance-id/region written from IMDS for
+region-aware DB host selection; CloudWatch agent ships the JSONL log file;
+`long_gateway_timeout.conf` sets `ProxyTimeout 1800`/`Timeout 1800` — **the fix for the
+"exactly 60 second" 504** (EB Apache→PHP-FPM defaults to 60s); `enforce_https.conf`.
+
+## CI — commit message policy
+
+`.github/workflows/true-devteam-requirements.yml` enforces **`TRUE-{ticket}: {Subject}`** —
+subject ≤80 chars, capitalized, no trailing period, **imperative mood** (rejects `fixed`/
+`added`/`updated`), >1 word. Merge commits exempt.
+
+## ⚠️ Security note (committed secrets)
+
+`Config/production.ini` (and dev configs) contain **plaintext production credentials** —
+Core/Client/Logs DB passwords, **AWS access key id + secret**, and third-party API keys
+(FedEx, UPS, PayPal, NetSuite, Cal.com, AIG, OptimumDesk, VAPI). `.ebextensions/git.json`
+carries a **GitHub PAT**. These should be rotated and moved to SSM Parameter Store / EB env
+properties. **Flag this if you touch config or deploy.**
+
+## When making changes here
+
+- **Adding/altering an endpoint is usually a data change, not code** — define `Records` +
+  `RecordFields` rows and grant `AclRecordPermissions`/`AclFieldPermissions`. The model
+  class lives in `_underscore` (`Model/Client/...`).
+- **Client-specific behavior** belongs in `_Model_<ClientSlug>_X` overrides and
+  `Client_ApiPayloadInterceptor` pre/post hooks — not in `V2.php`.
+- **Don't break `/health`** or weaken `enforce_https.conf`.
+- **504s:** verify `long_gateway_timeout.conf` on both this tier and the proxy
+  (LB 3600s → proxy Apache 1800s → proxy cURL 900s → api Apache 1800s).
+- **`transactionId` uniqueness is load-bearing** — the idempotency/audit key; don't bypass
+  the duplicate check.
+- Preserve the commit-logs / rollback-data-on-failure invariant when editing the controller
+  or `execute()`.

package/knowledge/2.0/apps/worker2/INDEX.md

@@ -0,0 +1,6 @@
+# worker2 (Worker) — 2.0 knowledge
+
+| Doc | Summary | Files |
+|-----|---------|-------|
+| [Worker (worker2) Architecture](architecture.md) | Worker (repo `worker2`) is an AWS Elastic Beanstalk **Worker Tier** application that processes background jobs. | worker2/Controller/Index.php, worker2/Worker/, worker2/LambdaFunctions/, _underscore/Worker.php |
+| [Creating Worker Actions](features/creating-worker-actions.md) | How to add a new callable Worker action — a PHP class whose `public static` methods are invoked as background jobs (via webhook, cron, or `_Worker::runTask()`). | worker2/Worker/, worker2/Controller/Index.php, _underscore/Worker.php |

package/knowledge/2.0/apps/worker2/architecture.md

@@ -0,0 +1,127 @@
+---
+title: Worker (worker2) Architecture
+framework: "2.0"
+repo: worker2
+project: Worker
+client: shared
+type: architecture
+status: active
+updated: 2026-06-08
+owners: [jcardinal]
+files:
+  - worker2/Controller/Index.php
+  - worker2/Worker/
+  - worker2/LambdaFunctions/
+  - _underscore/Worker.php
+related:
+  - ./features/creating-worker-actions.md
+---
+
+## Summary
+
+Worker (repo `worker2`) is an AWS Elastic Beanstalk **Worker Tier** application that
+processes background jobs. It's a `_underscore` 2.0 app (`index.php` is just
+`require '_underscore.php'`). Two job sources:
+
+1. **Webhooks** — external HTTP calls (ClickUp, GitHub, …) hit an AWS Lambda that inserts
+   a job into MySQL and sends its ID to SQS.
+2. **Cron jobs** — a Lambda scheduler fires timed jobs into MySQL and SQS every minute.
+
+**MySQL is the source of truth; SQS is delivery only.** All job state lives in
+`Core.WorkerJobs`. Every worker invocation reads from that table and writes its result back.
+
+## AWS infrastructure
+
+| Component | Notes |
+|---|---|
+| SQS `WorkerProductionQueue` | EB Worker tier listens here (`aws_worker_queue_url`). Visibility timeout 3600s; **no DLQ**. |
+| SQS `WebhookFallback` | S3 references for webhook payloads that failed the MySQL insert. |
+| S3 `agilant-worker-fallback` | Failed webhook payloads (`webhook-fallback/{uuid}.json`, 7-day lifecycle). |
+| API Gateway `WorkerWebhookIngestion` | `webhook.togahub.com` → Lambda. Lambda Function URLs are blocked by an AWS Org SCP, so API Gateway is the only public entry point. |
+
+## MySQL tables
+
+**`Core.WorkerJobs`** — unified execution record. Key columns: `uuid`,
+`jobType ENUM('ACTION','CRON')`, `cronJobId`, `dtQueued` (NULL = not yet queued),
+`dtStarted` (committed immediately at check-in), `dtCompleted`, `executionTime`,
+`instanceId`, `isSuccess` (NULL pending/running, 0 fail, 1 success), `action`,
+`parameters` JSON, `failureReason`.
+
+Derived status: `pending` (`dtQueued IS NULL AND isSuccess IS NULL`) → `queued` → `running`
+(`dtStarted` set, `dtCompleted` NULL) → `succeeded`/`failed`.
+
+**`Core.CronJobs`** — `isActive`, `name`, `schedule` (cron expr, **Central tz**),
+`maxExecutionTime` (watchdog seconds, default 300), `action`, `parameters` JSON.
+
+## The four Lambdas (`worker2/LambdaFunctions/`, Python)
+
+`boto3` is in the runtime; `pymysql`, `croniter`, `pytz` are vendored into the zip.
+
+1. **WebhookIngestion** — API Gateway → derives action from URL path (kebab → Pascal,
+   `/Webhook` appended; `/clickup/task-update` → `Clickup/TaskUpdate/Webhook`), logs to
+   `Logs.Webhook` (non-fatal), INSERTs `WorkerJobs` (`ACTION`, `dtQueued=NOW()`), sends
+   `{workerJobId}` to the queue. On SQS failure resets `dtQueued=NULL`; on MySQL failure
+   writes payload to S3 + `WebhookFallback`. **Always returns HTTP 200.**
+2. **PayloadRecovery** — every minute, recovers S3 fallback payloads into `WorkerJobs`.
+3. **JobScheduler** — every minute. **Part A:** picks up `dtQueued IS NULL AND isSuccess
+   IS NULL` (≤100), sets `dtQueued=NOW()`, sends to queue. **Part B (watchdog):** marks
+   jobs `isSuccess=0` when `NOW() > dtStarted + timeout` (CronJobs.maxExecutionTime, else
+   env `MAX_EXECUTION_TIME_WORKER_ACTION` default 300).
+4. **CronScheduler** — every minute, evaluates active `CronJobs.schedule` against the
+   current Central-tz minute (`croniter`); on match INSERTs a `CRON` job and queues it.
+
+## EB worker tier — `Controller/Index.php`
+
+The `worker()` method handles all SQS deliveries. **Critical rule: always return HTTP
+200** — even on failure. Returning 500 keeps the message in-flight until the visibility
+timeout; job outcomes are tracked in `WorkerJobs.isSuccess`, so SQS retry is unwanted.
+
+New path (`{workerJobId}` present):
+1. Fetch the job; guard: if missing or `isSuccess IS NOT NULL` → 200, skip.
+2. **Check-in:** `UPDATE dtStarted=NOW(), instanceId` — immediately committed via
+   `_Database::transactionCommit(DB_CORE)` so no rollback can undo it.
+3. Register DB_LOGS / DB_CLIENT_LOGS (try/catch → failure marks job failed, returns 200).
+4. Resolve class+method from action (`Clickup/Webhook` → `_Worker_Clickup::Webhook`).
+5. Call `initialize()` if present, then the action with spread parameters.
+6. **Check-out:** `UPDATE dtCompleted, executionTime, isSuccess, failureReason`. Return 200.
+
+Legacy path (`action` present, no `workerJobId`): used by `_Worker::runTask()` debug mode
+only; runs directly with no `WorkerJobs` tracking.
+
+## Action → class/method routing
+
+Split the action on `/`: last segment = method; everything before = class segments joined
+with `_` and prefixed `_Worker_`. `Team/Github/Merge` → `_Worker_Team_Github::Merge`,
+file `Worker/Team/Github.php`.
+
+## Programmatic invocation & manual retry
+
+- **`_Worker::runTask(action, parameters)`** (`_underscore/Worker.php`) — INSERT
+  `WorkerJobs` → **`transactionCommit(DB_CORE)` before SQS** → send `{workerJobId}`.
+  Debug mode posts synchronously with no tracking.
+- **`_Worker_Infrastructure_Worker::Retry($workerJobId)`** — resets the job fields,
+  commits before SQS, sends `{workerJobId}` immediately. Sets `dtQueued=NOW()` (not NULL)
+  so JobScheduler doesn't double-pick it.
+
+## Cleanup cron
+
+`_Worker_Infrastructure_Worker_Cleanup`: `WorkerJobs()` deletes `isSuccess=1` rows >90 days
+(≤1000/run); `WebhookLogs()` deletes `Logs.Webhook` >90 days. Two daily CronJobs at 2 AM.
+
+## Critical transaction pattern
+
+**Any INSERT/UPDATE that must be visible to another connection before an SQS message is
+delivered must be immediately committed.** Three sites: `Retry()`, `_Worker::runTask()`,
+and the `Controller/Index.php` check-in. Reason: the caller's MySQL transaction is still
+open when SQS delivers; the worker reads in a separate connection and, if uncommitted, its
+SELECT returns nothing and the guard fires ("already processed or not found — skipping").
+
+## Key design decisions
+
+MySQL-first (SQS on success) prevents lost jobs · always HTTP 200 (WorkerJobs tracks
+failures, SQS retry unwanted) · 3600s visibility timeout for hour-long jobs · no DLQ
+(developer-controlled `Retry()` preferred) · all fallback payloads to S3 (no size split) ·
+SQS failure → reset `dtQueued=NULL` (rescheduled next minute) · `maxExecutionTime` on
+CronJobs, env var for ACTION jobs · `/Webhook` appended enforces method-name convention ·
+API Gateway in front of Lambda (SCP blocks Function URLs) · `workerJobId` lookup keeps SQS
+messages tiny · commit before SQS avoids the delivery-before-commit race.

package/knowledge/2.0/apps/worker2/features/creating-worker-actions.md

@@ -0,0 +1,135 @@
+---
+title: Creating Worker Actions
+framework: "2.0"
+repo: worker2
+project: Worker
+client: shared
+type: feature
+status: active
+updated: 2026-06-08
+owners: [jcardinal]
+files:
+  - worker2/Worker/
+  - worker2/Controller/Index.php
+  - _underscore/Worker.php
+related:
+  - ../architecture.md
+---
+
+## Summary
+
+How to add a new callable Worker action — a PHP class whose `public static` methods are
+invoked as background jobs (via webhook, cron, or `_Worker::runTask()`). Actions follow a
+path convention that maps directly to class/method names; in most cases **no routing or
+Lambda code changes are needed** — you just create the PHP file.
+
+> This replaces the former `/worker2-action` skill. To build one, run `kickoff` for
+> `worker2` and then describe the action you want; the conventions below are the contract.
+
+## What you need to decide
+
+1. **Action path** — `Category/Sub/File/MethodName`. Determines the class and file:
+   - File: `Worker/Category/Sub/File.php`
+   - Class: `_Worker_Category_Sub_File` (replace `/` with `_`, prefix `_Worker_`, **abstract**)
+   - Method: `MethodName`
+   - Example: `Client/Acme/ImportData` → `Worker/Client/Acme.php`, `_Worker_Client_Acme::ImportData`.
+2. **Parameters** — name + PHP type (`string`/`int`/`bool`/`array`/`float`); optional ones
+   get PHP defaults. The framework spreads the `parameters` JSON as positional arguments.
+3. **Return** — return `string`; `json_encode()` structured data.
+4. **`initialize()`** — optional `public static function initialize()` the framework calls
+   automatically before any method in the class (register DB connections / shared setup).
+
+## Class & method rules
+
+- File `Worker/{Path}.php`; class `abstract class _Worker_{Path_With_Underscores}`.
+- All action methods are `public static` and typed.
+- Typed parameters map 1:1 to the JSON `parameters` keys; optional params use defaults.
+
+### Non-webhook action template
+
+```php
+<?php
+
+abstract class _Worker_Category_MyAction {
+
+    public static function initialize() {
+        // Register DB connections / shared resources. Called before any method runs.
+        _Database::register(
+            database: 'SomeDatabase',
+            hostname: _Config::database('some_host'),
+            username: _Config::database(_Config::DATABASE_USERNAME),
+            password: _Config::database(_Config::DATABASE_PASSWORD),
+            alias:    'some_alias'
+        );
+    }
+
+    public static function DoSomething(string $param1, int $count = 0): string {
+        // perform work
+        return json_encode(['result' => 'done']);
+    }
+}
+```
+
+Invoke programmatically:
+`_Worker::runTask('Category/MyAction/DoSomething', ['param1' => 'value', 'count' => 5]);`
+
+### Webhook action template
+
+Webhook endpoints receive `$payload` (raw request body string) and `$headers`
+(lowercase-keyed — API Gateway lowercases all header names). Both required.
+
+```php
+<?php
+
+abstract class _Worker_MyService {
+
+    public static function Webhook(mixed $payload, object $headers): string {
+        $payload = json_decode($payload);
+        $event   = $headers->{'x-my-service-event'} ?? null;
+        // handle the event
+        return 'ok';
+    }
+}
+```
+
+## Adding a new webhook endpoint
+
+No Lambda/routing changes — just create the file:
+1. Create `Worker/MyService.php` with `_Worker_MyService` and a `Webhook(mixed $payload,
+   object $headers): string` method.
+2. Register the URL `webhook.togahub.com/my-service` with the external service.
+3. Events arrive → Lambda converts `/my-service` → `MyService/Webhook` → job inserted →
+   worker calls `_Worker_MyService::Webhook($payload, $headers)`. (Multi-segment paths:
+   `/my-service/task-update` → `MyService/TaskUpdate/Webhook`.)
+
+## Adding a new cron job
+
+1. Create the PHP file + method (e.g. `Worker/Reports/Daily.php`, method `Generate()`).
+2. Insert a `Core.CronJobs` row:
+   ```sql
+   INSERT INTO Core.CronJobs (uuid, isActive, name, schedule, maxExecutionTime, action, parameters)
+   VALUES (UUID(), 1, 'Daily Report Generation', '0 6 * * *', 300, 'Reports/Daily/Generate', NULL);
+   ```
+   `schedule` is evaluated in **Central time**; `maxExecutionTime` is the watchdog timeout.
+3. CronScheduler fires it at the next matching minute — no code wiring needed.
+
+## Test any action directly
+
+```bash
+curl -X POST https://worker.togahub.com/ \
+  -H "Content-Type: application/json" \
+  -d '{
+    "action": "Category/MyAction/DoSomething",
+    "parameters": { "param1": "example_value", "count": 5 }
+  }'
+```
+
+Use realistic placeholder values, not empty strings/nulls.
+
+## Gotchas
+
+- Class must be **`abstract`** and methods **`public static`** or routing fails.
+- Webhook headers are **lowercased** by API Gateway (`X-GitHub-Event` → `x-github-event`).
+- If the action needs a client DB, register it in `initialize()` — it runs before the method.
+- See [architecture.md](../architecture.md) for the always-HTTP-200 rule and the
+  commit-before-SQS transaction pattern that the worker relies on.