toga-ai 1.0.808 → 1.0.809

package/knowledge/2.0/apps/_underscore/features/cloud-s3-helpers.md

@@ -13,6 +13,7 @@ files:
 related:
   - ../../worker2/features/oneuptime-worker2-monitoring.md
   - ../../worker2/features/compass-vip-support-importer.md
+  - ../../worker2/features/s3-folder-retention-cleanup.md
 ---
 
 ## Summary

package/knowledge/2.0/apps/worker2/INDEX.md

@@ -46,6 +46,7 @@
 | [Platform Cache Cleanup (_Worker_Platform_Cache — Clean + Truncate)](features/platform-cache-cleanup.md) | `_Worker_Platform_Cache` owns maintenance of the shared **Cache** cluster that backs api2's [multi-client data retrieval](../../api2/features/cross-client-data- |
 | [QA/QC Branch Automation (auto-mirror + rebuild-from-ledger revert)](features/qa-qc-branch-automation.md) | Two CTO-approved (AGREE) pieces of GitHub branch automation, centralized in **worker2** rather than per-repo GitHub Actions. |
 | [QA/QC Review Pipeline (ClickUp review chain, QC batch + defect/client-review)](features/qa-qc-review-pipeline.md) | The ClickUp-driven review + promotion chain for the platform-wide QA/QC dev process (per the approved `QA-QC Plan.txt`), moving a task through **Development → Q |
+| [S3 folder retention cleanup worker action (CleanS3Folder)](features/s3-folder-retention-cleanup.md) | `_Worker_Infrastructure_Aws_Cleanup::CleanS3Folder` (action path `Infrastructure/Aws/Cleanup/CleanS3Folder`) deletes S3 objects under **one folder** (key prefix |
 | [Service Request → Sales Order → Purchase Order generation (Sync/ServiceRequest)](features/service-request-sales-order-generation.md) | `_Worker_Sync_ServiceRequest` turns a **Service Request into a Sales Order, and then into one Purchase Order per vendor**, for **any** tenant. |
 | [SSO Stability Monitor (Monitor/Operations/SsoStability)](features/sso-stability-monitor.md) | `_Worker_Monitor_Operations::SsoStability(): string` is the **reporter half** of the SAML uptime monitor. |
 | [Startech Webhook Handler (worker2)](features/startech-webhook-handler.md) | Receives inbound webhook events from Startech (Easeedesk) and creates or updates the corresponding ticket in TOGA 2.0. |

package/knowledge/2.0/apps/worker2/features/cross-account-aws-access.md

@@ -6,7 +6,7 @@ project: Worker
 client: shared
 type: feature
 status: active
-updated: 2026-08-24
+updated: 2026-09-14
 owners: [jcardinal, bala]
 files:
   - worker2/Component/Aws/Workloads/Workloads.php
@@ -171,8 +171,40 @@ that account is the
 `agilant-worker-alpha` and `agilant-worker-beta` as **separate EB environments that have no role
 registry at all**. An account-level EC2 scan would sweep them in and misreport them.
 
+### `agilant-as2` is in the LEGACY account 502614707982 — S3 work on it is cross-account
+
+The `agilant-as2` bucket (the Office Depot AS2/EDI drop bucket, `us-west-2`) is **not** in the
+868 production hub where worker2 runs. Verified via `aws s3api list-buckets` per profile: only
+account **502614707982** lists it. So any S3 work on `agilant-as2` is **cross-account**.
+
+**The fix (same model as this component):** pass the bucket's **owner** account id as
+`awsAccountId` so the worker assumes `WorkloadsRuntime` **in that account** and touches the
+bucket **same-account**. Then an identity policy on the owner account's `WorkloadsRuntime`
+role is enough — **no bucket policy is needed**. Every `WorkloadsRuntime` trusts the 868
+instance role, so assuming into 502 from 654 works.
+
+**Diagnostic tell — the AccessDenied wording tells you the account.** The SDK error
+`AccessDenied ... is not authorized to perform: s3:ListBucket ... because no resource-based
+policy allows the s3:ListBucket action` is AWS's **cross-account** signal. A same-account
+identity-policy gap instead says *"no identity-based policy"*. Seeing **"no resource-based
+policy"** means the resource lives in another account — assume into the owner account.
+
+**IAM ARN gotcha (bucket-level vs object-level).** `s3:ListBucket` and
+`s3:GetBucketLocation` are **bucket-level** → the `Resource` must be the **bucket** ARN
+`arn:aws:s3:::agilant-as2` with **no** `/*`. `s3:DeleteObject` is **object-level** → the
+`Resource` must end in `/*` (scope it, e.g. `arn:aws:s3:::agilant-as2/OUTBOX/processed/*`).
+Putting `ListBucket` on the `/*` ARN is the classic mistake and returns exactly this
+AccessDenied.
+
+First S3 consumer of this pattern: the
+[S3 folder retention cleanup action](./s3-folder-retention-cleanup.md).
+
 ## Gotchas / known issues
 
+- **`agilant-as2` is cross-account (owner 502614707982), not in the 868 hub.** Assume into the
+  owner account; the `AccessDenied ... no resource-based policy` wording is the cross-account
+  tell, and `ListBucket`/`GetBucketLocation` need the bare bucket ARN while `DeleteObject`
+  needs the `/*` ARN. See [the section above](#agilant-as2-is-in-the-legacy-account-502614707982--s3-work-on-it-is-cross-account).
 - **Do not assume the 1.0 worker fleet is in the 868 production hub.** `AWS_WORKER_QUEUE_URL`
   (654654170868) is the SQS queue only; the instances are in the legacy account 502614707982,
   `us-west-2`. See [the section above](#which-account-the-10-worker-fleet-lives-in-easy-and-expensive-to-get-wrong).
@@ -191,6 +223,13 @@ registry at all**. An account-level EC2 scan would sweep them in and misreport t
   see [creating-worker-actions.md](./creating-worker-actions.md).
 
 ## Change history
+- 2026-09-14 — Recorded that **`agilant-as2` is owned by the legacy account 502614707982**
+  (`us-west-2`), not the 868 production hub — so S3 work on it is cross-account: pass the
+  owner account id as `awsAccountId` and no bucket policy is needed. Added the diagnostic tell
+  (`AccessDenied ... no resource-based policy` = cross-account; *no identity-based policy* =
+  same-account) and the IAM ARN gotcha (`ListBucket`/`GetBucketLocation` need the bare bucket
+  ARN, `DeleteObject` needs the `/*` ARN). Found while building the
+  [S3 folder retention cleanup action](./s3-folder-retention-cleanup.md). (jcardinal)
 - 2026-08-24 — Pinned down **which account the 1.0 `worker` fleet actually runs in**:
   **502614707982** (`us-west-2`, EB env `agilant-worker`), **not** the 654654170868 production hub
   that `App_Worker::AWS_WORKER_QUEUE_URL` implies — that account holds only the SQS queue the 1.0

package/knowledge/2.0/apps/worker2/features/s3-folder-retention-cleanup.md

@@ -0,0 +1,130 @@
+---
+title: S3 folder retention cleanup worker action (CleanS3Folder)
+framework: "2.0"
+repo: worker2
+project: Worker
+client: shared
+type: feature
+status: active
+updated: 2026-09-14
+owners: [jcardinal]
+files:
+  - worker2/Worker/Infrastructure/Aws/Cleanup.php
+  - dbchanges2/Core/2026-09-14a - Insert S3 Folder Cleanup CronJob.sql
+related:
+  - ./creating-worker-actions.md
+  - ./cross-account-aws-access.md
+  - ../../_underscore/features/cloud-s3-helpers.md
+  - ./platform-cache-cleanup.md
+---
+
+## Summary
+
+`_Worker_Infrastructure_Aws_Cleanup::CleanS3Folder` (action path
+`Infrastructure/Aws/Cleanup/CleanS3Folder`) deletes S3 objects under **one folder** (key
+prefix) that are older than a retention window. **Both the folder and the days-to-keep are
+cron parameters**, so one action + one `Core.CronJobs` row cleans any folder with **no code
+change**. First use case: keep 30 days of `s3://agilant-as2/OUTBOX/processed/`.
+
+> This is the **S3** folder cleanup. It is separate from the DB "Cleanup cron" — see
+> [platform-cache-cleanup](./platform-cache-cleanup.md) for that one.
+
+## Key files / entry points
+
+- `worker2/Worker/Infrastructure/Aws/Cleanup.php` — `_Worker_Infrastructure_Aws_Cleanup::CleanS3Folder`.
+- `dbchanges2/Core/2026-09-14a - Insert S3 Folder Cleanup CronJob.sql` — the seed
+  `Core.CronJobs` row (seed rows live in dbchanges2, **never** a `.sql` in worker2).
+
+## How it works
+
+1. Parses an `s3://bucket/prefix/` URI into bucket + prefix.
+2. Assumes credentials via
+   `_Component_Aws_Workloads::assumeCredentials($awsAccountId)` and builds **one**
+   `Aws\S3\S3Client` reused for both list and delete. It uses **no static keys** and does
+   **not** use the `_Cloud` static-key helpers — this is a role-based cross-account path (see
+   [cross-account-aws-access](./cross-account-aws-access.md)).
+3. Paginates `listObjectsV2`; keeps only keys whose `LastModified` is older than
+   `now - daysToKeep`.
+4. Batch-deletes via `deleteObjects` (max 1000 keys per call).
+5. Returns a JSON summary on **every** path: `bucket`, `prefix`, `region`, `daysToKeep`,
+   `cutoffUtc`, `dryRun`, `scanned`, `eligible`, `deleted`, `failed`, `failedSample`,
+   `complete` — a proof-of-work return, per the
+   [creating-worker-actions](./creating-worker-actions.md) null-return rule.
+
+### Self-chunking — carry the S3 list position, not a page number
+
+A non-scheduler action is hard-capped at **300s** (the watchdog only honours
+`CronJobs.maxExecutionTime` for scheduler-fired jobs, and it does not stop the process — see
+[creating-worker-actions](./creating-worker-actions.md#-the-real-time-budget-is-300s--and-for-a-hand-triggered-action-it-cannot-be-raised)).
+The action works a ~240s slice, then re-enqueues itself with `_Worker::runTask(self::ACTION, …)`.
+
+The cursor it carries in the `resume` param is the **S3 `continuationToken`**, not a page
+counter:
+
+- **Why a keyset-style cursor is required:** S3 lists keys in **name order, not date order**.
+  A chunk can burn its whole 240s budget paging through recent keys before it reaches any old
+  object, and delete nothing that pass. Re-listing from page 1 each chunk (an earlier draft
+  did this) then **stalls the sweep forever with no alert** — every chunk re-scans the same
+  front of the list. Carrying the `continuationToken` guarantees the sweep advances through
+  the whole prefix.
+- **Backstop:** `MAX_CHUNK_ATTEMPTS = 100`.
+- **Single-attempt hand-off:** `runTask` mints a fresh uuid per call, so a *retried* hand-off
+  would fork the chain — do not retry it (same rule as
+  [creating-worker-actions](./creating-worker-actions.md#do-not-retry-the-chunk-hand-off--a-failed-hand-off-must-break-the-chain)).
+- **Normalize `resume` with a full round trip** — `json_encode` then `json_decode(true)` —
+  because the dispatcher casts only the top level of parameters, so a nested object arrives as
+  `stdClass`.
+
+### Destructive-delete safety guards (reviewed by CTO + cso)
+
+This deletes production S3, so the action refuses to run unless the target is provably safe:
+
+- Refuses a missing `s3://` scheme.
+- Refuses an **empty prefix** (never a whole-bucket sweep).
+- Forces the prefix to end in `/`.
+- Refuses a prefix shallower than `MIN_PREFIX_DEPTH = 2` folder levels — so a mistyped row
+  cannot hit a top-level or a live feed like `OfficeDepot/`.
+- `daysToKeep` must be `1..3650`.
+- Skips the folder-placeholder key.
+- Validates `awsAccountId` against a **3-account allowlist** (`654654170868` prod,
+  `502614707982` legacy, `975050298201` non-prod) **before** assume-role, because
+  WorkloadsRuntime reaches all three with broad permissions.
+- Logs each failed `deleteObjects` key — `Errors[]` returns HTTP 200 and does **not** throw on
+  per-key failure — and carries a capped `failedSample`.
+
+## Scheduling
+
+The seed `Core.CronJobs` row is
+`dbchanges2/Core/2026-09-14a - Insert S3 Folder Cleanup CronJob.sql`:
+
+- action `Infrastructure/Aws/Cleanup/CleanS3Folder`
+- schedule `0 2 * * *` (daily, Central), `maxExecutionTime` 300
+- parameters
+  `{"folder":"s3://agilant-as2/OUTBOX/processed/","daysToKeep":30,"awsAccountId":"502614707982","awsRegion":"us-west-2"}`
+- guarded `INSERT ... SELECT ... FROM DUAL WHERE NOT EXISTS`, fresh random v4 uuid.
+
+`awsAccountId` is the **legacy** account `502614707982` because `agilant-as2` lives there — see
+[cross-account-aws-access](./cross-account-aws-access.md). Ship the worker2 code deploy and
+this dbchanges2 migration **together**.
+
+## Client variations
+
+None — shared infrastructure, not client-specific.
+
+## Gotchas / known issues
+
+- **S3 lists by name, not date** — the whole reason the cursor must be a `continuationToken`
+  and not a page number (see self-chunking above).
+- `deleteObjects` returns HTTP 200 with per-key `Errors[]` — it does not throw on a partial
+  failure, so you must read `Errors[]` yourself.
+- `agilant-as2` is in **us-west-2** and is owned by a **different AWS account** — pass the
+  right `awsRegion` and `awsAccountId` (see related docs).
+
+## Change history
+- 2026-09-14 — Created. Added the generic `CleanS3Folder` action (folder + days-to-keep are
+  cron params, one row per folder, no code change), its self-chunking sweep carrying the S3
+  `continuationToken` (S3 lists by name not date, so a page-number cursor stalls the sweep),
+  and the destructive-delete guards (allowlist accounts, min prefix depth, prefix must end in
+  `/`, never whole-bucket). Seed cron row in dbchanges2 `Core/2026-09-14a`. (jcardinal)
+</content>
+</invoke>

package/knowledge/INDEX.md

@@ -19,7 +19,7 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
 ## 2.0 framework
 
 - **_underscore** (_Underscore) _(framework core)_ — 83 doc(s) → [2.0/apps/_underscore/INDEX.md](2.0/apps/_underscore/INDEX.md)
-- **worker2** (Worker) — 68 doc(s) → [2.0/apps/worker2/INDEX.md](2.0/apps/worker2/INDEX.md)
+- **worker2** (Worker) — 69 doc(s) → [2.0/apps/worker2/INDEX.md](2.0/apps/worker2/INDEX.md)
 - **api2** (API) — 26 doc(s) → [2.0/apps/api2/INDEX.md](2.0/apps/api2/INDEX.md)
 - **dbchanges2** (Database Changes) _(framework core)_ — 19 doc(s) → [2.0/apps/dbchanges2/INDEX.md](2.0/apps/dbchanges2/INDEX.md)
 - **toga2-supply** (TOGa Supply) — 9 doc(s) → [2.0/apps/toga2-supply/INDEX.md](2.0/apps/toga2-supply/INDEX.md)

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "toga-ai",
-  "version": "1.0.808",
+  "version": "1.0.809",
   "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
   "keywords": [
     "claude",