toga-ai 1.0.344 → 1.0.345
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
|
@@ -57,8 +57,12 @@ and — with a per-row reason — what was skipped.
|
|
|
57
57
|
note in one column) is **not** a stop.
|
|
58
58
|
7. **Apply.** Staged updates are applied in batches inside one transaction
|
|
59
59
|
(`buildBatchUpdateSql` — a `CASE id WHEN … THEN …` update); rollback on any failure.
|
|
60
|
-
8. **
|
|
61
|
-
|
|
60
|
+
8. **Clean up the source.** On a successful run (and unless `TEST_MODE`): a local input file
|
|
61
|
+
is unlinked; a URL input has its **source S3 object deleted** so an input file (which may
|
|
62
|
+
contain PII) does not linger (see *Source cleanup* below). The downloaded temp copy is
|
|
63
|
+
always removed in the `finally`.
|
|
64
|
+
9. **Report.** Returns JSON `{updated, skipped, stoppedAtRow, sourceDeleted, skippedRows[]}`.
|
|
65
|
+
There is no echo logging — the returned JSON is the entire output.
|
|
62
66
|
|
|
63
67
|
### Matching keys
|
|
64
68
|
|
|
@@ -81,6 +85,29 @@ an `.xlsx` suffix** so PhpSpreadsheet's `IOFactory` resolves the reader by exten
|
|
|
81
85
|
file is **always removed in a `finally` block**. A local input file is unlinked after a
|
|
82
86
|
successful run (unless `TEST_MODE`).
|
|
83
87
|
|
|
88
|
+
### Source cleanup
|
|
89
|
+
|
|
90
|
+
After a successful run, `deleteRemoteSourceIfS3()` deletes the **source** S3 object behind a
|
|
91
|
+
URL input so the input file (which may contain PII) does not linger. `parseS3Url()` extracts
|
|
92
|
+
the bucket, key, **and region** from either a virtual-hosted (`<bucket>.s3.<region>.amazonaws.com`)
|
|
93
|
+
or path-style (`s3.<region>.amazonaws.com/<bucket>/<key>`) HTTPS URL; region defaults to
|
|
94
|
+
`us-east-1` when the URL omits it. The result JSON's `sourceDeleted` field reports the outcome.
|
|
95
|
+
Key decisions:
|
|
96
|
+
|
|
97
|
+
- **Uses the AWS SDK `S3Client->deleteObject` directly** (`use Aws\S3\S3Client`), **not**
|
|
98
|
+
`_Cloud::deleteFileFromS3`. That `_underscore` helper shells out via
|
|
99
|
+
`shell_exec("aws s3 rm …")`, and passing a user-supplied, URL-derived bucket/key into a shell
|
|
100
|
+
is a command-injection risk (the security rule forbids `shell_exec` with user input). The SDK
|
|
101
|
+
call avoids the shell entirely.
|
|
102
|
+
- **Client built with the bucket's own region** parsed from the URL, so a cross-region bucket
|
|
103
|
+
is deleted correctly — the same single-region constraint that breaks `_Cloud::getFileFromS3`
|
|
104
|
+
on cross-region reads applies to any S3 client that assumes the config-default region.
|
|
105
|
+
- **Best-effort — never throws.** The whole call is wrapped in try/catch; the import already
|
|
106
|
+
committed its DB updates, so a delete failure (e.g. a missing `s3:DeleteObject` permission) is
|
|
107
|
+
logged and surfaced as `sourceDeleted: "delete failed: …"` rather than failing the run.
|
|
108
|
+
- **Only on success**, and **skipped in `TEST_MODE`** (so the source is kept for re-runs) and
|
|
109
|
+
for non-S3 URLs / local paths (`sourceDeleted: "skipped — not an S3 URL"`).
|
|
110
|
+
|
|
84
111
|
## Client variations
|
|
85
112
|
|
|
86
113
|
Compass-only. Some spreadsheet rows reference users that belong to a **different Compass tenant
|
|
@@ -112,6 +139,12 @@ reason, which is expected, not an error.
|
|
|
112
139
|
[_Cloud S3 helpers](../../_underscore/features/cloud-s3-helpers.md).
|
|
113
140
|
|
|
114
141
|
## Change history
|
|
142
|
+
- 2026-07-15 — On a successful URL-input run, `ProcessVipFile` now deletes the source S3 object
|
|
143
|
+
(new `deleteRemoteSourceIfS3()` / `parseS3Url()`) so a PII input file does not linger, and
|
|
144
|
+
returns a `sourceDeleted` field. Uses the AWS SDK `S3Client->deleteObject` directly (bucket's
|
|
145
|
+
own region from the URL), **not** `_Cloud::deleteFileFromS3`, whose `shell_exec("aws s3 rm …")`
|
|
146
|
+
would put URL-derived bucket/key into a shell (command-injection risk). Best-effort (never
|
|
147
|
+
throws), success-only, skipped in `TEST_MODE`. (bala)
|
|
115
148
|
- 2026-07-15 — Adapted to Compass's new 12-column format (banner row above the header;
|
|
116
149
|
`Support Tech First`/`Support Tech Last` removed, so `Support Tech Email` moved col N→L);
|
|
117
150
|
added `findHeaderRowNumber()` scanning for `PERS. NO.` and switched to explicit
|
package/package.json
CHANGED