toga-ai 1.0.457 → 1.0.458
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
|
@@ -0,0 +1,132 @@
|
|
|
1
|
+
---
|
|
2
|
+
type: session
|
|
3
|
+
slug: multi-client-api
|
|
4
|
+
title: Cross-client (multi-client) API retrieval brought from untested branch to working
|
|
5
|
+
author: jcardinal
|
|
6
|
+
repos: [api2, _underscore, worker2, dbchanges2]
|
|
7
|
+
framework: "2.0"
|
|
8
|
+
client: shared
|
|
9
|
+
status: active
|
|
10
|
+
created: 2026-07-28
|
|
11
|
+
updated: 2026-07-28
|
|
12
|
+
---
|
|
13
|
+
|
|
14
|
+
# Session: multi-client-api
|
|
15
|
+
**Date:** 2026-07-28
|
|
16
|
+
**Project/Repo:** api2 + _underscore + worker2 + dbchanges2 (2.0), all on branch `cache-layer`
|
|
17
|
+
**Task:** Take the cross-client (multi-client) data-retrieval feature — which existed on the `cache-layer` branch as "code-complete + reviewer-hardened but NEVER runtime tested" — and make it actually work end to end, ready for external testing.
|
|
18
|
+
|
|
19
|
+
---
|
|
20
|
+
|
|
21
|
+
## What WORKED
|
|
22
|
+
|
|
23
|
+
Working end to end on local dev: multi-client fan-out across 3 clients, ACL-filtered `client` object per row, typed multi-column sorting, keyset paging to page 2+, narrowed `fields`, per-query cache isolation, full meta parity.
|
|
24
|
+
|
|
25
|
+
- **Wiring the feature so it could execute at all.** `client`, `seekValues`, `seekUuid` were never parsed into `$httpOptions` (V2 builds it from an explicit allowlist ~line 350). Added them. Evidence: `XC:` trace first showed `handleListing enter` after this.
|
|
26
|
+
- **Namespace fix** — `CrossClient` type-hinted `\_Component_Api_V2` (global) but the class is `api\_Component_Api_V2`. Fixed all 4 refs. Evidence: constructor TypeError gone.
|
|
27
|
+
- **uuid row identity** (replacing numeric primary key) — `rowUuid CHAR(36) COLLATE utf8mb4_bin`, `UNIQUE (clientId, rowUuid)`, cursor tiebreaker `seekUuid`. Evidence: cache went from 1 row/page to a full 25.
|
|
28
|
+
- **Typed per-query sort columns** — `TableResults_<Tables.id>` with one typed column per sort field (model field type, or `FIELDOPT_SQL_TYPE` for calculated `FIELD_SQL`). Evidence: `sort=-dateOrder` produced `sort1 date`, verified in `information_schema`.
|
|
29
|
+
- **Index direction baked in** — `KEY sort_idx (sort1 DESC, …, rowUuid ASC, clientId ASC)`. Any DESC sort against an all-ASC index filesorts the whole table (`primaryKeyId ASC` was a fixed tiebreaker, so even a single-column DESC sort broke it).
|
|
30
|
+
- **Multi-column keyset seek in V2** — `seekValues` (JSON array) + `seekUuid`, lexicographic with NULL-safe `<=>` links and typed literals; tiebreaker `CAST(uuid AS BINARY)`. Verified predicate matched 107,511 rows after a real cursor.
|
|
31
|
+
- **Deepen-on-demand paging** — `Tables.safeRecordCount` (watermark-safe count). Evidence: page 2 returns rows 26–50 and `TableResults_*` grows.
|
|
32
|
+
- **Watermark-only deepening (efficiency)** — only the least-advanced client is re-queried per pass. Verified with the `-number DESC` case: Compass holds the watermark, NYCHH is not re-queried. ~1 sub-request per pass instead of one per client.
|
|
33
|
+
- **ACL-filtered `client` object per row** — `getAclFieldPermissions()` against the Core `Clients` record (id 51, `aclDatabase: CORE`) using the caller's **CORE** roles (`id.core.roles`). With roles [3,1] this grants 8 fields.
|
|
34
|
+
- **Field injection/stripping** — with `fields=number`, the sub-request asks for `number,uuid,dateOrder` and `servePage()` strips the injected ones. Resolved per REQUEST (not per fan-out) so cache hits strip too.
|
|
35
|
+
- **Sub-request failure surfacing** — `meta.crossClient.failures[]` + an ERROR per failed client in `messages[]`; response no longer returns 200/success when every in-scope client failed. This is what made the remaining bugs diagnosable in one run each.
|
|
36
|
+
- **Origin header on the auth handshake** — `/auth/encrypted-user-uuid` is Origin-gated; the fan-out now resolves a registered `Core.Domains` row for the target client and sends it.
|
|
37
|
+
- **Dangling-FK tolerance** — `getFullModelData()` catches a failed relationship expansion, logs it, sets the field null. One orphaned pointer no longer 500s an entire listing. NOTE: affects every REST listing in api2.
|
|
38
|
+
- **worker2 `Platform/Cache/Clean` + `Truncate`**, `DB_CACHE` registration at both boot paths, `*/5` cron.
|
|
39
|
+
- **Cache DB resolved BY NAME** in both api2 and worker2 (deploy blocker — see decisions).
|
|
40
|
+
- **Knowledge captured + pushed**: commits `b478ae0`, `2e614ae`, `ab91519` on the team repo `_main`.
|
|
41
|
+
|
|
42
|
+
---
|
|
43
|
+
|
|
44
|
+
## What did NOT work — DO NOT RETRY THESE
|
|
45
|
+
|
|
46
|
+
- **Numeric primary key as row identity (`primaryKeyId` / `seekId`).** A V2 LIST response **never** exposes `id` — records carry `uuid` only. Every row parsed as id `0`, all 25 collided on `UNIQUE (clientId, primaryKeyId)` under `INSERT IGNORE`, so exactly **1 row** cached per page and the cursor could never advance. Do not reintroduce an id-based cursor.
|
|
47
|
+
- **Defaulting the sort to `id`.** Same root cause — `id` is never returned, so `sortValues` was `[null]` on every row. Cursor stored `{"sortValues":[null]}`. Default is now `uuid`.
|
|
48
|
+
- **Hardcoding a named collation** (`utf8mb4_0900_as_cs`) in the seek predicate/ORDER BY. `ERROR 1253: COLLATION 'utf8mb4_0900_as_cs' is not valid for CHARACTER SET 'latin1'` for `Client_Nychh`. Client DBs do **not** share a charset: `utf8mb4_unicode_ci` (Compass), `utf8mb4_0900_ai_ci` (True), `latin1_swedish_ci` (NYCHH). Use `CAST(... AS BINARY)`. I verified only Compass+Core and generalised — that is what caused this.
|
|
49
|
+
- **`http_build_query()` for the sub-request URL.** Encodes commas as `%2C`; V2 never url-decodes, so `fields=number,uuid,dateOrder` arrived as one field literally named `number%2Cuuid%2CdateOrder` → `EV-8` "A field specified in your request does not exist". Forward options **verbatim** from the raw QUERY_STRING; assemble the URL by hand.
|
|
50
|
+
- **Sending `seekValues` JSON without decoding it in V2.** `json_decode()` on the still-percent-encoded value returns `null`, the engagement check fails **silently**, V2 falls back to OFFSET and replays page 1 forever. V2 now `urldecode()`s `seekValues`/`seekUuid` specifically.
|
|
51
|
+
- **`empty($model->id)` on a `_Model` field.** Returned TRUE for a populated field (`id` = 49) while `$model->id ?? 'unset'` read 49 fine. Fields live in `$_model_fields[...]['value']` behind `__get()`; `empty()`/`isset()` route through `__isset()` and disagree. Cost 3 debugging rounds and presented as "not authenticated". Copy to a plain local, test the local.
|
|
52
|
+
- **Building a client DB name as `'Client_' . clientIdentifier`.** Diverges in practice: clientIdentifier `Compass_Usa`, database `Client_Compass`. Resolve `Core.Clients.clientDatabaseId` → `Core.Databases.name` (mirrors `_underscore/Email.php` log-DB resolution).
|
|
53
|
+
- **Relying on `$fieldConfig` in V2's seek-term branches.** Undefined on this path (`Undefined variable $fieldConfig`) — it is only populated on some request paths. Derive from `$model->getFieldConfig()`.
|
|
54
|
+
- **Hardcoding `CACHE_DATABASE_ID`** (tried 145, then 151). `Databases.id` is auto-assigned per Core instance; it would only ever match one environment and fail silently everywhere else. Both apps now resolve by `Databases.name = 'Cache'`.
|
|
55
|
+
- **Inferring environment ids from slug ordering** in the DatabaseHosts seed. There are **27** environments and the families are not contiguous — id 23 is `sandbox-dev` (I had labelled it `sandbox-client`), and 24–27 (`sandbox-client`, `client-beta`, `client-gamma`, `client-alpha`) were missing entirely. Query `Core.Environments`.
|
|
56
|
+
- **`_Model_Cache_Table` writes and column defaults.** The model writes every declared field explicitly, so a `NOT NULL DEFAULT 0` column still receives `NULL` unless assigned (`Column 'safeRecordCount' cannot be null`).
|
|
57
|
+
- **Treating `$decoded->data` as the record array.** A V2 LIST nests records under the route key (`data: {"salesOrders": [...]}`); requiring `is_array($decoded->data)` discarded every **successful** sub-response as unusable.
|
|
58
|
+
- **Counting `$collected` without dedupe.** Re-fetched rows were counted twice, inflating `safeRecordCount` to 50 against 25 real rows, so later pages skipped deepening and returned empty. Self-concealing: the first failure corrupts the bookkeeping that would trigger the retry.
|
|
59
|
+
- **`internalApiRequest()` for the fan-out** (rejected at design time, do not revisit without new information): it reuses the caller's auth/ACL context (defeats per-client ACL delegation, the whole security premise), is synchronous/in-process (no concurrency), and shares the outer transaction (V2.php comment at the method confirms).
|
|
60
|
+
- **`_Object::get($decoded, ['messages', 0, 'message'])`** — does not traverse a numeric list index; returns the whole messages array.
|
|
61
|
+
|
|
62
|
+
---
|
|
63
|
+
|
|
64
|
+
## Not tried yet (candidates for next session)
|
|
65
|
+
|
|
66
|
+
- **`Truncate()` has never been executed.** Written and lint-clean, reuses `Clean()`'s verified helpers, but unrun. Run it against local first.
|
|
67
|
+
- **No load or concurrency test.** The PHP-FPM self-call contention risk (parent holds a worker while waiting on children needing workers from the same pool) is unmeasured. Watermark-only deepening mitigates but does not remove it.
|
|
68
|
+
- **Multi-field sort** (`sort=-dateOrder,number`) never tested — only single-field. The N-tuple compare and multi-column seek predicate are implemented but unexercised.
|
|
69
|
+
- **`where` clauses over the fan-out** never tested. Given V2 never url-decodes, encoded operators are the most likely place the raw-forwarding fix is still insufficient.
|
|
70
|
+
- **`recordsPerPage` variations** — only 25 (the default) was used. It is part of `Tables_identity`, so a different page size forks a new cache entry.
|
|
71
|
+
- **`prohibitedRecordCount`** was never observed non-null; the plumbing exists but no ACL-withheld rows were produced.
|
|
72
|
+
- **Deep paging** (page 3+) and the `MAX_DEEPEN_ITERATIONS = 50` ceiling.
|
|
73
|
+
- **Deploy to any non-local environment** — all testing was localhost/XAMPP.
|
|
74
|
+
- **The 8 deferred concerns** now recorded in `2.0/apps/api2/architecture.md` → *Known issues / accepted risks*.
|
|
75
|
+
|
|
76
|
+
---
|
|
77
|
+
|
|
78
|
+
## Current file state
|
|
79
|
+
|
|
80
|
+
| File | Status | Notes |
|
|
81
|
+
|------|--------|-------|
|
|
82
|
+
| `api2/Component/Api/CrossClient/CrossClient.php` | Modified (uncommitted) | Near-total rewrite. uuid identity, typed sort columns, N-tuple merge, watermark-only deepening, field injection/stripping, ACL client object, failure capture. |
|
|
83
|
+
| `api2/Component/Api/V2/V2.php` | Modified (uncommitted) | Option parsing (`client`/`seekValues`/`seekUuid` + urldecode), multi-column keyset seek, uuid ORDER BY tiebreaker, cross-client delegation + failure surfacing, dangling-FK guard, `CACHE_DATABASE_ID` removed. |
|
|
84
|
+
| `api2/Controller/Index.php` | Modified (uncommitted) | Cache DB registration resolved by name. |
|
|
85
|
+
| `_underscore/Model/Cache/Table.php` | Committed on `cache-layer` | `safeRecordCount` added; no `homeClientApiId`. |
|
|
86
|
+
| `_underscore/Model/Cache/Tables/Client.php` | Committed on `cache-layer` | `keysetCursor` JSON replaces `lastSortValue`/`lastPrimaryKeyId`. |
|
|
87
|
+
| `_underscore/Model/Core/Record.php` | Committed on `cache-layer` | `ttlCache` field declared (column existed, model field did not). |
|
|
88
|
+
| `worker2/Worker/Platform/Cache.php` | Modified (uncommitted) | `Clean()` (scheduled) + `Truncate()` (manual, **untested**). |
|
|
89
|
+
| `worker2/Controller/Index.php` | Modified (uncommitted) | `DB_CACHE` registration at both boot paths, resolved by name. |
|
|
90
|
+
| `worker2/_.php` | Committed on `cache-layer` | `const DB_CACHE = 'Cache'`. |
|
|
91
|
+
| `dbchanges2/Cache/2026-06-30a - MultiClientCacheTables.sql` | Modified (uncommitted) | `Tables` + `Tables_Clients`; `safeRecordCount`; `keysetCursor` JSON; no `homeClientApiId`. |
|
|
92
|
+
| `dbchanges2/Core/2026-06-30b - CacheClusterRegistrationAndRecordTtl.sql` | Modified (uncommitted) | Cache DB + hosts for all **27** environments; `Records.ttlCache`. **3 production RDS endpoint placeholders still `<...>`.** |
|
|
93
|
+
| `dbchanges2/Core/2026-07-27a - PlatformCacheCleanCron.sql` | Untracked | `*/5` cron for `Platform/Cache/Clean`. |
|
|
94
|
+
| Team KB (8 docs) | **Pushed** | `b478ae0`, `2e614ae`, `ab91519` on `_main`. |
|
|
95
|
+
|
|
96
|
+
---
|
|
97
|
+
|
|
98
|
+
## Decisions made
|
|
99
|
+
|
|
100
|
+
- **Row identity = record `uuid`, not the numeric primary key.** Chosen because a V2 LIST response never exposes `id`. Rejected: having V2 emit `id` for internal sub-requests (would widen the API's trust surface and expose internal identifiers over HTTP); rejected page-number pagination for sub-requests (abandons the deep-pagination property the design exists for).
|
|
101
|
+
- **Byte-ordered comparison everywhere** (`utf8mb4_bin` in the cache, `CAST(... AS BINARY)` in client queries). PHP's string comparison is byte-wise, so the merge and MySQL agree *by construction*. Rejected: any named collation — invalid on at least one client charset.
|
|
102
|
+
- **Result tables are per-QUERY**, not per record type, so sort columns can be typed for that specific query. Rejected: generic pre-typed sort slots (`sortInt1`, `sortStr1`…) which avoid DDL but waste columns.
|
|
103
|
+
- **Cache database resolved BY NAME**, no `CACHE_DATABASE_ID` constant in either app. Deliberate deviation from `CORE_LOGS_DATABASE_ID = 11`.
|
|
104
|
+
- **Cross-client is USER-LEVEL ONLY.** API credentials (`Client.Apis`) have no entitlement bridge — `Users_Clients` is user-keyed, no `Apis_Clients` exists. `homeClientApiId` dropped from the schema.
|
|
105
|
+
- **`ttlCache = 0` means "effectively uncached" but is floored at 1 minute**, so an entry always outlives the request that built it and the cleanup worker cannot delete rows mid-read.
|
|
106
|
+
- **`safeRecordCount` (provably-complete count) gates deepening**, never the raw stored row count — a fast client can leave rows cached that a lagging client would displace.
|
|
107
|
+
- **Only the watermark-holding client is deepened per pass.** A client past the watermark cannot make any row safe.
|
|
108
|
+
- **Sub-request failures surface to the caller**, and an all-clients-failed result is no longer reported as 200/success — "results unknown" is not "no results".
|
|
109
|
+
|
|
110
|
+
---
|
|
111
|
+
|
|
112
|
+
## Blockers
|
|
113
|
+
|
|
114
|
+
None blocking further development. Two items gate deployment:
|
|
115
|
+
|
|
116
|
+
1. **3 production RDS endpoint placeholders** (`<prod-cache-N ...>`) in `Core/2026-06-30b` need real values from DevOps. The production `regionId IS NULL` fallback row also points at the us-east-1 cluster because no Aurora Global endpoint exists for Cache yet.
|
|
117
|
+
2. **Local `Core.DatabaseHosts` has a bad row**: env 23 (`sandbox-dev`) was seeded with `client.sandbox.database.togahub.com`. Re-running the migration will NOT fix it (the `NOT EXISTS` guard skips it). Delete and re-run:
|
|
118
|
+
```sql
|
|
119
|
+
DELETE FROM Core.DatabaseHosts
|
|
120
|
+
WHERE databaseId = (SELECT id FROM Core.`Databases` WHERE name = 'Cache')
|
|
121
|
+
AND environmentId = 23;
|
|
122
|
+
```
|
|
123
|
+
Also: the 27 host hostnames follow the stated slug convention but **none have been DNS-verified**, and existing `Logs`/`Client` rows use raw RDS endpoints for some environments — so the convention may not be universal.
|
|
124
|
+
|
|
125
|
+
---
|
|
126
|
+
|
|
127
|
+
## Exact next step
|
|
128
|
+
|
|
129
|
+
> Run `_Worker::runTask('Platform/Cache/Truncate', [])` against local to exercise `worker2/Worker/Platform/Cache.php::Truncate()` — the only piece written this session that has never been executed. Confirm `Cache.Tables` and `Cache.Tables_Clients` are empty and every `TableResults_*` table is dropped, then commit the 3 api2 + 2 worker2 + 2 dbchanges2 files on `cache-layer`.
|
|
130
|
+
|
|
131
|
+
---
|
|
132
|
+
_Saved by /session-save on 2026-07-28_
|
package/package.json
CHANGED