toga-ai 1.0.260 → 1.0.263

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -7,5 +7,6 @@
7
7
  | [Tools MVC — Routing, CSRF & App_Database Access Patterns](features/mvc-data-access-patterns.md) | The load-bearing 1.0 (`App_`) framework conventions a developer needs when adding a page to the Tools app — URL routing, CSRF, and DB access through `App_Databa | tools/_/app/nav.php, tools/mvc/get.php |
8
8
  | [Tools Persona-Gated Navigation (App_Nav)](features/persona-gated-navigation.md) | `App_Nav` is the Tools app's two-level, **persona-gated** navigation. | tools/_/app/nav.php, tools/mvc/get.php |
9
9
  | [Tools SAML SSO Consumer & Persona-Gated Auth (App_Auth)](features/saml-sso-auth.md) | `App_Auth` is the Tools app's authentication layer: it consumes the SAML gateway `?saml=` handoff (see the 2.0 SAML downstream integration contract), establishe | tools/_/app/auth.php, tools/mvc/sso/initiate/get.php, tools/mvc/sso/get.php, tools/mvc/login/get.php, tools/mvc/login/post.php, tools/mvc/logout/get.php, tools/mvc/get.php, tools/config.production.ini, tools/config.local.ini |
10
+ | [Talos Knowledge Base Admin UI (KB Documents + Vocabulary)](features/talos-kb-documents-admin.md) | A Tools (1.0) admin UI to browse/fix the Talos knowledge-base documents and manage the transcript-cleanup vocabulary — without a deploy. | tools/mvc/talos/kb-documents/get.php, tools/mvc/talos/kb-documents/post.php, tools/mvc/talos/vocabulary/get.php, tools/mvc/talos/vocabulary/post.php, tools/_/app/talos/s3.php, tools/_/app/worker.php, tools/_/app/nav.php |
10
11
  | [Talos Pricing UI (Onboarding, Dashboard, Benchmarks, Cost Factors + Estimator)](features/talos-pricing-ui.md) | The 1.0 (tools app) face of the **Talos Pricing Platform** — a "Talos Pricing" nav folder with four pages plus a client-side estimate engine. | tools/_/app/nav.php, tools/_/app/talos/estimator.php, tools/mvc/talos/onboarding/get.php, tools/mvc/talos/onboarding/post.php, tools/mvc/talos/pricing/get.php, tools/mvc/talos/benchmarks/get.php, tools/mvc/talos/factors/get.php, tools/mvc/talos/factors/post.php, tools/assets/css/style.css |
11
12
  | [Deploying Tools to Elastic Beanstalk (PHP 8.5 / Amazon Linux 2023)](workflows/deploy-to-elastic-beanstalk-al2023.md) | How the **Tools** 1.0 app boots on Elastic Beanstalk running `PHP 8.5 on 64bit Amazon Linux 2023/4.13.1 (aarch64)`. | tools/.ebextensions/004_http_to_https.config, tools/.ebextensions/006_mount-s3fs.config, tools/.ebextensions/007_setup_export_cache_folders.config, tools/.ebextensions/008_setup_ldap.config, tools/.ebextensions/009_setup_phpini.config, tools/.ebextensions/020_setup_git_libraries.config, tools/.ebextensions/050_register_instance_to_shared_application_load_balancer.config, tools/ebs/git.json |
@@ -0,0 +1,88 @@
1
+ ---
2
+ title: Talos Knowledge Base Admin UI (KB Documents + Vocabulary)
3
+ framework: "1.0"
4
+ repo: tools
5
+ project: Tools
6
+ client: shared
7
+ type: feature
8
+ status: active
9
+ updated: 2026-06-30
10
+ owners: [jcardinal]
11
+ files:
12
+ - tools/mvc/talos/kb-documents/get.php
13
+ - tools/mvc/talos/kb-documents/post.php
14
+ - tools/mvc/talos/vocabulary/get.php
15
+ - tools/mvc/talos/vocabulary/post.php
16
+ - tools/_/app/talos/s3.php
17
+ - tools/_/app/worker.php
18
+ - tools/_/app/nav.php
19
+ related:
20
+ - ./mvc-data-access-patterns.md
21
+ - ./persona-gated-navigation.md
22
+ - ../architecture.md
23
+ - ../../../2.0/apps/worker2/features/talos-transcript-ingestion.md
24
+ ---
25
+
26
+ ## Summary
27
+
28
+ A Tools (1.0) admin UI to browse/fix the Talos knowledge-base documents and manage the
29
+ transcript-cleanup vocabulary — without a deploy. It is the human front-end to the worker2
30
+ [Talos Transcript Ingestion](../../../2.0/apps/worker2/features/talos-transcript-ingestion.md)
31
+ pipeline. A new **"Talos Knowledge Base"** nav group (personas *Development Team* / *TOGa
32
+ Technology*) exposes two pages.
33
+
34
+ ## Pages
35
+
36
+ ### `/talos/kb-documents` — approved-doc browser
37
+ - KB list derived from the **actual S3 sub-folders** under `development-team/` (title-cased,
38
+ e.g. `office-depot` → "Office Depot") — not from `Team.KnowledgeBases`.
39
+ - Lists approved docs excluding `*.metadata.json`; columns Meeting Date (`n/j/y`) / Title /
40
+ Size / Last modified (`n/j/y g:i A`).
41
+ - Clickable rows open an overlay **modal** with a type-aware preview: text inline; PDF via a
42
+ presigned S3 URL in an iframe; Office files via the Google Docs viewer (presigned URL; MS
43
+ Office viewer noted as an alternative); unsupported → download/delete only.
44
+ - Download + delete are icon buttons in the modal. **Delete enqueues the worker `SyncKb`
45
+ re-sync** so Bedrock reflects the removal.
46
+
47
+ ### `/talos/vocabulary` — cleanup vocabulary manager
48
+ Single tabbed page: Prompt Template, Applications, Clients, People, Business Terms, Examples,
49
+ Replacements. (The old standalone `/talos/replacements` page was folded in here and deleted.)
50
+ Classic view → Edit → Save per tab, with **full-set reconciliation** (transactional, rolls
51
+ back on error), trash-icon row removal, and an auto-dismissing toast. Backs the `Team.*`
52
+ prompt/replacement tables that the worker2 pipeline reads at runtime.
53
+
54
+ ## Helpers
55
+
56
+ - **`App_Talos_S3`** (`_/app/talos/s3.php`) — S3 client; list KB slugs; list/count approved;
57
+ presigned URL; get/delete; slug sanitize+lowercase; approved-prefix path guard.
58
+ - **`App_Worker::enqueue()`** (`_/app/worker.php`) — inserts a `Core.WorkerJobs` row and sends
59
+ the `{"workerJobId": id}` SQS message, so a 1.0 app can enqueue worker2 jobs.
60
+
61
+ ## Deploy prerequisites
62
+
63
+ - `aws/aws-sdk-php` installed in `tools/vendor` (S3 + SQS).
64
+ - New `tools/config.*.ini` sections (none exist yet): `[database_core2]` (writable 2.0 Core,
65
+ dbname `Core`), `[worker]` (`queue_url` / `queue_region`), `[aws]` (production).
66
+ - Apply the dbchanges2 `Team/2026-06-30a..e` + `Core/2026-06-30a` migrations in order.
67
+ - Verify the Bedrock KB region / data-source name against the live account (TODOs in code).
68
+
69
+ ## Gotchas
70
+
71
+ - **KB list is derived from S3, not the DB.** The browse UI enumerates live S3 folders under
72
+ `development-team/`; `Team.KnowledgeBases` is not the source of truth for what's shown.
73
+ - **Delete is not just an S3 delete** — it also enqueues a worker `SyncKb` job to re-sync the
74
+ Bedrock data source.
75
+ - No new secret literals introduced; credentials come from the new config sections above.
76
+
77
+ ## Change history
78
+
79
+ - 2026-06-30 — Built the Talos KB admin UI: `/talos/kb-documents` (S3-derived KB list, approved
80
+ doc browser with type-aware preview modal, download + delete-with-resync) and `/talos/vocabulary`
81
+ (tabbed prompt/vocabulary/replacements manager with transactional full-set reconciliation;
82
+ folded in and deleted the old `/talos/replacements`). Added `App_Talos_S3` and
83
+ `App_Worker::enqueue()` (1.0 → worker2 SQS) helpers and the "Talos Knowledge Base" nav group. (jcardinal)
84
+
85
+ ## Related docs
86
+
87
+ - [Talos Transcript Ingestion Pipeline](../../../2.0/apps/worker2/features/talos-transcript-ingestion.md) — the worker2 backend.
88
+ - [Tools MVC — Routing, CSRF & App_Database Access Patterns](./mvc-data-access-patterns.md)
@@ -21,8 +21,9 @@
21
21
  | [Startech Webhook Handler (worker2)](features/startech-webhook-handler.md) | Receives inbound webhook events from Startech (Easeedesk) and creates or updates the corresponding ticket in TOGA 2.0. | worker2/Worker/Startech.php |
22
22
  | [Talos (TOGa IQ) Meeting-Notes Integration & Token Auto-Refresh (consumer)](features/talos-meeting-notes-integration.md) | How a **dev tool / agent consumes Talos (TOGa IQ)** to query the team meeting-notes corpus programmatically. | .claude/skills/plan-ticket/scripts/talos.js |
23
23
  | [Talos Pricing Automation (worker2 Cron — AWS Actuals, Calibration, Monthly Report)](features/talos-pricing-automation.md) | The worker2 half of the **Talos Pricing Platform** (see the talos `pricing-cogs-model` and tools `talos-pricing-ui` docs for the other halves). | worker2/Worker/Talos/Pricing.php, worker2/Database/TalosPricingCrons.sql |
24
+ | [Talos Transcript Ingestion Pipeline (worker2 → AWS Bedrock KBs)](features/talos-transcript-ingestion.md) | `_Worker_Team_Transcripts` (in addition to the upstream `Export` action — see [Teams Meeting Transcript Export](./teams-transcript-export.md)) now runs a fully | worker2/Worker/Team/Transcripts.php, worker2/bin/reprocess-transcripts.php, worker2/Config/production.ini, dbchanges2/Team/2026-06-30a, dbchanges2/Team/2026-06-30b, dbchanges2/Team/2026-06-30c, dbchanges2/Team/2026-06-30d, dbchanges2/Team/2026-06-30e, dbchanges2/Core/2026-06-30a |
24
25
  | [Team Sprint Management & Reporting](features/team-sprint-management.md) | `_Worker_Team_Sprint` (file `Worker/Team/Sprint.php`) is the engine behind TOGA's internal **development-sprint process and reporting**. | worker2/Worker/Team/Sprint.php |
25
- | [Teams Meeting Transcript Export](features/teams-transcript-export.md) | `_Worker_Team_Transcripts` (action `Team/Transcripts/Export`) polls Microsoft Graph for Teams meeting transcripts produced by a set of organizers, classifies ea | worker2/Worker/Team/Transcripts.php, worker2/Config/production.ini, worker2/Database/TeamsTranscriptExports.sql, dbchanges2/Core/2026-06-18a - Teams Transcript Export schedule.sql |
26
+ | [Teams Meeting Transcript Export](features/teams-transcript-export.md) | `_Worker_Team_Transcripts` (action `Team/Transcripts/Export`) polls Microsoft Graph for Teams meeting transcripts produced by a set of organizers and archives t | worker2/Worker/Team/Transcripts.php, worker2/Config/production.ini, worker2/Database/TeamsTranscriptExports.sql, dbchanges2/Core/2026-06-18a - Teams Transcript Export schedule.sql |
26
27
  | [VAPI Webhook Handler (worker2 — AI-BDR end-of-call processing)](features/vapi-webhook-handler.md) | `_Worker_Vapi` ([worker2/Worker/Vapi.php](worker2/Worker/Vapi.php)) is the **PHP side of the AI-BDR call loop** — the webhook that receives VAPI's end-of-call r | worker2/Worker/Vapi.php, worker2/Worker/Ai/Bdr/Vapi.php |
27
28
  | [WJE Freshservice Sync (worker2)](features/wje-freshservice-sync.md) | WJE ("WJE IT", helpdesk `wje.freshservice.com`) is a **Freshservice**-based help-desk client whose tickets, contacts, assets, groups, categories, and canned res | worker2/Worker/Wje.php, _underscore/Component/Api/Wje/Wje.php, _underscore/Model/Wje/Ticket.php, _underscore/Model/Wje/TicketNote.php, _underscore/Model/Wje/Contact.php, _underscore/Model/Wje/Unit.php, _underscore/Model/Wje/TicketTeam.php, _underscore/Model/Wje/TicketCategory.php, _underscore/Model/Wje/AssetType.php, _underscore/Model/Wje/PredefinedReply.php, library/app/api/wje.php, worker/crons/toga2/wje/import_supporting_records.php, worker/crons/toga2/wje/sync_togasupply_wje.php, worker/crons/notifications/reports/wje/wje_common.php, library/app/systemmonitor/wje.php, dbchanges2/Client_Wje/2024-10-04 - WjeOnboarding.sql |
28
29
  | [Ticket → ClickUp Pseudocode Planning (Talos-grounded)](workflows/ticket-to-pseudocode-planning.md) | A repeatable procedure for turning a ClickUp ticket into a reviewed, formatted implementation plan posted back to the ticket's `📝 Pseudocode` custom field. | test/@dave/clickup_md2delta.js |
@@ -0,0 +1,158 @@
1
+ ---
2
+ title: Talos Transcript Ingestion Pipeline (worker2 → AWS Bedrock KBs)
3
+ framework: "2.0"
4
+ repo: worker2
5
+ project: Worker
6
+ client: shared
7
+ type: feature
8
+ status: active
9
+ updated: 2026-06-30
10
+ owners: [jcardinal]
11
+ files:
12
+ - worker2/Worker/Team/Transcripts.php
13
+ - worker2/bin/reprocess-transcripts.php
14
+ - worker2/Config/production.ini
15
+ - dbchanges2/Team/2026-06-30a
16
+ - dbchanges2/Team/2026-06-30b
17
+ - dbchanges2/Team/2026-06-30c
18
+ - dbchanges2/Team/2026-06-30d
19
+ - dbchanges2/Team/2026-06-30e
20
+ - dbchanges2/Core/2026-06-30a
21
+ related:
22
+ - ./teams-transcript-export.md
23
+ - ./creating-worker-actions.md
24
+ - ../architecture.md
25
+ - ../../../1.0/apps/tools/features/talos-kb-documents-admin.md
26
+ - ../../../1.0/apps/test/features/talos-kb-pipeline.md
27
+ ---
28
+
29
+ ## Summary
30
+
31
+ `_Worker_Team_Transcripts` (in addition to the upstream `Export` action —
32
+ see [Teams Meeting Transcript Export](./teams-transcript-export.md)) now runs a fully
33
+ automated, cron-driven pipeline that ingests raw Teams transcripts into the **Talos / TOGa IQ**
34
+ AWS Bedrock knowledge bases. It replaces the old **manual** two-page web tooling
35
+ (`test/team/talos/kb_processor.php` + `kb_processor.ini` — see
36
+ [Talos Knowledge Base Pipeline](../../../1.0/apps/test/features/talos-kb-pipeline.md)); the
37
+ cleanup/classify/approve/archive/sync logic is a port of that script, but driven by cron jobs
38
+ and by **DB-editable** config instead of an `.ini`.
39
+
40
+ Flow: raw VTT at `s3://toga-private/transcripts/{date}/…` → `Scan` enqueues one `Process` job
41
+ per unprocessed key → `Process` AI-cleans + AI-classifies → approved `.txt` + Bedrock metadata
42
+ sidecar written to `s3://togaiq/development-team/{kb_slug}/approved/` → raw archived → Bedrock
43
+ `startIngestionJob` on that KB's data source.
44
+
45
+ ## Key files / entry points
46
+
47
+ - `Worker/Team/Transcripts.php` — actions `Scan`, `Process(string $sourceKey)`,
48
+ `SyncKb(string $kbSlug)` (all alongside the existing `Export`).
49
+ - `bin/reprocess-transcripts.php` — standalone CLI backlog reprocessor.
50
+ - `Config/production.ini` `[talos]` section — S3/Bedrock targets (see Configuration).
51
+ - DB config + ledger tables in the `Team.*` schema (see Data model), authored in dbchanges2
52
+ under `Team/2026-06-30a..e` + `Core/2026-06-30a`.
53
+
54
+ ## How it works
55
+
56
+ ### `Scan` (cron)
57
+ Lists unprocessed raw transcripts under the `toga-private` `transcripts/` prefix (skips any
58
+ `sourceKey` already in the `Team.TranscriptProcessing` ledger) and enqueues one `Process` job
59
+ each via `_Worker::runTask('Team/Transcripts/Process', ['sourceKey' => …])`. Scheduled by a
60
+ `Core.CronJobs` row (weekday business hours).
61
+
62
+ ### `Process(string $sourceKey)`
63
+ Port of `kb_processor`, per transcript:
64
+ 1. Fetch the raw VTT from S3 (`toga-private`, **us-west-2**).
65
+ 2. Call the TOGa IQ AI endpoint `https://api.togaiq.com/api/ai/generate`, model
66
+ `bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0`, with an **extended** `output_schema`
67
+ returning **both** `knowledge_doc` **and** `kb_slug` — the AI picks the single best active KB
68
+ from the supplied list, or `general`. The system prompt is **assembled at runtime** from DB
69
+ tables: a template body with `{{APPLICATIONS}}`/`{{CLIENTS}}`/`{{PEOPLE}}`/`{{BUSINESS_TERMS}}`/`{{EXAMPLES}}`
70
+ placeholders filled from `Team.TranscriptPromptTerms`.
71
+ 3. Apply DB-driven text replacements from `Team.TranscriptReplacements` (`:c` case-sensitive,
72
+ `:w` whole-word, `:cw`) via `preg_replace_callback` **so replacement backrefs stay literal**.
73
+ 4. Build the approved name `"{YYYY-MM-DD} - {Title Case}.txt"`.
74
+ 5. Upload to `s3://togaiq/development-team/{kb_slug}/approved/` (**us-east-1**) plus a
75
+ `{name}.txt.metadata.json` Bedrock sidecar.
76
+ 6. Archive the raw VTT to `…/{kb_slug}/archive/{date}/`, then delete the source.
77
+ 7. AWS Bedrock `startIngestionJob` on that KB's data source (retry 5×30s on
78
+ `ConflictException` / throttling). **If `kb_slug === 'general'`, sync ALL active KBs.**
79
+
80
+ Idempotency + status are tracked in `Team.TranscriptProcessing`
81
+ (`pending/cleaned/uploaded/archived/synced/failed`).
82
+
83
+ ### `SyncKb(string $kbSlug)`
84
+ On-demand single-KB re-sync (used by the Tools delete flow — see
85
+ [Talos KB Documents Admin](../../../1.0/apps/tools/features/talos-kb-documents-admin.md)).
86
+
87
+ ### `bin/reprocess-transcripts.php` (CLI backlog reprocessor)
88
+ Standalone CLI that bootstraps `_underscore`, lists existing raw transcripts under the
89
+ `toga-private` `transcripts/` prefix, and runs each through `Process()`. Flags
90
+ `--prefix` / `--limit` / `--dry-run`. Tolerates **both** the old `{slug}/{date}/` layout and the
91
+ new `{date}/` layout. Requires the `ENVIRONMENT` env var, e.g.
92
+ `ENVIRONMENT=production php bin/reprocess-transcripts.php --dry-run`.
93
+
94
+ ### Graph download hardening
95
+ The Graph transcript download is hardened against SSRF / token leak: `https`-only, a
96
+ `*.microsoft.com` host allowlist, and `FOLLOWLOCATION` off.
97
+
98
+ ## Data model (`Team.*` schema, DB alias core cluster)
99
+
100
+ Authored in dbchanges2 (`Team/2026-06-30a..e`, `Core/2026-06-30a`, all dated 2026-06-30). All
101
+ PKs `INT UNSIGNED` to match the existing Talos Team-table family.
102
+
103
+ - **`Team.TranscriptReplacements`** — `pattern, replacement, isCaseSensitive, isWholeWord, isActive`.
104
+ Seeded ~109 rules from the old `.ini`.
105
+ - **`Team.TranscriptPromptTerms`** — `category` ENUM (`application/client/person/businessTerm/example`),
106
+ `term, isActive, dtCreated, dtUpdated`. Seeded from the `.ini` CORRECTIONS lists. PEOPLE are
107
+ stored as full "First Last" names (only 2 confidently paired; the rest seeded as individual
108
+ tokens, intended to be reconciled via the Tools vocabulary UI).
109
+ - **`Team.TranscriptPromptTemplate`** — `name, bodyText` (w/ placeholders), `instruction, model,
110
+ temperature, maxTokens, timeoutSeconds, isActive`. One active row seeded from the `.ini` system prompt.
111
+ - **`Team.KnowledgeBases`** — `slug, name, isGeneral, bedrockKbId, isActive`. Seeded `general` +
112
+ `office-depot`. NOTE: the Tools browse UI derives the **live** KB list from S3 folders, not this table.
113
+ - **`Team.TranscriptProcessing`** — `sourceKey UNIQUE(255), kbSlug, approvedKey, archiveKey,
114
+ status` ENUM (`pending/cleaned/uploaded/archived/synced/failed`), `aiModel, failureReason,
115
+ dtProcessed`. Pipeline ledger + idempotency.
116
+ - **`Core.CronJobs`** INSERT for action `Team/Transcripts/Scan` (weekday business hours);
117
+ includes `dtCreated = NOW()` since the column has no default, and depends on the
118
+ `maxExecutionTime` column migration.
119
+
120
+ ## Configuration
121
+
122
+ `worker2/Config/production.ini` `[talos]`: `s3_bucket=togaiq`, `s3_region=us-east-1`,
123
+ `bedrock_region=us-east-1`, `kb_prefix=development-team-`, `s3_root_path=development-team/`.
124
+
125
+ ## Gotchas / known issues
126
+
127
+ - **Cross-region S3.** The raw bucket `toga-private` is **us-west-2** while `togaiq` is
128
+ **us-east-1**. `Process()` uses two S3 clients; the archive step is a **cross-region copy**.
129
+ - **`[talos]` keys were added to `production.ini` only** — the beta/development configs still
130
+ need them before the pipeline runs in those environments.
131
+ - **`general` KB fans out.** A `Process()` classified as `general` (or a `general` sync) triggers
132
+ a sync of **all** active KBs, not one.
133
+ - **Replacements must use `preg_replace_callback`** so replacement text with `$`/`\` backrefs is
134
+ treated literally (a plain `preg_replace` would interpret them).
135
+ - **worker2 had no prior CLI precedent** — the backlog reprocessor bootstraps by
136
+ `chdir($projectRoot)` then `require vendor/autoload.php` + `_underscore.php`, and requires the
137
+ `ENVIRONMENT` env var.
138
+ - **Deploy TODOs left in code:** verify the live Bedrock KB region and data-source name against
139
+ the actual account before first prod run.
140
+ - **Pre-existing plaintext secrets** in `worker2/Config/production.ini` and the reference
141
+ `kb_processor.ini` (AWS / aegra) should be rotated — not introduced by this work; no new secret
142
+ literals were added to any new file or migration.
143
+
144
+ ## Change history
145
+
146
+ - 2026-06-30 — Built the automated cron-driven ingestion pipeline (`Scan`/`Process`/`SyncKb`) +
147
+ the `bin/reprocess-transcripts.php` backlog CLI, porting the manual `kb_processor.php`. Added
148
+ the `Team.*` config/ledger schema (DB-editable replacements, prompt terms, prompt template, KB
149
+ list, processing ledger) and the `Scan` cron. Classification moved from title-based to AI (single
150
+ best KB, or `general`). Config vocabulary/replacements are now DB-driven and editable via the
151
+ Tools UI. (jcardinal)
152
+
153
+ ## Related docs
154
+
155
+ - [Teams Meeting Transcript Export](./teams-transcript-export.md) — the upstream S3 producer.
156
+ - [Talos KB Documents Admin](../../../1.0/apps/tools/features/talos-kb-documents-admin.md) — the Tools UI.
157
+ - [Talos Knowledge Base Pipeline](../../../1.0/apps/test/features/talos-kb-pipeline.md) — the manual script this ports.
158
+ - [Creating Worker Actions](./creating-worker-actions.md)
@@ -6,8 +6,8 @@ project: Worker
6
6
  client: shared
7
7
  type: feature
8
8
  status: active
9
- updated: 2026-06-18
10
- owners: ["ajean"]
9
+ updated: 2026-06-30
10
+ owners: ["ajean", "jcardinal"]
11
11
  files:
12
12
  - worker2/Worker/Team/Transcripts.php
13
13
  - worker2/Config/production.ini
@@ -16,13 +16,18 @@ files:
16
16
  related:
17
17
  - ../architecture.md
18
18
  - ./creating-worker-actions.md
19
+ - ./talos-transcript-ingestion.md
19
20
  ---
20
21
 
21
22
  ## Summary
22
23
 
23
24
  `_Worker_Team_Transcripts` (action `Team/Transcripts/Export`) polls Microsoft Graph for
24
- Teams meeting transcripts produced by a set of organizers, classifies each meeting by
25
- client (from the meeting title), and archives the raw WebVTT to S3. Scheduled via a single
25
+ Teams meeting transcripts produced by a set of organizers and archives the raw WebVTT to S3.
26
+ As of 2026-06-30 it **no longer classifies by client**: raw transcripts land **flat** at
27
+ `s3://toga-private/transcripts/{YYYY-MM-DD}/{HHMM}_{title-slug}_{shortId}.vtt` (no client
28
+ folder). Classification now happens later, by AI, in the downstream ingestion pipeline — see
29
+ [Talos Transcript Ingestion](./talos-transcript-ingestion.md), which is the consumer of this
30
+ S3 output. Scheduled via a single
26
31
  `Core.CronJobs` row firing weekdays at 9:30 / 11:30 / 13:30 / 14:30 / 15:30 / 16:30 / 17:30
27
32
  Central (`30 9,11,13,14,15,16,17 * * 1-5`).
28
33
 
@@ -69,11 +74,15 @@ date. `createdDateTime` is stamped minutes after the actual occurrence, so it da
69
74
  correctly. `getMeeting()` is therefore used only for the subject. `dtExported` uses
70
75
  `UTC_TIMESTAMP()` (not `NOW()`) so all three timestamps are UTC.
71
76
 
72
- **Classification:** case-insensitive substring match of the meeting title against active
73
- `Core.Clients` names + `[teamsClientAliases]`, longest needle wins, min length 3. No match → `general`.
77
+ **Classification: REMOVED (2026-06-30).** The old title-based classification (substring match
78
+ against `Core.Clients` + `[teamsClientAliases]`) was inaccurate and has been deleted
79
+ (`classify()`, `buildClientIndex()`, `loadClientNames()`, `loadClientAliases()`, along with the
80
+ old `Backfill()`/`moveObject()`). Classification is now done by the AI in the downstream
81
+ ingestion `Process()` step, which picks the single best knowledge base (or `general`).
74
82
 
75
- **S3 layout:** `s3://{[teams] s3_bucket}/{s3_prefix}{client-or-general}/{YYYY-MM-DD}/{HHMM}_{title-slug}_{shortId}.vtt`
76
- (date/time in Central). Currently `toga-private/transcripts/…`.
83
+ **S3 layout (current):** `s3://toga-private/transcripts/{YYYY-MM-DD}/{HHMM}_{title-slug}_{shortId}.vtt`
84
+ — **flat, no client/general folder** (date/time in Central). The old
85
+ `…/{client-or-general}/{date}/…` layout is history; the downstream reprocessor CLI tolerates both.
77
86
 
78
87
  ## Data model
79
88
 
@@ -162,6 +171,12 @@ Policy gap; a `400` only on a UPN means the id was never resolved to a GUID. (A
162
171
 
163
172
  ## Change history
164
173
 
174
+ - 2026-06-30 — **Removed title-based client classification and the client-folder S3 layout.**
175
+ Raw transcripts now land flat at `transcripts/{date}/…` with no client subfolder; deleted
176
+ `classify()`/`buildClientIndex()`/`loadClientNames()`/`loadClientAliases()` and the old
177
+ `Backfill()`/`moveObject()`. Classification moved downstream to the AI in the new
178
+ [Talos Transcript Ingestion](./talos-transcript-ingestion.md) pipeline, which is now the
179
+ consumer of this S3 output. (jcardinal)
165
180
  - 2026-06-18 — Fixed recurring-meeting date misfiling: archive date + `dtMeeting` now come from
166
181
  the transcript `createdDateTime`, not the series-anchor `meeting->startDateTime` (worker2 PR
167
182
  #82, merged + deployed). Added `Backfill()` and ran it in prod (61/66 objects re-filed, 0
@@ -12,12 +12,12 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
12
12
  - **walmarttechservices** (Walmart Tech Services) — 1 doc(s) → [1.0/apps/walmarttechservices/INDEX.md](1.0/apps/walmarttechservices/INDEX.md)
13
13
  - **test** (Test) — 12 doc(s) → [1.0/apps/test/INDEX.md](1.0/apps/test/INDEX.md)
14
14
  - **toga** (TOGa) — 2 doc(s) → [1.0/apps/toga/INDEX.md](1.0/apps/toga/INDEX.md)
15
- - **tools** (Tools) — 7 doc(s) → [1.0/apps/tools/INDEX.md](1.0/apps/tools/INDEX.md)
15
+ - **tools** (Tools) — 8 doc(s) → [1.0/apps/tools/INDEX.md](1.0/apps/tools/INDEX.md)
16
16
 
17
17
  ## 2.0 framework
18
18
 
19
19
  - **_underscore** (_Underscore) _(framework core)_ — 21 doc(s) → [2.0/apps/_underscore/INDEX.md](2.0/apps/_underscore/INDEX.md)
20
- - **worker2** (Worker) — 24 doc(s) → [2.0/apps/worker2/INDEX.md](2.0/apps/worker2/INDEX.md)
20
+ - **worker2** (Worker) — 25 doc(s) → [2.0/apps/worker2/INDEX.md](2.0/apps/worker2/INDEX.md)
21
21
  - **api2** (API) — 7 doc(s) → [2.0/apps/api2/INDEX.md](2.0/apps/api2/INDEX.md)
22
22
  - **dbchanges2** (Database Changes) _(framework core)_ — 3 doc(s) → [2.0/apps/dbchanges2/INDEX.md](2.0/apps/dbchanges2/INDEX.md)
23
23
  - **toga2-supply** (TOGa Supply) — 3 doc(s) → [2.0/apps/toga2-supply/INDEX.md](2.0/apps/toga2-supply/INDEX.md)
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "toga-ai",
3
- "version": "1.0.260",
3
+ "version": "1.0.263",
4
4
  "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
5
5
  "keywords": [
6
6
  "claude",
@@ -246,6 +246,11 @@ valid choice; this skill is opt-in.
246
246
 
247
247
  Tell the developer concisely:
248
248
  - Which framework(s), repo(s), and client are in scope, and the local path of each repo.
249
+ - **Name the framework core(s) explicitly and state the `app → core` dependency** for every
250
+ chosen app — e.g. "`worker` (1.0) → `library` core", "`api2` (2.0) → `_underscore` core".
251
+ Every repo depends on its framework's core (1.0 → `library`, 2.0 → `_underscore`); preflight
252
+ always includes it in `loadSet`. Do **not** bury the core inside a generic "loaded as
253
+ summaries" list — the developer must see that the core is loaded and which app pulled it in.
249
254
  - Which knowledge docs you loaded (by title). Where a repo/standard has **no knowledge yet**,
250
255
  say so explicitly: "No knowledge captured yet for X — `capture` will build it as you work."
251
256
  - Confirm you're primed and ready for their first task.