@talkpilot/core-db 1.3.38 → 1.3.39

package/README.md

@@ -1,721 +1,721 @@
-L1:# @talkpilot/core-db
-
-[NPM Version](https://www.npmjs.com/package/@talkpilot/core-db)
-`@talkpilot/core-db` is the shared TypeScript database package that wires TalkPilot APIs, municipal CRM integrations, and internal tools to a single, type-safe MongoDB surface. Every repo (MIS, CIS, TalkPilot Server, etc.) imports this package to avoid re-implementing connections, collections, or validation helpers.
-
-## Purpose
-
-- Provide a reliable, multi-domain MongoDB layer for TalkPilot and municipal data.
-- Export typed getters, vector search helpers, and document factories so services can focus on behavior instead of schema wiring.
-- Manage connection lifecycles, environment configuration, and test helpers from one place so every repo reuses the same plumbing.
-
-## Main Concepts
-
-- **Multi-domain clients** – `src/connection.ts` exposes `mongodbClient` (TalkPilot) and `municipalDataMongodbClient`, each of which resolves `MONGO_URI`, DB overrides, and default names.
-- **Domain-specific getters** – `src/talkpilot/` and `src/municipal/` host typed getters (agents, calls, streets, tickets, etc.), vector-search helpers, and service-friendly adapters that keep caller code DRY.
-- **Product-specific clients (future)** – While the package currently exposes the shared TalkPilot + municipal clients, we expect each product (CIS, MIS, TalkPilot Server) to eventually get its own domain-specific client helpers or wrappers so the shared core can remain stable while new consumers add targeted extensions.
-- **Test helpers** – `src/test-utils/` plus `src/__tests__/` reuse `MongoMemoryServer` and shared factories so tests start with clean data regardless of the consuming repo.
-- **Utility layers** – `src/utils/` contains shared validation, pagination, and environment helpers that complement the getters.
-- **Bulk-write helpers** – `src/bulkWrite/` exposes small, domain-agnostic functions (`buildSetOp`, `buildUpsertOp`, `buildInsertOp`, `buildDeleteOp`) that construct correctly-shaped MongoDB bulk-write operations from a filter and field values — no collection- or domain-specific knowledge required.
-- **Configuration domain** – `src/configuration/` hosts the `configuration` database (a `prompts` collection so far) — a place for cross-service settings that used to be hardcoded in consuming repos, editable without a deploy.
-- **Environment awareness** – Defaults, fallbacks, and `process.env` lookups ensure local, CI, and Cloud Run clients all connect using the right URI/DB names.
-
-## Key Components
-
-- `src/connection.ts` – Central connection logic that resolves URIs/DB names from env vars (`MONGO_URI`, `MONGODB_URI`, `TALKPILOT_DB_NAME`, `MUNICIPAL_DB_NAME`) and reuses a single `MongoClient`.
-- `src/talkpilot/` – Call history, agents, flows, sessions, leads, subscriptions, and support helpers exposed as getters plus helper enums/types for each collection.
-- `src/municipal/` – Municipal-specific collections (`cities`, `streets`, `departmentsSubjects`, `tickets`, etc.) plus vector search helpers and Ash Bina helpers used by MIS.
-- `src/utils/` – Shared helpers such as `resolveConnection`, pagination utilities, and schema validation helper functions.
-- `src/bulkWrite/` – Generic bulk-write op constructors (`buildSetOp`, `buildUpsertOp`, `buildInsertOp`, `buildDeleteOp`) usable with any collection's document type.
-- `src/configuration/` – `configurationMongodbClient` plus the `prompts` collection (`Prompt` type, `getPromptByName`, `findPrompts`, `getAllPrompts`, `getPromptsByProduct`).
-- `src/config.ts` – Centralized `process.env` reads for the configuration domain — add new env-backed fields here instead of reading `process.env` inline elsewhere.
-- `src/test-utils/` and `src/__tests__/` – Utilities that bootstrap `MongoMemoryServer`, expose factories, and make sure Jest environments can stub database calls predictably.
-- `dist/` – Compiled output consumed by downstream repos (CJS + ESM + type defs).
-
-## Domain APIs
-
-- **TalkPilot domain** – Imports like `findAgents`, `getFlows`, `findCalls`, and `vectorSearchCalls` live in `src/talkpilot`. These functions are the canonical access pattern for call history, session metadata, and provider configs.
-- **Municipal domain** – Helpers such as `findStreets`, `getMunicipalCities`, `findDepartmentSubjects`, and `createTicket` live under `src/municipal` and feed MIS workflows (street hints, subject matching, Ash Bina tickets).
-- **Configuration domain** – `getPromptByName`, `findPrompts`, `getAllPrompts`, and `getPromptsByProduct` live under `src/configuration` and back MIS's `moked_106` prompt lookups (defaults, overridable per-client via `clientConfig.toolsPrompts`).
-
-## Environment variables
-
-| Variable            | Purpose                                                                           | Required |
-|---------------------|-----------------------------------------------------------------------------------|----------|
-| `MONGO_URI`         | Primary MongoDB connection string for every domain (overridden by `MONGODB_URI`). | ✅        |
-| `MONGODB_URI`       | Alternate connection string used when Mongo needs a second URI parameter.         | ✅        |
-| `TALKPILOT_DB_NAME` | Optional override for the TalkPilot database name (defaults from URI path).       | ❌        |
-| `MUNICIPAL_DB_NAME` | Optional override for the municipal database name (defaults to `municipal-data`). | ❌        |
-| `CONFIGURATION_DB_NAME` | Optional override for the configuration database name (defaults to `configuration`). | ❌   |
-| `ENV`               | Free-form label used in logs/validators (defaults to `unknown`).                  | ❌        |
-
-If you pass a `uri` directly to `mongodbClient.connect()` or `municipalDataMongodbClient.connect()`, the client will prefer that value over the env vars.
-
-## Getting Started
-
-### Prerequisites
-
-- Node.js 22.x+ (aligns with downstream services).
-- npm 11+ or Yarn.
-- MongoDB accessible from your environment or a `MongoMemoryServer` for tests.
-
-### Setup
-
-1. Clone the repo and install dependencies:
-
-   ```bash
-   git clone https://github.com/talkpilot/core-db.git
-   cd core-db
-   npm install
-   ```
-
-2. Build the package before using it locally:
-
-   ```bash
-   npm run build
-   ```
-
-3. Import `@talkpilot/core-db` from another project by pointing `package.json` at the local path during development or installing the published release.
-
-## Sample `.env`
-
-```
-MONGO_URI=mongodb://localhost:27017
-TALKPILOT_DB_NAME=talkpilot-dev
-MUNICIPAL_DB_NAME=municipal-dev
-CONFIGURATION_DB_NAME=configuration-dev
-ENV=development
-```
-
-Adjust `MONGO_URI` to match the running Mongo instance and configure `talkpilot`/`municipal`/`configuration` DB names if you want to keep them separate.
-
-## Local development
-
-1. Run `npm install`.
-2. Build the compiled output: `npm run build`.
-3. Execute tests: `npm run test`.
-4. Use `npm link` or `npm pack` to consume the freshly built package from other repos (`CIS`, `MIS`, `TalkPilot Server`).
-
-## Development guide
-
-`DEVELOPMENT.md` contains the tactical steps for contributors. At a glance:
-
-- Node 18+/TypeScript is required (aligns with downstream services).
-- Run `npm install` → `npm run build` after cloning.
-- Use `npm link`/`npm link @talkpilot/core-db` to test the package locally before publishing.
-- When adding getters, define types, implement the function, export it through the domain `index.ts`, and add a corresponding test under the domain’s `__tests__` folder.
-- Always rely on the provided test factories (`src/test-utils/factories`) to seed data so tests remain consistent.
-- Jest with `mongodb-memory-server` is the only execution path we have to verify this core utility—unit tests are the safety net for every change.
-
-Refer to `DEVELOPMENT.md` for the full walkthrough, token instructions, and factory samples.
-
-## 🧪 Testing
-
-- `npm run test` – Jest suite (factories, utils, integration mocks) powered by `mongodb-memory-server`.
-- Tests rely on `src/__tests__/setup.ts` to bootstrap the in-memory Mongo instances and wire shared factories/helpers before each run.
-- When adding getters, helpers, or domain logic, create focused coverage inside the consuming domain’s `__tests__/` folder and use the provided factories to keep fixtures consistent.
-
-`@talkpilot/core-db` does not run in a product UI or feature branch—unit tests are the *only* reliable execution path to ensure your changes work. Every change must ship with a unit test so downstream repos can upgrade without surprises; treat the test suite as the canonical safety net for this core utility package.
-
-## 🧹 Lint & build verification
-
-- `npm run lint` – Run ESLint over `src/**/*.{ts,tsx}`.
-- `npm run format` – Format the source files with Prettier.
-- `npm run build` – Compile TypeScript and emit `dist/` (used by downstream consumers).
-
-## ✅ Pre-push checklist
-
-1. `npm run build`.
-2. `npm run test`.
-3. `npm run format`.
-
-## Publishing & release notes
-
-- Releases are handled by `npm version <patch|minor|major>` followed by `npm publish`. The package is a **private `@talkpilot` dependency**, so every contributor must install the shared npm automation token into their global `~/.npmrc` before running publish or `npm install`.
-- The shared token is rotated periodically—if you see authentication failures, request the refreshed token, update your `~/.npmrc`, and retry. Never commit credentials to source control.
-- After publishing, downstream repos (`CIS`, `MIS`, `TalkPilot Server`, etc.) should run `npm update @talkpilot/core-db` so they receive the latest helpers/bug fixes.
-- Cloud Build & Cloud Run jobs that depend on this package pick up the new version the next time they rebuild their container; the services pull the compiled `dist/` output and type definitions when installing the dependency.
-
-- Release process, when to publish, branch hygiene, and the function-signature versioning policy: see [`DEVELOPMENT.md`](./DEVELOPMENT.md).
-
-### Version history
-
-| Version | Note                                                                                                                                                                                                                                                                                                                                                         |
-|---------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| 1.1.9   | Maintenance release (build, test, format).                                                                                                                                                                                                                                                                                                                   |
-| 1.3.4   | Added new statistics getters for call & ticket dashboards (CP-149): summary, trend, hourly, and routing for calls, plus open/draft/subject ticket stats. **Removed** the legacy getters `getTicketsCountByCityAndDateRange` and `getTicketsSubjectStats`.                                                                                                    |
-| 1.3.5   | Added the `muniIssues` collection & module (CP-1115): the `MuniIssue` document type, `createMuniIssue` (accepts a caller-supplied `_id` for a two-way Jira link), `generateMuniIssueId`, `getMuniIssueById` / `getMuniIssueByJiraKey`, a `$jsonSchema` validator with indexes (unique `jira.key`), and `ensureMuniIssuesCollection` to provision it at boot. |
-| 1.3.6   | Added `contextNotes` and `products` modules under `src/talkpilot/`.                                                                                                                                                                                                                                                                                          |
-| 1.3.7   | Added `disableGenericPrompt` optional boolean field to the `Flow` type and schema (CP-704).                                                                                                                                                                                                                                                                  |
-| 1.3.8   | Ticket-count Map getters (CP-1200): deprecated `string[]` ticket getters and `CallsStatsFilter` fields, added `Map<string, number>` replacements; fixed ticket counting in `aggregateCallsSummary`.                                                                                                                                                          |
-| 1.3.9   | Fixed wrap-around hour filter boundary guards (CP-1166): excluded first-day morning calls and last-day evening calls from wrap-around windows.                                                                                                                                                                                                               |
-| 1.3.10  | Added optional `isDraft` flag to `Ticket` (CP-1390): explicit draft marking with legacy fallback for existing tickets.                                                                                                                                                                                                                                       |
-| 1.3.11  | Fixed pre-existing broken type declaration import in `clientsConfig.types.d.ts` that caused consumer `tsc` builds to fail.                                                                                                                                                                                                                                   |
-| 1.3.12  | Added `callsWithTickets` to `CallsSummaryAggregation` (CP-1200): distinct call count with at least one open ticket, enabling accurate `ticketOpenRate` and `not_opened` calculations in consumers.                                                                                                                                                           |
-| 1.3.13  | Faulty release (redundant). Published with `Flow.useRedisTranscription` at the flow level; superseded by 1.3.14.                                                                                                                                                                                                                                            |
-| 1.3.14  | Added `FlowTool.useRedisTranscription` optional flag to control whether cached Redis transcriptions are used (defaults to `false`).                                                                                                                                                                                                                          |
-| 1.3.15  | Multi-scan support for WebsiteTalk (CP-1372): `websiteUrls` collection, `Scan.websiteUrlId`, `OVERWRITTEN` status, `activeScanId` on `WebsiteTalkProduct`. **Breaking:** `Scan.baseUrl` removed; `getActiveScanStatus` removed. |
-| 1.3.16  | Refactored WebsiteTalk scan status constants (CP-1372): `SCAN_STATUSES` object map, `FINISHED_SCAN_STATUSES` derived from it, `isInProgressScan` moved to `scans.utils`. No breaking API changes. |
-| 1.3.17  | Twilio call-status semantics for Websitalk dashboard: `inCallCount` KPI (`in-progress` + `answered`), `busy` is line-busy only; `updateCallStatusByCallSid`; expanded `CallStatus` union; `timeSavedMinutes` from `completed` only. |
-| 1.3.18  | WebsiteTalk multi-website API (CP-1372): ships `websiteUrls` module, scan getters by `websiteUrlId`, `activeScanId` getters on `clientsConfig`. **Note:** 1.3.17 did not include these — they landed after merge. **Breaking:** `WebsiteTalkProduct.defaultBaseUrl` removed from type. |
-| 1.3.19  | WebsiteTalk flow provisioning (WTIS activate-scan): optional `flowId` on `WebsiteTalkProduct`; `bindClientPhoneToFlow(clientId, flowId)` sets `flow_id` on the client's primary `phone_numbers` document so inbound calls route to the provisioned TalkPilot flow. |
-| 1.3.20  | `FINAL_STATUSES` constant for terminal Twilio call statuses (`no-answer`, `completed`, `busy`, `failed`, `canceled`) in `calls.constants`. |
-| 1.3.21  | Added `ToolExecutionStep` type, `steps` field to `ToolExecution`, and `pushToolExecution` getter (CP-1093). Added `ToolExecutionStepLevel` named type and `TOOL_EXECUTIONS_DEFAULT_LIMIT`/`SKIP` constants. Widened `ToolExecutionStep.data` to `Record<string, unknown> \| string` — allows plain string steps alongside structured JSON. Added optional `ToolExecutionStep.payload` for structured drill-down data. |
-| 1.3.22  | `CallsFilterParams.status` typed as `CallStatus` instead of `string` — aligns call-query filters with the Twilio status union from 1.3.17. |
-| 1.3.23  | Merge of 1.3.21 and 1.3.22 — combines the `ToolExecutionStep`/`steps`/`pushToolExecution` work (CP-1093) with the `CallsFilterParams.status: CallStatus` fix. Adds `CallStatusValues` (const object) with `CallStatus` and `ALL_CALL_STATUSES` derived from it. `CallStatus` still does not include `"redirected"` — tracked separately. |
-| 1.3.24  | **Breaking:** removed `flowId` from `WebsiteTalkProduct` — flow source of truth is `phone_numbers.flow_id`. Added `getPrimaryPhoneFlowId(clientId)`. |
-| 1.3.25  | Added `disconnectDb()` to close all MongoDB clients opened by `ensureDbConnected()` — for one-shot scripts that must exit cleanly. |
-| 1.3.28  | Soft-delete (`isActive`) on Streets/DepartmentSubjects, new bulk-write getters, MongoDB type re-exports, and the generic `bulkWrite` op-builder module. |
-| 1.3.29  | Added client call-quota module (CP-1500): `ClientConfig.quota` for usage limits (`totalQuota`, `usedQuota`, `expiresAt`), alert settings on `clients`, and getters for state, increment/reset, threshold alerts, and expiry notification tracking. |
-| 1.3.32  | `clientDisplayName` lives on `clientsConfig.quota` (not top-level). `getClientDisplayName` returns `"-"` when unset; never exposes `clientId`. |
-| 1.3.33  | Added `configuration` database module (CP-1560): `configurationMongodbClient` + a read-only `Prompts`-style collection for prompts previously hardcoded in consuming repos (e.g. MIS). |
-| 1.3.35  | Fixed the configuration collection name (`Prompts` → `prompts`, matches the lowercase convention used elsewhere). **Breaking** if you inserted documents into a collection literally named `Prompts`. Added `Prompt.productName` (required) + `getPromptsByProduct`. Centralized configuration-DB env var reads into `src/config.ts`. |
-| 1.3.36  | Publish combining the 1.3.33/1.3.35 configuration-DB work with the latest `main` (client quota + `clientDisplayName` + bulk-write/streets/departmentSubjects updates — see 1.3.28/1.3.29/1.3.32). No new code of its own. |
-| 1.3.37  | Added `models` and `internalModels` collections under `configuration` (CP-1624): realtime/stt model catalog (`ModelDoc`, `getModelByModelId`, `getModelsByProvider`, `getModelsByType`, `getAvailableModelsByType`) and a name-keyed settings store (`InternalModel`, `getInternalModelByName`) for cross-service config previously hardcoded in consuming repos (e.g. TalkPilot Server's realtime/STT catalogs and its live transcription model). Read-only, same as `prompts` — documents are seeded/managed manually. |
-| 1.3.38  | Added optional `transcriptionComparison` field to `Call` (CP-1594): per-`{provider}-{model}` side-channel transcription comparison results, same segment shape as `Call.transcription`. |
-
-### 1.1.9
-
-1. `npm run build`.
-2. `npm run test`.
-3. `npm run format`.
-
-### 1.3.4 — Call & ticket statistics (CP-149)
-
-New call statistics getters for dashboards (summary, trend, hourly, routing) and ticket statistics scoped to the same date range (open tickets, draft tickets, subject breakdowns).
-
-**Removed**
-
-| Removed                             | Replacement                                                                                                     |
-|-------------------------------------|-----------------------------------------------------------------------------------------------------------------|
-| `getTicketsCountByCityAndDateRange` | `aggregateCallsSummary` + `findCallSidTicketCountsByCity` — note: returns `Map<string, number>`, not `string[]` |
-| `getTicketsSubjectStats`            | `findSubjectsByCityAndDateRange`                                                                                |
-| `SubjectStatsItem`                  | `SubjectItem`                                                                                                   |
-
-**Do I need to update my app?**
-
-Yes — if you used any of the removed functions above.
-
-### 1.3.5 — Muni issues (CP-1115)
-
-New `muniIssues` module under `src/municipal/` for issues reported from the call-log screen. The document is written to the municipal DB after its Jira ticket is created.
-
-**Added**
-
-| Export                                       | Purpose                                                                                                                                                 |
-|----------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------|
-| `MuniIssue` / `CreateMuniIssueInput`         | Document type and insert-input type (`issueContent` + `jira` domains).                                                                                  |
-| `createMuniIssue(input, id)`                 | Insert an issue; requires a caller-supplied `_id` from MIS (via `generateMuniIssueId`) so it matches the id embedded in the Jira ticket (two-way link). |
-| `generateMuniIssueId()`                      | Pre-generate the `_id` to pass to `createMuniIssue`.                                                                                                    |
-| `getMuniIssueById` / `getMuniIssueByJiraKey` | Lookups by our id or by the Jira key.                                                                                                                   |
-| `ensureMuniIssuesCollection()`               | Provision the `$jsonSchema` validator and indexes (unique `jira.key`); run once at MIS boot.                                                            |
-
-**Do I need to update my app?**
-
-No — purely additive. New consumers (MIS) import these from `@talkpilot/core-db`.
-
-### 1.3.6 — Context Notes & Products
-
-**Context Notes** — new `contextNotes` collection under `src/talkpilot/` for per-client, per-product notes injected into AI calls at runtime. Each document holds a `systemPrompt` and a list of time-bounded entries (`activeFrom`, `expiresAt`). Two setters with split ownership: `setContextNoteEntries` for the notes list and `setContextNoteConfig` for `systemPrompt` and `product`.
-
-**Products** — new `products` collection under `src/talkpilot/` cataloguing TalkPilot products. Each document has a stable `name` and an optional `displayName` locale map (`Record<string, string>`) for UI display.
-
-**Added**
-
-| Export                                       | Purpose                                                         |
-|----------------------------------------------|-----------------------------------------------------------------|
-| `getContextNotes(clientId, product)`         | Fetch context note documents for a client/product pair.         |
-| `createContextNote(input)`                   | Insert a new context note document.                             |
-| `setContextNoteEntries(id, clientId, notes)` | Replace the notes list on an existing document.                 |
-| `setContextNoteConfig(id, clientId, config)` | Update `systemPrompt` and/or `product` on an existing document. |
-| `getAllProducts()`                           | Fetch the full product catalogue.                               |
-
-**Do I need to update my app?**
-
-No — purely additive.
-
-### 1.3.7 — Generic prompt control (CP-704)
-
-Added an optional boolean field to the `Flow` type and MongoDB schema.
-
-**Added**
-
-| Export                      | Purpose                                                                                                                                  |
-|-----------------------------|------------------------------------------------------------------------------------------------------------------------------------------|
-| `Flow.disableGenericPrompt` | Optional `boolean`. When `true`, disables the generic prompt for the flow. Existing flows without this field continue to work unchanged. |
-
-**Do I need to update my app?**
-
-No — the field is optional and fully backwards-compatible.
-
-### 1.3.8 — Ticket-count Map getters (CP-1200)
-
-Adds `Map<string, number>` ticket getters that return per-call ticket counts. The old `string[]` getters and filter fields are kept and marked `@deprecated`.
-
-**Deprecated → Replacement**
-
-| Deprecated                                  | Replacement                                                  |
-|---------------------------------------------|--------------------------------------------------------------|
-| `findCallSidsWithTicketsByCity`             | `findCallSidTicketCountsByCity` → `Map<string, number>`      |
-| `findCallSidsWithDraftTicketsByCity`        | `findCallSidDraftTicketCountsByCity` → `Map<string, number>` |
-| `CallsStatsFilter.callSidsWithTickets`      | `CallsStatsFilter.callSidTicketCounts`                       |
-| `CallsStatsFilter.callSidsWithDraftTickets` | `CallsStatsFilter.callSidDraftTicketCounts`                  |
-
-**Do I need to update my app?**
-
-No — the old fields still compile. Migrate when convenient.
-
-### 1.3.9 — Wrap-around hour filter boundary guards (CP-1166)
-
-When `hourFrom` > `hourTo` (e.g. 23:00–13:00), two silent miscounting bugs existed:
-
-- **First-day morning**: calls before `hourTo` on the first day (e.g. 08:00 on Jun 8) were counted even though the window starts at `hourFrom` that day. Fixed by adding `dateLocal > startStr` to the after-midnight leg.
-- **Last-day evening**: calls at or after `hourFrom` on the last day (e.g. 23:30 on Jun 9) were counted even though the window ends at `hourTo` that day. Fixed by adding `dateLocal < endStr` to the before-midnight leg.
-
-**Affects:** `aggregateCallsTrend`, `aggregateCallsSummary`, `aggregateCallsHourlyByRange`, `aggregateCallsRouting`, `findFilteredCallSids`, `findSubjectsByCityAndDateRange`.
-
-**Do I need to update my app?**
-
-No API changes — both fixes correct silent miscounting. Upgrading is recommended if you use a wrap-around hour window.
-
-### 1.3.10 — Ticket draft flag (CP-1390)
-
-Added an optional `isDraft` boolean field to the `Ticket` type.
-
-**Added**
-
-| Export           | Purpose                                                                                                                                                                                 |
-|------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| `Ticket.isDraft` | Optional `boolean`. When `true`, the ticket is a draft. When `false`, it is not. When absent (legacy tickets), draft detection falls back to the previous `event_subject_id` heuristic. |
-
-**Affects:** `findCallSidDraftTicketCountsByCity`, `findCallSidsWithDraftTicketsByCity` (draft filtering in ticket statistics).
-
-**Do I need to update my app?**
-
-No — purely additive. Existing tickets without `isDraft` continue to work unchanged. New consumers (MIS) can pass `isDraft: true` when creating a draft ticket.
-
-### 1.3.11 — Fix clientsConfig type declaration import
-
-Fixed a pre-existing broken import path in the published `clientsConfig.types.d.ts` (introduced in an earlier release, unrelated to 1.3.10). Consumer TypeScript builds could fail with:
-
-`Cannot find module 'src/utils/shared.types' or its corresponding type declarations.`
-
-The import now uses a relative path (`../../utils/shared.types`) so `tsc` resolves it correctly from `node_modules/@talkpilot/core-db/dist/`.
-
-**Do I need to update my app?**
-
-Yes — upgrade to 1.3.11 if your build failed with the error above (affects any version that shipped the broken declaration). No API or behavior changes otherwise.
-
-### 1.3.12 — Calls-with-tickets count (CP-1200)
-
-Added `callsWithTickets` to `CallsSummaryAggregation` — the number of distinct calls that opened at least one ticket. Use this field (instead of `openTickets`) to compute call-level rates and "calls without a ticket" counts.
-
-**Do I need to update my app?**
-
-Yes — replace `openTickets` with `callsWithTickets` wherever you compute `ticketOpenRate` or `not_opened`.
-
-### 1.3.13 — Faulty release
-
-This release incorrectly placed `useRedisTranscription` on `Flow` instead of `FlowTool`. It is redundant and should be skipped.
-
-### 1.3.14 — Redis transcription flag
-
-Added an optional `FlowTool.useRedisTranscription` boolean that tells consumers whether to read transcriptions from Redis. Defaults to `false` when omitted.
-
-**Do I need to update my app?**
-
-No — this flag is optional. Only update if you want to opt into the Redis transcription path.
-
-### 1.3.15 — Multi-scan configuration (CP-1372)
-
-Adds `websiteUrls` collection (CRUD getters), migrates scans from `baseUrl` to `websiteUrlId`, adds `OVERWRITTEN` scan status, and replaces `defaultBaseUrl`/`logoUrl` on `WebsiteTalkProduct` with `activeScanId` (+ getters).
-
-**Breaking**
-
-| Removed / changed | Replacement |
-| --- | --- |
-| `Scan.baseUrl` | `Scan.websiteUrlId` |
-| `WebsiteTalkProduct.defaultBaseUrl` | `websiteUrls` collection |
-| `getActiveScanStatus(clientId)` | `getInProgressScanByWebsiteUrl`, `findInProgressScansByClient` |
-
-**Do I need to update my app?**
-
-Yes — WTIS, CoreBrain, and CoreCrawler must adopt `websiteUrlId` and the new getters before upgrading.
-
-### 1.3.16 — Scan status constants refactor (CP-1372)
-
-Centralizes scan status values under a single `SCAN_STATUSES` object map and derives terminal statuses from it.
-
-**Changed**
-
-| Area | What |
-| --- | --- |
-| `SCAN_STATUSES` | Object map (`SCAN_STATUSES.CHECKING`, `SCAN_STATUSES.COMPLETED`, …) instead of separate exported string constants. |
-| `FINISHED_SCAN_STATUSES` | Derived from `SCAN_STATUSES` (`COMPLETED`, `FAILED`, `OVERWRITTEN`). |
-| `isInProgressScan` | Moved to `scans.utils.ts` (exported from `scans` index). |
-| `ScanStatus` type | Derived from `(typeof SCAN_STATUSES)[keyof typeof SCAN_STATUSES]`. |
-
-**Do I need to update my app?**
-
-No — behavior is unchanged. If you imported removed aliases like `COMPLETED_SCAN_STATUS`, switch to `SCAN_STATUSES.COMPLETED` (or keep using string literals / `ScanStatus` type).
-
-### 1.3.17 — Twilio call-status semantics (CP-1521)
-
-Aligns dashboard KPIs and call-status updates with Twilio semantics so in-call counts update in real time.
-
-**Added**
-
-| Export | Purpose |
-| --- | --- |
-| `updateCallStatusByCallSid(callSid, status)` | Set the current call status by `callSid`. |
-| `IN_CALL_STATUSES` | `["in-progress", "answered"]` — statuses counted as in-call. |
-| `DashboardSummaryMetrics.inCallCount` | Replaces `busyCount`; counts in-progress + answered calls. |
-
-**Changed**
-
-| Area | What |
-| --- | --- |
-| Dashboard KPIs | `busyCount` → `inCallCount`; `busy` is line-busy only, not in-call. |
-| `timeSavedMinutes` | Computed from `completed` calls only. |
-| `CallStatus` union | Expanded for Twilio-aligned statuses. |
-
-**Do I need to update my app?**
-
-Yes — if you read `busyCount` from dashboard KPIs, switch to `inCallCount`.
-
-### 1.3.18 — WebsiteTalk multi-website API (CP-1372)
-
-Ships the full persistence API for multi-website / multi-scan. Versions 1.3.15–1.3.16 documented the design; **1.3.17 did not export these symbols** (CP-1372 merged into main after the 1.3.17 release). Upgrade to **1.3.18** if you see missing-export errors such as `getActiveScanId`, `getWebsiteUrlById`, or `WebsiteUrlDoc`.
-
-**`websiteUrls` module**
-
-| Export | Purpose |
-| --- | --- |
-| `WebsiteUrl` / `WebsiteUrlDoc` / `CreateWebsiteUrlInput` | Document types. |
-| `getWebsiteUrlsCollection()` | Collection access. |
-| `getWebsiteUrlsByClient(clientId)` | All websites for a client. |
-| `getWebsiteUrlById(id)` | Lookup by id. |
-| `findWebsiteUrls(filter?)` | Flexible query. |
-| `createWebsiteUrl({ clientId, name, baseUrl })` | Create. |
-| `updateWebsiteUrl(id, updates)` | Update `name` / `baseUrl`. |
-| `deleteWebsiteUrl(id)` | Delete by id. |
-
-**`clientsConfig` — selected scan**
-
-| Export | Purpose |
-| --- | --- |
-| `getActiveScanId(clientId)` | Read `products.websiteTalk.activeScanId`. |
-| `updateActiveScanId(clientId, scanId)` | Set the active scan for search. |
-| `clearActiveScanId(clientId)` | Unset `activeScanId`. |
-
-**`scans` — by `websiteUrlId`**
-
-| Export | Purpose |
-| --- | --- |
-| `getScanById(id)` | Lookup by id. |
-| `getScansByWebsiteUrl(websiteUrlId)` | All scans for a website. |
-| `getLatestScanByWebsiteUrl(websiteUrlId)` | Most recent scan. |
-| `getInProgressScanByWebsiteUrl(websiteUrlId)` | Latest scan if still in progress. |
-| `findInProgressScansByClient(clientId)` | All in-progress scans for a client. |
-| `hasScansForWebsiteUrl(websiteUrlId)` | Whether any scans exist. |
-| `getLatestCompletedScanByWebsiteUrl(websiteUrlId)` | Most recent `COMPLETED` scan. |
-| `updateScanDoc(scanId, updates)` | Partial update (e.g. mark `OVERWRITTEN`). |
-| `deleteScansByWebsiteUrl(websiteUrlId)` | Delete all scans for a website. |
-
-**Breaking / type changes**
-
-| Removed | Replacement |
-| --- | --- |
-| `WebsiteTalkProduct.defaultBaseUrl` | `websiteUrls` collection (`name`, `baseUrl`) |
-| `Scan.baseUrl` | `Scan.websiteUrlId` (see 1.3.15) |
-
-**`OVERWRITTEN` scan status**
-
-- `OVERWRITTEN` is part of `ScanStatus` / `SCAN_STATUSES`.
-- Marking previous `COMPLETED` scans as overwritten is **not** done in `createScanDoc` — implement in the service layer (e.g. via `updateScanDoc`).
-
-**Not in core-db (service layer)**
-
-- `resolveActiveWebsiteContext` — join `activeScanId` → scan → websiteUrl.
-- Migration from legacy `defaultBaseUrl`.
-
-**Do I need to update my app?**
-
-Yes — WTIS, CoreBrain, and CoreCrawler should upgrade to **1.3.18** and import the symbols above from `@talkpilot/core-db`. Stop reading `defaultBaseUrl` from `WebsiteTalkProduct`; use `websiteUrls` instead.
-
-### 1.3.19 — WebsiteTalk flow provisioning (WTIS activate-scan)
-
-Persists the TalkPilot flow id on the client product config and links the client's primary phone number to that flow so inbound calls use the WebsiteTalk flow after `activate-scan`.
-
-**`clientsConfig` — `WebsiteTalkProduct`**
-
-| Field | Purpose |
-| --- | --- |
-| `WebsiteTalkProduct.flowId?` | Optional TalkPilot flow `_id` (hex string). Set by WTIS after `POST /flows/add_new_flow`; reused on later activations via `PUT /flows/edit_flow/:id`. |
-
-**`phone_numbers`**
-
-| Export | Purpose |
-| --- | --- |
-| `bindClientPhoneToFlow(clientId, flowId)` | Updates the **primary** `phone_numbers` row for `clientId`, setting `flow_id` to the given TalkPilot flow. Throws if no primary phone exists. |
-
-**Typical service-layer flow (WTIS)**
-
-1. Build flow payload from template + `companyInfo` (generated at activate time).
-2. Create or update the flow via TalkPilot Server API (`add_new_flow` / `edit_flow`).
-3. On first create: `updateProductConfig(clientId, "websiteTalk", { flowId })`.
-4. `bindClientPhoneToFlow(clientId, flowId)` — ensure the phone routes to the flow.
-5. `updateActiveScanId(clientId, scanId)` — point search at the activated scan.
-
-**Not in core-db (service layer)**
-
-- Flow template substitution, Claude `companyInfo` generation, Firebase Storage template download, TalkPilot flow create/update HTTP calls.
-
-**Do I need to update my app?**
-
-No — purely additive. **WTIS** should upgrade to **1.3.19** (or later) and import `bindClientPhoneToFlow` plus `WebsiteTalkProduct.flowId` when implementing activate-scan flow provisioning.
-
-### 1.3.20 — `FINAL_STATUSES` call-status constant
-
-Centralizes terminal Twilio call statuses so consumers (e.g. TalkPilot Server `callResultService`) don't duplicate the list locally.
-
-**Added**
-
-| Export | Purpose |
-| --- | --- |
-| `FINAL_STATUSES` | `["no-answer", "completed", "busy", "failed", "canceled"]` — statuses where no further Twilio callbacks are expected. Pair with `IN_CALL_STATUSES` for live vs terminal semantics. |
-
-**Do I need to update my app?**
-
-No — purely additive. Replace local `FINAL_STATUSES` arrays with `import { FINAL_STATUSES } from "@talkpilot/core-db"` when convenient.
-
-### 1.3.21 — Tool execution steps & push getter (CP-1093)
-
-Added `ToolExecutionStep` type, `steps` field on `ToolExecution`, `pushToolExecution` getter, named `ToolExecutionStepLevel` type, and pagination constants.
-
-**Added**
-
-| Export | Purpose |
-| --- | --- |
-| `ToolExecutionStepLevel` | Named type: `"info" \| "warn" \| "error"`. |
-| `ToolExecution.steps` | Optional `ToolExecutionStep[]`. Ordered log of intermediate steps within a tool call (e.g. `audio_enrichment`, `google_places`, `subject_classification`). Absent on executions recorded before this version. |
-| `ToolExecutionStep.payload` | Optional `Record<string, unknown>`. Structured data for UI drill-down, alongside the human-readable `data` field. |
-| `pushToolExecution(callSid, execution)` | Append a `ToolExecution` to the call's `toolExecutions` array via `$push`. |
-| `getToolExecutionsByCallSid(callSid, opts?)` | Fetch the `toolExecutions` array for a call. `opts.skip` and `opts.limit` (default `TOOL_EXECUTIONS_DEFAULT_LIMIT` = 50) support pagination. Returns `[]` when the call has no executions or does not exist. |
-| `TOOL_EXECUTIONS_DEFAULT_LIMIT` | `50` — default page size for `getToolExecutionsByCallSid`. |
-| `TOOL_EXECUTIONS_DEFAULT_SKIP` | `0` — default skip for `getToolExecutionsByCallSid`. |
-
-**Do I need to update my app?**
-
-No — purely additive. All new fields are optional and existing documents continue to work unchanged.
-
-### 1.3.22 — `CallsFilterParams.status` type alignment
-
-Tightens the `status` filter field on `CallsFilterParams` from `string` to `CallStatus` so call-query consumers get compile-time validation against the Twilio-aligned status union introduced in 1.3.17.
-
-**Changed**
-
-| Area | What |
-| --- | --- |
-| `CallsFilterParams.status` | `string` → `CallStatus` |
-
-**Do I need to update my app?**
-
-Only if you pass a non-`CallStatus` string to `CallsFilterParams.status` — fix the value or cast. No runtime behavior change.
-
-### 1.3.23 — Merge of 1.3.21 and 1.3.22, plus `CallStatusValues`
-
-Combines the CP-1093 tool-execution-steps work (1.3.21) with the `CallsFilterParams.status` type fix (1.3.22); both were published independently from diverging branches.
-
-**Added**
-
-| Export | Purpose |
-| --- | --- |
-| `CallStatusValues` | `as const` object mapping named keys (`Completed`, `Failed`, ...) to the 9 raw Twilio-aligned status strings — gives enum-like dot access (`CallStatusValues.Completed`) while `CallStatus` stays a plain string-literal union (no breaking change for existing `status === "completed"` comparisons). |
-| `CallStatus` | Now derived from `CallStatusValues` (`(typeof CallStatusValues)[keyof typeof CallStatusValues]`) instead of a hand-written union — single source of truth. |
-| `ALL_CALL_STATUSES` | `CallStatus[]` — `Object.values(CallStatusValues)`, for runtime validation of raw input (e.g. HTTP query params) against `CallStatus`. |
-
-**Do I need to update my app?** No — `CallStatus` is structurally identical to before (same 9 string literals); this is a source-only refactor.
-
-`CallStatus` still does not include `"redirected"` — tracked separately.
-
-### 1.3.28 — Soft-delete support and generic bulk-write helpers
-
-Adds an `isActive` soft-delete convention, bulk-write getters for Streets/DepartmentSubjects, MongoDB type re-exports, and a generic bulk-write op-builder module.
-
-**Added**
-
-| Export | Purpose |
-| --- | --- |
-| `Street.isActive` / `DepartmentSubject.isActive` | Optional soft-delete flag; `createStreet` defaults it to `true`. |
-| `bulkWriteStreets` / `bulkWriteDepartmentsSubjects` | Passthrough bulk-write for each collection. |
-| `findDepartmentSubjectsByFilter` | Query subjects by an arbitrary filter. |
-| `updateDepartmentSubjectActive` | Set `isActive` on a subject by `_id`. |
-| `ObjectId`, `Filter`, `BulkWriteOp`, `BulkWriteError`, `OptionalUnlessRequiredId` | Re-exported from `mongodb`. |
-| `buildSetOp`, `buildUpsertOp`, `buildInsertOp`, `buildDeleteOp` | Generic bulk-write op builders — filter/fields in, a shaped op out. |
-
-**Do I need to update my app?**
-
-No — purely additive.
-
-### 1.3.29 — Client call quota (CP-1500)
-
-Per-client call quota with usage counters, optional expiration, and configurable threshold/expiry email alerts. Usage lives on clientsConfig; alert config and send-tracking live on clients. TalkPilot Server consumes these getters to block calls when quota is exceeded or expired and to send notification emails.
-
-**Data model**
-
-| Location | Field | Purpose |
-| --- | --- | --- |
-| `clientsConfig.quota` | `totalQuota`, `usedQuota`, `expiresAt?` | Call allowance and consumption. `getClientQuotaState` returns `null` when `totalQuota` is unset (quota disabled for that client). |
-| `clients.quota` | `alerts?`, `expiredSent?` | Up to 3 threshold alerts (each with up to 3 recipient emails) and a flag set after the expiry email is sent. |
-
-**Added**
-
-| Export | Purpose |
-| --- | --- |
-| `ClientQuotaUsage` | Usage shape: `totalQuota`, `usedQuota`, optional `expiresAt`. |
-| `ClientQuotaSettings` | Alert settings on the `clients` doc: `alerts`, `expiredSent`. |
-| `ClientQuotaState` | Merged view returned by `getClientQuotaState` (usage + settings). |
-| `QuotaThresholdAlert` / `QuotaAlertInput` | Threshold alert with `thresholdPct`, `recipientEmails`, and optional `thresholdSent` send-tracking. |
-| `getClientQuotaState(clientId)` | Read combined quota state; `null` when no quota is configured. |
-| `incrementClientQuotaUsage(clientId, amount?)` | Atomically increment `clientsConfig.quota.usedQuota` (default `amount = 1`). No-op when `quota.totalQuota` is absent. |
-| `resetClientQuotaUsage(clientId)` | Reset `usedQuota` to `0` and clear alert/expiry send flags via `resetClientQuotaSettings`. |
-| `updateQuotaAlertsConfig(clientId, alerts)` | Replace threshold alert config; preserves `thresholdSent` for unchanged thresholds. |
-| `markQuotaAlertSent(clientId, alertIndex)` | Idempotently mark a threshold alert as sent; returns whether the update applied. |
-| `markQuotaExpiredSent(clientId)` | Idempotently mark the expiry notification as sent; returns whether the update applied. |
-| `resetClientQuotaSettings(clientId)` | Clear `expiredSent` and strip `thresholdSent` from alerts (keeps alert definitions). |
-
-**Do I need to update my app?**
-
-No — purely additive. TalkPilot Server should upgrade to 1.3.29 (or later) to enforce quota on inbound/outbound calls and to drive threshold/expiry emails via the new getters.
-
-### 1.3.32 — Quota `clientDisplayName`
-
-Optional human-readable account label stored on `clientsConfig.quota` for quota notification emails. TalkPilot Server uses `getClientDisplayName` instead of exposing raw `clientId` in alert/expiry email copy.
-
-**Changed**
-
-| Area | What |
-| --- | --- |
-| `ClientQuotaUsage.clientDisplayName?` | Optional display name on `clientsConfig.quota` — e.g. `"Acme Corp"`. Included in `ClientQuotaState` via `getClientQuotaState`. |
-
-**Added**
-
-| Export | Purpose |
-| --- | --- |
-| `getClientDisplayName(clientId)` | Returns trimmed `quota.clientDisplayName`, or `"-"` when unset, missing, or blank. Never falls back to `clientId`. |
-
-**Do I need to update my app?**
-
-No — purely additive. Set `clientsConfig.quota.clientDisplayName` when provisioning quota for a client. **TalkPilot Server** should upgrade to **1.3.32** (or later) to use the label in quota emails.
-
-### 1.3.33 — Configuration database & Prompts collection (CP-1560)
-
-New `configuration` database with a prompts collection, so prompts previously hardcoded in consuming repos (e.g. MIS's `moked_106` street/subject search and transcription prompts) can be stored centrally and edited without a deploy. Ships the **read** path only — documents are seeded/managed manually (e.g. via Compass), not through this package.
-
-**Added**
-
-| Export | Purpose |
-| --- | --- |
-| `configurationMongodbClient` | MongoDB client for the `configuration` database (default name `configuration`, override with `CONFIGURATION_DB_NAME`). |
-| `getConfigurationDb` / `setConfigurationDb` / `ConfigurationObjectId` | DB handle accessors and `ObjectId`, matching the other domains. |
-| `Prompt` | Document type: `{ _id, name, content, createdAt, updatedAt }`, keyed by unique `name`. |
-| `getPromptByName(name)` | Fetch a single prompt by its unique `name`. Returns `null` if absent. |
-| `findPrompts(filter?)` | Flexible query over the collection. |
-| `getAllPrompts()` | Fetch every prompt. |
-| `getPromptsCollection()` | Raw collection access. |
-
-`ensureDbConnected()` / `disconnectDb()` now also connect/disconnect the configuration client alongside talkpilot/municipal/websitalk.
-
-**Do I need to update my app?**
-
-No — purely additive. Consumers opt in by importing the getters above.
-
-### 1.3.35 — Prompts fixes: collection name, `productName`, centralized config
-
-**Breaking**
-
-| Changed | What |
-| --- | --- |
-| Collection name | `"Prompts"` → `"prompts"` (lowercase, matching every other collection in core-db). MongoDB collection names are case-sensitive — if you inserted documents into a collection literally named `Prompts` under 1.3.33, move them to `prompts`. |
-
-**Added**
-
-| Export | Purpose |
-| --- | --- |
-| `Prompt.productName` | Required `KnownProductKey` (`"municipal" \| "clinics" \| "websiteTalk"`). Existing documents need this field backfilled. |
-| `getPromptsByProduct(productName)` | Fetch all prompts for a given product. |
-
-**Internal**
-
-- Configuration-DB env var reads (`MONGO_URI`, `MONGODB_URI`, `CONFIGURATION_DB_NAME`) centralized into `src/config.ts` (`configurationDbConfig`) — no consumer-facing change.
-
-**Do I need to update my app?**
-
-Only if you inserted documents into the old `Prompts` (capital P) collection — move them to `prompts`. Also backfill `productName` on any existing prompt documents (e.g. `"municipal"`), since it's now a required field on the type.
-
-### 1.3.36 — Merge with `main`
-
-Publishes the 1.3.33/1.3.35 configuration-DB work alongside the latest `main` — client call-quota, `clientDisplayName`, and bulk-write/streets/departmentSubjects updates (see 1.3.28, 1.3.29, 1.3.32). No new code of its own; this version exists to ship both branches' work together.
-
-**Do I need to update my app?**
-
-No — purely additive, same as the versions it combines.
-
-### 1.3.37 — Model catalog & internal model settings (CP-1624)
-
-New `models` and `internalModels` collections under the `configuration` database, so AI model catalogs and cross-service settings previously hardcoded in consuming repos (e.g. TalkPilot Server's realtime/STT model lists and its live transcription model) can be stored centrally and edited without a deploy. Ships the **read** path only, same as `prompts` — documents are seeded/managed manually (e.g. via Compass).
-
-**Added — `models`**
-
-| Export | Purpose |
-| --- | --- |
-| `ModelDoc` (`RealtimeModelDoc \| SttModelDoc`) | Document type discriminated by `type`: `{ _id, modelId, name, description?, available, features?, configDefaults?, createdAt, updatedAt }`, plus `provider` (`RealtimeModelProvider` for `type: "realtime"`, `SttModelProvider` for `type: "stt"`). |
-| `MODEL_TYPES`, `REALTIME_MODEL_PROVIDERS`, `STT_MODEL_PROVIDERS` | Known `type`/`provider` value lists the types above are derived from. |
-| `getModelByModelId(modelId, type?)` | Fetch a single model by its `modelId`, optionally scoped to a `type`. |
-| `getModelsByProvider(provider, type?)` | Fetch all models for a given `provider`, optionally scoped to a `type`. |
-| `getModelsByType(type)` | Fetch all models of a given `type`. |
-| `getAvailableModelsByType(type)` | Same as above, pre-filtered to `available: true`. |
-| `findModels(filter?)` | Flexible query over the collection. |
-| `getModelsCollection()` | Raw collection access. |
-
-**Added — `internalModels`**
-
-| Export | Purpose |
-| --- | --- |
-| `InternalModel` | Document type: `{ _id, name, provider, model, config?, createdAt, updatedAt }`, keyed by unique `name` — e.g. a `"transcriptionModel"` document. |
-| `getInternalModelByName(name)` | Fetch a single internal model config by its unique `name`. Returns `null` if absent. |
-| `findInternalModels(filter?)` | Flexible query over the collection. |
-| `getAllInternalModels()` | Fetch every internal model. |
-| `getInternalModelsCollection()` | Raw collection access. |
-
-`ensureDbConnected()` / `disconnectDb()` already covered the configuration client from 1.3.33 — no change there.
-
-**Do I need to update my app?**
-
-No — purely additive. Consumers opt in by importing the getters above.
-
-### 1.3.38 — Transcription comparison field (CP-1594)
-
-Added an optional `transcriptionComparison` field to `Call`, for consumers that run side-channel transcription comparisons across multiple providers/models alongside the real `transcription` field.
-
-**Added**
-
-| Export | Purpose |
-| --- | --- |
-| `Call.transcriptionComparison` | Optional `Record<string, TranscriptionSegment[]>` — one entry per `{provider}-{model}` key, each an array of `TranscriptionSegment` in the same shape as `Call.transcription`. |
-
-**Do I need to update my app?**
-
-No — purely additive. Existing documents and consumers continue to work unchanged.
-
-## 🛠 CI/CD & deployment
-
-- This package is consumed by Cloud Build-based services (TalkPilot Server, MIS, CIS) as a dependency when Docker images are built. Keep `dist/` in sync with your builds because the compiled artifact is what downstream services install.
-- Releases require the shared npm token documented in `DEVELOPMENT.md`; consult that guide for contribution, linking, and token rotation procedures.
-
-## Resources
-
-- [DEVELOPMENT.md](./DEVELOPMENT.md) (setup, tooling, publishing, token management)
-- [src/test-utils](src/test-utils) and the `__tests__` folder for examples of `MongoMemoryServer` wiring.
-
+L1:# @talkpilot/core-db
+
+[NPM Version](https://www.npmjs.com/package/@talkpilot/core-db)
+`@talkpilot/core-db` is the shared TypeScript database package that wires TalkPilot APIs, municipal CRM integrations, and internal tools to a single, type-safe MongoDB surface. Every repo (MIS, CIS, TalkPilot Server, etc.) imports this package to avoid re-implementing connections, collections, or validation helpers.
+
+## Purpose
+
+- Provide a reliable, multi-domain MongoDB layer for TalkPilot and municipal data.
+- Export typed getters, vector search helpers, and document factories so services can focus on behavior instead of schema wiring.
+- Manage connection lifecycles, environment configuration, and test helpers from one place so every repo reuses the same plumbing.
+
+## Main Concepts
+
+- **Multi-domain clients** – `src/connection.ts` exposes `mongodbClient` (TalkPilot) and `municipalDataMongodbClient`, each of which resolves `MONGO_URI`, DB overrides, and default names.
+- **Domain-specific getters** – `src/talkpilot/` and `src/municipal/` host typed getters (agents, calls, streets, tickets, etc.), vector-search helpers, and service-friendly adapters that keep caller code DRY.
+- **Product-specific clients (future)** – While the package currently exposes the shared TalkPilot + municipal clients, we expect each product (CIS, MIS, TalkPilot Server) to eventually get its own domain-specific client helpers or wrappers so the shared core can remain stable while new consumers add targeted extensions.
+- **Test helpers** – `src/test-utils/` plus `src/__tests__/` reuse `MongoMemoryServer` and shared factories so tests start with clean data regardless of the consuming repo.
+- **Utility layers** – `src/utils/` contains shared validation, pagination, and environment helpers that complement the getters.
+- **Bulk-write helpers** – `src/bulkWrite/` exposes small, domain-agnostic functions (`buildSetOp`, `buildUpsertOp`, `buildInsertOp`, `buildDeleteOp`) that construct correctly-shaped MongoDB bulk-write operations from a filter and field values — no collection- or domain-specific knowledge required.
+- **Configuration domain** – `src/configuration/` hosts the `configuration` database (a `prompts` collection so far) — a place for cross-service settings that used to be hardcoded in consuming repos, editable without a deploy.
+- **Environment awareness** – Defaults, fallbacks, and `process.env` lookups ensure local, CI, and Cloud Run clients all connect using the right URI/DB names.
+
+## Key Components
+
+- `src/connection.ts` – Central connection logic that resolves URIs/DB names from env vars (`MONGO_URI`, `MONGODB_URI`, `TALKPILOT_DB_NAME`, `MUNICIPAL_DB_NAME`) and reuses a single `MongoClient`.
+- `src/talkpilot/` – Call history, agents, flows, sessions, leads, subscriptions, and support helpers exposed as getters plus helper enums/types for each collection.
+- `src/municipal/` – Municipal-specific collections (`cities`, `streets`, `departmentsSubjects`, `tickets`, etc.) plus vector search helpers and Ash Bina helpers used by MIS.
+- `src/utils/` – Shared helpers such as `resolveConnection`, pagination utilities, and schema validation helper functions.
+- `src/bulkWrite/` – Generic bulk-write op constructors (`buildSetOp`, `buildUpsertOp`, `buildInsertOp`, `buildDeleteOp`) usable with any collection's document type.
+- `src/configuration/` – `configurationMongodbClient` plus the `prompts` collection (`Prompt` type, `getPromptByName`, `findPrompts`, `getAllPrompts`, `getPromptsByProduct`).
+- `src/config.ts` – Centralized `process.env` reads for the configuration domain — add new env-backed fields here instead of reading `process.env` inline elsewhere.
+- `src/test-utils/` and `src/__tests__/` – Utilities that bootstrap `MongoMemoryServer`, expose factories, and make sure Jest environments can stub database calls predictably.
+- `dist/` – Compiled output consumed by downstream repos (CJS + ESM + type defs).
+
+## Domain APIs
+
+- **TalkPilot domain** – Imports like `findAgents`, `getFlows`, `findCalls`, and `vectorSearchCalls` live in `src/talkpilot`. These functions are the canonical access pattern for call history, session metadata, and provider configs.
+- **Municipal domain** – Helpers such as `findStreets`, `getMunicipalCities`, `findDepartmentSubjects`, and `createTicket` live under `src/municipal` and feed MIS workflows (street hints, subject matching, Ash Bina tickets).
+- **Configuration domain** – `getPromptByName`, `findPrompts`, `getAllPrompts`, and `getPromptsByProduct` live under `src/configuration` and back MIS's `moked_106` prompt lookups (defaults, overridable per-client via `clientConfig.toolsPrompts`).
+
+## Environment variables
+
+| Variable            | Purpose                                                                           | Required |
+|---------------------|-----------------------------------------------------------------------------------|----------|
+| `MONGO_URI`         | Primary MongoDB connection string for every domain (overridden by `MONGODB_URI`). | ✅        |
+| `MONGODB_URI`       | Alternate connection string used when Mongo needs a second URI parameter.         | ✅        |
+| `TALKPILOT_DB_NAME` | Optional override for the TalkPilot database name (defaults from URI path).       | ❌        |
+| `MUNICIPAL_DB_NAME` | Optional override for the municipal database name (defaults to `municipal-data`). | ❌        |
+| `CONFIGURATION_DB_NAME` | Optional override for the configuration database name (defaults to `configuration`). | ❌   |
+| `ENV`               | Free-form label used in logs/validators (defaults to `unknown`).                  | ❌        |
+
+If you pass a `uri` directly to `mongodbClient.connect()` or `municipalDataMongodbClient.connect()`, the client will prefer that value over the env vars.
+
+## Getting Started
+
+### Prerequisites
+
+- Node.js 22.x+ (aligns with downstream services).
+- npm 11+ or Yarn.
+- MongoDB accessible from your environment or a `MongoMemoryServer` for tests.
+
+### Setup
+
+1. Clone the repo and install dependencies:
+
+   ```bash
+   git clone https://github.com/talkpilot/core-db.git
+   cd core-db
+   npm install
+   ```
+
+2. Build the package before using it locally:
+
+   ```bash
+   npm run build
+   ```
+
+3. Import `@talkpilot/core-db` from another project by pointing `package.json` at the local path during development or installing the published release.
+
+## Sample `.env`
+
+```
+MONGO_URI=mongodb://localhost:27017
+TALKPILOT_DB_NAME=talkpilot-dev
+MUNICIPAL_DB_NAME=municipal-dev
+CONFIGURATION_DB_NAME=configuration-dev
+ENV=development
+```
+
+Adjust `MONGO_URI` to match the running Mongo instance and configure `talkpilot`/`municipal`/`configuration` DB names if you want to keep them separate.
+
+## Local development
+
+1. Run `npm install`.
+2. Build the compiled output: `npm run build`.
+3. Execute tests: `npm run test`.
+4. Use `npm link` or `npm pack` to consume the freshly built package from other repos (`CIS`, `MIS`, `TalkPilot Server`).
+
+## Development guide
+
+`DEVELOPMENT.md` contains the tactical steps for contributors. At a glance:
+
+- Node 18+/TypeScript is required (aligns with downstream services).
+- Run `npm install` → `npm run build` after cloning.
+- Use `npm link`/`npm link @talkpilot/core-db` to test the package locally before publishing.
+- When adding getters, define types, implement the function, export it through the domain `index.ts`, and add a corresponding test under the domain’s `__tests__` folder.
+- Always rely on the provided test factories (`src/test-utils/factories`) to seed data so tests remain consistent.
+- Jest with `mongodb-memory-server` is the only execution path we have to verify this core utility—unit tests are the safety net for every change.
+
+Refer to `DEVELOPMENT.md` for the full walkthrough, token instructions, and factory samples.
+
+## 🧪 Testing
+
+- `npm run test` – Jest suite (factories, utils, integration mocks) powered by `mongodb-memory-server`.
+- Tests rely on `src/__tests__/setup.ts` to bootstrap the in-memory Mongo instances and wire shared factories/helpers before each run.
+- When adding getters, helpers, or domain logic, create focused coverage inside the consuming domain’s `__tests__/` folder and use the provided factories to keep fixtures consistent.
+
+`@talkpilot/core-db` does not run in a product UI or feature branch—unit tests are the *only* reliable execution path to ensure your changes work. Every change must ship with a unit test so downstream repos can upgrade without surprises; treat the test suite as the canonical safety net for this core utility package.
+
+## 🧹 Lint & build verification
+
+- `npm run lint` – Run ESLint over `src/**/*.{ts,tsx}`.
+- `npm run format` – Format the source files with Prettier.
+- `npm run build` – Compile TypeScript and emit `dist/` (used by downstream consumers).
+
+## ✅ Pre-push checklist
+
+1. `npm run build`.
+2. `npm run test`.
+3. `npm run format`.
+
+## Publishing & release notes
+
+- Releases are handled by `npm version <patch|minor|major>` followed by `npm publish`. The package is a **private `@talkpilot` dependency**, so every contributor must install the shared npm automation token into their global `~/.npmrc` before running publish or `npm install`.
+- The shared token is rotated periodically—if you see authentication failures, request the refreshed token, update your `~/.npmrc`, and retry. Never commit credentials to source control.
+- After publishing, downstream repos (`CIS`, `MIS`, `TalkPilot Server`, etc.) should run `npm update @talkpilot/core-db` so they receive the latest helpers/bug fixes.
+- Cloud Build & Cloud Run jobs that depend on this package pick up the new version the next time they rebuild their container; the services pull the compiled `dist/` output and type definitions when installing the dependency.
+
+- Release process, when to publish, branch hygiene, and the function-signature versioning policy: see [`DEVELOPMENT.md`](./DEVELOPMENT.md).
+
+### Version history
+
+| Version | Note                                                                                                                                                                                                                                                                                                                                                         |
+|---------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| 1.1.9   | Maintenance release (build, test, format).                                                                                                                                                                                                                                                                                                                   |
+| 1.3.4   | Added new statistics getters for call & ticket dashboards (CP-149): summary, trend, hourly, and routing for calls, plus open/draft/subject ticket stats. **Removed** the legacy getters `getTicketsCountByCityAndDateRange` and `getTicketsSubjectStats`.                                                                                                    |
+| 1.3.5   | Added the `muniIssues` collection & module (CP-1115): the `MuniIssue` document type, `createMuniIssue` (accepts a caller-supplied `_id` for a two-way Jira link), `generateMuniIssueId`, `getMuniIssueById` / `getMuniIssueByJiraKey`, a `$jsonSchema` validator with indexes (unique `jira.key`), and `ensureMuniIssuesCollection` to provision it at boot. |
+| 1.3.6   | Added `contextNotes` and `products` modules under `src/talkpilot/`.                                                                                                                                                                                                                                                                                          |
+| 1.3.7   | Added `disableGenericPrompt` optional boolean field to the `Flow` type and schema (CP-704).                                                                                                                                                                                                                                                                  |
+| 1.3.8   | Ticket-count Map getters (CP-1200): deprecated `string[]` ticket getters and `CallsStatsFilter` fields, added `Map<string, number>` replacements; fixed ticket counting in `aggregateCallsSummary`.                                                                                                                                                          |
+| 1.3.9   | Fixed wrap-around hour filter boundary guards (CP-1166): excluded first-day morning calls and last-day evening calls from wrap-around windows.                                                                                                                                                                                                               |
+| 1.3.10  | Added optional `isDraft` flag to `Ticket` (CP-1390): explicit draft marking with legacy fallback for existing tickets.                                                                                                                                                                                                                                       |
+| 1.3.11  | Fixed pre-existing broken type declaration import in `clientsConfig.types.d.ts` that caused consumer `tsc` builds to fail.                                                                                                                                                                                                                                   |
+| 1.3.12  | Added `callsWithTickets` to `CallsSummaryAggregation` (CP-1200): distinct call count with at least one open ticket, enabling accurate `ticketOpenRate` and `not_opened` calculations in consumers.                                                                                                                                                           |
+| 1.3.13  | Faulty release (redundant). Published with `Flow.useRedisTranscription` at the flow level; superseded by 1.3.14.                                                                                                                                                                                                                                            |
+| 1.3.14  | Added `FlowTool.useRedisTranscription` optional flag to control whether cached Redis transcriptions are used (defaults to `false`).                                                                                                                                                                                                                          |
+| 1.3.15  | Multi-scan support for WebsiteTalk (CP-1372): `websiteUrls` collection, `Scan.websiteUrlId`, `OVERWRITTEN` status, `activeScanId` on `WebsiteTalkProduct`. **Breaking:** `Scan.baseUrl` removed; `getActiveScanStatus` removed. |
+| 1.3.16  | Refactored WebsiteTalk scan status constants (CP-1372): `SCAN_STATUSES` object map, `FINISHED_SCAN_STATUSES` derived from it, `isInProgressScan` moved to `scans.utils`. No breaking API changes. |
+| 1.3.17  | Twilio call-status semantics for Websitalk dashboard: `inCallCount` KPI (`in-progress` + `answered`), `busy` is line-busy only; `updateCallStatusByCallSid`; expanded `CallStatus` union; `timeSavedMinutes` from `completed` only. |
+| 1.3.18  | WebsiteTalk multi-website API (CP-1372): ships `websiteUrls` module, scan getters by `websiteUrlId`, `activeScanId` getters on `clientsConfig`. **Note:** 1.3.17 did not include these — they landed after merge. **Breaking:** `WebsiteTalkProduct.defaultBaseUrl` removed from type. |
+| 1.3.19  | WebsiteTalk flow provisioning (WTIS activate-scan): optional `flowId` on `WebsiteTalkProduct`; `bindClientPhoneToFlow(clientId, flowId)` sets `flow_id` on the client's primary `phone_numbers` document so inbound calls route to the provisioned TalkPilot flow. |
+| 1.3.20  | `FINAL_STATUSES` constant for terminal Twilio call statuses (`no-answer`, `completed`, `busy`, `failed`, `canceled`) in `calls.constants`. |
+| 1.3.21  | Added `ToolExecutionStep` type, `steps` field to `ToolExecution`, and `pushToolExecution` getter (CP-1093). Added `ToolExecutionStepLevel` named type and `TOOL_EXECUTIONS_DEFAULT_LIMIT`/`SKIP` constants. Widened `ToolExecutionStep.data` to `Record<string, unknown> \| string` — allows plain string steps alongside structured JSON. Added optional `ToolExecutionStep.payload` for structured drill-down data. |
+| 1.3.22  | `CallsFilterParams.status` typed as `CallStatus` instead of `string` — aligns call-query filters with the Twilio status union from 1.3.17. |
+| 1.3.23  | Merge of 1.3.21 and 1.3.22 — combines the `ToolExecutionStep`/`steps`/`pushToolExecution` work (CP-1093) with the `CallsFilterParams.status: CallStatus` fix. Adds `CallStatusValues` (const object) with `CallStatus` and `ALL_CALL_STATUSES` derived from it. `CallStatus` still does not include `"redirected"` — tracked separately. |
+| 1.3.24  | **Breaking:** removed `flowId` from `WebsiteTalkProduct` — flow source of truth is `phone_numbers.flow_id`. Added `getPrimaryPhoneFlowId(clientId)`. |
+| 1.3.25  | Added `disconnectDb()` to close all MongoDB clients opened by `ensureDbConnected()` — for one-shot scripts that must exit cleanly. |
+| 1.3.28  | Soft-delete (`isActive`) on Streets/DepartmentSubjects, new bulk-write getters, MongoDB type re-exports, and the generic `bulkWrite` op-builder module. |
+| 1.3.29  | Added client call-quota module (CP-1500): `ClientConfig.quota` for usage limits (`totalQuota`, `usedQuota`, `expiresAt`), alert settings on `clients`, and getters for state, increment/reset, threshold alerts, and expiry notification tracking. |
+| 1.3.32  | `clientDisplayName` lives on `clientsConfig.quota` (not top-level). `getClientDisplayName` returns `"-"` when unset; never exposes `clientId`. |
+| 1.3.33  | Added `configuration` database module (CP-1560): `configurationMongodbClient` + a read-only `Prompts`-style collection for prompts previously hardcoded in consuming repos (e.g. MIS). |
+| 1.3.35  | Fixed the configuration collection name (`Prompts` → `prompts`, matches the lowercase convention used elsewhere). **Breaking** if you inserted documents into a collection literally named `Prompts`. Added `Prompt.productName` (required) + `getPromptsByProduct`. Centralized configuration-DB env var reads into `src/config.ts`. |
+| 1.3.36  | Publish combining the 1.3.33/1.3.35 configuration-DB work with the latest `main` (client quota + `clientDisplayName` + bulk-write/streets/departmentSubjects updates — see 1.3.28/1.3.29/1.3.32). No new code of its own. |
+| 1.3.37  | Added `models` and `internalModels` collections under `configuration` (CP-1624): realtime/stt model catalog (`ModelDoc`, `getModelByModelId`, `getModelsByProvider`, `getModelsByType`, `getAvailableModelsByType`) and a name-keyed settings store (`InternalModel`, `getInternalModelByName`) for cross-service config previously hardcoded in consuming repos (e.g. TalkPilot Server's realtime/STT catalogs and its live transcription model). Read-only, same as `prompts` — documents are seeded/managed manually. |
+| 1.3.38  | Added optional `transcriptionComparison` field to `Call` (CP-1594): per-`{provider}-{model}` side-channel transcription comparison results, same segment shape as `Call.transcription`. |
+
+### 1.1.9
+
+1. `npm run build`.
+2. `npm run test`.
+3. `npm run format`.
+
+### 1.3.4 — Call & ticket statistics (CP-149)
+
+New call statistics getters for dashboards (summary, trend, hourly, routing) and ticket statistics scoped to the same date range (open tickets, draft tickets, subject breakdowns).
+
+**Removed**
+
+| Removed                             | Replacement                                                                                                     |
+|-------------------------------------|-----------------------------------------------------------------------------------------------------------------|
+| `getTicketsCountByCityAndDateRange` | `aggregateCallsSummary` + `findCallSidTicketCountsByCity` — note: returns `Map<string, number>`, not `string[]` |
+| `getTicketsSubjectStats`            | `findSubjectsByCityAndDateRange`                                                                                |
+| `SubjectStatsItem`                  | `SubjectItem`                                                                                                   |
+
+**Do I need to update my app?**
+
+Yes — if you used any of the removed functions above.
+
+### 1.3.5 — Muni issues (CP-1115)
+
+New `muniIssues` module under `src/municipal/` for issues reported from the call-log screen. The document is written to the municipal DB after its Jira ticket is created.
+
+**Added**
+
+| Export                                       | Purpose                                                                                                                                                 |
+|----------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `MuniIssue` / `CreateMuniIssueInput`         | Document type and insert-input type (`issueContent` + `jira` domains).                                                                                  |
+| `createMuniIssue(input, id)`                 | Insert an issue; requires a caller-supplied `_id` from MIS (via `generateMuniIssueId`) so it matches the id embedded in the Jira ticket (two-way link). |
+| `generateMuniIssueId()`                      | Pre-generate the `_id` to pass to `createMuniIssue`.                                                                                                    |
+| `getMuniIssueById` / `getMuniIssueByJiraKey` | Lookups by our id or by the Jira key.                                                                                                                   |
+| `ensureMuniIssuesCollection()`               | Provision the `$jsonSchema` validator and indexes (unique `jira.key`); run once at MIS boot.                                                            |
+
+**Do I need to update my app?**
+
+No — purely additive. New consumers (MIS) import these from `@talkpilot/core-db`.
+
+### 1.3.6 — Context Notes & Products
+
+**Context Notes** — new `contextNotes` collection under `src/talkpilot/` for per-client, per-product notes injected into AI calls at runtime. Each document holds a `systemPrompt` and a list of time-bounded entries (`activeFrom`, `expiresAt`). Two setters with split ownership: `setContextNoteEntries` for the notes list and `setContextNoteConfig` for `systemPrompt` and `product`.
+
+**Products** — new `products` collection under `src/talkpilot/` cataloguing TalkPilot products. Each document has a stable `name` and an optional `displayName` locale map (`Record<string, string>`) for UI display.
+
+**Added**
+
+| Export                                       | Purpose                                                         |
+|----------------------------------------------|-----------------------------------------------------------------|
+| `getContextNotes(clientId, product)`         | Fetch context note documents for a client/product pair.         |
+| `createContextNote(input)`                   | Insert a new context note document.                             |
+| `setContextNoteEntries(id, clientId, notes)` | Replace the notes list on an existing document.                 |
+| `setContextNoteConfig(id, clientId, config)` | Update `systemPrompt` and/or `product` on an existing document. |
+| `getAllProducts()`                           | Fetch the full product catalogue.                               |
+
+**Do I need to update my app?**
+
+No — purely additive.
+
+### 1.3.7 — Generic prompt control (CP-704)
+
+Added an optional boolean field to the `Flow` type and MongoDB schema.
+
+**Added**
+
+| Export                      | Purpose                                                                                                                                  |
+|-----------------------------|------------------------------------------------------------------------------------------------------------------------------------------|
+| `Flow.disableGenericPrompt` | Optional `boolean`. When `true`, disables the generic prompt for the flow. Existing flows without this field continue to work unchanged. |
+
+**Do I need to update my app?**
+
+No — the field is optional and fully backwards-compatible.
+
+### 1.3.8 — Ticket-count Map getters (CP-1200)
+
+Adds `Map<string, number>` ticket getters that return per-call ticket counts. The old `string[]` getters and filter fields are kept and marked `@deprecated`.
+
+**Deprecated → Replacement**
+
+| Deprecated                                  | Replacement                                                  |
+|---------------------------------------------|--------------------------------------------------------------|
+| `findCallSidsWithTicketsByCity`             | `findCallSidTicketCountsByCity` → `Map<string, number>`      |
+| `findCallSidsWithDraftTicketsByCity`        | `findCallSidDraftTicketCountsByCity` → `Map<string, number>` |
+| `CallsStatsFilter.callSidsWithTickets`      | `CallsStatsFilter.callSidTicketCounts`                       |
+| `CallsStatsFilter.callSidsWithDraftTickets` | `CallsStatsFilter.callSidDraftTicketCounts`                  |
+
+**Do I need to update my app?**
+
+No — the old fields still compile. Migrate when convenient.
+
+### 1.3.9 — Wrap-around hour filter boundary guards (CP-1166)
+
+When `hourFrom` > `hourTo` (e.g. 23:00–13:00), two silent miscounting bugs existed:
+
+- **First-day morning**: calls before `hourTo` on the first day (e.g. 08:00 on Jun 8) were counted even though the window starts at `hourFrom` that day. Fixed by adding `dateLocal > startStr` to the after-midnight leg.
+- **Last-day evening**: calls at or after `hourFrom` on the last day (e.g. 23:30 on Jun 9) were counted even though the window ends at `hourTo` that day. Fixed by adding `dateLocal < endStr` to the before-midnight leg.
+
+**Affects:** `aggregateCallsTrend`, `aggregateCallsSummary`, `aggregateCallsHourlyByRange`, `aggregateCallsRouting`, `findFilteredCallSids`, `findSubjectsByCityAndDateRange`.
+
+**Do I need to update my app?**
+
+No API changes — both fixes correct silent miscounting. Upgrading is recommended if you use a wrap-around hour window.
+
+### 1.3.10 — Ticket draft flag (CP-1390)
+
+Added an optional `isDraft` boolean field to the `Ticket` type.
+
+**Added**
+
+| Export           | Purpose                                                                                                                                                                                 |
+|------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `Ticket.isDraft` | Optional `boolean`. When `true`, the ticket is a draft. When `false`, it is not. When absent (legacy tickets), draft detection falls back to the previous `event_subject_id` heuristic. |
+
+**Affects:** `findCallSidDraftTicketCountsByCity`, `findCallSidsWithDraftTicketsByCity` (draft filtering in ticket statistics).
+
+**Do I need to update my app?**
+
+No — purely additive. Existing tickets without `isDraft` continue to work unchanged. New consumers (MIS) can pass `isDraft: true` when creating a draft ticket.
+
+### 1.3.11 — Fix clientsConfig type declaration import
+
+Fixed a pre-existing broken import path in the published `clientsConfig.types.d.ts` (introduced in an earlier release, unrelated to 1.3.10). Consumer TypeScript builds could fail with:
+
+`Cannot find module 'src/utils/shared.types' or its corresponding type declarations.`
+
+The import now uses a relative path (`../../utils/shared.types`) so `tsc` resolves it correctly from `node_modules/@talkpilot/core-db/dist/`.
+
+**Do I need to update my app?**
+
+Yes — upgrade to 1.3.11 if your build failed with the error above (affects any version that shipped the broken declaration). No API or behavior changes otherwise.
+
+### 1.3.12 — Calls-with-tickets count (CP-1200)
+
+Added `callsWithTickets` to `CallsSummaryAggregation` — the number of distinct calls that opened at least one ticket. Use this field (instead of `openTickets`) to compute call-level rates and "calls without a ticket" counts.
+
+**Do I need to update my app?**
+
+Yes — replace `openTickets` with `callsWithTickets` wherever you compute `ticketOpenRate` or `not_opened`.
+
+### 1.3.13 — Faulty release
+
+This release incorrectly placed `useRedisTranscription` on `Flow` instead of `FlowTool`. It is redundant and should be skipped.
+
+### 1.3.14 — Redis transcription flag
+
+Added an optional `FlowTool.useRedisTranscription` boolean that tells consumers whether to read transcriptions from Redis. Defaults to `false` when omitted.
+
+**Do I need to update my app?**
+
+No — this flag is optional. Only update if you want to opt into the Redis transcription path.
+
+### 1.3.15 — Multi-scan configuration (CP-1372)
+
+Adds `websiteUrls` collection (CRUD getters), migrates scans from `baseUrl` to `websiteUrlId`, adds `OVERWRITTEN` scan status, and replaces `defaultBaseUrl`/`logoUrl` on `WebsiteTalkProduct` with `activeScanId` (+ getters).
+
+**Breaking**
+
+| Removed / changed | Replacement |
+| --- | --- |
+| `Scan.baseUrl` | `Scan.websiteUrlId` |
+| `WebsiteTalkProduct.defaultBaseUrl` | `websiteUrls` collection |
+| `getActiveScanStatus(clientId)` | `getInProgressScanByWebsiteUrl`, `findInProgressScansByClient` |
+
+**Do I need to update my app?**
+
+Yes — WTIS, CoreBrain, and CoreCrawler must adopt `websiteUrlId` and the new getters before upgrading.
+
+### 1.3.16 — Scan status constants refactor (CP-1372)
+
+Centralizes scan status values under a single `SCAN_STATUSES` object map and derives terminal statuses from it.
+
+**Changed**
+
+| Area | What |
+| --- | --- |
+| `SCAN_STATUSES` | Object map (`SCAN_STATUSES.CHECKING`, `SCAN_STATUSES.COMPLETED`, …) instead of separate exported string constants. |
+| `FINISHED_SCAN_STATUSES` | Derived from `SCAN_STATUSES` (`COMPLETED`, `FAILED`, `OVERWRITTEN`). |
+| `isInProgressScan` | Moved to `scans.utils.ts` (exported from `scans` index). |
+| `ScanStatus` type | Derived from `(typeof SCAN_STATUSES)[keyof typeof SCAN_STATUSES]`. |
+
+**Do I need to update my app?**
+
+No — behavior is unchanged. If you imported removed aliases like `COMPLETED_SCAN_STATUS`, switch to `SCAN_STATUSES.COMPLETED` (or keep using string literals / `ScanStatus` type).
+
+### 1.3.17 — Twilio call-status semantics (CP-1521)
+
+Aligns dashboard KPIs and call-status updates with Twilio semantics so in-call counts update in real time.
+
+**Added**
+
+| Export | Purpose |
+| --- | --- |
+| `updateCallStatusByCallSid(callSid, status)` | Set the current call status by `callSid`. |
+| `IN_CALL_STATUSES` | `["in-progress", "answered"]` — statuses counted as in-call. |
+| `DashboardSummaryMetrics.inCallCount` | Replaces `busyCount`; counts in-progress + answered calls. |
+
+**Changed**
+
+| Area | What |
+| --- | --- |
+| Dashboard KPIs | `busyCount` → `inCallCount`; `busy` is line-busy only, not in-call. |
+| `timeSavedMinutes` | Computed from `completed` calls only. |
+| `CallStatus` union | Expanded for Twilio-aligned statuses. |
+
+**Do I need to update my app?**
+
+Yes — if you read `busyCount` from dashboard KPIs, switch to `inCallCount`.
+
+### 1.3.18 — WebsiteTalk multi-website API (CP-1372)
+
+Ships the full persistence API for multi-website / multi-scan. Versions 1.3.15–1.3.16 documented the design; **1.3.17 did not export these symbols** (CP-1372 merged into main after the 1.3.17 release). Upgrade to **1.3.18** if you see missing-export errors such as `getActiveScanId`, `getWebsiteUrlById`, or `WebsiteUrlDoc`.
+
+**`websiteUrls` module**
+
+| Export | Purpose |
+| --- | --- |
+| `WebsiteUrl` / `WebsiteUrlDoc` / `CreateWebsiteUrlInput` | Document types. |
+| `getWebsiteUrlsCollection()` | Collection access. |
+| `getWebsiteUrlsByClient(clientId)` | All websites for a client. |
+| `getWebsiteUrlById(id)` | Lookup by id. |
+| `findWebsiteUrls(filter?)` | Flexible query. |
+| `createWebsiteUrl({ clientId, name, baseUrl })` | Create. |
+| `updateWebsiteUrl(id, updates)` | Update `name` / `baseUrl`. |
+| `deleteWebsiteUrl(id)` | Delete by id. |
+
+**`clientsConfig` — selected scan**
+
+| Export | Purpose |
+| --- | --- |
+| `getActiveScanId(clientId)` | Read `products.websiteTalk.activeScanId`. |
+| `updateActiveScanId(clientId, scanId)` | Set the active scan for search. |
+| `clearActiveScanId(clientId)` | Unset `activeScanId`. |
+
+**`scans` — by `websiteUrlId`**
+
+| Export | Purpose |
+| --- | --- |
+| `getScanById(id)` | Lookup by id. |
+| `getScansByWebsiteUrl(websiteUrlId)` | All scans for a website. |
+| `getLatestScanByWebsiteUrl(websiteUrlId)` | Most recent scan. |
+| `getInProgressScanByWebsiteUrl(websiteUrlId)` | Latest scan if still in progress. |
+| `findInProgressScansByClient(clientId)` | All in-progress scans for a client. |
+| `hasScansForWebsiteUrl(websiteUrlId)` | Whether any scans exist. |
+| `getLatestCompletedScanByWebsiteUrl(websiteUrlId)` | Most recent `COMPLETED` scan. |
+| `updateScanDoc(scanId, updates)` | Partial update (e.g. mark `OVERWRITTEN`). |
+| `deleteScansByWebsiteUrl(websiteUrlId)` | Delete all scans for a website. |
+
+**Breaking / type changes**
+
+| Removed | Replacement |
+| --- | --- |
+| `WebsiteTalkProduct.defaultBaseUrl` | `websiteUrls` collection (`name`, `baseUrl`) |
+| `Scan.baseUrl` | `Scan.websiteUrlId` (see 1.3.15) |
+
+**`OVERWRITTEN` scan status**
+
+- `OVERWRITTEN` is part of `ScanStatus` / `SCAN_STATUSES`.
+- Marking previous `COMPLETED` scans as overwritten is **not** done in `createScanDoc` — implement in the service layer (e.g. via `updateScanDoc`).
+
+**Not in core-db (service layer)**
+
+- `resolveActiveWebsiteContext` — join `activeScanId` → scan → websiteUrl.
+- Migration from legacy `defaultBaseUrl`.
+
+**Do I need to update my app?**
+
+Yes — WTIS, CoreBrain, and CoreCrawler should upgrade to **1.3.18** and import the symbols above from `@talkpilot/core-db`. Stop reading `defaultBaseUrl` from `WebsiteTalkProduct`; use `websiteUrls` instead.
+
+### 1.3.19 — WebsiteTalk flow provisioning (WTIS activate-scan)
+
+Persists the TalkPilot flow id on the client product config and links the client's primary phone number to that flow so inbound calls use the WebsiteTalk flow after `activate-scan`.
+
+**`clientsConfig` — `WebsiteTalkProduct`**
+
+| Field | Purpose |
+| --- | --- |
+| `WebsiteTalkProduct.flowId?` | Optional TalkPilot flow `_id` (hex string). Set by WTIS after `POST /flows/add_new_flow`; reused on later activations via `PUT /flows/edit_flow/:id`. |
+
+**`phone_numbers`**
+
+| Export | Purpose |
+| --- | --- |
+| `bindClientPhoneToFlow(clientId, flowId)` | Updates the **primary** `phone_numbers` row for `clientId`, setting `flow_id` to the given TalkPilot flow. Throws if no primary phone exists. |
+
+**Typical service-layer flow (WTIS)**
+
+1. Build flow payload from template + `companyInfo` (generated at activate time).
+2. Create or update the flow via TalkPilot Server API (`add_new_flow` / `edit_flow`).
+3. On first create: `updateProductConfig(clientId, "websiteTalk", { flowId })`.
+4. `bindClientPhoneToFlow(clientId, flowId)` — ensure the phone routes to the flow.
+5. `updateActiveScanId(clientId, scanId)` — point search at the activated scan.
+
+**Not in core-db (service layer)**
+
+- Flow template substitution, Claude `companyInfo` generation, Firebase Storage template download, TalkPilot flow create/update HTTP calls.
+
+**Do I need to update my app?**
+
+No — purely additive. **WTIS** should upgrade to **1.3.19** (or later) and import `bindClientPhoneToFlow` plus `WebsiteTalkProduct.flowId` when implementing activate-scan flow provisioning.
+
+### 1.3.20 — `FINAL_STATUSES` call-status constant
+
+Centralizes terminal Twilio call statuses so consumers (e.g. TalkPilot Server `callResultService`) don't duplicate the list locally.
+
+**Added**
+
+| Export | Purpose |
+| --- | --- |
+| `FINAL_STATUSES` | `["no-answer", "completed", "busy", "failed", "canceled"]` — statuses where no further Twilio callbacks are expected. Pair with `IN_CALL_STATUSES` for live vs terminal semantics. |
+
+**Do I need to update my app?**
+
+No — purely additive. Replace local `FINAL_STATUSES` arrays with `import { FINAL_STATUSES } from "@talkpilot/core-db"` when convenient.
+
+### 1.3.21 — Tool execution steps & push getter (CP-1093)
+
+Added `ToolExecutionStep` type, `steps` field on `ToolExecution`, `pushToolExecution` getter, named `ToolExecutionStepLevel` type, and pagination constants.
+
+**Added**
+
+| Export | Purpose |
+| --- | --- |
+| `ToolExecutionStepLevel` | Named type: `"info" \| "warn" \| "error"`. |
+| `ToolExecution.steps` | Optional `ToolExecutionStep[]`. Ordered log of intermediate steps within a tool call (e.g. `audio_enrichment`, `google_places`, `subject_classification`). Absent on executions recorded before this version. |
+| `ToolExecutionStep.payload` | Optional `Record<string, unknown>`. Structured data for UI drill-down, alongside the human-readable `data` field. |
+| `pushToolExecution(callSid, execution)` | Append a `ToolExecution` to the call's `toolExecutions` array via `$push`. |
+| `getToolExecutionsByCallSid(callSid, opts?)` | Fetch the `toolExecutions` array for a call. `opts.skip` and `opts.limit` (default `TOOL_EXECUTIONS_DEFAULT_LIMIT` = 50) support pagination. Returns `[]` when the call has no executions or does not exist. |
+| `TOOL_EXECUTIONS_DEFAULT_LIMIT` | `50` — default page size for `getToolExecutionsByCallSid`. |
+| `TOOL_EXECUTIONS_DEFAULT_SKIP` | `0` — default skip for `getToolExecutionsByCallSid`. |
+
+**Do I need to update my app?**
+
+No — purely additive. All new fields are optional and existing documents continue to work unchanged.
+
+### 1.3.22 — `CallsFilterParams.status` type alignment
+
+Tightens the `status` filter field on `CallsFilterParams` from `string` to `CallStatus` so call-query consumers get compile-time validation against the Twilio-aligned status union introduced in 1.3.17.
+
+**Changed**
+
+| Area | What |
+| --- | --- |
+| `CallsFilterParams.status` | `string` → `CallStatus` |
+
+**Do I need to update my app?**
+
+Only if you pass a non-`CallStatus` string to `CallsFilterParams.status` — fix the value or cast. No runtime behavior change.
+
+### 1.3.23 — Merge of 1.3.21 and 1.3.22, plus `CallStatusValues`
+
+Combines the CP-1093 tool-execution-steps work (1.3.21) with the `CallsFilterParams.status` type fix (1.3.22); both were published independently from diverging branches.
+
+**Added**
+
+| Export | Purpose |
+| --- | --- |
+| `CallStatusValues` | `as const` object mapping named keys (`Completed`, `Failed`, ...) to the 9 raw Twilio-aligned status strings — gives enum-like dot access (`CallStatusValues.Completed`) while `CallStatus` stays a plain string-literal union (no breaking change for existing `status === "completed"` comparisons). |
+| `CallStatus` | Now derived from `CallStatusValues` (`(typeof CallStatusValues)[keyof typeof CallStatusValues]`) instead of a hand-written union — single source of truth. |
+| `ALL_CALL_STATUSES` | `CallStatus[]` — `Object.values(CallStatusValues)`, for runtime validation of raw input (e.g. HTTP query params) against `CallStatus`. |
+
+**Do I need to update my app?** No — `CallStatus` is structurally identical to before (same 9 string literals); this is a source-only refactor.
+
+`CallStatus` still does not include `"redirected"` — tracked separately.
+
+### 1.3.28 — Soft-delete support and generic bulk-write helpers
+
+Adds an `isActive` soft-delete convention, bulk-write getters for Streets/DepartmentSubjects, MongoDB type re-exports, and a generic bulk-write op-builder module.
+
+**Added**
+
+| Export | Purpose |
+| --- | --- |
+| `Street.isActive` / `DepartmentSubject.isActive` | Optional soft-delete flag; `createStreet` defaults it to `true`. |
+| `bulkWriteStreets` / `bulkWriteDepartmentsSubjects` | Passthrough bulk-write for each collection. |
+| `findDepartmentSubjectsByFilter` | Query subjects by an arbitrary filter. |
+| `updateDepartmentSubjectActive` | Set `isActive` on a subject by `_id`. |
+| `ObjectId`, `Filter`, `BulkWriteOp`, `BulkWriteError`, `OptionalUnlessRequiredId` | Re-exported from `mongodb`. |
+| `buildSetOp`, `buildUpsertOp`, `buildInsertOp`, `buildDeleteOp` | Generic bulk-write op builders — filter/fields in, a shaped op out. |
+
+**Do I need to update my app?**
+
+No — purely additive.
+
+### 1.3.29 — Client call quota (CP-1500)
+
+Per-client call quota with usage counters, optional expiration, and configurable threshold/expiry email alerts. Usage lives on clientsConfig; alert config and send-tracking live on clients. TalkPilot Server consumes these getters to block calls when quota is exceeded or expired and to send notification emails.
+
+**Data model**
+
+| Location | Field | Purpose |
+| --- | --- | --- |
+| `clientsConfig.quota` | `totalQuota`, `usedQuota`, `expiresAt?` | Call allowance and consumption. `getClientQuotaState` returns `null` when `totalQuota` is unset (quota disabled for that client). |
+| `clients.quota` | `alerts?`, `expiredSent?` | Up to 3 threshold alerts (each with up to 3 recipient emails) and a flag set after the expiry email is sent. |
+
+**Added**
+
+| Export | Purpose |
+| --- | --- |
+| `ClientQuotaUsage` | Usage shape: `totalQuota`, `usedQuota`, optional `expiresAt`. |
+| `ClientQuotaSettings` | Alert settings on the `clients` doc: `alerts`, `expiredSent`. |
+| `ClientQuotaState` | Merged view returned by `getClientQuotaState` (usage + settings). |
+| `QuotaThresholdAlert` / `QuotaAlertInput` | Threshold alert with `thresholdPct`, `recipientEmails`, and optional `thresholdSent` send-tracking. |
+| `getClientQuotaState(clientId)` | Read combined quota state; `null` when no quota is configured. |
+| `incrementClientQuotaUsage(clientId, amount?)` | Atomically increment `clientsConfig.quota.usedQuota` (default `amount = 1`). No-op when `quota.totalQuota` is absent. |
+| `resetClientQuotaUsage(clientId)` | Reset `usedQuota` to `0` and clear alert/expiry send flags via `resetClientQuotaSettings`. |
+| `updateQuotaAlertsConfig(clientId, alerts)` | Replace threshold alert config; preserves `thresholdSent` for unchanged thresholds. |
+| `markQuotaAlertSent(clientId, alertIndex)` | Idempotently mark a threshold alert as sent; returns whether the update applied. |
+| `markQuotaExpiredSent(clientId)` | Idempotently mark the expiry notification as sent; returns whether the update applied. |
+| `resetClientQuotaSettings(clientId)` | Clear `expiredSent` and strip `thresholdSent` from alerts (keeps alert definitions). |
+
+**Do I need to update my app?**
+
+No — purely additive. TalkPilot Server should upgrade to 1.3.29 (or later) to enforce quota on inbound/outbound calls and to drive threshold/expiry emails via the new getters.
+
+### 1.3.32 — Quota `clientDisplayName`
+
+Optional human-readable account label stored on `clientsConfig.quota` for quota notification emails. TalkPilot Server uses `getClientDisplayName` instead of exposing raw `clientId` in alert/expiry email copy.
+
+**Changed**
+
+| Area | What |
+| --- | --- |
+| `ClientQuotaUsage.clientDisplayName?` | Optional display name on `clientsConfig.quota` — e.g. `"Acme Corp"`. Included in `ClientQuotaState` via `getClientQuotaState`. |
+
+**Added**
+
+| Export | Purpose |
+| --- | --- |
+| `getClientDisplayName(clientId)` | Returns trimmed `quota.clientDisplayName`, or `"-"` when unset, missing, or blank. Never falls back to `clientId`. |
+
+**Do I need to update my app?**
+
+No — purely additive. Set `clientsConfig.quota.clientDisplayName` when provisioning quota for a client. **TalkPilot Server** should upgrade to **1.3.32** (or later) to use the label in quota emails.
+
+### 1.3.33 — Configuration database & Prompts collection (CP-1560)
+
+New `configuration` database with a prompts collection, so prompts previously hardcoded in consuming repos (e.g. MIS's `moked_106` street/subject search and transcription prompts) can be stored centrally and edited without a deploy. Ships the **read** path only — documents are seeded/managed manually (e.g. via Compass), not through this package.
+
+**Added**
+
+| Export | Purpose |
+| --- | --- |
+| `configurationMongodbClient` | MongoDB client for the `configuration` database (default name `configuration`, override with `CONFIGURATION_DB_NAME`). |
+| `getConfigurationDb` / `setConfigurationDb` / `ConfigurationObjectId` | DB handle accessors and `ObjectId`, matching the other domains. |
+| `Prompt` | Document type: `{ _id, name, content, createdAt, updatedAt }`, keyed by unique `name`. |
+| `getPromptByName(name)` | Fetch a single prompt by its unique `name`. Returns `null` if absent. |
+| `findPrompts(filter?)` | Flexible query over the collection. |
+| `getAllPrompts()` | Fetch every prompt. |
+| `getPromptsCollection()` | Raw collection access. |
+
+`ensureDbConnected()` / `disconnectDb()` now also connect/disconnect the configuration client alongside talkpilot/municipal/websitalk.
+
+**Do I need to update my app?**
+
+No — purely additive. Consumers opt in by importing the getters above.
+
+### 1.3.35 — Prompts fixes: collection name, `productName`, centralized config
+
+**Breaking**
+
+| Changed | What |
+| --- | --- |
+| Collection name | `"Prompts"` → `"prompts"` (lowercase, matching every other collection in core-db). MongoDB collection names are case-sensitive — if you inserted documents into a collection literally named `Prompts` under 1.3.33, move them to `prompts`. |
+
+**Added**
+
+| Export | Purpose |
+| --- | --- |
+| `Prompt.productName` | Required `KnownProductKey` (`"municipal" \| "clinics" \| "websiteTalk"`). Existing documents need this field backfilled. |
+| `getPromptsByProduct(productName)` | Fetch all prompts for a given product. |
+
+**Internal**
+
+- Configuration-DB env var reads (`MONGO_URI`, `MONGODB_URI`, `CONFIGURATION_DB_NAME`) centralized into `src/config.ts` (`configurationDbConfig`) — no consumer-facing change.
+
+**Do I need to update my app?**
+
+Only if you inserted documents into the old `Prompts` (capital P) collection — move them to `prompts`. Also backfill `productName` on any existing prompt documents (e.g. `"municipal"`), since it's now a required field on the type.
+
+### 1.3.36 — Merge with `main`
+
+Publishes the 1.3.33/1.3.35 configuration-DB work alongside the latest `main` — client call-quota, `clientDisplayName`, and bulk-write/streets/departmentSubjects updates (see 1.3.28, 1.3.29, 1.3.32). No new code of its own; this version exists to ship both branches' work together.
+
+**Do I need to update my app?**
+
+No — purely additive, same as the versions it combines.
+
+### 1.3.37 — Model catalog & internal model settings (CP-1624)
+
+New `models` and `internalModels` collections under the `configuration` database, so AI model catalogs and cross-service settings previously hardcoded in consuming repos (e.g. TalkPilot Server's realtime/STT model lists and its live transcription model) can be stored centrally and edited without a deploy. Ships the **read** path only, same as `prompts` — documents are seeded/managed manually (e.g. via Compass).
+
+**Added — `models`**
+
+| Export | Purpose |
+| --- | --- |
+| `ModelDoc` (`RealtimeModelDoc \| SttModelDoc`) | Document type discriminated by `type`: `{ _id, modelId, name, description?, available, features?, configDefaults?, createdAt, updatedAt }`, plus `provider` (`RealtimeModelProvider` for `type: "realtime"`, `SttModelProvider` for `type: "stt"`). |
+| `MODEL_TYPES`, `REALTIME_MODEL_PROVIDERS`, `STT_MODEL_PROVIDERS` | Known `type`/`provider` value lists the types above are derived from. |
+| `getModelByModelId(modelId, type?)` | Fetch a single model by its `modelId`, optionally scoped to a `type`. |
+| `getModelsByProvider(provider, type?)` | Fetch all models for a given `provider`, optionally scoped to a `type`. |
+| `getModelsByType(type)` | Fetch all models of a given `type`. |
+| `getAvailableModelsByType(type)` | Same as above, pre-filtered to `available: true`. |
+| `findModels(filter?)` | Flexible query over the collection. |
+| `getModelsCollection()` | Raw collection access. |
+
+**Added — `internalModels`**
+
+| Export | Purpose |
+| --- | --- |
+| `InternalModel` | Document type: `{ _id, name, provider, model, config?, createdAt, updatedAt }`, keyed by unique `name` — e.g. a `"transcriptionModel"` document. |
+| `getInternalModelByName(name)` | Fetch a single internal model config by its unique `name`. Returns `null` if absent. |
+| `findInternalModels(filter?)` | Flexible query over the collection. |
+| `getAllInternalModels()` | Fetch every internal model. |
+| `getInternalModelsCollection()` | Raw collection access. |
+
+`ensureDbConnected()` / `disconnectDb()` already covered the configuration client from 1.3.33 — no change there.
+
+**Do I need to update my app?**
+
+No — purely additive. Consumers opt in by importing the getters above.
+
+### 1.3.38 — Transcription comparison field (CP-1594)
+
+Added an optional `transcriptionComparison` field to `Call`, for consumers that run side-channel transcription comparisons across multiple providers/models alongside the real `transcription` field.
+
+**Added**
+
+| Export | Purpose |
+| --- | --- |
+| `Call.transcriptionComparison` | Optional `Record<string, TranscriptionSegment[]>` — one entry per `{provider}-{model}` key, each an array of `TranscriptionSegment` in the same shape as `Call.transcription`. |
+
+**Do I need to update my app?**
+
+No — purely additive. Existing documents and consumers continue to work unchanged.
+
+## 🛠 CI/CD & deployment
+
+- This package is consumed by Cloud Build-based services (TalkPilot Server, MIS, CIS) as a dependency when Docker images are built. Keep `dist/` in sync with your builds because the compiled artifact is what downstream services install.
+- Releases require the shared npm token documented in `DEVELOPMENT.md`; consult that guide for contribution, linking, and token rotation procedures.
+
+## Resources
+
+- [DEVELOPMENT.md](./DEVELOPMENT.md) (setup, tooling, publishing, token management)
+- [src/test-utils](src/test-utils) and the `__tests__` folder for examples of `MongoMemoryServer` wiring.
+