@talkpilot/core-db 1.3.7 → 1.3.9

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (199) hide show
  1. package/.claude/settings.local.json +11 -0
  2. package/.cursor/rules/development.mdc +65 -65
  3. package/DEVELOPMENT.md +141 -141
  4. package/README.md +257 -190
  5. package/README_OLD.md +160 -160
  6. package/dist/municipal/tickets/tickets.deprecated.getters.d.ts +12 -0
  7. package/dist/municipal/tickets/tickets.deprecated.getters.d.ts.map +1 -0
  8. package/dist/municipal/tickets/tickets.deprecated.getters.js +131 -0
  9. package/dist/municipal/tickets/tickets.deprecated.getters.js.map +1 -0
  10. package/dist/municipal/tickets/tickets.statistics.dates.d.ts +7 -0
  11. package/dist/municipal/tickets/tickets.statistics.dates.d.ts.map +1 -0
  12. package/dist/municipal/tickets/tickets.statistics.dates.js +40 -0
  13. package/dist/municipal/tickets/tickets.statistics.dates.js.map +1 -0
  14. package/dist/municipal/tickets/tickets.statistics.getters.d.ts +4 -0
  15. package/dist/municipal/tickets/tickets.statistics.getters.d.ts.map +1 -1
  16. package/dist/municipal/tickets/tickets.statistics.getters.js +30 -3
  17. package/dist/municipal/tickets/tickets.statistics.getters.js.map +1 -1
  18. package/dist/municipal/tickets/tickets.statistics.pipeline.d.ts +53 -0
  19. package/dist/municipal/tickets/tickets.statistics.pipeline.d.ts.map +1 -0
  20. package/dist/municipal/tickets/tickets.statistics.pipeline.js +112 -0
  21. package/dist/municipal/tickets/tickets.statistics.pipeline.js.map +1 -0
  22. package/dist/municipal/tickets/tickets.statistics.utils.d.ts +7 -0
  23. package/dist/municipal/tickets/tickets.statistics.utils.d.ts.map +1 -0
  24. package/dist/municipal/tickets/tickets.statistics.utils.js +40 -0
  25. package/dist/municipal/tickets/tickets.statistics.utils.js.map +1 -0
  26. package/dist/talkpilot/calls/calls.statistics.getters.d.ts.map +1 -1
  27. package/dist/talkpilot/calls/calls.statistics.getters.js +18 -18
  28. package/dist/talkpilot/calls/calls.statistics.getters.js.map +1 -1
  29. package/dist/talkpilot/calls/calls.statistics.ticketScope.d.ts +12 -0
  30. package/dist/talkpilot/calls/calls.statistics.ticketScope.d.ts.map +1 -0
  31. package/dist/talkpilot/calls/calls.statistics.ticketScope.js +37 -0
  32. package/dist/talkpilot/calls/calls.statistics.ticketScope.js.map +1 -0
  33. package/dist/talkpilot/calls/calls.statistics.tickets.d.ts +17 -0
  34. package/dist/talkpilot/calls/calls.statistics.tickets.d.ts.map +1 -0
  35. package/dist/talkpilot/calls/calls.statistics.tickets.js +33 -0
  36. package/dist/talkpilot/calls/calls.statistics.tickets.js.map +1 -0
  37. package/dist/talkpilot/calls/calls.statistics.types.d.ts +4 -1
  38. package/dist/talkpilot/calls/calls.statistics.types.d.ts.map +1 -1
  39. package/dist/utils/statistics.aggregation.d.ts +20 -0
  40. package/dist/utils/statistics.aggregation.d.ts.map +1 -0
  41. package/dist/utils/statistics.aggregation.js +43 -0
  42. package/dist/utils/statistics.aggregation.js.map +1 -0
  43. package/jest.config.js +20 -20
  44. package/package.json +46 -46
  45. package/src/__tests__/setup.ts +20 -20
  46. package/src/connection.ts +54 -54
  47. package/src/index.ts +26 -26
  48. package/src/municipal/__tests__/validation.spec.ts +62 -62
  49. package/src/municipal/cities/cities.getters.ts +50 -50
  50. package/src/municipal/cities/cities.types.ts +11 -11
  51. package/src/municipal/cities/index.ts +2 -2
  52. package/src/municipal/departmentsSubjects/departmentsSubjects.getters.ts +282 -282
  53. package/src/municipal/departmentsSubjects/departmentsSubjects.types.ts +72 -72
  54. package/src/municipal/departmentsSubjects/index.ts +9 -9
  55. package/src/municipal/index.ts +22 -22
  56. package/src/municipal/mongodb-client.ts +61 -61
  57. package/src/municipal/muniIssues/__tests__/muniIssues.setters.spec.ts +92 -92
  58. package/src/municipal/muniIssues/__tests__/muniIssues.setup.spec.ts +92 -92
  59. package/src/municipal/muniIssues/index.ts +17 -17
  60. package/src/municipal/muniIssues/muniIssues.constants.ts +18 -18
  61. package/src/municipal/muniIssues/muniIssues.getters.ts +24 -24
  62. package/src/municipal/muniIssues/muniIssues.schema.ts +66 -66
  63. package/src/municipal/muniIssues/muniIssues.setters.ts +24 -24
  64. package/src/municipal/muniIssues/muniIssues.setup.ts +65 -65
  65. package/src/municipal/muniIssues/muniIssues.types.ts +37 -37
  66. package/src/municipal/streets/index.ts +2 -2
  67. package/src/municipal/streets/streets.getters.ts +125 -125
  68. package/src/municipal/streets/streets.types.ts +18 -18
  69. package/src/municipal/systemInstructions/__tests__/getters.spec.ts +113 -113
  70. package/src/municipal/systemInstructions/__tests__/setters.spec.ts +274 -274
  71. package/src/municipal/systemInstructions/index.ts +7 -7
  72. package/src/municipal/systemInstructions/instructions.getters.ts +57 -57
  73. package/src/municipal/systemInstructions/instructions.setters.ts +119 -119
  74. package/src/municipal/systemInstructions/instructions.types.ts +30 -30
  75. package/src/municipal/tickets/__tests__/tickets.getters.spec.ts +30 -30
  76. package/src/municipal/tickets/__tests__/tickets.statistics.spec.ts +68 -51
  77. package/src/municipal/tickets/index.ts +3 -3
  78. package/src/municipal/tickets/tickets.constants.ts +8 -8
  79. package/src/municipal/tickets/tickets.getters.ts +121 -121
  80. package/src/municipal/tickets/tickets.statistics.aggregation.ts +96 -96
  81. package/src/municipal/tickets/tickets.statistics.getters.ts +109 -71
  82. package/src/municipal/tickets/tickets.types.ts +53 -53
  83. package/src/municipal/utils/types.ts +11 -11
  84. package/src/talkpilot/__tests__/db.spec.ts +38 -38
  85. package/src/talkpilot/__tests__/mongodb-client.spec.ts +18 -18
  86. package/src/talkpilot/__tests__/validation.spec.ts +68 -68
  87. package/src/talkpilot/agents/__tests__/agents.getters.spec.ts +29 -29
  88. package/src/talkpilot/agents/agents.getters.ts +34 -34
  89. package/src/talkpilot/agents/agents.types.ts +14 -14
  90. package/src/talkpilot/agents/index.ts +2 -2
  91. package/src/talkpilot/backgroundToolResults/__tests__/backgroundToolResults.getters.spec.ts +147 -147
  92. package/src/talkpilot/backgroundToolResults/backgroundToolResults.getters.ts +65 -65
  93. package/src/talkpilot/backgroundToolResults/backgroundToolResults.types.ts +23 -23
  94. package/src/talkpilot/backgroundToolResults/index.ts +2 -2
  95. package/src/talkpilot/calls/__tests__/callStats.utils.spec.ts +128 -128
  96. package/src/talkpilot/calls/__tests__/calls.dashboard.spec.ts +149 -149
  97. package/src/talkpilot/calls/__tests__/calls.spec.ts +270 -270
  98. package/src/talkpilot/calls/__tests__/calls.statistics.spec.ts +375 -344
  99. package/src/talkpilot/calls/calls.constants.ts +20 -20
  100. package/src/talkpilot/calls/calls.getters.ts +248 -248
  101. package/src/talkpilot/calls/calls.statistics.getters.ts +590 -587
  102. package/src/talkpilot/calls/calls.statistics.types.ts +47 -44
  103. package/src/talkpilot/calls/calls.types.ts +119 -119
  104. package/src/talkpilot/calls/dashboard/calls.dashboard.ts +292 -292
  105. package/src/talkpilot/calls/dashboard/calls.dashboard.types.ts +57 -57
  106. package/src/talkpilot/calls/index.ts +6 -6
  107. package/src/talkpilot/clientAudioBuffers/__tests__/clientAudioBuffer.getters.spec.ts +160 -160
  108. package/src/talkpilot/clientAudioBuffers/clientAudioBuffer.getters.ts +117 -117
  109. package/src/talkpilot/clientAudioBuffers/clientsAudioBuffers.types.ts +25 -25
  110. package/src/talkpilot/clientAudioBuffers/index.ts +2 -2
  111. package/src/talkpilot/clients/clients.getters.ts +16 -16
  112. package/src/talkpilot/clients/clients.types.ts +14 -14
  113. package/src/talkpilot/clients/index.ts +2 -2
  114. package/src/talkpilot/clientsConfig/__tests__/clientsConfig.getters.spec.ts +53 -53
  115. package/src/talkpilot/clientsConfig/__tests__/clientsConfig.spec.ts +190 -190
  116. package/src/talkpilot/clientsConfig/clientsConfig.getters.ts +55 -55
  117. package/src/talkpilot/clientsConfig/clientsConfig.types.ts +127 -127
  118. package/src/talkpilot/clientsConfig/index.ts +2 -2
  119. package/src/talkpilot/contextNotes/__tests__/contextNotes.getters.spec.ts +176 -176
  120. package/src/talkpilot/contextNotes/contextNotes.getters.ts +86 -86
  121. package/src/talkpilot/contextNotes/contextNotes.types.ts +20 -20
  122. package/src/talkpilot/contextNotes/index.ts +2 -2
  123. package/src/talkpilot/flows/__tests__/flows.schema.spec.ts +96 -96
  124. package/src/talkpilot/flows/flows.getter.ts +14 -14
  125. package/src/talkpilot/flows/flows.schema.ts +154 -154
  126. package/src/talkpilot/flows/flows.types.ts +187 -187
  127. package/src/talkpilot/flows/index.ts +2 -2
  128. package/src/talkpilot/groups/__tests__/groups.spec.ts +90 -90
  129. package/src/talkpilot/groups/__tests__/phone.utils.spec.ts +32 -32
  130. package/src/talkpilot/groups/groups.getters.ts +30 -30
  131. package/src/talkpilot/groups/groups.types.ts +29 -29
  132. package/src/talkpilot/groups/index.ts +3 -3
  133. package/src/talkpilot/groups/phone.utils.ts +46 -46
  134. package/src/talkpilot/index.ts +31 -31
  135. package/src/talkpilot/leads/index.ts +2 -2
  136. package/src/talkpilot/leads/leads.getter.ts +6 -6
  137. package/src/talkpilot/leads/leads.schema.ts +33 -33
  138. package/src/talkpilot/leads/leads.types.ts +20 -20
  139. package/src/talkpilot/mongodb-client.ts +78 -78
  140. package/src/talkpilot/phone_numbers/__tests__/phone_numbers.spec.ts +252 -252
  141. package/src/talkpilot/phone_numbers/index.ts +2 -2
  142. package/src/talkpilot/phone_numbers/phone_numbers.getter.ts +158 -158
  143. package/src/talkpilot/phone_numbers/phone_numbers.schema.ts +17 -17
  144. package/src/talkpilot/phone_numbers/phone_numbers.types.ts +30 -30
  145. package/src/talkpilot/plans/__tests__/plans.spec.ts +70 -70
  146. package/src/talkpilot/plans/index.ts +2 -2
  147. package/src/talkpilot/plans/plans.getters.ts +132 -132
  148. package/src/talkpilot/plans/plans.types.ts +89 -89
  149. package/src/talkpilot/products/__tests__/products.getters.spec.ts +45 -45
  150. package/src/talkpilot/products/index.ts +2 -2
  151. package/src/talkpilot/products/products.getters.ts +12 -12
  152. package/src/talkpilot/products/products.types.ts +9 -9
  153. package/src/talkpilot/results/index.ts +7 -7
  154. package/src/talkpilot/results/results.getter.ts +39 -39
  155. package/src/talkpilot/results/results.schema.ts +25 -25
  156. package/src/talkpilot/results/results.types.ts +34 -34
  157. package/src/talkpilot/retry_analyze/__tests__/retryAnalyze.getters.spec.ts +156 -156
  158. package/src/talkpilot/retry_analyze/index.ts +2 -2
  159. package/src/talkpilot/retry_analyze/retryAnalyze.getters.ts +84 -84
  160. package/src/talkpilot/retry_analyze/retryAnalyze.types.ts +13 -13
  161. package/src/talkpilot/sessions/__tests__/sessions.spec.ts +147 -147
  162. package/src/talkpilot/sessions/index.ts +2 -2
  163. package/src/talkpilot/sessions/sessions.getter.ts +92 -92
  164. package/src/talkpilot/sessions/sessions.schema.ts +34 -34
  165. package/src/talkpilot/sessions/sessions.types.ts +30 -30
  166. package/src/talkpilot/subscriptions/__tests__/subscriptions.getters.utils.spec.ts +45 -45
  167. package/src/talkpilot/subscriptions/index.ts +3 -3
  168. package/src/talkpilot/subscriptions/subscriptions.getters.ts +146 -146
  169. package/src/talkpilot/subscriptions/subscriptions.getters.utils.ts +33 -33
  170. package/src/talkpilot/subscriptions/subscriptions.types.ts +66 -66
  171. package/src/talkpilot/utils/__tests__/query.utils.spec.ts +49 -49
  172. package/src/talkpilot/utils/query.utils.ts +21 -21
  173. package/src/test-utils/db-utils.ts +26 -26
  174. package/src/test-utils/factories/index.ts +15 -15
  175. package/src/test-utils/factories/municipal/cities.ts +16 -16
  176. package/src/test-utils/factories/municipal/departmentsSubjects.ts +37 -37
  177. package/src/test-utils/factories/municipal/muniIssues.ts +32 -32
  178. package/src/test-utils/factories/municipal/streets.ts +22 -22
  179. package/src/test-utils/factories/municipal/tickets.ts +39 -39
  180. package/src/test-utils/factories/talkpilot/agents.ts +19 -19
  181. package/src/test-utils/factories/talkpilot/calls.ts +37 -37
  182. package/src/test-utils/factories/talkpilot/clientAudioBuffers.ts +20 -20
  183. package/src/test-utils/factories/talkpilot/clientsConfig.ts +18 -18
  184. package/src/test-utils/factories/talkpilot/contextNotes.ts +33 -33
  185. package/src/test-utils/factories/talkpilot/flows.ts +33 -33
  186. package/src/test-utils/factories/talkpilot/groups.ts +33 -33
  187. package/src/test-utils/factories/talkpilot/phone_numbers.ts +22 -22
  188. package/src/test-utils/factories/talkpilot/sessions.ts +35 -35
  189. package/src/test-utils/factories/websitalk/scans.ts +23 -23
  190. package/src/utils/date.utils.ts +116 -116
  191. package/src/utils/shared.types.ts +4 -4
  192. package/src/utils/validation.ts +23 -23
  193. package/src/websitalk/index.ts +15 -15
  194. package/src/websitalk/mongodb-client.ts +61 -61
  195. package/src/websitalk/scans/__tests__/scans.spec.ts +218 -218
  196. package/src/websitalk/scans/index.ts +2 -2
  197. package/src/websitalk/scans/scans.getters.ts +113 -113
  198. package/src/websitalk/scans/scans.types.ts +53 -53
  199. package/tsconfig.json +23 -23
package/README.md CHANGED
@@ -1,190 +1,257 @@
1
- L1:# @talkpilot/core-db
2
-
3
- `@talkpilot/core-db` is the shared TypeScript database package that wires TalkPilot APIs, municipal CRM integrations, and internal tools to a single, type-safe MongoDB surface. Every repo (MIS, CIS, TalkPilot Server, etc.) imports this package to avoid re-implementing connections, collections, or validation helpers.
4
-
5
- ## Purpose
6
-
7
- - Provide a reliable, multi-domain MongoDB layer for TalkPilot and municipal data.
8
- - Export typed getters, vector search helpers, and document factories so services can focus on behavior instead of schema wiring.
9
- - Manage connection lifecycles, environment configuration, and test helpers from one place so every repo reuses the same plumbing.
10
-
11
- ## Main Concepts
12
-
13
- - **Multi-domain clients** – `src/connection.ts` exposes `mongodbClient` (TalkPilot) and `municipalDataMongodbClient`, each of which resolves `MONGO_URI`, DB overrides, and default names.
14
- - **Domain-specific getters** – `src/talkpilot/` and `src/municipal/` host typed getters (agents, calls, streets, tickets, etc.), vector-search helpers, and service-friendly adapters that keep caller code DRY.
15
- - **Product-specific clients (future)** – While the package currently exposes the shared TalkPilot + municipal clients, we expect each product (CIS, MIS, TalkPilot Server) to eventually get its own domain-specific client helpers or wrappers so the shared core can remain stable while new consumers add targeted extensions.
16
- - **Test helpers** – `src/test-utils/` plus `src/__tests__/` reuse `MongoMemoryServer` and shared factories so tests start with clean data regardless of the consuming repo.
17
- - **Utility layers** – `src/utils/` contains shared validation, pagination, and environment helpers that complement the getters.
18
- - **Environment awareness** – Defaults, fallbacks, and `process.env` lookups ensure local, CI, and Cloud Run clients all connect using the right URI/DB names.
19
-
20
- ## Key Components
21
-
22
- - `src/connection.ts` – Central connection logic that resolves URIs/DB names from env vars (`MONGO_URI`, `MONGODB_URI`, `TALKPILOT_DB_NAME`, `MUNICIPAL_DB_NAME`) and reuses a single `MongoClient`.
23
- - `src/talkpilot/` – Call history, agents, flows, sessions, leads, subscriptions, and support helpers exposed as getters plus helper enums/types for each collection.
24
- - `src/municipal/` – Municipal-specific collections (`cities`, `streets`, `departmentsSubjects`, `tickets`, etc.) plus vector search helpers and Ash Bina helpers used by MIS.
25
- - `src/utils/` – Shared helpers such as `resolveConnection`, pagination utilities, and schema validation helper functions.
26
- - `src/test-utils/` and `src/__tests__/` – Utilities that bootstrap `MongoMemoryServer`, expose factories, and make sure Jest environments can stub database calls predictably.
27
- - `dist/` – Compiled output consumed by downstream repos (CJS + ESM + type defs).
28
-
29
- ## Domain APIs
30
-
31
- - **TalkPilot domain** – Imports like `findAgents`, `getFlows`, `findCalls`, and `vectorSearchCalls` live in `src/talkpilot`. These functions are the canonical access pattern for call history, session metadata, and provider configs.
32
- - **Municipal domain** – Helpers such as `findStreets`, `getMunicipalCities`, `findDepartmentSubjects`, and `createTicket` live under `src/municipal` and feed MIS workflows (street hints, subject matching, Ash Bina tickets).
33
-
34
- ## Environment variables
35
-
36
- | Variable | Purpose | Required |
37
- |---------------------------|--------------------------------------------------------------------------------------|----------|
38
- | `MONGO_URI` | Primary MongoDB connection string for every domain (overridden by `MONGODB_URI`). | ✅ |
39
- | `MONGODB_URI` | Alternate connection string used when Mongo needs a second URI parameter. | ✅ |
40
- | `TALKPILOT_DB_NAME` | Optional override for the TalkPilot database name (defaults from URI path). | ❌ |
41
- | `MUNICIPAL_DB_NAME` | Optional override for the municipal database name (defaults to `municipal-data`). | ❌ |
42
- | `ENV` | Free-form label used in logs/validators (defaults to `unknown`). | ❌ |
43
-
44
- If you pass a `uri` directly to `mongodbClient.connect()` or `municipalDataMongodbClient.connect()`, the client will prefer that value over the env vars.
45
-
46
- ## Getting Started
47
-
48
- ### Prerequisites
49
-
50
- - Node.js 22.x+ (aligns with downstream services).
51
- - npm 11+ or Yarn.
52
- - MongoDB accessible from your environment or a `MongoMemoryServer` for tests.
53
-
54
- ### Setup
55
-
56
- 1. Clone the repo and install dependencies:
57
-
58
- ```bash
59
- git clone https://github.com/talkpilot/core-db.git
60
- cd core-db
61
- npm install
62
- ```
63
-
64
- 2. Build the package before using it locally:
65
-
66
- ```bash
67
- npm run build
68
- ```
69
-
70
- 3. Import `@talkpilot/core-db` from another project by pointing `package.json` at the local path during development or installing the published release.
71
-
72
- ## Sample `.env`
73
-
74
- ```
75
- MONGO_URI=mongodb://localhost:27017
76
- TALKPILOT_DB_NAME=talkpilot-dev
77
- MUNICIPAL_DB_NAME=municipal-dev
78
- ENV=development
79
- ```
80
-
81
- Adjust `MONGO_URI` to match the running Mongo instance and configure `talkpilot`/`municipal` DB names if you want to keep them separate.
82
-
83
- ## Local development
84
-
85
- 1. Run `npm install`.
86
- 2. Build the compiled output: `npm run build`.
87
- 3. Execute tests: `npm run test`.
88
- 4. Use `npm link` or `npm pack` to consume the freshly built package from other repos (`CIS`, `MIS`, `TalkPilot Server`).
89
-
90
- ## Development guide
91
-
92
- `DEVELOPMENT.md` contains the tactical steps for contributors. At a glance:
93
-
94
- - Node 18+/TypeScript is required (aligns with downstream services).
95
- - Run `npm install` → `npm run build` after cloning.
96
- - Use `npm link`/`npm link @talkpilot/core-db` to test the package locally before publishing.
97
- - When adding getters, define types, implement the function, export it through the domain `index.ts`, and add a corresponding test under the domain’s `__tests__` folder.
98
- - Always rely on the provided test factories (`src/test-utils/factories`) to seed data so tests remain consistent.
99
- - Jest with `mongodb-memory-server` is the only execution path we have to verify this core utility—unit tests are the safety net for every change.
100
-
101
- Refer to `DEVELOPMENT.md` for the full walkthrough, token instructions, and factory samples.
102
-
103
- ## 🧪 Testing
104
-
105
- - `npm run test` – Jest suite (factories, utils, integration mocks) powered by `mongodb-memory-server`.
106
- - Tests rely on `src/__tests__/setup.ts` to bootstrap the in-memory Mongo instances and wire shared factories/helpers before each run.
107
- - When adding getters, helpers, or domain logic, create focused coverage inside the consuming domain’s `__tests__/` folder and use the provided factories to keep fixtures consistent.
108
-
109
- `@talkpilot/core-db` does not run in a product UI or feature branch—unit tests are the *only* reliable execution path to ensure your changes work. Every change must ship with a unit test so downstream repos can upgrade without surprises; treat the test suite as the canonical safety net for this core utility package.
110
-
111
- ## 🧹 Lint & build verification
112
-
113
- - `npm run lint` – Run ESLint over `src/**/*.{ts,tsx}`.
114
- - `npm run format` – Format the source files with Prettier.
115
- - `npm run build` – Compile TypeScript and emit `dist/` (used by downstream consumers).
116
-
117
- ## ✅ Pre-push checklist
118
-
119
- 1. `npm run build`.
120
- 2. `npm run test`.
121
- 3. `npm run format`.
122
-
123
- ## Publishing & release notes
124
-
125
- - Releases are handled by `npm version <patch|minor|major>` followed by `npm publish`. The package is a **private `@talkpilot` dependency**, so every contributor must install the shared npm automation token into their global `~/.npmrc` before running publish or `npm install`.
126
- - The shared token is rotated periodically—if you see authentication failures, request the refreshed token, update your `~/.npmrc`, and retry. Never commit credentials to source control.
127
- - After publishing, downstream repos (`CIS`, `MIS`, `TalkPilot Server`, etc.) should run `npm update @talkpilot/core-db` so they receive the latest helpers/bug fixes.
128
- - Cloud Build & Cloud Run jobs that depend on this package pick up the new version the next time they rebuild their container; the services pull the compiled `dist/` output and type definitions when installing the dependency.
129
-
130
- - Release process, when to publish, branch hygiene, and the function-signature versioning policy: see [`DEVELOPMENT.md`](./DEVELOPMENT.md).
131
-
132
- ### Version history
133
-
134
- | Version | Note |
135
- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
136
- | 1.1.9 | Maintenance release (build, test, format). |
137
- | 1.3.4 | Added new statistics getters for call & ticket dashboards (CP-149): summary, trend, hourly, and routing for calls, plus open/draft/subject ticket stats. **Removed** the legacy getters `getTicketsCountByCityAndDateRange` and `getTicketsSubjectStats`. |
138
- | 1.3.5 | Added the `muniIssues` collection & module (CP-1115): the `MuniIssue` document type, `createMuniIssue` (accepts a caller-supplied `_id` for a two-way Jira link), `generateMuniIssueId`, `getMuniIssueById` / `getMuniIssueByJiraKey`, a `$jsonSchema` validator with indexes (unique `jira.key`), and `ensureMuniIssuesCollection` to provision it at boot. |
139
-
140
-
141
- ### 1.1.9
142
-
143
- 1. `npm run build`.
144
- 2. `npm run test`.
145
- 3. `npm run format`.
146
-
147
- ### 1.3.4 Call & ticket statistics (CP-149)
148
-
149
- New call statistics getters for dashboards (summary, trend, hourly, routing) and ticket statistics scoped to the same date range (open tickets, draft tickets, subject breakdowns).
150
-
151
- **Removed**
152
-
153
- | Removed | Replacement |
154
- | --- | --- |
155
- | `getTicketsCountByCityAndDateRange` | `aggregateCallsSummary` + `findCallSidsWithTicketsByCity` |
156
- | `getTicketsSubjectStats` | `findSubjectsByCityAndDateRange` |
157
- | `SubjectStatsItem` | `SubjectItem` |
158
-
159
- **Do I need to update my app?**
160
-
161
- Yes — if you used any of the removed functions above.
162
-
163
- ### 1.3.5 — Muni issues (CP-1115)
164
-
165
- New `muniIssues` module under `src/municipal/` for issues reported from the call-log screen. The document is written to the municipal DB after its Jira ticket is created.
166
-
167
- **Added**
168
-
169
- | Export | Purpose |
170
- | --- | --- |
171
- | `MuniIssue` / `CreateMuniIssueInput` | Document type and insert-input type (`issueContent` + `jira` domains). |
172
- | `createMuniIssue(input, id)` | Insert an issue; requires a caller-supplied `_id` from MIS (via `generateMuniIssueId`) so it matches the id embedded in the Jira ticket (two-way link). |
173
- | `generateMuniIssueId()` | Pre-generate the `_id` to pass to `createMuniIssue`. |
174
- | `getMuniIssueById` / `getMuniIssueByJiraKey` | Lookups by our id or by the Jira key. |
175
- | `ensureMuniIssuesCollection()` | Provision the `$jsonSchema` validator and indexes (unique `jira.key`); run once at MIS boot. |
176
-
177
- **Do I need to update my app?**
178
-
179
- No — purely additive. New consumers (MIS) import these from `@talkpilot/core-db`.
180
-
181
- ## 🛠 CI/CD & deployment
182
-
183
- - This package is consumed by Cloud Build-based services (TalkPilot Server, MIS, CIS) as a dependency when Docker images are built. Keep `dist/` in sync with your builds because the compiled artifact is what downstream services install.
184
- - Releases require the shared npm token documented in `DEVELOPMENT.md`; consult that guide for contribution, linking, and token rotation procedures.
185
-
186
- ## Resources
187
-
188
- - [DEVELOPMENT.md](./DEVELOPMENT.md) (setup, tooling, publishing, token management)
189
- - [src/test-utils](src/test-utils) and the `__tests__` folder for examples of `MongoMemoryServer` wiring.
190
-
1
+ L1:# @talkpilot/core-db
2
+
3
+ `@talkpilot/core-db` is the shared TypeScript database package that wires TalkPilot APIs, municipal CRM integrations, and internal tools to a single, type-safe MongoDB surface. Every repo (MIS, CIS, TalkPilot Server, etc.) imports this package to avoid re-implementing connections, collections, or validation helpers.
4
+
5
+ ## Purpose
6
+
7
+ - Provide a reliable, multi-domain MongoDB layer for TalkPilot and municipal data.
8
+ - Export typed getters, vector search helpers, and document factories so services can focus on behavior instead of schema wiring.
9
+ - Manage connection lifecycles, environment configuration, and test helpers from one place so every repo reuses the same plumbing.
10
+
11
+ ## Main Concepts
12
+
13
+ - **Multi-domain clients** – `src/connection.ts` exposes `mongodbClient` (TalkPilot) and `municipalDataMongodbClient`, each of which resolves `MONGO_URI`, DB overrides, and default names.
14
+ - **Domain-specific getters** – `src/talkpilot/` and `src/municipal/` host typed getters (agents, calls, streets, tickets, etc.), vector-search helpers, and service-friendly adapters that keep caller code DRY.
15
+ - **Product-specific clients (future)** – While the package currently exposes the shared TalkPilot + municipal clients, we expect each product (CIS, MIS, TalkPilot Server) to eventually get its own domain-specific client helpers or wrappers so the shared core can remain stable while new consumers add targeted extensions.
16
+ - **Test helpers** – `src/test-utils/` plus `src/__tests__/` reuse `MongoMemoryServer` and shared factories so tests start with clean data regardless of the consuming repo.
17
+ - **Utility layers** – `src/utils/` contains shared validation, pagination, and environment helpers that complement the getters.
18
+ - **Environment awareness** – Defaults, fallbacks, and `process.env` lookups ensure local, CI, and Cloud Run clients all connect using the right URI/DB names.
19
+
20
+ ## Key Components
21
+
22
+ - `src/connection.ts` – Central connection logic that resolves URIs/DB names from env vars (`MONGO_URI`, `MONGODB_URI`, `TALKPILOT_DB_NAME`, `MUNICIPAL_DB_NAME`) and reuses a single `MongoClient`.
23
+ - `src/talkpilot/` – Call history, agents, flows, sessions, leads, subscriptions, and support helpers exposed as getters plus helper enums/types for each collection.
24
+ - `src/municipal/` – Municipal-specific collections (`cities`, `streets`, `departmentsSubjects`, `tickets`, etc.) plus vector search helpers and Ash Bina helpers used by MIS.
25
+ - `src/utils/` – Shared helpers such as `resolveConnection`, pagination utilities, and schema validation helper functions.
26
+ - `src/test-utils/` and `src/__tests__/` – Utilities that bootstrap `MongoMemoryServer`, expose factories, and make sure Jest environments can stub database calls predictably.
27
+ - `dist/` – Compiled output consumed by downstream repos (CJS + ESM + type defs).
28
+
29
+ ## Domain APIs
30
+
31
+ - **TalkPilot domain** – Imports like `findAgents`, `getFlows`, `findCalls`, and `vectorSearchCalls` live in `src/talkpilot`. These functions are the canonical access pattern for call history, session metadata, and provider configs.
32
+ - **Municipal domain** – Helpers such as `findStreets`, `getMunicipalCities`, `findDepartmentSubjects`, and `createTicket` live under `src/municipal` and feed MIS workflows (street hints, subject matching, Ash Bina tickets).
33
+
34
+ ## Environment variables
35
+
36
+ | Variable | Purpose | Required |
37
+ |---------------------------|--------------------------------------------------------------------------------------|----------|
38
+ | `MONGO_URI` | Primary MongoDB connection string for every domain (overridden by `MONGODB_URI`). | ✅ |
39
+ | `MONGODB_URI` | Alternate connection string used when Mongo needs a second URI parameter. | ✅ |
40
+ | `TALKPILOT_DB_NAME` | Optional override for the TalkPilot database name (defaults from URI path). | ❌ |
41
+ | `MUNICIPAL_DB_NAME` | Optional override for the municipal database name (defaults to `municipal-data`). | ❌ |
42
+ | `ENV` | Free-form label used in logs/validators (defaults to `unknown`). | ❌ |
43
+
44
+ If you pass a `uri` directly to `mongodbClient.connect()` or `municipalDataMongodbClient.connect()`, the client will prefer that value over the env vars.
45
+
46
+ ## Getting Started
47
+
48
+ ### Prerequisites
49
+
50
+ - Node.js 22.x+ (aligns with downstream services).
51
+ - npm 11+ or Yarn.
52
+ - MongoDB accessible from your environment or a `MongoMemoryServer` for tests.
53
+
54
+ ### Setup
55
+
56
+ 1. Clone the repo and install dependencies:
57
+
58
+ ```bash
59
+ git clone https://github.com/talkpilot/core-db.git
60
+ cd core-db
61
+ npm install
62
+ ```
63
+
64
+ 2. Build the package before using it locally:
65
+
66
+ ```bash
67
+ npm run build
68
+ ```
69
+
70
+ 3. Import `@talkpilot/core-db` from another project by pointing `package.json` at the local path during development or installing the published release.
71
+
72
+ ## Sample `.env`
73
+
74
+ ```
75
+ MONGO_URI=mongodb://localhost:27017
76
+ TALKPILOT_DB_NAME=talkpilot-dev
77
+ MUNICIPAL_DB_NAME=municipal-dev
78
+ ENV=development
79
+ ```
80
+
81
+ Adjust `MONGO_URI` to match the running Mongo instance and configure `talkpilot`/`municipal` DB names if you want to keep them separate.
82
+
83
+ ## Local development
84
+
85
+ 1. Run `npm install`.
86
+ 2. Build the compiled output: `npm run build`.
87
+ 3. Execute tests: `npm run test`.
88
+ 4. Use `npm link` or `npm pack` to consume the freshly built package from other repos (`CIS`, `MIS`, `TalkPilot Server`).
89
+
90
+ ## Development guide
91
+
92
+ `DEVELOPMENT.md` contains the tactical steps for contributors. At a glance:
93
+
94
+ - Node 18+/TypeScript is required (aligns with downstream services).
95
+ - Run `npm install` → `npm run build` after cloning.
96
+ - Use `npm link`/`npm link @talkpilot/core-db` to test the package locally before publishing.
97
+ - When adding getters, define types, implement the function, export it through the domain `index.ts`, and add a corresponding test under the domain’s `__tests__` folder.
98
+ - Always rely on the provided test factories (`src/test-utils/factories`) to seed data so tests remain consistent.
99
+ - Jest with `mongodb-memory-server` is the only execution path we have to verify this core utility—unit tests are the safety net for every change.
100
+
101
+ Refer to `DEVELOPMENT.md` for the full walkthrough, token instructions, and factory samples.
102
+
103
+ ## 🧪 Testing
104
+
105
+ - `npm run test` – Jest suite (factories, utils, integration mocks) powered by `mongodb-memory-server`.
106
+ - Tests rely on `src/__tests__/setup.ts` to bootstrap the in-memory Mongo instances and wire shared factories/helpers before each run.
107
+ - When adding getters, helpers, or domain logic, create focused coverage inside the consuming domain’s `__tests__/` folder and use the provided factories to keep fixtures consistent.
108
+
109
+ `@talkpilot/core-db` does not run in a product UI or feature branch—unit tests are the *only* reliable execution path to ensure your changes work. Every change must ship with a unit test so downstream repos can upgrade without surprises; treat the test suite as the canonical safety net for this core utility package.
110
+
111
+ ## 🧹 Lint & build verification
112
+
113
+ - `npm run lint` – Run ESLint over `src/**/*.{ts,tsx}`.
114
+ - `npm run format` – Format the source files with Prettier.
115
+ - `npm run build` – Compile TypeScript and emit `dist/` (used by downstream consumers).
116
+
117
+ ## ✅ Pre-push checklist
118
+
119
+ 1. `npm run build`.
120
+ 2. `npm run test`.
121
+ 3. `npm run format`.
122
+
123
+ ## Publishing & release notes
124
+
125
+ - Releases are handled by `npm version <patch|minor|major>` followed by `npm publish`. The package is a **private `@talkpilot` dependency**, so every contributor must install the shared npm automation token into their global `~/.npmrc` before running publish or `npm install`.
126
+ - The shared token is rotated periodically—if you see authentication failures, request the refreshed token, update your `~/.npmrc`, and retry. Never commit credentials to source control.
127
+ - After publishing, downstream repos (`CIS`, `MIS`, `TalkPilot Server`, etc.) should run `npm update @talkpilot/core-db` so they receive the latest helpers/bug fixes.
128
+ - Cloud Build & Cloud Run jobs that depend on this package pick up the new version the next time they rebuild their container; the services pull the compiled `dist/` output and type definitions when installing the dependency.
129
+
130
+ - Release process, when to publish, branch hygiene, and the function-signature versioning policy: see [`DEVELOPMENT.md`](./DEVELOPMENT.md).
131
+
132
+ ### Version history
133
+
134
+ | Version | Note |
135
+ | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
136
+ | 1.1.9 | Maintenance release (build, test, format). |
137
+ | 1.3.4 | Added new statistics getters for call & ticket dashboards (CP-149): summary, trend, hourly, and routing for calls, plus open/draft/subject ticket stats. **Removed** the legacy getters `getTicketsCountByCityAndDateRange` and `getTicketsSubjectStats`. |
138
+ | 1.3.5 | Added the `muniIssues` collection & module (CP-1115): the `MuniIssue` document type, `createMuniIssue` (accepts a caller-supplied `_id` for a two-way Jira link), `generateMuniIssueId`, `getMuniIssueById` / `getMuniIssueByJiraKey`, a `$jsonSchema` validator with indexes (unique `jira.key`), and `ensureMuniIssuesCollection` to provision it at boot. |
139
+ | 1.3.6 | Added `contextNotes` and `products` modules under `src/talkpilot/`. |
140
+ | 1.3.7 | Added `disableGenericPrompt` optional boolean field to the `Flow` type and schema (CP-704). |
141
+ | 1.3.8 | Ticket-count Map getters (CP-1200): deprecated `string[]` ticket getters and `CallsStatsFilter` fields, added `Map<string, number>` replacements; fixed ticket counting in `aggregateCallsSummary`. |
142
+ | 1.3.9 | Fixed wrap-around hour filter boundary guards (CP-1166): excluded first-day morning calls and last-day evening calls from wrap-around windows. |
143
+
144
+ ### 1.1.9
145
+
146
+ 1. `npm run build`.
147
+ 2. `npm run test`.
148
+ 3. `npm run format`.
149
+
150
+ ### 1.3.4 — Call & ticket statistics (CP-149)
151
+
152
+ New call statistics getters for dashboards (summary, trend, hourly, routing) and ticket statistics scoped to the same date range (open tickets, draft tickets, subject breakdowns).
153
+
154
+ **Removed**
155
+
156
+ | Removed | Replacement |
157
+ | --- | --- |
158
+ | `getTicketsCountByCityAndDateRange` | `aggregateCallsSummary` + `findCallSidTicketCountsByCity` — note: returns `Map<string, number>`, not `string[]` |
159
+ | `getTicketsSubjectStats` | `findSubjectsByCityAndDateRange` |
160
+ | `SubjectStatsItem` | `SubjectItem` |
161
+
162
+ **Do I need to update my app?**
163
+
164
+ Yes — if you used any of the removed functions above.
165
+
166
+ ### 1.3.5 — Muni issues (CP-1115)
167
+
168
+ New `muniIssues` module under `src/municipal/` for issues reported from the call-log screen. The document is written to the municipal DB after its Jira ticket is created.
169
+
170
+ **Added**
171
+
172
+ | Export | Purpose |
173
+ | --- | --- |
174
+ | `MuniIssue` / `CreateMuniIssueInput` | Document type and insert-input type (`issueContent` + `jira` domains). |
175
+ | `createMuniIssue(input, id)` | Insert an issue; requires a caller-supplied `_id` from MIS (via `generateMuniIssueId`) so it matches the id embedded in the Jira ticket (two-way link). |
176
+ | `generateMuniIssueId()` | Pre-generate the `_id` to pass to `createMuniIssue`. |
177
+ | `getMuniIssueById` / `getMuniIssueByJiraKey` | Lookups by our id or by the Jira key. |
178
+ | `ensureMuniIssuesCollection()` | Provision the `$jsonSchema` validator and indexes (unique `jira.key`); run once at MIS boot. |
179
+
180
+ **Do I need to update my app?**
181
+
182
+ No — purely additive. New consumers (MIS) import these from `@talkpilot/core-db`.
183
+
184
+ ### 1.3.6 Context Notes & Products
185
+
186
+ **Context Notes** — new `contextNotes` collection under `src/talkpilot/` for per-client, per-product notes injected into AI calls at runtime. Each document holds a `systemPrompt` and a list of time-bounded entries (`activeFrom`, `expiresAt`). Two setters with split ownership: `setContextNoteEntries` for the notes list and `setContextNoteConfig` for `systemPrompt` and `product`.
187
+
188
+ **Products** — new `products` collection under `src/talkpilot/` cataloguing TalkPilot products. Each document has a stable `name` and an optional `displayName` locale map (`Record<string, string>`) for UI display.
189
+
190
+ **Added**
191
+
192
+ | Export | Purpose |
193
+ | --- | --- |
194
+ | `getContextNotes(clientId, product)` | Fetch context note documents for a client/product pair. |
195
+ | `createContextNote(input)` | Insert a new context note document. |
196
+ | `setContextNoteEntries(id, clientId, notes)` | Replace the notes list on an existing document. |
197
+ | `setContextNoteConfig(id, clientId, config)` | Update `systemPrompt` and/or `product` on an existing document. |
198
+ | `getAllProducts()` | Fetch the full product catalogue. |
199
+
200
+ **Do I need to update my app?**
201
+
202
+ No — purely additive.
203
+
204
+ ### 1.3.7 — Generic prompt control (CP-704)
205
+
206
+ Added an optional boolean field to the `Flow` type and MongoDB schema.
207
+
208
+ **Added**
209
+
210
+ | Export | Purpose |
211
+ | --- | --- |
212
+ | `Flow.disableGenericPrompt` | Optional `boolean`. When `true`, disables the generic prompt for the flow. Existing flows without this field continue to work unchanged. |
213
+
214
+ **Do I need to update my app?**
215
+
216
+ No — the field is optional and fully backwards-compatible.
217
+
218
+ ### 1.3.8 — Ticket-count Map getters (CP-1200)
219
+
220
+ Adds `Map<string, number>` ticket getters that return per-call ticket counts. The old `string[]` getters and filter fields are kept and marked `@deprecated`.
221
+
222
+ **Deprecated → Replacement**
223
+
224
+ | Deprecated | Replacement |
225
+ | --- | --- |
226
+ | `findCallSidsWithTicketsByCity` | `findCallSidTicketCountsByCity` → `Map<string, number>` |
227
+ | `findCallSidsWithDraftTicketsByCity` | `findCallSidDraftTicketCountsByCity` → `Map<string, number>` |
228
+ | `CallsStatsFilter.callSidsWithTickets` | `CallsStatsFilter.callSidTicketCounts` |
229
+ | `CallsStatsFilter.callSidsWithDraftTickets` | `CallsStatsFilter.callSidDraftTicketCounts` |
230
+
231
+ **Do I need to update my app?**
232
+
233
+ No — the old fields still compile. Migrate when convenient.
234
+
235
+ ### 1.3.9 — Wrap-around hour filter boundary guards (CP-1166)
236
+
237
+ When `hourFrom` > `hourTo` (e.g. 23:00–13:00), two silent miscounting bugs existed:
238
+
239
+ - **First-day morning**: calls before `hourTo` on the first day (e.g. 08:00 on Jun 8) were counted even though the window starts at `hourFrom` that day. Fixed by adding `dateLocal > startStr` to the after-midnight leg.
240
+ - **Last-day evening**: calls at or after `hourFrom` on the last day (e.g. 23:30 on Jun 9) were counted even though the window ends at `hourTo` that day. Fixed by adding `dateLocal < endStr` to the before-midnight leg.
241
+
242
+ **Affects:** `aggregateCallsTrend`, `aggregateCallsSummary`, `aggregateCallsHourlyByRange`, `aggregateCallsRouting`, `findFilteredCallSids`, `findSubjectsByCityAndDateRange`.
243
+
244
+ **Do I need to update my app?**
245
+
246
+ No API changes — both fixes correct silent miscounting. Upgrading is recommended if you use a wrap-around hour window.
247
+
248
+ ## 🛠 CI/CD & deployment
249
+
250
+ - This package is consumed by Cloud Build-based services (TalkPilot Server, MIS, CIS) as a dependency when Docker images are built. Keep `dist/` in sync with your builds because the compiled artifact is what downstream services install.
251
+ - Releases require the shared npm token documented in `DEVELOPMENT.md`; consult that guide for contribution, linking, and token rotation procedures.
252
+
253
+ ## Resources
254
+
255
+ - [DEVELOPMENT.md](./DEVELOPMENT.md) (setup, tooling, publishing, token management)
256
+ - [src/test-utils](src/test-utils) and the `__tests__` folder for examples of `MongoMemoryServer` wiring.
257
+