@terreno/ai 58.0.0-beta.1 → 58.0.0-beta.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -20,6 +20,52 @@ Upgrade notes for consumer action live in [`mcp-server/src/docs/upgrades/`](mcp-
20
20
 
21
21
  Unreleased changes live in [`changelog/unreleased/`](changelog/unreleased/). Add one Markdown file per feature (see that directory's README) instead of editing this section.
22
22
 
23
+ ## [58.0.0-beta.2] - 2026-11-03
24
+
25
+ Upgrade note: [`mcp-server/src/docs/upgrades/58.0.0-beta.2.md`](mcp-server/src/docs/upgrades/58.0.0-beta.2.md).
26
+
27
+ ### Breaking
28
+
29
+ - Terreno 58 targets Expo SDK 58 (`~58.0.14`), React Native `0.88.0`, and React `19.3.0`. `@terreno/ui` peers `react-native` `0.88.0`. Native catalog moves with the SDK: `react-native-gesture-handler` `~3.3.0`, `react-native-reanimated` `4.7.1` + `react-native-worklets` `0.13.0`, `react-native-screens` `~4.30.0`, `react-native-webview` `14.0.1`, `@sentry/react-native` `~8.28.0`, `@shopify/flash-list` `2.3.2`, `@shopify/react-native-skia` `2.14.0`. Rebuild dev clients and store binaries.
30
+ - Expo Router 58: set `export const unstable_settings = {initialRouteName}` on the layout instead of passing `initialRouteName` to `<Slot>` / navigators. `SidebarNavigation` ignores the prop.
31
+ - Expo Router 58 static rendering: a custom `app/+html.tsx` must render `headNodes` and `bodyNodes` from `useServerDocumentContext()` (`expo-router/html`), or exported pages ship without their JavaScript. `example-frontend` is updated.
32
+ - React Native 0.88 generated types: type refs as `ViewInstance`, `TextInstance`, `ScrollViewInstance`, and `TextInputInstance` (`Box.scrollRef`, `TextField.inputRef`, `Text` `ref`).
33
+ - `create-terreno-app` scaffolds Expo `~58.0.14`, React Native `0.88.0`, and React `19.3.0`.
34
+ - The Terreno 58 native module baseline lands in `example-frontend` and `demo`: `@stripe/stripe-react-native` 0.81.0, `react-native-purchases` and `react-native-purchases-ui` 10.12.2, `expo-device`, `expo-crypto`, `expo-local-authentication`, `expo-system-ui`, and `expo-build-properties` (Android Kotlin 2.2.21 and `compileSdkVersion` 37 pins). Rebuild every dev client and store binary. `react-native-otp-verify` is left out because it does not support the New Architecture. The demo has a hidden **Native Modules** page that shows each module's version and whether the running binary links it. See the 58.0.0 upgrade note.
35
+ - `TerrenoApp` turns HTTP rate limiting on by default. Omitting `rateLimit` now mounts the limiter with the memory store (auth 20 / 15 min, api 600 / 15 min). Pass `rateLimit: false` to turn it off. On Cloud Run pass `rateLimit: {trustProxy: 1}` so unauthenticated traffic keys by client IP. See the 58.0.0 upgrade note.
36
+ - `@terreno/admin-frontend` no longer takes an RTK `api`. The `api` prop is removed from every admin component (`AdminProvider`, `AdminShellLayout`, `AdminHome`, `AdminModelList`, `AdminModelTable`, `AdminModelForm`, `AdminScreenRouter`, `ConfigurationScreen`, `DocumentStorageBrowser`, `ConsentHistory`, comms, jobs, org, consent, and announcement screens, and widget props), and admin hooks drop their `api` argument (`useAdminConfig(apiBase)`, `useAdminScripts(apiBase)`, `useAdminRoles(apiBase)`, `useConfigurationApi({basePath})`, `useDocumentStorageApi(basePath)`, `useConsentHistory(baseUrl)`, `useOrganizationsApi(basePath)`, `useCommsDashboardApi()`, `useJobsDashboardApi()`, and the `useAiObservability*Api()` hooks).
37
+ - `useAdminApi(api, routePath, modelName)` is removed. Use `useAdminModelApi(routePath, modelName)`, which keeps the same hook names and calls the same `/admin/{routePath}` URLs through the admin fetch client. `AdminApi`, `EndpointBuilder`, and admin `injectEndpoints` are gone; `@reduxjs/toolkit` and `react-redux` are no longer peers of admin-frontend.
38
+ - `AdminProvider` is now the required auth binding for every admin request, including ObjectId and API-only model CRUD. Cookie SPAs pass `credentials="same-origin"`; embedded apps pass `apiOrigin` and `getAuthHeaders`. Product screens that reuse admin components wrap them in `AdminProvider`. See the 58.0.0 upgrade guide.
39
+ - Removed the legacy RTK realtime path. Use `sync` on `modelRouter` with `@terreno/syncdb` for live collection updates. `RealtimeApp`, `SyncApp`, socket auth, `sync:delta`, and admin `admin:model.changed` events are unchanged. See the 58.0.0 upgrade note.
40
+ - `@terreno/api`: `modelRouter` no longer accepts `realtime` (now a TypeScript error), and the change stream no longer emits the `sync` websocket event. Removed `RealtimeConfig`, `RealtimeEvent`, `DocumentSubscription`, `QuerySubscription`, `registerRealtime`, `getRealtimeRegistry`, `findRegistryEntryByCollection`, `findRegistryEntryByRoutePath`, `updateRealtimeRegistryOptions`, `clearRealtimeRegistry`, `RealtimeRegistryEntry`, the query-subscription store (`addQuerySubscription`, `removeQuerySubscription`, `removeAllSocketQueries`, `getQuerySubscriptionsForCollection`, `computeQueryId`, `clearQueryStore`), and `MAX_MODEL_SUBSCRIPTIONS` / `MAX_DOCUMENT_SUBSCRIPTIONS` / `MAX_QUERY_SUBSCRIPTIONS`. Sockets no longer handle `subscribe:model`, `subscribe:document`, or `subscribe:query`.
41
+ - `@terreno/rtk`: removed `realtimeList`, `realtimeDocument`, `setRealtimeSocket`, `getRealtimeSocket`, `useSyncConnection`, and the `RealtimeEvent` type. The rest of `@terreno/rtk` (including `useSocketConnection` and `useRealtimeDebug`) is removed in the same release; see `remove-rtk.md`.
42
+ - Removed `@terreno/rtk` and every Redux dependency (`@reduxjs/toolkit`, `react-redux`, `redux-persist`, `@rtk-query/codegen-openapi`). `@terreno/rtk` is not published for 58. `@terreno/syncdb` is the only frontend data, request, and session layer. Follow `docs/how-to/migrate-rtk-to-syncdb.md`, which maps every removed export to its replacement, and the 58.0.0 upgrade note.
43
+ - `@terreno/syncdb` adds `@terreno/syncdb/api` (`createApiClient`, `ApiError`), request hooks in `@terreno/syncdb/react` (`ApiClientProvider`, `useApiQuery`, `useLazyApiQuery`, `useApiMutation`, `useSocketEvent`), `@terreno/syncdb/config` (`baseUrl`, `baseWebsocketsUrl`, `appHeaders`), `@terreno/syncdb/auth` (`createBetterAuthClient`, `SessionProvider`, `useSession`, `jwtAuthAdapter`, `useJwtAuth`, token-expiry helpers), `@terreno/syncdb/flags` (`useFeatureFlags`, `useTerrenoFeatureFlags` on the sync socket), `@terreno/syncdb/app` (`useUpgradeCheck`, `hasPermission`, `canOpenAdminPage`, `installDevConsoleLogger`, `useMCPTools`), and `@terreno/syncdb/buildNumber`.
44
+ - `terreno-syncdb-codegen --api-out` generates typed hooks for every non-synced OpenAPI operation (same names as the RTK codegen), replacing `@rtk-query/codegen-openapi`. `bun run sdk` writes `store/apiSdk.ts`.
45
+ - `@terreno/ui` `ConsentNavigator`, `AnnouncementNavigator`, `useConsentForms`, `useSubmitConsent`, `useAnnouncements`, and `useAcknowledgeAnnouncement` take a request `client` instead of an RTK `api`.
46
+ - `GptHistory` (`gpt/histories`) is a synced collection and keeps its ObjectId `_id` (no migration). `McpServiceToken` (`mcp/service-tokens`, read-only) is a synced collection with String ids; run `mcpServiceTokenStringIdsMigration` (`@terreno/api`) before deploying 58 frontends.
47
+ - `create-terreno-app`, the `terreno_bootstrap` prompt, and the `terreno_generate_screen` tool scaffold apps without Redux or `@terreno/rtk`. The terreno-mcp-local `get_rtk_state` tool, the MCP resource `terreno://docs/rtk`, and `terreno state --slice rtk` are removed; use `get_syncdb_state` and `terreno state`.
48
+
49
+ ### Added
50
+
51
+ - `DataTable` columns size themselves when `width` is omitted. Each auto column starts at the wider of its header and its content (sampled from at most 50 rows, measured with a canvas on web and estimated on native), clamped to 80–360px, and auto columns grow to fill the table's container. Columns that set `width` keep it. `DataTableColumn.width` is now optional.
52
+ - `@terreno/syncdb` `createSyncDb({idFormats})` mints 24-hex ObjectIds for a collection set to `"objectId"`, so a model that keeps Mongoose's default ObjectId `_id` can sync without migrating its ids. `mutate()` throws on an explicit non-hex id for such a collection. `generateObjectId` and the `SyncIdFormat` type are exported.
53
+ - Sync `validation` nacks carry an optional `reason`: `invalid_id` when an ObjectId-keyed collection gets a non-hex id (rejected before the mutation is claimed), and `id_collision` when a create names an `_id` another document holds. A collision never touches the existing document and its message never names the holder. The ledger records the reason (`SyncMutation.nackReason`) so a redelivery replays it.
54
+ - `GET /sync/entities` leaves out non-hex ids for an ObjectId-keyed model instead of answering 400, so a rejected create queued for repair cannot fail the rest of the repair batch.
55
+
56
+ ### Changed
57
+
58
+ - The repository's agent rules now generate with rulesync 17 (up from 9). The root rule `.rulesync/rules/00-root.md` now generates `AGENTS.md`, which Devin and other AGENTS.md readers load, in place of `.devin/rules/00-root.md`. Never edit `AGENTS.md` by hand: edit the root rule and run `bun run rules`. The examples that existed only in the old hand-edited `AGENTS.md` (syncdb and RTK SDK usage) moved into the root rule, so `CLAUDE.md`, the Cursor root rule, and Copilot instructions get them too. Links in the root rule now resolve from the repository root. Generated Copilot hooks use the portable `command` field. Generated Copilot subagents no longer list `agent/runSubagent` as their only tool, so they get every default tool. Generated skills carry their `license`, `disable-model-invocation`, and Devin `triggers` frontmatter.
59
+ - `@terreno/syncdb/auth` `SessionProvider`: `logout()` no longer clears the session before the server answers. The session stays in the state until `authClient.signOut()` finishes (`signOutTimeoutMs`, default 10 seconds), and then it is cleared. Meanwhile the new `isSigningOut` state (`useSession().isSigningOut`) is true; `isLoading` is left alone, so a navigator gated on it keeps its screen. Before, a login screen that opened at once could find the still-live session and sign the user straight back in. A sign-out that fails or times out checks the session first. A session that is still live, or a cookie or native session the server cannot be asked about, stays signed in, and `error` is set to `SIGN_OUT_FAILED_MESSAGE` ("Couldn't sign out. Check your connection and try again."). A web bearer client, which drops its token before the request leaves, ends signed out. The global `auth/logout` action (a refused session) now ends signed out even when the sign-out request fails. With `apisToReset` (57.17.0), `logout()` purges the cache when the session clears, not when it is dispatched: a purge at dispatch refetched every mounted screen mid-sign-out, and a refetch that could not reach the server answered 401 and signed the user out even though the sign-out failed. A failed sign-out keeps both the session and the cache. The global `auth/logout` action still purges at once (only the first of several in flight) and again when the session clears. A sign-in for another user that lands while a sign-out is in flight is kept; a sign-out that started signed out asks `getSession` first, so a restore of the session it revoked is cleared. Show `selectBetterAuthIsSigningOut` on the sign-out control and `selectBetterAuthError` after a failed sign-out.
60
+
61
+ ### Removed
62
+
63
+ - `@terreno/ui` no longer exports the `Filter` and `FilterProps` aliases deprecated in 57.7.0. Import `DropdownPanel` and `DropdownPanelProps` instead. `FilterSelectMenu`, `FilterBoolean`, `FilterAccordion`, and `FilterChangesBadge` keep their names.
64
+
65
+ ### Fixed
66
+
67
+ - `@terreno/syncdb`: logging out while `syncDb.start()` is still running no longer loses local writes or logs errors. `start()` now resolves the web encryption key from `GET /sync/key` before it loads the store, so the flush in `stop()` never needs the network after the session has ended. A `stop()` or `signOut()` call now cancels a `start()` that is running or queued ahead of it, so the transport never connects without a session. A failed key fetch is now retried on the next save instead of failing every save for the rest of the session.
68
+
23
69
  ## [58.0.0-beta.1] - 2026-11-02
24
70
 
25
71
  Upgrade note: [`mcp-server/src/docs/upgrades/58.0.0-beta.1.md`](mcp-server/src/docs/upgrades/58.0.0-beta.1.md).
package/dist/index.d.ts CHANGED
@@ -12,7 +12,6 @@ export { compilePrompt, createPrompt, getPrompt, invalidatePromptCache } from ".
12
12
  export { initTracing, shutdownTracing } from "./langfuseTracing";
13
13
  export type { ChatMessage, GetPromptOptions, LangfuseAppOptions, LangfuseCachedPrompt, PaginatedResult, PreparePromptResult, PromptListItem, ScoreSubmission, ScoringFunction, TelemetrySettings, TraceListItem, } from "./langfuseTypes";
14
14
  export { createTelemetryConfig, preparePromptForAI } from "./langfuseVercelAi";
15
- export { GPT_HISTORY_STRING_IDS_MIGRATION, type GptHistoryStringIdsResult, gptHistoryStringIdsMigration, migrateGptHistoryStringIds, } from "./migrations/gptHistoryStringIds";
16
15
  export { AIDataset } from "./models/aiDataset";
17
16
  export { AIRequest } from "./models/aiRequest";
18
17
  export { FileAttachment } from "./models/fileAttachment";
package/dist/index.js CHANGED
@@ -14,8 +14,8 @@ var __exportStar = (this && this.__exportStar) || function(m, exports) {
14
14
  for (var p in m) if (p !== "default" && !Object.prototype.hasOwnProperty.call(exports, p)) __createBinding(exports, m, p);
15
15
  };
16
16
  Object.defineProperty(exports, "__esModule", { value: true });
17
- exports.isLocalObservabilityPluginOn = exports.buildPlaygroundAiStatus = exports.buildObservabilityStatus = exports.resetObservabilityApp = exports.ObservabilityApp = exports.getObservabilityApp = exports.MemoryTraceSink = exports.MemoryScoreSink = exports.LocalTraceStore = exports.LocalTraceSink = exports.LocalScoreSink = exports.LocalReviewStore = exports.LocalPromptStore = exports.createLocalObservabilityPlugin = exports.createLocalObservabilityBundle = exports.LocalExperimentRunner = exports.LocalEvaluatorStore = exports.LocalDatasetStore = exports.observabilityAdminScreens = exports.AI_OBSERVABILITY_GROUP = exports.Project = exports.getProjectModel = exports.GptHistory = exports.FileAttachment = exports.AIRequest = exports.AIDataset = exports.migrateGptHistoryStringIds = exports.gptHistoryStringIdsMigration = exports.GPT_HISTORY_STRING_IDS_MIGRATION = exports.preparePromptForAI = exports.createTelemetryConfig = exports.shutdownTracing = exports.initTracing = exports.invalidatePromptCache = exports.getPrompt = exports.createPrompt = exports.compilePrompt = exports.shutdownLangfuseClient = exports.isLangfuseInitialized = exports.initLangfuseClient = exports.getLangfuseClient = exports.setCached = exports.LangfuseCache = exports.invalidateCache = exports.getCached = exports.LangfuseApp = exports.AiApp = exports.AIAdminApp = exports.Output = exports.jsonSchema = void 0;
18
- exports.verifyVertexModelsEnabled = exports.normalizeVertexModelId = exports.listEnabledVertexModels = exports.isVertexModelAllowed = exports.DEFAULT_VERTEX_LOCATION = exports.createVertexProvider = exports.assertVertexModelsEnabled = exports.toggleChecklistHostAction = exports.scaleStepperHostAction = exports.TRANSLATION_PROMPT = exports.TITLE_GENERATION_PROMPT = exports.TERRENO_ASKS_SYSTEM_PROMPT = exports.REMIX_PROMPT = exports.JSON_VALUE_SYSTEM_PROMPT = exports.DEFAULT_GPT_MEMORY = exports.CONTENT_SUMMARY_PROMPT = exports.COMPACT_SURFACE_SYSTEM_PROMPT = exports.parseAiJson = exports.normalizeLlmJsonTextForStructuredOutput = exports.MCPService = exports.getMCPTools = exports.normalizeGeminiModelId = exports.listGeminiApiModels = exports.GEMINI_API_BASE_URL = exports.FileStorageService = exports.runBufferedChatTurn = exports.createAskTools = exports.TemperaturePresets = exports.AIService = exports.registerAiDataset = exports.configureAiDatasets = exports.findAgentBlock = exports.addProjectRoutes = exports.addMcpRoutes = exports.addGptHistoryRoutes = exports.addGptRoutes = exports.addFileRoutes = exports.addAiRequestsExplorerRoutes = exports.validateObservabilityConfig = exports.resolveObservabilityControl = exports.DEFAULT_OBSERVABILITY_CONTROL = void 0;
17
+ exports.validateObservabilityConfig = exports.resolveObservabilityControl = exports.DEFAULT_OBSERVABILITY_CONTROL = exports.isLocalObservabilityPluginOn = exports.buildPlaygroundAiStatus = exports.buildObservabilityStatus = exports.resetObservabilityApp = exports.ObservabilityApp = exports.getObservabilityApp = exports.MemoryTraceSink = exports.MemoryScoreSink = exports.LocalTraceStore = exports.LocalTraceSink = exports.LocalScoreSink = exports.LocalReviewStore = exports.LocalPromptStore = exports.createLocalObservabilityPlugin = exports.createLocalObservabilityBundle = exports.LocalExperimentRunner = exports.LocalEvaluatorStore = exports.LocalDatasetStore = exports.observabilityAdminScreens = exports.AI_OBSERVABILITY_GROUP = exports.Project = exports.getProjectModel = exports.GptHistory = exports.FileAttachment = exports.AIRequest = exports.AIDataset = exports.preparePromptForAI = exports.createTelemetryConfig = exports.shutdownTracing = exports.initTracing = exports.invalidatePromptCache = exports.getPrompt = exports.createPrompt = exports.compilePrompt = exports.shutdownLangfuseClient = exports.isLangfuseInitialized = exports.initLangfuseClient = exports.getLangfuseClient = exports.setCached = exports.LangfuseCache = exports.invalidateCache = exports.getCached = exports.LangfuseApp = exports.AiApp = exports.AIAdminApp = exports.Output = exports.jsonSchema = void 0;
18
+ exports.verifyVertexModelsEnabled = exports.normalizeVertexModelId = exports.listEnabledVertexModels = exports.isVertexModelAllowed = exports.DEFAULT_VERTEX_LOCATION = exports.createVertexProvider = exports.assertVertexModelsEnabled = exports.toggleChecklistHostAction = exports.scaleStepperHostAction = exports.TRANSLATION_PROMPT = exports.TITLE_GENERATION_PROMPT = exports.TERRENO_ASKS_SYSTEM_PROMPT = exports.REMIX_PROMPT = exports.JSON_VALUE_SYSTEM_PROMPT = exports.DEFAULT_GPT_MEMORY = exports.CONTENT_SUMMARY_PROMPT = exports.COMPACT_SURFACE_SYSTEM_PROMPT = exports.parseAiJson = exports.normalizeLlmJsonTextForStructuredOutput = exports.MCPService = exports.getMCPTools = exports.normalizeGeminiModelId = exports.listGeminiApiModels = exports.GEMINI_API_BASE_URL = exports.FileStorageService = exports.runBufferedChatTurn = exports.createAskTools = exports.TemperaturePresets = exports.AIService = exports.registerAiDataset = exports.configureAiDatasets = exports.findAgentBlock = exports.addProjectRoutes = exports.addMcpRoutes = exports.addGptHistoryRoutes = exports.addGptRoutes = exports.addFileRoutes = exports.addAiRequestsExplorerRoutes = void 0;
19
19
  var ai_1 = require("ai");
20
20
  Object.defineProperty(exports, "jsonSchema", { enumerable: true, get: function () { return ai_1.jsonSchema; } });
21
21
  Object.defineProperty(exports, "Output", { enumerable: true, get: function () { return ai_1.Output; } });
@@ -46,10 +46,6 @@ Object.defineProperty(exports, "shutdownTracing", { enumerable: true, get: funct
46
46
  var langfuseVercelAi_1 = require("./langfuseVercelAi");
47
47
  Object.defineProperty(exports, "createTelemetryConfig", { enumerable: true, get: function () { return langfuseVercelAi_1.createTelemetryConfig; } });
48
48
  Object.defineProperty(exports, "preparePromptForAI", { enumerable: true, get: function () { return langfuseVercelAi_1.preparePromptForAI; } });
49
- var gptHistoryStringIds_1 = require("./migrations/gptHistoryStringIds");
50
- Object.defineProperty(exports, "GPT_HISTORY_STRING_IDS_MIGRATION", { enumerable: true, get: function () { return gptHistoryStringIds_1.GPT_HISTORY_STRING_IDS_MIGRATION; } });
51
- Object.defineProperty(exports, "gptHistoryStringIdsMigration", { enumerable: true, get: function () { return gptHistoryStringIds_1.gptHistoryStringIdsMigration; } });
52
- Object.defineProperty(exports, "migrateGptHistoryStringIds", { enumerable: true, get: function () { return gptHistoryStringIds_1.migrateGptHistoryStringIds; } });
53
49
  var aiDataset_1 = require("./models/aiDataset");
54
50
  Object.defineProperty(exports, "AIDataset", { enumerable: true, get: function () { return aiDataset_1.AIDataset; } });
55
51
  var aiRequest_1 = require("./models/aiRequest");
@@ -30,11 +30,11 @@ const aiDatasetSchema = new mongoose_1.default.Schema({
30
30
  type: Date,
31
31
  },
32
32
  historyId: {
33
- description: "The conversation whose tool stored these rows (a GptHistory String id)",
33
+ description: "The conversation whose tool stored these rows",
34
34
  index: true,
35
35
  ref: "GptHistory",
36
36
  required: true,
37
- type: String,
37
+ type: mongoose_1.default.Schema.Types.ObjectId,
38
38
  },
39
39
  rowCount: {
40
40
  description: "How many rows were stored",
@@ -4,7 +4,6 @@ var __importDefault = (this && this.__importDefault) || function (mod) {
4
4
  };
5
5
  Object.defineProperty(exports, "__esModule", { value: true });
6
6
  exports.GptHistory = void 0;
7
- const node_crypto_1 = require("node:crypto");
8
7
  const api_1 = require("@terreno/api");
9
8
  const blocks_1 = require("@terreno/blocks");
10
9
  const mongoose_1 = __importDefault(require("mongoose"));
@@ -133,14 +132,6 @@ const gptHistoryPromptSchema = new mongoose_1.default.Schema({
133
132
  },
134
133
  }, { _id: false, strict: "throw" });
135
134
  const gptHistorySchema = new mongoose_1.default.Schema({
136
- _id: {
137
- // Synced through @terreno/syncdb: offline clients mint entity ids locally and the sync
138
- // mutation channel writes them through as _id. Histories created before Terreno 58 keep
139
- // their ObjectId hex string after the gpt-history-string-ids migration.
140
- default: () => (0, node_crypto_1.randomUUID)(),
141
- description: "The conversation id (a String so offline sync clients can mint ids)",
142
- type: String,
143
- },
144
135
  pendingAsk: {
145
136
  description: "The ask this conversation is waiting on; cleared when the user answers or the ask is cancelled",
146
137
  type: pendingAskSchema,
@@ -1,9 +1,13 @@
1
1
  "use strict";
2
+ var __importDefault = (this && this.__importDefault) || function (mod) {
3
+ return (mod && mod.__esModule) ? mod : { "default": mod };
4
+ };
2
5
  Object.defineProperty(exports, "__esModule", { value: true });
3
6
  exports.addGptActionRoutes = exports.HOST_ACTION_TIMEOUT_MS = void 0;
4
7
  const api_1 = require("@terreno/api");
5
8
  const blocks_1 = require("@terreno/blocks");
6
9
  const luxon_1 = require("luxon");
10
+ const mongoose_1 = __importDefault(require("mongoose"));
7
11
  const aiRequest_1 = require("../models/aiRequest");
8
12
  const gptHistory_1 = require("../models/gptHistory");
9
13
  /**
@@ -120,6 +124,9 @@ const addGptActionRoutes = (router, options) => {
120
124
  title: "historyId, messageId, blockId, elementId, and name are required",
121
125
  });
122
126
  }
127
+ if (!mongoose_1.default.Types.ObjectId.isValid(historyId)) {
128
+ throw new api_1.APIError({ status: 400, title: "historyId is not valid" });
129
+ }
123
130
  const userId = requestUserId(req);
124
131
  const history = await gptHistory_1.GptHistory.findOneOrNone({ _id: historyId, deleted: false });
125
132
  if (!history) {
@@ -91,8 +91,7 @@ const registerAiDataset = async ({ columns, historyId, rows, userId, }) => {
91
91
  assertRows(columns, rows);
92
92
  const created = await aiDataset_1.AIDataset.create({
93
93
  columns,
94
- // GptHistory ids are Strings; an ObjectId from an older caller is stored as its hex string.
95
- historyId: String(historyId),
94
+ historyId,
96
95
  rowCount: rows.length,
97
96
  rows,
98
97
  userId,
@@ -155,9 +155,10 @@ const loadHistory = async ({ historyId, projectId, userId, }) => {
155
155
  if (!historyId) {
156
156
  return new gptHistory_1.GptHistory({ prompts: [], userId, ...(projectId ? { projectId } : {}) });
157
157
  }
158
- // History ids are Strings (synced collection); an object here would be read as a query operator.
159
- if (typeof historyId !== "string") {
160
- throw new api_1.APIError({ status: 400, title: "historyId must be a string" });
158
+ // The id arrives as the ObjectId's hex string. An object here would be read as a query
159
+ // operator, and any other string would surface as a CastError 500.
160
+ if (typeof historyId !== "string" || !/^[0-9a-fA-F]{24}$/.test(historyId)) {
161
+ throw new api_1.APIError({ status: 400, title: "historyId is not valid" });
161
162
  }
162
163
  const history = await gptHistory_1.GptHistory.findById(historyId);
163
164
  if (!history) {
@@ -113,9 +113,11 @@ interface GptHistoryPendingAskState {
113
113
  }
114
114
  /** The ask a history is waiting on. At most one per history. */
115
115
  export type GptHistoryPendingAsk = Ask & GptHistoryPendingAskState;
116
- /** Synced through @terreno/syncdb, so `_id` is a String (UUID for new rows, ObjectId hex for migrated rows). */
117
- export interface GptHistoryDocument extends mongoose.Document<string> {
118
- _id: string;
116
+ /**
117
+ * Synced through @terreno/syncdb with the default ObjectId `_id`; sync clients that create
118
+ * histories configure `idFormats: {"gpt/histories": "objectId"}`.
119
+ */
120
+ export interface GptHistoryDocument extends mongoose.Document<mongoose.Types.ObjectId> {
119
121
  created: Date;
120
122
  deleted: boolean;
121
123
  pendingAsk?: GptHistoryPendingAsk;
@@ -437,8 +439,7 @@ export interface AIDatasetDocument extends mongoose.Document<mongoose.Types.Obje
437
439
  created: Date;
438
440
  deleted: boolean;
439
441
  expiresAt?: Date;
440
- /** The GptHistory String `_id`. */
441
- historyId: string;
442
+ historyId: mongoose.Types.ObjectId;
442
443
  rowCount: number;
443
444
  rows: AIDatasetCell[][];
444
445
  updated: Date;
package/package.json CHANGED
@@ -10,7 +10,7 @@
10
10
  "@langfuse/otel": "^5.0.1",
11
11
  "@opentelemetry/api": "^1.9.0",
12
12
  "@opentelemetry/sdk-node": "0.222.0",
13
- "@terreno/blocks": "58.0.0-beta.1",
13
+ "@terreno/blocks": "58.0.0-beta.2",
14
14
  "ai": "^6.0.97",
15
15
  "ajv": "8.20.0",
16
16
  "csv-parse": "^6.1.0",
@@ -22,10 +22,10 @@
22
22
  "description": "Provider-agnostic AI service with streaming chat, request logging, and admin tools — part of Terreno, the batteries-included TypeScript framework for universal apps.",
23
23
  "devDependencies": {
24
24
  "@biomejs/biome": "^2.3.6",
25
- "@terreno/admin-backend": "58.0.0-beta.1",
26
- "@terreno/api": "58.0.0-beta.1",
27
- "@terreno/jobs": "58.0.0-beta.1",
28
- "@terreno/test": "58.0.0-beta.1",
25
+ "@terreno/admin-backend": "58.0.0-beta.2",
26
+ "@terreno/api": "58.0.0-beta.2",
27
+ "@terreno/jobs": "58.0.0-beta.2",
28
+ "@terreno/test": "58.0.0-beta.2",
29
29
  "@types/bun": "^1.4.0",
30
30
  "@types/express": "^5.0.6",
31
31
  "@types/luxon": "^3.7.1",
@@ -137,8 +137,8 @@
137
137
  "peerDependencies": {
138
138
  "@ai-sdk/google-vertex": "^5.0.92",
139
139
  "@ai-sdk/provider": "^4.0.26",
140
- "@terreno/api": "58.0.0-beta.1",
141
- "@terreno/jobs": "58.0.0-beta.1",
140
+ "@terreno/api": "58.0.0-beta.2",
141
+ "@terreno/jobs": "58.0.0-beta.2",
142
142
  "mongoose": "^8.0.0 || ^9.0.0"
143
143
  },
144
144
  "peerDependenciesMeta": {
@@ -171,5 +171,5 @@
171
171
  "typecheck": "tsc -p tsconfig.typecheck.json"
172
172
  },
173
173
  "types": "dist/index.d.ts",
174
- "version": "58.0.0-beta.1"
174
+ "version": "58.0.0-beta.2"
175
175
  }
package/src/index.ts CHANGED
@@ -36,12 +36,6 @@ export type {
36
36
  TraceListItem,
37
37
  } from "./langfuseTypes";
38
38
  export {createTelemetryConfig, preparePromptForAI} from "./langfuseVercelAi";
39
- export {
40
- GPT_HISTORY_STRING_IDS_MIGRATION,
41
- type GptHistoryStringIdsResult,
42
- gptHistoryStringIdsMigration,
43
- migrateGptHistoryStringIds,
44
- } from "./migrations/gptHistoryStringIds";
45
39
  export {AIDataset} from "./models/aiDataset";
46
40
  export {AIRequest} from "./models/aiRequest";
47
41
  export {FileAttachment} from "./models/fileAttachment";
@@ -12,7 +12,7 @@ describe("AIDataset model", () => {
12
12
  const userId = new mongoose.Types.ObjectId();
13
13
  const dataset = await AIDataset.create({
14
14
  columns: [{name: "count", type: "number"}],
15
- historyId: new mongoose.Types.ObjectId().toString(),
15
+ historyId: new mongoose.Types.ObjectId(),
16
16
  rowCount: 1,
17
17
  rows: [[3]],
18
18
  userId,
@@ -32,11 +32,11 @@ const aiDatasetSchema: AIDatasetSchema = new mongoose.Schema(
32
32
  type: Date,
33
33
  },
34
34
  historyId: {
35
- description: "The conversation whose tool stored these rows (a GptHistory String id)",
35
+ description: "The conversation whose tool stored these rows",
36
36
  index: true,
37
37
  ref: "GptHistory",
38
38
  required: true,
39
- type: String,
39
+ type: mongoose.Schema.Types.ObjectId,
40
40
  },
41
41
  rowCount: {
42
42
  description: "How many rows were stored",
@@ -4,12 +4,6 @@ import mongoose from "mongoose";
4
4
  import type {GptHistoryDocument, GptHistoryPendingAsk, GptHistoryPrompt} from "../types/index";
5
5
  import {GptHistory} from "./gptHistory";
6
6
 
7
- // GptHistory ids are strings, but the driver types the raw collection with ObjectId ids.
8
- const stringIdCollection = (): mongoose.mongo.Collection<{_id: string} & mongoose.mongo.Document> =>
9
- GptHistory.collection as unknown as mongoose.mongo.Collection<
10
- {_id: string} & mongoose.mongo.Document
11
- >;
12
-
13
7
  describe("GptHistory Model", () => {
14
8
  beforeEach(async () => {
15
9
  await GptHistory.collection.deleteMany({});
@@ -277,8 +271,7 @@ describe("GptHistory Model", () => {
277
271
 
278
272
  it("loads a history saved before asks existed", async () => {
279
273
  const userId = new mongoose.Types.ObjectId();
280
- const {insertedId} = await stringIdCollection().insertOne({
281
- _id: new mongoose.Types.ObjectId().toHexString(),
274
+ const {insertedId} = await GptHistory.collection.insertOne({
282
275
  created: new Date("2026-01-05T09:00:00.000Z"),
283
276
  deleted: false,
284
277
  prompts: [
@@ -310,7 +303,7 @@ describe("GptHistory Model", () => {
310
303
  userId,
311
304
  });
312
305
 
313
- const stored = await stringIdCollection().findOne({_id: history._id});
306
+ const stored = await GptHistory.collection.findOne({_id: history._id});
314
307
 
315
308
  expect(stored?.pendingAsk).toEqual(pendingAsk);
316
309
  expect(stored?.prompts[1].ask).toEqual({kind: "choice", status: "pending"});
@@ -323,7 +316,7 @@ describe("GptHistory Model", () => {
323
316
  history.pendingAsk = pendingAsk;
324
317
  await history.save();
325
318
 
326
- const stored = await stringIdCollection().findOne({_id: history._id});
319
+ const stored = await GptHistory.collection.findOne({_id: history._id});
327
320
  expect(stored?.pendingAsk.responseMessages[0].content[0]).toEqual({
328
321
  input: {},
329
322
  toolCallId: "call_lookup",
@@ -1,5 +1,3 @@
1
- import {randomUUID} from "node:crypto";
2
-
3
1
  import {
4
2
  createdUpdatedPlugin,
5
3
  findExactlyOne,
@@ -161,14 +159,6 @@ const gptHistoryPromptSchema = new mongoose.Schema(
161
159
 
162
160
  const gptHistorySchema = new mongoose.Schema<GptHistoryDocument, GptHistoryModel>(
163
161
  {
164
- _id: {
165
- // Synced through @terreno/syncdb: offline clients mint entity ids locally and the sync
166
- // mutation channel writes them through as _id. Histories created before Terreno 58 keep
167
- // their ObjectId hex string after the gpt-history-string-ids migration.
168
- default: (): string => randomUUID(),
169
- description: "The conversation id (a String so offline sync clients can mint ids)",
170
- type: String,
171
- },
172
162
  pendingAsk: {
173
163
  description:
174
164
  "The ask this conversation is waiting on; cleared when the user answers or the ask is cancelled",
@@ -1730,6 +1730,18 @@ describe("/gpt/prompt asks", () => {
1730
1730
  expect(res.status).toBe(404);
1731
1731
  expect(res.body.title).toBe("History not found");
1732
1732
  });
1733
+
1734
+ it("returns 400 for a history id that is not an ObjectId hex string", async () => {
1735
+ const model = createScriptedModel({steps: []});
1736
+ const agent = await authAsUser(buildApp({asks: true, model}), "notAdmin");
1737
+
1738
+ for (const historyId of [crypto.randomUUID(), "abcdefghijkl", {$ne: null}]) {
1739
+ const res = await agent.post("/gpt/prompt").send({historyId, prompt: "Hi"});
1740
+ expect(res.status).toBe(400);
1741
+ expect(res.body.title).toBe("historyId is not valid");
1742
+ }
1743
+ expect(model.doStream).not.toHaveBeenCalled();
1744
+ });
1733
1745
  });
1734
1746
 
1735
1747
  describe("a prompt while an ask is pending", () => {
@@ -198,7 +198,7 @@ describe("POST /gpt/actions", () => {
198
198
  expect(res.body.meta.fields["blocks[0].color"]).toEqual(expect.stringContaining("UNKNOWN_KEY"));
199
199
  });
200
200
 
201
- it("rejects a missing body and a history that does not exist, by any string id", async () => {
201
+ it("rejects a missing body, a bad history id, and a history that does not exist", async () => {
202
202
  await ensureTestUsers();
203
203
  const app = buildApp({
204
204
  model,
@@ -215,10 +215,9 @@ describe("POST /gpt/actions", () => {
215
215
  messageId: "m1",
216
216
  name: "export_csv",
217
217
  };
218
- // History ids are Strings (UUIDs or migrated ObjectId hex), so any string is looked up.
219
- const unknownId = await agent.post("/gpt/actions").send(body);
220
- expect(unknownId.status).toBe(404);
221
- expect(unknownId.body.title).toBe("History not found");
218
+ const badId = await agent.post("/gpt/actions").send(body);
219
+ expect(badId.status).toBe(400);
220
+ expect(badId.body.title).toBe("historyId is not valid");
222
221
 
223
222
  const missing = await agent.post("/gpt/actions").send({
224
223
  ...body,
@@ -8,7 +8,7 @@ import {
8
8
  import {type BlocksDocument, validateBlocks} from "@terreno/blocks";
9
9
  import type express from "express";
10
10
  import {DateTime} from "luxon";
11
- import type mongoose from "mongoose";
11
+ import mongoose from "mongoose";
12
12
 
13
13
  import {AIRequest} from "../models/aiRequest";
14
14
  import {GptHistory} from "../models/gptHistory";
@@ -158,6 +158,9 @@ export const addGptActionRoutes = (router: express.Router, options: GptRouteOpti
158
158
  });
159
159
  }
160
160
 
161
+ if (!mongoose.Types.ObjectId.isValid(historyId)) {
162
+ throw new APIError({status: 400, title: "historyId is not valid"});
163
+ }
161
164
  const userId = requestUserId(req);
162
165
  const history = await GptHistory.findOneOrNone({_id: historyId, deleted: false});
163
166
  if (!history) {
@@ -1,7 +1,7 @@
1
1
  import {afterEach, beforeAll, describe, expect, it} from "bun:test";
2
2
  import {SyncApp, TerrenoApp} from "@terreno/api";
3
3
  import type express from "express";
4
- import type mongoose from "mongoose";
4
+ import mongoose from "mongoose";
5
5
 
6
6
  import {GptHistory} from "../models/gptHistory";
7
7
  import {
@@ -11,7 +11,6 @@ import {
11
11
  ensureTestUsers,
12
12
  historyRouterOptions,
13
13
  terrenoUserModel,
14
- UserModel,
15
14
  } from "../tests/helpers";
16
15
  import {addGptHistoryRoutes} from "./gptHistories";
17
16
 
@@ -22,7 +21,7 @@ const OTHER_USER = {
22
21
  password: "password",
23
22
  };
24
23
 
25
- const UUID_PATTERN = /^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/;
24
+ const OBJECT_ID_PATTERN = /^[0-9a-f]{24}$/;
26
25
 
27
26
  interface SnapshotEntity {
28
27
  data: Record<string, unknown>;
@@ -66,12 +65,12 @@ describe("/gpt/histories over sync", () => {
66
65
  await GptHistory.collection.deleteMany({});
67
66
  });
68
67
 
69
- it("gives new histories a UUID String id", async () => {
68
+ it("gives new histories a server-made ObjectId", async () => {
70
69
  const agent = await authAsUser(buildSyncApp(), "notAdmin");
71
70
 
72
71
  const res = await agent.post("/gpt/histories").send({title: "Fresh"}).expect(201);
73
72
 
74
- expect(res.body.data._id).toMatch(UUID_PATTERN);
73
+ expect(res.body.data._id).toMatch(OBJECT_ID_PATTERN);
75
74
  const history = await GptHistory.findById(res.body.data._id);
76
75
  expect(history?.title).toBe("Fresh");
77
76
  });
@@ -119,7 +118,9 @@ describe("/gpt/histories over sync", () => {
119
118
 
120
119
  const res = await agent.get(snapshotPath(notAdminId)).expect(200);
121
120
 
122
- const entity = (res.body.entities as SnapshotEntity[]).find((item) => item.id === history._id);
121
+ const entity = (res.body.entities as SnapshotEntity[]).find(
122
+ (item) => item.id === String(history._id)
123
+ );
123
124
  const pendingAsk = entity?.data.pendingAsk as Record<string, unknown>;
124
125
  expect(pendingAsk.toolCallId).toBe("call_confirm");
125
126
  expect(pendingAsk).not.toHaveProperty("promptIndex");
@@ -138,4 +139,92 @@ describe("/gpt/histories over sync", () => {
138
139
  );
139
140
  expect(entity?.deleted).toBe(true);
140
141
  });
142
+
143
+ it("creates a history through sync with a client-minted ObjectId and edits it by that id", async () => {
144
+ const agent = await authAsUser(buildSyncApp(), "notAdmin");
145
+ const id = new mongoose.Types.ObjectId().toHexString();
146
+
147
+ const created = await agent
148
+ .post("/sync/mutate")
149
+ .send({
150
+ collection: "gpt/histories",
151
+ data: {prompts: [], title: "Offline"},
152
+ id,
153
+ mutationId: `create-${id}`,
154
+ operation: "create",
155
+ })
156
+ .expect(200);
157
+ expect(created.body.ack.id).toBe(id);
158
+ const stored = await GptHistory.findById(id);
159
+ expect(stored?._id).toBeInstanceOf(mongoose.Types.ObjectId);
160
+ expect(String(stored?.userId)).toBe(notAdminId);
161
+
162
+ const renamed = await agent
163
+ .post("/sync/mutate")
164
+ .send({
165
+ baseVersion: created.body.ack.seq,
166
+ collection: "gpt/histories",
167
+ data: {title: "Renamed"},
168
+ id,
169
+ mutationId: `rename-${id}`,
170
+ operation: "update",
171
+ })
172
+ .expect(200);
173
+ expect((await GptHistory.findById(id))?.title).toBe("Renamed");
174
+
175
+ await agent
176
+ .post("/sync/mutate")
177
+ .send({
178
+ collection: "gpt/histories",
179
+ id,
180
+ mutationId: `delete-${id}`,
181
+ operation: "delete",
182
+ })
183
+ .expect(200);
184
+ const res = await agent.get(snapshotPath(notAdminId)).expect(200);
185
+ const entity = (res.body.entities as SnapshotEntity[]).find((item) => item.id === id);
186
+ expect(entity?.deleted).toBe(true);
187
+ expect(entity?.seq).toBeGreaterThan(renamed.body.ack.seq);
188
+ });
189
+
190
+ it("nacks a UUID history id as invalid_id without writing anything", async () => {
191
+ const agent = await authAsUser(buildSyncApp(), "notAdmin");
192
+
193
+ const res = await agent
194
+ .post("/sync/mutate")
195
+ .send({
196
+ collection: "gpt/histories",
197
+ data: {prompts: [], title: "UUID"},
198
+ id: crypto.randomUUID(),
199
+ mutationId: "create-uuid-history",
200
+ operation: "create",
201
+ })
202
+ .expect(422);
203
+
204
+ expect(res.body.nack.code).toBe("validation");
205
+ expect(res.body.nack.reason).toBe("invalid_id");
206
+ expect(await GptHistory.collection.countDocuments({})).toBe(0);
207
+ });
208
+
209
+ it("nacks id_collision when a client-minted id is another user's history", async () => {
210
+ const agent = await authAsUser(buildSyncApp(), "notAdmin");
211
+ const theirs = await GptHistory.create({prompts: [], title: "Theirs", userId: otherId});
212
+
213
+ const res = await agent
214
+ .post("/sync/mutate")
215
+ .send({
216
+ collection: "gpt/histories",
217
+ data: {prompts: [], title: "Mine"},
218
+ id: String(theirs._id),
219
+ mutationId: "create-colliding-history",
220
+ operation: "create",
221
+ })
222
+ .expect(422);
223
+
224
+ expect(res.body.nack.reason).toBe("id_collision");
225
+ expect(res.body.nack.message).toBe("Entity id already exists");
226
+ const stored = await GptHistory.findById(theirs._id);
227
+ expect(stored?.title).toBe("Theirs");
228
+ expect(String(stored?.userId)).toBe(otherId);
229
+ });
141
230
  });
@@ -133,8 +133,7 @@ export const registerAiDataset = async ({
133
133
  assertRows(columns, rows);
134
134
  const created = await AIDataset.create({
135
135
  columns,
136
- // GptHistory ids are Strings; an ObjectId from an older caller is stored as its hex string.
137
- historyId: String(historyId),
136
+ historyId,
138
137
  rowCount: rows.length,
139
138
  rows,
140
139
  userId,
@@ -415,9 +415,10 @@ const loadHistory = async ({
415
415
  if (!historyId) {
416
416
  return new GptHistory({prompts: [], userId, ...(projectId ? {projectId} : {})});
417
417
  }
418
- // History ids are Strings (synced collection); an object here would be read as a query operator.
419
- if (typeof historyId !== "string") {
420
- throw new APIError({status: 400, title: "historyId must be a string"});
418
+ // The id arrives as the ObjectId's hex string. An object here would be read as a query
419
+ // operator, and any other string would surface as a CastError 500.
420
+ if (typeof historyId !== "string" || !/^[0-9a-fA-F]{24}$/.test(historyId)) {
421
+ throw new APIError({status: 400, title: "historyId is not valid"});
421
422
  }
422
423
  const history = await GptHistory.findById(historyId);
423
424
  if (!history) {
@@ -154,9 +154,11 @@ interface GptHistoryPendingAskState {
154
154
  /** The ask a history is waiting on. At most one per history. */
155
155
  export type GptHistoryPendingAsk = Ask & GptHistoryPendingAskState;
156
156
 
157
- /** Synced through @terreno/syncdb, so `_id` is a String (UUID for new rows, ObjectId hex for migrated rows). */
158
- export interface GptHistoryDocument extends mongoose.Document<string> {
159
- _id: string;
157
+ /**
158
+ * Synced through @terreno/syncdb with the default ObjectId `_id`; sync clients that create
159
+ * histories configure `idFormats: {"gpt/histories": "objectId"}`.
160
+ */
161
+ export interface GptHistoryDocument extends mongoose.Document<mongoose.Types.ObjectId> {
160
162
  created: Date;
161
163
  deleted: boolean;
162
164
  pendingAsk?: GptHistoryPendingAsk;
@@ -517,8 +519,7 @@ export interface AIDatasetDocument extends mongoose.Document<mongoose.Types.Obje
517
519
  created: Date;
518
520
  deleted: boolean;
519
521
  expiresAt?: Date;
520
- /** The GptHistory String `_id`. */
521
- historyId: string;
522
+ historyId: mongoose.Types.ObjectId;
522
523
  rowCount: number;
523
524
  rows: AIDatasetCell[][];
524
525
  updated: Date;
@@ -1,41 +0,0 @@
1
- /**
2
- * Data migration for Terreno 58: GptHistory became a synced collection with String `_id`s.
3
- * Histories saved before 58 have ObjectId `_id`s, which sync clients and the String-typed
4
- * schema cannot address. This migration re-inserts each one with `_id` set to the same id's
5
- * hex string, so URLs and client caches that hold the hex id keep working, rewrites
6
- * `AIDataset.historyId` refs to that hex string, and then deletes the ObjectId originals.
7
- *
8
- * It writes through the native driver: `syncPlugin` forbids the hard deletes and
9
- * multi-document writes a move needs, and the copies keep every stored field as-is.
10
- * Running it again is a no-op.
11
- */
12
- import type { MigrationContext, MigrationLogger } from "@terreno/api";
13
- import type mongoose from "mongoose";
14
- /** Slug for the timestamped file an app adds to its `migrations/` directory. */
15
- export declare const GPT_HISTORY_STRING_IDS_MIGRATION = "gpt-history-string-ids";
16
- export interface GptHistoryStringIdsResult {
17
- /** AIDataset rows whose ObjectId `historyId` was rewritten (or would be, in a dry run). */
18
- datasetsUpdated: number;
19
- /** Histories moved to a String `_id` (or that would move, in a dry run). */
20
- historiesMigrated: number;
21
- }
22
- /**
23
- * Moves every ObjectId-id GptHistory to its hex-string id in batches and rewrites the
24
- * AIDataset refs. In a dry run it only counts what it would change.
25
- */
26
- export declare const migrateGptHistoryStringIds: ({ batchSize, checkCancellation, connection, dryRun, logger, }: {
27
- batchSize?: number;
28
- checkCancellation?: () => Promise<void>;
29
- connection: mongoose.Connection;
30
- dryRun: boolean;
31
- logger?: MigrationLogger;
32
- }) => Promise<GptHistoryStringIdsResult>;
33
- /**
34
- * Ready-made `up` for an app's `migrations/<YYYYMMDDHHmmss>-gpt-history-string-ids.ts` file.
35
- * Run it before deploying Terreno 58 frontends, which address histories by String id. It has
36
- * no `down`: new histories get UUIDs, which have no ObjectId to go back to.
37
- */
38
- export declare const gptHistoryStringIdsMigration: {
39
- slug: string;
40
- up: (ctx: MigrationContext) => Promise<void>;
41
- };
@@ -1,94 +0,0 @@
1
- "use strict";
2
- Object.defineProperty(exports, "__esModule", { value: true });
3
- exports.gptHistoryStringIdsMigration = exports.migrateGptHistoryStringIds = exports.GPT_HISTORY_STRING_IDS_MIGRATION = void 0;
4
- const aiDataset_1 = require("../models/aiDataset");
5
- const gptHistory_1 = require("../models/gptHistory");
6
- /** Slug for the timestamped file an app adds to its `migrations/` directory. */
7
- exports.GPT_HISTORY_STRING_IDS_MIGRATION = "gpt-history-string-ids";
8
- const DEFAULT_BATCH_SIZE = 500;
9
- const DUPLICATE_KEY_CODE = 11000;
10
- /** Only ObjectId `_id`s still need a move; String ids are already migrated or new. */
11
- const OBJECT_ID_FILTER = { _id: { $type: "objectId" } };
12
- /** Server-side ObjectId → hex string, so refs move without reading each dataset. */
13
- const HISTORY_ID_TO_STRING = [{ $set: { historyId: { $toString: "$historyId" } } }];
14
- /** An insert that failed only because the hex-id copy exists from an interrupted earlier run. */
15
- const isOnlyDuplicateKeyError = (error) => {
16
- const writeError = error;
17
- if (writeError.writeErrors?.length) {
18
- return writeError.writeErrors.every((item) => item.code === DUPLICATE_KEY_CODE);
19
- }
20
- return writeError.code === DUPLICATE_KEY_CODE;
21
- };
22
- const insertCopies = async ({ collection, copies, }) => {
23
- try {
24
- await collection.insertMany(copies, { ordered: false });
25
- }
26
- catch (error) {
27
- if (!isOnlyDuplicateKeyError(error)) {
28
- throw error;
29
- }
30
- }
31
- };
32
- /**
33
- * Moves every ObjectId-id GptHistory to its hex-string id in batches and rewrites the
34
- * AIDataset refs. In a dry run it only counts what it would change.
35
- */
36
- const migrateGptHistoryStringIds = async ({ batchSize = DEFAULT_BATCH_SIZE, checkCancellation, connection, dryRun, logger, }) => {
37
- const histories = connection.collection(gptHistory_1.GptHistory.collection.collectionName);
38
- const datasets = connection.collection(aiDataset_1.AIDataset.collection.collectionName);
39
- const datasetFilter = { historyId: { $type: "objectId" } };
40
- if (dryRun) {
41
- const result = {
42
- datasetsUpdated: await datasets.countDocuments(datasetFilter),
43
- historiesMigrated: await histories.countDocuments(OBJECT_ID_FILTER),
44
- };
45
- logger?.info(`[${exports.GPT_HISTORY_STRING_IDS_MIGRATION}] dry run`, result);
46
- return result;
47
- }
48
- let historiesMigrated = 0;
49
- let datasetsUpdated = 0;
50
- while (true) {
51
- await checkCancellation?.();
52
- const batch = await histories.find(OBJECT_ID_FILTER).limit(batchSize).toArray();
53
- if (batch.length === 0) {
54
- break;
55
- }
56
- const originalIds = batch.map((doc) => doc._id);
57
- // Copy first and delete second, so an interrupted run never loses a history: the next
58
- // run finds the original again and the copy's duplicate key is ignored.
59
- await insertCopies({
60
- collection: histories,
61
- copies: batch.map((doc) => ({ ...doc, _id: String(doc._id) })),
62
- });
63
- const moved = await datasets.updateMany({ historyId: { $in: originalIds } }, HISTORY_ID_TO_STRING);
64
- datasetsUpdated += moved.modifiedCount;
65
- await histories.deleteMany({ _id: { $in: originalIds } });
66
- historiesMigrated += batch.length;
67
- logger?.info(`[${exports.GPT_HISTORY_STRING_IDS_MIGRATION}] moved ${historiesMigrated} histories to String ids`);
68
- }
69
- // Refs whose history was moved by an interrupted run, or never existed, still hold ObjectIds.
70
- const remaining = await datasets.updateMany(datasetFilter, HISTORY_ID_TO_STRING);
71
- datasetsUpdated += remaining.modifiedCount;
72
- const result = { datasetsUpdated, historiesMigrated };
73
- logger?.info(`[${exports.GPT_HISTORY_STRING_IDS_MIGRATION}] done`, result);
74
- return result;
75
- };
76
- exports.migrateGptHistoryStringIds = migrateGptHistoryStringIds;
77
- /**
78
- * Ready-made `up` for an app's `migrations/<YYYYMMDDHHmmss>-gpt-history-string-ids.ts` file.
79
- * Run it before deploying Terreno 58 frontends, which address histories by String id. It has
80
- * no `down`: new histories get UUIDs, which have no ObjectId to go back to.
81
- */
82
- exports.gptHistoryStringIdsMigration = {
83
- slug: exports.GPT_HISTORY_STRING_IDS_MIGRATION,
84
- up: async (ctx) => {
85
- const result = await (0, exports.migrateGptHistoryStringIds)({
86
- checkCancellation: ctx.checkCancellation,
87
- connection: ctx.mongoose.connection,
88
- dryRun: ctx.dryRun,
89
- logger: ctx.logger,
90
- });
91
- await ctx.addLog?.("info", `[${exports.GPT_HISTORY_STRING_IDS_MIGRATION}] ${ctx.dryRun ? "would move" : "moved"} ` +
92
- `${result.historiesMigrated} histories and ${result.datasetsUpdated} dataset refs`);
93
- },
94
- };
@@ -1,231 +0,0 @@
1
- import {afterEach, beforeEach, describe, expect, it} from "bun:test";
2
- import type {MigrationContext} from "@terreno/api";
3
- import mongoose from "mongoose";
4
-
5
- import {AIDataset} from "../models/aiDataset";
6
- import {GptHistory} from "../models/gptHistory";
7
- import {gptHistoryStringIdsMigration, migrateGptHistoryStringIds} from "./gptHistoryStringIds";
8
-
9
- const userId = new mongoose.Types.ObjectId();
10
-
11
- const historiesCollection = (): mongoose.mongo.Collection =>
12
- mongoose.connection.collection(GptHistory.collection.collectionName);
13
-
14
- const datasetsCollection = (): mongoose.mongo.Collection =>
15
- mongoose.connection.collection(AIDataset.collection.collectionName);
16
-
17
- const legacyHistory = (title: string): mongoose.mongo.Document => ({
18
- _id: new mongoose.Types.ObjectId(),
19
- created: new Date("2026-01-05T09:00:00.000Z"),
20
- deleted: false,
21
- prompts: [
22
- {text: `Question about ${title}`, type: "user"},
23
- {model: "gemini", text: `Answer about ${title}`, type: "assistant"},
24
- ],
25
- title,
26
- updated: new Date("2026-01-05T09:01:00.000Z"),
27
- userId,
28
- });
29
-
30
- const legacyDataset = (historyId: unknown): mongoose.mongo.Document => ({
31
- _id: new mongoose.Types.ObjectId(),
32
- columns: [{name: "count", type: "number"}],
33
- created: new Date("2026-01-05T09:02:00.000Z"),
34
- deleted: false,
35
- historyId,
36
- rowCount: 1,
37
- rows: [[1]],
38
- updated: new Date("2026-01-05T09:02:00.000Z"),
39
- userId,
40
- });
41
-
42
- interface Seeded {
43
- histories: mongoose.mongo.Document[];
44
- stringHistoryId: string;
45
- }
46
-
47
- /** Five ObjectId histories (two batches at batchSize 2, plus one), a String one, and refs. */
48
- const seed = async (): Promise<Seeded> => {
49
- const histories = ["a", "b", "c", "d", "e"].map(legacyHistory);
50
- const stringHistoryId = "already-a-string-id";
51
- await historiesCollection().insertMany([
52
- ...histories,
53
- {...legacyHistory("string"), _id: stringHistoryId},
54
- ]);
55
- await datasetsCollection().insertMany([
56
- legacyDataset(histories[0]._id),
57
- legacyDataset(histories[0]._id),
58
- legacyDataset(histories[3]._id),
59
- // A ref whose history was deleted long ago still becomes a String.
60
- legacyDataset(new mongoose.Types.ObjectId()),
61
- legacyDataset(stringHistoryId),
62
- ]);
63
- return {histories, stringHistoryId};
64
- };
65
-
66
- const idTypes = async (collection: mongoose.mongo.Collection, field: string): Promise<string[]> => {
67
- const rows = await collection
68
- .aggregate<{type: string}>([{$project: {_id: 0, type: {$type: `$${field}`}}}])
69
- .toArray();
70
- return [...new Set(rows.map((row) => row.type))].sort();
71
- };
72
-
73
- const migrationContext = (dryRun: boolean, logs: string[] = []): MigrationContext => ({
74
- addLog: async (_level, message): Promise<void> => {
75
- logs.push(message);
76
- },
77
- dryRun,
78
- logger: {debug: () => {}, error: () => {}, info: () => {}, warn: () => {}},
79
- mongoose,
80
- });
81
-
82
- describe("gpt-history-string-ids migration", () => {
83
- beforeEach(async () => {
84
- await historiesCollection().deleteMany({});
85
- await datasetsCollection().deleteMany({});
86
- });
87
-
88
- afterEach(async () => {
89
- await historiesCollection().deleteMany({});
90
- await datasetsCollection().deleteMany({});
91
- });
92
-
93
- it("moves ObjectId histories to their hex string ids, rewrites refs, and is a no-op the second time", async () => {
94
- const {histories, stringHistoryId} = await seed();
95
- const historyCount = await historiesCollection().countDocuments();
96
- const datasetCount = await datasetsCollection().countDocuments();
97
-
98
- const first = await migrateGptHistoryStringIds({
99
- batchSize: 2,
100
- connection: mongoose.connection,
101
- dryRun: false,
102
- });
103
- const second = await migrateGptHistoryStringIds({
104
- batchSize: 2,
105
- connection: mongoose.connection,
106
- dryRun: false,
107
- });
108
-
109
- expect(first).toEqual({datasetsUpdated: 4, historiesMigrated: 5});
110
- expect(second).toEqual({datasetsUpdated: 0, historiesMigrated: 0});
111
- expect(await historiesCollection().countDocuments()).toBe(historyCount);
112
- expect(await datasetsCollection().countDocuments()).toBe(datasetCount);
113
- expect(await idTypes(historiesCollection(), "_id")).toEqual(["string"]);
114
- expect(await idTypes(datasetsCollection(), "historyId")).toEqual(["string"]);
115
-
116
- const ids = (await historiesCollection().find({}).toArray()).map((doc) => doc._id).sort();
117
- expect(ids).toEqual([...histories.map((doc) => doc._id.toHexString()), stringHistoryId].sort());
118
- const firstHex = histories[0]._id.toHexString();
119
- expect(await datasetsCollection().countDocuments({historyId: firstHex})).toBe(2);
120
- expect(
121
- await datasetsCollection().countDocuments({historyId: histories[3]._id.toHexString()})
122
- ).toBe(1);
123
- expect(await datasetsCollection().countDocuments({historyId: stringHistoryId})).toBe(1);
124
- });
125
-
126
- it("keeps every stored field, so the moved history loads through the model", async () => {
127
- const {histories} = await seed();
128
-
129
- await migrateGptHistoryStringIds({connection: mongoose.connection, dryRun: false});
130
-
131
- const hexId = histories[1]._id.toHexString();
132
- const history = await GptHistory.findById(hexId);
133
- expect(history?._id).toBe(hexId);
134
- expect(history?.title).toBe("b");
135
- expect(history?.userId.toString()).toBe(userId.toString());
136
- expect(history?.prompts.map((prompt) => prompt.text)).toEqual([
137
- "Question about b",
138
- "Answer about b",
139
- ]);
140
- const datasets = await AIDataset.find({historyId: histories[0]._id.toHexString()});
141
- expect(datasets).toHaveLength(2);
142
- });
143
-
144
- it("finishes a run that stopped after copying a history but before deleting the original", async () => {
145
- const {histories} = await seed();
146
- const interrupted = histories[2];
147
- await historiesCollection().insertOne({...interrupted, _id: interrupted._id.toHexString()});
148
-
149
- const result = await migrateGptHistoryStringIds({
150
- connection: mongoose.connection,
151
- dryRun: false,
152
- });
153
-
154
- expect(result.historiesMigrated).toBe(5);
155
- expect(await historiesCollection().countDocuments()).toBe(6);
156
- expect(await idTypes(historiesCollection(), "_id")).toEqual(["string"]);
157
- });
158
-
159
- it("rewrites refs left as ObjectIds when a run stopped after moving their history", async () => {
160
- const orphanedHistory = new mongoose.Types.ObjectId();
161
- const movedHistory: mongoose.mongo.Document = {
162
- ...legacyHistory("moved"),
163
- _id: orphanedHistory.toHexString(),
164
- };
165
- await historiesCollection().insertOne(movedHistory);
166
- await datasetsCollection().insertOne(legacyDataset(orphanedHistory));
167
-
168
- const result = await migrateGptHistoryStringIds({
169
- connection: mongoose.connection,
170
- dryRun: false,
171
- });
172
-
173
- expect(result).toEqual({datasetsUpdated: 1, historiesMigrated: 0});
174
- expect(
175
- await datasetsCollection().countDocuments({historyId: orphanedHistory.toHexString()})
176
- ).toBe(1);
177
- });
178
-
179
- it("only counts in a dry run", async () => {
180
- await seed();
181
-
182
- const result = await migrateGptHistoryStringIds({
183
- connection: mongoose.connection,
184
- dryRun: true,
185
- });
186
-
187
- expect(result).toEqual({datasetsUpdated: 4, historiesMigrated: 5});
188
- expect(await idTypes(historiesCollection(), "_id")).toEqual(["objectId", "string"]);
189
- expect(await historiesCollection().countDocuments({_id: {$type: "objectId"}})).toBe(5);
190
- });
191
-
192
- it("runs as a migration up through the migration context", async () => {
193
- await seed();
194
-
195
- const logs: string[] = [];
196
- await gptHistoryStringIdsMigration.up(migrationContext(true, logs));
197
- expect(await historiesCollection().countDocuments({_id: {$type: "objectId"}})).toBe(5);
198
-
199
- await gptHistoryStringIdsMigration.up(migrationContext(false, logs));
200
- await gptHistoryStringIdsMigration.up(migrationContext(false, logs));
201
- expect(gptHistoryStringIdsMigration.slug).toBe("gpt-history-string-ids");
202
- expect(logs).toEqual([
203
- "[gpt-history-string-ids] would move 5 histories and 4 dataset refs",
204
- "[gpt-history-string-ids] moved 5 histories and 4 dataset refs",
205
- "[gpt-history-string-ids] moved 0 histories and 0 dataset refs",
206
- ]);
207
- expect(await historiesCollection().countDocuments({_id: {$type: "objectId"}})).toBe(0);
208
- expect(await datasetsCollection().countDocuments({historyId: {$type: "objectId"}})).toBe(0);
209
- });
210
-
211
- it("stops between batches when the run is cancelled", async () => {
212
- await seed();
213
- let checks = 0;
214
- const checkCancellation = async (): Promise<void> => {
215
- checks += 1;
216
- if (checks > 1) {
217
- throw new Error("cancelled");
218
- }
219
- };
220
-
221
- await expect(
222
- migrateGptHistoryStringIds({
223
- batchSize: 2,
224
- checkCancellation,
225
- connection: mongoose.connection,
226
- dryRun: false,
227
- })
228
- ).rejects.toThrow("cancelled");
229
- expect(await historiesCollection().countDocuments({_id: {$type: "objectId"}})).toBe(3);
230
- });
231
- });
@@ -1,150 +0,0 @@
1
- /**
2
- * Data migration for Terreno 58: GptHistory became a synced collection with String `_id`s.
3
- * Histories saved before 58 have ObjectId `_id`s, which sync clients and the String-typed
4
- * schema cannot address. This migration re-inserts each one with `_id` set to the same id's
5
- * hex string, so URLs and client caches that hold the hex id keep working, rewrites
6
- * `AIDataset.historyId` refs to that hex string, and then deletes the ObjectId originals.
7
- *
8
- * It writes through the native driver: `syncPlugin` forbids the hard deletes and
9
- * multi-document writes a move needs, and the copies keep every stored field as-is.
10
- * Running it again is a no-op.
11
- */
12
- import type {MigrationContext, MigrationLogger} from "@terreno/api";
13
- import type mongoose from "mongoose";
14
-
15
- import {AIDataset} from "../models/aiDataset";
16
- import {GptHistory} from "../models/gptHistory";
17
-
18
- /** Slug for the timestamped file an app adds to its `migrations/` directory. */
19
- export const GPT_HISTORY_STRING_IDS_MIGRATION = "gpt-history-string-ids";
20
-
21
- const DEFAULT_BATCH_SIZE = 500;
22
-
23
- const DUPLICATE_KEY_CODE = 11000;
24
-
25
- /** Only ObjectId `_id`s still need a move; String ids are already migrated or new. */
26
- const OBJECT_ID_FILTER = {_id: {$type: "objectId" as const}};
27
-
28
- /** Server-side ObjectId → hex string, so refs move without reading each dataset. */
29
- const HISTORY_ID_TO_STRING = [{$set: {historyId: {$toString: "$historyId"}}}];
30
-
31
- export interface GptHistoryStringIdsResult {
32
- /** AIDataset rows whose ObjectId `historyId` was rewritten (or would be, in a dry run). */
33
- datasetsUpdated: number;
34
- /** Histories moved to a String `_id` (or that would move, in a dry run). */
35
- historiesMigrated: number;
36
- }
37
-
38
- interface WriteErrorLike {
39
- code?: number;
40
- writeErrors?: Array<{code?: number}>;
41
- }
42
-
43
- /** An insert that failed only because the hex-id copy exists from an interrupted earlier run. */
44
- const isOnlyDuplicateKeyError = (error: unknown): boolean => {
45
- const writeError = error as WriteErrorLike;
46
- if (writeError.writeErrors?.length) {
47
- return writeError.writeErrors.every((item) => item.code === DUPLICATE_KEY_CODE);
48
- }
49
- return writeError.code === DUPLICATE_KEY_CODE;
50
- };
51
-
52
- const insertCopies = async ({
53
- collection,
54
- copies,
55
- }: {
56
- collection: mongoose.mongo.Collection;
57
- copies: mongoose.mongo.Document[];
58
- }): Promise<void> => {
59
- try {
60
- await collection.insertMany(copies, {ordered: false});
61
- } catch (error) {
62
- if (!isOnlyDuplicateKeyError(error)) {
63
- throw error;
64
- }
65
- }
66
- };
67
-
68
- /**
69
- * Moves every ObjectId-id GptHistory to its hex-string id in batches and rewrites the
70
- * AIDataset refs. In a dry run it only counts what it would change.
71
- */
72
- export const migrateGptHistoryStringIds = async ({
73
- batchSize = DEFAULT_BATCH_SIZE,
74
- checkCancellation,
75
- connection,
76
- dryRun,
77
- logger,
78
- }: {
79
- batchSize?: number;
80
- checkCancellation?: () => Promise<void>;
81
- connection: mongoose.Connection;
82
- dryRun: boolean;
83
- logger?: MigrationLogger;
84
- }): Promise<GptHistoryStringIdsResult> => {
85
- const histories = connection.collection(GptHistory.collection.collectionName);
86
- const datasets = connection.collection(AIDataset.collection.collectionName);
87
- const datasetFilter = {historyId: {$type: "objectId" as const}};
88
-
89
- if (dryRun) {
90
- const result = {
91
- datasetsUpdated: await datasets.countDocuments(datasetFilter),
92
- historiesMigrated: await histories.countDocuments(OBJECT_ID_FILTER),
93
- };
94
- logger?.info(`[${GPT_HISTORY_STRING_IDS_MIGRATION}] dry run`, result);
95
- return result;
96
- }
97
-
98
- let historiesMigrated = 0;
99
- let datasetsUpdated = 0;
100
- while (true) {
101
- await checkCancellation?.();
102
- const batch = await histories.find(OBJECT_ID_FILTER).limit(batchSize).toArray();
103
- if (batch.length === 0) {
104
- break;
105
- }
106
- const originalIds = batch.map((doc) => doc._id);
107
- // Copy first and delete second, so an interrupted run never loses a history: the next
108
- // run finds the original again and the copy's duplicate key is ignored.
109
- await insertCopies({
110
- collection: histories,
111
- copies: batch.map((doc) => ({...doc, _id: String(doc._id)})),
112
- });
113
- const moved = await datasets.updateMany({historyId: {$in: originalIds}}, HISTORY_ID_TO_STRING);
114
- datasetsUpdated += moved.modifiedCount;
115
- await histories.deleteMany({_id: {$in: originalIds}});
116
- historiesMigrated += batch.length;
117
- logger?.info(
118
- `[${GPT_HISTORY_STRING_IDS_MIGRATION}] moved ${historiesMigrated} histories to String ids`
119
- );
120
- }
121
-
122
- // Refs whose history was moved by an interrupted run, or never existed, still hold ObjectIds.
123
- const remaining = await datasets.updateMany(datasetFilter, HISTORY_ID_TO_STRING);
124
- datasetsUpdated += remaining.modifiedCount;
125
- const result = {datasetsUpdated, historiesMigrated};
126
- logger?.info(`[${GPT_HISTORY_STRING_IDS_MIGRATION}] done`, result);
127
- return result;
128
- };
129
-
130
- /**
131
- * Ready-made `up` for an app's `migrations/<YYYYMMDDHHmmss>-gpt-history-string-ids.ts` file.
132
- * Run it before deploying Terreno 58 frontends, which address histories by String id. It has
133
- * no `down`: new histories get UUIDs, which have no ObjectId to go back to.
134
- */
135
- export const gptHistoryStringIdsMigration = {
136
- slug: GPT_HISTORY_STRING_IDS_MIGRATION,
137
- up: async (ctx: MigrationContext): Promise<void> => {
138
- const result = await migrateGptHistoryStringIds({
139
- checkCancellation: ctx.checkCancellation,
140
- connection: ctx.mongoose.connection,
141
- dryRun: ctx.dryRun,
142
- logger: ctx.logger,
143
- });
144
- await ctx.addLog?.(
145
- "info",
146
- `[${GPT_HISTORY_STRING_IDS_MIGRATION}] ${ctx.dryRun ? "would move" : "moved"} ` +
147
- `${result.historiesMigrated} histories and ${result.datasetsUpdated} dataset refs`
148
- );
149
- },
150
- };